Leveraging Edge Computing for Video Data Streaming in UAV-Based Emergency Response Systems.

The rapid advancement of technology has greatly expanded the capabilities of unmanned aerial vehicles (UAVs) in wireless communication and edge computing domains. The primary objective of UAVs is the seamless transfer of video data streams to emergency responders. However, live video data streaming is inherently latency dependent, wherein the value of the video frames diminishes with any delay in the stream. This becomes particularly critical during emergencies, where live video streaming provides vital information about the current conditions. Edge computing seeks to address this latency issue in live video streaming by bringing computing resources closer to users. Nonetheless, the mobile nature of UAVs necessitates additional trajectory supervision alongside the management of computation and networking resources. Consequently, efficient system optimization is required to maximize the overall effectiveness of the collaborative system with limited UAV resources. This study explores a scenario where multiple UAVs collaborate with end users and edge servers to establish an emergency response system. The proposed idea takes a comprehensive approach by considering the entire emergency response system from the incident site to video distribution at the user level. It includes an adaptive resource management strategy, leveraging deep reinforcement learning by simultaneously addressing video streaming latency, UAV and user mobility factors, and varied bandwidth resources.

Semi-Supervised Building Extraction with Optical Flow Correction Based on Satellite Video Data in a Tsunami-Induced Disaster Scene.

Data and reports indicate an increasing frequency and intensity of natural disasters worldwide. Buildings play a crucial role in disaster responses and damage assessments, aiding in planning rescue efforts and evaluating losses. Despite advances in applying deep learning to building extraction, challenges remain in handling complex natural disaster scenes and reducing reliance on labeled datasets. Recent advances in satellite video are opening a new avenue for efficient and accurate building extraction research. By thoroughly mining the characteristics of disaster video data, this work provides a new semantic segmentation model for accurate and efficient building extraction based on a limited number of training data, which consists of two parts: the prediction module and the automatic correction module. The prediction module, based on a base encoder-decoder structure, initially extracts buildings using a limited amount of training data that are obtained instantly. Then, the automatic correction module takes the output of the prediction module as input, constructs a criterion for identifying pixels with erroneous semantic information, and uses optical flow values to extract the accurate corresponding semantic information on the corrected frame. The experimental results demonstrate that the proposed method outperforms other methods in accuracy and computational complexity in complicated natural disaster scenes.

Exploration of factors affecting webcam-based automated gaze coding.

Online experiments have been transforming the field of behavioral research, enabling researchers to increase sample sizes, access diverse populations, lower the costs of data collection, and promote reproducibility. The field of developmental psychology increasingly exploits such online testing approaches. Since infants cannot give explicit behavioral responses, one key outcome measure is infants' gaze behavior. In the absence of automated eyetrackers in participants' homes, automatic gaze classification from webcam data would make it possible to avoid painstaking manual coding. However, the lack of a controlled experimental environment may lead to various noise factors impeding automatic face detection or gaze classification. We created an adult webcam dataset that systematically reproduced noise factors from infant webcam studies which might affect automated gaze coding accuracy. We varied participants' left-right offset, distance to the camera, facial rotation, and the direction of the lighting source. Running two state-of-the-art classification algorithms (iCatcher+ and OWLET) revealed that facial detection performance was particularly affected by the lighting source, while gaze coding accuracy was consistently affected by the distance to the camera and lighting source. Morphing participants' faces to be unidentifiable did not generally affect the results, suggesting facial anonymization could be used when making online video data publicly available, for purposes of further study and transparency. Our findings will guide improving study design for infant and adult participants during online experiments. Moreover, training algorithms using our dataset will allow researchers to improve robustness and allow developmental psychologists to leverage online testing more efficiently.

Accuracy Evaluation of a Three-Dimensional Model Generated from Patient-Specific Monocular Video Data for Maxillofacial Prosthetic Rehabilitation: A Pilot Study.

PURPOSE: To evaluate if the combination of a monoscopic photogrammetry technique and smartphone-recorded monocular video data could be appropriately applied to maxillofacial prosthesis fabrication. MATERIALS AND METHODS: Smartphone video and laser scanning data were recorded for five healthy volunteers (24.1 ± 0.7 years). Three-dimensional (3D) facial models were generated using photogrammetry software and a laser scanner. Smartphone-recorded video data were used to generate a photogrammetric 3D model. The videos were recorded at two resolutions: 1080 × 1920 (high resolution) and 720 × 1280 pixels (low resolution). The lengths of five nasal component parts (nose height, nasal dorsum length, nasal column length, nasal ala length, and nose breadth) were compared in the photogrammetric 3D models (as the test model) and the laser scanned 3D models (as the validation model) using reverse engineering software. RESULTS: There was a significant difference in the nasal dorsum length between the test model and the validation model (high resolution; 95% confidence interval, 2.05-5.07, Low resolution; confidence interval, 2.19-5.69). In contrast to the nasal dorsum length, there were no significant differences in nose height, nose breadth, nasal ala length, and nasal column length. CONCLUSION: Using smartphone-recorded video data and a photogrammetry technique may be a promising technique to apply in the maxillofacial prosthetic rehabilitation workflow.

The blenderFace method: video-based measurement of raw movement data during facial expressions of emotion using open-source software.

This article proposes an optical measurement of movement applied to data from video recordings of facial expressions of emotion. The approach offers a way to capture motion adapted from the film industry in which markers placed on the skin of the face can be tracked with a pattern-matching algorithm. The method records and postprocesses raw facial movement data (coordinates per frame) of distinctly placed markers and is intended for use in facial expression research (e.g., microexpressions) in laboratory settings. Due to the explicit use of specifically placed, artificial markers, the procedure offers the simultaneous measurement of several emotionally relevant markers in a (psychometrically) objective and artifact-free way, even for facial regions without natural landmarks (e.g., the cheeks). In addition, the proposed procedure is fully based on open-source software and is transparent at every step of data processing. Two worked examples demonstrate the practicability of the proposed procedure: In Study 1(N= 39), the participants were instructed to show the emotions happiness, sadness, disgust, and anger, and in Study 2 (N= 113), they were asked to present both a neutral face and the emotions happiness, disgust, and fear. Study 2 involved the simultaneous tracking of 14 markers for approximately 12 min per participant with a time resolution of 33 ms. The measured facial movements corresponded closely to the assumptions of established measurement instruments (EMFACS, FACSAID, Friesen & Ekman, 1983; Ekman & Hager, 2002). In addition, the measurement was found to be very precise with sub-second, sub-pixel, and sub-millimeter accuracy.

Agency and clientship in public encounters: co-constructing 'neediness' and 'worthiness' in shelter placement meetings.

This article seeks to develop our understanding of the agency of vulnerable groups who at first sight may not seem to have much agency in their lives. It explores the co-constructed nature of agency in three Danish homeless shelters. Unlike earlier interview-based studies, our research is based on naturalistic data drawn from 23 video-recorded placement meetings. Using concepts from Goffman, we examine how versions of the neediness and worthiness of homeless people are negotiated verbally and bodily between staff and clients. We find that homeless people have to juggle two partly contradictory roles when they are given or take the roles of either a (active) citizen or a (passive) client. Clientship is actively negotiated by both parties and demonstrates the agency of homeless people: they can collaborate with (as clients) or challenge (as citizens) the staff's attempts to formulate solutions to their troubles. We further examine how the professional ideology of client centredness affects the meeting between the two parties. However, we show that, like any discourse, client centredness has no intrinsic meaning and is played out by actors in very different ways. In work with the homeless, the discourse of client centredness is related to discourses of 'neediness', 'worthiness' and 'value for money' that define agency in different ways and make three different client positions available: the resolute client, the acquiescent client and the passive client.

OptiMouse: a comprehensive open source program for reliable detection and analysis of mouse body and nose positions.

BACKGROUND: Accurate determination of mouse positions from video data is crucial for various types of behavioral analyses. While detection of body positions is straightforward, the correct identification of nose positions, usually more informative, is far more challenging. The difficulty is largely due to variability in mouse postures across frames. RESULTS: Here, we present OptiMouse, an extensively documented open-source MATLAB program providing comprehensive semiautomatic analysis of mouse position data. The emphasis in OptiMouse is placed on minimizing errors in position detection. This is achieved by allowing application of multiple detection algorithms to each video, including custom user-defined algorithms, by selection of the optimal algorithm for each frame, and by correction when needed using interpolation or manual specification of positions. CONCLUSIONS: At a basic level, OptiMouse is a simple and comprehensive solution for analysis of position data. At an advanced level, it provides an open-source and expandable environment for a detailed analysis of mouse position data.

A High Precision Quality Inspection System for Steel Bars Based on Machine Vision.

Steel bars play an important role in modern construction projects and their quality enormously affects the safety of buildings. It is urgent to detect whether steel bars meet the specifications or not. However, the existing manual detection methods are costly, slow and offer poor precision. In order to solve these problems, a high precision quality inspection system for steel bars based on machine vision is developed. We propose two algorithms: the sub-pixel boundary location method (SPBLM) and fast stitch method (FSM). A total of five sensors, including a CMOS, a level sensor, a proximity switch, a voltage sensor, and a current sensor have been used to detect the device conditions and capture image or video. The device could capture abundant and high-definition images and video taken by a uniform and stable smartphone at the construction site. Then data could be processed in real-time on a smartphone. Furthermore, the detection results, including steel bar diameter, spacing, and quantity would be given by a practical APP. The system has a rather high accuracy (as low as 0.04 mm (absolute error) and 0.002% (relative error) of calculating diameter and spacing; zero error in counting numbers of steel bars) when doing inspection tasks, and three parameters can be detected at the same time. None of these features are available in existing systems and the device and method can be widely used to steel bar quality inspection at the construction site.

"Having friends is like having marshmallows": Perspectives of transition-aged youths with intellectual and developmental disabilities on friendship.

BACKGROUND: The literature emphasizes that friendships are essential to youths' successful transition to and adjustment in adulthood. Few studies have explored the friendships of transition-aged youths with intellectual and developmental disabilities (IDD), and even fewer include youths' own perspectives. This qualitative study explored the perspectives of transition-aged youths with IDD regarding their own experiences of friendship. METHOD: A subset of video-recorded data of 21 interviews with seven participants (20-24 years) was extracted from a larger study for secondary analysis. Data were analysed using thematic analysis, informed by constructivist grounded theory methods. RESULTS: Three themes were revealed: meanings of friends and friendship, deepening self-knowledge and negotiating in(ter)dependence. CONCLUSIONS: The findings expand existing knowledge about friendships of transition-aged youths with IDD. Findings can potentially inform development of new services or enhancement of existing services aimed at facilitating transition to adulthood for youths with IDD and point to key areas for future research.

AI in evaluating ambulation of stroke patients: severity classification with video and functional ambulation category scale.

BACKGROUND: The evaluation of gait function and severity classification of stroke patients are important to determine the rehabilitation goal and the level of exercise. Physicians often qualitatively evaluate patients' walking ability through visual gait analysis using naked eye, video images, or standardized assessment tools. Gait evaluation through observation relies on the doctor's empirical judgment, potentially introducing subjective opinions. Therefore, conducting research to establish a basis for more objective judgment is crucial. OBJECTIVE: To verify a deep learning model that classifies gait image data of stroke patients according to Functional Ambulation Category (FAC) scale. METHODS: Gait vision data from 203 stroke patients and 182 healthy individuals recruited from six medical institutions were collected to train a deep learning model for classifying gait severity in stroke patients. The recorded videos were processed using OpenPose. The dataset was randomly split into 80% for training and 20% for testing. RESULTS: The deep learning model attained a training accuracy of 0.981 and test accuracy of 0.903. Area Under the Curve(AUC) values of 0.93, 0.95, and 0.96 for discriminating among the mild, moderate, and severe stroke groups, respectively. CONCLUSION: This confirms the potential of utilizing human posture estimation based on vision data not only to develop gait parameter models but also to develop models to classify severity according to the FAC criteria used by physicians. To develop an AI-based severity classification model, a large amount and variety of data is necessary and data collected in non-standardized real environments, not in laboratories, can also be used meaningfully.

Effects of Virtual Reality Pilates Training on Duration of Posture Maintenance and Flow in Young, Healthy Individuals: Randomized Crossover Trial.

BACKGROUND: This study explored the use of virtual reality (VR) technology to enhance the effectiveness and duration of low-intensity movements and postures in Pilates-derived exercises. We postulate that by leveraging the flow state in VR, individuals can engage in these exercises for longer periods while maintaining a high level of flow. OBJECTIVE: The purpose of this study was to compare differences in posture maintenance and flow between VR Pilates training and conventional Pilates training, and the correlation between the 2 factors. METHODS: The 18 participants in each group received either VR training or conventional training and were switched to the other training type after a 2-day wash-out period. Each group performed Pilates movements in a VR environment and a conventional environment, divided into 4 types. After training sessions, participants were evaluated for flow using a self-report questionnaire. In addition, a sports video analysis program was used to measure the duration of posture maintenance in 2 video-recorded sessions. Repeated-measures ANOVA and correlation analysis were performed on the measured duration of posture maintenance and flow scores. In all cases, the statistical significance level was set at P<.05. RESULTS: Results for the duration of posture maintenance verification by type showed that simple behavior (F1,16=17.631; P<.001), upper body-arm coordination behavior (F1,16=6.083; P=.04), upper body-leg coordination behavior (F1,16=8.359; P<.001), and whole-body coordination behavior (F1,16=8.426; P<.001) all showed an interaction effect at P<.05. Flow (F1,16=15.250; P<.001) also showed an interaction effect. In addition, significant correlations were determined between duration of all types of posture maintenance and flow in the VR training group at P<.05. CONCLUSIONS: Our results indicate that VR Pilates training may be more useful than conventional Pilates training in improving the duration of posture maintenance and that it promotes a significantly higher degree of flow when compared with conventional Pilates training.

Video-based fully automatic assessment of open surgery suturing skills.

PURPOSE: The goal of this study was to develop a new reliable open surgery suturing simulation system for training medical students in situations where resources are limited or in the domestic setup. Namely, we developed an algorithm for tools and hands localization as well as identifying the interactions between them based on simple webcam video data, calculating motion metrics for assessment of surgical skill. METHODS: Twenty-five participants performed multiple suturing tasks using our simulator. The YOLO network was modified to a multi-task network for the purpose of tool localization and tool-hand interaction detection. This was accomplished by splitting the YOLO detection heads so that they supported both tasks with minimal addition to computer run-time. Furthermore, based on the outcome of the system, motion metrics were calculated. These metrics included traditional metrics such as time and path length as well as new metrics assessing the technique participants use for holding the tools. RESULTS: The dual-task network performance was similar to that of two networks, while computational load was only slightly bigger than one network. In addition, the motion metrics showed significant differences between experts and novices. CONCLUSION: While video capture is an essential part of minimal invasive surgery, it is not an integral component of open surgery. Thus, new algorithms, focusing on the unique challenges open surgery videos present, are required. In this study, a dual-task network was developed to solve both a localization task and a hand-tool interaction task. The dual network may be easily expanded to a multi-task network, which may be useful for images with multiple layers and for evaluating the interaction between these different layers.

Age-specific, population-level pedigree of wild black bears provides insights into reproduction, paternity, and maternal effects on offspring apparent survival.

Wildlife pedigrees provide insights into ecological and evolutionary processes. DNA obtained from noninvasively collected hair is often used to determine individual identities for pedigrees and other genetic analyses. However, detection rates associated with some noninvasive DNA studies can be relatively low, and genetic data do not provide information on individual birth year. Supplementing hair DNA stations with video cameras should increase the individual detection rate, assuming accurate identification of individuals via video data. Video data can also provide birth year information for individuals captured as young of the year, which can enrich population-level pedigrees. We placed video cameras at hair stations and combined genetic and video data to reconstruct an age-specific, population-level pedigree of wild black bears during 2010-2020. Combining individual birth year with mother-offspring relatedness, we also estimated litter size, interlitter interval, primiparity, and fecundity. We used the Cormack-Jolly-Seber model in Program Mark to evaluate the effect of maternal identity on offspring apparent survival. We compared model rankings of apparent survival and parameter estimates based on combined genetic and video data with those based on only genetic data. We observed 42 mother-offspring relationships. Of these, 21 (50%) would not have been detected had we used hair DNA alone. Moreover, video data allowed for the cub and yearling age classes to be determined. Mean annual fecundity was 0.42 (95% CI: 0.27, 0.56). Maternal identity influenced offspring apparent survival, where offspring of one mother experienced significantly lower apparent survival (0.39; SE = 0.15) than that of offspring of four other mothers (0.89-1.00; SE = 0.00-0.06). We video-documented cub abandonment by the mother whose offspring experienced low apparent survival, indicating individual behaviors (e.g., maternal care) may scale up to affect population-level parameters (e.g., cub survival). Our findings provide insights into evolutionary processes and are broadly relevant to wildlife ecology and conservation.

Open surgery tool classification and hand utilization using a multi-camera system.

PURPOSE: The goal of this work is to use multi-camera video to classify open surgery tools as well as identify which tool is held in each hand. Multi-camera systems help prevent occlusions in open surgery video data. Furthermore, combining multiple views such as a top-view camera covering the full operative field and a close-up camera focusing on hand motion and anatomy may provide a more comprehensive view of the surgical workflow. However, multi-camera data fusion poses a new challenge: A tool may be visible in one camera and not the other. Thus, we defined the global ground truth as the tools being used regardless their visibility. Therefore, tools that are out of the image should be remembered for extensive periods of time while the system responds quickly to changes visible in the video. METHODS: Participants (n = 48) performed a simulated open bowel repair. A top-view and a close-up cameras were used. YOLOv5 was used for tool and hand detection. A high-frequency LSTM with a 1-second window at 30 frames per second (fps) and a low-frequency LSTM with a 40-second window at 3 fps were used for spatial, temporal, and multi-camera integration. RESULTS: The accuracy and F1 of the six systems were: top-view (0.88/0.88), close-up (0.81,0.83), both cameras (0.9/0.9), high-fps LSTM (0.92/0.93), low-fps LSTM (0.9/0.91), and our final architecture the multi-camera classifier(0.93/0.94). CONCLUSION: Since each camera in a multi-camera system may have a partial view of the procedure, we defined a 'global ground truth.' Defining this at the data labeling phase emphasized this requirement at the learning phase, eliminating the need for any heuristic decisions. By combining a system with a high fps and a low fps from the multiple camera array, we improved the classification abilities of the global ground truth.

(Hyper)active Data Curation: A Video Case Study from Behavioral Science.

Video data are uniquely suited for research reuse and for documenting research methods and findings. However, curation of video data is a serious hurdle for researchers in the social and behavioral sciences, where behavioral video data are obtained session by session and data sharing is not the norm. To eliminate the onerous burden of post hoc curation at the time of publication (or later), we describe best practices in active data curation-where data are curated and uploaded immediately after each data collection to allow instantaneous sharing with one button press at any time. Indeed, we recommend that researchers adopt "hyperactive" data curation where they openly share every step of their research process. The necessary infrastructure and tools are provided by Databrary-a secure, web-based data library designed for active curation and sharing of personally identifiable video data and associated metadata. We provide a case study of hyperactive curation of video data from the Play and Learning Across a Year (PLAY) project, where dozens of researchers developed a common protocol to collect, annotate, and actively curate video data of infants and mothers during natural activity in their homes at research sites across North America. PLAY relies on scalable standardized workflows to facilitate collaborative research, assure data quality, and prepare the corpus for sharing and reuse throughout the entire research process.

The Particle Jako ("Like") in Spoken Czech: From Expressing Comparison to Mobilizing Affiliative Responses.

This contribution investigates the use of the Czech particle jako ("like"/"as") in naturally occurring conversations. Inspired by interactional research on unfinished or suspended utterances and on turn-final conjunctions and particles, the analysis aims to trace the possible development of jako from conjunction to a tag-like particle that can be exploited for mobilizing affiliative responses. Traditionally, jako has been described as conjunction used for comparing two elements or for providing a specification of a first element ["X (is) like Y"]. In spoken Czech, however, jako can be flexibly positioned within a speaking turn and does not seem to operate as a coordinating or hypotactic conjunction. As a result, prior studies have described jako as a polyfunctional particle. This article will try to shed light on the meaning of jako in spoken discourse by focusing on its apparent fuzzy or "filler" uses, i.e., when it is found in a mid-turn position in multi-unit turns and in the immediate vicinity of hesitations, pauses, and turn suspensions. Based on examples from mundane, video-recorded conversations and on a sequential and multimodal approach to social interaction, the analyses will first show that jako frequently frames discursive objects that co-participants should respond to. By using jako before a pause and concurrently adopting specific embodied displays, participants can more explicitly seek to mobilize responsive action. Moreover, as jako tends to cluster in multi-unit turns involving the formulation of subjective experience or stance, it can be shown to be specifically designed for mobilizing affiliative responses. Finally, it will be argued that the potential of jako to open up interactive turn spaces can be linked to the fundamental comparative semantics of the original conjunction.

Video from user-generated content as a source of pre-crash scenario naturalistic driving data.

OBJECTIVE: The objective of this study was to investigate the use of public video from internet user-generated content as a means of collecting naturalistic driving data. METHODS: A convenience sample of 38 videos comprised of 203 events was extracted from publicly available channels on the YouTube™ platform. Each event was manually reviewed and pseudo-coded according to a subset of current CRSS variables. Pre-crash scenarios were coded using categories developed for prior NHTSA analysis. RESULTS: Crashes represented 67% of the reviewed cases. Collisions with motor vehicles accounted for 84% of all crashes in the sample. Pre-crash scenarios were able to be determined for all crashes and near-crashes. The most prevalent pre-crash scenario types in the video data were Crossing Paths (41%), Rear End (21%), and Lane Change (17%). The top pre-crash scenarios from Swanson et al., were Rear End (31%), Crossing Paths (21%), and Lane Change (12%). The most prevalent pre-near crash scenario types in the video data were Crossing Paths (32%), Lane Change (30%), and Pedestrian (12%). CONCLUSIONS: The most prevalent pre-crash scenarios in the video data were similar to those in data from FARS and NASS-GES. Though not nationally representative, this preliminary study demonstrated that user-generated content may be useful as a source of inexpensive naturalistic data and provides sufficient detail to capture important pre-crash, near-crash and crash information.

Hand gestures for emergency situations: A video dataset based on words from Indian sign language.

Automatic sign language recognition provides better services to the deaf as it avoids the existing communication gap between them and the rest of the society. Hand gestures, the primary mode of sign language communication, plays a key role in improving sign language recognition. This article presents a video dataset of the hand gestures of Indian sign language (ISL) words used in emergency situations. The videos of eight ISL words have been collected from 26 individuals (including 12 males and 14 females) in the age group of 22 to 26 years with two samples from each individual in an indoor environment with normal lighting conditions. Such a video dataset is highly needed for automatic recognition of emergency situations from the sign language for the benefit of the deaf. The dataset is useful for the researchers working on vision based sign language recognition (SLR) as well as hand gesture recognition (HGR). Moreover, support vector machine based classification and deep learning based classification of the emergency gestures has been carried out and the base classification performance shows that the database can be used as a benchmarking dataset for developing novel and improved techniques for recognizing the hand gestures of emergency words in Indian sign language.

Looking Tasks Online: Utilizing Webcams to Collect Video Data from Home.

Online experimentation is emerging as a new methodology within classical data acquisition in psychology. It allows for easy, fast, broad, and cheap data conduction from the comfort of people's homes. To add another method to the array of available tools, here we used recent developments in web technology to investigate the technical feasibility of online HyperText Markup Language-5/JavaScript-based video data recording. We employed a preferential looking task with children between 4 and 24 months. Parents and their children participated from home through a three-stage process: First, interested adults registered and took pictures through a webcam-based photo application. In the second step, we edited the pictures and integrated them into the design. Lastly, participants returned to the website and the video data acquisition took place through their webcam. In sum, we were able to create and employ the video recording application with participants as young as 4 months old. Quality-wise, no participant had to be removed due to the framerate or quality of videos and only 7% of data was excluded due to behavioral factors (lack of concentration). Results-wise, interrater reliability of rated looking side (left/right) showed a high agreement between raters, Fleiss' Kappa, κ = 0.97, which can be translated to sufficient data quality for further analyses. With regard to on-/off-screen attention attribution, we found that children lost interest after about 10 s after trial onset using a static image presentation or 60 s total experimental time. Taken together, we were able to show that online video data recording is possible and viable for developmental psychology and beyond.