Automated Video Debriefing Using Computer Vision Techniques.

INTRODUCTION: Within any training event, debriefing is a vital component that highlights areas of proficiency and deficiency, enables reflection, and ultimately provides opportunity for remediation. Video-based debriefing is desirable to capture performance and replay events, but the reality is rife with challenges, principally lengthy video and occlusions that block line of sight from camera equipment to participants. METHODS: To address this issue, researchers automated the editing of a video debrief, using a system of person-worn cameras and computer vision techniques. The cameras record a simulation event, and the video is processed using computer vision. Researchers investigated a variety of computer vision techniques, ultimately focusing on the scale invariant feature transform detection method and a convolutional neural network. The system was trained to detect and tag medically relevant segments of video and assess a single exemplar medical intervention, in this case the application of a tourniquet. RESULTS: The system tagged medically relevant video segments with 92% recall and 66% precision, resulting in an F1 (harmonic mean of precision and recall) of 72% (N = 23). The exemplar medical intervention was successfully assessed in 39.5% of videos (N = 39). CONCLUSION: The system showed suitable accuracy tagging medically relevant video segments, but requires additional research to improve medical intervention assessment accuracy. Computer vision has the potential to automate video debrief creation to augment existing debriefing strategies.

Highly-automated, high-throughput replication of yeast-based logic circuit design assessments.

We describe an experimental campaign that replicated the performance assessment of logic gates engineered into cells of Saccharomyces cerevisiae by Gander et al. Our experimental campaign used a novel high-throughput experimentation framework developed under Defense Advanced Research Projects Agency's Synergistic Discovery and Design program: a remote robotic lab at Strateos executed a parameterized experimental protocol. Using this protocol and robotic execution, we generated two orders of magnitude more flow cytometry data than the original experiments. We discuss our results, which largely, but not completely, agree with the original report and make some remarks about lessons learned. Graphical Abstract.

Round Trip: An Automated Pipeline for Experimental Design, Execution, and Analysis.

Synthetic biology is a complex discipline that involves creating detailed, purpose-built designs from genetic parts. This process is often phrased as a Design-Build-Test-Learn loop, where iterative design improvements can be made, implemented, measured, and analyzed. Automation can potentially improve both the end-to-end duration of the process and the utility of data produced by the process. One of the most important considerations for the development of effective automation and quality data is a rigorous description of implicit knowledge encoded as a formal knowledge representation. The development of knowledge representation for the process poses a number of challenges, including developing effective human-machine interfaces, protecting against and repairing user error, providing flexibility for terminological mismatches, and supporting extensibility to new experimental types. We address these challenges with the DARPA SD2 Round Trip software architecture. The Round Trip is an open architecture that automates many of the key steps in the Test and Learn phases of a Design-Build-Test-Learn loop for high-throughput laboratory science. The primary contribution of the Round Trip is to assist with and otherwise automate metadata creation, curation, standardization, and linkage with experimental data. The Round Trip's focus on metadata supports fast, automated, and replicable analysis of experiments as well as experimental situational awareness and experimental interpretability. We highlight the major software components and data representations that enable the Round Trip to speed up the design and analysis of experiments by 2 orders of magnitude over prior ad hoc methods. These contributions support a number of experimental protocols and experimental types, demonstrating the Round Trip's breadth and extensibility. We describe both an illustrative use case using the Round Trip for an on-the-loop experimental campaign and overall contributions to reducing experimental analysis time and increasing data product volume in the SD2 program.

Intent Parser: A Tool for Codification and Sharing of Experimental Design.

Communicating information about experimental design among a team of collaborators is challenging because different people tend to describe experiments in different ways and with different levels of detail. Sometimes, humans can interpret missing information by making assumptions and drawing inferences from information already provided. Doing so, however, is error-prone and typically requires a high level of interpersonal communication. In this paper, we present a tool that addresses this challenge by providing a simple interface for incremental formal codification of experiment designs. Users interact with a Google Docs word-processing interface with structured tables, backed by assisted linking to machine-readable definitions in a data repository (SynBioHub) and specification of available protocols and requests for execution in the Open Protocol Interface Language (OPIL). The result is an easy-to-use tool for generating machine-readable descriptions of experiment designs with which users in the DARPA SD2 program have collected data from 80 208 samples using a variety of protocols and instruments over the course of 181 experiment runs.

TASBE Image Analytics: A Processing Pipeline for Quantifying Cell Organization from Fluorescent Microscopy.

Laboratory automation now commonly allows high-throughput sample preparation, culturing, and acquisition of microscopy images, but quantitative image analysis is often still a painstaking and subjective process. This is a problem especially significant for work on programmed morphogenesis, where the spatial organization of cells and cell types is of paramount importance. To address the challenges of quantitative analysis for such experiments, we have developed TASBE Image Analytics, a software pipeline for automatically segmenting collections of cells using the fluorescence channels of microscopy images. With TASBE Image Analytics, collections of cells can be grouped into spatially disjoint segments, the movement or development of these segments tracked over time, and rich statistical data output in a standardized format for analysis. Processing is readily configurable, rapid, and produces results that closely match hand annotation by humans for all but the smallest and dimmest segments. TASBE Image Analytics can thus provide the analysis necessary to complete the design-build-test-learn cycle for high-throughput experiments in programmed morphogenesis, as validated by our application of this pipeline to process experiments on shape formation with engineered CHO and HEK293 cells.

Use of Computer Vision Tools to Identify Behavioral Markers of Pediatric Obsessive-Compulsive Disorder: A Pilot Study.

OBJECTIVES: The clinical presentation of pediatric obsessive-compulsive disorder (OCD) is heterogeneous, which is a stumbling block to understanding pathophysiology and to developing new treatments. A major shift in psychiatry, embodied in the Research Domain Criteria (RDoC) initiative of National Institute of Mental Health, recognizes the pitfalls of categorizing mental illnesses using diagnostic criteria. Instead, RDoC encourages researchers to use a dimensional approach, focusing on narrower domains of psychopathology to characterize brain-behavior relationships. Our aim in this multidisciplinary pilot study was to use computer vision tools to record OCD behaviors and to cross-validate these behavioral markers with standard clinical measures. METHODS: Eighteen youths with OCD and 21 healthy controls completed tasks in an innovation laboratory (free arrangement of objects, hand washing, arrangement of objects on contrasting carpets). Tasks were video-recorded. Videos were coded by blind raters for OCD-related behaviors. Children's Yale-Brown Obsessive Compulsive Scale (CY-BOCS) and other scales were administered. We compared video-recorded measures of behavior in OCD versus healthy controls and correlated video measures and clinical measures of OCD. RESULTS: Behavioral measures on the videos were significantly correlated with specific CY-BOCS dimension scores. During the free arrangement task, more time spent ordering objects and more moves of objects were both significantly associated with higher CY-BOCS ordering/repeating dimension scores. Longer duration of hand washing was significantly correlated with higher scores on CY-BOCS ordering/repeating and forbidden thoughts dimensions. During arrangement of objects on contrasting carpets, more moves and more adjustment of objects were significantly associated with higher CY-BOCS ordering/repeating dimension scores. CONCLUSION: Preliminary data suggest that measurement of behavior using video recording is a valid approach for quantifying OCD psychopathology. This methodology could serve as a new tool for investigating OCD using an RDoC approach. This objective, novel behavioral measurement technique may benefit both researchers and clinicians in assessing pediatric OCD and in identifying new behavioral markers of OCD. Clinical Trial Registry: Development of an Instrument That Monitors Behaviors Associated With OCD. NCT02866422. http://clinicaltrials.gov.