Gamified Crowdsourcing as a Novel Approach to Lung Ultrasound Data Set Labeling: Prospective Analysis.

BACKGROUND: Machine learning (ML) models can yield faster and more accurate medical diagnoses; however, developing ML models is limited by a lack of high-quality labeled training data. Crowdsourced labeling is a potential solution but can be constrained by concerns about label quality. OBJECTIVE: This study aims to examine whether a gamified crowdsourcing platform with continuous performance assessment, user feedback, and performance-based incentives could produce expert-quality labels on medical imaging data. METHODS: In this diagnostic comparison study, 2384 lung ultrasound clips were retrospectively collected from 203 emergency department patients. A total of 6 lung ultrasound experts classified 393 of these clips as having no B-lines, one or more discrete B-lines, or confluent B-lines to create 2 sets of reference standard data sets (195 training clips and 198 test clips). Sets were respectively used to (1) train users on a gamified crowdsourcing platform and (2) compare the concordance of the resulting crowd labels to the concordance of individual experts to reference standards. Crowd opinions were sourced from DiagnosUs (Centaur Labs) iOS app users over 8 days, filtered based on past performance, aggregated using majority rule, and analyzed for label concordance compared with a hold-out test set of expert-labeled clips. The primary outcome was comparing the labeling concordance of collated crowd opinions to trained experts in classifying B-lines on lung ultrasound clips. RESULTS: Our clinical data set included patients with a mean age of 60.0 (SD 19.0) years; 105 (51.7%) patients were female and 114 (56.1%) patients were White. Over the 195 training clips, the expert-consensus label distribution was 114 (58%) no B-lines, 56 (29%) discrete B-lines, and 25 (13%) confluent B-lines. Over the 198 test clips, expert-consensus label distribution was 138 (70%) no B-lines, 36 (18%) discrete B-lines, and 24 (12%) confluent B-lines. In total, 99,238 opinions were collected from 426 unique users. On a test set of 198 clips, the mean labeling concordance of individual experts relative to the reference standard was 85.0% (SE 2.0), compared with 87.9% crowdsourced label concordance (P=.15). When individual experts' opinions were compared with reference standard labels created by majority vote excluding their own opinion, crowd concordance was higher than the mean concordance of individual experts to reference standards (87.4% vs 80.8%, SE 1.6 for expert concordance; P<.001). Clips with discrete B-lines had the most disagreement from both the crowd consensus and individual experts with the expert consensus. Using randomly sampled subsets of crowd opinions, 7 quality-filtered opinions were sufficient to achieve near the maximum crowd concordance. CONCLUSIONS: Crowdsourced labels for B-line classification on lung ultrasound clips via a gamified approach achieved expert-level accuracy. This suggests a strategic role for gamified crowdsourcing in efficiently generating labeled image data sets for training ML systems.

Accuracy of "TICS" ultrasound protocol in detecting simple and complicated diverticulitis: A prospective cohort study.

BACKGROUND: Point-of-care ultrasound (US) has been suggested as the primary imaging in evaluating patients with suspected diverticulitis. Discrimination between simple and complicated diverticulitis may help to expedite emergent surgical consults and determine the risk of complications. This study aimed to: (1) determine the accuracy of an US protocol (TICS) for diagnosing diverticulitis in the emergency department (ED) setting and (2) assess the ability of TICS to distinguish between simple and complicated diverticulitis. METHODS: Patients with clinically suspected diverticulitis who underwent a diagnostic computed tomography (CT) scan were identified prospectively in the ED. Emergency US faculty and fellows blinded to the CT results performed and interpreted US scans. The presence of simple or complicated diverticulitis was recorded after each US evaluation. The diagnostic ability of the US was compared to CT as the criterion standard. Modified Hinchey classification was used to distinguish between simple and complicated diverticulitis. RESULTS: A total of 149 patients (55% female, mean ± SD age 58 ± 16 years) were enrolled and included in the final analyses. Diverticulitis was the final diagnosis in 75 of 149 patients (50.3%), of whom 53 had simple diverticulitis and 22 had perforated diverticulitis (29.4%). TICS protocol's test characteristics for simple diverticulitis include a sensitivity of 95% (95% confidence interval [CI] 87%-99%), specificity of 76% (95% CI 65%-86%), positive predictive value of 80% (95% CI 71%-88%), and negative predictive value of 93% (95% CI 84%-98%). TICS protocol correctly identified 12 of 22 patients with complicated diverticulitis (sensitivity 55% [95% CI 32%-76%]) and specificity was 96% (95% CI 91%-99%). Eight of 10 missed diagnoses of complicated diverticulitis were identified as simple diverticulitis, and two were recorded as negative. CONCLUSIONS: In ED patients with suspected diverticulitis, US demonstrated high accuracy in ruling out or diagnosing diverticulitis, but its reliability in differentiating complicated from simple diverticulitis is unsatisfactory.

Novice-performed point-of-care ultrasound for home-based imaging.

Patient-performed point-of-care ultrasound (POCUS) may be feasible for use in home-based healthcare. We investigated whether novice users can obtain lung ultrasound (LUS) images via self-scanning with similar interpretability and quality as experts. Adult participants with no prior medical or POCUS training, who were capable of viewing PowerPoint slides in their home and who could hold a probe to their chest were recruited. After training, volunteers self-performed 8-zone LUS and saved images using a hand-held POCUS device in their own home. Each 8-zone LUS scan was repeated by POCUS experts. Clips were independently viewed and scored by POCUS experts blinded to performing sonographers. Quality and interpretability scores of novice- and expert-obtained LUS images were compared. Thirty volunteers with average age of 42.8 years (Standard Deviation (SD) 15.8), and average body mass index of 23.7 (SD 3.1) were recruited. Quality of novice and expert scans did not differ (median score 2.6, interquartile range (IQR) 2.3-2.9 vs. 2.8, IQR 2.3-3.0, respectively p = 0.09). Individual zone quality also did not differ (P > 0.05). Interpretability of LUS was similar between expert and novice scanners (median 7 zones interpretable, IQR 6-8, for both groups, p = 0.42). Interpretability of novice-obtained scans did not differ from expert scans (median 7 out of 8 zones, IQR 6-8, p = 0.42). Novice-users can self-obtain interpretable, expert-quality LUS clips with minimal training. Patient-performed LUS may be feasible for outpatient home monitoring.

Perineural Dexamethasone as a Peripheral Nerve Block Adjuvant in the Emergency Department: A Case Series.

BACKGROUND: Acute pain is one of the most common complaints encountered in the emergency department (ED). Single-injection peripheral nerve blocks are a safe and effective pain management tool when performed in the ED. Dexamethasone has been explored as an adjuvant to prolong duration of analgesia from peripheral nerve blocks in peri- and postoperative settings; however, data surrounding the use of dexamethasone for ED-performed nerve blocks are lacking. CASE SERIES: In this case series we discuss our experience with adjunctive perineural dexamethasone in ED-performed regional anesthesia. Why Should an Emergency Physician be Aware of This?: Nerve blocks performed with adjuvant perineural dexamethasone may be a safe additive to provide analgesia beyond the expected half-life of local anesthetic alone. Prospective studies exploring the role of adjuvant perineural dexamethasone in ED-performed nerve blocks are needed. © 2021 Elsevier Inc.

Healthy Adults Display Long-Term Trait-Like Neurobehavioral Resilience and Vulnerability to Sleep Loss.

Sleep loss produces well-characterized cognitive deficits, although there are large individual differences, with marked vulnerability or resilience among individuals. Such differences are stable with repeated exposures to acute total sleep deprivation (TSD) within a short-time interval (weeks). Whether such stability occurs with chronic sleep restriction (SR) and whether it endures across months to years in TSD, indicating a true trait, remains unknown. In 23 healthy adults, neurobehavioral vulnerability to TSD exposures, separated by 27-2,091 days (mean: 444 days; median: 210 days), showed trait-like stability in performance and subjective measures (82-95% across measures). Similarly, in 24 healthy adults, neurobehavioral vulnerability to SR exposures, separated by 78-3,058 days (mean: 935 days; median: 741 days), also showed stability (72-92% across measures). Cognitive performance outcomes and subjective ratings showed consistency across objective measures, and consistency across subjective measures, but not between objective and subjective domains. We demonstrate for the first time the stability of phenotypic neurobehavioral responses in the same individuals to SR and to TSD over long-time intervals. Across multiple measures, prior sleep loss responses are strong predictors of individual responses to subsequent sleep loss exposures chronically or intermittently, across months and years, thus validating the need for biomarkers and predictors.