Gamified Crowdsourcing as a Novel Approach to Lung Ultrasound Data Set Labeling: Prospective Analysis.

BACKGROUND: Machine learning (ML) models can yield faster and more accurate medical diagnoses; however, developing ML models is limited by a lack of high-quality labeled training data. Crowdsourced labeling is a potential solution but can be constrained by concerns about label quality. OBJECTIVE: This study aims to examine whether a gamified crowdsourcing platform with continuous performance assessment, user feedback, and performance-based incentives could produce expert-quality labels on medical imaging data. METHODS: In this diagnostic comparison study, 2384 lung ultrasound clips were retrospectively collected from 203 emergency department patients. A total of 6 lung ultrasound experts classified 393 of these clips as having no B-lines, one or more discrete B-lines, or confluent B-lines to create 2 sets of reference standard data sets (195 training clips and 198 test clips). Sets were respectively used to (1) train users on a gamified crowdsourcing platform and (2) compare the concordance of the resulting crowd labels to the concordance of individual experts to reference standards. Crowd opinions were sourced from DiagnosUs (Centaur Labs) iOS app users over 8 days, filtered based on past performance, aggregated using majority rule, and analyzed for label concordance compared with a hold-out test set of expert-labeled clips. The primary outcome was comparing the labeling concordance of collated crowd opinions to trained experts in classifying B-lines on lung ultrasound clips. RESULTS: Our clinical data set included patients with a mean age of 60.0 (SD 19.0) years; 105 (51.7%) patients were female and 114 (56.1%) patients were White. Over the 195 training clips, the expert-consensus label distribution was 114 (58%) no B-lines, 56 (29%) discrete B-lines, and 25 (13%) confluent B-lines. Over the 198 test clips, expert-consensus label distribution was 138 (70%) no B-lines, 36 (18%) discrete B-lines, and 24 (12%) confluent B-lines. In total, 99,238 opinions were collected from 426 unique users. On a test set of 198 clips, the mean labeling concordance of individual experts relative to the reference standard was 85.0% (SE 2.0), compared with 87.9% crowdsourced label concordance (P=.15). When individual experts' opinions were compared with reference standard labels created by majority vote excluding their own opinion, crowd concordance was higher than the mean concordance of individual experts to reference standards (87.4% vs 80.8%, SE 1.6 for expert concordance; P<.001). Clips with discrete B-lines had the most disagreement from both the crowd consensus and individual experts with the expert consensus. Using randomly sampled subsets of crowd opinions, 7 quality-filtered opinions were sufficient to achieve near the maximum crowd concordance. CONCLUSIONS: Crowdsourced labels for B-line classification on lung ultrasound clips via a gamified approach achieved expert-level accuracy. This suggests a strategic role for gamified crowdsourcing in efficiently generating labeled image data sets for training ML systems.

Internal Rotation Traction Radiograph Improves Proximal Femoral Fracture Classification Accuracy and Agreement.

OBJECTIVE: The objective of this study is to assess the clinical utility of internal rotation traction radiography in the classification of proximal femoral fractures. MATERIALS AND METHODS: The study cohort included 78 consecutive patients who were surgically treated for a proximal femoral fracture and for whom preoperative physician-assisted internal rotation traction radiographs of the fractured hip were obtained in addition to standard radiographs. Two radiologists who were blinded to clinical information independently classified each fracture without the traction view and then with the traction view. The radiologists also reported their confidence (expressed as a percentage) in their classifications. The reference standard was the consensus interpretation of intraoperative C-arm fluoroscopic images by two orthopedic surgeons and one radiologist. Classification accuracy was compared using the McNemar test. Subjective confidence and confidence-weighted accuracy were compare using paired t tests. Agreement with the reference standard and interreader agreement were calculated using the kappa statistic and were compared using the z-test after bootstrapping was performed to obtain the standard error. RESULTS: With the traction view, the pooled accuracy increased from 44.9% to 72.4%, subjective confidence increased from 87% to 94%, and confidence-weighted accuracy increased from 51.7% to 74.3% (p < 0.001). With the traction view, the kappa statistic for agreement with the reference standard increased from 0.530 to 0.791 and from 0.381 to 0.625 for the two readers, and interreader agreement increased from 0.480 to 0.678 (p < 0.001). CONCLUSION: The addition of an internal rotation traction radiographic view significantly improves radiologist accuracy and confidence as well as interreader agreement in the classification of proximal femoral fractures, all of which would be expected to best guide appropriate surgical management.

Color postprocessing of conventional CT images: preliminary results in assessment of nondisplaced proximal femoral fractures.

PURPOSE: The purpose of this study was to demonstrate the diagnostic performance and effect on reader confidence of a custom computed tomography (CT) color postprocessing algorithm for assessment of nondisplaced proximal femoral fractures. MATERIALS AND METHODS: Four radiologists, including two PGY-3 radiology residents and two emergency radiologists, independently interpreted 30 CT examinations of the hip and/or pelvis performed for trauma, consisting of a total of 15 cases positive for nondisplaced hip fracture and 15 age and sex-matched controls. Images were reviewed first with conventional CT images and after at least 8 weeks, all images were reviewed again with the addition of coronal color postprocessed images. Sensitivity and specificity were compared with McNemar's test, and diagnostic confidence was compared with paired t tests. RESULTS: There was no significant difference in diagnostic performance between conventional and postprocessed images, although there was nominally increased sensitivity and decreased specificity with the postprocessed images: for all readers, the sensitivity and specificity for conventional images was 88.3 and 95.0%, compared to 93.3% (p = 0.25) and 88.3% (p = 0.14) for postprocessed images. Three of four readers (including both attending radiologists) reported an increase in confidence with postprocessed images for cases negative for fracture (10-point confidence scale of 7.25 for conventional images, compared to 8.2 for postprocessed images for all readers, p = 0.0053). There was no difference in diagnostic confidence for cases positive for fracture. CONCLUSIONS: A custom color CT postprocessing algorithm did not demonstrate a significant difference in diagnostic performance for assessment of nondisplaced proximal femoral fractures within the limitations of a relatively small sample size; however, postprocessing increases confidence of experienced readers in cases negative for fracture.

Potential Utility of C-reactive Protein for Tuberculosis Risk Stratification Among Patients With Non-Meningitic Symptoms at HIV Diagnosis in Low- and Middle-income Countries.

Background: The World Health Organization recommends initiating same-day antiretroviral therapy (ART) while tuberculosis (TB) testing is under way for patients with non-meningitic symptoms at HIV diagnosis, though safety data are limited. C-reactive protein (CRP) testing may improve TB risk stratification in this population. Methods: In this baseline analysis of 498 adults (>18 years) with TB symptoms at HIV diagnosis who were enrolled in a trial of rapid ART initiation in Haiti, we describe test characteristics of varying CRP thresholds in the diagnosis of TB. We also assessed predictors of high CRP as a continuous variable using generalized linear models. Results: Eighty-seven (17.5%) participants were diagnosed with baseline TB. The median CRP was 33.0 mg/L (interquartile range: 5.1, 85.5) in those with TB, and 2.6 mg/L (interquartile range: 0.8, 11.7) in those without TB. As the CRP threshold increased from ≥1 mg/L to ≥10 mg/L, the positive predictive value for TB increased from 22.4% to 35.4% and negative predictive value decreased from 96.9% to 92.3%. With CRP thresholds varying from <1 to <10 mg/L, a range from 25.5% to 64.9% of the cohort would have been eligible for same-day ART and 0.8% to 5.0% would have untreated TB at ART initiation. Conclusions: CRP concentrations can be used to improve TB risk stratification, facilitating same-day decisions about ART initiation. Depending on the CRP threshold, one-quarter to two-thirds of patients could be eligible for same-day ART, with a reduction of 3- to 20-fold in the proportion with untreated TB, compared with a strategy of same-day ART while awaiting TB test results.

Acute respiratory distress syndrome after spontaneous intracerebral hemorrhage*.

OBJECTIVES: Acute respiratory distress syndrome develops commonly in critically ill patients in response to an injurious stimulus. The prevalence and risk factors for development of acute respiratory distress syndrome after spontaneous intracerebral hemorrhage have not been reported. We sought to determine the prevalence of acute respiratory distress syndrome after intracerebral hemorrhage, characterize risk factors for its development, and assess its impact on patient outcomes. DESIGN: Retrospective cohort study at two academic centers. PATIENTS: We included consecutive patients presenting from June 1, 2000, to November 1, 2010, with intracerebral hemorrhage requiring mechanical ventilation. We excluded patients with age less than 18 years, intracerebral hemorrhage secondary to trauma, tumor, ischemic stroke, or structural lesion; if they required intubation only during surgery; if they were admitted for comfort measures; or for a history of immunodeficiency. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: Data were collected both prospectively as part of an ongoing cohort study and by retrospective chart review. Of 1,665 patients identified by database query, 697 met inclusion criteria. The prevalence of acute respiratory distress syndrome was 27%. In unadjusted analysis, high tidal volume ventilation was associated with an increased risk of acute respiratory distress syndrome (hazard ratio, 1.79 [95% CI, 1.13-2.83]), as were male sex, RBC and plasma transfusion, higher fluid balance, obesity, hypoxemia, acidosis, tobacco use, emergent hematoma evacuation, and vasopressor dependence. In multivariable modeling, high tidal volume ventilation was the strongest risk factor for acute respiratory distress syndrome development (hazard ratio, 1.74 [95% CI, 1.08-2.81]) and for inhospital mortality (hazard ratio, 2.52 [95% CI, 1.46-4.34]). CONCLUSIONS: Development of acute respiratory distress syndrome is common after intubation for intracerebral hemorrhage. Modifiable risk factors, including high tidal volume ventilation, are associated with its development and in-patient mortality.

Potential Utility of C-reactive Protein for Tuberculosis Risk Stratification among Patients with Non-Meningitic Symptoms at HIV Diagnosis in Low- and Middle-Income Countries.

Article Summary: We assessed the association between C-reactive protein (CRP) and Mycobacterium tuberculosis (TB) diagnosis in symptomatic patients at HIV diagnosis. We found that CRP concentrations can improve tuberculosis risk stratification, facilitating decision making about whether (specific) tuberculosis testing is indicated before antiretroviral therapy initiation. Background: The World Health Organization recommends initiating same-day ART while tuberculosis testing is underway for patients with non-meningitic symptoms at HIV diagnosis, though safety data are limited. C-reactive protein (CRP) testing may improve tuberculosis risk stratification in this population. Methods: In this baseline analysis of 498 adults (>18 years) with tuberculosis symptoms at HIV diagnosis who were enrolled in a trial of rapid ART initiation in Haiti, we describe test characteristics of varying CRP thresholds in the diagnosis of TB. We also assessed predictors of high CRP (≥3 mg/dL) using generalized linear models. Results: Eighty-seven (17.5%) patients were diagnosed with baseline TB. The median CRP was 33.0 mg/L (IQR: 5.1, 85.5) in those with TB, and 2.6 mg/L (IQR: 0.8, 11.7) in those without TB. As the CRP threshold increased from ≥1 mg/L to ≥10 mg/L, the positive predictive value for TB increased from 22.4% to 35.4%, and negative predictive value decreased from 96.9% to 92.3%. With CRP thresholds varying from <1 to <10 mg/L, a range from 25.5% to 64.9% of the cohort would have been eligible for same-day ART, and 0.8% to 5.0% would have untreated TB at ART initiation. Conclusions: CRP concentrations can be used to improve TB risk stratification, facilitating same-day decisions about ART initiation. Depending on the CRP threshold, one-quarter to two-thirds of patients could be eligible for same-day ART, with a reduction of 3-fold to 20-fold in the proportion with untreated TB, compared with a strategy of same-day ART while awaiting TB test results.

Reduced-dose CT: effect on reader evaluation in detection of pulmonary embolism.

OBJECTIVE: The purpose of this study was to evaluate the effect of reduction in radiation dose on CT detection of pulmonary embolism. SUBJECTS AND METHODS: Emergency department patients were evaluated for pulmonary embolism with standard and simulated reduced-dose CT angiography. Simulated lower-dose CT angiograms obtained at 90, 45, 22, and 10 mAs(eff) were reconstructed by mathematical addition of noise to the standard dose (180 mAs(eff)) data from the images of 18 patients with and 20 patients without pulmonary embolism. Four radiologists blinded to the study parameters separately interpreted each CT angiogram. Dose trends for subjective measures (diagnostic certainty, image quality, and perceived technical limitations) were evaluated, test characteristics for the detection of pulmonary embolism were computed, and clot burden was measured. RESULTS: Readers indicated significant reductions in diagnostic certainty (p < 0.02) and image quality (p < 0.02) and an increase in perceived technical limitations (p < 0.01) as the simulated radiation dose was decreased. These subjective measures also showed significant adverse dose trends when the mAs(eff) was reduced (p < 0.001). At reduced radiation doses, the sensitivity and positive predictive value for detection of pulmonary embolism diminished significantly. The sensitivity was 0.94 (lower bound of 0.95 CI, 0.92); specificity, 0.99 (lower bound of 0.95 CI, 0.98); positive predictive value, 0.95 (lower bound of 0.95 CI, 0.92); and negative predictive value, 0.99 (lower bound of 0.95 CI, 0.97). All patients had a low to moderate clot burden. CONCLUSION: Reduction in dose for CT angiography in the detection of pulmonary embolism has a significant adverse effect on readers' subjective assessment of diagnostic confidence and image quality. Detection of pulmonary embolism also decreases as the tube current dose is reduced.

Traumatic right atrial pseudoaneurysm.

Pseudoaneurysm is defined as contained blood pooling due to rupture of vascular wall. They have higher risk of rupture and hence are usually managed aggressively. Trauma, infection and prior surgery are the most common etiologies for pseudoaneurysm of most sites. Traumatic cardiac pseudoaneurysm are rare and poses a diagnostic challenge to the treating physician since there is no specific symptoms associated with pseudoaneurysm and electrocardiogram; cardiac enzymes and echocardiogram may be unrevealing or inconclusive in many cases. Cross-sectional imaging [computed tomography (CT) and magnetic resonance imaging (MRI)] is done in cases with high index of suspicion and is the standard modality for diagnosis for cardiac pseudoaneurysm. We present a case of conservatively managed post-traumatic right atrial pseudoaneurysm with aim to highlight the role of electrocardiography (ECG)-gated multidetector computed tomography (MDCT) in diagnosis and follow-up of this rare entity that ultimately underwent rapid partial thrombosis.