Deep Learning Assistance Closes the Accuracy Gap in Fracture Detection Across Clinician Types.

BACKGROUND: Missed fractures are the most common diagnostic errors in musculoskeletal imaging and can result in treatment delays and preventable morbidity. Deep learning, a subfield of artificial intelligence, can be used to accurately detect fractures by training algorithms to emulate the judgments of expert clinicians. Deep learning systems that detect fractures are often limited to specific anatomic regions and require regulatory approval to be used in practice. Once these hurdles are overcome, deep learning systems have the potential to improve clinician diagnostic accuracy and patient care. QUESTIONS/PURPOSES: This study aimed to evaluate whether a Food and Drug Administration-cleared deep learning system that identifies fractures in adult musculoskeletal radiographs would improve diagnostic accuracy for fracture detection across different types of clinicians. Specifically, this study asked: (1) What are the trends in musculoskeletal radiograph interpretation by different clinician types in the publicly available Medicare claims data? (2) Does the deep learning system improve clinician accuracy in diagnosing fractures on radiographs and, if so, is there a greater benefit for clinicians with limited training in musculoskeletal imaging? METHODS: We used the publicly available Medicare Part B Physician/Supplier Procedure Summary data provided by the Centers for Medicare & Medicaid Services to determine the trends in musculoskeletal radiograph interpretation by clinician type. In addition, we conducted a multiple-reader, multiple-case study to assess whether clinician accuracy in diagnosing fractures on radiographs was superior when aided by the deep learning system compared with when unaided. Twenty-four clinicians (radiologists, orthopaedic surgeons, physician assistants, primary care physicians, and emergency medicine physicians) with a median (range) of 16 years (2 to 37) of experience postresidency each assessed 175 unique musculoskeletal radiographic cases under aided and unaided conditions (4200 total case-physician pairs per condition). These cases were comprised of radiographs from 12 different anatomic regions (ankle, clavicle, elbow, femur, forearm, hip, humerus, knee, pelvis, shoulder, tibia and fibula, and wrist) and were randomly selected from 12 hospitals and healthcare centers. The gold standard for fracture diagnosis was the majority opinion of three US board-certified orthopaedic surgeons or radiologists who independently interpreted the case. The clinicians' diagnostic accuracy was determined by the area under the curve (AUC) of the receiver operating characteristic (ROC) curve, sensitivity, and specificity. Secondary analyses evaluated the fracture miss rate (1-sensitivity) by clinicians with and without extensive training in musculoskeletal imaging. RESULTS: Medicare claims data revealed that physician assistants showed the greatest increase in interpretation of musculoskeletal radiographs within the analyzed time period (2012 to 2018), although clinicians with extensive training in imaging (radiologists and orthopaedic surgeons) still interpreted the majority of the musculoskeletal radiographs. Clinicians aided by the deep learning system had higher accuracy diagnosing fractures in radiographs compared with when unaided (unaided AUC: 0.90 [95% CI 0.89 to 0.92]; aided AUC: 0.94 [95% CI 0.93 to 0.95]; difference in least square mean per the Dorfman, Berbaum, Metz model AUC: 0.04 [95% CI 0.01 to 0.07]; p < 0.01). Clinician sensitivity increased when aided compared with when unaided (aided: 90% [95% CI 88% to 92%]; unaided: 82% [95% CI 79% to 84%]), and specificity increased when aided compared with when unaided (aided: 92% [95% CI 91% to 93%]; unaided: 89% [95% CI 88% to 90%]). Clinicians with limited training in musculoskeletal imaging missed a higher percentage of fractures when unaided compared with radiologists (miss rate for clinicians with limited imaging training: 20% [95% CI 17% to 24%]; miss rate for radiologists: 14% [95% CI 9% to 19%]). However, when assisted by the deep learning system, clinicians with limited training in musculoskeletal imaging reduced their fracture miss rate, resulting in a similar miss rate to radiologists (miss rate for clinicians with limited imaging training: 9% [95% CI 7% to 12%]; miss rate for radiologists: 10% [95% CI 6% to 15%]). CONCLUSION: Clinicians were more accurate at diagnosing fractures when aided by the deep learning system, particularly those clinicians with limited training in musculoskeletal image interpretation. Reducing the number of missed fractures may allow for improved patient care and increased patient mobility. LEVEL OF EVIDENCE: Level III, diagnostic study.

Optical Mammography in Patients with Breast Cancer Undergoing Neoadjuvant Chemotherapy: Individual Clinical Response Index.

RATIONALE AND OBJECTIVES: We present an optical mammography study that aims to develop quantitative measures of pathologic response to neoadjuvant chemotherapy (NAC) in patients with breast cancer. Such quantitative measures are based on the concentrations of oxyhemoglobin ([HbO2]), deoxyhemoglobin ([Hb]), total hemoglobin ([HbT]), and hemoglobin saturation (SO2) in breast tissue at the tumor location and at sequential time points during chemotherapy. MATERIALS AND METHODS: Continuous-wave, spectrally resolved optical mammography was performed in transmission and parallel-plate geometry on 10 patients before treatment initiation and at each NAC administration (mean number of optical mammography sessions: 12, range: 7-18). Data on two patients were discarded for technical reasons. The patients were categorized as responders (R, >50% decrease in tumor size), or nonresponders (NR, <50% decrease in tumor size) based on imaging and histopathology results. RESULTS: At 50% completion of the NAC regimen (therapy midpoint), R (6/8) demonstrated significant decreases in SO2 (-27% ± 4%) and [HbT] (-35 ± 4 µM) at the tumor location with respect to baseline values. By contrast, NR (2/8) showed nonsignificant changes in SO2 and [HbT] at therapy midpoint. We introduce a cumulative response index as a quantitative measure of the individual patient's response to therapy. At therapy midpoint, the SO2-based cumulative response index had a sensitivity of 100% and a specificity of 100% for the identification of R. CONCLUSIONS: These results show that optical mammography is a promising tool to assess individual response to NAC at therapy midpoint to guide further decision making for neoadjuvant therapy.

Broadband optical mammography instrument for depth-resolved imaging and local dynamic measurements.

We present a continuous-wave instrument for non-invasive diffuse optical imaging of the breast in a parallel-plate transmission geometry. The instrument measures continuous spectra in the wavelength range 650-1000 nm, with an intensity noise level <1.5% and a spatial sampling rate of 5 points/cm in the x- and y-directions. We collect the optical transmission at four locations, one collinear and three offset with respect to the illumination optical fiber, to recover the depth of optical inhomogeneities in the tissue. We imaged a tissue-like, breast shaped, silicone phantom (6 cm thick) with two embedded absorbing structures: a black circle (1.7 cm in diameter) and a black stripe (3 mm wide), designed to mimic a tumor and a blood vessel, respectively. The use of a spatially multiplexed detection scheme allows for the generation of on-axis and off-axis projection images simultaneously, as opposed to requiring multiple scans, thus decreasing scan-time and motion artifacts. This technique localizes detected inhomogeneities in 3D and accurately assigns their depth to within 1 mm in the ideal conditions of otherwise homogeneous tissue-like phantoms. We also measured induced hemodynamic changes in the breast of a healthy human subject at a selected location (no scanning). We applied a cyclic, arterial blood pressure perturbation by alternating inflation (to a pressure of 200 mmHg) and deflation of a pneumatic cuff around the subject's thigh at a frequency of 0.05 Hz, and measured oscillations with amplitudes up to 1 µM and 0.2 µM in the tissue concentrations of oxyhemoglobin and deoxyhemoglobin, respectively. These hemodynamic oscillations provide information about the vascular structure and functional integrity in tissue, and may be used to assess healthy or abnormal perfusion in a clinical setting.

Optical mammography: bilateral breast symmetry in hemoglobin saturation maps.

We present a study of the bilateral symmetry of human breast hemoglobin saturation maps measured with a broadband optical mammography instrument. We have imaged 21 patients with unilateral breast cancer, 32 patients with unilateral benign lesions, and 27 healthy patients. An image registration process was applied to the bilateral hemoglobin saturation (SO 2 SO2 ) images by assigning each pixel to the low, middle, or high range of SO 2 SO2 values, where the thresholds for the categories were the 15th and 85th percentiles of the individual saturation range. The Dice coefficient, which is a measure of similarity, was calculated for each patient's pair of right and left breast SO 2 SO2 images. The invasive cancer patients were found to have an average Dice coefficient value of 0.55±0.07 0.55±0.07 , which was significantly lower than the benign and healthy groups (0.61±0.11 0.61±0.11 and 0.62±0.12 0.62±0.12 , respectively). Although differences were seen in a group analysis, the healthy patient Dice coefficients spanned a wide range, limiting the diagnostic capabilities of this SO 2 SO2 symmetry analysis on an individual basis. Our results suggest that for assessing the SO 2 SO2 contrast of breast lesions, it may be better to select a reference tissue in the ipsilateral rather than the contralateral breast.

Broadband optical mammography: chromophore concentration and hemoglobin saturation contrast in breast cancer.

This study reports the optical characterization and quantitative oximetry of human breast cancer using spectrally-resolved images collected with a broadband, continuous-wave optical mammography instrument. On twenty-six cancer patients, we collected two-dimensional optical mammograms and created maps of the concentrations of hemoglobin, water, and lipids, as well as the oxygen saturation of hemoglobin. For each cancerous breast, we analyzed the difference between the tumor region (as identified by x-ray and optical mammography) and the remainder of breast tissue. With respect to the surrounding tissue, we found that cancer regions have significantly higher concentrations of total hemoglobin (+2.4 ± 0.4 µM) and water (+7 ± 1% v/v), and significantly lower lipid concentration (8 ± 2% v/v) and oxygen saturation of hemoglobin (5 ± 1%). We also found a significant correlation between the tumor optical contrast and the grade of breast cancer as quantified by the Nottingham histologic score; this demonstrates how optical signatures may be representative of metabolic and morphological features, as well as the aggressive potential of the tumor.

Depth discrimination in diffuse optical transmission imaging by planar scanning off-axis fibers: initial applications to optical mammography.

We present a method for depth discrimination in parallel-plate, transmission mode, diffuse optical imaging. The method is based on scanning a set of detector pairs, where the two detectors in each pair are separated by a distance δDi along direction δ D i within the x-y scanning plane. A given optical inhomogeneity appears shifted by αi δ D i (with 0≤ αi ≤1) in the images collected with the two detection fibers of the i-th pair. Such a spatial shift can be translated into a measurement of the depth z of the inhomogeneity, and the depth measurements based on each detector pair are combined into a specially designed weighted average. This depth assessment is demonstrated on tissue-like phantoms for simple inhomogeneities such as straight rods in single-rod or multiple-rod configurations, and for more complex curved structures which mimic blood vessels in the female breast. In these phantom tests, the method has recovered the depth of single inhomogeneities in the central position of the phantom to within 4 mm of their actual value, and within 7 mm for more superficial inhomogeneities, where the thickness of the phantom was 65 mm. The application of this method to more complex images, such as optical mammograms, requires a robust approach to identify corresponding structures in the images collected with the two detectors of a given pair. To this aim, we propose an approach based on the inner product of the skeleton images collected with the two detectors of each pair, and we present an application of this approach to optical in vivo images of the female breast. This depth discrimination method can enhance the spatial information content of 2D projection images of the breast by assessing the depth of detected structures, and by allowing for 3D localization of breast tumors.

Parametric estimation of 3D tubular structures for diffuse optical tomography.

We explore the use of diffuse optical tomography (DOT) for the recovery of 3D tubular shapes representing vascular structures in breast tissue. Using a parametric level set method (PaLS) our method incorporates the connectedness of vascular structures in breast tissue to reconstruct shape and absorption values from severely limited data sets. The approach is based on a decomposition of the unknown structure into a series of two dimensional slices. Using a simplified physical model that ignores 3D effects of the complete structure, we develop a novel inter-slice regularization strategy to obtain global regularity. We report on simulated and experimental reconstructions using realistic optical contrasts where our method provides a more accurate estimate compared to an unregularized approach and a pixel based reconstruction.

Effect of graphite concentration on shear-wave speed in gelatin-based tissue-mimicking phantoms.

Elasticity-based imaging modalities are becoming popular diagnostic tools in clinical practice. Gelatin-based, tissue mimicking phantoms that contain graphite as the acoustic scattering material are commonly used in testing and validating elasticity-imaging methods to quantify tissue stiffness. The gelatin bloom strength and concentration are used to control phantom stiffness. While it is known that graphite concentration can be modulated to control acoustic attenuation, the impact of graphite concentration on phantom elasticity has not been characterized in these gelatin phantoms. This work investigates the impact of graphite concentration on phantom shear stiffness as characterized by shear-wave speed measurements using impulsive acoustic-radiation-force excitations. Phantom shear-wave speed increased by 0.83 (m/s)/(dB/(cm MHz)) when increasing the attenuation coefficient slope of the phantom material through increasing graphite concentration. Therefore, gelatin-phantom stiffness can be affected by the conventional ways that attenuation is modulated through graphite concentration in these phantoms.