MULTIMODAL ASSESSMENTS OF DRUSENOID PIGMENT EPITHELIAL DETACHMENTS IN THE AGE-RELATED EYE DISEASE STUDY 2 ANCILLARY SPECTRAL-DOMAIN OPTICAL COHERENCE TOMOGRAPHY STUDY COHORT.

PURPOSE: To identify features correlating with drusenoid pigment epithelial detachment (DPED) progression in the Age-Related Eye Disease Study 2 Ancillary spectral-domain optical coherence tomography study cohort. METHODS: In this retrospective analysis of a prospective longitudinal study, eyes with intermediate age-related macular degeneration and DPEDs were followed longitudinally with annual multimodal imaging. RESULTS: Thirty-one eyes of 25 participants (mean age 72.6 years) in the Age-Related Eye Disease Study 2 Ancillary spectral-domain OCT substudy (A2A study) had DPED identified in color fundus images. Spectral-domain optical coherence tomography inspection confirmed a subretinal pigment epithelium drusenoid elevation of ≥433 µm diameter in 25 eyes (80.6%). Twenty-four of these eyes were followed longitudinally (median 4.0 years), during which 7 eyes (29.2%) underwent DPED collapse (with 3/7 further progressing to geographic atrophy), 6 (25.0%) developing neovascular age-related macular degeneration, and 11 (45.8%) maintaining DPED persistence without late age-related macular degeneration. On Kaplan-Meier analysis, mean time to DPED collapse was 3.9 years. Both DPED collapse and progression to neovascular age-related macular degeneration were preceded by the presence of hyperreflective foci over the DPED. CONCLUSION: The natural history of DPED comprises collapse (sometimes followed by the development of atrophy), vascularization followed by exudation, or DPED persistence. Spectral-domain optical coherence tomography can confirm retinal pigment epithelial elevation caused by drusenoid accumulation and facilitate the identification of high-risk features that correlate with progression.

QUANTITATIVE ANALYSIS OF LONGITUDINAL CHANGES IN MULTIMODAL IMAGING OF LATE-ONSET RETINAL DEGENERATION.

PURPOSE: To quantitatively analyze clinically relevant features on longitudinal multimodal imaging of late-onset retinal degeneration to characterize disease progression. METHODS: Fundus autofluorescence (FAF), infrared reflectance, and optical coherence tomography imaging of 4 patients with late-onset retinal degeneration were acquired over 3 to 15 years (20 visits total). Corresponding regions of interest were analyzed on FAF (reticular pseudodrusen [RPD], "speckled FAF," and chorioretinal atrophy) and infrared reflectance (hyporeflective RPD and target RPD) using quantitative measurements, including contour area, distance to fovea, contour overlap, retinal thickness, and texture features. RESULTS: Cross-sectional analysis revealed a moderate correlation (RPD FAF ∩ RPD infrared reflectance = 63%) between contour area across modalities. Quantification of retinal thickness and texture analysis of areas contoured on FAF objectively differentiated the contour types. A longitudinal analysis of aligned images demonstrates that the contoured region of atrophy both encroaches toward the fovea and grows monotonically with a rate of 0.531 mm/year to 1.969 mm/year (square root of area, n = 5 eyes). A retrospective analysis of precursor lesions of atrophy reveals quantifiable progression from RPD to speckled FAF to atrophy. CONCLUSION: Image analysis of time points before the development of atrophy reveals consistent patterns over time and space in late-onset retinal degeneration that may provide useful outcomes for this and other degenerative retinal diseases.

MASSIVE ADVANCING NONEXUDATIVE TYPE 1 CHOROIDAL NEOVASCULARIZATION IN CTRP5 LATE-ONSET RETINAL DEGENERATION: Longitudinal Findings on Multimodal Imaging and Implications for Age-Related Macular Degeneration.

PURPOSE: To describe longitudinal multimodal imaging findings of nonexudative choroidal neovascularization in CTRP5 late-onset retinal degeneration. METHODS: Four patients with CTRP5-positive late-onset retinal degeneration underwent repeated ophthalmoscopic examination and multimodal imaging. All four patients (two siblings and their cousins, from a pedigree described previously) had the heterozygous S163R mutation. RESULTS: All four patients demonstrated large subretinal lesions in the mid-peripheral retina of both eyes. The lesions were characterized by confluent hypercyanescence with hypocyanescent borders on indocyanine green angiography, faintly visible branching vascular networks with absent/minimal leakage on fluorescein angiography, Type 1 neovascularization on optical coherence tomography angiography, and absent retinal fluid, consistent with nonexudative choroidal neovascularization. The neovascular membranes enlarged substantially over time and the birth of new membranes was observed, but all lesions remained nonexudative/minimally exudative. Without treatment, all involved retinal areas remained free of atrophy and subretinal fibrosis. CONCLUSION: We report the existence of massive advancing nonexudative Type 1 choroidal neovascularization in CTRP5 late-onset retinal degeneration. These findings have implications for age-related macular degeneration. They provide a monogenic model system for studying the mechanisms underlying the distinct events of choroidal neovascularization development, enlargement, progression to exudation, and atrophy in age-related macular degeneration. They suggest that choroidal hypoperfusion precedes neovascularization and that nonexudative neovascularization may protect against atrophy.

OCT Intensity of the Region between Outer Retina Band 2 and Band 3 as a Biomarker for Retinal Degeneration and Therapy.

Optical coherence tomography (OCT) is widely used to probe retinal structure and function. This study investigated the outer retina band (ORB) pattern and reflective intensity for the region between bands 2 and 3 (Dip) in three mouse models of inherited retinal degeneration (Rs1KO, TTLL5KO, RPE65KO) and in human AMD patients from the A2A database. OCT images were manually graded, and reflectivity signals were used to calculate the Dip ratio. Qualitative analyses demonstrated the progressive merging band 2 and band 3 in all three mouse models, leading to a reduction in the Dip ratio compared to wildtype (WT) controls. Gene replacement therapy in Rs1KO mice reverted the ORB pattern to one resembling WT and increased the Dip ratio. The degree of anatomical rescue in these mice was highly correlated with level of transgenic RS1 expression and with the restoration of ERG b-wave amplitudes. While the inner retinal cavity was significantly enlarged in dark-adapted Rs1KO mice, the Dip ratio was not altered. A reduction of the Dip ratio was also detected in AMD patients compared with healthy controls and was also positively correlated with AMD severity on the AMD score. We propose that the ORB and Dip ratio can be used as non-invasive early biomarkers for retina health, which can be used to probe therapeutic gene expression and to evaluate the effectiveness of therapy.

Quantitative analysis of optical coherence tomography imaging in patients with different severities of hydroxychloroquine toxicity.

PURPOSE: To determine the diagnostic validity of quantitative measures derived from optical coherence tomography (OCT) images in their ability to discriminate between cohorts of eyes unaffected by hydroxychloroquine (HCQ) and those with a range of toxicity severities, including mild toxicity. METHODS: Prospective, single-centre, case-control study conducted between August 2010 and May 2017. Participants were exposed to HCQ for at least 5 years (mean±SD =14±7.2 years) and classified into affected and unaffected cohorts based on the American Academy of Ophthalmology's 2016 recommendations. For affected eyes, severity (groups 1-4) was assigned based on the extent of ellipsoid zone loss. For all eyes, spectral domain-OCT scans were analysed quantitatively to compute inner retinal thickness (IRT), outer retinal thickness (ORT), and minimum signal intensity (MI) and compared across toxicity groups. RESULTS: Of the 85 participants (mean age 59±12 years, 93% female), 30 had retinal toxicity. Significant differences in ORT and MI were observed between each affected severity group and unaffected eyes. Significant differences in IRT were observed for groups 3-4 but not groups 1-2. ORT and MI were each able to discriminate between unaffected and group 1 eyes with the highest discrimination at the inner subfields (areas under the curve, AUC=0.96 for ORT and AUC=0.93 for MI). CONCLUSIONS: Quantitative analysis of OCT scans revealed significant differences between eyes with and without toxicity in two different measures. Each individual metric could discriminate between the unaffected and the lowest severity category, suggesting their potential utility in screening for HCQ toxicity in patients at risk.

Visual Field Sensitivity Prediction Using Optical Coherence Tomography Analysis in Hydroxychloroquine Toxicity.

Purpose: This study investigates the association between local retina structure and visual function in a cohort with long-term hydroxychloroquine (HCQ) use. Methods: The study included 84 participants (54 participants without toxicity and 30 participants with toxicity) with history of chronic HCQ use (14.5 ± 7.4 years) who had testing with spectral-domain optical coherence tomography (SD-OCT) imaging and Humphrey 10-2 visual fields. Optical coherence tomography (OCT) metrics (total and outer retina thickness [TRT and ORT], minimum intensity [MinI], and ellipsoid zone [EZ] loss) were sampled in regions corresponding to visual field test locations. Univariate linear correlations were investigated and a multivariate random forest regression using a combination of OCT metrics was used to predict visual field sensitivity by locus using a leave-one-out cross-validation strategy. Results: In univariate linear regression, EZ loss demonstrated the strongest relationship with visual field sensitivities in the parafoveal ring with R2 = 0.58. TRT and ORT revealed positive correlations with visual field sensitivity (R2 = 0.57 and 0.40, respectively), whereas total and outer retinal MinI yielded negative correlations (R2 = 0.10 and 0.22). The multivariate model improved correlations (R2 = 0.66) yielding a root mean squared error of 3.8 decibel (dB). Feature importance analysis identified EZ loss as the most relevant predictor of function. Conclusions: Multiple OCT-derived quantitative metrics used in combination can provide information to predict local sensitivities. The results indicate a strong relationship between retinal function and OCT measures, which contribute to the understanding of the retinal toxicity caused by HCQ as well as being applicable to outcome development for other degenerative diseases of the outer retina.

Evaluation of Cone- and Rod-Mediated Parameters in Dark Adaptation Testing as Outcome Measures in Age-Related Macular Degeneration.

PURPOSE: To investigate the suitability of 6 rod- or cone-mediated dark adaptation (DA) parameters as outcome measures for clinical trials in age-related macular degeneration (AMD), including their retest reliability, association with age and disease severity, and measurable longitudinal change over time. DESIGN: Prospective, longitudinal study (Clinicaltrials.gov: NCT01352975). PARTICIPANTS: A total of 191 patients with AMD and older participants followed longitudinally over 5 years. METHODS: Dark adaptation testing was performed using the AdaptDx dark adaptometer with a maximum test time of 40 minutes. A 2-part exponential-linear curve was fitted to obtain values for cone decay, cone plateau, time to rod-cone break, rod intercept time (RIT), rod adaptation rate (S2), and area under the curve. Intersession retest reliability was assessed in tests performed within 2 weeks using the Bland-Altman analysis. The relationship of DA parameters with age, AMD severity, and reticular pseudodrusen (RPD) presence was evaluated using linear mixed models. MAIN OUTCOME MEASURES: Retest reliability, association with disease severity, and longitudinal change of 6 DA parameters. RESULTS: A total of 1329 DA curves were analyzed. Rod intercept time was the parameter that showed the greatest reliability (intraclass correlation coefficient of 0.88) and greatest association with age, AMD severity, and RPD (marginal R2 of 0.38), followed by the rod-mediated parameters area under the curve and rod-cone break. Cone plateau appeared constant at lower RIT values but increased with progressive rod dysfunction (RIT > 22.8 minutes) with a slope of 0.07 log units per 10 minutes RIT prolongation. Therefore, it might provide additional information in the advanced stages of AMD. CONCLUSIONS: Age-related macular degeneration severity and RPD presence are each associated with large differences in multiple DA curve parameters. In addition, substantial differences in some parameters occur with age, even accounting for AMD severity and RPD status. This supports the 2-hit hypothesis of age and disease status on DA (and perhaps AMD pathophysiology itself). Of the DA parameters, RIT has the highest retest reliability, closest correlation with AMD severity and RPD, and largest longitudinal changes. This underscores the suitability of RIT as an outcome measure in clinical trials. The cone plateau increases only in advanced stages of kinetic rod dysfunction, indicating rod dysfunction preceding cone dysfunction and degeneration in the temporal sequence of pathology in AMD.

Retinal layer segmentation in optical coherence tomography (OCT) using a 3D deep-convolutional regression network for patients with age-related macular degeneration.

Introduction - Retinal layer segmentation in optical coherence tomography (OCT) images is an important approach for detecting and prognosing disease. Automating segmentation using robust machine learning techniques lead to computationally efficient solutions and significantly reduces the cost of labor-intensive labeling, which is traditionally performed by trained graders at a reading center, sometimes aided by semi-automated algorithms. Although several algorithms have been proposed since the revival of deep learning, eyes with severe pathological conditions continue to challenge fully automated segmentation approaches. There remains an opportunity to leverage the underlying spatial correlations between the retinal surfaces in the segmentation approach. Methods - Some of these proposed traditional methods can be expanded to utilize the three-dimensional spatial context governing the retinal image volumes by replacing the use of 2D filters with 3D filters. Towards this purpose, we propose a spatial-context, continuity and anatomical relationship preserving semantic segmentation algorithm, which utilizes the 3D spatial context from the image volumes with the use of 3D filters. We propose a 3D deep neural network capable of learning the surface positions of the layers in the retinal volumes. Results - We utilize a dataset of OCT images from patients with Age-related Macular Degeneration (AMD) to assess performance of our model and provide both qualitative (including segmentation maps and thickness maps) and quantitative (including error metric comparisons and volumetric comparisons) results, which demonstrate that our proposed method performs favorably even for eyes with pathological changes caused by severe retinal diseases. The Mean Absolute Error (MAE) and Root Mean Squared Error (RMSE) for patients with a wide range of AMD severity scores (0-11) were within 0.84±0.41 and 1.33±0.73 pixels, respectively, which are significantly better than some of the other state-of-the-art algorithms. Conclusion - The results demonstrate the utility of extracting features from the entire OCT volume by treating the volume as a correlated entity and show the benefit of utilizing 3D autoencoder based regression networks for smoothing the approximated retinal layers by inducing shape based regularization constraints.

Longitudinal characterization and treatment response of retinal arterial macroaneurysms in adult-onset coats disease.

Purpose: To perform longitudinal analysis of retinal arterial macroaneurysms in 3 patients with adult-onset Coats disease. Observations: Three eyes of three patients with adult-onset Coats disease were followed longitudinally for 4-15 years. Ultra-widefield images and montage color fundus photographs of affected eyes were analyzed. Size, retinal location, and grading for predominant characteristic (hemorrhagic, exudative, or quiescent) of each individual macroaneurysm were followed longitudinally from the time of onset. Fifty-one individual retinal arterial macroaneurysms were identified. The distance of any lesion-associated hemorrhage or exudation present from the foveal center was measured. Macroaneurysms were located in all quadrants of the retina, with the majority (37/51) graded as hemorrhagic at lesion onset. Hemorrhagic and exudative macroaneurysms that entered the quiescent phase remained quiescent for an average of 26 months. Seven macroaneurysms were found to have hemorrhage or exudation that came within 125 µm of the fovea and all three eyes followed demonstrated a longitudinal decrease in visual acuity despite laser and intravitreal injection therapy. At the initial visit, visual acuities ranged from 20/40 to 20/200, but decreased to 20/80 to 20/320 by the last follow-up visit. Conclusion and Importance: There are many challenges in treating patients with adult-onset Coats disease. Long-term loss of visual acuity often results from sequelae of hemorrhage and exudation affecting the macula.

Deep Learning-Based Modeling of the Dark Adaptation Curve for Robust Parameter Estimation.

Purpose: This study investigates deep-learning (DL) sequence modeling techniques to reliably fit dark adaptation (DA) curves and estimate their key parameters in patients with age-related macular degeneration (AMD) to improve robustness and curve predictions. Methods: A long-short-term memory autoencoder was used as the DL method to model the DA curve. The performance was compared against the classical nonlinear regression method using goodness-of-fit and repeatability metrics. Experiments were performed to predict the latter portion of the curve using data from early measurements. The prediction accuracy was quantified as the rod intercept time (RIT) prediction error between predicted and actual curves. Results: The two models had comparable goodness-of-fit measures, with root mean squared error (RMSE; SD) = 0.11 (0.04) log-units (LU) for the classical model and RMSE = 0.13 (0.06) LU for the DL model. Repeatability of the curve fits evaluated after introduction of random perturbations, and after performing repeated testing, demonstrated superiority of the DL method, especially among parameters related to cone decay. The DL method exhibited superior ability to predict the curve and RIT using points prior to -2 LU, with 3.1 ± 3.1 minutes RIT prediction error, compared to 19.1 ± 18.6 minutes RIT error for the classical method. Conclusions: The parameters obtained from the DL method demonstrated superior robustness as well as predictability of the curve. These could provide important advances in using multiple DA curve parameters to characterize AMD severity. Translational Relevance: Dark adaptation is an important functional measure in studies of AMD and curve modeling using DL methods can lead to improved clinical trial end points.

Quantification of prostate deformation due to needle insertion during TRUS-guided biopsy: comparison of hand-held and mechanically stabilized systems.

PURPOSE: Prostate biopsy is the clinical standard for the definitive diagnosis of prostate cancer. To overcome the limitations of 2D TRUS-guided biopsy systems when targeting preplanned locations, systems have been developed with 3D guidance to improve the accuracy of cancer detection. Prostate deformation due to needle insertion and biopsy gun firing is a potential source of error that can cause target misalignments during biopsies. METHODS: The authors used nonrigid registration of 2D TRUS images to quantify the deformation that occurs during the needle insertion and the biopsy gun firing procedure and compare this effect in biopsies performed using a hand-held TRUS probe to those performed using a mechanically assisted 3D TRUS-guided biopsy system. The authors calculated a spatially varying 95% confidence interval on the prostate tissue motion and analyzed this motion both as a function of distance to the biopsy needle and as a function of distance to the lower piercing point of the prostate. The former is relevant because biopsy targets lie along the needle axis, and the latter is of particular importance due to the reported high concentration of prostate cancer in the peripheral zone, a substantial portion of which lies on the posterior side of the prostate where biopsy needles enter the prostate after penetrating the rectal wall during transrectal biopsy. RESULTS: The results show that for both systems, the tissue deformation is such that throughout the length of the needle axis, including regions proximal to the lower piercing point, spherical tumors with a radius of 2.1 mm or more can be sampled with 95% confidence under the assumption of zero error elsewhere in the biopsy system. More deformation was observed in the direction orthogonal to the needle axis compared to the direction parallel to the needle axis; this is of particular importance given the long, narrow shape of the biopsy core. The authors measured lateral tissue motion proximal to the needle axis of not more than 1.5 mm, with 95% confidence. The authors observed a statistically significant and clinically insignificant maximum difference of 0.38 mm in the deformation, resulting from the hand-held and mechanically assisted systems along the needle axis, and the mechanical system resulted in a lower relative increase in deformation proximal to the needle axis during needle insertion, as well as lower variability of deformation during biopsy gun firing. CONCLUSIONS: Given the clinical need to biopsy tumors of volume greater than or equal to 0.5 cm3, corresponding to spherical tumors with a radius of 5 mm or more, the tissue motion induced by needle insertion and gun firing is an important consideration when setting the design specifications for TRUS-guided prostate biopsy systems.

Deep-learning based multi-modal retinal image registration for the longitudinal analysis of patients with age-related macular degeneration.

This work reports a deep-learning based registration algorithm that aligns multi-modal retinal images collected from longitudinal clinical studies to achieve accuracy and robustness required for analysis of structural changes in large-scale clinical data. Deep-learning networks that mirror the architecture of conventional feature-point-based registration were evaluated with different networks that solved for registration affine parameters, image patch displacements, and patch displacements within the region of overlap. The ground truth images for deep learning-based approaches were derived from successful conventional feature-based registration. Cross-sectional and longitudinal affine registrations were performed across color fundus photography (CFP), fundus autofluorescence (FAF), and infrared reflectance (IR) image modalities. For mono-modality longitudinal registration, the conventional feature-based registration method achieved mean errors in the range of 39-53 µm (depending on the modality) whereas the deep learning method with region overlap prediction exhibited mean errors in the range 54-59 µm. For cross-sectional multi-modality registration, the conventional method exhibited gross failures with large errors in more than 50% of the cases while the proposed deep-learning method achieved robust performance with no gross failures and mean errors in the range 66-69 µm. Thus, the deep learning-based method achieved superior overall performance across all modalities. The accuracy and robustness reported in this work provide important advances that will facilitate clinical research and enable a detailed study of the progression of retinal diseases such as age-related macular degeneration.

Deep Learning-Based Automatic Detection of Ellipsoid Zone Loss in Spectral-Domain OCT for Hydroxychloroquine Retinal Toxicity Screening.

Purpose: Retinal toxicity resulting from hydroxychloroquine use manifests photoreceptor loss and disruption of the ellipsoid zone (EZ) reflectivity band detectable on spectral-domain (SD) OCT imaging. This study investigated whether an automatic deep learning-based algorithm can detect and quantitate EZ loss on SD OCT images with an accuracy comparable with that of human annotations. Design: Retrospective analysis of data acquired in a prospective, single-center, case-control study. Participants: Eighty-five patients (168 eyes) who were long-term hydroxychloroquine users (average exposure time, 14 ± 7.2 years). Methods: A mask region-based convolutional neural network (M-RCNN) was implemented and trained on individual OCT B-scans. Scan-by-scan detections were aggregated to produce an en face map of EZ loss per 3-dimensional SD OCT volume image. To improve the accuracy and robustness of the EZ loss map, a dual network architecture was proposed that learns to detect EZ loss in parallel using horizontal (horizontal mask region-based convolutional neural network [M-RCNNH]) and vertical (vertical mask region-based convolutional neural network [M-RCNNV]) B-scans independently. To quantify accuracy, 10-fold cross-validation was performed. Main Outcome Measures: Precision, recall, intersection over union (IOU), F1-score metrics, and measured total EZ loss area were compared against human grader annotations and with the determination of toxicity based on the recommended screening guidelines. Results: The combined projection network demonstrated the best overall performance: precision, 0.90 ± 0.09; recall, 0.88 ± 0.08; and F1 score, 0.89 ± 0.07. The combined model performed superiorly to the M-RCNNH only model (precision, 0.79 ± 0.17; recall, 0.96 ± 0.04; IOU, 0.78 ± 0.15; and F1 score, 0.86 ± 0.12) and M-RCNNV only model (precision, 0.71 ± 0.21; recall, 0.94 ± 0.06; IOU, 0.69 ± 0.21; and F1 score, 0.79 ± 0.16). The accuracy was comparable with the variability of human experts: precision, 0.85 ± 0.09; recall, 0.98 ± 0.01; IOU, 0.82 ± 0.12; and F1 score, 0.91 ± 0.06. Automatically generated en face EZ loss maps provide quantitative SD OCT metrics for accurate toxicity determination combined with other functional testing. Conclusions: The algorithm can provide a fast, objective, automatic method for measuring areas with EZ loss and can serve as a quantitative assistance tool to screen patients for the presence and extent of toxicity.

Drill-mounted video guidance for orthopaedic trauma surgery.

Purpose: Percutaneous fracture fixation is a challenging procedure that requires accurate interpretation of fluoroscopic images to insert guidewires through narrow bone corridors. We present a guidance system with a video camera mounted onboard the surgical drill to achieve real-time augmentation of the drill trajectory in fluoroscopy and/or CT. Approach: The camera was mounted on the drill and calibrated with respect to the drill axis. Markers identifiable in both video and fluoroscopy are placed about the surgical field and co-registered by feature correspondences. If available, a preoperative CT can also be co-registered by 3D-2D image registration. Real-time guidance is achieved by virtual overlay of the registered drill axis on fluoroscopy or in CT. Performance was evaluated in terms of target registration error (TRE), conformance within clinically relevant pelvic bone corridors, and runtime. Results: Registration of the drill axis to fluoroscopy demonstrated median TRE of 0.9 mm and 2.0 deg when solved with two views (e.g., anteroposterior and lateral) and five markers visible in both video and fluoroscopy-more than sufficient to provide Kirschner wire (K-wire) conformance within common pelvic bone corridors. Registration accuracy was reduced when solved with a single fluoroscopic view ( TRE = 3.4 mm and 2.7 deg) but was also sufficient for K-wire conformance within pelvic bone corridors. Registration was robust with as few as four markers visible within the field of view. Runtime of the initial implementation allowed fluoroscopy overlay and/or 3D CT navigation with freehand manipulation of the drill up to 10 frames / s . Conclusions: A drill-mounted video guidance system was developed to assist with K-wire placement. Overall workflow is compatible with fluoroscopically guided orthopaedic trauma surgery and does not require markers to be placed in preoperative CT. The initial prototype demonstrates accuracy and runtime that could improve the accuracy of K-wire placement, motivating future work for translation to clinical studies.

Improving Interpretability in Machine Diagnosis: Detection of Geographic Atrophy in OCT Scans.

Purpose: Manually identifying geographic atrophy (GA) presence and location on OCT volume scans can be challenging and time consuming. This study developed a deep learning model simultaneously (1) to perform automated detection of GA presence or absence from OCT volume scans and (2) to provide interpretability by demonstrating which regions of which B-scans show GA. Design: Med-XAI-Net, an interpretable deep learning model was developed to detect GA presence or absence from OCT volume scans using only volume scan labels, as well as to interpret the most relevant B-scans and B-scan regions. Participants: One thousand two hundred eighty-four OCT volume scans (each containing 100 B-scans) from 311 participants, including 321 volumes with GA and 963 volumes without GA. Methods: Med-XAI-Net simulates the human diagnostic process by using a region-attention module to locate the most relevant region in each B-scan, followed by an image-attention module to select the most relevant B-scans for classifying GA presence or absence in each OCT volume scan. Med-XAI-Net was trained and tested (80% and 20% participants, respectively) using gold standard volume scan labels from human expert graders. Main Outcome Measures: Accuracy, area under the receiver operating characteristic (ROC) curve, F1 score, sensitivity, and specificity. Results: In the detection of GA presence or absence, Med-XAI-Net obtained superior performance (91.5%, 93.5%, 82.3%, 82.8%, and 94.6% on accuracy, area under the ROC curve, F1 score, sensitivity, and specificity, respectively) to that of 2 other state-of-the-art deep learning methods. The performance of ophthalmologists grading only the 5 B-scans selected by Med-XAI-Net as most relevant (95.7%, 95.4%, 91.2%, and 100%, respectively) was almost identical to that of ophthalmologists grading all volume scans (96.0%, 95.7%, 91.8%, and 100%, respectively). Even grading only 1 region in 1 B-scan, the ophthalmologists demonstrated moderately high performance (89.0%, 87.4%, 77.6%, and 100%, respectively). Conclusions: Despite using ground truth labels during training at the volume scan level only, Med-XAI-Net was effective in locating GA in B-scans and selecting relevant B-scans within each volume scan for GA diagnosis. These results illustrate the strengths of Med-XAI-Net in interpreting which regions and B-scans contribute to GA detection in the volume scan.

SpineCloud: image analytics for predictive modeling of spine surgery outcomes.

Purpose: Data-intensive modeling could provide insight on the broad variability in outcomes in spine surgery. Previous studies were limited to analysis of demographic and clinical characteristics. We report an analytic framework called "SpineCloud" that incorporates quantitative features extracted from perioperative images to predict spine surgery outcome. Approach: A retrospective study was conducted in which patient demographics, imaging, and outcome data were collected. Image features were automatically computed from perioperative CT. Postoperative 3- and 12-month functional and pain outcomes were analyzed in terms of improvement relative to the preoperative state. A boosted decision tree classifier was trained to predict outcome using demographic and image features as predictor variables. Predictions were computed based on SpineCloud and conventional demographic models, and features associated with poor outcome were identified from weighting terms evident in the boosted tree. Results: Neither approach was predictive of 3- or 12-month outcomes based on preoperative data alone in the current, preliminary study. However, SpineCloud predictions incorporating image features obtained during and immediately following surgery (i.e., intraoperative and immediate postoperative images) exhibited significant improvement in area under the receiver operating characteristic (AUC): AUC = 0.72 ( CI 95 = 0.59 to 0.83) at 3 months and AUC = 0.69 ( CI 95 = 0.55 to 0.82) at 12 months. Conclusions: Predictive modeling of lumbar spine surgery outcomes was improved by incorporation of image-based features compared to analysis based on conventional demographic data. The SpineCloud framework could improve understanding of factors underlying outcome variability and warrants further investigation and validation in a larger patient cohort.

Automatic analysis of global spinal alignment from simple annotation of vertebral bodies.

Purpose: Measurement of global spinal alignment (GSA) is an important aspect of diagnosis and treatment evaluation for spinal deformity but is subject to a high level of inter-reader variability. Approach: Two methods for automatic GSA measurement are proposed to mitigate such variability and reduce the burden of manual measurements. Both approaches use vertebral labels in spine computed tomography (CT) as input: the first (EndSeg) segments vertebral endplates using input labels as seed points; and the second (SpNorm) computes a two-dimensional curvilinear fit to the input labels. Studies were performed to characterize the performance of EndSeg and SpNorm in comparison to manual GSA measurement by five clinicians, including measurements of proximal thoracic kyphosis, main thoracic kyphosis, and lumbar lordosis. Results: For the automatic methods, 93.8% of endplate angle estimates were within the inter-reader 95% confidence interval ( CI 95 ). All GSA measurements for the automatic methods were within the inter-reader CI 95 , and there was no statistically significant difference between automatic and manual methods. The SpNorm method appears particularly robust as it operates without segmentation. Conclusions: Such methods could improve the reproducibility and reliability of GSA measurements and are potentially suitable to applications in large datasets-e.g., for outcome assessment in surgical data science.

Learning-based deformable image registration: effect of statistical mismatch between train and test images.

Convolutional neural networks (CNNs) offer a promising means to achieve fast deformable image registration with accuracy comparable to conventional, physics-based methods. A persistent question with CNN methods, however, is whether they will be able to generalize to data outside of the training set. We investigated this question of mismatch between train and test data with respect to first- and second-order image statistics (e.g., spatial resolution, image noise, and power spectrum). A UNet-based architecture was built and trained on simulated CT images for various conditions of image noise (dose), spatial resolution, and deformation magnitude. Target registration error was measured as a function of the difference in statistical properties between the test and training data. Generally, registration error is minimized when the training data exactly match the statistics of the test data; however, networks trained with data exhibiting a diversity in statistical characteristics generalized well across the range of statistical conditions considered. Furthermore, networks trained on simulated image content with first- and second-order statistics selected to match that of real anatomical data were shown to provide reasonable registration performance on real anatomical content, offering potential new means for data augmentation. Characterizing the behavior of a CNN in the presence of statistical mismatch is an important step in understanding how these networks behave when deployed on new, unobserved data. Such characterization can inform decisions on whether retraining is necessary and can guide the data collection and/or augmentation process for training.

A Statistical Model for Rigid Image Registration Performance: The Influence of Soft-Tissue Deformation as a Confounding Noise Source.

Soft-tissue deformation presents a confounding factor to rigid image registration by introducing image content inconsistent with the underlying motion model, presenting non-correspondent structure with potentially high power, and creating local minima that challenge iterative optimization. In this paper, we introduce a model for registration performance that includes deformable soft tissue as a power-law noise distribution within a statistical framework describing the Cramer-Rao lower bound (CRLB) and root-mean-squared error (RMSE) in registration performance. The model incorporates both cross-correlation and gradient-based similarity metrics, and the model was tested in application to 3D-2D (CT-to-radiograph) and 3D-3D (CT-to-CT) image registration. Predictions accurately reflect the trends in registration error as a function of dose (quantum noise), and the choice of similarity metrics for both registration scenarios. Incorporating soft-tissue deformation as a noise source yields important insight on the limits of registration performance with respect to algorithm design and the clinical application or anatomical context. For example, the model quantifies the advantage of gradient-based similarity metrics in 3D-2D registration, identifies the low-dose limits of registration performance, and reveals the conditions for which the registration performance is fundamentally limited by soft-tissue deformation.

Virtual fluoroscopy for intraoperative C-arm positioning and radiation dose reduction.

Positioning of an intraoperative C-arm to achieve clear visualization of a particular anatomical feature often involves repeated fluoroscopic views, which cost time and radiation exposure to both the patient and surgical staff. A system for virtual fluoroscopy (called FluoroSim) that could dramatically reduce time- and dose-spent "fluoro-hunting" by leveraging preoperative computed tomography (CT), encoded readout of C-arm gantry position, and automatic 3D-2D image registration has been developed. The method is consistent with existing surgical workflow and does not require additional tracking equipment. Real-time virtual fluoroscopy was achieved via mechanical encoding of the C-arm motion, C-arm geometric calibration, and patient registration using a single radiograph. The accuracy, time, and radiation dose associated with C-arm positioning were measured for FluoroSim in comparison with conventional methods. Five radiology technologists were tasked with acquiring six standard pelvic views pertinent to sacro-illiac, anterior-inferior iliac spine, and superior-ramus screw placement in an anthropomorphic pelvis phantom using conventional and FluoroSim approaches. The positioning accuracy, exposure time, number of exposures, and total time for each trial were recorded, and radiation dose was characterized in terms of entrance skin dose and in-room scatter. The geometric accuracy of FluoroSim was measured to be [Formula: see text]. There was no significant difference ([Formula: see text]) observed in the accuracy or total elapsed time for C-arm positioning. However, the total fluoroscopy time required to achieve the desired view decreased by 4.1 s ([Formula: see text] for conventional, compared with [Formula: see text] for FluoroSim, [Formula: see text]), and the total number of exposures reduced by 4.0 ([Formula: see text] for conventional, compared with [Formula: see text] for FluoroSim, [Formula: see text]). These reductions amounted to a 50% to 78% decrease in patient entrance skin dose and a 55% to 70% reduction in in-room scatter. FluoroSim was found to reduce the radiation exposure required in C-arm positioning without diminishing positioning time or accuracy, providing a potentially valuable tool to assist technologists and surgeons.