Key Parameters of Tumor Epitope Immunogenicity Revealed Through a Consortium Approach Improve Neoantigen Prediction.

Many approaches to identify therapeutically relevant neoantigens couple tumor sequencing with bioinformatic algorithms and inferred rules of tumor epitope immunogenicity. However, there are no reference data to compare these approaches, and the parameters governing tumor epitope immunogenicity remain unclear. Here, we assembled a global consortium wherein each participant predicted immunogenic epitopes from shared tumor sequencing data. 608 epitopes were subsequently assessed for T cell binding in patient-matched samples. By integrating peptide features associated with presentation and recognition, we developed a model of tumor epitope immunogenicity that filtered out 98% of non-immunogenic peptides with a precision above 0.70. Pipelines prioritizing model features had superior performance, and pipeline alterations leveraging them improved prediction performance. These findings were validated in an independent cohort of 310 epitopes prioritized from tumor sequencing data and assessed for T cell binding. This data resource enables identification of parameters underlying effective anti-tumor immunity and is available to the research community.

A community challenge to predict clinical outcomes after immune checkpoint blockade in non-small cell lung cancer.

BACKGROUND: Predictive biomarkers of immune checkpoint inhibitor (ICI) efficacy are currently lacking for non-small cell lung cancer (NSCLC). Here, we describe the results from the Anti-PD-1 Response Prediction DREAM Challenge, a crowdsourced initiative that enabled the assessment of predictive models by using data from two randomized controlled clinical trials (RCTs) of ICIs in first-line metastatic NSCLC. METHODS: Participants developed and trained models using public resources. These were evaluated with data from the CheckMate 026 trial (NCT02041533), according to the model-to-data paradigm to maintain patient confidentiality. The generalizability of the models with the best predictive performance was assessed using data from the CheckMate 227 trial (NCT02477826). Both trials were phase III RCTs with a chemotherapy control arm, which supported the differentiation between predictive and prognostic models. Isolated model containers were evaluated using a bespoke strategy that considered the challenges of handling transcriptome data from clinical trials. RESULTS: A total of 59 teams participated, with 417 models submitted. Multiple predictive models, as opposed to a prognostic model, were generated for predicting overall survival, progression-free survival, and progressive disease status with ICIs. Variables within the models submitted by participants included tumor mutational burden (TMB), programmed death ligand 1 (PD-L1) expression, and gene-expression-based signatures. The best-performing models showed improved predictive power over reference variables, including TMB or PD-L1. CONCLUSIONS: This DREAM Challenge is the first successful attempt to use protected phase III clinical data for a crowdsourced effort towards generating predictive models for ICI clinical outcomes and could serve as a blueprint for similar efforts in other tumor types and disease states, setting a benchmark for future studies aiming to identify biomarkers predictive of ICI efficacy. TRIAL REGISTRATION: CheckMate 026; NCT02041533, registered January 22, 2014. CheckMate 227; NCT02477826, registered June 23, 2015.

Insights from the IronTract challenge: Optimal methods for mapping brain pathways from multi-shell diffusion MRI.

Limitations in the accuracy of brain pathways reconstructed by diffusion MRI (dMRI) tractography have received considerable attention. While the technical advances spearheaded by the Human Connectome Project (HCP) led to significant improvements in dMRI data quality, it remains unclear how these data should be analyzed to maximize tractography accuracy. Over a period of two years, we have engaged the dMRI community in the IronTract Challenge, which aims to answer this question by leveraging a unique dataset. Macaque brains that have received both tracer injections and ex vivo dMRI at high spatial and angular resolution allow a comprehensive, quantitative assessment of tractography accuracy on state-of-the-art dMRI acquisition schemes. We find that, when analysis methods are carefully optimized, the HCP scheme can achieve similar accuracy as a more time-consuming, Cartesian-grid scheme. Importantly, we show that simple pre- and post-processing strategies can improve the accuracy and robustness of many tractography methods. Finally, we find that fiber configurations that go beyond crossing (e.g., fanning, branching) are the most challenging for tractography. The IronTract Challenge remains open and we hope that it can serve as a valuable validation tool for both users and developers of dMRI analysis methods.

Cell-to-cell and type-to-type heterogeneity of signaling networks: insights from the crowd.

Recent technological developments allow us to measure the status of dozens of proteins in individual cells. This opens the way to understand the heterogeneity of complex multi-signaling networks across cells and cell types, with important implications to understand and treat diseases such as cancer. These technologies are, however, limited to proteins for which antibodies are available and are fairly costly, making predictions of new markers and of existing markers under new conditions a valuable alternative. To assess our capacity to make such predictions and boost further methodological development, we organized the Single Cell Signaling in Breast Cancer DREAM challenge. We used a mass cytometry dataset, covering 36 markers in over 4,000 conditions totaling 80 million single cells across 67 breast cancer cell lines. Through four increasingly difficult subchallenges, the participants predicted missing markers, new conditions, and the time-course response of single cells to stimuli in the presence and absence of kinase inhibitors. The challenge results show that despite the stochastic nature of signal transduction in single cells, the signaling events are tightly controlled and machine learning methods can accurately predict new experimental data.

Revisiting the T2 spectrum imaging inverse problem: Bayesian regularized non-negative least squares.

Multi-echo T2 magnetic resonance images contain information about the distribution of T2 relaxation times of compartmentalized water, from which we can estimate relevant brain tissue properties such as the myelin water fraction (MWF). Regularized non-negative least squares (NNLS) is the tool of choice for estimating non-parametric T2 spectra. However, the estimation is ill-conditioned, sensitive to noise, and highly affected by the employed regularization weight. The purpose of this study is threefold: first, we want to underline that the apparently innocuous use of two alternative parameterizations for solving the inverse problem, which we called the standard and alternative regularization forms, leads to different solutions; second, to assess the performance of both parameterizations; and third, to propose a new Bayesian regularized NNLS method (BayesReg). The performance of BayesReg was compared with that of two conventional approaches (L-curve and Chi-square (X2) fitting) using both regularization forms. We generated a large dataset of synthetic data, acquired in vivo human brain data in healthy participants for conducting a scan-rescan analysis, and correlated the myelin content derived from histology with the MWF estimated from ex vivo data. Results from synthetic data indicate that BayesReg provides accurate MWF estimates, comparable to those from L-curve and X2, and with better overall stability across a wider signal-to-noise range. Notably, we obtained superior results by using the alternative regularization form. The correlations reported in this study are higher than those reported in previous studies employing the same ex vivo and histological data. In human brain data, the estimated maps from L-curve and BayesReg were more reproducible. However, the T2 spectra produced by BayesReg were less affected by over-smoothing than those from L-curve. These findings suggest that BayesReg is a good alternative for estimating T2 distributions and MWF maps.

Tractography dissection variability: What happens when 42 groups dissect 14 white matter bundles on the same dataset?

White matter bundle segmentation using diffusion MRI fiber tractography has become the method of choice to identify white matter fiber pathways in vivo in human brains. However, like other analyses of complex data, there is considerable variability in segmentation protocols and techniques. This can result in different reconstructions of the same intended white matter pathways, which directly affects tractography results, quantification, and interpretation. In this study, we aim to evaluate and quantify the variability that arises from different protocols for bundle segmentation. Through an open call to users of fiber tractography, including anatomists, clinicians, and algorithm developers, 42 independent teams were given processed sets of human whole-brain streamlines and asked to segment 14 white matter fascicles on six subjects. In total, we received 57 different bundle segmentation protocols, which enabled detailed volume-based and streamline-based analyses of agreement and disagreement among protocols for each fiber pathway. Results show that even when given the exact same sets of underlying streamlines, the variability across protocols for bundle segmentation is greater than all other sources of variability in the virtual dissection process, including variability within protocols and variability across subjects. In order to foster the use of tractography bundle dissection in routine clinical settings, and as a fundamental analytical tool, future endeavors must aim to resolve and reduce this heterogeneity. Although external validation is needed to verify the anatomical accuracy of bundle dissections, reducing heterogeneity is a step towards reproducible research and may be achieved through the use of standard nomenclature and definitions of white matter bundles and well-chosen constraints and decisions in the dissection process.

Prediction of overall survival for patients with metastatic castration-resistant prostate cancer: development of a prognostic model through a crowdsourced challenge with open clinical trial data.

BACKGROUND: Improvements to prognostic models in metastatic castration-resistant prostate cancer have the potential to augment clinical trial design and guide treatment strategies. In partnership with Project Data Sphere, a not-for-profit initiative allowing data from cancer clinical trials to be shared broadly with researchers, we designed an open-data, crowdsourced, DREAM (Dialogue for Reverse Engineering Assessments and Methods) challenge to not only identify a better prognostic model for prediction of survival in patients with metastatic castration-resistant prostate cancer but also engage a community of international data scientists to study this disease. METHODS: Data from the comparator arms of four phase 3 clinical trials in first-line metastatic castration-resistant prostate cancer were obtained from Project Data Sphere, comprising 476 patients treated with docetaxel and prednisone from the ASCENT2 trial, 526 patients treated with docetaxel, prednisone, and placebo in the MAINSAIL trial, 598 patients treated with docetaxel, prednisone or prednisolone, and placebo in the VENICE trial, and 470 patients treated with docetaxel and placebo in the ENTHUSE 33 trial. Datasets consisting of more than 150 clinical variables were curated centrally, including demographics, laboratory values, medical history, lesion sites, and previous treatments. Data from ASCENT2, MAINSAIL, and VENICE were released publicly to be used as training data to predict the outcome of interest-namely, overall survival. Clinical data were also released for ENTHUSE 33, but data for outcome variables (overall survival and event status) were hidden from the challenge participants so that ENTHUSE 33 could be used for independent validation. Methods were evaluated using the integrated time-dependent area under the curve (iAUC). The reference model, based on eight clinical variables and a penalised Cox proportional-hazards model, was used to compare method performance. Further validation was done using data from a fifth trial-ENTHUSE M1-in which 266 patients with metastatic castration-resistant prostate cancer were treated with placebo alone. FINDINGS: 50 independent methods were developed to predict overall survival and were evaluated through the DREAM challenge. The top performer was based on an ensemble of penalised Cox regression models (ePCR), which uniquely identified predictive interaction effects with immune biomarkers and markers of hepatic and renal function. Overall, ePCR outperformed all other methods (iAUC 0·791; Bayes factor >5) and surpassed the reference model (iAUC 0·743; Bayes factor >20). Both the ePCR model and reference models stratified patients in the ENTHUSE 33 trial into high-risk and low-risk groups with significantly different overall survival (ePCR: hazard ratio 3·32, 95% CI 2·39-4·62, p<0·0001; reference model: 2·56, 1·85-3·53, p<0·0001). The new model was validated further on the ENTHUSE M1 cohort with similarly high performance (iAUC 0·768). Meta-analysis across all methods confirmed previously identified predictive clinical variables and revealed aspartate aminotransferase as an important, albeit previously under-reported, prognostic biomarker. INTERPRETATION: Novel prognostic factors were delineated, and the assessment of 50 methods developed by independent international teams establishes a benchmark for development of methods in the future. The results of this effort show that data-sharing, when combined with a crowdsourced challenge, is a robust and powerful framework to develop new prognostic models in advanced prostate cancer. FUNDING: Sanofi US Services, Project Data Sphere.

Unusual locations of primary subepithelial squamous cell carcinomas of the vulva.

OBJECTIVE: This study aimed to report 2 cases of squamous cell carcinoma (SCC) of the vulva, arising from unusual subepithelial locations. MATERIALS AND METHODS: The first case was of an 85-year-old woman with a 65-year history of a mass in the left labium majus. The second case was of a 54-year-old woman who presented with a 1-week history of a painful left inguinal mass. She had previously sought medical attention for a 2-year history of a left-sided painless vestibular mass. RESULTS: In the first case, a simple excision showed a purely dermal SCC with no attachment to the epidermis. After the diagnosis of SCC, a wide, deep local excision of the left side of the vulva and left inguinofemoral lymph node dissection were performed. Pathological findings showed no residual tumor in the vulva, and the lymph nodes were clear. Based on the long history of a mass at the same site, the pathogenesis of the SCC was considered to be malignant degeneration of a previously benign epidermal cyst.In the second case, SCC was diagnosed on fine-needle aspirations of the vulvar and groin masses. The patient was treated with primary chemoradiation. Subsequently, wide, deep local excision of the left side of the vulva and left inguinofemoral lymph node dissection were performed. No residual tumor was found in the vulva, although atrophic Bartholin gland tissue was found at the site of the SCC. One 5-mm inguinofemoral lymph node metastasis was found in the groin node dissection. In the absence of any evidence of another pathogenesis, it was believed that the SCC had arisen from an unusually anterior located Bartholin gland. CONCLUSIONS: Subepithelial SCC should be considered in the differential diagnosis of unusually located vulvar masses.

Evaluation of crowdsourced mortality prediction models as a framework for assessing artificial intelligence in medicine.

OBJECTIVE: Applications of machine learning in healthcare are of high interest and have the potential to improve patient care. Yet, the real-world accuracy of these models in clinical practice and on different patient subpopulations remains unclear. To address these important questions, we hosted a community challenge to evaluate methods that predict healthcare outcomes. We focused on the prediction of all-cause mortality as the community challenge question. MATERIALS AND METHODS: Using a Model-to-Data framework, 345 registered participants, coalescing into 25 independent teams, spread over 3 continents and 10 countries, generated 25 accurate models all trained on a dataset of over 1.1 million patients and evaluated on patients prospectively collected over a 1-year observation of a large health system. RESULTS: The top performing team achieved a final area under the receiver operator curve of 0.947 (95% CI, 0.942-0.951) and an area under the precision-recall curve of 0.487 (95% CI, 0.458-0.499) on a prospectively collected patient cohort. DISCUSSION: Post hoc analysis after the challenge revealed that models differ in accuracy on subpopulations, delineated by race or gender, even when they are trained on the same data. CONCLUSION: This is the largest community challenge focused on the evaluation of state-of-the-art machine learning methods in a healthcare system performed to date, revealing both opportunities and pitfalls of clinical AI.

A Fetal Brain magnetic resonance Acquisition Numerical phantom (FaBiAN).

Accurate characterization of in utero human brain maturation is critical as it involves complex and interconnected structural and functional processes that may influence health later in life. Magnetic resonance imaging is a powerful tool to investigate equivocal neurological patterns during fetal development. However, the number of acquisitions of satisfactory quality available in this cohort of sensitive subjects remains scarce, thus hindering the validation of advanced image processing techniques. Numerical phantoms can mitigate these limitations by providing a controlled environment with a known ground truth. In this work, we present FaBiAN, an open-source Fetal Brain magnetic resonance Acquisition Numerical phantom that simulates clinical T2-weighted fast spin echo sequences of the fetal brain. This unique tool is based on a general, flexible and realistic setup that includes stochastic fetal movements, thus providing images of the fetal brain throughout maturation comparable to clinical acquisitions. We demonstrate its value to evaluate the robustness and optimize the accuracy of an algorithm for super-resolution fetal brain magnetic resonance imaging from simulated motion-corrupted 2D low-resolution series compared to a synthetic high-resolution reference volume. We also show that the images generated can complement clinical datasets to support data-intensive deep learning methods for fetal brain tissue segmentation.

A Scalable Quality Assurance Process for Curating Oncology Electronic Health Records: The Project GENIE Biopharma Collaborative Approach.

PURPOSE: The American Association for Cancer Research Project Genomics Evidence Neoplasia Information Exchange Biopharma Collaborative is a multi-institution effort to build a pan-cancer repository of genomic and clinical data curated from the electronic health record. For the research community to be confident that data extracted from electronic health record text are reliable, transparency of the approach used to ensure data quality is essential. MATERIALS AND METHODS: Four institutions participating in AACR's Project GENIE created an observational cohort of patients with cancer for whom tumor molecular profiling data, therapeutic exposures, and treatment outcomes are available and will be shared publicly with the research community. A comprehensive approach to quality assurance included assessments of (1) feasibility of the curation model through pressure test cases; (2) accuracy through programmatic queries and comparison with source data; and (3) reproducibility via double curation and code review. RESULTS: Assessments of feasibility resulted in critical modifications to the curation directives. Queries and comparison with source data identified errors that were rectified via data correction and curator retraining. Assessment of intercurator reliability indicated a reliable curation model. CONCLUSION: The transparent quality assurance processes for the GENIE BPC data ensure that the data can be used for analyses that support clinical decision making and advances in precision oncology.

Variability and reproducibility of multi-echo T2 relaxometry: Insights from multi-site, multi-session and multi-subject MRI acquisitions.

Quantitative magnetic resonance imaging (qMRI) can increase the specificity and sensitivity of conventional weighted MRI to underlying pathology by comparing meaningful physical or chemical parameters, measured in physical units, with normative values acquired in a healthy population. This study focuses on multi-echo T2 relaxometry, a qMRI technique that probes the complex tissue microstructure by differentiating compartment-specific T2 relaxation times. However, estimation methods are still limited by their sensitivity to the underlying noise. Moreover, estimating the model's parameters is challenging because the resulting inverse problem is ill-posed, requiring advanced numerical regularization techniques. As a result, the estimates from distinct regularization strategies are different. In this work, we aimed to investigate the variability and reproducibility of different techniques for estimating the transverse relaxation time of the intra- and extra-cellular space (T2IE) in gray (GM) and white matter (WM) tissue in a clinical setting, using a multi-site, multi-session, and multi-run T2 relaxometry dataset. To this end, we evaluated three different techniques for estimating the T2 spectra (two regularized non-negative least squares methods and a machine learning approach). Two independent analyses were performed to study the effect of using raw and denoised data. For both the GM and WM regions, and the raw and denoised data, our results suggest that the principal source of variance is the inter-subject variability, showing a higher coefficient of variation (CoV) than those estimated for the inter-site, inter-session, and inter-run, respectively. For all reconstruction methods studied, the CoV ranged between 0.32 and 1.64%. Interestingly, the inter-session variability was close to the inter-scanner variability with no statistical differences, suggesting that T2IE is a robust parameter that could be employed in multi-site neuroimaging studies. Furthermore, the three tested methods showed consistent results and similar intra-class correlation (ICC), with values superior to 0.7 for most regions. Results from raw data were slightly more reproducible than those from denoised data. The regularized non-negative least squares method based on the L-curve technique produced the best results, with ICC values ranging from 0.72 to 0.92.

A Crowdsourcing Approach to Develop Machine Learning Models to Quantify Radiographic Joint Damage in Rheumatoid Arthritis.

Importance: An automated, accurate method is needed for unbiased assessment quantifying accrual of joint space narrowing and erosions on radiographic images of the hands and wrists, and feet for clinical trials, monitoring of joint damage over time, assisting rheumatologists with treatment decisions. Such a method has the potential to be directly integrated into electronic health records. Objectives: To design and implement an international crowdsourcing competition to catalyze the development of machine learning methods to quantify radiographic damage in rheumatoid arthritis (RA). Design, Setting, and Participants: This diagnostic/prognostic study describes the Rheumatoid Arthritis 2-Dialogue for Reverse Engineering Assessment and Methods (RA2-DREAM Challenge), which used existing radiographic images and expert-curated Sharp-van der Heijde (SvH) scores from 2 clinical studies (674 radiographic sets from 562 patients) for training (367 sets), leaderboard (119 sets), and final evaluation (188 sets). Challenge participants were tasked with developing methods to automatically quantify overall damage (subchallenge 1), joint space narrowing (subchallenge 2), and erosions (subchallenge 3). The challenge was finished on June 30, 2020. Main Outcomes and Measures: Scores derived from submitted algorithms were compared with the expert-curated SvH scores, and a baseline model was created for benchmark comparison. Performances were ranked using weighted root mean square error (RMSE). The performance and reproductivity of each algorithm was assessed using Bayes factor from bootstrapped data, and further evaluated with a postchallenge independent validation data set. Results: The RA2-DREAM Challenge received a total of 173 submissions from 26 participants or teams in 7 countries for the leaderboard round, and 13 submissions were included in the final evaluation. The weighted RMSEs metric showed that the winning algorithms produced scores that were very close to the expert-curated SvH scores. Top teams included Team Shirin for subchallenge 1 (weighted RMSE, 0.44), HYL-YFG (Hongyang Li and Yuanfang Guan) subchallenge 2 (weighted RMSE, 0.38), and Gold Therapy for subchallenge 3 (weighted RMSE, 0.43). Bootstrapping/Bayes factor approach and the postchallenge independent validation confirmed the reproducibility and the estimation concordance indices between final evaluation and postchallenge independent validation data set were 0.71 for subchallenge 1, 0.78 for subchallenge 2, and 0.82 for subchallenge 3. Conclusions and Relevance: The RA2-DREAM Challenge resulted in the development of algorithms that provide feasible, quick, and accurate methods to quantify joint damage in RA. Ultimately, these methods could help research studies on RA joint damage and may be integrated into electronic health records to help clinicians serve patients better by providing timely, reliable, and quantitative information for making treatment decisions to prevent further damage.

AACR Project GENIE: 100,000 Cases and Beyond.

The American Association for Cancer Research (AACR) Project Genomics Evidence Neoplasia Information Exchange (GENIE) is an international pan-cancer registry with the goal to inform cancer research and clinical care worldwide. Founded in late 2015, the milestone GENIE 9.1-public release contains data from >110,000 tumors from >100,000 people treated at 19 cancer centers from the United States, Canada, the United Kingdom, France, the Netherlands, and Spain. Here, we demonstrate the use of these real-world data, harmonized through a centralized data resource, to accurately predict enrollment on genome-guided trials, discover driver alterations in rare tumors, and identify cancer types without actionable mutations that could benefit from comprehensive genomic analysis. The extensible data infrastructure and governance framework support additional deep patient phenotyping through biopharmaceutical collaborations and expansion to include new data types such as cell-free DNA sequencing. AACR Project GENIE continues to serve a global precision medicine knowledge base of increasing impact to inform clinical decision-making and bring together cancer researchers internationally. SIGNIFICANCE: AACR Project GENIE has now accrued data from >110,000 tumors, placing it among the largest repository of publicly available, clinically annotated genomic data in the world. GENIE has emerged as a powerful resource to evaluate genome-guided clinical trial design, uncover drivers of cancer subtypes, and inform real-world use of genomic data. This article is highlighted in the In This Issue feature, p. 2007.

Enhanced optical properties of chemical vapor deposited single crystal diamond by low-pressure/high-temperature annealing.

Single crystal diamond produced by chemical vapor deposition (CVD) at very high growth rates (up to 150 microm/h) has been successfully annealed without graphitization at temperatures up to 2200 degrees C and pressures <300 torr. Crystals were annealed in a hydrogen environment by using microwave plasma techniques for periods of time ranging from a fraction of minute to a few hours. This low-pressure/high-temperature (LPHT) annealing enhances the optical properties of this high-growth rate CVD single crystal diamond. Significant decreases are observed in UV, visible, and infrared absorption and photoluminescence spectra. The decrease in optical absorption after the LPHT annealing arises from the changes in defect structure associated with hydrogen incorporation during CVD growth. There is a decrease in sharp line spectral features indicating a reduction in nitrogen-vacancy-hydrogen (NVH(-)) defects. These measurements indicate an increase in relative concentration of nitrogen-vacancy (NV) centers in nitrogen-containing LPHT-annealed diamond as compared with as-grown CVD material. The large overall changes in optical properties and the specific types of alterations in defect structure induced by this facile LPHT processing of high-growth rate single-crystal CVD diamond will be useful in the creation of diamond for a variety of scientific and technological applications.

MPRAGE to MP2RAGE UNI translation via generative adversarial network improves the automatic tissue and lesion segmentation in multiple sclerosis patients.

BACKGROUND AND OBJECTIVE: Compared to the conventional magnetization-prepared rapid gradient-echo imaging (MPRAGE) MRI sequence, the specialized magnetization prepared 2 rapid acquisition gradient echoes (MP2RAGE) shows a higher brain tissue and lesion contrast in multiple sclerosis (MS) patients. The goal of this work is to retrospectively generate realistic-looking MP2RAGE uniform images (UNI) from already acquired MPRAGE images in order to improve the automatic lesion and tissue segmentation. METHODS: For this task we propose a generative adversarial network (GAN). Multi-contrast MRI data of 12 healthy controls and 44 patients diagnosed with MS was retrospectively analyzed. Imaging was acquired at 3T using a SIEMENS scanner with MPRAGE, MP2RAGE, FLAIR, and DIR sequences. We train the GAN with both healthy controls and MS patients to generate synthetic MP2RAGE UNI images. These images were then compared to the real MP2RAGE UNI (considered as ground truth) analyzing the output of automatic brain tissue and lesion segmentation tools. Reference-based metrics as well as the lesion-wise true and false positives, Dice coefficient, and volume difference were considered for the evaluation. Statistical differences were assessed with the Wilcoxon signed-rank test. RESULTS: The synthetic MP2RAGE UNI significantly improves the lesion and tissue segmentation masks in terms of Dice coefficient and volume difference (p-values < 0.001) compared to the MPRAGE. For the segmentation metrics analyzed no statistically significant differences are found between the synthetic and acquired MP2RAGE UNI. CONCLUSION: Synthesized MP2RAGE UNI images are visually realistic and improve the output of automatic segmentation tools.

Model-informed machine learning for multi-component T2 relaxometry.

Recovering the T2 distribution from multi-echo T2 magnetic resonance (MR) signals is challenging but has high potential as it provides biomarkers characterizing the tissue micro-structure, such as the myelin water fraction (MWF). In this work, we propose to combine machine learning and aspects of parametric (fitting from the MRI signal using biophysical models) and non-parametric (model-free fitting of the T2 distribution from the signal) approaches to T2 relaxometry in brain tissue by using a multi-layer perceptron (MLP) for the distribution reconstruction. For training our network, we construct an extensive synthetic dataset derived from biophysical models in order to constrain the outputs with a priori knowledge of in vivo distributions. The proposed approach, called Model-Informed Machine Learning (MIML), takes as input the MR signal and directly outputs the associated T2 distribution. We evaluate MIML in comparison to a Gaussian Mixture Fitting (parametric) and Regularized Non-Negative Least Squares algorithms (non-parametric) on synthetic data, an ex vivo scan, and high-resolution scans of healthy subjects and a subject with Multiple Sclerosis. In synthetic data, MIML provides more accurate and noise-robust distributions. In real data, MWF maps derived from MIML exhibit the greatest conformity to anatomical scans, have the highest correlation to a histological map of myelin volume, and the best unambiguous lesion visualization and localization, with superior contrast between lesions and normal appearing tissue. In whole-brain analysis, MIML is 22 to 4980 times faster than the non-parametric and parametric methods, respectively.

Fetal Brain Biometric Measurements on 3D Super-Resolution Reconstructed T2-Weighted MRI: An Intra- and Inter-observer Agreement Study.

We present the comparison of two-dimensional (2D) fetal brain biometry on magnetic resonance (MR) images using orthogonal 2D T2-weighted sequences (T2WSs) vs. one 3D super-resolution (SR) reconstructed volume and evaluation of the level of confidence and concordance between an experienced pediatric radiologist (obs1) and a junior radiologist (obs2). Twenty-five normal fetal brain MRI scans (18-34 weeks of gestation) including orthogonal 3-mm-thick T2WSs were analyzed retrospectively. One 3D SR volume was reconstructed per subject based on multiple series of T2WSs. The two observers performed 11 2D biometric measurements (specifying their level of confidence) on T2WS and SR volumes. Measurements were compared using the paired Wilcoxon rank sum test between observers for each dataset (T2WS and SR) and between T2WS and SR for each observer. Bland-Altman plots were used to assess the agreement between each pair of measurements. Measurements were made with low confidence in three subjects by obs1 and in 11 subjects by obs2 (mostly concerning the length of the corpus callosum on T2WS). Inter-rater intra-dataset comparisons showed no significant difference (p > 0.05), except for brain axial biparietal diameter (BIP) on T2WS and for brain and skull coronal BIP and coronal transverse cerebellar diameter (DTC) on SR. None of them remained significant after correction for multiple comparisons. Inter-dataset intra-rater comparisons showed statistical differences in brain axial and coronal BIP for both observers, skull coronal BIP for obs1, and axial and coronal DTC for obs2. After correction for multiple comparisons, only axial brain BIP remained significantly different, but differences were small (2.95 ± 1.73 mm). SR allows similar fetal brain biometry as compared to using the conventional T2WS while improving the level of confidence in the measurements and using a single reconstructed volume.

Comparison of non-parametric T2 relaxometry methods for myelin water quantification.

Multi-component T2 relaxometry allows probing tissue microstructure by assessing compartment-specific T2 relaxation times and water fractions, including the myelin water fraction. Non-negative least squares (NNLS) with zero-order Tikhonov regularization is the conventional method for estimating smooth T2 distributions. Despite the improved estimation provided by this method compared to non-regularized NNLS, the solution is still sensitive to the underlying noise and the regularization weight. This is especially relevant for clinically achievable signal-to-noise ratios. In the literature of inverse problems, various well-established approaches to promote smooth solutions, including first-order and second-order Tikhonov regularization, and different criteria for estimating the regularization weight have been proposed, such as L-curve, Generalized Cross-Validation, and Chi-square residual fitting. However, quantitative comparisons between the available reconstruction methods for computing the T2 distribution, and between different approaches for selecting the optimal regularization weight, are lacking. In this study, we implemented and evaluated ten reconstruction algorithms, resulting from the individual combinations of three penalty terms with three criteria to estimate the regularization weight, plus non-regularized NNLS. Their performance was evaluated both in simulated data and real brain MRI data acquired from healthy volunteers through a scan-rescan repeatability analysis. Our findings demonstrate the need for regularization. As a result of this work, we provide a list of recommendations for selecting the optimal reconstruction algorithms based on the acquired data. Moreover, the implemented methods were packaged in a freely distributed toolbox to promote reproducible research, and to facilitate further research and the use of this promising quantitative technique in clinical practice.

A Continuously Benchmarked and Crowdsourced Challenge for Rapid Development and Evaluation of Models to Predict COVID-19 Diagnosis and Hospitalization.

Importance: Machine learning could be used to predict the likelihood of diagnosis and severity of illness. Lack of COVID-19 patient data has hindered the data science community in developing models to aid in the response to the pandemic. Objectives: To describe the rapid development and evaluation of clinical algorithms to predict COVID-19 diagnosis and hospitalization using patient data by citizen scientists, provide an unbiased assessment of model performance, and benchmark model performance on subgroups. Design, Setting, and Participants: This diagnostic and prognostic study operated a continuous, crowdsourced challenge using a model-to-data approach to securely enable the use of regularly updated COVID-19 patient data from the University of Washington by participants from May 6 to December 23, 2020. A postchallenge analysis was conducted from December 24, 2020, to April 7, 2021, to assess the generalizability of models on the cumulative data set as well as subgroups stratified by age, sex, race, and time of COVID-19 test. By December 23, 2020, this challenge engaged 482 participants from 90 teams and 7 countries. Main Outcomes and Measures: Machine learning algorithms used patient data and output a score that represented the probability of patients receiving a positive COVID-19 test result or being hospitalized within 21 days after receiving a positive COVID-19 test result. Algorithms were evaluated using area under the receiver operating characteristic curve (AUROC) and area under the precision recall curve (AUPRC) scores. Ensemble models aggregating models from the top challenge teams were developed and evaluated. Results: In the analysis using the cumulative data set, the best performance for COVID-19 diagnosis prediction was an AUROC of 0.776 (95% CI, 0.775-0.777) and an AUPRC of 0.297, and for hospitalization prediction, an AUROC of 0.796 (95% CI, 0.794-0.798) and an AUPRC of 0.188. Analysis on top models submitting to the challenge showed consistently better model performance on the female group than the male group. Among all age groups, the best performance was obtained for the 25- to 49-year age group, and the worst performance was obtained for the group aged 17 years or younger. Conclusions and Relevance: In this diagnostic and prognostic study, models submitted by citizen scientists achieved high performance for the prediction of COVID-19 testing and hospitalization outcomes. Evaluation of challenge models on demographic subgroups and prospective data revealed performance discrepancies, providing insights into the potential bias and limitations in the models.