Multi-center validation of an artificial intelligence system for detection of COVID-19 on chest radiographs in symptomatic patients.

OBJECTIVES: While chest radiograph (CXR) is the first-line imaging investigation in patients with respiratory symptoms, differentiating COVID-19 from other respiratory infections on CXR remains challenging. We developed and validated an AI system for COVID-19 detection on presenting CXR. METHODS: A deep learning model (RadGenX), trained on 168,850 CXRs, was validated on a large international test set of presenting CXRs of symptomatic patients from 9 study sites (US, Italy, and Hong Kong SAR) and 2 public datasets from the US and Europe. Performance was measured by area under the receiver operator characteristic curve (AUC). Bootstrapped simulations were performed to assess performance across a range of potential COVID-19 disease prevalence values (3.33 to 33.3%). Comparison against international radiologists was performed on an independent test set of 852 cases. RESULTS: RadGenX achieved an AUC of 0.89 on 4-fold cross-validation and an AUC of 0.79 (95%CI 0.78-0.80) on an independent test cohort of 5,894 patients. Delong's test showed statistical differences in model performance across patients from different regions (p < 0.01), disease severity (p < 0.001), gender (p < 0.001), and age (p = 0.03). Prevalence simulations showed the negative predictive value increases from 86.1% at 33.3% prevalence, to greater than 98.5% at any prevalence below 4.5%. Compared with radiologists, McNemar's test showed the model has higher sensitivity (p < 0.001) but lower specificity (p < 0.001). CONCLUSION: An AI model that predicts COVID-19 infection on CXR in symptomatic patients was validated on a large international cohort providing valuable context on testing and performance expectations for AI systems that perform COVID-19 prediction on CXR. KEY POINTS: • An AI model developed using CXRs to detect COVID-19 was validated in a large multi-center cohort of 5,894 patients from 9 prospectively recruited sites and 2 public datasets. • Differences in AI model performance were seen across region, disease severity, gender, and age. • Prevalence simulations on the international test set demonstrate the model's NPV is greater than 98.5% at any prevalence below 4.5%.

Betting strategies on fluctuations in the transient response of greenhouse warming.

We examine a series of betting strategies on the transient response of greenhouse warming, expressed by changes in 15-year mean global surface temperature from one 15-year period to the next. Over the last century, these bets are increasingly dominated by positive changes (warming), reflecting increasing greenhouse forcing and its rising contribution to temperature changes on this time scale. The greenhouse contribution to 15-year trends is now of a similar magnitude to typical naturally occurring 15-year trends. Negative 15-year changes (decreases) have not occurred since about 1970, and are still possible, but now rely on large, and therefore infrequent, natural variations. Model projections for even intermediate warming scenarios show very low likelihoods of obtaining negative 15-year changes over the coming century. Betting against greenhouse warming, even on these short time scales, is no longer a rational proposition.

Fishing elevates variability in the abundance of exploited species.

The separation of the effects of environmental variability from the impacts of fishing has been elusive, but is essential for sound fisheries management. We distinguish environmental effects from fishing effects by comparing the temporal variability of exploited versus unexploited fish stocks living in the same environments. Using the unique suite of 50-year-long larval fish surveys from the California Cooperative Oceanic Fisheries Investigations we analyse fishing as a treatment effect in a long-term ecological experiment. Here we present evidence from the marine environment that exploited species exhibit higher temporal variability in abundance than unexploited species. This remains true after accounting for life-history effects, abundance, ecological traits and phylogeny. The increased variability of exploited populations is probably caused by fishery-induced truncation of the age structure, which reduces the capacity of populations to buffer environmental events. Therefore, to avoid collapse, fisheries must be managed not only to sustain the total viable biomass but also to prevent the significant truncation of age structure. The double jeopardy of fishing to potentially deplete stock sizes and, more immediately, to amplify the peaks and valleys of population variability, calls for a precautionary management approach.