Receiver Operating Characteristic (ROC) Curves: The Basics and Beyond.

Diagnostic tests and clinical prediction rules are frequently used to help estimate the probability of a disease or outcome. How well a test or rule distinguishes between disease or no disease (discrimination) can be measured by plotting a receiver operating characteristic (ROC) curve and calculating the area under it (AUROC). In this paper, we review the features of ROC curves and interpretation of ROC curves and AUROC values. We highlight 5 underappreciated features of ROC curves: (1) the slope of the ROC curve over a test result interval is the likelihood ratio for that interval; (2) the optimal cutoff for calling a test positive depends not only on the shape of the ROC curve, but also on the pretest probability of disease and relative harms of false-positive and false-negative results; (3) the AUROC measures discrimination only, not the accuracy of the predicted probabilities; (4) the AUROC is not a good measure of discrimination if the slope of the ROC curve is not consistently decreasing; and (5) the AUROC can be increased by including a large number of people correctly identified as being at very low risk for the outcome of interest. We illustrate this last concept using 3 published studies.

A structure-function analysis shows SARS-CoV-2 BA.2.86 balances antibody escape and ACE2 affinity.

BA.2.86, a recently described sublineage of SARS-CoV-2 Omicron, contains many mutations in the spike gene. It appears to have originated from BA.2 and is distinct from the XBB variants responsible for many infections in 2023. The global spread and plethora of mutations in BA.2.86 has caused concern that it may possess greater immune-evasive potential, leading to a new wave of infection. Here, we examine the ability of BA.2.86 to evade the antibody response to infection using a panel of vaccinated or naturally infected sera and find that it shows marginally less immune evasion than XBB.1.5. We locate BA.2.86 in the antigenic landscape of recent variants and look at its ability to escape panels of potent monoclonal antibodies generated against contemporary SARS-CoV-2 infections. We demonstrate, and provide a structural explanation for, increased affinity of BA.2.86 to ACE2, which may increase transmissibility.

Determining the Association Between the Origin of Sepsis and the Severity of Sepsis in Intensive Care Unit (ICU) Patients Using Acute Physiology and Chronic Health Evaluation (APACHE) IV.

Objective The objective of this study is to compare the outcomes of hospital mortality, the requirement of invasive ventilation, vasopressor requirement, duration of vasopressor requirement, and duration of intensive care unit (ICU) stay among the different causes of sepsis and to determine which cause of sepsis had the most severe outcomes. Methods A retrospective chart review was done in critically ill adult patients who were admitted with sepsis to the ICU from July 2017 until July 2019. Acute Physiology and Chronic Health Evaluation (APACHE) IV scores were calculated on patients admitted to ICU on day one of ICU admission. Each patient was then evaluated for outcomes of hospital mortality, need for invasive ventilation, requirement of vasopressors, duration of vasopressors, and duration of ICU stay. The outcomes were then compared between the different sources of sepsis to determine which source of sepsis had the highest severity. Results In total, 176 patients were included in the study. Ninety-three patients were admitted with respiratory sepsis, 26 patients were admitted with gastrointestinal sepsis, 31 patients were admitted with urosepsis, and 26 patients were admitted with other miscellaneous causes of sepsis. The hospital mortality was highest in the respiratory sepsis group at 32%, with a trend towards statistical significance with a P value of 0.057. ICU stay duration was highest in patients with respiratory sepsis at six days, with a statistically significant P value of < 0.001. The need for invasive ventilation was highest in patients with respiratory sepsis at 64%, with a statistically significant P value of < 0.001. The requirement of vasopressor support was highest in patients with respiratory sepsis at 47% and the duration of vasopressors was highest in both respiratory and gastrointestinal sepsis at three days, however, there was no statistical significance. Conclusion Among the different origins of sepsis, the patients with respiratory sepsis had the most severe outcomes, with the highest need for invasive ventilation and the highest ICU stay duration.

Clinical Course of Children 1 to 24 Months Old With Positive Urine Cultures Without Pyuria.

OBJECTIVE: To describe the clinical course of children with positive urine cultures without pyuria who were not given antibiotics initially, identify predictors of subsequent antibiotic treatment, and evaluate the association between subsequent treatment and urinary tract infection (UTI) within 30 days. METHODS: We conducted a multicenter retrospective cohort study of children 1 to 24 months old who had positive urine cultures without pyuria and who were not started on antibiotics upon presentation to 3 health care systems from 2010 to 2021. Outcomes included clinical status at the time urine cultures resulted, escalation of care (emergency department visit or hospitalization) and subsequent antibiotic treatment within 7 days, and subsequent UTI within 30 days of presentation. RESULTS: Of 202 included children, 61% were female and median age was 9 months. Of 151 patients with clinical status information when cultures resulted, 107 (70.8%, 95% confidence interval [CI] 62.9-77.9%) were improved. Two of 202 children (1.0%, 95% CI 0.2-4.0%) experienced care escalation. Antibiotics were started in 142 (82.2%) children, and treatment was associated with prior UTI (risk ratio [RR] 1.20, 95% CI 1.15-1.26) and lack of improvement (RR 1.22, 95% CI 1.13-1.33). Subsequent UTI was diagnosed in 2 of 164 (1.2%, 95% CI 0.1-4.3%) treated and 0 of 36 (0%, 95% CI 0-9.7%) untreated children. CONCLUSIONS: Seventy percent of children with positive urine cultures without pyuria improved before starting antibiotics; however, >80% were ultimately treated. Future research should study the impact of diagnostic stewardship interventions and various urine testing strategies to optimize the management of children evaluated for UTI.retain-->.

Maternal treatment with selective serotonin reuptake inhibitors during pregnancy and delayed neonatal adaptation: a population-based cohort study.

OBJECTIVE: Selective serotonin reuptake inhibitor (SSRI) use is common in pregnancy. It is associated with delayed neonatal adaptation. Most previous studies have not adjusted for the severity of maternal mental health disorders or examined the impact of SSRI type and dosage. We examined whether treatment with SSRIs in late pregnancy (after 20 weeks) is associated with delayed neonatal adaptation independent of maternal depression and anxiety. DESIGN, SETTING AND PATIENTS: Retrospective population-based birth cohort of 280 090 term infants born at 15 Kaiser Permanente Northern California hospitals, 2011-2019. Individual-level pharmacy, maternal, pregnancy and neonatal data were obtained from electronic medical records. EXPOSURE: Dispensed maternal SSRI prescription after 20 weeks of pregnancy. MAIN OUTCOME MEASURES: Delayed neonatal adaptation defined as a 5 min Apgar score ≤5, resuscitation at birth or admission to a neonatal intensive care unit for respiratory support. Secondary outcomes included each individual component of the primary outcome and more severe neonatal outcomes (pulmonary hypertension, hypoxic-ischaemic encephalopathy and seizures). RESULTS: 7573 (2.7%) infants were exposed to SSRIs in late pregnancy. Delayed neonatal adaptation occurred in 11.2% of exposed vs 4.4% of unexposed infants (relative risk 2.52 (95% CI 2.36 to 2.70)). After multivariable adjustment, there was an association between SSRI exposure and delayed neonatal adaptation (adjusted OR 2.14 (95% CI 1.96 to 2.32)). This association was dose dependent. Escitalopram and fluoxetine were associated with the highest risk of delayed neonatal adaptation. CONCLUSIONS: Infants exposed to SSRIs have increased risks of delayed adaptation in a type and dose-dependent relationship, pointing toward a causal relationship.

Perinatal Hypoxic-Ischemic Encephalopathy: Incidence Over Time Within a Modern US Birth Cohort.

BACKGROUND: Recent studies suggest that the incidence of perinatal hypoxic-ischemic encephalopathy (HIE) may be increasing in developed countries. However, this observed increase may be due to increased ascertainment and increased treatment with therapeutic hypothermia rather than an increase in disease burden. In a US population-based cross-sectional study, we determined the incidence of perinatal HIE over time. METHODS: The study population included all 289,793 live-born infants ≥35 weeks gestational age born at 15 Kaiser Permanente Northern California hospitals between 2012 and 2019. Perinatal HIE was defined as the presence of both neonatal acidosis (i.e., cord blood pH < 7 or base deficit ≥10, or base deficit ≥10 on first infant gas) and neonatal encephalopathy confirmed by medical record review. Hospital discharge diagnoses of HIE were determined by extracting International Classification of Disease diagnostic codes for HIE assigned upon hospital discharge. RESULTS: The population incidence of perinatal HIE was 1.7 per 1000. Although the incidence of perinatal HIE did not change significantly, both hospital discharge diagnoses of HIE and treatment with therapeutic hypothermia increased significantly during the study period. The sensitivity and positive predictive value of a hospital discharge diagnosis of HIE for identifying perinatal HIE confirmed by chart review were 72% and 79%, respectively. CONCLUSIONS: During the study years, the incidence of perinatal HIE remained stable despite increases in hospital discharge diagnoses of HIE and in the use of therapeutic hypothermia. Our findings underscore the importance of applying stringent diagnostic criteria when diagnosing this complex condition.

CD4+ and CD8+ T cells and antibodies are associated with protection against Delta vaccine breakthrough infection: a nested case-control study within the PITCH study.

IMPORTANCE: Defining correlates of protection against severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) vaccine breakthrough infection informs vaccine policy for booster doses and future vaccine designs. Existing studies demonstrate humoral correlates of protection, but the role of T cells in protection is still unclear. In this study, we explore antibody and T cell immune responses associated with protection against Delta variant vaccine breakthrough infection in a well-characterized cohort of UK Healthcare Workers (HCWs). We demonstrate evidence to support a role for CD4+ and CD8+ T cells as well as antibodies against Delta vaccine breakthrough infection. In addition, our results suggest a potential role for cross-reactive T cells in vaccine breakthrough.

The walking man approach to interpreting the receiver operating characteristic curve and area under the receiver operating characteristic curve.

An accuracy study of a test that produces a wide range of results will often present a receiver operating characteristic (ROC) curve and report the area under the ROC curve (AUROC). The AUROC is a summary measure of how well the test discriminates between those with the condition or disease in question and those without it. A test that perfectly separates individuals with and without the condition has an AUROC of 1.0, and a test that doesn't separate them at all has an AUROC of 0.5. The AUROC is also the probability that a random individual with the condition will have a more abnormal test result than a random individual without the condition. In this Key Concepts article, we present our "walking man" approach to understanding ROC curves and the AUROC.

Rapid virtual fractional flow reserve using 3D computational fluid dynamics.

Aims: Over the last ten years, virtual Fractional Flow Reserve (vFFR) has improved the utility of Fractional Flow Reserve (FFR), a globally recommended assessment to guide coronary interventions. Although the speed of vFFR computation has accelerated, techniques utilising full 3D computational fluid dynamics (CFD) solutions rather than simplified analytical solutions still require significant time to compute. Methods and results: This study investigated the speed, accuracy and cost of a novel 3D-CFD software method based upon a graphic processing unit (GPU) computation, compared with the existing fastest central processing unit (CPU)-based 3D-CFD technique, on 40 angiographic cases. The novel GPU simulation was significantly faster than the CPU method (median 31.7 s (Interquartile Range (IQR) 24.0-44.4s) vs. 607.5 s (490-964 s), P < 0.0001). The novel GPU technique was 99.6% (IQR 99.3-99.9) accurate relative to the CPU method. The initial cost of the GPU hardware was greater than the CPU (£4080 vs. £2876), but the median energy consumption per case was significantly less using the GPU method (8.44 (6.80-13.39) Wh vs. 2.60 (2.16-3.12) Wh, P < 0.0001). Conclusion: This study demonstrates that vFFR can be computed using 3D-CFD with up to 28-fold acceleration than previous techniques with no clinically significant sacrifice in accuracy.

Establishing SARS-CoV-2 membrane protein-specific antibodies as a valuable serological target via high-content microscopy.

The prevalence and strength of serological responses mounted toward SARS-CoV-2 proteins other than nucleocapsid (N) and spike (S), which may be of use as additional serological markers, remains underexplored. Using high-content microscopy to assess antibody responses against full-length StrepTagged SARS-CoV-2 proteins, we found that 85% (166/196) of unvaccinated individuals with RT-PCR confirmed SARS-CoV-2 infections and 74% (31/42) of individuals infected after being vaccinated developed detectable IgG against the structural protein M, which is higher than previous estimates. Compared with N antibodies, M IgG displayed a shallower time-dependent decay and greater specificity. Sensitivity for SARS-CoV-2 seroprevalence was enhanced when N and M IgG detection was combined. These findings indicate that screening for M seroconversion may be a good approach for detecting additional vaccine breakthrough infections and highlight the potential to use HCM as a rapidly deployable method to identify the most immunogenic targets of newly emergent pathogens.

Research Methods: Diagnostic Test Characteristics.

The goal of a diagnostic test is to provide information on the probability of disease. In this article, we review the principles of diagnostic test characteristics, including sensitivity, specificity, positive and negative predictive value, receiver operating characteristics curves, likelihood ratios, and interval likelihood ratios. We illustrate how interval likelihood ratios optimize the information that can be obtained from test results that can take on >2 values, how they are reflected in the slope of the receiver operating characteristics curve, and how they can be easily calculated from published data.

Evolution of long-term vaccine-induced and hybrid immunity in healthcare workers after different COVID-19 vaccine regimens.

BACKGROUND: Both infection and vaccination, alone or in combination, generate antibody and T cell responses against severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). However, the maintenance of such responses-and hence protection from disease-requires careful characterization. In a large prospective study of UK healthcare workers (HCWs) (Protective Immunity from T Cells in Healthcare Workers [PITCH], within the larger SARS-CoV-2 Immunity and Reinfection Evaluation [SIREN] study), we previously observed that prior infection strongly affected subsequent cellular and humoral immunity induced after long and short dosing intervals of BNT162b2 (Pfizer/BioNTech) vaccination. METHODS: Here, we report longer follow-up of 684 HCWs in this cohort over 6-9 months following two doses of BNT162b2 or AZD1222 (Oxford/AstraZeneca) vaccination and up to 6 months following a subsequent mRNA booster vaccination. FINDINGS: We make three observations: first, the dynamics of humoral and cellular responses differ; binding and neutralizing antibodies declined, whereas T and memory B cell responses were maintained after the second vaccine dose. Second, vaccine boosting restored immunoglobulin (Ig) G levels; broadened neutralizing activity against variants of concern, including Omicron BA.1, BA.2, and BA.5; and boosted T cell responses above the 6-month level after dose 2. Third, prior infection maintained its impact driving larger and broader T cell responses compared with never-infected people, a feature maintained until 6 months after the third dose. CONCLUSIONS: Broadly cross-reactive T cell responses are well maintained over time-especially in those with combined vaccine and infection-induced immunity ("hybrid" immunity)-and may contribute to continued protection against severe disease. FUNDING: Department for Health and Social Care, Medical Research Council.

Longitudinal Lung Function Assessment of Patients Hospitalized With COVID-19 Using 1H and 129Xe Lung MRI.

BACKGROUND: Microvascular abnormalities and impaired gas transfer have been observed in patients with COVID-19. The progression of pulmonary changes in these patients remains unclear. RESEARCH QUESTION: Do patients hospitalized with COVID-19 without evidence of architectural distortion on structural imaging exhibit longitudinal improvements in lung function measured by using 1H and 129Xe MRI between 6 and 52 weeks following hospitalization? STUDY DESIGN AND METHODS: Patients who were hospitalized with COVID-19 pneumonia underwent a pulmonary 1H and 129Xe MRI protocol at 6, 12, 25, and 51 weeks following hospital admission in a prospective cohort study between November 2020 and February 2022. The imaging protocol was as follows: 1H ultra-short echo time, contrast-enhanced lung perfusion, 129Xe ventilation, 129Xe diffusion-weighted, and 129Xe spectroscopic imaging of gas exchange. RESULTS: Nine patients were recruited (age 57 ± 14 [median ± interquartile range] years; six of nine patients were male). Patients underwent MRI at 6 (n = 9), 12 (n = 9), 25 (n = 6), and 51 (n = 8) weeks following hospital admission. Patients with signs of interstitial lung damage were excluded. At 6 weeks, patients exhibited impaired 129Xe gas transfer (RBC to membrane fraction), but lung microstructure was not increased (apparent diffusion coefficient and mean acinar airway dimensions). Minor ventilation abnormalities present in four patients were largely resolved in the 6- to 25-week period. At 12 weeks, all patients with lung perfusion data (n = 6) showed an increase in both pulmonary blood volume and flow compared with 6 weeks, although this was not statistically significant. At 12 weeks, significant improvements in 129Xe gas transfer were observed compared with 6-week examinations; however, 129Xe gas transfer remained abnormally low at weeks 12, 25, and 51. INTERPRETATION: 129Xe gas transfer was impaired up to 1 year following hospitalization in patients who were hospitalized with COVID-19 pneumonia, without evidence of architectural distortion on structural imaging, whereas lung ventilation was normal at 52 weeks.

NICU Versus Mother/Baby Unit Admission for Low-Acuity Infants Born at 35 Weeks' Gestation.

OBJECTIVES: To estimate the effect of NICU admission of low-acuity infants born at 35 weeks' gestation versus care in a mother/baby unit, on inpatient and outpatient medical outcomes. METHODS: This retrospective cohort study included 5929 low-acuity infants born at 350/7 to 356/7 weeks' gestation at 13 Kaiser Permanente Northern California hospitals with level II or level III NICUs between January 1, 2011, and December 31, 2021. Exclusion criteria included congenital anomalies and early respiratory support or antibiotics. We used multivariable regression and regression discontinuity analyses to control for confounding variables. RESULTS: Infants admitted to the NICU within 2 hours of birth (n = 862, 14.5%) had a 58 hour adjusted (98-hour unadjusted) longer length of stay. NICU admission was associated with an increased probability of a length of stay ≥96 hours (67% vs 21%; adjusted odds ratio [aOR], 4.94; 95% confidence interval [CI], 3.96-6.16). Regression discontinuity results suggested a similar (57 hour) increase in length of stay. Readmission risk, primarily for jaundice, was lower for those admitted to the NICU (3% vs 6%; aOR, 0.43; 95% CI, 0.27-0.69). Infants admitted to the NICU were slightly less likely to be receiving exclusive breast milk at 6-month follow-up (15% vs 25%; aOR, 0.73; 95% CI, 0.55-0.97; adjusted marginal risk difference -5%). CONCLUSIONS: Admitting low-acuity infants born at 35 weeks' gestation to the NICU was associated with decreased readmission, but with longer length of stay and decreased exclusive breast milk feeding at 6 months. Routine NICU admission may be unnecessary for low-acuity infants born at 35 weeks' gestation.

Rapid escape of new SARS-CoV-2 Omicron variants from BA.2-directed antibody responses.

In November 2021, Omicron BA.1, containing a raft of new spike mutations, emerged and quickly spread globally. Intense selection pressure to escape the antibody response produced by vaccines or severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection then led to a rapid succession of Omicron sub-lineages with waves of BA.2 and then BA.4/5 infection. Recently, many variants have emerged such as BQ.1 and XBB, which carry up to 8 additional receptor-binding domain (RBD) amino acid substitutions compared with BA.2. We describe a panel of 25 potent monoclonal antibodies (mAbs) generated from vaccinees suffering BA.2 breakthrough infections. Epitope mapping shows potent mAb binding shifting to 3 clusters, 2 corresponding to early-pandemic binding hotspots. The RBD mutations in recent variants map close to these binding sites and knock out or severely knock down neutralization activity of all but 1 potent mAb. This recent mAb escape corresponds with large falls in neutralization titer of vaccine or BA.1, BA.2, or BA.4/5 immune serum.

Evidence Suggests a Decrease in the Incidence of Kernicterus in California.

We identified children diagnosed with kernicterus in the California Department of Developmental Services and estimated an incidence of 0.42 per 100 000 births from 1988 to 2014, significantly decreasing to 0.04 per 100 000 births after 2009. We also examined national infant kernicterus mortality from 1979 to 2016 using CDC data. It did not decrease significantly.

Discriminative Ability of Lower limb Strength and Power Measures in Lacrosse Athletes.

The objective of this investigation was to compare isokinetic strength, countermovement jump and drop jump variables between high-contributors and low-contributors within NCAA Division I Men's and Women's lacrosse athletes. Men's (N=36) and Women's (N=30) NCAA Division I lacrosse athletes completed strength testing of the quadriceps and hamstring across three speeds (60°·s-1, 180°·s-1, 300°·s-1), countermovement and drop jumps. To determine the discriminative ability of select lower-limb strength and power characteristics participants were categorized as high-contributors (Males N=18, age=20.3±0.4 yrs, height=183.9±5.5 cm, mass=90.8±5.8 kg; Females N=15, age=20.8±0.8 yrs, height=169.3±6.7 cm, mass=64.1±7.2 kg) or low-contributors (Males N=18, age=19.5±0.2 yrs, height=184.1±5.6 cm; mass=87.9±8.1 kg; Females N=15, age=19.7±0.2 yrs, height=169.8±7.0 cm, mass=62.9±7.7 kg ) based upon the number of games the participants competed in during the regular season. Within the male cohort, moderate significant (p<0.05) differences were observed between high-contributors and low-contributors in isokinetic hamstring strength of the left leg at 300°·s-1 (d=0.69) and peak power in countermovement jump (d=0.68). Within the women's cohort a large (d=0.87) significant difference (p<0.05) in isokinetic strength of the left hamstring was observed between high-contributors and low-contributors at 60°·s-1. Hamstring strength and lower-limb power are important strength measures for lacrosse performance and should be prioritized in training prescription for lacrosse athletes.