Fatigue and vigilance in medical experts detecting breast cancer.

An abundance of laboratory-based experiments has described a vigilance decrement of reducing accuracy to detect targets with time on task, but there are few real-world studies, none of which have previously controlled the environment to control for bias. We describe accuracy in clinical practice for 360 experts who examined >1 million women's mammograms for signs of cancer, whilst controlling for potential biases. The vigilance decrement pattern was not observed. Instead, test accuracy improved over time, through a reduction in false alarms and an increase in speed, with no significant change in sensitivity. The multiple-decision model explains why experts miss targets in low prevalence settings through a change in decision threshold and search quit threshold and propose it should be adapted to explain these observed patterns of accuracy with time on task. What is typically thought of as standard and robust research findings in controlled laboratory settings may not directly apply to real-world environments and instead large, controlled studies in relevant environments are needed.

Optimising the diagnostic accuracy of First post-contrAst SubtracTed breast MRI (FAST MRI) through interpretation-training: a multicentre e-learning study, mapping the learning curve of NHS Breast Screening Programme (NHSBSP) mammogram readers using an enriched dataset.

BACKGROUND: Abbreviated breast MRI (FAST MRI) is being introduced into clinical practice to screen women with mammographically dense breasts or with a personal history of breast cancer. This study aimed to optimise diagnostic accuracy through the adaptation of interpretation-training. METHODS: A FAST MRI interpretation-training programme (short presentations and guided hands-on workstation teaching) was adapted to provide additional training during the assessment task (interpretation of an enriched dataset of 125 FAST MRI scans) by giving readers feedback about the true outcome of each scan immediately after each scan was interpreted (formative assessment). Reader interaction with the FAST MRI scans used developed software (RiViewer) that recorded reader opinions and reading times for each scan. The training programme was additionally adapted for remote e-learning delivery. STUDY DESIGN: Prospective, blinded interpretation of an enriched dataset by multiple readers. RESULTS: 43 mammogram readers completed the training, 22 who interpreted breast MRI in their clinical role (Group 1) and 21 who did not (Group 2). Overall sensitivity was 83% (95%CI 81-84%; 1994/2408), specificity 94% (95%CI 93-94%; 7806/8338), readers' agreement with the true outcome kappa = 0.75 (95%CI 0.74-0.77) and diagnostic odds ratio = 70.67 (95%CI 61.59-81.09). Group 1 readers showed similar sensitivity (84%) to Group 2 (82% p = 0.14), but slightly higher specificity (94% v. 93%, p = 0.001). Concordance with the ground truth increased significantly with the number of FAST MRI scans read through the formative assessment task (p = 0.002) but by differing amounts depending on whether or not a reader had previously attended FAST MRI training (interaction p = 0.02). Concordance with the ground truth was significantly associated with reading batch size (p = 0.02), tending to worsen when more than 50 scans were read per batch. Group 1 took a median of 56 seconds (range 8-47,466) to interpret each FAST MRI scan compared with 78 (14-22,830, p < 0.0001) for Group 2. CONCLUSIONS: Provision of immediate feedback to mammogram readers during the assessment test set reading task increased specificity for FAST MRI interpretation and achieved high diagnostic accuracy. Optimal reading-batch size for FAST MRI was 50 reads per batch. Trial registration (25/09/2019): ISRCTN16624917.

Software using artificial intelligence for nodule and cancer detection in CT lung cancer screening: systematic review of test accuracy studies.

OBJECTIVES: To examine the accuracy and impact of artificial intelligence (AI) software assistance in lung cancer screening using CT. METHODS: A systematic review of CE-marked, AI-based software for automated detection and analysis of nodules in CT lung cancer screening was conducted. Multiple databases including Medline, Embase and Cochrane CENTRAL were searched from 2012 to March 2023. Primary research reporting test accuracy or impact on reading time or clinical management was included. QUADAS-2 and QUADAS-C were used to assess risk of bias. We undertook narrative synthesis. RESULTS: Eleven studies evaluating six different AI-based software and reporting on 19 770 patients were eligible. All were at high risk of bias with multiple applicability concerns. Compared with unaided reading, AI-assisted reading was faster and generally improved sensitivity (+5% to +20% for detecting/categorising actionable nodules; +3% to +15% for detecting/categorising malignant nodules), with lower specificity (-7% to -3% for correctly detecting/categorising people without actionable nodules; -8% to -6% for correctly detecting/categorising people without malignant nodules). AI assistance tended to increase the proportion of nodules allocated to higher risk categories. Assuming 0.5% cancer prevalence, these results would translate into additional 150-750 cancers detected per million people attending screening but lead to an additional 59 700 to 79 600 people attending screening without cancer receiving unnecessary CT surveillance. CONCLUSIONS: AI assistance in lung cancer screening may improve sensitivity but increases the number of false-positive results and unnecessary surveillance. Future research needs to increase the specificity of AI-assisted reading and minimise risk of bias and applicability concerns through improved study design. PROSPERO REGISTRATION NUMBER: CRD42021298449.

Optimizing the Pairs of Radiologists That Double Read Screening Mammograms.

Background Despite variation in performance characteristics among radiologists, the pairing of radiologists for the double reading of screening mammograms is performed randomly. It is unknown how to optimize pairing to improve screening performance. Purpose To investigate whether radiologist performance characteristics can be used to determine the optimal set of pairs of radiologists to double read screening mammograms for improved accuracy. Materials and Methods This retrospective study was performed with reading outcomes from breast cancer screening programs in Sweden (2008-2015), England (2012-2014), and Norway (2004-2018). Cancer detection rates (CDRs) and abnormal interpretation rates (AIRs) were calculated, with AIR defined as either reader flagging an examination as abnormal. Individual readers were divided into performance categories based on their high and low CDR and AIR. The performance of individuals determined the classification of pairs. Random pair performance, for which any type of pair was equally represented, was compared with the performance of specific pairing strategies, which consisted of pairs of readers who were either opposite or similar in AIR and/or CDR. Results Based on a minimum number of examinations per reader and per pair, the final study sample consisted of 3 592 414 examinations (Sweden, n = 965 263; England, n = 837 048; Norway, n = 1 790 103). The overall AIRs and CDRs for all specific pairing strategies (Sweden AIR range, 45.5-56.9 per 1000 examinations and CDR range, 3.1-3.6 per 1000; England AIR range, 68.2-70.5 per 1000 and CDR range, 8.9-9.4 per 1000; Norway AIR range, 81.6-88.1 per 1000 and CDR range, 6.1-6.8 per 1000) were not significantly different from the random pairing strategy (Sweden AIR, 54.1 per 1000 examinations and CDR, 3.3 per 1000; England AIR, 69.3 per 1000 and CDR, 9.1 per 1000; Norway AIR, 84.1 per 1000 and CDR, 6.3 per 1000). Conclusion Pairing a set of readers based on different pairing strategies did not show a significant difference in screening performance when compared with random pairing. © RSNA, 2023.

Evaluating the effectiveness of abbreviated breast MRI (abMRI) interpretation training for mammogram readers: a multi-centre study assessing diagnostic performance, using an enriched dataset.

BACKGROUND: Abbreviated breast MRI (abMRI) is being introduced in breast screening trials and clinical practice, particularly for women with dense breasts. Upscaling abMRI provision requires the workforce of mammogram readers to learn to effectively interpret abMRI. The purpose of this study was to examine the diagnostic accuracy of mammogram readers to interpret abMRI after a single day of standardised small-group training and to compare diagnostic performance of mammogram readers experienced in full-protocol breast MRI (fpMRI) interpretation (Group 1) with that of those without fpMRI interpretation experience (Group 2). METHODS: Mammogram readers were recruited from six NHS Breast Screening Programme sites. Small-group hands-on workstation training was provided, with subsequent prospective, independent, blinded interpretation of an enriched dataset with known outcome. A simplified form of abMRI (first post-contrast subtracted images (FAST MRI), displayed as maximum-intensity projection (MIP) and subtracted slice stack) was used. Per-breast and per-lesion diagnostic accuracy analysis was undertaken, with comparison across groups, and double-reading simulation of a consecutive screening subset. RESULTS: 37 readers (Group 1: 17, Group 2: 20) completed the reading task of 125 scans (250 breasts) (total = 9250 reads). Overall sensitivity was 86% (95% confidence interval (CI) 84-87%; 1776/2072) and specificity 86% (95%CI 85-86%; 6140/7178). Group 1 showed significantly higher sensitivity (843/952; 89%; 95%CI 86-91%) and higher specificity (2957/3298; 90%; 95%CI 89-91%) than Group 2 (sensitivity = 83%; 95%CI 81-85% (933/1120) p < 0.0001; specificity = 82%; 95%CI 81-83% (3183/3880) p < 0.0001). Inter-reader agreement was higher for Group 1 (kappa = 0.73; 95%CI 0.68-0.79) than for Group 2 (kappa = 0.51; 95%CI 0.45-0.56). Specificity improved for Group 2, from the first 55 cases (81%) to the remaining 70 (83%) (p = 0.02) but not for Group 1 (90-89% p = 0.44), whereas sensitivity remained consistent for both Group 1 (88-89%) and Group 2 (83-84%). CONCLUSIONS: Single-day abMRI interpretation training for mammogram readers achieved an overall diagnostic performance within benchmarks published for fpMRI but was insufficient for diagnostic accuracy of mammogram readers new to breast MRI to match that of experienced fpMRI readers. Novice MRI reader performance improved during the reading task, suggesting that additional training could further narrow this performance gap.

Optimising breast cancer screening reading: blinding the second reader to the first reader's decisions.

OBJECTIVES: In breast cancer screening, two readers separately examine each woman's mammograms for signs of cancer. We examined whether preventing the two readers from seeing each other's decisions (blinding) affects behaviour and outcomes. METHODS: This cohort study used data from the CO-OPS breast-screening trial (1,119,191 women from 43 screening centres in England) where all discrepant readings were arbitrated. Multilevel models were fitted using Markov chain Monte Carlo to measure whether reader 2 conformed to the decisions of reader 1 when they were not blinded, and the effect of blinding on overall rates of recall for further tests and cancer detection. Differences in positive predictive value (PPV) were assessed using Pearson's chi-squared test. RESULTS: When reader 1 recalls, the probability of reader 2 also recalling was higher when not blinded than when blinded, suggesting readers may be influenced by the other's decision. Overall, women were less likely to be recalled when reader 2 was blinded (OR 0.923; 95% credible interval 0.864, 0.986), with no clear pattern in cancer detection rate (OR 1.029; 95% credible interval 0.970, 1.089; Bayesian p value 0.832). PPV was 22.1% for blinded versus 20.6% for not blinded (p < 0.001). CONCLUSIONS: Our results suggest that when not blinded, reader 2 is influenced by reader 1's decisions to recall (alliterative bias) which would result in bypassing arbitration and negate some of the benefits of double-reading. We found a relationship between blinding the second reader and slightly higher PPV of breast cancer screening, although this analysis may be confounded by other centre characteristics. KEY POINTS: • In Europe, it is recommended that breast screening mammograms are analysed by two readers but there is little evidence on the effect of 'blinding' the readers so they cannot see each other's decisions. • We found evidence that when the second reader is not blinded, they are more likely to agree with a recall decision from the first reader and less likely to make an independent judgement (alliterative error). This may reduce overall accuracy through bypassing arbitration. • This observational study suggests an association between blinding the second reader and higher positive predictive value of screening, but this may be confounded by centre characteristics.

Comparing outcomes from tailored meta-analysis with outcomes from a setting specific test accuracy study using routine data of faecal calprotectin testing for inflammatory bowel disease.

BACKGROUND: Meta-analyses of test accuracy studies may provide estimates that are highly improbable in clinical practice. Tailored meta-analysis produces plausible estimates for the accuracy of a test within a specific setting by tailoring the selection of included studies compatible with a specific setting using information from the target setting. The aim of this study was to validate the tailored meta-analysis approach by comparing outcomes from tailored meta-analysis with outcomes from a setting specific test accuracy study. METHODS: A retrospective cohort study of primary care electronic health records provided setting-specific data on the test positive rate and disease prevalence. This was used to tailor the study selection from a review of faecal calprotectin testing for inflammatory bowel disease for meta-analysis using the binomial method and the Mahalanobis distance method. Tailored estimates were compared to estimates from a study of test accuracy in primary care using the same routine dataset. RESULTS: Tailoring resulted in the inclusion of 3/14 (binomial method) and 9/14 (Mahalanobis distance method) studies in meta-analysis. Sensitivity and specificity from tailored meta-analysis using the binomial method were 0.87 (95% CI 0.77 to 0.94) and 0.65 (95% CI 0.60 to 0.69) and 0.98 (95% CI 0.83 to 0.999) and 0.68 (95% CI 0.65 to 0.71), respectively using the Mahalanobis distance method. The corresponding estimates for the conventional meta-analysis were 0.94 (95% CI 0.90 to 0.97) and 0.67 (95% CI 0.57 to 0.76) and for the FC test accuracy study of primary care data 0.93 (95%CI 0.89 to 0.96) and 0.61 (95% CI 0.6 to 0.63) to detect IBD at a threshold of 50 µg/g. Although the binomial method produced a plausible estimate, the tailored estimates of sensitivity and specificity were not closer to the primary study estimates than the estimates from conventional meta-analysis including all 14 studies. CONCLUSIONS: Tailored meta-analysis does not always produce estimates of sensitivity and specificity that lie closer to the estimates derived from a primary study in the setting in question. Potentially, tailored meta-analysis may be improved using a constrained model approach and this requires further investigation.

Rapid, point-of-care antigen tests for diagnosis of SARS-CoV-2 infection.

BACKGROUND: Accurate rapid diagnostic tests for SARS-CoV-2 infection would be a useful tool to help manage the COVID-19 pandemic. Testing strategies that use rapid antigen tests to detect current infection have the potential to increase access to testing, speed detection of infection, and inform clinical and public health management decisions to reduce transmission. This is the second update of this review, which was first published in 2020. OBJECTIVES: To assess the diagnostic accuracy of rapid, point-of-care antigen tests for diagnosis of SARS-CoV-2 infection. We consider accuracy separately in symptomatic and asymptomatic population groups. Sources of heterogeneity investigated included setting and indication for testing, assay format, sample site, viral load, age, timing of test, and study design. SEARCH METHODS: We searched the COVID-19 Open Access Project living evidence database from the University of Bern (which includes daily updates from PubMed and Embase and preprints from medRxiv and bioRxiv) on 08 March 2021. We included independent evaluations from national reference laboratories, FIND and the Diagnostics Global Health website. We did not apply language restrictions. SELECTION CRITERIA: We included studies of people with either suspected SARS-CoV-2 infection, known SARS-CoV-2 infection or known absence of infection, or those who were being screened for infection. We included test accuracy studies of any design that evaluated commercially produced, rapid antigen tests. We included evaluations of single applications of a test (one test result reported per person) and evaluations of serial testing (repeated antigen testing over time). Reference standards for presence or absence of infection were any laboratory-based molecular test (primarily reverse transcription polymerase chain reaction (RT-PCR)) or pre-pandemic respiratory sample. DATA COLLECTION AND ANALYSIS: We used standard screening procedures with three people. Two people independently carried out quality assessment (using the QUADAS-2 tool) and extracted study results. Other study characteristics were extracted by one review author and checked by a second. We present sensitivity and specificity with 95% confidence intervals (CIs) for each test, and pooled data using the bivariate model. We investigated heterogeneity by including indicator variables in the random-effects logistic regression models. We tabulated results by test manufacturer and compliance with manufacturer instructions for use and according to symptom status. MAIN RESULTS: We included 155 study cohorts (described in 166 study reports, with 24 as preprints). The main results relate to 152 evaluations of single test applications including 100,462 unique samples (16,822 with confirmed SARS-CoV-2). Studies were mainly conducted in Europe (101/152, 66%), and evaluated 49 different commercial antigen assays. Only 23 studies compared two or more brands of test. Risk of bias was high because of participant selection (40, 26%); interpretation of the index test (6, 4%); weaknesses in the reference standard for absence of infection (119, 78%); and participant flow and timing 41 (27%). Characteristics of participants (45, 30%) and index test delivery (47, 31%) differed from the way in which and in whom the test was intended to be used. Nearly all studies (91%) used a single RT-PCR result to define presence or absence of infection. The 152 studies of single test applications reported 228 evaluations of antigen tests. Estimates of sensitivity varied considerably between studies, with consistently high specificities. Average sensitivity was higher in symptomatic (73.0%, 95% CI 69.3% to 76.4%; 109 evaluations; 50,574 samples, 11,662 cases) compared to asymptomatic participants (54.7%, 95% CI 47.7% to 61.6%; 50 evaluations; 40,956 samples, 2641 cases). Average sensitivity was higher in the first week after symptom onset (80.9%, 95% CI 76.9% to 84.4%; 30 evaluations, 2408 cases) than in the second week of symptoms (53.8%, 95% CI 48.0% to 59.6%; 40 evaluations, 1119 cases). For those who were asymptomatic at the time of testing, sensitivity was higher when an epidemiological exposure to SARS-CoV-2 was suspected (64.3%, 95% CI 54.6% to 73.0%; 16 evaluations; 7677 samples, 703 cases) compared to where COVID-19 testing was reported to be widely available to anyone on presentation for testing (49.6%, 95% CI 42.1% to 57.1%; 26 evaluations; 31,904 samples, 1758 cases). Average specificity was similarly high for symptomatic (99.1%) or asymptomatic (99.7%) participants. We observed a steady decline in summary sensitivities as measures of sample viral load decreased. Sensitivity varied between brands. When tests were used according to manufacturer instructions, average sensitivities by brand ranged from 34.3% to 91.3% in symptomatic participants (20 assays with eligible data) and from 28.6% to 77.8% for asymptomatic participants (12 assays). For symptomatic participants, summary sensitivities for seven assays were 80% or more (meeting acceptable criteria set by the World Health Organization (WHO)). The WHO acceptable performance criterion of 97% specificity was met by 17 of 20 assays when tests were used according to manufacturer instructions, 12 of which demonstrated specificities above 99%. For asymptomatic participants the sensitivities of only two assays approached but did not meet WHO acceptable performance standards in one study each; specificities for asymptomatic participants were in a similar range to those observed for symptomatic people. At 5% prevalence using summary data in symptomatic people during the first week after symptom onset, the positive predictive value (PPV) of 89% means that 1 in 10 positive results will be a false positive, and around 1 in 5 cases will be missed. At 0.5% prevalence using summary data for asymptomatic people, where testing was widely available and where epidemiological exposure to COVID-19 was suspected, resulting PPVs would be 38% to 52%, meaning that between 2 in 5 and 1 in 2 positive results will be false positives, and between 1 in 2 and 1 in 3 cases will be missed. AUTHORS' CONCLUSIONS: Antigen tests vary in sensitivity. In people with signs and symptoms of COVID-19, sensitivities are highest in the first week of illness when viral loads are higher. Assays that meet appropriate performance standards, such as those set by WHO, could replace laboratory-based RT-PCR when immediate decisions about patient care must be made, or where RT-PCR cannot be delivered in a timely manner. However, they are more suitable for use as triage to RT-PCR testing. The variable sensitivity of antigen tests means that people who test negative may still be infected. Many commercially available rapid antigen tests have not been evaluated in independent validation studies. Evidence for testing in asymptomatic cohorts has increased, however sensitivity is lower and there is a paucity of evidence for testing in different settings. Questions remain about the use of antigen test-based repeat testing strategies. Further research is needed to evaluate the effectiveness of screening programmes at reducing transmission of infection, whether mass screening or targeted approaches including schools, healthcare setting and traveller screening.

Antibody tests for identification of current and past infection with SARS-CoV-2.

BACKGROUND: The diagnostic challenges associated with the COVID-19 pandemic resulted in rapid development of diagnostic test methods for detecting SARS-CoV-2 infection. Serology tests to detect the presence of antibodies to SARS-CoV-2 enable detection of past infection and may detect cases of SARS-CoV-2 infection that were missed by earlier diagnostic tests. Understanding the diagnostic accuracy of serology tests for SARS-CoV-2 infection may enable development of effective diagnostic and management pathways, inform public health management decisions and understanding of SARS-CoV-2 epidemiology. OBJECTIVES: To assess the accuracy of antibody tests, firstly, to determine if a person presenting in the community, or in primary or secondary care has current SARS-CoV-2 infection according to time after onset of infection and, secondly, to determine if a person has previously been infected with SARS-CoV-2. Sources of heterogeneity investigated included: timing of test, test method, SARS-CoV-2 antigen used, test brand, and reference standard for non-SARS-CoV-2 cases. SEARCH METHODS: The COVID-19 Open Access Project living evidence database from the University of Bern (which includes daily updates from PubMed and Embase and preprints from medRxiv and bioRxiv) was searched on 30 September 2020. We included additional publications from the Evidence for Policy and Practice Information and Co-ordinating Centre (EPPI-Centre) 'COVID-19: Living map of the evidence' and the Norwegian Institute of Public Health 'NIPH systematic and living map on COVID-19 evidence'. We did not apply language restrictions. SELECTION CRITERIA: We included test accuracy studies of any design that evaluated commercially produced serology tests, targeting IgG, IgM, IgA alone, or in combination. Studies must have provided data for sensitivity, that could be allocated to a predefined time period after onset of symptoms, or after a positive RT-PCR test. Small studies with fewer than 25 SARS-CoV-2 infection cases were excluded. We included any reference standard to define the presence or absence of SARS-CoV-2 (including reverse transcription polymerase chain reaction tests (RT-PCR), clinical diagnostic criteria, and pre-pandemic samples). DATA COLLECTION AND ANALYSIS: We use standard screening procedures with three reviewers. Quality assessment (using the QUADAS-2 tool) and numeric study results were extracted independently by two people. Other study characteristics were extracted by one reviewer and checked by a second. We present sensitivity and specificity with 95% confidence intervals (CIs) for each test and, for meta-analysis, we fitted univariate random-effects logistic regression models for sensitivity by eligible time period and for specificity by reference standard group. Heterogeneity was investigated by including indicator variables in the random-effects logistic regression models. We tabulated results by test manufacturer and summarised results for tests that were evaluated in 200 or more samples and that met a modification of UK Medicines and Healthcare products Regulatory Agency (MHRA) target performance criteria. MAIN RESULTS: We included 178 separate studies (described in 177 study reports, with 45 as pre-prints) providing 527 test evaluations. The studies included 64,688 samples including 25,724 from people with confirmed SARS-CoV-2; most compared the accuracy of two or more assays (102/178, 57%). Participants with confirmed SARS-CoV-2 infection were most commonly hospital inpatients (78/178, 44%), and pre-pandemic samples were used by 45% (81/178) to estimate specificity. Over two-thirds of studies recruited participants based on known SARS-CoV-2 infection status (123/178, 69%). All studies were conducted prior to the introduction of SARS-CoV-2 vaccines and present data for naturally acquired antibody responses. Seventy-nine percent (141/178) of studies reported sensitivity by week after symptom onset and 66% (117/178) for convalescent phase infection. Studies evaluated enzyme-linked immunosorbent assays (ELISA) (165/527; 31%), chemiluminescent assays (CLIA) (167/527; 32%) or lateral flow assays (LFA) (188/527; 36%). Risk of bias was high because of participant selection (172, 97%); application and interpretation of the index test (35, 20%); weaknesses in the reference standard (38, 21%); and issues related to participant flow and timing (148, 82%). We judged that there were high concerns about the applicability of the evidence related to participants in 170 (96%) studies, and about the applicability of the reference standard in 162 (91%) studies. Average sensitivities for current SARS-CoV-2 infection increased by week after onset for all target antibodies. Average sensitivity for the combination of either IgG or IgM was 41.1% in week one (95% CI 38.1 to 44.2; 103 evaluations; 3881 samples, 1593 cases), 74.9% in week two (95% CI 72.4 to 77.3; 96 evaluations, 3948 samples, 2904 cases) and 88.0% by week three after onset of symptoms (95% CI 86.3 to 89.5; 103 evaluations, 2929 samples, 2571 cases). Average sensitivity during the convalescent phase of infection (up to a maximum of 100 days since onset of symptoms, where reported) was 89.8% for IgG (95% CI 88.5 to 90.9; 253 evaluations, 16,846 samples, 14,183 cases), 92.9% for IgG or IgM combined (95% CI 91.0 to 94.4; 108 evaluations, 3571 samples, 3206 cases) and 94.3% for total antibodies (95% CI 92.8 to 95.5; 58 evaluations, 7063 samples, 6652 cases). Average sensitivities for IgM alone followed a similar pattern but were of a lower test accuracy in every time slot. Average specificities were consistently high and precise, particularly for pre-pandemic samples which provide the least biased estimates of specificity (ranging from 98.6% for IgM to 99.8% for total antibodies). Subgroup analyses suggested small differences in sensitivity and specificity by test technology however heterogeneity in study results, timing of sample collection, and smaller sample numbers in some groups made comparisons difficult. For IgG, CLIAs were the most sensitive (convalescent-phase infection) and specific (pre-pandemic samples) compared to both ELISAs and LFAs (P < 0.001 for differences across test methods). The antigen(s) used (whether from the Spike-protein or nucleocapsid) appeared to have some effect on average sensitivity in the first weeks after onset but there was no clear evidence of an effect during convalescent-phase infection. Investigations of test performance by brand showed considerable variation in sensitivity between tests, and in results between studies evaluating the same test. For tests that were evaluated in 200 or more samples, the lower bound of the 95% CI for sensitivity was 90% or more for only a small number of tests (IgG, n = 5; IgG or IgM, n = 1; total antibodies, n = 4). More test brands met the MHRA minimum criteria for specificity of 98% or above (IgG, n = 16; IgG or IgM, n = 5; total antibodies, n = 7). Seven assays met the specified criteria for both sensitivity and specificity. In a low-prevalence (2%) setting, where antibody testing is used to diagnose COVID-19 in people with symptoms but who have had a negative PCR test, we would anticipate that 1 (1 to 2) case would be missed and 8 (5 to 15) would be falsely positive in 1000 people undergoing IgG or IgM testing in week three after onset of SARS-CoV-2 infection. In a seroprevalence survey, where prevalence of prior infection is 50%, we would anticipate that 51 (46 to 58) cases would be missed and 6 (5 to 7) would be falsely positive in 1000 people having IgG tests during the convalescent phase (21 to 100 days post-symptom onset or post-positive PCR) of SARS-CoV-2 infection. AUTHORS' CONCLUSIONS: Some antibody tests could be a useful diagnostic tool for those in whom molecular- or antigen-based tests have failed to detect the SARS-CoV-2 virus, including in those with ongoing symptoms of acute infection (from week three onwards) or those presenting with post-acute sequelae of COVID-19. However, antibody tests have an increasing likelihood of detecting an immune response to infection as time since onset of infection progresses and have demonstrated adequate performance for detection of prior infection for sero-epidemiological purposes. The applicability of results for detection of vaccination-induced antibodies is uncertain.

Investigating informed choice in screening programmes: a mixed methods analysis.

BACKGROUND: Screening programmes aim to identify individuals at higher risk of developing a disease or condition. While globally, there is agreement that people who attend screening should be fully informed, there is no consensus about how this should be achieved. We conducted a mixed methods study across eight different countries to understand how countries address informed choice across two screening programmes: breast cancer and fetal trisomy anomaly screening. METHODS: Fourteen senior level employees from organisations who produce and deliver decision aids to assist informed choice were interviewed, and their decision aids (n = 15) were evaluated using documentary analysis. RESULTS: We discovered that attempts to achieve informed choice via decision aids generate two key tensions (i) between improving informed choice and increasing uptake and (ii) between improving informed choice and comprehensibility of the information presented. Comprehensibility is fundamentally at tension with an aim of being fully informed. These tensions emerged in both the interviews and documentary analysis. CONCLUSION: We conclude that organisations need to decide whether their overarching aim is ensuring high levels of uptake or maximising informed choice to participate in screening programmes. Consideration must then be given to all levels of development and distribution of information produced to reflect each organisation's aim. The comprehensibility of the DA must also be considered, as this may be reduced when informed choice is prioritised.

Enhanced axillary assessment using intradermally injected microbubbles and contrast-enhanced ultrasound (CEUS) before neoadjuvant systemic therapy (NACT) identifies axillary disease missed by conventional B-mode ultrasound that may be clinically relevant.

PURPOSE: The purpose of this study is to measure pre-treatment diagnostic yield of malignant lymph nodes (LN) using contrast-enhanced ultrasound (CEUS) in addition to B-mode axillary ultrasound and compare clinicopathological features, response to NACT and long-term outcomes of patients with malignant LN detected with B-mode ultrasound versus CEUS. METHODS: Between August 2009 and October 2016, NACT patients were identified from a prospective database. Follow-up data were collected until May 2019. RESULTS: 288 consecutive NACT patients were identified; 77 were excluded, 110 had malignant LN identified by B-mode ultrasound (Group A) and 101 patients with negative B-mode axillary ultrasound had CEUS with biopsy of sentinel lymph nodes (SLN). In two cases CEUS failed. Malignant SLN were identified in 35/99 (35%) of B-mode ultrasound-negative cases (Group B). Patients in Group A were similar to those in Group B in age, mean diagnostic tumour size, grade and oestrogen receptor status. More Group A patients had a ductal phenotype. In the breast, 34 (31%) Group A patients and 8 (23%) Group B patients achieved a pathological complete response (PCR). In the axilla, 41 (37%) and 13 (37%) Groups A and B patients, respectively, had LN PCR. The systemic relapse rate was not statistically different (5% and 16% for Groups A and B, respectively). CONCLUSIONS: Enhanced assessment with CEUS before NACT identifies patients with axillary metastases missed by conventional B-mode ultrasound. Without CEUS, 22 (63%) of cases in Group B (negative B-mode ultrasound) may have been erroneously classed as progressive disease by surgical SLN excision after NACT.

Testing for lynch syndrome in people with endometrial cancer using immunohistochemistry and microsatellite instability-based testing strategies - A systematic review of test accuracy.

BACKGROUND: Lynch syndrome is an inherited genetic condition that is associated with an increased risk of cancer, including endometrial and colorectal cancer. We assessed the test accuracy of immunohistochemistry and microsatellite instability-based testing (with or without MLH1 promoter methylation testing) for Lynch syndrome in women with endometrial cancer. METHODS: We conducted a systematic review of literature published up to August 2019. We searched bibliographic databases, contacted experts and checked reference lists of relevant studies. Two reviewers conducted each stage of the review. RESULTS: Thirteen studies were identified that included approximately 3500 participants. None of the studies was at low risk of bias in all domains. Data could not be pooled due to the small number of heterogeneous studies. Sensitivity ranged from 60.7-100% for immunohistochemistry, 41.7-100% for microsatellite instability-based testing, and 90.5-100% for studies combining immunohistochemistry, microsatellite instability-based testing, and MLH1 promoter methylation testing. Specificity ranged from 60.9-83.3% (excluding 1 study with highly selective inclusion criteria) for immunohistochemistry, 69.2-89.9% for microsatellite instability-based testing, and 72.4-92.3% (excluding 1 study with highly selective inclusion criteria) for testing strategies that included immunohistochemistry, microsatellite instability-based testing, and MLH1 promoter methylation. We found no statistically significant differences in test accuracy estimates (sensitivity, specificity) in head-to-head studies of immunohistochemistry versus microsatellite instability-based testing. Reported test failures were rare. CONCLUSIONS: Sensitivity of the index tests were generally high, though most studies had much lower specificity. We found no evidence that test accuracy differed between IHC and MSI based strategies. The evidence base is currently small and at high risk of bias.

Optimum screening mammography reading volumes: evidence from the NHS Breast Screening Programme.

OBJECTIVES: Minimum caseload standards for professionals examining breast screening mammograms vary from 480 (US) to 5000 (Europe). We measured the relationship between the number of women's mammograms examined per year and reader performance. METHODS: We extracted routine records from the English NHS Breast Screening Programme for readers examining between 1000 and 45,000 mammograms between April 2014 and March 2017. We measured the relationship between the volume of cases read and screening performance (cancer detection rate, recall rate, positive predictive value of recall (PPV) and discrepant cancers) using linear logistic regression. We also examined the effect of reader occupational group on performance. RESULTS: In total, 759 eligible mammography readers (445 consultant radiologists, 235 radiography advanced practitioners, 79 consultant radiographers) examined 6.1 million women's mammograms during the study period. PPV increased from 12.9 to 14.4 to 17.0% for readers examining 2000, 5000 and 10000 cases per year respectively. This was driven by decreases in recall rates from 5.8 to 5.3 to 4.5 with increasing volume read, and no change in cancer detection rate (from 7.6 to 7.6 to 7.7). There was no difference in cancer detection rate with reader occupational group. Consultant radiographers had higher recall rate and lower PPV compared to radiologists (OR 1.105, p = 0.012; OR 0.874, p = 0.002, unadjusted). CONCLUSION: Positive predictive value of screening increases with the total volume of cases examined per reader, through decreases in numbers of cases recalled with no concurrent change in numbers of cancers detected. KEY POINTS: • In the English Breast Screening Programme, readers who examined a larger number of cases per year had a higher positive predictive value, because they recalled fewer women for further tests but detected the same number of cancers. • Reader type did not affect cancer detection rate, but consultant radiographers had a higher recall rate and lower positive predictive value than consultant radiologists, although this was not adjusted for length of experience.

The incidence and prevalence of inflammatory bowel disease in UK primary care: a retrospective cohort study of the IQVIA Medical Research Database.

BACKGROUND: Our knowledge of the incidence and prevalence of inflammatory bowel disease (IBD) is uncertain. Recent studies reported an increase in prevalence. However, they excluded a high proportion of ambiguous cases from general practice. Estimates are needed to inform health care providers who plan the provision of services for IBD patients. We aimed to estimate the IBD incidence and prevalence in UK general practice. METHODS: We undertook a retrospective cohort study of routine electronic health records from the IQVIA Medical Research Database covering 14 million patients. Adult patients from 2006 to 2016 were included. IBD was defined as an IBD related Read code or record of IBD specific medication. Annual incidence and 12-month period prevalence were calculated. RESULTS: The prevalence of IBD increased between 2006 and 2016 from 106.2 (95% CI 105.2-107.3) to 142.1 (95% CI 140.7-143.5) IBD cases per 10,000 patients which is a 33.8% increase. Incidence varied across the years. The incidence across the full study period was 69.5 (95% CI 68.6-70.4) per 100,000 person years. CONCLUSIONS: In this large study we found higher estimates of IBD incidence and prevalence than previously reported. Estimates are highly dependent on definitions of disease and previously may have been underestimated.

Rapid, point-of-care antigen and molecular-based tests for diagnosis of SARS-CoV-2 infection.

BACKGROUND: Accurate rapid diagnostic tests for SARS-CoV-2 infection could contribute to clinical and public health strategies to manage the COVID-19 pandemic. Point-of-care antigen and molecular tests to detect current infection could increase access to testing and early confirmation of cases, and expediate clinical and public health management decisions that may reduce transmission. OBJECTIVES: To assess the diagnostic accuracy of point-of-care antigen and molecular-based tests for diagnosis of SARS-CoV-2 infection. We consider accuracy separately in symptomatic and asymptomatic population groups. SEARCH METHODS: Electronic searches of the Cochrane COVID-19 Study Register and the COVID-19 Living Evidence Database from the University of Bern (which includes daily updates from PubMed and Embase and preprints from medRxiv and bioRxiv) were undertaken on 30 Sept 2020. We checked repositories of COVID-19 publications and included independent evaluations from national reference laboratories, the Foundation for Innovative New Diagnostics and the Diagnostics Global Health website to 16 Nov 2020. We did not apply language restrictions. SELECTION CRITERIA: We included studies of people with either suspected SARS-CoV-2 infection, known SARS-CoV-2 infection or known absence of infection, or those who were being screened for infection. We included test accuracy studies of any design that evaluated commercially produced, rapid antigen or molecular tests suitable for a point-of-care setting (minimal equipment, sample preparation, and biosafety requirements, with results within two hours of sample collection). We included all reference standards that define the presence or absence of SARS-CoV-2 (including reverse transcription polymerase chain reaction (RT-PCR) tests and established diagnostic criteria). DATA COLLECTION AND ANALYSIS: Studies were screened independently in duplicate with disagreements resolved by discussion with a third author. Study characteristics were extracted by one author and checked by a second; extraction of study results and assessments of risk of bias and applicability (made using the QUADAS-2 tool) were undertaken independently in duplicate. We present sensitivity and specificity with 95% confidence intervals (CIs) for each test and pooled data using the bivariate model separately for antigen and molecular-based tests. We tabulated results by test manufacturer and compliance with manufacturer instructions for use and according to symptom status. MAIN RESULTS: Seventy-eight study cohorts were included (described in 64 study reports, including 20 pre-prints), reporting results for 24,087 samples (7,415 with confirmed SARS-CoV-2). Studies were mainly from Europe (n = 39) or North America (n = 20), and evaluated 16 antigen and five molecular assays. We considered risk of bias to be high in 29 (50%) studies because of participant selection; in 66 (85%) because of weaknesses in the reference standard for absence of infection; and in 29 (45%) for participant flow and timing. Studies of antigen tests were of a higher methodological quality compared to studies of molecular tests, particularly regarding the risk of bias for participant selection and the index test. Characteristics of participants in 35 (45%) studies differed from those in whom the test was intended to be used and the delivery of the index test in 39 (50%) studies differed from the way in which the test was intended to be used. Nearly all studies (97%) defined the presence or absence of SARS-CoV-2 based on a single RT-PCR result, and none included participants meeting case definitions for probable COVID-19. Antigen tests Forty-eight studies reported 58 evaluations of antigen tests. Estimates of sensitivity varied considerably between studies. There were differences between symptomatic (72.0%, 95% CI 63.7% to 79.0%; 37 evaluations; 15530 samples, 4410 cases) and asymptomatic participants (58.1%, 95% CI 40.2% to 74.1%; 12 evaluations; 1581 samples, 295 cases). Average sensitivity was higher in the first week after symptom onset (78.3%, 95% CI 71.1% to 84.1%; 26 evaluations; 5769 samples, 2320 cases) than in the second week of symptoms (51.0%, 95% CI 40.8% to 61.0%; 22 evaluations; 935 samples, 692 cases). Sensitivity was high in those with cycle threshold (Ct) values on PCR ≤25 (94.5%, 95% CI 91.0% to 96.7%; 36 evaluations; 2613 cases) compared to those with Ct values >25 (40.7%, 95% CI 31.8% to 50.3%; 36 evaluations; 2632 cases). Sensitivity varied between brands. Using data from instructions for use (IFU) compliant evaluations in symptomatic participants, summary sensitivities ranged from 34.1% (95% CI 29.7% to 38.8%; Coris Bioconcept) to 88.1% (95% CI 84.2% to 91.1%; SD Biosensor STANDARD Q). Average specificities were high in symptomatic and asymptomatic participants, and for most brands (overall summary specificity 99.6%, 95% CI 99.0% to 99.8%). At 5% prevalence using data for the most sensitive assays in symptomatic people (SD Biosensor STANDARD Q and Abbott Panbio), positive predictive values (PPVs) of 84% to 90% mean that between 1 in 10 and 1 in 6 positive results will be a false positive, and between 1 in 4 and 1 in 8 cases will be missed. At 0.5% prevalence applying the same tests in asymptomatic people would result in PPVs of 11% to 28% meaning that between 7 in 10 and 9 in 10 positive results will be false positives, and between 1 in 2 and 1 in 3 cases will be missed. No studies assessed the accuracy of repeated lateral flow testing or self-testing. Rapid molecular assays Thirty studies reported 33 evaluations of five different rapid molecular tests. Sensitivities varied according to test brand. Most of the data relate to the ID NOW and Xpert Xpress assays. Using data from evaluations following the manufacturer's instructions for use, the average sensitivity of ID NOW was 73.0% (95% CI 66.8% to 78.4%) and average specificity 99.7% (95% CI 98.7% to 99.9%; 4 evaluations; 812 samples, 222 cases). For Xpert Xpress, the average sensitivity was 100% (95% CI 88.1% to 100%) and average specificity 97.2% (95% CI 89.4% to 99.3%; 2 evaluations; 100 samples, 29 cases). Insufficient data were available to investigate the effect of symptom status or time after symptom onset. AUTHORS' CONCLUSIONS: Antigen tests vary in sensitivity. In people with signs and symptoms of COVID-19, sensitivities are highest in the first week of illness when viral loads are higher. The assays shown to meet appropriate criteria, such as WHO's priority target product profiles for COVID-19 diagnostics ('acceptable' sensitivity ≥ 80% and specificity ≥ 97%), can be considered as a replacement for laboratory-based RT-PCR when immediate decisions about patient care must be made, or where RT-PCR cannot be delivered in a timely manner. Positive predictive values suggest that confirmatory testing of those with positive results may be considered in low prevalence settings. Due to the variable sensitivity of antigen tests, people who test negative may still be infected. Evidence for testing in asymptomatic cohorts was limited. Test accuracy studies cannot adequately assess the ability of antigen tests to differentiate those who are infectious and require isolation from those who pose no risk, as there is no reference standard for infectiousness. A small number of molecular tests showed high accuracy and may be suitable alternatives to RT-PCR. However, further evaluations of the tests in settings as they are intended to be used are required to fully establish performance in practice. Several important studies in asymptomatic individuals have been reported since the close of our search and will be incorporated at the next update of this review. Comparative studies of antigen tests in their intended use settings and according to test operator (including self-testing) are required.

Screening for Prediabetes and Type 2 Diabetes: Updated Evidence Report and Systematic Review for the US Preventive Services Task Force.

Importance: Type 2 diabetes is common and is a leading cause of morbidity and disability. Objective: To review the evidence on screening for prediabetes and diabetes to inform the US Preventive Services Task Force (USPSTF). Data Sources: PubMed/MEDLINE, Cochrane Library, and trial registries through September 2019; references; and experts; literature surveillance through May 21, 2021. Study Selection: English-language controlled studies evaluating screening or interventions for prediabetes or diabetes that was screen detected or recently diagnosed. Data Extraction and Synthesis: Dual review of abstracts, full-text articles, and study quality; qualitative synthesis of findings; meta-analyses conducted when at least 3 similar studies were available. Main Outcomes and Measures: Mortality, cardiovascular morbidity, diabetes-related morbidity, development of diabetes, quality of life, and harms. Results: The review included 89 publications (N = 68 882). Two randomized clinical trials (RCTs) (25 120 participants) found no significant difference between screening and control groups for all-cause or cause-specific mortality at 10 years. For harms (eg, anxiety or worry), the trials reported no significant differences between screening and control groups. For recently diagnosed (not screen-detected) diabetes, 5 RCTs (5138 participants) were included. In the UK Prospective Diabetes Study, health outcomes were improved with intensive glucose control with sulfonylureas or insulin. For example, for all-cause mortality the relative risk (RR) was 0.87 (95% CI, 0.79 to 0.96) over 20 years (10-year posttrial assessment). For overweight persons, intensive glucose control with metformin improved health outcomes at the 10-year follow-up (eg, all-cause mortality: RR, 0.64 [95% CI, 0.45 to 0.91]), and benefits were maintained longer term. Lifestyle interventions (most involving >360 minutes) for obese or overweight persons with prediabetes were associated with reductions in the incidence of diabetes (23 RCTs; pooled RR, 0.78 [95% CI, 0.69 to 0.88]). Lifestyle interventions were also associated with improved intermediate outcomes, such as reduced weight, body mass index, systolic blood pressure, and diastolic blood pressure (pooled weighted mean difference, -1.7 mm Hg [95% CI, -2.6 to -0.8] and -1.2 mm Hg [95% CI, -2.0 to -0.4], respectively). Metformin was associated with a significant reduction in diabetes incidence (pooled RR, 0.73 [95% CI, 0.64 to 0.83]) and reduction in weight and body mass index. Conclusions and Relevance: Trials of screening for diabetes found no significant mortality benefit but had insufficient data to assess other health outcomes; evidence on harms of screening was limited. For persons with recently diagnosed (not screen-detected) diabetes, interventions improved health outcomes; for obese or overweight persons with prediabetes, interventions were associated with reduced incidence of diabetes and improvement in other intermediate outcomes.

The use of electronic healthcare records for colorectal cancer screening referral decisions and risk prediction model development.

BACKGROUND: The database used for the NHS Bowel Cancer Screening Programme (BCSP) derives participant information from primary care records. Combining predictors with FOBTs has shown to improve referral decisions and accuracy. The richer data available from GP databases could be used to complement screening referral decisions by identifying those at greatest risk of colorectal cancer. We determined the availability of data for key predictors and whether this information could be used to inform more accurate screening referral decisions. METHODS: An English BCSP cohort was derived using the electronic notifications received from the BCSP database to GP records. The cohort covered a period between 13th May 2009 to 17th January 2017. Completeness of variables and univariable associations were assessed. Risk prediction models were developed using Cox regression and multivariable fractional polynomials with backwards elimination. Optimism adjusted performance metrics were reported. The sensitivity and specificity of a combined approach using the negative FOBT model plus FOBT positive patients was determined using a probability equivalent to a 3% PPV NICE guidelines level. RESULTS: 292,059 participants aged 60-74 were derived for the BCSP screening cohort. A model including the screening test result had a C-statistic of 0.860, c-slope of 0.997, and R2 of 0.597. A model developed for negative screening results only had a C-statistic of 0.597, c-slope of 0.940, and R2 of 0.062. Risk predictors included in the models included; age, sex, alcohol consumption, IBS diagnosis, family history of gastrointestinal cancer, smoking status, previous negatives and whether a GP had ordered a blood test. For the combined screening approach, sensitivity increased slightly from 53.90% (FOBT only) to 58.82% but at the expense of an increased referral rate. CONCLUSIONS: This research has identified several potential predictors for CRC in a BCSP population. A risk prediction model developed for BCSP FOBT negative patients was not clinically useful due to a low sensitivity and increased referral rate. The predictors identified in this study should be investigated in a refined algorithm combining the quantitative FIT result. Combining data from multiple sources enables fuller patient profiles using the primary care and screening database interface.

Antenatal screening for fetal trisomies using microarray-based cell-free DNA testing: A systematic review and meta-analysis.

OBJECTIVE: To evaluate the test accuracy of non-invasive prenatal testing (NIPT) for fetal trisomy 21, 18, and 13 using cell-free (cf) DNA analysis in maternal plasma with microarray quantitation. METHOD: Systematic review and meta-analysis. Searches in MEDLINE, Pre-MEDLINE, EMBASE, Web of Science, and the Cochrane Library to 09.07.2018. RESULTS: Five studies analyzing 3074 samples, including 187 trisomy 21, 43 trisomy 18, and 19 trisomy 13 cases, were identified. Risk of bias was high in all studies, introduced particularly by exclusions from analysis and by the role of the sponsor. Sensitivity of microarray-based cfDNA testing was 99.5% (95%CI 96.3%-99.9%) for trisomy 21, 97.7% (95%CI 87.9%-99.6%) for trisomy 18, and 100% (95%CI 83.2%-100%) for trisomy 13. Specificity was 100% (95% CI 99.87%-100%) for trisomy 21, 99.97% (95%CI 99.81%-99.99%) for trisomy 18, and 99.97% (95%CI 99.81%-99.99%) for trisomy 13. Pooled test failure rate was 1.1%. A direct comparison of microarray- and sequencing-based cfDNA found equivalent test accuracy. CONCLUSION: Included studies suggest that NIPT using microarray-based cfDNA testing has high sensitivity and specificity for detecting fetal trisomy 21, 18, and 13. However, the evidence base is small and at high risk of bias.

Rapid, point-of-care antigen and molecular-based tests for diagnosis of SARS-CoV-2 infection.

BACKGROUND: Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) and the resulting COVID-19 pandemic present important diagnostic challenges. Several diagnostic strategies are available to identify or rule out current infection, identify people in need of care escalation, or to test for past infection and immune response. Point-of-care antigen and molecular tests to detect current SARS-CoV-2 infection have the potential to allow earlier detection and isolation of confirmed cases compared to laboratory-based diagnostic methods, with the aim of reducing household and community transmission. OBJECTIVES: To assess the diagnostic accuracy of point-of-care antigen and molecular-based tests to determine if a person presenting in the community or in primary or secondary care has current SARS-CoV-2 infection. SEARCH METHODS: On 25 May 2020 we undertook electronic searches in the Cochrane COVID-19 Study Register and the COVID-19 Living Evidence Database from the University of Bern, which is updated daily with published articles from PubMed and Embase and with preprints from medRxiv and bioRxiv. In addition, we checked repositories of COVID-19 publications. We did not apply any language restrictions. SELECTION CRITERIA: We included studies of people with suspected current SARS-CoV-2 infection, known to have, or not to have SARS-CoV-2 infection, or where tests were used to screen for infection. We included test accuracy studies of any design that evaluated antigen or molecular tests suitable for a point-of-care setting (minimal equipment, sample preparation, and biosafety requirements, with results available within two hours of sample collection). We included all reference standards to define the presence or absence of SARS-CoV-2 (including reverse transcription polymerase chain reaction (RT-PCR) tests and established clinical diagnostic criteria). DATA COLLECTION AND ANALYSIS: Two review authors independently screened studies and resolved any disagreements by discussion with a third review author. One review author independently extracted study characteristics, which were checked by a second review author. Two review authors independently extracted 2x2 contingency table data and assessed risk of bias and applicability of the studies using the QUADAS-2 tool. We present sensitivity and specificity, with 95% confidence intervals (CIs), for each test using paired forest plots. We pooled data using the bivariate hierarchical model separately for antigen and molecular-based tests, with simplifications when few studies were available. We tabulated available data by test manufacturer. MAIN RESULTS: We included 22 publications reporting on a total of 18 study cohorts with 3198 unique samples, of which 1775 had confirmed SARS-CoV-2 infection. Ten studies took place in North America, two in South America, four in Europe, one in China and one was conducted internationally. We identified data for eight commercial tests (four antigen and four molecular) and one in-house antigen test. Five of the studies included were only available as preprints. We did not find any studies at low risk of bias for all quality domains and had concerns about applicability of results across all studies. We judged patient selection to be at high risk of bias in 50% of the studies because of deliberate over-sampling of samples with confirmed COVID-19 infection and unclear in seven out of 18 studies because of poor reporting. Sixteen (89%) studies used only a single, negative RT-PCR to confirm the absence of COVID-19 infection, risking missing infection. There was a lack of information on blinding of index test (n = 11), and around participant exclusions from analyses (n = 10). We did not observe differences in methodological quality between antigen and molecular test evaluations. Antigen tests Sensitivity varied considerably across studies (from 0% to 94%): the average sensitivity was 56.2% (95% CI 29.5 to 79.8%) and average specificity was 99.5% (95% CI 98.1% to 99.9%; based on 8 evaluations in 5 studies on 943 samples). Data for individual antigen tests were limited with no more than two studies for any test. Rapid molecular assays Sensitivity showed less variation compared to antigen tests (from 68% to 100%), average sensitivity was 95.2% (95% CI 86.7% to 98.3%) and specificity 98.9% (95% CI 97.3% to 99.5%) based on 13 evaluations in 11 studies of on 2255 samples. Predicted values based on a hypothetical cohort of 1000 people with suspected COVID-19 infection (with a prevalence of 10%) result in 105 positive test results including 10 false positives (positive predictive value 90%), and 895 negative results including 5 false negatives (negative predictive value 99%). Individual tests We calculated pooled results of individual tests for ID NOW (Abbott Laboratories) (5 evaluations) and Xpert Xpress (Cepheid Inc) (6 evaluations). Summary sensitivity for the Xpert Xpress assay (99.4%, 95% CI 98.0% to 99.8%) was 22.6 (95% CI 18.8 to 26.3) percentage points higher than that of ID NOW (76.8%, (95% CI 72.9% to 80.3%), whilst the specificity of Xpert Xpress (96.8%, 95% CI 90.6% to 99.0%) was marginally lower than ID NOW (99.6%, 95% CI 98.4% to 99.9%; a difference of -2.8% (95% CI -6.4 to 0.8)) AUTHORS' CONCLUSIONS: This review identifies early-stage evaluations of point-of-care tests for detecting SARS-CoV-2 infection, largely based on remnant laboratory samples. The findings currently have limited applicability, as we are uncertain whether tests will perform in the same way in clinical practice, and according to symptoms of COVID-19, duration of symptoms, or in asymptomatic people. Rapid tests have the potential to be used to inform triage of RT-PCR use, allowing earlier detection of those testing positive, but the evidence currently is not strong enough to determine how useful they are in clinical practice. Prospective and comparative evaluations of rapid tests for COVID-19 infection in clinically relevant settings are urgently needed. Studies should recruit consecutive series of eligible participants, including both those presenting for testing due to symptoms and asymptomatic people who may have come into contact with confirmed cases. Studies should clearly describe symptomatic status and document time from symptom onset or time since exposure. Point-of-care tests must be conducted on samples according to manufacturer instructions for use and be conducted at the point of care. Any future research study report should conform to the Standards for Reporting of Diagnostic Accuracy (STARD) guideline.

Antibody tests for identification of current and past infection with SARS-CoV-2.

BACKGROUND: The severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) virus and resulting COVID-19 pandemic present important diagnostic challenges. Several diagnostic strategies are available to identify current infection, rule out infection, identify people in need of care escalation, or to test for past infection and immune response. Serology tests to detect the presence of antibodies to SARS-CoV-2 aim to identify previous SARS-CoV-2 infection, and may help to confirm the presence of current infection. OBJECTIVES: To assess the diagnostic accuracy of antibody tests to determine if a person presenting in the community or in primary or secondary care has SARS-CoV-2 infection, or has previously had SARS-CoV-2 infection, and the accuracy of antibody tests for use in seroprevalence surveys. SEARCH METHODS: We undertook electronic searches in the Cochrane COVID-19 Study Register and the COVID-19 Living Evidence Database from the University of Bern, which is updated daily with published articles from PubMed and Embase and with preprints from medRxiv and bioRxiv. In addition, we checked repositories of COVID-19 publications. We did not apply any language restrictions. We conducted searches for this review iteration up to 27 April 2020. SELECTION CRITERIA: We included test accuracy studies of any design that evaluated antibody tests (including enzyme-linked immunosorbent assays, chemiluminescence immunoassays, and lateral flow assays) in people suspected of current or previous SARS-CoV-2 infection, or where tests were used to screen for infection. We also included studies of people either known to have, or not to have SARS-CoV-2 infection. We included all reference standards to define the presence or absence of SARS-CoV-2 (including reverse transcription polymerase chain reaction tests (RT-PCR) and clinical diagnostic criteria). DATA COLLECTION AND ANALYSIS: We assessed possible bias and applicability of the studies using the QUADAS-2 tool. We extracted 2x2 contingency table data and present sensitivity and specificity for each antibody (or combination of antibodies) using paired forest plots. We pooled data using random-effects logistic regression where appropriate, stratifying by time since post-symptom onset. We tabulated available data by test manufacturer. We have presented uncertainty in estimates of sensitivity and specificity using 95% confidence intervals (CIs). MAIN RESULTS: We included 57 publications reporting on a total of 54 study cohorts with 15,976 samples, of which 8526 were from cases of SARS-CoV-2 infection. Studies were conducted in Asia (n = 38), Europe (n = 15), and the USA and China (n = 1). We identified data from 25 commercial tests and numerous in-house assays, a small fraction of the 279 antibody assays listed by the Foundation for Innovative Diagnostics. More than half (n = 28) of the studies included were only available as preprints. We had concerns about risk of bias and applicability. Common issues were use of multi-group designs (n = 29), inclusion of only COVID-19 cases (n = 19), lack of blinding of the index test (n = 49) and reference standard (n = 29), differential verification (n = 22), and the lack of clarity about participant numbers, characteristics and study exclusions (n = 47). Most studies (n = 44) only included people hospitalised due to suspected or confirmed COVID-19 infection. There were no studies exclusively in asymptomatic participants. Two-thirds of the studies (n = 33) defined COVID-19 cases based on RT-PCR results alone, ignoring the potential for false-negative RT-PCR results. We observed evidence of selective publication of study findings through omission of the identity of tests (n = 5). We observed substantial heterogeneity in sensitivities of IgA, IgM and IgG antibodies, or combinations thereof, for results aggregated across different time periods post-symptom onset (range 0% to 100% for all target antibodies). We thus based the main results of the review on the 38 studies that stratified results by time since symptom onset. The numbers of individuals contributing data within each study each week are small and are usually not based on tracking the same groups of patients over time. Pooled results for IgG, IgM, IgA, total antibodies and IgG/IgM all showed low sensitivity during the first week since onset of symptoms (all less than 30.1%), rising in the second week and reaching their highest values in the third week. The combination of IgG/IgM had a sensitivity of 30.1% (95% CI 21.4 to 40.7) for 1 to 7 days, 72.2% (95% CI 63.5 to 79.5) for 8 to 14 days, 91.4% (95% CI 87.0 to 94.4) for 15 to 21 days. Estimates of accuracy beyond three weeks are based on smaller sample sizes and fewer studies. For 21 to 35 days, pooled sensitivities for IgG/IgM were 96.0% (95% CI 90.6 to 98.3). There are insufficient studies to estimate sensitivity of tests beyond 35 days post-symptom onset. Summary specificities (provided in 35 studies) exceeded 98% for all target antibodies with confidence intervals no more than 2 percentage points wide. False-positive results were more common where COVID-19 had been suspected and ruled out, but numbers were small and the difference was within the range expected by chance. Assuming a prevalence of 50%, a value considered possible in healthcare workers who have suffered respiratory symptoms, we would anticipate that 43 (28 to 65) would be missed and 7 (3 to 14) would be falsely positive in 1000 people undergoing IgG/IgM testing at days 15 to 21 post-symptom onset. At a prevalence of 20%, a likely value in surveys in high-risk settings, 17 (11 to 26) would be missed per 1000 people tested and 10 (5 to 22) would be falsely positive. At a lower prevalence of 5%, a likely value in national surveys, 4 (3 to 7) would be missed per 1000 tested, and 12 (6 to 27) would be falsely positive. Analyses showed small differences in sensitivity between assay type, but methodological concerns and sparse data prevent comparisons between test brands. AUTHORS' CONCLUSIONS: The sensitivity of antibody tests is too low in the first week since symptom onset to have a primary role for the diagnosis of COVID-19, but they may still have a role complementing other testing in individuals presenting later, when RT-PCR tests are negative, or are not done. Antibody tests are likely to have a useful role for detecting previous SARS-CoV-2 infection if used 15 or more days after the onset of symptoms. However, the duration of antibody rises is currently unknown, and we found very little data beyond 35 days post-symptom onset. We are therefore uncertain about the utility of these tests for seroprevalence surveys for public health management purposes. Concerns about high risk of bias and applicability make it likely that the accuracy of tests when used in clinical care will be lower than reported in the included studies. Sensitivity has mainly been evaluated in hospitalised patients, so it is unclear whether the tests are able to detect lower antibody levels likely seen with milder and asymptomatic COVID-19 disease. The design, execution and reporting of studies of the accuracy of COVID-19 tests requires considerable improvement. Studies must report data on sensitivity disaggregated by time since onset of symptoms. COVID-19-positive cases who are RT-PCR-negative should be included as well as those confirmed RT-PCR, in accordance with the World Health Organization (WHO) and China National Health Commission of the People's Republic of China (CDC) case definitions. We were only able to obtain data from a small proportion of available tests, and action is needed to ensure that all results of test evaluations are available in the public domain to prevent selective reporting. This is a fast-moving field and we plan ongoing updates of this living systematic review.