From driverless dilemmas to more practical commonsense tests for automated vehicles.

For the first time in history, automated vehicles (AVs) are being deployed in populated environments. This unprecedented transformation of our everyday lives demands a significant undertaking: endowing complex autonomous systems with ethically acceptable behavior. We outline how one prominent, ethically relevant component of AVs-driving behavior-is inextricably linked to stakeholders in the technical, regulatory, and social spheres of the field. Whereas humans are presumed (rightly or wrongly) to have the "common sense" to behave ethically in new driving situations beyond a standard driving test, AVs do not (and probably should not) enjoy this presumption. We examine, at a high level, how to test the common sense of an AV. We start by reviewing discussions of "driverless dilemmas," adaptions of the traditional "trolley dilemmas" of philosophy that have sparked discussion on AV ethics but have limited use to the technical and legal spheres. Then, we explain how to substantially change the premises and features of these dilemmas (while preserving their behavioral diagnostic spirit) in order to lay the foundations for a more practical and relevant framework that tests driving common sense as an integral part of road rules testing.

Outcome choice and definition in systematic reviews leads to few eligible studies included in meta-analyses: a case study.

BACKGROUND: There is broad recognition of the importance of evidence in informing clinical decisions. When information from all studies included in a systematic review ("review") does not contribute to a meta-analysis, decision-makers can be frustrated. Our objectives were to use the field of eyes and vision as a case study and examine the extent to which authors of Cochrane reviews conducted meta-analyses for their review's pre-specified main outcome domain and the reasons that some otherwise eligible studies were not incorporated into meta-analyses. METHODS: We examined all completed systematic reviews published by Cochrane Eyes and Vision, as of August 11, 2017. We extracted information about each review's outcomes and, using an algorithm, categorized one outcome as its "main" outcome. We calculated the percentage of included studies incorporated into meta-analyses for any outcome and for the main outcome. We examined reasons for non-inclusion of studies into the meta-analysis for the main outcome. RESULTS: We identified 175 completed reviews, of which 125 reviews included two or more studies. Across these 125 reviews, the median proportions of studies incorporated into at least one meta-analysis for any outcome and for the main outcome were 74% (interquartile range [IQR] 0-100%) and 28% (IQR 0-71%), respectively. Fifty-one reviews (41%) could not conduct a meta-analysis for the main outcome, mostly because fewer than two included studies measured the outcome (21/51 reviews) or the specific measurements for the outcome were inconsistent (16/51 reviews). CONCLUSIONS: Outcome choice during systematic reviews can lead to few eligible studies included in meta-analyses. Core outcome sets and improved reporting of outcomes can help solve some of these problems.

Treatments of Primary Basal Cell Carcinoma of the Skin: A Systematic Review and Network Meta-analysis.

Background: Most interventions for basal cell carcinoma (BCC) have not been compared in head-to-head randomized trials. Purpose: To evaluate the comparative effectiveness and safety of treatments of primary BCC in adults. Data Sources: English-language searches of MEDLINE, Cochrane Central Register of Controlled Trials, Cochrane Database of Systematic Reviews, and Embase from inception to May 2018; reference lists of guidelines and systematic reviews; and a search of ClinicalTrials.gov in August 2016. Study Selection: Comparative studies of treatments currently used in adults with primary BCC. Data Extraction: One investigator extracted data on recurrence, histologic clearance, clinical clearance, cosmetic outcomes, quality of life, and mortality, and a second reviewer verified extractions. Several investigators evaluated risk of bias for each study. Data Synthesis: Forty randomized trials and 5 nonrandomized studies compared 18 interventions in 9 categories. Relative intervention effects and mean outcome frequencies were estimated using frequentist network meta-analyses. Estimated recurrence rates were similar for excision (3.8% [95% CI, 1.5% to 9.5%]), Mohs surgery (3.8% [CI, 0.7% to 18.2%]), curettage and diathermy (6.9% [CI, 0.9% to 36.6%]), and external-beam radiation (3.5% [CI, 0.7% to 16.8%]). Recurrence rates were higher for cryotherapy (22.3% [CI, 10.2% to 42.0%]), curettage and cryotherapy (19.9% [CI, 4.6% to 56.1%]), 5-fluorouracil (18.8% [CI, 10.1% to 32.5%]), imiquimod (14.1% [CI, 5.4% to 32.4%]), and photodynamic therapy using methyl-aminolevulinic acid (18.8% [CI, 10.1% to 32.5%]) or aminolevulinic acid (16.6% [CI, 7.5% to 32.8%]). The proportion of patients reporting good or better cosmetic outcomes was better for photodynamic therapy using methyl-aminolevulinic acid (93.8% [CI, 79.2% to 98.3%]) or aminolevulinic acid (95.8% [CI, 84.2% to 99.0%]) than for excision (77.8% [CI, 44.8% to 93.8%]) or cryotherapy (51.1% [CI, 15.8% to 85.4%]). Data on quality of life and mortality were too sparse for quantitative synthesis. Limitation: Data are sparse, and effect estimates are imprecise and informed by indirect comparisons. Conclusion: Surgical treatments and external-beam radiation have low recurrence rates for the treatment of low-risk BCC, but substantial uncertainty exists about their comparative effectiveness versus other treatments. Gaps remain regarding high-risk BCC subtypes and important outcomes, including costs. Primary Funding Source: Agency for Healthcare Research and Quality. (PROSPERO: CRD42016043353).

The Systematic Review Data Repository (SRDR): descriptive characteristics of publicly available data and opportunities for research.

BACKGROUND: Conducting systematic reviews ("reviews") requires a great deal of effort and resources. Making data extracted during reviews available publicly could offer many benefits, including reducing unnecessary duplication of effort, standardizing data, supporting analyses to address secondary research questions, and facilitating methodologic research. Funded by the US Agency for Healthcare Research and Quality (AHRQ), the Systematic Review Data Repository (SRDR) is a free, web-based, open-source, data management and archival platform for reviews. Our specific objectives in this paper are to describe (1) the current extent of usage of SRDR and (2) the characteristics of all projects with publicly available data on the SRDR website. METHODS: We examined all projects with data made publicly available through SRDR as of November 12, 2019. We extracted information about the characteristics of these projects. Two investigators extracted and verified the data. RESULTS: SRDR has had 2552 individual user accounts belonging to users from 80 countries. Since SRDR's launch in 2012, data have been made available publicly for 152 of the 735 projects in SRDR (21%), at a rate of 24.5 projects per year, on average. Most projects are in clinical fields (144/152 projects; 95%); most have evaluated interventions (therapeutic or preventive) (109/152; 72%). The most frequent health areas addressed are mental and behavioral disorders (31/152; 20%) and diseases of the eye and ocular adnexa (23/152; 15%). Two-thirds of the projects (104/152; 67%) were funded by AHRQ, and one-sixth (23/152; 15%) are Cochrane reviews. The 152 projects each address a median of 3 research questions (IQR 1-5) and include a median of 70 studies (IQR 20-130). CONCLUSIONS: Until we arrive at a future in which the systematic review and broader research communities are comfortable with the accuracy of automated data extraction, re-use of data extracted by humans has the potential to help reduce redundancy and costs. The 152 projects with publicly available data through SRDR, and the more than 15,000 studies therein, are freely available to researchers and the general public who might be working on similar reviews or updates of reviews or who want access to the data for decision-making, meta-research, or other purposes.

Features and functioning of Data Abstraction Assistant, a software application for data abstraction during systematic reviews.

INTRODUCTION: During systematic reviews, data abstraction is labor- and time-intensive and error-prone. Existing data abstraction systems do not track specific locations and contexts of abstracted information. To address this limitation, we developed a software application, the Data Abstraction Assistant (DAA) and surveyed early users about their experience using DAA. FEATURES OF DAA: We designed DAA to encompass three essential features: (1) a platform for indicating the source of abstracted information, (2) compatibility with a variety of data abstraction systems, and (3) user-friendliness. HOW DAA FUNCTIONS: DAA (1) converts source documents from PDF to HTML format (to enable tracking of source of abstracted information), (2) transmits the HTML to the data abstraction system, and (3) displays the HTML in an area adjacent to the data abstraction form in the data abstraction system. The data abstractor can mark locations on the HTML that DAA associates with items on the data abstraction form. EXPERIENCES OF EARLY USERS OF DAA: When we surveyed 52 early users of DAA, 83% reported that using DAA was either very or somewhat easy; 71% are very or somewhat likely to use DAA in the future; and 87% are very or somewhat likely to recommend that others use DAA in the future. DISCUSSION: DAA, a user-friendly software for linking abstracted data with their exact source, is likely to be a very useful tool in the toolbox of systematic reviewers. DAA facilitates verification of abstracted data and provides an audit trail that is crucial for reproducible research.

A randomized trial provided new evidence on the accuracy and efficiency of traditional vs. electronically annotated abstraction approaches in systematic reviews.

OBJECTIVES: Data Abstraction Assistant (DAA) is a software for linking items abstracted into a data collection form for a systematic review to their locations in a study report. We conducted a randomized cross-over trial that compared DAA-facilitated single-data abstraction plus verification ("DAA verification"), single data abstraction plus verification ("regular verification"), and independent dual data abstraction plus adjudication ("independent abstraction"). STUDY DESIGN AND SETTING: This study is an online randomized cross-over trial with 26 pairs of data abstractors. Each pair abstracted data from six articles, two per approach. Outcomes were the proportion of errors and time taken. RESULTS: Overall proportion of errors was 17% for DAA verification, 16% for regular verification, and 15% for independent abstraction. DAA verification was associated with higher odds of errors when compared with regular verification (adjusted odds ratio [OR] = 1.08; 95% confidence interval [CI]: 0.99-1.17) or independent abstraction (adjusted OR = 1.12; 95% CI: 1.03-1.22). For each article, DAA verification took 20 minutes (95% CI: 1-40) longer than regular verification, but 46 minutes (95% CI: 26 to 66) shorter than independent abstraction. CONCLUSION: Independent abstraction may only be necessary for complex data items. DAA provides an audit trail that is crucial for reproducible research.

Response to 'Increasing value and reducing waste in data extraction for systematic reviews: tracking data in data extraction forms'.

ᅟ: This is a response to a Letter. Data abstraction is a time-consuming and error-prone systematic review task. Shokraneh and Adams categorize available techniques for tracking data during data abstraction into three methods: simple annotation, descriptive addressing, and Cartesian coordinate system. While we agree with the categorization of the techniques, we disagree with the authors' statement that descriptive addressing is a PDF-independent method, i.e., any sort of descriptive addressing must reference a specific version of PDF file and not just any PDF of said report. Different versions of PDFs of the same report might place text and tables on different locations of the same page and/or on different pages. Consequently, it is our opinion that any kind of source location information should be accompanied by the source or linked by an intermediary service such as the Data Abstraction Assistant (DAA).

Strategies for improving the lives of US women aged 40 and above living with HIV/AIDS: an evidence map.

BACKGROUND: While in its early years the HIV epidemic affected primarily the male and the young, nowadays, the population living with HIV/AIDS is approximately 24% women, and its age composition has shifted towards older ages. Many of the older women who live with HIV/AIDS also live with the medical and social conditions that accompany aging. This work aims to identify and characterize empirical studies of strategies for the comprehensive management of women over 40, including transgender women, who live with HIV/AIDS. Forty was chosen as an operational age cutoff to identify premenopausal women who are less likely to bear children, as well as peri- and postmenopausal women. METHODS: We conducted a literature search after discussions with a diverse panel of content experts and other stakeholders and developed an evidence map that identified 890 citations that address questions having to do with programs and barriers to engaging with programs, as well as the role of insurance and comorbidities, and have enrolled older women who live with HIV/AIDS. RESULTS: Of these, only 37 (4%) reported results of interest for women over 40 who live with HIV/AIDS, or examined interactions between gender and older age that would allow predictions in this subgroup. Few of the 37 eligible studies focused on women facing obvious challenges, such as immigrants, transgender, physically abused, or those recently released from prison. No studies focused on women caring for dependents, including children and grandchildren, or those diagnosed after age 40. CONCLUSION: The evidence base that is directly applicable to women over 40 who live with HIV/AIDS in the USA is limited, and the research need is broad. We propose research prioritization strategies for this population.

Omega-3 Fatty Acids and Cardiovascular Disease: An Updated Systematic Review.

BACKGROUND: The effect and association of omega-3 fatty acids (n-3 FA) intake and biomarker levels with cardiovascular (CV) clinical and intermediate outcomes remains controversial. We update prior Evidence Reports of n-3 FA and clinical and intermediate CV disease (CVD) outcomes. OBJECTIVES: Evaluate the effect of n-3 FA on clinical and selected intermediate CV outcomes and the association of n-3 FA intake and biomarkers with CV outcomes. The n-3 FA under review include eicosapentaenoic acid (EPA), docosahexaenoic acid (DHA), docosapentaenoic acid (DPA), stearidonic acid (SDA), and alphalinolenic acid (ALA). DATA SOURCES: MEDLINE®, Embase®, the Cochrane Central Register of Controlled Trials, Cochrane Database of Systematic Reviews, and CAB Abstracts from 2000 or 2002 to June 8, 2015, and eligible studies from the original reports and relevant existing systematic reviews. REVIEW METHODS: We included randomized controlled trials (RCTs) of any n-3 FA intake compared to no, lower, or other n-3 FA intake with an outcome of interest conducted in healthy adults, those at risk for CVD, or those with CVD. We also included prospective observational studies of the association between baseline n-3 FA intake or biomarker level and followup outcomes. We required 1 year or more of followup for clinical outcomes and 4 weeks for intermediate outcomes (blood pressure [BP] and lipids). RESULTS: From 11,440 citations (from electronic literature searches and existing systematic reviews), 829 abstracts met basic eligibility criteria; 61 RCTs and 37 longitudinal observational studies (in 147 articles) were included. Most RCTs and observational studies had few risk-of-bias concerns.Total n-3 FA: There is low strength of evidence (SoE) of no association between total n-3 FA intake and stroke death or myocardial infarction. There is insufficient evidence for other outcomes.Marine oils, total: There is moderate to high SoE that higher marine oil intake lowers triglycerides (Tg), raises high density lipoprotein cholesterol (HDL-c), and lowers the ratio of total cholesterol to HDL-c but raises low density lipoprotein cholesterol (LDL-c); also that higher marine oil intake does not affect major adverse CV events, all-cause death, total stroke, sudden cardiac death, coronary revascularization, atrial fibrillation, or BP. There is low SoE of associations between higher marine oil intake and decreased risk of CVD death, coronary heart disease (CHD), myocardial infarction, ischemic stroke, and congestive heart failure (CHF). There is low SoE of no association with CHD death or hemorrhagic stroke. There is insufficient evidence for other outcomes.Marine oil FA individually: There is low SoE of no associations between EPA or DHA intake (separately) and CHD, and between EPA or DPA and atrial fibrillation. There is low SoE of no association between EPA biomarkers and atrial fibrillation, but moderate SoE of no effect of purified DHA supplementation on BP or LDL-c. There is insufficient evidence for other specific marine oil FA and outcomes.ALA: There is moderate SoE of no effect of ALA intake on BP, LDL-c, HDL-c, or Tg. There is low SoE of no association between ALA intake or biomarker level and CHD, CHD death, atrial fibrillation, and CHF. There is insufficient evidence for other outcomes.Other n-3 FA analyses: There is insufficient evidence comparing n-3 FA with each other or for SDA.Subgroup analyses: Nineteen of 22 studies found no interaction of sex on any effect of n-3 FA. Likewise, 19 of 20 studies found no differential effect by statin co-use. Within 16 studies evaluating diabetes subgroups, 2 found statistically significant beneficial effects of n-3 FA in those with diabetes but not in those without diabetes, but no test of interaction was reported. CONCLUSIONS: The 61 RCTs mostly compared marine oil supplements with placebo on CVD outcomes in populations at risk for CVD or with CVD, while the 37 observational studies mostly examined associations between various individual n-3 FA and long-term CVD events in generally healthy populations. Compared with the prior report on n-3 FA and CVD, there is more robust RCT evidence on ALA and on clinical CV outcomes; also, by design there are newly added data on associations between n-3 FA biomarkers and CV outcomes. However, conclusions regarding the effect of n-3 FA intake on CV outcomes or associations with outcomes remain substantially unchanged. Marine oils statistically significantly raise HDL-c and LDL-c by similar amounts (≤2 mg/dL), while lowering Tg in a dose-dependent manner, particularly in individuals with elevated Tg; they have no significant effect on BP. ALA has no significant effect on intermediate outcomes. Limited data were available from RCTs on the effect of n-3 FA on clinical CVD outcomes. Observational studies suggest that higher marine oil intake (including from dietary fish) is associated with lower risk of several CVD outcomes. No clear differences in effects or associations were evident based on population, demographic features, or cointerventions. Future RCTs would be needed to establish adequate evidence of the effect of n-3 FA on CVD outcomes or to clarify differential effects in different groups of people. However, future trials are unlikely to alter conclusions about the effects of n-3 FA supplementation on intermediate cardiovascular outcomes (BP, LDL-c, HDL-c, or Tg).

Evaluating Data Abstraction Assistant, a novel software application for data abstraction during systematic reviews: protocol for a randomized controlled trial.

BACKGROUND: Data abstraction, a critical systematic review step, is time-consuming and prone to errors. Current standards for approaches to data abstraction rest on a weak evidence base. We developed the Data Abstraction Assistant (DAA), a novel software application designed to facilitate the abstraction process by allowing users to (1) view study article PDFs juxtaposed to electronic data abstraction forms linked to a data abstraction system, (2) highlight (or "pin") the location of the text in the PDF, and (3) copy relevant text from the PDF into the form. We describe the design of a randomized controlled trial (RCT) that compares the relative effectiveness of (A) DAA-facilitated single abstraction plus verification by a second person, (B) traditional (non-DAA-facilitated) single abstraction plus verification by a second person, and (C) traditional independent dual abstraction plus adjudication to ascertain the accuracy and efficiency of abstraction. METHODS: This is an online, randomized, three-arm, crossover trial. We will enroll 24 pairs of abstractors (i.e., sample size is 48 participants), each pair comprising one less and one more experienced abstractor. Pairs will be randomized to abstract data from six articles, two under each of the three approaches. Abstractors will complete pre-tested data abstraction forms using the Systematic Review Data Repository (SRDR), an online data abstraction system. The primary outcomes are (1) proportion of data items abstracted that constitute an error (compared with an answer key) and (2) total time taken to complete abstraction (by two abstractors in the pair, including verification and/or adjudication). DISCUSSION: The DAA trial uses a practical design to test a novel software application as a tool to help improve the accuracy and efficiency of the data abstraction process during systematic reviews. Findings from the DAA trial will provide much-needed evidence to strengthen current recommendations for data abstraction approaches. TRIAL REGISTRATION: The trial is registered at National Information Center on Health Services Research and Health Care Technology (NICHSR) under Registration # HSRP20152269: https://wwwcf.nlm.nih.gov/hsr_project/view_hsrproj_record.cfm?NLMUNIQUE_ID=20152269&SEARCH_FOR=Tianjing%20Li . All items from the World Health Organization Trial Registration Data Set are covered at various locations in this protocol. Protocol version and date: This is version 2.0 of the protocol, dated September 6, 2016. As needed, we will communicate any protocol amendments to the Institutional Review Boards (IRBs) of Johns Hopkins Bloomberg School of Public Health (JHBSPH) and Brown University. We also will make appropriate as-needed modifications to the NICHSR website in a timely fashion.