Surviving in the Valley of Death: Opportunities and Challenges in Translating Academic Drug Discoveries.

With pharmaceutical companies shrinking their research departments and exiting out of efforts related to unprofitable diseases, society has become increasingly dependent on academic institutions to perform drug discovery and early-stage translational research. Academic drug discovery and translational research programs assist in shepherding promising therapeutic opportunities through the so-called valley of death in the hope that a successful new drug will result in saved lives, improved health, economic growth, and financial return. We have interviewed directors of 16 such academic programs in the United States and found that these programs and the projects therein face numerous challenges in reaching the clinic, including limited funding, lack of know-how, and lack of a regional drug development ecosystem. If these issues can be addressed through novel industry partnerships, the revision of government policies, and expanded programs in translational education, more effective new therapies are more likely to reach patients in need.

Ultrasonographic Optic Nerve Sheath Diameter Measurement to Detect Intracranial Hypertension in Children With Neurological Injury: A Systematic Review.

OBJECTIVES: Ultrasound measured optic nerve sheath diameter is a noninvasive, nonirradiating tool for estimating intracranial hypertension. The objective of this systematic review and meta-analysis is summarization of the current evidence for accuracy of ultrasound measured optic nerve sheath diameter in detecting intracranial hypertension in pediatric patients. DATA SOURCES: Medical subject heading terms were used to search MEDLINE, Embase, Google Scholar, Web of Science, and the Cochrane Library for relevant citations. Publications from January 1, 2000, to June 30, 2019, were included in the search strategy. STUDY SELECTION: Studies were included if they involved patients less than 18 years, where ultrasound measured optic nerve sheath diameter was compared to conventional, nonophthalmic tests for intracranial hypertension. Studies were excluded if there was insufficient data to compute a sensitivity/specificity table. Case reports, case series, and manuscripts not published in English were also excluded. DATA EXTRACTION: The initial search returned 573 citations. Of these, 57 were selected for review. DATA SYNTHESIS: Eleven citations were included in the final meta-analysis. A bivariate random-effects meta-analysis was performed, which revealed a pooled sensitivity for ultrasound measured optic nerve sheath diameter of 93% (95% CI, 74-99%), a specificity of 74% (95% CI, 52-88%), and a diagnostic odds ratio of 39.00 (95% CI, 4.16-365.32). The area under the curve of the hierarchical summary receiver operating characteristic curve was 0.90 (95% CI, 0.87-0.93). Subgroup analyses of the test's performance evaluating new-onset intracranial hypertension and in comparison to invasively measured intracranial pressure were performed. The test performance in these instances was similar to findings in the primary analysis. CONCLUSIONS: We are unable to identify a threshold value in ultrasound measured optic nerve sheath diameter for the determination of intracranial hypertension in children. Even though the ultrasound measured optic nerve sheath diameter measurement is highly sensitive to the presence of increased intracranial pressure, the test has only moderate specificity. Therefore, other confirmatory methods and further investigation is necessary in the clinical care of children. The technique is likely not sufficiently precise for clinical use in the absence of other confirmatory methods, and further investigation is necessary to determine clinical protocols for its use in children.

Limitations and Misinterpretations of E-Values for Sensitivity Analyses of Observational Studies.

The E-value was recently introduced on the basis of earlier work as "the minimum strength of association…that an unmeasured confounder would need to have with both the treatment and the outcome to fully explain away a specific treatment-outcome association, conditional on the measured covariates." E-values have been proposed for wide application in observational studies evaluating causality. However, they have limitations and are prone to misinterpretation. E-values have a monotonic, almost linear relationship with effect estimates and thus offer no additional information beyond what effect estimates can convey. Whereas effect estimates are based on real data, E-values may make unrealistic assumptions. No general rule can exist about what is a "small enough" E-value, and users of the biomedical literature are not familiar with how to interpret a range of E-values. Problems arise for any measure dependent on effect estimates and their CIs-for example, bias due to selective reporting and dependence on choice of exposure contrast and level of confidence. The automation of E-values may give an excuse not to think seriously about confounding. Moreover, biases other than confounding may still undermine results. Instead of misused or misinterpreted E-values, the authors recommend judicious use of existing methods for sensitivity analyses with careful assumptions; systematic assessments of whether and how known confounders have been handled, along with consideration of their prevalence and magnitude; thorough discussion of the potential for unknown confounders considering the study design and field of application; and explicit caution in making causal claims from observational studies.

Implementing and monitoring high-quality community health worker care in adult primary care at New York City Health + Hospitals.

BACKGROUND: This study describes how New York City (NYC) Health + Hospitals implemented a large-scale Community Health Worker (CHW) program in adult primary care clinics between January 2022 and December 2023 and established metrics to monitor program implementation. This study is timely as healthcare systems consider how to scale high-quality CHW programs. METHODS: We collected metrics in the following areas: (1) Workforce demographics, team structure, and training; (2) Enrolled patient demographics; (3) Patient-centered metrics, such as patient counts (e.g. patients outreached and enrolled) and engagement (e.g. median time in program, caseloads per CHW), and goals (e.g. median number of goals identified and completed). Metrics are based on standard data elements captured through CHW documentation in the electronic health record collected during program implementation. Data cleaning is completed using SQL queries and R scripts. RESULTS: In June 2023, there were a total of 97 CHW and 22 CHW Supervisor staff lines in adult primary care across 17 healthcare sites. There were 4.6 CHWs to 1 CHW supervisor on average though this ranged by facility from 1:1 to 1:6. Compared to the population that receives primary care at NYC H + H, CHWs served more African American/Black patients (40% vs. 32%) and an older patient population (35% older than 65 vs. 21% older than 65). From January 2022 to December 2023, 13,812 patients were outreached by CHWs. Of these, 9,069 (66%) were referred by clinicians, 7,331 (53%) were enrolled, and 5,044 (37%) successfully graduated. The median number of goals identified by patients was four, and the median number of goals completed with a CHW per patient was three. The top three goals were primary care engagement (47%), specialty care engagement (46%), and food insecurity (45%). CONCLUSION: Establishing clear implementation and process metrics helps to ensure that CHWs embedded in health systems can meaningfully engage adult patients in health care, address patient-centered goals, and connect patients to community and government services.

Empirical assessment of bias in machine learning diagnostic test accuracy studies.

OBJECTIVE: Machine learning (ML) diagnostic tools have significant potential to improve health care. However, methodological pitfalls may affect diagnostic test accuracy studies used to appraise such tools. We aimed to evaluate the prevalence and reporting of design characteristics within the literature. Further, we sought to empirically assess whether design features may be associated with different estimates of diagnostic accuracy. MATERIALS AND METHODS: We systematically retrieved 2 × 2 tables (n = 281) describing the performance of ML diagnostic tools, derived from 114 publications in 38 meta-analyses, from PubMed. Data extracted included test performance, sample sizes, and design features. A mixed-effects metaregression was run to quantify the association between design features and diagnostic accuracy. RESULTS: Participant ethnicity and blinding in test interpretation was unreported in 90% and 60% of studies, respectively. Reporting was occasionally lacking for rudimentary characteristics such as study design (28% unreported). Internal validation without appropriate safeguards was used in 44% of studies. Several design features were associated with larger estimates of accuracy, including having unreported (relative diagnostic odds ratio [RDOR], 2.11; 95% confidence interval [CI], 1.43-3.1) or case-control study designs (RDOR, 1.27; 95% CI, 0.97-1.66), and recruiting participants for the index test (RDOR, 1.67; 95% CI, 1.08-2.59). DISCUSSION: Significant underreporting of experimental details was present. Study design features may affect estimates of diagnostic performance in the ML diagnostic test accuracy literature. CONCLUSIONS: The present study identifies pitfalls that threaten the validity, generalizability, and clinical value of ML diagnostic tools and provides recommendations for improvement.

Use of E-values for addressing confounding in observational studies-an empirical assessment of the literature.

BACKGROUND: E-values are a recently introduced approach to evaluate confounding in observational studies. We aimed to empirically assess the current use of E-values in published literature. METHODS: We conducted a systematic literature search for all publications, published up till the end of 2018, which cited at least one of two inceptive E-value papers and presented E-values for original data. For these case publications we identified control publications, matched by journal and issue, where the authors had not calculated E-values. RESULTS: In total, 87 papers presented 516 E-values. Of the 87 papers, 14 concluded that residual confounding likely threatens at least some of the main conclusions. Seven of these 14 named potential uncontrolled confounders. 19 of 87 papers related E-value magnitudes to expected strengths of field-specific confounders. The median E-value was 1.88, 1.82, and 2.02 for the 43, 348, and 125 E-values where confounding was felt likely to affect the results, unlikely to affect the results, or not commented upon, respectively. The 69 case-control publication pairs dealt with effect sizes of similar magnitude. Of 69 control publications, 52 did not comment on unmeasured confounding and 44/69 case publications concluded that confounding was unlikely to affect study conclusions. CONCLUSIONS: Few papers using E-values conclude that confounding threatens their results, and their E-values overlap in magnitude with those of papers acknowledging susceptibility to confounding. Facile automation in calculating E-values may compound the already poor handling of confounding. E-values should not be a substitute for careful consideration of potential sources of unmeasured confounding. If used, they should be interpreted in the context of expected confounding in specific fields.

An empirical assessment of research practices across 163 clinical trials of tumor-bearing companion dogs.

Comparative clinical trials of domestic dogs with spontaneously-occurring cancers are increasingly common. Canine cancers are likely more representative of human cancers than induced murine tumors. These trials could bridge murine models and human trials and better prioritize drug candidates. Such investigations also benefit veterinary patients. We aimed to evaluate the design and reporting practices of clinical trials containing ≥2 arms and involving tumor-bearing dogs. 163 trials containing 8552 animals were systematically retrieved from PubMed (searched 1/18/18). Data extracted included sample sizes, response criteria, study design, and outcome reporting. Low sample sizes were prevalent (median n = 33). The median detectable hazard ratio was 0.3 for overall survival and 0.06 for disease progression. Progressive disease thresholds for studies that did not adopt VCOG-RECIST guidelines varied in stringency. Additionally, there was significant underreporting across all Cochrane risk of bias categories. The proportion of studies with unclear reporting ranged from 44% (randomization) to 94% (selective reporting). 72% of studies also failed to define a primary outcome. The present study confirms previous findings that clinical trials in dogs need to be improved, particularly regarding low statistical power and underreporting of design and outcomes.

Cost Effectiveness of Chimeric Antigen Receptor T-Cell Therapy in Relapsed or Refractory Pediatric B-Cell Acute Lymphoblastic Leukemia.

PURPOSE: The anti-CD19 chimeric antigen receptor T-cell therapy tisagenlecleucel was recently approved to treat relapsed or refractory pediatric acute lymphoblastic leukemia. With a one-time infusion cost of $475,000, tisagenlecleucel is currently the most expensive oncologic therapy. We aimed to determine whether tisagenlecleucel is cost effective compared with currently available treatments. METHODS: Markov modeling was used to evaluate tisagenlecleucel in pediatric relapsed or refractory acute lymphoblastic leukemia from a US health payer perspective over a lifetime horizon. The model was informed by recent multicenter, single-arm clinical trials. Tisagenlecleucel (under a range of plausible long-term effectiveness) was compared with blinatumomab, clofarabine combination therapy (clofarabine, etoposide, and cyclophosphamide), and clofarabine monotherapy. Scenario and probabilistic sensitivity analyses were used to explore uncertainty. Main outcomes were life-years, discounted lifetime costs, discounted quality-adjusted life-years (QALYs), and incremental cost-effectiveness ratio (3% discount rate). RESULTS: With an assumption of a 40% 5-year relapse-free survival rate, tisagenlecleucel increased life expectancies by 12.1 years and cost $61,000/QALY gained. However, at a 20% 5-year relapse-free survival rate, life-expectancies were more modest (3.8 years) and expensive ($151,000/QALY gained). At a 0% 5-year relapse-free survival rate and with use as a bridge to transplant, tisagenlecleucel increased life expectancies by 5.7 years and cost $184,000/QALY gained. Reduction of the price of tisagenlecleucel to $200,000 or $350,000 would allow it to meet a $100,000/QALY or $150,000/QALY willingness-to-pay threshold in all scenarios. CONCLUSION: The long-term effectiveness of tisagenlecleucel is a critical but uncertain determinant of its cost effectiveness. At its current price, tisagenlecleucel represents reasonable value if it can keep a substantial fraction of patients in remission without transplantation; however, if all patients ultimately require a transplantation to remain in remission, it will not be cost effective at generally accepted thresholds. Price reductions would favorably influence cost effectiveness even if long-term clinical outcomes are modest.