Large language models encode clinical knowledge.

Large language models (LLMs) have demonstrated impressive capabilities, but the bar for clinical applications is high. Attempts to assess the clinical knowledge of models typically rely on automated evaluations based on limited benchmarks. Here, to address these limitations, we present MultiMedQA, a benchmark combining six existing medical question answering datasets spanning professional medicine, research and consumer queries and a new dataset of medical questions searched online, HealthSearchQA. We propose a human evaluation framework for model answers along multiple axes including factuality, comprehension, reasoning, possible harm and bias. In addition, we evaluate Pathways Language Model1 (PaLM, a 540-billion parameter LLM) and its instruction-tuned variant, Flan-PaLM2 on MultiMedQA. Using a combination of prompting strategies, Flan-PaLM achieves state-of-the-art accuracy on every MultiMedQA multiple-choice dataset (MedQA3, MedMCQA4, PubMedQA5 and Measuring Massive Multitask Language Understanding (MMLU) clinical topics6), including 67.6% accuracy on MedQA (US Medical Licensing Exam-style questions), surpassing the prior state of the art by more than 17%. However, human evaluation reveals key gaps. To resolve this, we introduce instruction prompt tuning, a parameter-efficient approach for aligning LLMs to new domains using a few exemplars. The resulting model, Med-PaLM, performs encouragingly, but remains inferior to clinicians. We show that comprehension, knowledge recall and reasoning improve with model scale and instruction prompt tuning, suggesting the potential utility of LLMs in medicine. Our human evaluations reveal limitations of today's models, reinforcing the importance of both evaluation frameworks and method development in creating safe, helpful LLMs for clinical applications.

An uncharacteristically low-potential flavin governs the energy landscape of electron bifurcation.

Electron bifurcation, an energy-conserving process utilized extensively throughout all domains of life, represents an elegant means of generating high-energy products from substrates with less reducing potential. The coordinated coupling of exergonic and endergonic reactions has been shown to operate over an electrochemical potential of ∼1.3 V through the activity of a unique flavin cofactor in the enzyme NADH-dependent ferredoxin-NADP+ oxidoreductase I. The inferred energy landscape has features unprecedented in biochemistry and presents novel energetic challenges, the most intriguing being a large thermodynamically uphill step for the first electron transfer of the bifurcation reaction. However, ambiguities in the energy landscape at the bifurcating site deriving from overlapping flavin spectral signatures have impeded a comprehensive understanding of the specific mechanistic contributions afforded by thermodynamic and kinetic factors. Here, we elucidate an uncharacteristically low two-electron potential of the bifurcating flavin, resolving the energetic challenge of the first bifurcation event.

Engineering the cellulolytic bacterium, Clostridium thermocellum, to co-utilize hemicellulose.

Consolidated bioprocessing (CBP) of lignocellulosic biomass holds promise to realize economic production of second-generation biofuels/chemicals, and Clostridium thermocellum is a leading candidate for CBP due to it being one of the fastest degraders of crystalline cellulose and lignocellulosic biomass. However, CBP by C. thermocellum is approached with co-cultures, because C. thermocellum does not utilize hemicellulose. When compared with a single-species fermentation, the co-culture system introduces unnecessary process complexity that may compromise process robustness. In this study, we engineered C. thermocellum to co-utilize hemicellulose without the need for co-culture. By evolving our previously engineered xylose-utilizing strain in xylose, an evolved clonal isolate (KJC19-9) was obtained and showed improved specific growth rate on xylose by ∼3-fold and displayed comparable growth to a minimally engineered strain grown on the bacteria's naturally preferred substrate, cellobiose. To enable full xylan deconstruction to xylose, we recombinantly expressed three different ß-xylosidase enzymes originating from Thermoanaerobacterium saccharolyticum into KJC19-9 and demonstrated growth on xylan with one of the enzymes. This recombinant strain was capable of co-utilizing cellulose and xylan simultaneously, and we integrated the ß-xylosidase gene into the KJC19-9 genome, creating the KJCBXint strain. The strain, KJC19-9, consumed monomeric xylose but accumulated xylobiose when grown on pretreated corn stover, whereas the final KJCBXint strain showed significantly greater deconstruction of xylan and xylobiose. This is the first reported C. thermocellum strain capable of degrading and assimilating hemicellulose polysaccharide while retaining its cellulolytic capabilities, unlocking significant potential for CBP in advancing the bioeconomy.

Deep Learning Detection of Active Pulmonary Tuberculosis at Chest Radiography Matched the Clinical Performance of Radiologists.

Background The World Health Organization (WHO) recommends chest radiography to facilitate tuberculosis (TB) screening. However, chest radiograph interpretation expertise remains limited in many regions. Purpose To develop a deep learning system (DLS) to detect active pulmonary TB on chest radiographs and compare its performance to that of radiologists. Materials and Methods A DLS was trained and tested using retrospective chest radiographs (acquired between 1996 and 2020) from 10 countries. To improve generalization, large-scale chest radiograph pretraining, attention pooling, and semisupervised learning ("noisy-student") were incorporated. The DLS was evaluated in a four-country test set (China, India, the United States, and Zambia) and in a mining population in South Africa, with positive TB confirmed with microbiological tests or nucleic acid amplification testing (NAAT). The performance of the DLS was compared with that of 14 radiologists. The authors studied the efficacy of the DLS compared with that of nine radiologists using the Obuchowski-Rockette-Hillis procedure. Given WHO targets of 90% sensitivity and 70% specificity, the operating point of the DLS (0.45) was prespecified to favor sensitivity. Results A total of 165 754 images in 22 284 subjects (mean age, 45 years; 21% female) were used for model development and testing. In the four-country test set (1236 subjects, 17% with active TB), the receiver operating characteristic (ROC) curve of the DLS was higher than those for all nine India-based radiologists, with an area under the ROC curve of 0.89 (95% CI: 0.87, 0.91). Compared with these radiologists, at the prespecified operating point, the DLS sensitivity was higher (88% vs 75%, P < .001) and specificity was noninferior (79% vs 84%, P = .004). Trends were similar within other patient subgroups, in the South Africa data set, and across various TB-specific chest radiograph findings. In simulations, the use of the DLS to identify likely TB-positive chest radiographs for NAAT confirmation reduced the cost by 40%-80% per TB-positive patient detected. Conclusion A deep learning method was found to be noninferior to radiologists for the determination of active tuberculosis on digital chest radiographs. © RSNA, 2022 Online supplemental material is available for this article. See also the editorial by van Ginneken in this issue.

Customization scenarios for de-identification of clinical notes.

BACKGROUND: Automated machine-learning systems are able to de-identify electronic medical records, including free-text clinical notes. Use of such systems would greatly boost the amount of data available to researchers, yet their deployment has been limited due to uncertainty about their performance when applied to new datasets. OBJECTIVE: We present practical options for clinical note de-identification, assessing performance of machine learning systems ranging from off-the-shelf to fully customized. METHODS: We implement a state-of-the-art machine learning de-identification system, training and testing on pairs of datasets that match the deployment scenarios. We use clinical notes from two i2b2 competition corpora, the Physionet Gold Standard corpus, and parts of the MIMIC-III dataset. RESULTS: Fully customized systems remove 97-99% of personally identifying information. Performance of off-the-shelf systems varies by dataset, with performance mostly above 90%. Providing a small labeled dataset or large unlabeled dataset allows for fine-tuning that improves performance over off-the-shelf systems. CONCLUSION: Health organizations should be aware of the levels of customization available when selecting a de-identification deployment solution, in order to choose the one that best matches their resources and target performance level.

CO2-fixing one-carbon metabolism in a cellulose-degrading bacterium Clostridium thermocellum.

Clostridium thermocellum can ferment cellulosic biomass to formate and other end products, including CO2 This organism lacks formate dehydrogenase (Fdh), which catalyzes the reduction of CO2 to formate. However, feeding the bacterium 13C-bicarbonate and cellobiose followed by NMR analysis showed the production of 13C-formate in C. thermocellum culture, indicating the presence of an uncharacterized pathway capable of converting CO2 to formate. Combining genomic and experimental data, we demonstrated that the conversion of CO2 to formate serves as a CO2 entry point into the reductive one-carbon (C1) metabolism, and internalizes CO2 via two biochemical reactions: the reversed pyruvate:ferredoxin oxidoreductase (rPFOR), which incorporates CO2 using acetyl-CoA as a substrate and generates pyruvate, and pyruvate-formate lyase (PFL) converting pyruvate to formate and acetyl-CoA. We analyzed the labeling patterns of proteinogenic amino acids in individual deletions of all five putative PFOR mutants and in a PFL deletion mutant. We identified two enzymes acting as rPFOR, confirmed the dual activities of rPFOR and PFL crucial for CO2 uptake, and provided physical evidence of a distinct in vivo "rPFOR-PFL shunt" to reduce CO2 to formate while circumventing the lack of Fdh. Such a pathway precedes CO2 fixation via the reductive C1 metabolic pathway in C. thermocellum These findings demonstrated the metabolic versatility of C. thermocellum, which is thought of as primarily a cellulosic heterotroph but is shown here to be endowed with the ability to fix CO2 as well.

Engineering cellulolytic bacterium Clostridium thermocellum to co-ferment cellulose- and hemicellulose-derived sugars simultaneously.

Cellulose and hemicellulose are the most abundant components in plant biomass. A preferred Consolidated Bioprocessing (CBP) system is one which can directly convert both cellulose and hemicellulose into target products without adding the costly hydrolytic enzyme cocktail. In this work, the thermophilic, cellulolytic, and anaerobic bacterium, Clostridium thermocellum DSM 1313, was engineered to grow on xylose in addition to cellulose. Both xylA (encoding for xylose isomerase) and xylB (encoding for xylulokinase) genes from the thermophilic anaerobic bacterium Thermoanaerobacter ethanolicus were introduced to enable xylose utilization while still retaining its inherent ability to grow on 6-carbon substrates. Targeted integration of xylAB into C. thermocellum genome realized simultaneous fermentation of xylose with glucose, with cellobiose (glucose dimer), and with cellulose, respectively, without carbon catabolite repression. We also showed that the respective H2 and ethanol production were twice as much when both xylose and cellulose were consumed simultaneously than when consuming cellulose alone. Moreover, the engineered xylose consumer can also utilize xylo-oligomers (with degree of polymerization of 2-7) in the presence of xylose. Isotopic tracer studies also revealed that the engineered xylose catabolism contributed to the production of ethanol from xylan which is a model hemicellulose in mixed sugar fermentation, demonstrating immense potential of this enhanced CBP strain in co-utilizing both cellulose and hemicellulose for the production of fuels and chemicals.

Screening adolescents for sexually transmitted infections in the pediatric emergency department.

OBJECTIVES: The objectives of this study were to determine the prevalence of Chlamydia trachomatis (CT) and Neisseria gonorrhoeae (GC) infections among adolescents presenting to a pediatric emergency department (PED), to assess the association between these infections and certain risk factors, and to assess the feasibility of routine screening for sexually transmitted infections (STIs) in the PED. METHODS: This was a prospective, observational cohort study. Three hundred seven adolescents aged 13 to 17 years in an urban PED in Bronx, NY, were enrolled in the study. Subjects provided urine samples for nucleic acid amplification testing for CT and GC and self-completed a confidential questionnaire to assess health care-seeking patterns, high-risk social behaviors, and the presence of abuse, depression, and suicidal ideation. Outcome measures include prevalence of STIs and association of STIs with responses to the confidential questionnaire. RESULTS: Twenty subjects (6.5%) tested positive for an STI. Seventeen (5.5%) were positive for CT, 2 (0.7%) for GC, and 1 (0.3%) for both. Fourteen adolescents (70%) with a positive test were asymptomatic. Logistic regression yielded 4 factors significantly associated with an STI: female sex (odds ratio [OR], 4.02; 95% confidence interval [CI], 1.1-15.2), illicit drug use (OR, 3.3; 95% CI, 1.1-9.3), disclosure of sexual activity (OR, 9.3; 95% CI, 1.1-76.9), and report of a sexual encounter resulting in pregnancy (OR, 3.7; 95% CI, 1.3-10.4). CONCLUSIONS: Sexually transmitted infections were common in asymptomatic adolescents presenting to the PED. We identified 4 risk factors that were significantly associated with STIs. Our findings may facilitate identification of adolescents at highest risk for STIs, help prevent further transmission of infection, and decrease morbidity in this marginalized population.

Polyphosphate kinase deletion increases laboratory productivity in cyanobacteria.

Identification and manipulation of cellular energy regulation mechanisms may be a strategy to increase productivity in photosynthetic organisms. This work tests the hypothesis that polyphosphate synthesis and degradation play a role in energy management by storing or dissipating energy in the form of ATP. A polyphosphate kinase (ppk) knock-out strain unable to synthesize polyphosphate was generated in the cyanobacterium Synechocystis sp. PCC 6803. This mutant strain demonstrated higher ATP levels and faster growth than the wildtype strain in high-carbon conditions and had a growth defect under multiple stress conditions. In a strain that combined ppk deletion with heterologous expression of ethylene-forming enzyme, higher ethylene productivity was observed than in the wildtype background. These results support the role of polyphosphate synthesis and degradation as an energy regulation mechanism and suggest that such mechanisms may be effective targets in biocontainment design.