Retrieval augmented scientific claim verification.

Objective: To automate scientific claim verification using PubMed abstracts. Materials and Methods: We developed CliVER, an end-to-end scientific Claim VERification system that leverages retrieval-augmented techniques to automatically retrieve relevant clinical trial abstracts, extract pertinent sentences, and use the PICO framework to support or refute a scientific claim. We also created an ensemble of three state-of-the-art deep learning models to classify rationale of support, refute, and neutral. We then constructed CoVERt, a new COVID VERification dataset comprising 15 PICO-encoded drug claims accompanied by 96 manually selected and labeled clinical trial abstracts that either support or refute each claim. We used CoVERt and SciFact (a public scientific claim verification dataset) to assess CliVER's performance in predicting labels. Finally, we compared CliVER to clinicians in the verification of 19 claims from 6 disease domains, using 189 648 PubMed abstracts extracted from January 2010 to October 2021. Results: In the evaluation of label prediction accuracy on CoVERt, CliVER achieved a notable F1 score of 0.92, highlighting the efficacy of the retrieval-augmented models. The ensemble model outperforms each individual state-of-the-art model by an absolute increase from 3% to 11% in the F1 score. Moreover, when compared with four clinicians, CliVER achieved a precision of 79.0% for abstract retrieval, 67.4% for sentence selection, and 63.2% for label prediction, respectively. Conclusion: CliVER demonstrates its early potential to automate scientific claim verification using retrieval-augmented strategies to harness the wealth of clinical trial abstracts in PubMed. Future studies are warranted to further test its clinical utility.

Polygenic risk alters the penetrance of monogenic kidney disease.

Chronic kidney disease (CKD) is determined by an interplay of monogenic, polygenic, and environmental risks. Autosomal dominant polycystic kidney disease (ADPKD) and COL4A-associated nephropathy (COL4A-AN) represent the most common forms of monogenic kidney diseases. These disorders have incomplete penetrance and variable expressivity, and we hypothesize that polygenic factors explain some of this variability. By combining SNP array, exome/genome sequence, and electronic health record data from the UK Biobank and All-of-Us cohorts, we demonstrate that the genome-wide polygenic score (GPS) significantly predicts CKD among ADPKD monogenic variant carriers. Compared to the middle tertile of the GPS for noncarriers, ADPKD variant carriers in the top tertile have a 54-fold increased risk of CKD, while ADPKD variant carriers in the bottom tertile have only a 3-fold increased risk of CKD. Similarly, the GPS significantly predicts CKD in COL4A-AN carriers. The carriers in the top tertile of the GPS have a 2.5-fold higher risk of CKD, while the risk for carriers in the bottom tertile is not different from the average population risk. These results suggest that accounting for polygenic risk improves risk stratification in monogenic kidney disease.

Evaluating large language models on medical evidence summarization.

Recent advances in large language models (LLMs) have demonstrated remarkable successes in zero- and few-shot performance on various downstream tasks, paving the way for applications in high-stakes domains. In this study, we systematically examine the capabilities and limitations of LLMs, specifically GPT-3.5 and ChatGPT, in performing zero-shot medical evidence summarization across six clinical domains. We conduct both automatic and human evaluations, covering several dimensions of summary quality. Our study demonstrates that automatic metrics often do not strongly correlate with the quality of summaries. Furthermore, informed by our human evaluations, we define a terminology of error types for medical evidence summarization. Our findings reveal that LLMs could be susceptible to generating factually inconsistent summaries and making overly convincing or uncertain statements, leading to potential harm due to misinformation. Moreover, we find that models struggle to identify the salient information and are more error-prone when summarizing over longer textual contexts.

Implementation and Feasibility of Clinical Genome Sequencing Embedded Into the Outpatient Nephrology Care for Patients With Proteinuric Kidney Disease.

Introduction: The diagnosis and management of proteinuric kidney diseases such as focal segmental glomerulosclerosis (FSGS) are challenging. Genetics holds the promise to improve clinical decision making for these diseases; however, it is often performed too late to enable timely clinical action and it is not implemented within routine outpatient nephrology visits. Methods: We sought to test the implementation and feasibility of clinical rapid genome sequencing (GS) in guiding decision making in patients with proteinuric kidney disease in real-time and embedded in the outpatient nephrology setting. Results: We enrolled 10 children or young adults with biopsy-proven FSGS (9 cases) or minimal change disease (1 case). The mean age at enrollment was 16.2 years (range 2-30). The workflow did not require referral to external genetics clinics but was conducted entirely during the nephrology standard-of-care appointments. The total turn-around-time from enrollment to return-of-results and clinical decision averaged 21.8 days (12.4 for GS), which is well within a time frame that allows clinically relevant treatment decisions. A monogenic or APOL1-related form of kidney disease was diagnosed in 5 of 10 patients. The genetic findings resulted in a rectified diagnosis in 6 patients. Both positive and negative GS findings determined a change in pharmacological treatment. In 3 patients, the results were instrumental for transplant evaluation, donor selection, and the immunosuppressive treatment. All patients and families received genetic counseling. Conclusion: Clinical GS is feasible and can be implemented in real-time in the outpatient care to help guiding clinical management. Additional studies are needed to confirm the cost-effectiveness and broader utility of clinical GS across the phenotypic and demographic spectrum of kidney diseases.

Evaluating Large Language Models on Medical Evidence Summarization.

Recent advances in large language models (LLMs) have demonstrated remarkable successes in zero- and few-shot performance on various downstream tasks, paving the way for applications in high-stakes domains. In this study, we systematically examine the capabilities and limitations of LLMs, specifically GPT-3.5 and ChatGPT, in performing zero-shot medical evidence summarization across six clinical domains. We conduct both automatic and human evaluations, covering several dimensions of summary quality. Our study has demonstrated that automatic metrics often do not strongly correlate with the quality of summaries. Furthermore, informed by our human evaluations, we define a terminology of error types for medical evidence summarization. Our findings reveal that LLMs could be susceptible to generating factually inconsistent summaries and making overly convincing or uncertain statements, leading to potential harm due to misinformation. Moreover, we find that models struggle to identify the salient information and are more error-prone when summarizing over longer textual contexts.

Polygenic risk affects the penetrance of monogenic kidney disease.

Background: Chronic kidney disease (CKD) is a genetically complex disease determined by an interplay of monogenic, polygenic, and environmental risks. Most forms of monogenic kidney diseases have incomplete penetrance and variable expressivity. It is presently unknown if some of the variability in penetrance can be attributed to polygenic factors. Methods: Using the UK Biobank (N=469,835 participants) and the All of Us (N=98,622 participants) datasets, we examined two most common forms of monogenic kidney disorders, autosomal dominant polycystic kidney disease (ADPKD) caused by deleterious variants in the PKD1 or PKD2 genes, and COL4A-associated nephropathy (COL4A-AN caused by deleterious variants in COL4A3, COL4A4, or COL4A5 genes). We used the eMERGE-III electronic CKD phenotype to define cases (estimated glomerular filtration rate (eGFR) <60 mL/min/1.73m2 or kidney failure) and controls (eGFR >90 mL/min/1.73m2 in the absence of kidney disease diagnoses). The effects of the genome-wide polygenic score (GPS) for CKD were tested in monogenic variant carriers and non-carriers using logistic regression controlling for age, sex, diabetes, and genetic ancestry. Results: As expected, the carriers of known pathogenic and rare predicted loss-of-function variants in PKD1 or PKD2 had a high risk of CKD (ORmeta=17.1, 95% CI: 11.1-26.4, P=1.8E-37). The GPS was comparably predictive of CKD in both ADPKD variant carriers (ORmeta=2.28 per SD, 95%CI: 1.55-3.37, P=2.6E-05) and non-carriers (ORmeta=1.72 per SD, 95% CI=1.69-1.76, P< E-300) independent of age, sex, diabetes, and genetic ancestry. Compared to the middle tertile of the GPS distribution for non-carriers, ADPKD variant carriers in the top tertile had a 54-fold increased risk of CKD, while ADPKD variant carriers in the bottom tertile had only a 3-fold increased risk of CKD. Similarly, the GPS was predictive of CKD in both COL4-AN variant carriers (ORmeta=1.78, 95% CI=1.22-2.58, P=2.38E-03) and non-carriers (ORmeta=1.70, 95%CI: 1.68-1.73 P

Rare Single Nucleotide and Copy Number Variants and the Etiology of Congenital Obstructive Uropathy: Implications for Genetic Diagnosis.

SIGNIFICANCE STATEMENT: Congenital obstructive uropathy (COU) is a prevalent human developmental defect with highly heterogeneous clinical presentations and outcomes. Genetics may refine diagnosis, prognosis, and treatment, but the genomic architecture of COU is largely unknown. Comprehensive genomic screening study of 733 cases with three distinct COU subphenotypes revealed disease etiology in 10.0% of them. We detected no significant differences in the overall diagnostic yield among COU subphenotypes, with characteristic variable expressivity of several mutant genes. Our findings therefore may legitimize a genetic first diagnostic approach for COU, especially when burdening clinical and imaging characterization is not complete or available. BACKGROUND: Congenital obstructive uropathy (COU) is a common cause of developmental defects of the urinary tract, with heterogeneous clinical presentation and outcome. Genetic analysis has the potential to elucidate the underlying diagnosis and help risk stratification. METHODS: We performed a comprehensive genomic screen of 733 independent COU cases, which consisted of individuals with ureteropelvic junction obstruction ( n =321), ureterovesical junction obstruction/congenital megaureter ( n =178), and COU not otherwise specified (COU-NOS; n =234). RESULTS: We identified pathogenic single nucleotide variants (SNVs) in 53 (7.2%) cases and genomic disorders (GDs) in 23 (3.1%) cases. We detected no significant differences in the overall diagnostic yield between COU sub-phenotypes, and pathogenic SNVs in several genes were associated to any of the three categories. Hence, although COU may appear phenotypically heterogeneous, COU phenotypes are likely to share common molecular bases. On the other hand, mutations in TNXB were more often identified in COU-NOS cases, demonstrating the diagnostic challenge in discriminating COU from hydronephrosis secondary to vesicoureteral reflux, particularly when diagnostic imaging is incomplete. Pathogenic SNVs in only six genes were found in more than one individual, supporting high genetic heterogeneity. Finally, convergence between data on SNVs and GDs suggest MYH11 as a dosage-sensitive gene possibly correlating with severity of COU. CONCLUSIONS: We established a genomic diagnosis in 10.0% of COU individuals. The findings underscore the urgent need to identify novel genetic susceptibility factors to COU to better define the natural history of the remaining 90% of cases without a molecular diagnosis.

Racial and Ethnic Disparities in Acute Care Utilization Among Patients With Glomerular Disease.

RATIONALE & OBJECTIVE: The effects of race, ethnicity, socioeconomic status (SES), and disease severity on acute care utilization in patients with glomerular disease are unknown. STUDY DESIGN: Prospective cohort study. SETTING & PARTICIPANTS: 1,456 adults and 768 children with biopsy-proven glomerular disease enrolled in the Cure Glomerulonephropathy (CureGN) cohort. EXPOSURE: Race and ethnicity as a participant-reported social factor. OUTCOME: Acute care utilization defined as hospitalizations or emergency department visits. ANALYTICAL APPROACH: Multivariable recurrent event proportional rate models were used to estimate associations between race and ethnicity and acute care utilization. RESULTS: Black or Hispanic participants had lower SES and more severe glomerular disease than White or Asian participants. Acute care utilization rates were 45.6, 29.5, 25.8, and 19.2 per 100 person-years in Black, Hispanic, White, and Asian adults, respectively, and 55.8, 42.5, 40.8, and 13.0, respectively, for children. Compared with the White race (reference group), Black race was significantly associated with acute care utilization in adults (rate ratio [RR], 1.76 [95% CI, 1.37-2.27]), although this finding was attenuated after multivariable adjustment (RR, 1.31 [95% CI, 1.03-1.68]). Black race was not significantly associated with acute care utilization in children; Asian race was significantly associated with lower acute care utilization in children (RR, 0.32 [95% CI 0.14-0.70]); no significant associations between Hispanic ethnicity and acute care utilization were identified. LIMITATIONS: We used proxies for SES and lacked direct information on income, household unemployment, or disability. CONCLUSIONS: Significant differences in acute care utilization rates were observed across racial and ethnic groups in persons with prevalent glomerular disease, although many of these difference were explained by differences in SES and disease severity. Measures to combat socioeconomic disadvantage in Black patients and to more effectively prevent and treat glomerular disease are needed to reduce disparities in acute care utilization, improve patient wellbeing, and reduce health care costs.

OARD: Open annotations for rare diseases and their phenotypes based on real-world data.

Diagnosis for rare genetic diseases often relies on phenotype-driven methods, which hinge on the accuracy and completeness of the rare disease phenotypes in the underlying annotation knowledgebase. Existing knowledgebases are often manually curated with additional annotations found in published case reports. Despite their potential, real-world data such as electronic health records (EHRs) have not been fully exploited to derive rare disease annotations. Here, we present open annotation for rare diseases (OARD), a real-world-data-derived resource with annotation for rare-disease-related phenotypes. This resource is derived from the EHRs of two academic health institutions containing more than 10 million individuals spanning wide age ranges and different disease subgroups. By leveraging ontology mapping and advanced natural-language-processing (NLP) methods, OARD automatically and efficiently extracts concepts for both rare diseases and their phenotypic traits from billing codes and lab tests as well as over 100 million clinical narratives. The rare disease prevalence derived by OARD is highly correlated with those annotated in the original rare disease knowledgebase. By performing association analysis, we identified more than 1 million novel disease-phenotype association pairs that were previously missed by human annotation, and >60% were confirmed true associations via manual review of a list of sampled pairs. Compared to the manual curated annotation, OARD is 100% data driven and its pipeline can be shared across different institutions. By supporting privacy-preserving sharing of aggregated summary statistics, such as term frequencies and disease-phenotype associations, it fills an important gap to facilitate data-driven research in the rare disease community.

Impact of education on APOL1 testing attitudes among prospective living kidney donors.

It is unknown how providing prospective living donors with information about APOL1, including the benefits and drawbacks of testing, influences their desire for testing. In this study, we surveyed 102 participants with self-reported African ancestry and positive family history of kidney disease, recruited from our nephrology waiting room. We assessed views on APOL1 testing before and after presentation of a set of potential benefits and drawbacks of testing and quantified the self-reported level of influence individual benefits and drawbacks had on participants' desire for testing in the proposed context of living donation. The majority of participants (92%) were aware of organ donation and more than half (56%) had considered living donation. And though we found no significant change in response following presentation of the potential benefits and the drawbacks of APOL1 testing by study end significance, across all participants, "becoming aware of the potential risk of kidney disease among your immediate family" was the benefit with the highest mean influence (3.3±1.4), while the drawback with the highest mean influence (2.9±1.5) was "some transplant centers may not allow you to donate to a loved one". This study provides insights into the priorities of prospective living donors and suggests concern for how the information affects family members may strongly influence desires for testing. It also highlights the need for greater community engagement to gain a deeper understanding of the priorities that influence decision making on APOL1 testing.

Racial-ethnic differences in health-related quality of life among adults and children with glomerular disease.

INTRODUCTION: Disparities in health-related quality of life (HRQOL) have been inadequately studied in patients with glomerular disease. The aim of this study was to identify relationships between race/ethnicity, socioeconomic status, disease severity, and HRQOL in an ethnically and racially diverse cohort of patients with glomerular disease. METHODS: Cure Glomerulonephropathy (CureGN) is a multinational cohort study of patients with biopsy-proven glomerular disease. Associations between race/ethnicity and HRQOL were determined by the following: 1. Missed school or work due to kidney disease; 2. Responses to Patient Reported Outcomes Measurement Information System (PROMIS) questionnaires. We adjusted for demographics, socioeconomic status, and disease characteristics using multivariable logistic and linear regression. RESULTS: Black and Hispanic participants had worse socioeconomic status and more severe glomerular disease than White or Asian participants. Black adults missed work or school most frequently due to kidney disease (30% versus 16-23% in the other three groups, p=0.04), and had the worst self-reported global physical health (median score 44.1 versus 48.0-48.2, p<0.001) and fatigue (53.8 versus 48.5-51.1, p=0.002), compared to other racial/ethnic groups. However, these findings were not statistically significant with adjustment for socioeconomic status and disease severity, both of which were strongly associated with HRQOL in adults. Among children, disease severity but not race/ethnicity or socioeconomic status were associated with HRQOL. CONCLUSIONS: Among patients with glomerular disease enrolled in CureGN, the worse HRQOL reported by Black adults was attributable to lower socioeconomic status and more severe glomerular disease. No racial/ethnic differences in HRQOL were observed in children.

Genomic considerations for FHIR®; eMERGE implementation lessons.

Structured representation of clinical genetic results is necessary for advancing precision medicine. The Electronic Medical Records and Genomics (eMERGE) Network's Phase III program initially used a commercially developed XML message format for standardized and structured representation of genetic results for electronic health record (EHR) integration. In a desire to move towards a standard representation, the network created a new standardized format based upon Health Level Seven Fast Healthcare Interoperability Resources (HL7® FHIR®), to represent clinical genomics results. These new standards improve the utility of HL7® FHIR® as an international healthcare interoperability standard for management of genetic data from patients. This work advances the establishment of standards that are being designed for broad adoption in the current health information technology landscape.

An electronic health record (EHR) log analysis shows limited clinician engagement with unsolicited genetic test results.

How clinicians utilize medically actionable genomic information, displayed in the electronic health record (EHR), in medical decision-making remains unknown. Participating sites of the Electronic Medical Records and Genomics (eMERGE) Network have invested resources into EHR integration efforts to enable the display of genetic testing data across heterogeneous EHR systems. To assess clinicians' engagement with unsolicited EHR-integrated genetic test results of eMERGE participants within a large tertiary care academic medical center, we analyzed automatically generated EHR access log data. We found that clinicians viewed only 1% of all the eMERGE genetic test results integrated in the EHR. Using a cluster analysis, we also identified different user traits associated with varying degrees of engagement with the EHR-integrated genomic data. These data contribute important empirical knowledge about clinicians limited and brief engagements with unsolicited EHR-integrated genetic test results of eMERGE participants. Appreciation for user-specific roles provide additional context for why certain users were more or less engaged with the unsolicited results. This study highlights opportunities to use EHR log data as a performance metric to more precisely inform ongoing EHR-integration efforts and decisions about the allocation of informatics resources in genomic research.

Evaluation of the MC4R gene across eMERGE network identifies many unreported obesity-associated variants.

BACKGROUND/OBJECTIVES: Melanocortin-4 receptor (MC4R) plays an essential role in food intake and energy homeostasis. More than 170 MC4R variants have been described over the past two decades, with conflicting reports regarding the prevalence and phenotypic effects of these variants in diverse cohorts. To determine the frequency of MC4R variants in large cohort of different ancestries, we evaluated the MC4R coding region for 20,537 eMERGE participants with sequencing data plus additional 77,454 independent individuals with genome-wide genotyping data at this locus. SUBJECTS/METHODS: The sequencing data were obtained from the eMERGE phase III study, in which multisample variant call format calls have been generated, curated, and annotated. In addition to penetrance estimation using body mass index (BMI) as a binary outcome, GWAS and PheWAS were performed using median BMI in linear regression analyses. All results were adjusted for principal components, age, sex, and sites of genotyping. RESULTS: Targeted sequencing data of MC4R revealed 125 coding variants in 1839 eMERGE participants including 30 unreported coding variants that were predicted to be functionally damaging. Highly penetrant unreported variants included (L325I, E308K, D298N, S270F, F261L, T248A, D111V, and Y80F) in which seven participants had obesity class III defined as BMI ≥ 40 kg/m2. In GWAS analysis, in addition to known risk haplotype upstream of MC4R (best variant rs6567160 (P = 5.36 × 10-25, Beta = 0.37), a novel rare haplotype was detected which was protective against obesity and encompassed the V103I variant with known gain-of-function properties (P = 6.23 × 10-08, Beta = -0.62). PheWAS analyses extended this protective effect of V103I to type 2 diabetes, diabetic nephropathy, and chronic renal failure independent of BMI. CONCLUSIONS: MC4R screening in a large eMERGE cohort confirmed many previous findings, extend the MC4R pleotropic effects, and discovered additional MC4R rare alleles that probably contribute to obesity.

Pilot Study of Return of Genetic Results to Patients in Adult Nephrology.

BACKGROUND AND OBJECTIVES: Actionable genetic findings have implications for care of patients with kidney disease, and genetic testing is an emerging tool in nephrology practice. However, there are scarce data regarding best practices for return of results and clinical application of actionable genetic findings for kidney patients. DESIGN, SETTING, PARTICIPANTS, & MEASUREMENTS: We developed a return of results workflow in collaborations with clinicians for the retrospective recontact of adult nephrology patients who had been recruited into a biobank research study for exome sequencing and were identified to have medically actionable genetic findings. RESULTS: Using this workflow, we attempted to recontact a diverse pilot cohort of 104 nephrology research participants with actionable genetic findings, encompassing 34 different monogenic etiologies of nephropathy and five single-gene disorders recommended by the American College of Medical Genetics and Genomics for return as medically actionable secondary findings. We successfully recontacted 64 (62%) participants and returned results to 41 (39%) individuals. In each case, the genetic diagnosis had meaningful implications for the patients' nephrology care. Through implementation efforts and qualitative interviews with providers, we identified over 20 key challenges associated with returning results to study participants, and found that physician knowledge gaps in genomics was a recurrent theme. We iteratively addressed these challenges to yield an optimized workflow, which included standardized consultation notes with tailored management recommendations, monthly educational conferences on core topics in genomics, and a curated list of expert clinicians for patients requiring extranephrologic referrals. CONCLUSIONS: Developing the infrastructure to support return of genetic results in nephrology was resource-intensive, but presented potential opportunities for improving patient care. PODCAST: This article contains a podcast at https://www.asn-online.org/media/podcast/CJASN/2020_04_16_12481019.mp3.

Towards precision nephrology: the opportunities and challenges of genomic medicine.

The expansion of genomic medicine is furthering our understanding of many human diseases. This is well illustrated in the field of nephrology, through the characterization, discovery, and growing insight into various renal diseases through use of Next Generation Sequencing (NGS) technologies. This review will provide an overview of the diagnostic opportunities of using genetic testing in the clinical setting by describing notable discoveries regarding inherited forms of renal disease that have advanced the field and by highlighting some of the potential benefits of establishing a molecular diagnosis in a clinical practice. In addition, it will discuss some of the challenges associated with the expansion of genetic testing into the clinical setting, including clinical variant interpretation and return of genetic results.