Best practices to evaluate the impact of biomedical research software-metric collection beyond citations.

MOTIVATION: Software is vital for the advancement of biology and medicine. Impact evaluations of scientific software have primarily emphasized traditional citation metrics of associated papers, despite these metrics inadequately capturing the dynamic picture of impact and despite challenges with improper citation. RESULTS: To understand how software developers evaluate their tools, we conducted a survey of participants in the Informatics Technology for Cancer Research (ITCR) program funded by the National Cancer Institute (NCI). We found that although developers realize the value of more extensive metric collection, they find a lack of funding and time hindering. We also investigated software among this community for how often infrastructure that supports more nontraditional metrics were implemented and how this impacted rates of papers describing usage of the software. We found that infrastructure such as social media presence, more in-depth documentation, the presence of software health metrics, and clear information on how to contact developers seemed to be associated with increased mention rates. Analysing more diverse metrics can enable developers to better understand user engagement, justify continued funding, identify novel use cases, pinpoint improvement areas, and ultimately amplify their software's impact. Challenges are associated, including distorted or misleading metrics, as well as ethical and security concerns. More attention to nuances involved in capturing impact across the spectrum of biomedical software is needed. For funders and developers, we outline guidance based on experience from our community. By considering how we evaluate software, we can empower developers to create tools that more effectively accelerate biological and medical research progress. AVAILABILITY AND IMPLEMENTATION: More information about the analysis, as well as access to data and code is available at https://github.com/fhdsl/ITCR_Metrics_manuscript_website.

Environmental and structural factors associated with bacterial diversity in household dust across the Arizona-Sonora border.

We previously reported that asthma prevalence was higher in the United States (US) compared to Mexico (MX) (25.8% vs. 8.4%). This investigation assessed differences in microbial dust composition in relation to demographic and housing characteristics on both sides of the US-MX Border. Forty homes were recruited in the US and MX. Home visits collected floor dust and documented occupants' demographics, asthma prevalence, housing structure, and use characteristics. US households were more likely to have inhabitants who reported asthma when compared with MX households (30% vs. 5%) and had significantly different flooring types. The percentage of households on paved roads, with flushing toilets, with piped water and with air conditioning was higher in the US, while dust load was higher in MX. Significant differences exist between countries in the microbial composition of the floor dust. Dust from Mexican homes was enriched with Alishewanella, Paracoccus, Rheinheimera genera and Intrasporangiaceae family. A predictive metagenomics analysis identified 68 significantly differentially abundant functional pathways between US and MX. This study documented multiple structural, environmental, and demographic differences between homes in the US and MX that may contribute to significantly different microbial composition of dust observed in these two countries.

Cabozantinib and nivolumab with or without live bacterial supplementation in metastatic renal cell carcinoma: a randomized phase 1 trial.

Supplementation with CBM588, a bifidogenic live bacterial product, has been associated with improved clinical outcomes in persons with metastatic renal cell carcinoma (mRCC) receiving nivolumab and ipilimumab. However, its effect on those receiving tyrosine kinase inhibitor-based combinations is unknown. In this open-label, randomized, investigator-initiated, phase 1 study, 30 participants with locally advanced or mRCC with histological confirmation of clear cell, papillary or sarcomatoid component were randomized in a 2:1 fashion to receive cabozantinib (an inhibitor of vascular endothelial growth factor receptor, MET and AXL) and nivolumab (anti-programmed cell death protein 1) with or without CBM588 as first-line treatment. Metagenomic sequencing was performed on stool samples to characterize their gut microbiome at baseline and 13 weeks into treatment. The primary endpoint was a change in the relative abundance of Bifidobacterium spp.; secondary endpoints included objective response rate (ORR), progression-free survival (PFS) and toxicity profile. The primary endpoint of the study was not met and the addition of CBM588 to cabozantinib and nivolumab did not result in a difference in the relative abundance of Bifidobacterium spp. or alpha diversity (as measured by the Shannon index). However, ORR was significantly higher in participants treated with CBM588 compared to those in the control arm (14 of 19, 74% versus 2 of 10, 20%; P = 0.01). PFS at 6 months was 84% (16 of 19) and 60% (6 of 10) in the experimental and control arms, respectively. No significant difference in toxicity profile was seen between the study arms. Our results provide a preliminary signal of improved clinical activity with CBM588 in treatment-naive participants with mRCC receiving cabozantinib and nivolumab. Further investigation is needed to confirm these findings and better characterize the underlying mechanism driving this effect.ClinicalTrials.gov identifier: NCT05122546.

Pan-Enterovirus Characterization Reveals Cryptic Circulation of Clinically Relevant Subtypes in Arizona Wastewater.

Background: Most seasonally circulating enteroviruses result in asymptomatic or mildly symptomatic infections. In rare cases, however, infection with some subtypes can result in paralysis or death. Of the 300 subtypes known, only poliovirus is reportable, limiting our understanding of the distribution of other enteroviruses that can cause clinical disease. Objective: The overarching objectives of this study were to: 1) describe the distribution of enteroviruses in Arizona during the late summer and fall of 2022, the time of year when they are thought to be most abundant, and 2) demonstrate the utility of viral pan-assay approaches for semi-agnostic discovery that can be followed up by more targeted assays and phylogenomics. Methods: This study utilizes pooled nasal samples collected from school-aged children and long-term care facility residents, and wastewater from multiple locations in Arizona during July-October of 2022. We used PCR to amplify and sequence a region common to all enteroviruses, followed by species-level bioinformatic characterization using the QIIME 2 platform. For Enterovirus-D68 (EV-D68), detection was carried out using RT-qPCR, followed by confirmation using near-complete whole EV-D68 genome sequencing using a newly designed tiled amplicon approach. Results: In the late summer and early fall of 2022, multiple enterovirus species were identified in Arizona wastewater, with Coxsackievirus A6, EV-D68, and Coxsackievirus A19 composing 86% of the characterized reads sequenced. While EV-D68 was not identified in pooled human nasal samples, and the only reported acute flaccid myelitis case in Arizona did not test positive for the virus, an in-depth analysis of EV-D68 in wastewater revealed that the virus was circulating from August through mid-October. A phylogenetic analysis on this relatively limited dataset revealed just a few importations into the state, with a single clade indicating local circulation. Significance: This study further supports the utility of wastewater-based epidemiology to identify potential public health threats. Our further investigations into EV-D68 shows how these data might help inform healthcare diagnoses for children presenting with concerning neurological symptoms.