Best practices to evaluate the impact of biomedical research software-metric collection beyond citations.

MOTIVATION: Software is vital for the advancement of biology and medicine. Impact evaluations of scientific software have primarily emphasized traditional citation metrics of associated papers, despite these metrics inadequately capturing the dynamic picture of impact and despite challenges with improper citation. RESULTS: To understand how software developers evaluate their tools, we conducted a survey of participants in the Informatics Technology for Cancer Research (ITCR) program funded by the National Cancer Institute (NCI). We found that although developers realize the value of more extensive metric collection, they find a lack of funding and time hindering. We also investigated software among this community for how often infrastructure that supports more nontraditional metrics were implemented and how this impacted rates of papers describing usage of the software. We found that infrastructure such as social media presence, more in-depth documentation, the presence of software health metrics, and clear information on how to contact developers seemed to be associated with increased mention rates. Analysing more diverse metrics can enable developers to better understand user engagement, justify continued funding, identify novel use cases, pinpoint improvement areas, and ultimately amplify their software's impact. Challenges are associated, including distorted or misleading metrics, as well as ethical and security concerns. More attention to nuances involved in capturing impact across the spectrum of biomedical software is needed. For funders and developers, we outline guidance based on experience from our community. By considering how we evaluate software, we can empower developers to create tools that more effectively accelerate biological and medical research progress. AVAILABILITY AND IMPLEMENTATION: More information about the analysis, as well as access to data and code is available at https://github.com/fhdsl/ITCR_Metrics_manuscript_website.

Reintroducing mothur: 10 Years Later.

More than 10 years ago, we published the paper describing the mothur software package in Applied and Environmental Microbiology Our goal was to create a comprehensive package that allowed users to analyze amplicon sequence data using the most robust methods available. mothur has helped lead the community through the ongoing sequencing revolution and continues to provide this service to the microbial ecology community. Beyond its success and impact on the field, mothur's development exposed a series of observations that are generally translatable across science. Perhaps the observation that stands out the most is that all science is done in the context of prevailing ideas and available technologies. Although it is easy to criticize choices that were made 10 years ago through a modern lens, if we were to wait for all of the possible limitations to be solved before proceeding, science would stall. Even preceding the development of mothur, it was necessary to address the most important problems and work backwards to other problems that limited access to robust sequence analysis tools. At the same time, we strive to expand mothur's capabilities in a data-driven manner to incorporate new ideas and accommodate changes in data and desires of the research community. It has been edifying to see the benefit that a simple set of tools can bring to so many other researchers.

Dynamics and associations of microbial community types across the human body.

A primary goal of the Human Microbiome Project (HMP) was to provide a reference collection of 16S ribosomal RNA gene sequences collected from sites across the human body that would allow microbiologists to better associate changes in the microbiome with changes in health. The HMP Consortium has reported the structure and function of the human microbiome in 300 healthy adults at 18 body sites from a single time point. Using additional data collected over the course of 12-18 months, we used Dirichlet multinomial mixture models to partition the data into community types for each body site and made three important observations. First, there were strong associations between whether individuals had been breastfed as an infant, their gender, and their level of education with their community types at several body sites. Second, although the specific taxonomic compositions of the oral and gut microbiomes were different, the community types observed at these sites were predictive of each other. Finally, over the course of the sampling period, the community types from sites within the oral cavity were the least stable, whereas those in the vagina and gut were the most stable. Our results demonstrate that even with the considerable intra- and interpersonal variation in the human microbiome, this variation can be partitioned into community types that are predictive of each other and are probably the result of life-history characteristics. Understanding the diversity of community types and the mechanisms that result in an individual having a particular type or changing types, will allow us to use their community types to assess disease risk and to personalize therapies.

Biogeography and environmental conditions shape bacteriophage-bacteria networks across the human microbiome.

Viruses and bacteria are critical components of the human microbiome and play important roles in health and disease. Most previous work has relied on studying bacteria and viruses independently, thereby reducing them to two separate communities. Such approaches are unable to capture how these microbial communities interact, such as through processes that maintain community robustness or allow phage-host populations to co-evolve. We implemented a network-based analytical approach to describe phage-bacteria network diversity throughout the human body. We built these community networks using a machine learning algorithm to predict which phages could infect which bacteria in a given microbiome. Our algorithm was applied to paired viral and bacterial metagenomic sequence sets from three previously published human cohorts. We organized the predicted interactions into networks that allowed us to evaluate phage-bacteria connectedness across the human body. We observed evidence that gut and skin network structures were person-specific and not conserved among cohabitating family members. High-fat diets appeared to be associated with less connected networks. Network structure differed between skin sites, with those exposed to the external environment being less connected and likely more susceptible to network degradation by microbial extinction events. This study quantified and contrasted the diversity of virome-microbiome networks across the human body and illustrated how environmental factors may influence phage-bacteria interactive dynamics. This work provides a baseline for future studies to better understand system perturbations, such as disease states, through ecological networks.

Mathematical modeling of primary succession of murine intestinal microbiota.

Understanding the nature of interpopulation interactions in host-associated microbial communities is critical to understanding gut colonization, responses to perturbations, and transitions between health and disease. Characterizing these interactions is complicated by the complexity of these communities and the observation that even if populations can be cultured, their in vitro and in vivo phenotypes differ significantly. Dynamic models are the cornerstone of computational systems biology and a key objective of computational systems biologists is the reconstruction of biological networks (i.e., network inference) from high-throughput data. When such computational models reflect biology, they provide an opportunity to generate testable hypotheses as well as to perform experiments that are impractical or not feasible in vivo or in vitro. We modeled time-series data for murine microbial communities using statistical approaches and systems of ordinary differential equations. To obtain the dense time-series data, we sequenced the 16S ribosomal RNA (rRNA) gene from DNA isolated from the fecal material of germfree mice colonized with cecal contents of conventionally raised animals. The modeling results suggested a lack of mutualistic interactions within the community. Among the members of the Bacteroidetes, there was evidence for closely related pairs of populations to exhibit parasitic interactions. Among the Firmicutes, the interactions were all competitive. These results suggest future animal and in silico experiments. Our modeling approach can be applied to other systems to provide a greater understanding of the dynamics of communities associated with health and disease.

Multicenter Comparison of Lung and Oral Microbiomes of HIV-infected and HIV-uninfected Individuals.

RATIONALE: Improved understanding of the lung microbiome in HIV-infected individuals could lead to better strategies for diagnosis, therapy, and prophylaxis of HIV-associated pneumonias. Differences in the oral and lung microbiomes in HIV-infected and HIV-uninfected individuals are not well defined. Whether highly active antiretroviral therapy influences these microbiomes is unclear. OBJECTIVES: We determined whether oral and lung microbiomes differed in clinically healthy groups of HIV-infected and HIV-uninfected subjects. METHODS: Participating sites in the Lung HIV Microbiome Project contributed bacterial 16S rRNA sequencing data from oral washes and bronchoalveolar lavages (BALs) obtained from HIV-uninfected individuals (n = 86), HIV-infected individuals who were treatment naive (n = 18), and HIV-infected individuals receiving antiretroviral therapy (n = 38). MEASUREMENTS AND MAIN RESULTS: Microbial populations differed in the oral washes among the subject groups (Streptococcus, Actinomyces, Rothia, and Atopobium), but there were no individual taxa that differed among the BALs. Comparison of oral washes and BALs demonstrated similar patterns from HIV-uninfected individuals and HIV-infected individuals receiving antiretroviral therapy, with multiple taxa differing in abundance. The pattern observed from HIV-infected individuals who were treatment naive differed from the other two groups, with differences limited to Veillonella, Rothia, and Granulicatella. CD4 cell counts did not influence the oral or BAL microbiome in these relatively healthy, HIV-infected subjects. CONCLUSIONS: The overall similarity of the microbiomes in participants with and without HIV infection was unexpected, because HIV-infected individuals with relatively preserved CD4 cell counts are at higher risk for lower respiratory tract infections, indicating impaired local immune function.

Dynamics and establishment of Clostridium difficile infection in the murine gastrointestinal tract.

Clostridium difficile infection (CDI) following antibiotic therapy is a major public health threat. While antibiotic disruption of the indigenous microbiota underlies the majority of cases of CDI, the early dynamics of infection in the disturbed intestinal ecosystem are poorly characterized. This study defines the dynamics of infection with C. difficile strain VPI 10463 throughout the gastrointestinal (GI) tract using a murine model of infection. After inducing susceptibility to C. difficile colonization via antibiotic administration, we followed the dynamics of spore germination, colonization, sporulation, toxin activity, and disease progression throughout the GI tract. C. difficile spores were able to germinate within 6 h postchallenge, resulting in the establishment of vegetative bacteria in the distal GI tract. Spores and cytotoxin activity were detected by 24 h postchallenge, and histopathologic colitis developed by 30 h. Within 36 h, all infected mice succumbed to infection. We correlated the establishment of infection with changes in the microbiota and bile acid profile of the small and large intestines. Antibiotic administration resulted in significant changes to the microbiota in the small and large intestines, as well as a significant shift in the abundance of primary and secondary bile acids. Ex vivo analysis suggested the small intestine as the site of spore germination. This study provides an integrated understanding of the timing and location of the events surrounding C. difficile colonization and identifies potential targets for the development of new therapeutic strategies.

Intra- and interindividual variations mask interspecies variation in the microbiota of sympatric peromyscus populations.

Using populations of two sympatric Peromyscus species, we characterized the importance of the host species, physiology, environment, diet, and other factors in shaping the structure and dynamics of their gut microbiota. We performed a capture-mark-release experiment in which we obtained 16S rRNA gene sequence data from 49 animals at multiple time points. In addition, we performed 18S rRNA gene sequencing of the same samples to characterize the diet of each individual. Our analysis could not distinguish between the two species of Peromyscus on the basis of the structures of their microbiotas. However, we did observe a set of bacterial populations that were found in every animal. Most notable were abundant representatives of the genera Lactobacillus and Helicobacter. When we combined the 16S and 18S rRNA gene sequence analyses, we were unable to distinguish the communities on the basis of the animal's diet. Furthermore, there were no discernible differences in the structure of the gut communities based on the capture site or their developmental or physiological status. Finally, in contrast to humans, where each individual has a unique microbiota when sampled over years, among the animals captured in this study, the uniqueness of each microbiota was lost within a week of the original sampling. Wild populations provide an opportunity to study host-microbiota interactions as they originally evolved, and the ability to perform natural experiments will facilitate a greater understanding of the factors that shape the structure and function of the gut microbiota.

Decade-long bacterial community dynamics in cystic fibrosis airways.

The structure and dynamics of bacterial communities in the airways of persons with cystic fibrosis (CF) remain largely unknown. We characterized the bacterial communities in 126 sputum samples representing serial collections spanning 8-9 y from six age-matched male CF patients. Sputum DNA was analyzed by bar-coded pyrosequencing of the V3-V5 hypervariable region of the 16S rRNA gene, defining 662 operational taxonomic units (OTUs) from >633,000 sequences. Bacterial community diversity decreased significantly over time in patients with typically progressive lung disease but remained relatively stable in patients with a mild lung disease phenotype. Antibiotic use, rather than patient age or lung function, was the primary driver of decreasing diversity. Interpatient variability in community structure exceeded intrapatient variability in serial samples. Antibiotic treatment was associated with pronounced shifts in community structure, but communities showed both short- and long-term resilience after antibiotic perturbation. There was a positive correlation between OTU occurrence and relative abundance, with a small number of persistent OTUs accounting for the greatest abundance. Significant changes in community structure, diversity, or total bacterial density at the time of pulmonary exacerbation were not observed. Despite decreasing community diversity in patients with progressive disease, total bacterial density remained relatively stable over time. These findings show the critical relationship between airway bacterial community structure, disease stage, and clinical state at the time of sample collection. These features are the key parameters with which to assess the complex ecology of the CF airway.

Low prevalence of Clostridium septicum fecal carriage in an adult population.

Clostridium septicum is an uncommon cause of severe infection. Real-time PCR against the C. septicum-specific alpha-toxin gene (csa) was used to estimate the prevalence of this microbe in human stool from 161 asymptomatic community-dwelling adults and 192 hospitalized patients with diarrhea. All samples were negative, suggesting a low prevalence.