Perceptions of Data Set Experts on Important Characteristics of Health Data Sets Ready for Machine Learning: A Qualitative Study.

Importance: The lack of data quality frameworks to guide the development of artificial intelligence (AI)-ready data sets limits their usefulness for machine learning (ML) research in health care and hinders the diagnostic excellence of developed clinical AI applications for patient care. Objective: To discern what constitutes high-quality and useful data sets for health and biomedical ML research purposes according to subject matter experts. Design, Setting, and Participants: This qualitative study interviewed data set experts, particularly those who are creators and ML researchers. Semistructured interviews were conducted in English and remotely through a secure video conferencing platform between August 23, 2022, and January 5, 2023. A total of 93 experts were invited to participate. Twenty experts were enrolled and interviewed. Using purposive sampling, experts were affiliated with a diverse representation of 16 health data sets/databases across organizational sectors. Content analysis was used to evaluate survey information and thematic analysis was used to analyze interview data. Main Outcomes and Measures: Data set experts' perceptions on what makes data sets AI ready. Results: Participants included 20 data set experts (11 [55%] men; mean [SD] age, 42 [11] years), of whom all were health data set creators, and 18 of the 20 were also ML researchers. Themes (3 main and 11 subthemes) were identified and integrated into an AI-readiness framework to show their association within the health data ecosystem. Participants partially determined the AI readiness of data sets using priority appraisal elements of accuracy, completeness, consistency, and fitness. Ethical acquisition and societal impact emerged as appraisal considerations in that participant samples have not been described to date in prior data quality frameworks. Factors that drive creation of high-quality health data sets and mitigate risks associated with data reuse in ML research were also relevant to AI readiness. The state of data availability, data quality standards, documentation, team science, and incentivization were associated with elements of AI readiness and the overall perception of data set usefulness. Conclusions and Relevance: In this qualitative study of data set experts, participants contributed to the development of a grounded framework for AI data set quality. Data set AI readiness required the concerted appraisal of many elements and the balancing of transparency and ethical reflection against pragmatic constraints. The movement toward more reliable, relevant, and ethical AI and ML applications for patient care will inevitably require strategic updates to data set creation practices.

Organizational Factors in Clinical Data Sharing for Artificial Intelligence in Health Care.

Importance: Limited sharing of data sets that accurately represent disease and patient diversity limits the generalizability of artificial intelligence (AI) algorithms in health care. Objective: To explore the factors associated with organizational motivation to share health data for AI development. Design, Setting, and Participants: This qualitative study investigated organizational readiness for sharing health data across the academic, governmental, nonprofit, and private sectors. Using a multiple case studies approach, 27 semistructured interviews were conducted with leaders in data-sharing roles from August 29, 2022, to January 9, 2023. The interviews were conducted in the English language using a video conferencing platform. Using a purposive and nonprobabilistic sampling strategy, 78 individuals across 52 unique organizations were identified. Of these, 35 participants were enrolled. Participant recruitment concluded after 27 interviews, as theoretical saturation was reached and no additional themes emerged. Main Outcome and Measure: Concepts defining organizational readiness for data sharing and the association between data-sharing factors and organizational behavior were mapped through iterative qualitative analysis to establish a framework defining organizational readiness for sharing clinical data for AI development. Results: Interviews included 27 leaders from 18 organizations (academia: 10, government: 7, nonprofit: 8, and private: 2). Organizational readiness for data sharing centered around 2 main constructs: motivation and capabilities. Motivation related to the alignment of an organization's values with data-sharing priorities and was associated with its engagement in data-sharing efforts. However, organizational motivation could be modulated by extrinsic incentives for financial or reputational gains. Organizational capabilities comprised infrastructure, people, expertise, and access to data. Cross-sector collaboration was a key strategy to mitigate barriers to access health data. Conclusions and Relevance: This qualitative study identified sector-specific factors that may affect the data-sharing behaviors of health organizations. External incentives may bolster cross-sector collaborations by helping overcome barriers to accessing health data for AI development. The findings suggest that tailored incentives may boost organizational motivation and facilitate sustainable flow of health data for AI development.

Email-Based Recruitment Into the Health eHeart Study: Cohort Analysis of Invited Eligible Patients.

BACKGROUND: Web- or app-based digital health studies allow for more efficient collection of health data for research. However, remote recruitment into digital health studies can enroll nonrepresentative study samples, hindering the robustness and generalizability of findings. Through the comprehensive evaluation of an email-based campaign on recruitment into the Health eHeart Study, we aim to uncover key sociodemographic and clinical factors that contribute to enrollment. OBJECTIVE: This study sought to understand the factors related to participation, specifically regarding enrollment, in the Health eHeart Study as a result of a large-scale remote email recruitment campaign. METHODS: We conducted a cohort analysis on all invited University of California, San Francisco (UCSF) patients to identify sociodemographic and clinical predictors of enrollment into the Health eHeart Study. The primary outcome was enrollment, defined by account registration and consent into the Health eHeart Study. The email recruitment campaign was carried out from August 2015 to February 2016, with electronic health record data extracted between September 2019 and December 2019. RESULTS: The email recruitment campaign delivered at least 1 email invitation to 93.5% (193,606/206,983) of all invited patients and yielded a 3.6% (7012/193,606) registration rate among contacted patients and an 84.1% (5899/7012) consent rate among registered patients. Adjusted multivariate logistic regression models analyzed independent sociodemographic and clinical predictors of (1) registration among contacted participants and (2) consent among registered participants. Odds of registration were higher among patients who are older, women, non-Hispanic White, active patients with commercial insurance or Medicare, with a higher comorbidity burden, with congestive heart failure, and randomized to receive up to 2 recruitment emails. The odds of registration were lower among those with medical conditions such as dementia, chronic pulmonary disease, moderate or severe liver disease, paraplegia or hemiplegia, renal disease, or cancer. Odds of subsequent consent after initial registration were different, with an inverse trend of being lower among patients who are older and women. The odds of consent were also lower among those with peripheral vascular disease. However, the odds of consent remained higher among patients who were non-Hispanic White and those with commercial insurance. CONCLUSIONS: This study provides important insights into the potential returns on participant enrollment when digital health study teams invest resources in using email for recruitment. The findings show that participant enrollment was driven more strongly by sociodemographic factors than clinical factors. Overall, email is an extremely efficient means of recruiting participants from a large list into the Health eHeart Study. Despite some improvements in representation, the formulation of truly diverse studies will require additional resources and strategies to overcome persistent participation barriers.

Usability, inclusivity, and content evaluation of COVID-19 contact tracing apps in the United States.

We evaluated the usability of mobile COVID-19 contact tracing apps, especially for individuals with barriers to communication and limited digital literacy skills. We searched the Apple App Store, Google Play, peer-reviewed literature, and lay press to find contact tracing apps in the United States. We evaluated apps with a framework focused on user characteristics and user interface. Of the final 26 apps, 77% were on both iPhone and Android. 69% exceeded 9th grade readability, and 65% were available only in English. Only 12% had inclusive illustrations (different genders, skin tones, physical abilities). 92% alerted users of an exposure, 42% linked to a testing site, and 62% linked to a public health website within 3 clicks. Most apps alert users of COVID-19 exposure but require high English reading levels and are not fully inclusive of the U.S. population, which may limit their reach as public health tools.

IRF8 acts in lineage-committed rather than oligopotent progenitors to control neutrophil vs monocyte production.

Interferon regulatory factor 8 (IRF8) is a key regulator of myelopoiesis in mice and humans. IRF8-deficient mice exhibit increased neutrophil numbers but defective monocyte and dendritic cell (DC) production. It has therefore been hypothesized that IRF8 regulates granulocyte vs monocyte/DC lineage commitment by oligopotent progenitors. Alternatively, IRF8 could control the differentiation of lineage-committed progenitors. In this study, we defined the role of IRF8 in lineage commitment and neutrophil vs monocyte differentiation using a novel sorting strategy that for the first time allows us to separate oligopotent granulocyte-monocyte progenitors (GMPs) and their lineage-committed progeny: granulocyte progenitors (GPs) and monocyte progenitors (MPs). We show that IRF8 is highly expressed by both GPs and MPs, but not GMPs, and is not required for GP or MP production by GMPs. In fact, IRF8-deficient mice have more GPs and MPs. This is not due to IRF8-mediated suppression of GP and MP production by GMPs, but rather to selective effects in GPs and MPs. We identify roles for IRF8 in regulating progenitor survival and differentiation and preventing leukemic cell accumulation. Thus, IRF8 does not regulate granulocytic vs monocytic fate in GMPs, but instead acts downstream of lineage commitment to selectively control neutrophil and monocyte production.

Detection of a TLR2 agonist by hematopoietic stem and progenitor cells impacts the function of the macrophages they produce.

Several groups have shown that detection of microbial components by TLRs on hematopoietic stem and progenitor cells (HSPCs) instructs myeloid cell generation, raising interest in the possibility of targeting TLRs on HSPCs to boost myelopoiesis. However, although "TLR-derived" cells exhibit myeloid cell characteristics (phagocytosis, cytokine production, antigen presentation), it is not clear whether they are functionally equivalent to macrophages derived in the absence of TLR activation. Our in vitro and in vivo studies show that macrophages derived from mouse and human HSPC subsets (including stem cells) exposed to a TLR2 agonist prior to or during macrophage differentiation produce lower levels of inflammatory cytokines (TNF-α, IL-6, and IL-1ß) and reactive oxygen species. This is in contrast to prior exposure of differentiated macrophages to the TLR2 agonist ("tolerance"), which suppresses inflammatory cytokine production, but elevates reactive oxygen species. Soluble factors produced following exposure of HSPCs to a TLR2 agonist can also act in a paracrine manner to influence the function of macrophages derived from unexposed HSPCs. Our data demonstrate that macrophage function can be influenced by TLR signaling in the HSPCs from which they are derived, and that this may impact the clinical utility of targeting TLRs on HSPCs to boost myelopoiesis.

Preferential biological processes in the human limbus by differential gene profiling.

Corneal epithelial stem cells or limbal stem cells (LSCs) are responsible for the maintenance of the corneal epithelium in humans. The exact location of LSCs is still under debate, but the increasing need for identifying the biological processes in the limbus, where LSCs are located, is of great importance in the regulation of LSCs. In our current study we identified 146 preferentially expressed genes in the human limbus in direct comparison to that in the cornea and conjunctiva. The expression of newly identified limbal transcripts endomucin, fibromodulin, paired-like homeodomain 2 (PITX2) and axin-2 were validated using qRT-PCR. Further protein analysis on the newly identified limbal transcripts showed protein localization of PITX2 in the basal and suprabasal layer of the limbal epithelium and very low expression in the cornea and conjunctiva. Two other limbal transcripts, frizzled-7 and tenascin-C, were expressed in the basal epithelial layer of the limbus. Gene ontology and network analysis of the overexpressed limbal genes revealed cell-cell adhesion, Wnt and TGF-ß/BMP signaling components among other developmental processes in the limbus. These results could aid in a better understanding of the regulatory elements in the LSC microenvironment.

Wnt/ß-catenin signaling regulates proliferation of human cornea epithelial stem/progenitor cells.

PURPOSE: To investigate the expression and role of the Wnt signaling pathway in human limbal stem cells (LSCs). METHODS: Total RNA was isolated from the human limbus and central cornea. Limbal or cornea-specific transcripts were identified through quantitative real-time PCR. Protein expression of Wnt molecules was confirmed by immunohistochemistry on human ocular tissue. Activation of Wnt signaling using lithium chloride was achieved in vitro and its effects on LSC differentiation and proliferation were evaluated. RESULTS: Expression of Wnt2, Wnt6, Wnt11, Wnt16b, and four Wnt inhibitors were specific to the limbal region, whereas Wnt3, Wnt7a, Wnt7b, and Wnt10a were upregulated in the central cornea. Nuclear localization of ß-catenin was observed in a very small subset of basal epithelial cells only at the limbus. Activation of Wnt/ß-catenin signaling increased the proliferation and colony-forming efficiency of primary human LSCs. The stem cell phenotype was maintained, as shown by higher expression levels of putative corneal epithelial stem cell markers, ATP-binding cassette family G2 and ΔNp63α, and low expression levels of mature cornea epithelial cell marker, cytokeratin 12. CONCLUSIONS: These findings demonstrate for the first time that Wnt signaling is present in the ocular surface epithelium and plays an important role in the regulation of LSC proliferation. Modulation of Wnt signaling could be of clinical application to increase the efficiency of ex vivo expansion of corneal epithelial stem/progenitor cells for transplantation.