Spatiotemporal trends in COVID-19 vaccine sentiments on a social media platform and correlations with reported vaccine coverage.

Objective: To assess spatiotemporal trends in, and determinants of, the acceptance of coronavirus disease 2019 (COVID-19) vaccination globally, as expressed on the social media platform X (formerly Twitter). Methods: We collected over 13 million posts on the platform regarding COVID-19 vaccination made between November 2020 and March 2022 in 90 languages. Multilingual deep learning XLM-RoBERTa models annotated all posts using an annotation framework after being fine-tuned on 8125 manually annotated, English-language posts. The annotation results were used to assess spatiotemporal trends in COVID-19 vaccine acceptance and confidence as expressed by platform users in 135 countries and territories. We identified associations between spatiotemporal trends in vaccine acceptance and country-level characteristics and public policies by using univariate and multivariate regression analysis. Findings: A greater proportion of platform users in the World Health Organization's South-East Asia, Eastern Mediterranean and Western Pacific Regions expressed vaccine acceptance than users in the rest of the world. Countries in which a greater proportion of platform users expressed vaccine acceptance had higher COVID-19 vaccine coverage rates. Trust in government was also associated with greater vaccine acceptance. Internationally, vaccine acceptance and confidence declined among platform users as: (i) vaccination eligibility was extended to adolescents; (ii) vaccine supplies became sufficient; (iii) nonpharmaceutical interventions were relaxed; and (iv) global reports on adverse events following vaccination appeared. Conclusion: Social media listening could provide an effective and expeditious means of informing public health policies during pandemics, and could supplement existing public health surveillance approaches in addressing global health issues.

A roadmap to artificial intelligence (AI): Methods for designing and building AI ready data to promote fairness.

OBJECTIVES: We evaluated methods for preparing electronic health record data to reduce bias before applying artificial intelligence (AI). METHODS: We created methods for transforming raw data into a data framework for applying machine learning and natural language processing techniques for predicting falls and fractures. Strategies such as inclusion and reporting for multiple races, mixed data sources such as outpatient, inpatient, structured codes, and unstructured notes, and addressing missingness were applied to raw data to promote a reduction in bias. The raw data was carefully curated using validated definitions to create data variables such as age, race, gender, and healthcare utilization. For the formation of these variables, clinical, statistical, and data expertise were used. The research team included a variety of experts with diverse professional and demographic backgrounds to include diverse perspectives. RESULTS: For the prediction of falls, information extracted from radiology reports was converted to a matrix for applying machine learning. The processing of the data resulted in an input of 5,377,673 reports to the machine learning algorithm, out of which 45,304 were flagged as positive and 5,332,369 as negative for falls. Processed data resulted in lower missingness and a better representation of race and diagnosis codes. For fractures, specialized algorithms extracted snippets of text around keywork "femoral" from dual x-ray absorptiometry (DXA) scans to identify femoral neck T-scores that are important for predicting fracture risk. The natural language processing algorithms yielded 98% accuracy and 2% error rate The methods to prepare data for input to artificial intelligence processes are reproducible and can be applied to other studies. CONCLUSION: The life cycle of data from raw to analytic form includes data governance, cleaning, management, and analysis. When applying artificial intelligence methods, input data must be prepared optimally to reduce algorithmic bias, as biased output is harmful. Building AI-ready data frameworks that improve efficiency can contribute to transparency and reproducibility. The roadmap for the application of AI involves applying specialized techniques to input data, some of which are suggested here. This study highlights data curation aspects to be considered when preparing data for the application of artificial intelligence to reduce bias.

Defining facets of social distancing during the COVID-19 pandemic: Twitter analysis.

OBJECTIVES: Using Twitter, we aim to (1) define and quantify the prevalence and evolution of facets of social distancing during the COVID-19 pandemic in the US in a spatiotemporal context and (2) examine amplified tweets among social distancing facets. MATERIALS AND METHODS: We analyzed English and US-based tweets containing "coronavirus" between January 23-March 24, 2020 using the Twitter API. Tweets containing keywords were grouped into six social distancing facets: implementation, purpose, social disruption, adaptation, positive emotions, and negative emotions. RESULTS: A total of 259,529 unique tweets were included in the analyses. Social distancing tweets became more prevalent from late January to March but were not geographically uniform. Early facets of social distancing appeared in Los Angeles, San Francisco, and Seattle: the first cities impacted by the COVID-19 outbreak. Tweets related to the "implementation" and "negative emotions" facets largely dominated in combination with topics of "social disruption" and "adaptation", albeit to lesser degree. Social disruptiveness tweets were most retweeted, and implementation tweets were most favorited. DISCUSSION: Social distancing can be defined by facets that respond to and represent certain events in a pandemic, including travel restrictions and rising case counts. For example, Miami had a low volume of social distancing tweets but grew in March corresponding with the rise of COVID-19 cases. CONCLUSION: The evolution of social distancing facets on Twitter reflects actual events and may signal potential disease hotspots. Our facets can also be used to understand public discourse on social distancing which may inform future public health measures.

Identification of Patients with Nontraumatic Intracranial Hemorrhage Using Administrative Claims Data.

INTRODUCTION: Nontraumatic intracranial hemorrhage (ICH) is a neurological emergency of research interest; however, unlike ischemic stroke, has not been well studied in large datasets due to the lack of an established administrative claims-based definition. We aimed to evaluate both explicit diagnosis codes and machine learning methods to create a claims-based definition for this clinical phenotype. METHODS: We examined all patients admitted to our tertiary medical center with a primary or secondary International Classification of Disease version 9 (ICD-9) or 10 (ICD-10) code for ICH in claims from any portion of the hospitalization in 2014-2015. As a gold standard, we defined the nontraumatic ICH phenotype based on manual chart review. We tested explicit definitions based on ICD-9 and ICD-10 that had been previously published in the literature as well as four machine learning classifiers including support vector machine (SVM), logistic regression with LASSO, random forest and xgboost. We report five standard measures of model performance for each approach. RESULTS: A total of 1830 patients with 2145 unique ICD-10 codes were included in the initial dataset, of which 437 (24%) were true positive based on manual review. The explicit ICD-10 definition performed best (Sensitivity = 0.89 (95% CI 0.85-0.92), Specificity = 0.83 (0.81-0.85), F-score = 0.73 (0.69-0.77)) and improves on an explicit ICD-9 definition (Sensitivity = 0.87 (0.83-0.90), Specificity = 0.77 (0.74-0.79), F-score = 0.67 (0.63-0.71). Among machine learning classifiers, SVM performed best (Sensitivity = 0.78 (0.75-0.82), Specificity = 0.84 (0.81-0.87), AUC = 0.89 (0.87-0.92), F-score = 0.66 (0.62-0.69)). CONCLUSIONS: An explicit ICD-10 definition can be used to accurately identify patients with a nontraumatic ICH phenotype with substantially better performance than ICD-9. An explicit ICD-10 based definition is easier to implement and quantitatively not appreciably improved with the additional application of machine learning classifiers. Future research utilizing large datasets should utilize this definition to address important research gaps.

Automatic extraction of informal topics from online suicidal ideation.

BACKGROUND: Suicide is an alarming public health problem accounting for a considerable number of deaths each year worldwide. Many more individuals contemplate suicide. Understanding the attributes, characteristics, and exposures correlated with suicide remains an urgent and significant problem. As social networking sites have become more common, users have adopted these sites to talk about intensely personal topics, among them their thoughts about suicide. Such data has previously been evaluated by analyzing the language features of social media posts and using factors derived by domain experts to identify at-risk users. RESULTS: In this work, we automatically extract informal latent recurring topics of suicidal ideation found in social media posts. Our evaluation demonstrates that we are able to automatically reproduce many of the expertly determined risk factors for suicide. Moreover, we identify many informal latent topics related to suicide ideation such as concerns over health, work, self-image, and financial issues. CONCLUSIONS: These informal topics topics can be more specific or more general. Some of our topics express meaningful ideas not contained in the risk factors and some risk factors do not have complimentary latent topics. In short, our analysis of the latent topics extracted from social media containing suicidal ideations suggests that users of these systems express ideas that are complementary to the topics defined by experts but differ in their scope, focus, and precision of language.

Exploiting MEDLINE for gene molecular function prediction via NMF based multi-label classification.

Gene ontology (GO) provides a representation of terms and categories used to describe genes and their molecular functions, cellular components and biological processes. GO has been the standard for describing the functions of specific genes in different model organisms. GO annotation, or the tagging of genes with GO terms, has mostly been a manual and time-consuming curation process. Although many automated approaches have been proposed for annotation, few have utilized knowledge available in the literature. In this manuscript, we describe the development and evaluation of an innovative predictive system to automatically assign molecular functions (GO terms) to genes using the biomedical literature. Because genes could be associated with multiple molecular functions, we posed the GO molecular function annotation as a multi-label classification problem with several classes. We used non-negative matrix factorization (NMF) for feature reduction and then classified the genes. To address the multi-label aspect of the data, we used the binary-relevance method. Although we experimented with several classifiers, the combination of binary-relevance and K-nearest neighbor (KNN) classifier performed best. Our evaluation on UniProtKB/Swiss-Prot dataset showed the best performance of 0.84 in terms of F1-measure.

Unlocking echocardiogram measurements for heart disease research through natural language processing.

BACKGROUND: In order to investigate the mechanisms of cardiovascular disease in HIV infected and uninfected patients, an analysis of echocardiogram reports is required for a large longitudinal multi-center study. IMPLEMENTATION: A natural language processing system using a dictionary lookup, rules, and patterns was developed to extract heart function measurements that are typically recorded in echocardiogram reports as measurement-value pairs. Curated semantic bootstrapping was used to create a custom dictionary that extends existing terminologies based on terms that actually appear in the medical record. A novel disambiguation method based on semantic constraints was created to identify and discard erroneous alternative definitions of the measurement terms. The system was built utilizing a scalable framework, making it available for processing large datasets. RESULTS: The system was developed for and validated on notes from three sources: general clinic notes, echocardiogram reports, and radiology reports. The system achieved F-scores of 0.872, 0.844, and 0.877 with precision of 0.936, 0.982, and 0.969 for each dataset respectively averaged across all extracted values. Left ventricular ejection fraction (LVEF) is the most frequently extracted measurement. The precision of extraction of the LVEF measure ranged from 0.968 to 1.0 across different document types. CONCLUSIONS: This system illustrates the feasibility and effectiveness of a large-scale information extraction on clinical data. New clinical questions can be addressed in the domain of heart failure using retrospective clinical data analysis because key heart function measurements can be successfully extracted using natural language processing.

Estimating healthcare mobility in the Veterans Affairs Healthcare System.

BACKGROUND: Healthcare mobility, defined as healthcare utilization in more than one distinct healthcare system, may have detrimental effects on outcomes of care. We characterized healthcare mobility and associated characteristics among a national sample of Veterans. METHODS: Using the Veterans Health Administration Electronic Health Record, we conducted a retrospective cohort study to quantify healthcare mobility within a four year period. We examined the association between sociodemographic and clinical characteristics and healthcare mobility, and characterized possible temporal and geographic patterns of healthcare mobility. RESULTS: Approximately nine percent of the sample were healthcare mobile. Younger Veterans, divorced or separated Veterans, and those with hepatitis C virus and psychiatric disorders were more likely to be healthcare mobile. We demonstrated two possible patterns of healthcare mobility, related to specialty care and lifestyle, in which Veterans repeatedly utilized two different healthcare systems. CONCLUSIONS: Healthcare mobility is associated with young age, marital status changes, and also diseases requiring intensive management. This type of mobility may affect disease prevention and management and has implications for healthcare systems that seek to improve population health.

Baseline cluster membership demonstrates positive associations with first occurrence of multiple gerontologic outcomes over 10 years.

UNLABELLED: BACKGROUND/STUDY CONTEXT: The potential of cluster analysis (CA) as a baseline predictor of multivariate gerontologic outcomes over a long period of time has not been previously demonstrated. METHODS: Restricting candidate variables to a small group of established predictors of deleterious gerontologic outcomes, various CA methods were applied to baseline values from 754 nondisabled, community-living persons, aged 70 years or older. The best cluster solution yielded at baseline was subsequently used as a fixed explanatory variable in time-to-event models of the first occurrence of the following outcomes: any disability in four activities of daily living, any disability in four mobility measures, and death. Each outcome was recorded through a maximum of 129 months or death. Associations between baseline ordinal cluster level and first occurrence of all three outcomes were modeled over a 10-year period with proportional hazards regression and compared with the associations yielded by the analogous latent class analysis (LCA) solution. RESULTS: The final cluster-defining variables were continuous measures of cognitive status and depressive symptoms, and dichotomous indicators of slow gait and exhaustion. The best solution yielded by baseline values of these variables was obtained with a K-means algorithm and cosine similarity and consisted of three clusters representing increasing levels of impairment. After adjustment for age, sex, ethnic group, and number of chronic conditions, baseline ordinal cluster level demonstrated significantly positive associations with all three outcomes over a 10-year period that were equivalent to those from the corresponding LCA solution. CONCLUSION: These findings suggest that baseline clusters based on previously established explanatory variables have potential to predict multivariate gerontologic outcomes over a long period of time.

Deploying a national clinical text processing infrastructure.

OBJECTIVES: Clinical text processing offers a promising avenue for improving multiple aspects of healthcare, though operational deployment remains a substantial challenge. This case report details the implementation of a national clinical text processing infrastructure within the Department of Veterans Affairs (VA). METHODS: Two foundational use cases, cancer case management and suicide and overdose prevention, illustrate how text processing can be practically implemented at scale for diverse clinical applications using shared services. RESULTS: Insights from these use cases underline both commonalities and differences, providing a replicable model for future text processing applications. CONCLUSIONS: This project enables more efficient initiation, testing, and future deployment of text processing models, streamlining the integration of these use cases into healthcare operations. This project implementation is in a large integrated health delivery system in the United States, but we expect the lessons learned to be relevant to any health system, including smaller local and regional health systems in the United States.

Cell Maps for Artificial Intelligence: AI-Ready Maps of Human Cell Architecture from Disease-Relevant Cell Lines.

This article describes the Cell Maps for Artificial Intelligence (CM4AI) project and its goals, methods, standards, current datasets, software tools , status, and future directions. CM4AI is the Functional Genomics Data Generation Project in the U.S. National Institute of Health's (NIH) Bridge2AI program. Its overarching mission is to produce ethical, AI-ready datasets of cell architecture, inferred from multimodal data collected for human cell lines, to enable transformative biomedical AI research.

Complementary ensemble clustering of biomedical data.

The rapidly growing availability of electronic biomedical data has increased the need for innovative data mining methods. Clustering in particular has been an active area of research in many different application areas, with existing clustering algorithms mostly focusing on one modality or representation of the data. Complementary ensemble clustering (CEC) is a recently introduced framework in which Kmeans is applied to a weighted, linear combination of the coassociation matrices obtained from separate ensemble clustering of different data modalities. The strength of CEC is its extraction of information from multiple aspects of the data when forming the final clusters. This study assesses the utility of CEC in biomedical data, which often have multiple data modalities, e.g., text and images, by applying CEC to two distinct biomedical datasets (PubMed images and radiology reports) that each have two modalities. Referent to five different clustering approaches based on the Kmeans algorithm, CEC exhibited equal or better performance in the metrics of micro-averaged precision and Normalized Mutual Information across both datasets. The reference methods included clustering of single modalities as well as ensemble clustering of separate and merged data modalities. Our experimental results suggest that CEC is equivalent or more efficient than comparable Kmeans based clustering methods using either single or merged data modalities.

A Roadmap to Artificial Intelligence (AI): Methods for Designing and Building AI ready Data for Women's Health Studies.

Objectives: Evaluating methods for building data frameworks for application of AI in large scale datasets for women's health studies. Methods: We created methods for transforming raw data to a data framework for applying machine learning (ML) and natural language processing (NLP) techniques for predicting falls and fractures. Results: Prediction of falls was higher in women compared to men. Information extracted from radiology reports was converted to a matrix for applying machine learning. For fractures, by applying specialized algorithms, we extracted snippets from dual x-ray absorptiometry (DXA) scans for meaningful terms usable for predicting fracture risk. Discussion: Life cycle of data from raw to analytic form includes data governance, cleaning, management, and analysis. For applying AI, data must be prepared optimally to reduce algorithmic bias. Conclusion: Algorithmic bias is harmful for research using AI methods. Building AI ready data frameworks that improve efficiency can be especially valuable for women's health.

Understanding headache classification coding within the veterans health administration using ICD-9-CM and ICD-10-CM in fiscal years 2014-2017.

OBJECTIVES: Understand the continuity and changes in headache not-otherwise-specified (NOS), migraine, and post-traumatic headache (PTH) diagnoses after the transition from ICD-9-CM to ICD-10-CM in the Veterans Health Administration (VHA). BACKGROUND: Headache is one of the most commonly diagnosed chronic conditions managed within primary and specialty care clinics. The VHA transitioned from ICD-9-CM to ICD-10-CM on October-1-2015. The effect transitioning on coding of specific headache diagnoses is unknown. Accuracy of headache diagnosis is important since different headache types respond to different treatments. METHODS: We mapped headache diagnoses from ICD-9-CM (FY 2014/2015) onto ICD-10-CM (FY 2016/2017) and computed coding proportions two years before/after the transition in VHA. We used queries to determine the change in transition pathways. We report the odds of ICD-10-CM coding associated with ICD-9-CM controlling for provider type, and patient age, sex, and race/ethnicity. RESULTS: Only 37%, 58% and 34% of patients with ICD-9-CM coding of NOS, migraine, and PTH respectively had an ICD-10-CM headache diagnosis. Of those with an ICD-10-CM diagnosis, 73-79% had a single headache diagnosis. The odds ratios for receiving the same code in both ICD-9-CM and ICD-10-CM after adjustment for ICD-9-CM and ICD-10-CM headache comorbidities and sociodemographic factors were high (range 6-26) and statistically significant. Specifically, 75% of patients with headache NOS had received one headache diagnoses (Adjusted headache NOS-ICD-9-CM OR for headache NOS-ICD-10-CM = 6.1, 95% CI 5.89-6.32. 79% of migraineurs had one headache diagnoses, mostly migraine (Adjusted migraine-ICD-9-CM OR for migraine-ICD-10-CM = 26.43, 95% CI 25.51-27.38). The same held true for PTH (Adjusted PTH-ICD-9-CM OR for PTH-ICD-10-CM = 22.92, 95% CI: 18.97-27.68). These strong associations remained after adjustment for specialist care in ICD-10-CM follow-up period. DISCUSSION: The majority of people with ICD-9-CM headache diagnoses did not have an ICD-10-CM headache diagnosis. However, a given diagnosis in ICD-9-CM by a primary care provider (PCP) was significantly predictive of its assignment in ICD-10-CM as was seeing either a neurologist or physiatrist (compared to a generalist) for an ICD-10-CM headache diagnosis. CONCLUSION: When a veteran had a specific diagnosis in ICD-9-CM, the odds of being coded with the same diagnosis in ICD-10-CM were significantly higher. Specialist visit during the ICD-10-CM period was independently associated with all three ICD-10-CM headaches.

Identifying the mechanisms of patient-centred communication in secure messages between clinicians and cancer patients.

Objective: Identify how patients and clinicians incorporate patient-centered communication (PCC) within secure messaging. Methods: A random sample of 199 secure messages from patient portal communication between patients and clinicians were collected and analyzed. Via manual annotation, the task of tagging target words/phrases in text, we identified five components of PCC: information giving, information seeking, emotional support, partnership, and shared decision-making. Textual analysis was also performed to understand the context of PCC expressions within messages. Results: Information-giving was the predominant (n = 346, 68.1%) PCC category used in secure messaging, more than double of the other four PCC codes, information-seeking (n = 82, 16.1%), emotional support (n = 52, 10.2%), shared decision making (n = 5, 1.0%), combined. The textual analysis revealed that clinicians informed patients about appointment reminders and new protocols while patients reminded clinicians about upcoming procedures and outcomes of test results conducted by other clinicians. Although less common, patients expressed statements of concern, uncertainty, and fear; enabling clinicians to provide support. Conclusion: Secure messaging is mainly used for exchanging information, but other aspects of PCC emerge using this channel of communication. Innovation: Meaningful discussions can occur via secure messaging, and clinicians should be mindful of incorporating PCC when communicating with patients through secure messaging.

Mental Health Diagnoses are Not Associated With Indicators of Lower Quality Pain Care in Electronic Health Records of a National Sample of Veterans Treated in Veterans Health Administration Primary Care Settings.

Prior research has demonstrated disparities in general medical care for patients with mental health conditions, but little is known about disparities in pain care. The objective of this retrospective cohort study was to determine whether mental health conditions are associated with indicators of pain care quality (PCQ) as documented by primary care clinicians in the Veterans Health Administration (VHA). We used natural language processing to analyze electronic health record data from a national sample of Veterans with moderate to severe musculoskeletal pain during primary care visits in the Fiscal Year 2017. Twelve PCQ indicators were annotated from clinician progress notes as present or absent; PCQ score was defined as the sum of these indicators. Generalized estimating equation Poisson models examined associations among mental health diagnosis categories and PCQ scores. The overall mean PCQ score across 135,408 person-visits was 8.4 (SD = 2.3). In the final adjusted model, post-traumatic stress disorder was associated with higher PCQ scores (RR = 1.006, 95%CI 1.002-1.010, P = .007). Depression, alcohol use disorder, other substance use disorder, schizophrenia, and bipolar disorder diagnoses were not associated with PCQ scores. Overall, results suggest that in this patient population, presence of a mental health condition is not associated with lower quality pain care. PERSPECTIVE: This study used a natural language processing approach to analyze medical records to determine whether mental health conditions are associated with indicators of pain care quality as documented by primary care clinicians. Findings suggest that presence of a diagnosed mental health condition is not associated with lower quality pain care.

Complementary and Integrative Health Approaches and Pain Care Quality in the Veterans Health Administration Primary Care Setting: A Quasi-Experimental Analysis.

Background: Complementary and integrative health (CIH) approaches have been recommended in national and international clinical guidelines for chronic pain management. We set out to determine whether exposure to CIH approaches is associated with pain care quality (PCQ) in the Veterans Health Administration (VHA) primary care setting. Methods: We followed a cohort of 62,721 Veterans with newly diagnosed musculoskeletal disorders between October 2016 and September 2017 over 1-year. PCQ scores were derived from primary care progress notes using natural language processing. CIH exposure was defined as documentation of acupuncture, chiropractic or massage therapies by providers. Propensity scores (PSs) were used to match one control for each Veteran with CIH exposure. Generalized estimating equations were used to examine associations between CIH exposure and PCQ scores, accounting for potential selection and confounding bias. Results: CIH was documented for 14,114 (22.5%) Veterans over 16,015 primary care clinic visits during the follow-up period. The CIH exposure group and the 1:1 PS-matched control group achieved superior balance on all measured baseline covariates, with standardized differences ranging from 0.000 to 0.045. CIH exposure was associated with an adjusted rate ratio (aRR) of 1.147 (95% confidence interval [CI]: 1.142, 1.151) on PCQ total score (mean: 8.36). Sensitivity analyses using an alternative PCQ scoring algorithm (aRR: 1.155; 95% CI: 1.150-1.160) and redefining CIH exposure by chiropractic alone (aRR: 1.118; 95% CI: 1.110-1.126) derived consistent results. Discussion: Our data suggest that incorporating CIH approaches may reflect higher overall quality of care for patients with musculoskeletal pain seen in primary care settings, supporting VHA initiatives and the Declaration of Astana to build comprehensive, sustainable primary care capacity for pain management. Future investigation is warranted to better understand whether and to what degree the observed association may reflect the therapeutic benefits patients actually received or other factors such as empowering provider-patient education and communication about these approaches.

Measuring pain care quality in the Veterans Health Administration primary care setting.

ABSTRACT: The lack of a reliable approach to assess quality of pain care hinders quality improvement initiatives. Rule-based natural language processing algorithms were used to extract pain care quality (PCQ) indicators from documents of Veterans Health Administration primary care providers for veterans diagnosed within the past year with musculoskeletal disorders with moderate-to-severe pain intensity across 2 time periods 2013 to 2014 (fiscal year [FY] 2013) and 2017 to 2018 (FY 2017). Patterns of documentation of PCQ indicators for 64,444 veterans and 124,408 unique visits (FY 2013) and 63,427 veterans and 146,507 visits (FY 2017) are described. The most commonly documented PCQ indicators in each cohort were presence of pain, etiology or source, and site of pain (greater than 90% of progress notes), while least commonly documented were sensation, what makes pain better or worse, and pain's impact on function (documented in fewer than 50%). A PCQ indicator score (maximum = 12) was calculated for each visit in FY 2013 (mean = 7.8, SD = 1.9) and FY 2017 (mean = 8.3, SD = 2.3) by adding one point for every indicator documented. Standardized Cronbach alpha for total PCQ scores was 0.74 in the most recent data (FY 2017). The mean PCQ indicator scores across patient characteristics and types of healthcare facilities were highly stable. Estimates of the frequency of documentation of PCQ indicators have face validity and encourage further evaluation of the reliability, validity, and utility of the measure. A reliable measure of PCQ fills an important scientific knowledge and practice gap.