Associations of Primary Care Provider Burnout with Quality Improvement, Patient Experience Measurement, Clinic Culture, and Job Satisfaction.

BACKGROUND: Burnout among providers negatively impacts patient care experiences and safety. Providers at Federally Qualified Health Centers (FQHC) are at high risk for burnout due to high patient volumes; inadequate staffing; and balancing the demands of patients, families, and team members. OBJECTIVE: Examine associations of provider burnout with their perspectives on quality improvement (QI), patient experience measurement, clinic culture, and job satisfaction. DESIGN: We conducted a cross-sectional provider survey about their perspectives including the single-item burnout measure. We fit separate regression models, controlling for provider type, gender, being multilingual, and fixed effects for clinic predicting outcome measures from burnout. PARTICIPANTS: Seventy-four providers from 44 clinics in large, urban FQHC (52% response rate; n = 174). MAIN MEASURES: Survey included a single-item, self-defined burnout measure adapted from the Physician Worklife Survey, and measures from the RAND AMA Study survey, Heath Tracking Physician survey, TransforMed Clinician and Staff Questionnaire, Physician Worklife Survey, Minimizing Errors Maximizing Outcomes survey, and surveys by Friedberg et al. 31 and Walling et al. 32 RESULTS: Thirty percent of providers reported burnout. Providers in clinics with more facilitative leadership reported not being burned out (compared to those reporting burnout; p-values < 0.05). More pressures related to patient care and lower job satisfaction were associated with burnout (p-values < 0.05). CONCLUSIONS: Creating provider-team relationships and environments where providers have the time and space necessary to discuss changes to improve care, ideas are shared, leadership supports QI, and QI is monitored and discussed were related to not being burned out. Reducing time pressures and improving support needed for providers to address the high-need levels of FQHC patients can also decrease burnout. Such leadership and support to improving care may be a separate protective factor against burnout. Research is needed to further examine which aspects of leadership drive down burnout and increase provider involvement in change efforts and improving care.

Comparison of patient-reported outcomes measurement information system (PROMIS®)-29 and PROMIS global physical and mental health scores.

PURPOSE: The Patient-Reported Outcomes Measurement and Information System (PROMIS®): includes the PROMIS-29 physical and mental health summary and the PROMIS global physical and mental health scores. It is unknown how these scores coincide with one another. This study examines whether the scores yield similar or different information. METHODS: The PROMIS-29 and the PROMIS global health items were administered to 5804 adults from Amazon's Mechanical Turk (MTurk) in 2021-2022 and to 4060 adults in the Ipsos KnowledgePanel (KP) in 2022. RESULTS: The median age of those in MTurk (KP) was 36 (54) and 53% (50%) were male. Mean T-scores on the PROMIS-29 and PROMIS global physical health scales were similar, but PROMIS global mental health was 3-4 points lower than the PROMIS-29 mental health summary score. Product-moment correlations ranged from 0.69 to 0.81 between the PROMIS-29 physical health and PROMIS global physical health scales and 0.56-0.69 between the mental health scales. Multi-trait multimethod analyses indicated that only a small proportion of the correlations between the two methods of measuring mental health were significantly more highly correlated with one another than correlations between physical and mental health. CONCLUSIONS: PROMIS-29 and PROMIS global mental health scales provide different information and, therefore, study conclusions may vary depending on which measure is used. Interpretation of results needs to consider that the PROMIS-29 mental health scale is a weighted combination of specific domains while the PROMIS global mental health scale is based on general mental health perceptions. Further comparisons of methods of assessing mental health are needed.

The PROMIS-16 reproduces the PROMIS-29 physical and mental health summary scores accurately in a probability-based internet panel.

PURPOSE: The Patient-Reported Outcomes Measurement Information System® (PROMIS)-16 assesses the same multi-item domains but does not include the pain intensity item in the PROMIS-29. We evaluate how well physical and mental health summary scores estimated from the PROMIS-16 reproduce those estimated using the PROMIS-29. METHODS: An evaluation of data collected from 4130 respondents from the KnowledgePanel. Analyses include confirmatory factor analysis to assess physical and mental health latent variables based on PROMIS-16 scores, reliability estimates for the PROMIS measures, mean differences and correlations of scores estimated by the PROMIS-16 with those estimated by the PROMIS-29, and associations between differences in corresponding PROMIS-16 and PROMIS-29 scores by sociodemographic characteristics. RESULTS: A two-factor (physical and mental health) model adequately fits the PROMIS-16 scores. Reliability estimates for the PROMIS-16 measures were slightly lower than for the PROMIS-29 measures. There were minimal differences between PROMIS physical and mental health summary scores estimated using the PROMIS-16 or the PROMIS-29. PROMIS-16 and PROMIS-29 score differences by sociodemographic characteristics were small. Using the PROMIS pain intensity item when scoring the PROMIS-16 produced similar estimates of physical and mental health summary scores. CONCLUSION: The PROMIS-16 provides similar estimates of the PROMIS-29 physical and mental health summary scores. The high reliability of these scores indicates they are accurate enough for use with individual patients.

Development of an ultra-short measure of eight domains of health-related quality of life for research and clinical care: the patient-reported outcomes measurement information system® PROMIS®-16 profile.

PURPOSE: We describe development of a short health-related quality of life measure, the patient-reported outcomes measurement information system® (PROMIS®)-16 Profile, which generates domain-specific scores for physical function, ability to participate in social roles and activities, anxiety, depression, sleep disturbance, pain interference, cognitive function, and fatigue. METHODS: An empirical evaluation of 50 candidate PROMIS items and item pairs was conducted using data from a sample of 5775 respondents from Amazon's Mechanical Turk (MTurk). Results and item response theory information curves for a subset of item pairs were presented and discussed in a stakeholder meeting to narrow the candidate item sets. A survey of the stakeholders and 124 MTurk adults was conducted to solicit preferences among remaining candidate items and finalize the measure. RESULTS: Empirical evaluation showed minimal differences in basic descriptive statistics (e.g., means, correlations) and associations with the PROMIS-29 + 2 Profile, thus item pairs were further considered primarily based on item properties and content. Stakeholders discussed and identified subsets of candidate item pairs for six domains, and final item pairs were agreed upon for two domains. Final items were selected based on stakeholder and MTurk-respondent preferences. The PROMIS-16 profile generates eight domain scores with strong psychometric properties. CONCLUSION: The PROMIS-16 Profile provides an attractive brief measure of eight distinct domains of health-related quality of life, representing an ideal screening tool for clinical care, which can help clinicians quickly identify distinct areas of concern that may require further assessment and follow-up. Further research is needed to confirm and extend these findings.

Performance of the Physical Functioning Activities of Daily Living Scale in the 2020 Medicare Health Outcomes Survey.

OBJECTIVE: Assessing functional limitations for adults at high risk of frailty yields valuable information for identifying those in need of therapy. We evaluate a self-report measure used to assess physical function among Medicare recipients in the United States. DESIGN: Secondary analysis of the 2020 Medicare Health Outcomes Survey. SETTING: A random sample of adult enrollees of 510 managed care plans. PARTICIPANTS: 287,476 adults (37% completion rate): 58% women; 16% were <65 years old (entitled via disability), 50% 65-74, and 34% 75 or older; 77% White, 14% Black, and 8% another race; 19% had

How Well Do Seven Self-Report Measures Represent Underlying Back Pain Impact?

BACKGROUND: The extent to which different measures of back pain impact represent an underlying common factor has implications for decisions about which one to use in studies of pain management and estimating one score from others. AIMS: To determine if different self-report back pain impact measures represent an underlying pain latent variable and estimate associations with it. METHOD: Seven pain impact measures completed by Amazon Mechanical Turk adults are used to estimate internal consistency reliability and associations: Oswestry Disability Index (ODI), Roland-Morris Disability Questionnaire (RMDQ), short form of the Örebro Musculoskeletal Pain Questionnaire (OMPQ), Subgroups for Targeted Treatment (STarT) Back Tool, the Graded Chronic Pain Scale (GCPS) disability score, PEG (Pain intensity, interference with Enjoyment of life, interference with General activity), and Impact Stratification Score (ISS). RESULTS: The sample of 1,874 adults with back pain had an average age of 41 and 52% were female. Sixteen percent were Hispanic, 7% non-Hispanic Black, 5% non-Hispanic Asian, and 71% non-Hispanic White. Internal consistency reliability estimates ranged from 0.710 (OMPQ) to 0.923 (GCPS). Correlations among the measures ranged from 0.609 (RMDQ with OMPQ) to 0.812 (PEG with GCPS). Standardized factor loadings on the pain latent variable ranged from 0.782 (RMDQ) to 0.870 (ISS). CONCLUSIONS: Scores of each measure can be estimated from the others for use in research.

Narrative comments about pediatric inpatient experiences yield substantial information beyond answers to closed-ended CAHPS survey questions.

PURPOSE: Adults' comments on patient experience surveys explain variation in provider ratings, with negative comments providing more actionable information than positive comments. We investigate if narrative comments on the Consumer Assessment of Healthcare Providers and Systems (CAHPS®) survey of inpatient pediatric care (Child HCAHPS) account for global perceptions of the hospital beyond that explained by reports about specific aspects of care. METHODS: We analyzed 545 comments from 927 Child HCAHPS surveys completed by parents and guardians of hospitalized children with at least a 24-h hospital stay from July 2017 to December 2020 at an urban children's hospital. Comments were coded for valence (positive/negative/mixed) and actionability and used to predict Overall Hospital Rating and Willingness to Recommend the Hospital along with Child HCAHPS composite scores. RESULTS: Comments were provided more often by White and more educated respondents. Negative comments and greater actionability of comments were significantly associated with Child HCAHPS global rating measures, controlling for responses to closed-ended questions, and child and respondent characteristics. Each explained an additional 8% of the variance in respondents' overall hospital ratings and an additional 5% in their willingness to recommend the hospital. CONCLUSIONS: Child HCAHPS narrative comment data provide significant additional information about what is important to parents and guardians during inpatient pediatric care beyond closed-ended composites. PRACTICE IMPLICATIONS: Quality improvement efforts should include a review of narrative comments alongside closed-ended responses to help identify ways to improve inpatient care experiences. To promote health equity, comments should be encouraged for racial-and-ethnic minority patients and those with less educational attainment.

Health-related quality of life in a systematically assessed cohort of children and adults with urea cycle disorders.

PURPOSE: Individuals with urea cycle disorders (UCDs) may develop recurrent hyperammonemia, episodic encephalopathy, and neurological sequelae which can impact Health-related Quality of Life (HRQoL). To date, there have been no systematic studies of HRQoL in people with UCDs. METHODS: We reviewed HRQoL and clinical data for 190 children and 203 adults enrolled in a multicenter UCD natural history study. Physical and psychosocial HRQoL in people with UCDs were compared to HRQoL in healthy people and people with phenylketonuria (PKU) and diabetes mellitus. We assessed relationships between HRQoL, UCD diagnosis, and disease severity. Finally, we calculated sample sizes required to detect changes in these HRQoL measures. RESULTS: Individuals with UCDs demonstrated worse physical and psychosocial HRQoL than their healthy peers and peers with PKU and diabetes. In children, HRQoL scores did not differ by diagnosis or severity. In adults, individuals with decreased severity had worse psychosocial HRQoL. Finally, we show that a large number of individuals would be required in clinical trials to detect differences in HRQoL in UCDs. CONCLUSION: Individuals with UCDs have worse HRQoL compared to healthy individuals and those with PKU and diabetes. Future work should focus on the impact of liver transplantation and other clinical variables on HRQoL in UCDs.

Development of a Patient-Reported Outcome Measure to Assess Symptoms Associated with Cataract Surgery and Intraocular Lens Implants.

PURPOSE: To develop a standardized patient-reported outcome measure of visual perceptions and symptoms for implanted premium and monofocal intraocular lenses (IOLs). DESIGN: Observational study before and after IOL implants to assess the measure and symptom experience. PARTICIPANTS: Adults scheduled for binocular implantation of the same IOL type completed the survey at baseline prior to surgery (n = 716) and postoperatively (n = 554). Most respondents were female (64%), White (81%), 61 or older (89%), and had some college or more education (62%). METHODS: Administration was by web survey with mail follow-up and phone reminders. MAIN OUTCOME MEASURES: Frequency, severity, and level of symptom bother in the last 7 days for 14 symptoms: (1) glare, (2) hazy vision, (3) blurry vision, (4) starbursts, (5) halos, (6) snowballs, (7) floaters, (8) double images, (9) rings and spider webs, (10) distortion, (11) light flashes with eyes closed, (12) light flashes with eyes open, (13) shimmering images, and (14) dark shadows. RESULTS: The median correlation among having 14 symptoms at baseline was only 0.19. Mean uncorrected binocular visual acuity improved from a preoperative value of 0.47 logarithm of the minimum angle of resolution (logMAR; Snellen 20/59) to a postoperative value of 0.12 (20/26) and best-corrected binocular visual acuity improved from 0.23 logMAR (20/34) preoperative to 0.05 logMAR (20/22) postoperative. The most bothersome symptoms were reduced after surgery: preoperative/postoperative glare (84%/36%), blurry vision (68%/22%), starbursts (66%/28%), hazy vision (63%/18%), snowballs (55%/17%), and halos (52%/22%). All symptoms decreased significantly (P < 0.0001) from before to after surgery except for dark crescent-shaped shadows (4%/4%). The percentage of symptoms rated as quite a bit or extremely bothersome declined from before to after surgery except for dark crescent-shaped shadows (29%/32%): blurry vision (54%/15%), snowballs (52%/14%), glare (49%/15%), and halos (46%/14%). Having monofocal IOL implants was associated with significantly more reduction in halos, starbursts, glare, and rings and spider webs, but less improvement in self-reported general vision. CONCLUSIONS: This study provides support for the 37-item Assessment of IntraOcular Lens Implant Symptoms (AIOLIS) instrument for use to assess symptoms and general perceptions of vision in clinical studies and clinical care. FINANCIAL DISCLOSURE(S): Proprietary or commercial disclosure may be found after the references.

Symptoms and Satisfaction Levels Associated with Intraocular Lens Implants in the Monofocal and Premium IOL Patient-Reported Outcome Measure Study.

PURPOSE: To develop a questionnaire with standardized questions and images about visual symptoms and satisfaction administered before and after cataract surgery with monofocal and various (premium) intraocular lenses (IOLs). DESIGN: A prospective, observational study of cataract surgery patients completing a self-administered questionnaire preoperatively and postoperatively at 4 to 6 months. PARTICIPANTS: Five hundred fifty-four patients with plans to undergo implantation of the same IOL in both eyes on separate occasions in 20 ophthalmology practices. METHODS: An 86-item questionnaire with standardized images assessed the following 14 symptoms: glare, blurry vision, starbursts, hazy vision, snowballs, halos, floaters, double images, rings and spider webs, light flashes with eyes closed, distortion, light flashes with eyes open, shimmering images, and dark crescent-shaped shadows. MAIN OUTCOME MEASURES: Symptom severity and level of symptom bother, satisfaction with vision, quality of vision, and ability to see without corrective lenses or eyeglasses. RESULTS: Except for dark crescent-shaped shadows, the report of visual symptoms significantly decreased postoperatively. Best uncorrected binocular visual acuity improved from 0.47 (20/59 Snellen visual acuity values) ± 0.35 logarithm of the minimum angle of resolution (logMAR) preoperatively to 0.12 (20/26 Snellen visual acuity values) ± 0.12 logMAR postoperatively. Patients' ratings of intermediate vision as good to excellent improved significantly from 12% preoperatively to 71% postoperatively, and patients' ratings of distance vision improved from 8% preoperatively to 85% postoperatively. After surgery, 84% reported that they were somewhat, very, or completely satisfied with their vision. Most patients (88%) reported that they could see pretty well, very well, or perfectly well without corrective lenses after surgery. CONCLUSIONS: The Assessment of IntraOcular Lens Implant Symptoms questionnaire can be used across a wide variety of IOLs to evaluate visual symptoms and satisfaction with a growing segment of the market, premium IOLs, that target intermediate and near vision, in addition to distance vision. Compared to patients receiving monofocal IOLs, patients receiving premium IOLs appear to be more challenging to satisfy because of their requirements for distance, intermediate, and near vision, and their desire to be free of eyeglasses postoperatively. This instrument provides a structured, uniform tool for regulators, researchers, and ophthalmologists in everyday practice to gain insights into patients' experiences. FINANCIAL DISCLOSURE(S): The author(s) have no proprietary or commercial interest in any materials discussed in this article.

Follow-Up Shadow Coaching Improves Primary Care Provider-Patient Interactions and Maintains Improvements When Conducted Regularly: A Spline Model Analysis.

INTRODUCTION: Shadow coaching improves provider-patient interactions, as measured by CG-CAHPS® overall provider rating (OPR) and provider communication (PC). However, these improvements erode over time. AIM: Examine whether a second coaching session (re-coaching) improves and sustains patient experience. SETTING: Large, urban Federally Qualified Health Center PROGRAM: Trained providers observed patient care by colleagues and provided suggestions for improvement. Providers with OPRs<90 (0-100-point scale) were eligible. EVALUATION: We used stratified randomization based on provider type and OPR to assign half of the 40 eligible providers to re-coaching. For OPR and PC, we fit mixed-effects regression models with random-effects for provider (level of treatment assignment) and fixed-effects for time (linear spline with knots and possible "jump" at initial coaching and re-coaching), previous OPR, patient characteristics, and sites. We observed a statistically significant medium jump among re-coached providers after re-coaching on OPR (3.7 points) and PC (3.5 points); differences of 1, 3, and ≥5-points for CAHPS measures are considered small, medium, and large. Improvements from re-coaching persisted for 12 months for OPR and 8 months for PC. DISCUSSION: Re-coaching improved patient experience more than initial coaching, suggesting the reactivation of knowledge from initial coaching. However, re-coaching gains also eroded. Coaching should occur every 6 to 12 months to maintain behaviors and scores.

Shadow Coaching Improves Patient Experience for English-Preferring Patients but not for Spanish-Preferring Patients.

BACKGROUND: Shadow coaching, a type of one-on-one provider counseling by trained peers, is an effective strategy for improving provider behaviors and patient interactions, but its effects on improving patient experience for English- and Spanish-preferring patients is unknown. OBJECTIVE: Assess effects of shadow coaching on patient experience for English- and for Spanish-preferring patients. DESIGN: We analyzed 2012-2019 Clinician and Group Consumer Assessment of Healthcare Providers and Systems (CG-CAHPS) data (n=46,089) from an urban Federally Qualified Health Center with 44 primary care practices and 320 providers. One-third (n=14,631) were Spanish-preferring patients. We fit mixed-effects regression models with random effects for provider (the level of treatment assignment) and fixed effects for time (a linear spline for time with a knot and "jump" at coaching date), patient characteristics, and site indicators, stratified by preferred language. PARTICIPANTS: The 74 providers who had a 6-month average top-box score on the CAHPS overall provider rating below 90 (on a 100-point scale) were shadow coached. Similar percentages of English-preferring (45%) and Spanish-preferring patients (43%) were seen by coached providers. INTERVENTION: Trained providers observed patient care by colleagues and provided suggestions for improvement. Verbal feedback was provided immediately after the observation and the participant received a written report summarizing the comments and recommendations from the coaching session. MAIN MEASURES: CG-CAHPS Visit Survey 2.0 provider communication composite and overall provider rating (0-100 scoring). KEY RESULTS: We found a statistically significant 2-point (small) jump in CAHPS provider communication and overall provider rating among English-preferring patients of coached providers. There was no evidence of a coaching effect on patient experience for Spanish-preferring patients. CONCLUSIONS: Coaching improved care experiences for English-preferring patients but may not have improved patient experience for Spanish-preferring patients. Selection and training of providers to communicate effectively with Spanish-preferring patients is needed to extend the benefits of shadow coaching to Spanish-preferring patients.

Telephone Outreach Enhances Recruitment of Underrepresented Seriously Ill Patients for an Advance Care Planning Pragmatic Trial.

BACKGROUND: Patients experiencing systemic patterns of disadvantage, such as racial/ethnic minorities and those with limited English proficiency, are underrepresented in research. This is particularly true for large pragmatic trials of potentially sensitive research topics, such as advance care planning (ACP). It is unclear how phone outreach may affect research participation by underrepresented individuals. OBJECTIVE: To assess the effect of phone outreach, in addition to standard mail survey recruitment, in a population-based ACP pragmatic trial at three academic health systems in California. DESIGN: Retrospective cohort study PATIENTS: Primary care patients with serious illness were mailed a survey in their preferred language. Patients who did not initially respond by mail received up to three reminder phone calls with the option of survey completion by phone. MAIN MEASURES: Effect of phone outreach on survey response rate associated with respondent demographic characteristics (e.g., Social Vulnerability Index [SVI], range 0 (low) to 1 (high)). RESULTS: Across the health systems, 5998 seriously ill patients were mailed surveys. We obtained completed surveys from 1215 patients (20% response rate); 787 (65%) responded after mail alone and 428 (35%) participated only after phone outreach. Patients recruited after phone outreach compared to mail alone were more socially vulnerable (SVI 0.41 v 0.35, P < 0.001), were more likely to report being a racial/ethnic minority (35% v 28%, P = 0.006), and non-English speaking (16% v 10%, P = 0.005). Age and gender did not differ significantly. The inclusion of phone outreach resulted in a sample that better represented the baseline population than mail alone in racial/ethnic minority (28% mail alone, 30% including phone outreach, 36% baseline population), non-English language preference (10%, 12%, 15%, respectively), and SVI (0.35, 0.37, 0.38, respectively). CONCLUSIONS: Phone outreach for a population-based survey in a pragmatic trial concerning a potentially sensitive topic significantly enhanced recruitment of underrepresented seriously ill patients.

Likely change indexes improve estimates of individual change on patient-reported outcomes.

PURPOSE: Individual change on a patient-reported outcome (PRO) measure can be assessed by statistical significance and meaningfulness to patients. We explored the relationship between these two criteria by varying the confidence levels of the coefficient of repeatability (CR) on the Patient-Reported Outcomes Measurement Information System (R) Physical Function (PF) 10a (PF10a) measure. METHODS: In a sample of 1129 adult cancer patients, we estimated individual-change thresholds on the PF10a from baseline to 6 weeks later with the CR at 50%, 68%, and 95% confidence. We also assessed agreement with group- and individual-level thresholds from anchor-based methods [mean change and receiver operating characteristic (ROC) curve] using a PF-specific patient global impression of change (PGIC). RESULTS: CRs at 50%, 68%, and 95% confidence were 3, 4, and 7 raw score points, respectively. The ROC- and mean-change-based thresholds for deterioration were -4 and -6; for improvement they were both 2. Kappas for agreement between anchor-based thresholds and CRs for deterioration ranged between κ = 0.65 and 1.00, while for improvement, they ranged between 0.35 and 0.83. Agreement between the PGIC and all CRs always fell below "good" (κ < 0.40) for deterioration (0.30-0.33) and were lower for improvement (0.16-0.28). CONCLUSIONS: In comparison to the CR at 95% confidence, CRs at 50% and 68% confidence (considered likely change indexes) have the advantage of maximizing the proportion of patients appropriately classified as changed according to statistical significance and meaningfulness.

Vision-related quality of life compared to generic measures in retinoblastoma survivors.

PURPOSE: To (1) Compare vision-related quality of life (VRQOL) in adolescent and young adult (AYA) unilateral versus bilateral retinoblastoma (RB) survivors using a vision-targeted measure and a generic health-related quality of life (HRQOL) measure and (2) Assess associations among VRQOL and generic HRQOL domains and overall QOL and estimate associations of the VRQOL and HRQOL domains with overall QOL. METHODS: The National Institute for Health (NIH) Toolbox® VRQOL instrument, PROMIS®-29 Profile v 2.1, and a single-item QOL measure were administered in a cross-sectional study of 101 RB survivors. Reliability for multi-item scales was estimated. Product-moment and Spearman rank correlation coefficients and stepwise ordinary least squares were used to measure associations of other variables with overall QOL. RESULTS: Significantly worse VRQOL was reported by bilateral than unilateral RB survivors. Cronbach's alpha coefficients for all VRQOL scales ranged from 0.83 to 0.95. Medium to large correlations were found between all NIH Toolbox® VRQOL scales and the PROMIS®-29 measures. Depression and ability to participate in social roles and activities from the PROMIS®-29 Profile accounted for 38% of the variance in overall QOL with the psychosocial domain of the NIH Toolbox® VRQOL explaining 16% of the variance. CONCLUSION: VRQOL is impaired in bilateral RB survivors. VRQOL is associated substantially with the PROMIS-29 generic HRQOL measure but has significant unique associations with overall QOL. The NIH Toolbox® VRQOL measure provides important information about the vision-related effects on daily life of AYA RB survivors.

Simulation study comparing analytical methods for single-item longitudinal patient-reported outcomes data.

PURPOSE: Efficient analytical methods are necessary to make reproducible inferences on single-item longitudinal ordinal patient-reported outcome (PRO) data. A thorough simulation study was performed to compare the performance of the semiparametric probabilistic index models (PIM) with a longitudinal analysis using parametric cumulative logit mixed models (CLMM). METHODS: In the setting of a control and intervention arm, we compared the power of the PIM and CLMM to detect differences in PRO adverse event (AE) between these groups using several existing and novel summary scores of PROs. For each scenario, PRO data were simulated using copula multinomial models. Comparisons were also exemplified using clinical trial data. RESULTS: On average, CLMM provided substantially greater power than the PIM to detect differences in PRO-AEs between the groups when the baseline-adjusted method was used, and a small advantage in power when using the baseline symptom as a covariate. CONCLUSION: Although the CLMM showed the best performance among analytical methods, it relies on assumptions difficult to verify and that might not be fulfilled in the real world, therefore our recommendation is the use of PIM models with baseline symptom as a covariate.

Crosswalking the National Institutes of Health Impact Stratification Score to the PEG.

OBJECTIVE: To crosswalk the National Institutes of Health (NIH) Pain Consortium's Research Task Force proposed Impact Stratification Score (ISS) to the PEG (Pain Intensity, Interference With Enjoyment of Life, Interference With General Activity) Scale. DESIGN: Cross-sectional data collected in 2021. Ordinary least squares regression analyses of ISS and PEG. SETTING: Amazon Mechanical Turk workers. PARTICIPANTS: 1931 adults with back pain with an average age of 41 (range, 19-77); 48% were female, 16% Hispanic, 7% non-Hispanic Black, 5% non-Hispanic Asian, and 71% non-Hispanic White (N=1931). INTERVENTIONS: Not applicable. MAIN OUTCOME MEASURES: The Patient-Reported Outcomes Measurement Information System (PROMIS)-29+2 v2.1 survey that includes the ISS, and the 3-item PEG. RESULTS: The ISS and PEG had a correlation coefficient of 0.74. The ISS accounted for 55% of the adjusted variance in the PEG and the standardized average deviation between observed and predicted scores (normalized mean absolute error) was 0.53. Likewise, the PEG explained 55% of the variance in the ISS with a normalized mean absolute error of 0.52. CONCLUSIONS: This study provides a crosswalk between the ISS and PEG that can be used to predict one from the other. The regression equations can facilitate comparisons in studies that use different measures.

Estimating individual health-related quality of life changes in low back pain patients.

BACKGROUND: There is a need to evaluate different options for estimating individual change in health-related quality of life for patients with low back pain. METHODS: Secondary analysis of data collected at baseline and 6 weeks later in a randomized trial of 749 adults with low back pain receiving usual medical care (UMC) or UMC plus chiropractic care at a small hospital at a military training site or two large military medical centers. The mean age was 31; 76% were male and 67% were White. The study participants completed the Patient-Reported Outcomes Measurement Information System (PROMIS®)-29 v 1.0 physical function, pain interference, pain intensity, fatigue, sleep disturbance, depression, anxiety, satisfaction with participation in social roles, physical summary, and mental health summary scores (T-scored with mean = 50 and standard deviation (SD) = 10 in the U.S. general population). RESULTS: Reliability estimates at the baseline ranged from 0.700 to 0.969. Six-week test-retest intraclass correlation estimates were substantially lower than these estimates: the median test-retest intraclass correlation for the two-way mixed-effects model was 0. 532. Restricting the test-retest reliability estimates to the subset who reported they were about the same as at baseline on a retrospective rating of change item increased the median test-retest reliability to 0.686. The amount of individual change that was statistically significant varied by how reliability was estimated, and which SD was used. The smallest change needed was found when internal consistency reliability and the SD at baseline were used. When these values were used, the amount of change needed to be statistically significant (p < .05) at the individual level ranged from 3.33 (mental health summary scale) to 12.30 (pain intensity item) T-score points. CONCLUSIONS: We recommend that in research studies estimates of the magnitude of individual change needed for statistical significance be provided for multiple reliability and standard deviation estimates. Whenever possible, patients should be classified based on whether they 1) improved significantly and perceived they got better, 2) improved significantly but did not perceive they were better, 3) did not improve significantly but felt they got better, or 4) did not improve significantly or report getting better.

Classifying patients with non-specific chronic low back pain using the impact stratification score in an online convenience sample.

BACKGROUND: In 2014, the National Institute of Health Pain Consortium's research task force (RTF) on research standards for chronic low back pain (CLBP) proposed the Impact Stratification Score (ISS) as a patient-reported outcome measure that could stratify patients by the impact CLBP has on their lives. This work compares three newly developed ISS-based classifications to the RTF's original to provide an optimal recommendation. METHODS: The online sample included 1226 individuals from Amazon's Mechanical Turk who indicated having non-specific CLBP, average age of 40, 49% female, and 67% White. Participants completed the PROMIS-29 v2.1 profile survey that contains the 9 ISS items as well the Roland-Morris Disability Questionnaire (RMDQ) and Graded Chronic Pain Scale (GCPS). Other items included high-impact chronic pain; not working due to health problems; overall health; and number of healthcare visits for back pain in the past 6 months. Three new classifications were created using quartiles (Classification 2), latent profile analysis (Classification 3), and one modeled after the GCPS (Classification 4). Classifications were subsequently compared to the RTF-proposed classification (Classification 1) on several concurrent and prognostic criteria. RESULTS: Classification 1 had three CLBP severity groups, four in Classification 2, three in Classification 3, and four in Classification 4. All novel classifications improved upon the original. Classification 2 performed best at minimizing the classification of those with negative outcomes into the lowest severity groups at baseline (e.g., 11% with RMDQ ≥ 7) and 6 months (e.g., 8.2% had fair/poor health). Classification 4 performed best at maximizing classification of those with negative outcomes into the most severe group concurrently (e.g., 100% had GCPS grade ≥ 2) and at 6 months (e.g., 100% with RMDQ ≥ 7). CONCLUSIONS: We developed three ISS-based classification schemes and tested them against several outcomes. All three improved upon the original scheme. While appearing more optimal than other classifications in the lowest severity groups, Classification 2 presents some considerations and limitations. Given that Classification 4 was an improvement at the lowest end of severity and was the best at the highest end, it is our tentative recommendation that this approach be adopted to classify individuals with non-specific CLBP.

Effects of Excluding Those Who Report Having "Syndomitis" or "Chekalism" on Data Quality: Longitudinal Health Survey of a Sample From Amazon's Mechanical Turk.

BACKGROUND: Researchers have implemented multiple approaches to increase data quality from existing web-based panels such as Amazon's Mechanical Turk (MTurk). OBJECTIVE: This study extends prior work by examining improvements in data quality and effects on mean estimates of health status by excluding respondents who endorse 1 or both of 2 fake health conditions ("Syndomitis" and "Chekalism"). METHODS: Survey data were collected in 2021 at baseline and 3 months later from MTurk study participants, aged 18 years or older, with an internet protocol address in the United States, and who had completed a minimum of 500 previous MTurk "human intelligence tasks." We included questions about demographic characteristics, health conditions (including the 2 fake conditions), and the Patient Reported Outcomes Measurement Information System (PROMIS)-29+2 (version 2.1) preference-based score survey. The 3-month follow-up survey was only administered to those who reported having back pain and did not endorse a fake condition at baseline. RESULTS: In total, 15% (996/6832) of the sample endorsed at least 1 of the 2 fake conditions at baseline. Those who endorsed a fake condition at baseline were more likely to identify as male, non-White, younger, report more health conditions, and take longer to complete the survey than those who did not endorse a fake condition. They also had substantially lower internal consistency reliability on the PROMIS-29+2 scales than those who did not endorse a fake condition: physical function (0.69 vs 0.89), pain interference (0.80 vs 0.94), fatigue (0.80 vs 0.92), depression (0.78 vs 0.92), anxiety (0.78 vs 0.90), sleep disturbance (-0.27 vs 0.84), ability to participate in social roles and activities (0.77 vs 0.92), and cognitive function (0.65 vs 0.77). The lack of reliability of the sleep disturbance scale for those endorsing a fake condition was because it includes both positively and negatively worded items. Those who reported a fake condition reported significantly worse self-reported health scores (except for sleep disturbance) than those who did not endorse a fake condition. Excluding those who endorsed a fake condition improved the overall mean PROMIS-29+2 (version 2.1) T-scores by 1-2 points and the PROMIS preference-based score by 0.04. Although they did not endorse a fake condition at baseline, 6% (n=59) of them endorsed at least 1 of them on the 3-month survey and they had lower PROMIS-29+2 score internal consistency reliability and worse mean scores on the 3-month survey than those who did not report having a fake condition. Based on these results, we estimate that 25% (1708/6832) of the MTurk respondents provided careless or dishonest responses. CONCLUSIONS: This study provides evidence that asking about fake health conditions can help to screen out respondents who may be dishonest or careless. We recommend this approach be used routinely in samples of members of MTurk.