Selecting patient-reported outcome measures: "what" and "for whom".

Patient-reported outcomes (PROMs) are becoming more widely implemented across health care for important reasons. However, with thousands of PROMs available and the science of psychometrics becoming more widely applied in health measurement, choosing the right ones to implement can be puzzling. This article provides a framework of the different types of PROMs by organizing them into 4 categories based upon "what" is being measured and "from whom" the questions are asked: (1) condition-specific and domain-specific, (2) condition-specific and global, (3) universal and global, and (4) universal and domain-specific. We delve deeper into each category with clinical examples. This framework can empower health care leaders and policymakers to make more informed decisions when selecting the best PROMs to implement, ensuring PROMs deliver on their potential to promote high quality, patient-centered care.

Seamless Integration of Computer-Adaptive Patient Reported Outcomes into an Electronic Health Record.

BACKGROUND: Patient-reported outcome (PRO) measures have become an essential component of quality measurement, quality improvement, and capturing the voice of the patient in clinical care. In 2004, the National Institutes of Health endorsed the importance of PROs by initiating the Patient-Reported Outcomes Measurement Information System (PROMIS), which leverages computer-adaptive tests (CATs) to reduce patient burden while maintaining measurement precision. Historically, PROMIS CATs have been used in a large number of research studies outside the electronic health record (EHR), but growing demand for clinical use of PROs requires creative information technology solutions for integration into the EHR. OBJECTIVES: This paper describes the introduction of PROMIS CATs into the Epic Systems EHR at a large academic medical center using a tight integration; we describe the process of creating a secure, automatic connection between the application programming interface (API) which scores and selects CAT items and Epic. METHODS: The overarching strategy was to make CATs appear indistinguishable from conventional measures to clinical users, patients, and the EHR software itself. We implemented CATs in Epic without compromising patient data security by creating custom middleware software within the organization's existing middleware framework. This software communicated between the Assessment Center API for item selection and scoring and Epic for item presentation and results. The middleware software seamlessly administered CATs alongside fixed-length, conventional PROs while maintaining the display characteristics and functions of other Epic measures, including automatic display of PROMIS scores in the patient's chart. Pilot implementation revealed differing workflows for clinicians using the software. RESULTS: The middleware software was adopted in 27 clinics across the hospital system. In the first 2 years of hospital-wide implementation, 793 providers collected 70,446 PROs from patients using this system. CONCLUSION: This project demonstrated the importance of regular communication across interdisciplinary teams in the design and development of clinical software. It also demonstrated that implementation relies on buy-in from clinical partners as they integrate new tools into their existing clinical workflow.

Translation and cultural adaptation of drug use stigma and HIV stigma measures among people who use drugs in Tanzania.

INTRODUCTION: People who use drugs (PWUD) experience stigma from multiple sources due to their drug use. HIV seroprevalence for PWUD in Tanzania is estimated to range from 18 to 25%. So, many PWUD will also experience HIV stigma. Both HIV and drug use stigma have negative health and social outcomes, it is therefore important to measure their magnitude and impact. However, no contextually and linguistically adapted measures are available to assess either HIV or drug use stigma among PWUD in Tanzania. In response, we translated and culturally adapted HIV and drug use stigma measures among Tanzanian PWUD and described that process in this study. METHODS: This was a cross-sectional study. We translated and adapted existing validated stigma measures by following a modified version of Wild's ten steps for translation and adaptation. We also added new items on stigmatizing actions that were not included in the original measures. Following translation and back translation, we conducted 40 cognitive debriefs among 19 PWUD living with and 21 PWUD not living with HIV in Dar es Salaam to assess comprehension of the original and new items. For challenging items, we made adaptations and repeated cognitive debriefs among ten new PWUD participants where half of them were living with HIV. RESULTS: Most of the original items (42/54, 78%), response options and all items with new 12 stigmatizing actions were understood by participants. Challenges included response options for a few items; translation to Swahili; and differences in participants' interpretation of Swahili words. We made changes to these items and the final versions were understood by PWUD participants. CONCLUSION: Drug use and HIV stigma measures can successfully be translated and culturally adapted among Tanzanian PWUD living with and without HIV. We are currently conducting research to determine the stigma measures' psychometric properties and we will report the results separately.

Monitoring Adverse Effects of Radiation Therapy in Patients With Head and Neck Cancer: The FACT-HN-RAD Patient-Reported Outcome Measure.

Importance: Patients undergoing radiation therapy (RT) for head and neck squamous cell carcinoma (HNSCC) experience a range of debilitating adverse effects (AEs). Patient-reported outcome (PRO) measures to quantify these AEs are a necessary and important component of health care; however, currently available PRO options often measure only disease-related symptoms or AEs of non-RT treatments. Objective: To develop a brief PRO measure of the most common AEs associated with RT for HNSCC. Design, Setting, and Participants: This was a qualitative study that followed the US Food and Drug Administration guidelines to develop a brief measure of patient-reported RT-related AEs (the Functional Assessment of Cancer Therapy-Head and Neck Radiotherapy [FACT-HN-RAD] measure). The study included (1) a literature review of clinical trials; (2) secondary analysis of retrospective concept elicitation interviews (CEIs); (3) electronic surveys of practicing radiation oncologists; (4) mapping of existing items to inform the development of the draft version of the measure; and (5) validation of content and face validity via patient cognitive interviews. Analysis was performed of CEI data and interviews with practicing radiation oncologists. Data analysis was conducted from July 1, 2022, to April 21, 2023. Exposures: Surveys and qualitative interviews. Main Outcomes and Measures: The most common patient-reported RT-related AEs among patients with HNSCC. Results: Of 19 CEI participants, 14 (mean [range] age, 67 [49-86] years; 12 [86%] men and 2 [14%] women) described RT-related AEs and were included in the secondary analysis. Eleven (79%) patients reported difficulty swallowing; 8 (57%), oral pain; 7 (50%), dry mouth; 7 (50%), weight loss; 6 (43%), skin burning; 5 (36%), loss of taste; 5 (36%), voice changes (36%); and 5 (36%), fatigue. Nine radiation oncologists (mean [range] time in practice, 8 [1-42] years; 5 [56%] men and 4 [44%] women) reported the most common AEs: 9 (100%) reported dysgeusia; 7 (78%), xerostomia; 7 (78%), mucositis or oral pain; 8 (89%), dysphagia or odynophagia; 6 (67%), dermatitis; and 3 (33%), fatigue. Together these data informed the development of an 8-item AE-focused measure of pain, dysphagia, xerostomia, dysgeusia, voice changes, dermatitis, fatigue, and weight loss. Cognitive interviews with 10 patients (mean [range] age, 61 [29-84] years; 8 [80%] men and 2 [20%] women) demonstrated strong face validity; all (100%) reported that the measure reflected their experience with RT and stated that the length of the questionnaire was "just right." Conclusions and Relevance: The 8-item FACT-HN-RAD measure captures the most common patient- and physician-reported AEs related to RT for HNSCC. This measure offers a means to serially monitor patient-reported treatment-related AEs and recovery over time in both clinical and research settings. Future work will evaluate the psychometric validity of the measure.

Evaluating a web-based personalized decision report for total knee or hip replacement: Lessons learned from patients.

RATIONALE: Patient-reported outcomes (PROs) are increasingly used in the context of clinical care, but evaluation of patients' perspectives of PRO-based applications in routine care remains limited. AIMS AND OBJECTIVES: This paper investigates patients' acceptability of a personalized web-based decision report for total knee or hip replacement and identifies opportunities to refine the report. METHOD: This qualitative evaluation was embedded in a pragmatic cluster randomized trial of the report. We interviewed 25 patients with knee and hip osteoarthritis about their experiences using the personalized decision report in the context of a surgical consultation. The web-based report contained current descriptive PRO scores of pain, function and general physical health; tailored predicted postoperative PRO scores (i.e., personalized likely outcomes based on actual knee or hip replacement outcomes of similar patients in a national registry); and information about alternative nonoperative treatments. Two trained researchers analysed the interview data qualitatively using a combination of inductive and deductive coding. RESULTS: We identified three major categories for evaluation: content of report, presentation of data in report and engagement with report. Patients generally liked the report overall but specifically valued different pages of the report based on where they were in the surgical decision-making process. Patients identified areas of confusion in data presentation related to graph orientation, terminology and interpretation of T-scores. Patients also highlighted support needs to meaningfully engage with the information in the report. CONCLUSION: Our findings highlight areas of opportunity to further refine this personalized web-based decision report and similar patient-facing PRO applications for routine clinical care. Specific examples include additional tailoring of reports via filterable web-based dashboards and scalable educational supports to facilitate more independent patient understanding and use.

Using bookmarking methods with orthopedic clinicians and patients with fractures produces score interpretation labels for patient-reported outcome measures.

OBJECTIVE: The objective of this study was to determine the patient-reported outcome measure (PROM) score ranges associated with descriptive labels (i.e., within normal limits, mild, moderate, severe) by using bookmarking methods with orthopedic clinicians and patients who have experienced a bone fracture. STUDY DESIGN AND SETTING: We created vignettes comprised of six items and responses from the Patient-Reported Outcomes Measurement Information System (PROMIS) Upper Extremity Function, Physical Function, and Pain Interference item banks reflecting different levels of severity. Two groups of patients with fractures (n = 11) and two groups of orthopedic clinicians (n = 16) reviewed the vignettes and assigned descriptive labels independently and then discussed as a group until reaching consensus via a videoconference platform. RESULTS: PROMIS Physical Function and Pain Interference thresholds (T = 50, 40, 25/30 and T = 50/55, 60, 65/70, respectively) for patients with bone fractures were consistent with the results from other patient populations. Upper Extremity thresholds were about 10 points (1 SD) more severe (T = 40, 30, 25/20) compared to the other measures. Patient and clinician perspectives were similar. CONCLUSION: Bookmarking methods generated meaningful score thresholds for PROMIS measures. These thresholds between severity categories varied by domain. Threshold values for severity represent important supplemental information to interpret PROMIS scores clinically.

The need for a measurement-based care professional practice guideline.

Professional practice guidelines (PPGs) are intended to promote a high level of professional practice and serve as an educational resource, providing pragmatic guidance in a clinical area for psychologists. Measurement-based care (MBC) is an evidence-based psychological practice with accumulating empirical support and alignment with patient-centered care. In connection with the American Psychological Association's Advisory Committee for Measurement-based Care and the Mental and Behavioral Health Registry, this article outlines various lines of support for the development and implementation of an MBC PPG. In addition to research evidence, we address the demonstrated need of this guideline across three domains: public benefit, professional guidance, and legal and regulatory issues. Consistent with the aspirational spirit of a PPG, this article proposes a draft PPG statement and highlights how an MBC PPG would improve service delivery, facilitate implementation of an evidence-based practice associated with symptom reduction, improved retention, and greater patient satisfaction, as well as create a framework that will better align changes in reimbursement models with patients' and providers' treatment goals. We also identify key future directions and critical gaps in MBC science and implementation that require attention. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

One report, multiple aims: orthopedic surgeons vary how they use patient-reported outcomes with patients.

PURPOSE: We conducted semi-structured qualitative interviews with surgeons to assess their goals for incorporating a patient-reported outcome measure (PROM)-based shared decision report into discussions around surgical and non-surgical treatment options for osteoarthritis of the knee and hip. METHODS: Surgeons actively enrolling patients into a study incorporating a standardized PROM-based shared decision report were invited to participate in a semi-structured interview lasting 30 min. Open-ended questions explored how the surgeon used report content, features that were helpful, confusing, or could be improved, and how use of the report fit into the surgeon's workflow. We used a conventional content analysis approach. RESULTS: Of the 16 eligible surgeons, 11 agreed to participate with 9 completing the interview and 2 withdrawing due to work demands. We identified 8 themes related to PROM-based report use: Acceptability, Patient Characteristics, Communication Goals, Useful Content, Not Useful Content, Challenges, Training Needs, and Recommended Improvements. Additional sub-themes emerged for Communication Goals (7) and Challenges (8). All surgeons shared positive feedback about using the report as part of clinical care. Whereas surgeons described the use of the report to achieve different goals, the most common uses related to setting expectations for post-surgical outcomes (89%) and educating patients (100%). CONCLUSION: Surgeons tailor their use of a PROM-based report with individual patients to achieve a range of aims. This study suggests multiple opportunities to further our understanding of the ways PROMs can be used in clinical practice. The way PROM information is visually displayed and multi-component reports are assembled can facilitate diverse aims.

Patients' perspectives on the benefits of feedback on patient-reported outcome measures in a web-based personalized decision report for hip and knee osteoarthritis.

BACKGROUND: Applications of patient-reported outcome measures (PROMs) for individual patient management are expanding with the support of digital tools. Providing PROM-based information to patients can potentially improve care experiences and outcomes through informing and activating patients. This study explored patients' perspectives on the benefits of receiving feedback on PROMs in the context of a web-based personalized decision report to guide care for their hip or knee osteoarthritis. METHODS: This qualitative descriptive interview study was nested in a pragmatic clinical trial of a personalized report, which includes descriptive PROM scores and predicted postoperative PROM scores. Patients completed a semi-structured interview within 6 weeks of an office visit with an orthopaedic surgeon. Only patients who reported receiving the report and reviewing it with the surgeon and/or a health educator were included. Data were iteratively analyzed using a combination of deductive and inductive coding strategies. RESULTS: Twenty-five patients aged 49-82 years (60% female, 72% surgical treatment decision) participated and described three primary benefits of the PROM feedback within the report: 1. Gaining Information About My Health Status, including data teaching new information, confirming what was known, or providing a frame of reference; 2. Fostering Communication Between Patient and Surgeon, encompassing use of the data to set expectations, ask and answer questions, and facilitate shared understanding; and 3. Increasing My Confidence and Trust, relating to the treatment outcomes, treatment decision, and surgeon. CONCLUSIONS: Patients identified actual and hypothetical benefits of receiving feedback on PROM scores in the context of a web-based decision report, including advantages for those who had already made a treatment decision before seeing the surgeon. Findings provide insight into patients' perspectives on how digital PROM data can promote patient-centered care. Results should be considered in the context of the homogeneous sample and complex trial. While participants perceived value in this personalized report, questions remain regarding best practices in patient-facing data presentation and engagement. TRIAL REGISTRATION: ClinicalTrials.gov, NCT03102580. Registered on 5 April 2017.

The development and initial validation of the PROMIS®+HF-27 and PROMIS+HF-10 profiles.

AIMS: Heart failure (HF) is a common and morbid condition impacting multiple health domains. We previously reported the development of the PROMIS®-Plus-HF (PROMIS+HF) profile measure, including universal and HF-specific items. To facilitate use, we developed shorter, PROMIS+HF profiles intended for research and clinical use. METHODS AND RESULTS: Candidate items were selected based on psychometric properties and symptom range coverage. HF clinicians (n = 43) rated item importance and clinical actionability. Based on these results, we developed the PROMIS+HF-27 and PROMIS+HF-10 profiles with summary scores (0-100) for overall, physical, mental, and social health. In a cross-sectional sample (n = 600), we measured internal consistency reliability (Cronbach's alpha and Spearman-Brown), test-retest reliability (intraclass coefficient; n = 100), known-groups validity via New York Heart Association (NYHA) class, and convergent validity with Kansas City Cardiomyopathy Questionnaire (KCCQ) scores. In a longitudinal sample (n = 75), we evaluated responsiveness of baseline/follow-up scores by calculating mean differences and Cohen's d and comparing with paired t-tests. Internal consistency was good to excellent (α 0.82-0.94) for all PROMIS+HF-27 scores and acceptable to good (α/Spearman-Brown 0.60-0.85) for PROMIS+HF-10 scores. Test-retest intraclass coefficients were acceptable to excellent (0.75-0.97). Both profiles demonstrated known-groups validity for the overall and physical health summary scores based on NYHA class, and convergent validity for nearly all scores compared with KCCQ scores. In the longitudinal sample, we demonstrated responsiveness for PROMIS+HF-27 and PROMIS+HF-10 overall and physical summary scores. For the PROMIS+HF overall summary scores, a group-based increase of 7.6-8.3 points represented a small to medium change (Cohen's d = 0.40-0.42). For the PROMIS+HF physical summary scores, a group-based increase of 5.0-5.9 points represented a small to medium change (Cohen's d = 0.29-0.35). CONCLUSIONS: The PROMIS+HF-27 and PROMIS+HF-10 profiles demonstrated good psychometric characteristics with evidence of responsiveness for overall and physical health. These new measures can facilitate patient-centred research and clinical care, such as improving care quality through symptom monitoring, facilitating shared decision-making, evaluating quality of care, assessing new interventions, and monitoring during the initiation and titration of guideline-directed medical therapy.

Motivational interviewing intervention for increasing physical activity and improving dietary behaviors: The Lupus Intervention Fatigue Trial protocol.

The Lupus Intervention Fatigue Trial (LIFT) is a prospective, randomized controlled trial to assess the effectiveness of a six-month motivational interviewing intervention program versus an educational control to reduce fatigue in persons with systematic lupus erythematosus (SLE). Participants are randomized using a stratified, 1:1 allocation design to the LIFT intervention or control arm. We plan to enroll 236 participants to achieve the target of 200 persons with six-month follow-up for the primary endpoint. Specific aims of this study are to evaluate the impact of the LIFT intervention on 1) self-reported measures of fatigue and 2) impact on accelerometer-measured physical activity. The primary study outcome is six-month change in fatigue from baseline, assessed by the Fatigue Severity Score (FSS). Additional outcomes include objective measures of physical activity, including non-sedentary behavior and moderate-to-vigorous activity (secondary outcome), and adherence to the LIFT dietary intervention, as assessed by nutrient density (diet quality) and recommended food groups/eating patterns (exploratory outcome) in persons with SLE. Intervention effectiveness will be assessed using an intention-to-treat two-arm comparison of six-month change in FSS, with one interim monitoring analysis. A two-sample independent group t-test will compare the six-month changes in FSS between the study arms. Intervention effect durability will be assessed 12-months after baseline (6 months after completion of the intervention). Enrollment began in June 2019 and is expected to end in June 2023. This study will inform future intervention strategies that promote physical activity and improved diet quality to reduce fatigue in persons with SLE.

Implementing an Application Programming Interface for PROMIS Measures at Three Medical Centers.

BACKGROUND: There is an increasing body of literature advocating for the collection of patient-reported outcomes (PROs) in clinical care. Unfortunately, there are many barriers to integrating PRO measures, particularly computer adaptive tests (CATs), within electronic health records (EHRs), thereby limiting access to advances in PRO measures in clinical care settings. OBJECTIVE: To address this obstacle, we created and evaluated a software integration of an Application Programming Interface (API) service for administering and scoring Patient-Reported Outcomes Measurement Information System (PROMIS) measures with the EHR system. METHODS: We created a RESTful API and evaluated the technical feasibility and impact on clinical workflow at three academic medical centers. RESULTS: Collaborative teams (i.e., clinical, information technology [IT] and administrative staff) performed these integration efforts addressing issues such as software integration as well as impact on clinical workflow. All centers considered their implementation successful based on the high rate of completed PROMIS assessments (between January 2016 and January 2021) and minimal workflow disruptions. CONCLUSION: These case studies demonstrate not only the feasibility but also the pathway for the integration of PROMIS CATs into the EHR and routine clinical care. All sites utilized diverse teams with support and commitment from institutional leadership, initial implementation in a single clinic, a process for monitoring and optimization, and use of custom software to minimize staff burden and error.

Strategies for Effective Implementation of Patient-Reported Outcome Measures in Arthroplasty Practice.

ABSTRACT: Patient-reported outcome measures (PROMs) will be an important component of real-world evidence, but best practices for capture and integration are not yet defined. While digital tools support patients, clinicians, and researchers to collect PROMs, PROM capture in clinical care remains challenging. We synthesized PROM implementation strategies that are successfully used by hundreds of arthroplasty surgeons and early PROM-adopter clinical systems. This information can guide health systems that are preparing to implement PROMs to inform clinical care, drive quality-improvement activities, and support reporting for payer-sponsored incentives. Specific information is included to guide each step in the implementation process, including selecting PROMs, redesigning office procedures, programming of information technology, and informing interpretation and clinical use. While no one solution exists for successful PROM implementation in total hip and total knee replacement, referred to as total joint replacement, these guidelines can inform optimal PROM deployment and the capture of complete PROM data. In addition, we outline future research that is needed to define methods for optimal patient engagement, technology infrastructure, and operational systems to seamlessly integrate PROM collection in clinical care.

Considerations to Support Use of Patient-Reported Outcomes Measurement Information System Pediatric Measures in Ambulatory Clinics.

OBJECTIVE: To identify challenges to the use of Patient-Reported Outcomes Measurement Information System (PROMIS) Pediatric measures in the ambulatory pediatric setting and possible solutions to these challenges. STUDY DESIGN: Eighteen semistructured telephone interviews of health system leaders, measurement implementers, and ambulatory pediatric clinicians were conducted. Five coders used applied thematic analysis to iteratively identify and refine themes in interview data. RESULTS: Most interviewees had roles in leadership or the implementation of patient-centered outcomes; 39% were clinicians. Some had experience using PROMIS clinically (44%) and 6% were considering this use. Analyses yielded 6 themes: (1) selection of PROMIS measures, (2) method of administration, (3) use of PROMIS Parent Proxy measures, (4) privacy and confidentiality of PROMIS responses, (5) interpretation of PROMIS scores, and (6) using PROMIS scores clinically. Within the themes, interviewees illuminated specific unique considerations for using PROMIS with children, including care transitions and privacy. CONCLUSIONS: Real-world challenges continue to hamper PROMIS use. Ongoing efforts to disseminate information about the integration of PROMIS measures in clinical care is critical to impacting the health of children.

Development and validation of an interpretive guide for PROMIS scores.

BACKGROUND: Accurate score interpretation is required for the appropriate use of patient-reported outcome measures in clinical practice. OBJECTIVE: To create and evaluate figures (T-score Maps) to facilitate the interpretation of scores on Patient-Reported Outcome Measurement Information System (PROMIS) measures. METHODS: For 21 PROMIS® short forms, item-level information was used to predict the most probable responses to items for the range of possible scores on each short form. Predicted responses were then "mapped" graphically along the range of possible scores. In a previously conducted longitudinal study, 1594 adult participants with chronic conditions (e.g., multiple sclerosis) responded to four items each of a subset of these PROMIS short forms. Participants' responses to these items were compared to those predicted by the T-score Maps. Difference scores were calculated between observed and predicted scores, and Spearman correlations were calculated. RESULTS: We constructed T-score Maps for 21 PROMIS short forms for adults and pediatric self- and parent-proxy report. For the clinical population, participants' actual responses were strongly correlated with their predicted responses (r = 0.762 to 0.950). The majority of predicted responses exactly matched observed responses (range 69.5% to 85.3%). CONCLUSION: Results support the validity of the predicted responses used to construct T-score Maps. T-score Maps are ready to be tested as interpretation aids in a variety of applications.

A reporting checklist for HealthMeasures' patient-reported outcomes: ASCQ-Me, Neuro-QoL, NIH Toolbox, and PROMIS.

BACKGROUND: ASCQ-Me®, Neuro-QoL™, NIH Toolbox®, and PROMIS®, which are health-related quality of life measures collectively known as HealthMeasures, have experienced rapid uptake in the scientific community with over 1700 peer-reviewed publications through 2018. Because of their proliferation across multiple research disciplines, there has been significant heterogeneity in the description and reporting of these measures. Here, we provide a publication checklist to promote standardization and comparability across different reports. This checklist can be used across all HealthMeasures systems. Checklist Development: Authors drafted a draft checklist, circulated among the HealthMeasures Steering Committee and PROMIS Health Organization until the members reached consensus. Checklist: The final checklist has 21 entries in 4 categories: measure details, administration, scoring, and reporting. Most entries (11) specify necessary measure-specific details including version number and administration language(s). Administration (4 entries) reminds authors to include details such as use of proxy respondents and the assessment platform. Scoring (3 entries) is necessary to ensure replication and cross-study comparisons. Reporting (3 entries) reminds authors to always report scores on the T-score metric. CONCLUSION: Consistent documentation is necessary to ensure transparent and reproducible methods and support the accumulation of evidence across studies. This checklist promotes standardization and completeness in documentation for ASCQ-Me, Neuro-QoL, PROMIS, and NIH Toolbox measures.

Integration of Patient-reported Outcomes in a Total Joint Arthroplasty Program at a High-volume Academic Medical Center.

INTRODUCTION: Despite widely appreciated barriers to successful clinical implementation, the literature regarding how to operationalize electronic health record-integrated patient-reported outcomes (PROs) remains sparse. We offer a detailed summary of the implementation of PROs into the standard of care at a major tertiary academic medical center. METHODS: Collection of four Patient-Reported Outcomes Measurement Information System computer adaptive tests was piloted in a large academic orthopaedic surgery ambulatory clinic starting in October 2016. The Patient-Reported Outcomes Measurement Information System computer adaptive tests (Physical Function, Pain Intensity, Pain Interference, and Ability to Return to Social Roles and Activities) were initially implemented as manual order sets to be administered before surgery through 2 years after surgery. Completion rate over time, mean time to completion for all PRO domains, and the overall distribution of symptom severity were used to evaluate the success of the pilot. A subsequent optimization and redesign of the pilot was conducted using tablets, automation of questionnaire deployment, and improved results review to address obstacles encountered during the pilot phase. RESULTS: Two thousand nine distinct joint arthroplasty patients (mean age = 65) completed at least one set of PRO assessments, with overall completion rates reaching 68% and mean completion time of 3 minutes. Focal points during the implementation process included engagement and training of staff, selection of an appropriate patient population and outcome measures, and user friendly data displays for patients and providers. CONCLUSION: Our pilot program successfully demonstrated that PROs can be administered, scored, and made immediately available within the electronic health record to patients and their providers with minimal disruption of clinical workflows. Although considerable operational and technological challenges remain, we found that the implementation of PROs in clinical care within an ambulatory practice at an academic medical center can be achieved through a constellation of several key factors.

Planning for patient-reported outcome implementation: Development of decision tools and practical experience across four clinics.

INTRODUCTION: Many institutions are attempting to implement patient-reported outcome (PRO) measures. Because PROs often change clinical workflows significantly for patients and providers, implementation choices can have major impact. While various implementation guides exist, a stepwise list of decision points covering the full implementation process and drawing explicitly on a sociotechnical conceptual framework does not exist. METHODS: To facilitate real-world implementation of PROs in electronic health records (EHRs) for use in clinical practice, members of the EHR Access to Seamless Integration of Patient-Reported Outcomes Measurement Information System (PROMIS) Consortium developed structured PRO implementation planning tools. Each institution pilot tested the tools. Joint meetings led to the identification of critical sociotechnical success factors. RESULTS: Three tools were developed and tested: (1) a PRO Planning Guide summarizes the empirical knowledge and guidance about PRO implementation in routine clinical care; (2) a Decision Log allows decision tracking; and (3) an Implementation Plan Template simplifies creation of a sharable implementation plan. Seven lessons learned during implementation underscore the iterative nature of planning and the importance of the clinician champion, as well as the need to understand aims, manage implementation barriers, minimize disruption, provide ample discussion time, and continuously engage key stakeholders. CONCLUSIONS: Highly structured planning tools, informed by a sociotechnical perspective, enabled the construction of clear, clinic-specific plans. By developing and testing three reusable tools (freely available for immediate use), our project addressed the need for consolidated guidance and created new materials for PRO implementation planning. We identified seven important lessons that, while common to technology implementation, are especially critical in PRO implementation.

The expansion and validation of a new upper extremity item bank for the Patient-Reported Outcomes Measurement Information System® (PROMIS).

BACKGROUND: The Patient-Reported Outcomes Measurement Information System® (PROMIS) includes a Physical Function (PF) item bank and an Upper Extremity (UE) item bank, which is composed of a subset of items from the PF bank. The UE item bank has few items and known ceiling effects. Therefore, this study aimed to expand the item bank to assess a wider range of functioning. With the additional content, other psychometric properties-improved content validity, item bank depth, range of measurement, and score reliability-were also evaluated. We convened an expert panel to review potential items, and then conducted psychometric analyses on both extant and newly-collected data. RESULTS: Expert focus groups reviewed the PF item bank for items that were "sufficiently" related to upper extremity functioning for inclusion in the expanded UE item bank. The candidate item bank was quantitatively evaluated in a new sample of 600 people. The final items were calibrated in an aggregated dataset (n = 11,635) from two existing datasets, and the newly collected sample. The original UE item bank included 15 items. After expert review and quantitative evaluation, 31 items were added. The combined 46 items were calibrated using item response theory (IRT). Then computer adaptive tests (CATs) were simulated based off of the psychometric results. These indicated that the new UE item bank has an extended measurement range compared to the original version. CONCLUSIONS: The expanded PROMIS UE item bank assesses a wider range of upper extremity functioning compared to the initial UE item bank. However, ceiling effects remain a concern for unimpaired groups. The new UE item bank is recommended for individuals with known or suspected upper extremity limitations.

Establishing clinically-relevant terms and severity thresholds for Patient-Reported Outcomes Measurement Information System® (PROMIS®) measures of physical function, cognitive function, and sleep disturbance in people with cancer using standard setting.

PURPOSE: Patient-Reported Outcomes Measurement Information System® (PROMIS®) physical function, cognitive function, and sleep disturbance measures are increasingly used in cancer care. However, there is limited guidance for interpreting the clinical meaning of scores. This study aimed to apply bookmarking, a standard setting methodology, to identify PROMIS score thresholds in the context of cancer care. METHODS: Using item parameters, we constructed vignettes of five items covering the range of possible scores. Focus groups were held with cancer care providers and people with cancer. Terminology for categorizing levels of severity was explored. Participants rank ordered vignettes by severity and then placed bookmarks between vignettes representing different levels of severity. Group discussion was held until consensus on bookmark placement was reached. RESULTS: Clinicians selected "within normal limits," "mild," "moderate," and "severe" to describe levels of severity. Both patients and clinicians were able to apply these labels, but there was not unanimous support for any set of descriptors. Clinicians and patients agreed on all severity thresholds for sleep disturbance. For cognitive and physical function, clinicians and patients agreed on the threshold between "within normal limits" and "mild." However, patients required greater dysfunction than clinicians before applying "moderate" and "severe" labels. CONCLUSIONS: Bookmarking can be applied to develop provisional score interpretation for PROMIS measures. Patients and clinicians were frequently consistent in their bookmark placement. When there was variance, patients required more dysfunction before assigning more severity. Additional research with other cancer samples is needed to evaluate the replicability and generalizability of our findings.