Risk of significant functional impairment across cancer diagnosis and care continuum.

BACKGROUND: The authors examined baseline physical functional (PF) impairment among cancer outpatients in the National Cancer Institute Cancer Moonshot study Northwestern University Improving the Management of Symptoms During and Following Cancer Treatment (NU IMPACT). They hypothesized that PF impairment, measured with the Patient Reported Outcome Measurement Information System-Physical Function (PROMIS-PF) survey, would (1) be common and more prevalent for patients receiving treatment compared with no treatment and (2) differ across tumor types, independent of cancer continuum phase. METHODS: Adults who were diagnosed with cancer in NU IMPACT (n = 2273) were sampled, and their PROMIS-PF scores were compared across tumor types and cancer continuum (curative, noncurative, or no treatment), with scores ≤40 indicating moderate-severe impairment. Multivariable logistic regression models were used to evaluate the relation among patient and cancer factors and PF scores using a 95% confidence interval. RESULTS: Forty percent of the surveyed patients reported moderate-severe PF impairment. Patients with melanoma reported the least impairment, and those with lung cancer were 6.5 times more likely to have moderate-severe impairment (95% confidence interval, 2.393-17.769). The noncurative group was 1.5 times more likely to have moderate-severe impairment (95% confidence interval, 1.045-2.145; mean score, 43; p < .001) than the curative (mean score, 6) and no treatment (mean score, 48) groups. One-third of those who reported PF impairment also had significant pain and/or fatigue. CONCLUSIONS: A sizeable minority experienced PF impairment across tumor types for which pain and/or fatigue co-occurred, particularly in the noncurative group. The PROMIS-PF survey effectively identified variations in physical function. Future studies will explore how screening for PF impairment can be used to refer patients for appropriate cancer rehabilitation services.

Quantifying Patient Expectations for Total Knee Arthroplasty: Expectations for Improvement Are Greater Than Minimal Clinically Important Difference.

BACKGROUND: Achieving a minimal clinically important difference (MCID) in patient-reported outcomes following total knee arthroplasty (TKA) is common, yet up to 20% patient dissatisfaction persists. Unmet expectations may explain post-TKA dissatisfaction. No prior studies have quantified patient expectations using the same patient-reported outcome metric as used for MCID to allow direct comparison. METHODS: This was a prospective study of patients undergoing TKA with 5 fellowship-trained arthroplasty surgeons at one academic center. Baseline Patient-Reported Outcomes Measurement Information System (PROMIS) Physical Function (PF) and Pain Interference (PI) domains were assessed. Expected PROMIS scores were determined by asking patients to indicate the outcomes they were expecting at 12 months postoperatively. Predicted scores were generated from a predictive model validated in the Function and Outcomes Research for Comparative Effectiveness in Total Joint Replacement (FORCE-TJR) dataset. T-tests were used to compare baseline, expected, and predicted PROMIS scores. Expected scores were compared to PROMIS MCID values obtained from the literature. Regression models were used to identify patient characteristics associated with high expectations. RESULTS: There were 93 patients included. Mean age was 67 years (range, 30 to 85) and 55% were women. Mean baseline PROMIS PF and PI was 34.4 ± 6.7 and 62.2 ± 6.4, respectively. Patients expected significant improvement for PF of 1.9 times the MCID (MCID = 11.3; mean expected improvement = 21.6, 95% confidence interval [CI] 19.6 to 23.5, P < .001) and for PI of 2.3 times the MCID (MCID = 8.9; mean expected improvement = 20.6, 95% CI 19.1-22.2, P < .001). Predicted scores were significantly lower than expected scores (mean difference = 9.5, 95% CI 7.7 to 11.3, P < .001). No unique patient characteristics were associated with high expectations (P > .05). CONCLUSIONS: To our knowledge, this study is the first to quantify preoperative patient expectations using the same metric as MCID to allow for direct comparison. Patient expectations for improvement following TKA are ∼2× greater than MCID and are significantly greater than predicted outcome scores. This discrepancy challenges currently accepted standards of success after TKA and indicates a need for improved expectation setting prior to surgery.

Agreement of the American Academy of Orthopaedic Surgeons Appropriate Use Criteria With Treatment Recommendations From Arthroplasty Surgeons.

Background: The American Academy of Orthopaedic Surgeons (AAOS) Appropriate Use Criteria (AUC) for Surgical Management of Osteoarthritis of the Knee (2016) and Management of Osteoarthritis of the Hip (2017) are intended to provide treatment recommendations for osteoarthritis (OA). This study examined the agreement of AUC appropriateness classifications with arthroplasty surgeon recommendations for total knee arthroplasty (TKA) and total hip arthroplasty (THA). Methods: The cohort included 558 OA patients (397 knee, 161 hip) referred to a specialty arthroplasty clinic. Surgeons completed the online AAOS AUC patient profiles to generate appropriateness ratings. Surgeons' recommendations for treatment were recorded. We performed univariate and bivariate analyses to evaluate relationships between AUC appropriateness and surgeon recommendations. Results: The knee OA AUC classified TKA as "appropriate" for 309 (77.8%) of the 397 knee OA patients. Surgeons recommended TKA for 123 (31.0%), resulting in 46.8% (n = 186) higher rate of "appropriate" classification by AUC than TKA recommendation by surgeons. Weighted Cohen's κ demonstrated slight agreement (κ = 0.06, 95% confidence interval: 0.04, 0.09) between AUC appropriateness and surgeon TKA recommendation. The hip OA AUC classified THA as "appropriate" for 98 (60.9%) of the 161 hip OA patients. Surgeons recommended THA for 76 (47.2%), resulting in 13.7% (n = 22) higher rate of "appropriate" classification by AUC than THA recommendation by surgeons. Weighted Cohen's κ demonstrated moderate agreement (κ = 0.47, 95% confidence interval: 0.37, 0.57) between the AUC appropriateness classification and the surgeon's THA recommendation. Conclusions: AAOS AUC guidelines indicated surgical appropriateness significantly more than arthroplasty surgeons. AUC agreed slightly with surgeons for TKA and moderately for THA.

Assessment of learning health system science competency in the equity and justice domain.

Seven knowledge domains were originally defined for the learning health system (LHS) scientist. To assess proficiency in each of these domains, we developed and published an assessment tool for use by emerging LHS scientists and training programs. (LHS, October 2022). In mid-2022, the AHRQ adopted an eighth LHS knowledge domain, Equity and Justice. The addition of this eighth domain emphasizes the importance and centrality of equity in the LHS and improvement science. To extend our prior LHS competency assessment, we developed a proficiency assessment for the new equity and justice domain. Content experts and trainees iteratively defined, reviewed, and edited the assessment criteria. The items were developed by trainees and experts at one LHS training center with experience conducting research focused on healthcare inequities among marginalized populations. The proficiency assessment criteria for the Equity domain apply the same four levels of mastery: "no exposure," "foundational awareness," "emerging," and "proficient" as were used for original competencies. LHS training programs can use these proficiency criteria to monitor skills among emerging scientists across the eight domains, with particular attention to equity and justice.

Social risk and patient-reported outcomes after total knee replacement: Implications for Medicare policy.

OBJECTIVE: To determine whether county-level or patient-level social risk factors are associated with patient-reported outcomes after total knee replacement when added to the comprehensive joint replacement risk-adjustment model. DATA SOURCES AND STUDY SETTING: Patient and outcomes data from the Function and Outcomes Research for Comparative Effectiveness in Total Joint Replacement cohort were merged with the Social Vulnerability Index from the Centers for Disease Control and Prevention. STUDY DESIGN: This prospective longitudinal cohort measured the change in patient-reported pain and physical function from baseline to 12 months after surgery. The cohort included a nationally diverse sample of adult patients who received elective unilateral knee replacement between 2012 and 2015. DATA COLLECTION/EXTRACTION METHODS: Using a national network of over 230 surgeons in 28 states, the cohort study enrolled patients from diverse settings and collected one-year outcomes after the surgery. Patients <65 years of age or who did not report outcomes were excluded. PRINCIPAL FINDINGS: After adjusting for clinical and demographic factors, we found patient-reported race, education, and income were associated with patient-reported pain or functional scores. Pain improvement was negatively associated with Black race (CI = -8.71, -3.02) and positively associated with higher annual incomes (≥$45,00) (CI = 0.07, 2.33). Functional improvement was also negatively associated with Black race (CI = -5.81, -0.35). Patients with higher educational attainment (CI = -2.35, -0.06) reported significantly less functional improvement while patients in households with three adults reported greater improvement (CI = 0.11, 4.57). We did not observe any associations between county-level social vulnerability and change in pain or function. CONCLUSIONS: We found patient-level social factors were associated with patient-reported outcomes after total knee replacement, but county-level social vulnerability was not. Our findings suggest patient-level social factors warrant further investigation to promote health equity in patient-reported outcomes after total knee replacement.

Engaging primary care clinicians in the selection of implementation strategies for toddler social-emotional health promotion in community health centers.

BACKGROUND: Social-emotional risk for subsequent behavioral health problems can be identified at toddler age, a period where prevention has a heightened impact. This study aimed to meaningfully engage pediatric clinicians, given the emphasis on health promotion and broad reach of primary care, to prepare an Implementation Research Logic Model to guide the implementation of a screening and referral process for toddlers with elevated social-emotional risk. METHOD: Using an adaptation of a previously published community partner engagement method, six pediatricians from community health centers (CHCs) comprised a Clinical Partner Work Group. The group was engaged in identifying determinants (barriers/facilitators), selecting and specifying strategies, strategy-determinant matching, a modified Delphi approach for strategy prioritization, and user-centered design methods. The data gathered from individual interviews, two group sessions, and a follow-up survey resulted in a completed Implementation Research Logic Model. RESULTS: The Clinical Partner Work Group identified 16 determinants, including barriers (e.g., patient access to electronic devices) and facilitators (e.g., clinician buy-in). They then selected and specified 14 strategies, which were prioritized based on ratings of feasibility, effectiveness, and priority. The highest-rated strategies (e.g., integration of the screener into the electronic health record) provided coverage of all identified barriers and comprised the primary implementation strategy "package" to be used and tested. CONCLUSIONS: Clinical partners provided important context and insights for implementation strategy selection and specification to support the implementation of social-emotional risk screening and referral in pediatric primary care. The methodology described herein can improve partner engagement in implementation efforts and increase the likelihood of success. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Optimizing PROM Implementation in Orthopedic Clinics for Longitudinal Outcome Monitoring: Lessons from a Multisite Study.

BACKGROUND: Systematic patient-reported outcome measure (PROM) collection is challenging for clinics, particularly when patients are not in the office. The Arthritis care through Shared Knowledge (ASK) study deployed multimodal approaches to collect PROMs using a clinical microsystem framework. CONCEPTUAL MODEL: Informed by the clinical microsystem model, the authors coached 12 orthopedic practices to implement shared processes to support best practices for PROM collection and use. Orthopedic sites collected PROMs from new patients before the first office visit; patients completed the PROM from home via an online assessment in a personalized e-mail. Site staff placed follow-up phone reminders. At 6 and 12 months after the visit, PROMs were collected from home, prompted by an e-mail or phone call. PERFORMANCE OUTCOMES: Of the 25,043 new patients identified by clinical sites during the study enrollment, approximately 60% completed a pre-visit PROM-36.6% completed the online PROM after receiving a single automated e-mail, and an additional 31.1% completed the PROM after receiving a text, an e-mail, or a phone call from the staff. The remaining 32.2% of PROMs were collected on arrival at the office. Of patients completing PROMs, 11,140 were eligible to participate in longitudinal collection, and 51.3% consented. Of these, approximately 84% completed a 6-month survey, more than 83% completed a 12-month survey, and more than 91% completed either a 6-month PROM, a 12-month PROM, or both. LEARNING: This study illustrates that a multimodal approach to PROM collection using a clinical microsystem approach sustainably supports PROM completion rates. Further efforts are needed to define strategies to engage all patients in understanding and reporting PROMs to inform their care.

Evaluating a web-based personalized decision report for total knee or hip replacement: Lessons learned from patients.

RATIONALE: Patient-reported outcomes (PROs) are increasingly used in the context of clinical care, but evaluation of patients' perspectives of PRO-based applications in routine care remains limited. AIMS AND OBJECTIVES: This paper investigates patients' acceptability of a personalized web-based decision report for total knee or hip replacement and identifies opportunities to refine the report. METHOD: This qualitative evaluation was embedded in a pragmatic cluster randomized trial of the report. We interviewed 25 patients with knee and hip osteoarthritis about their experiences using the personalized decision report in the context of a surgical consultation. The web-based report contained current descriptive PRO scores of pain, function and general physical health; tailored predicted postoperative PRO scores (i.e., personalized likely outcomes based on actual knee or hip replacement outcomes of similar patients in a national registry); and information about alternative nonoperative treatments. Two trained researchers analysed the interview data qualitatively using a combination of inductive and deductive coding. RESULTS: We identified three major categories for evaluation: content of report, presentation of data in report and engagement with report. Patients generally liked the report overall but specifically valued different pages of the report based on where they were in the surgical decision-making process. Patients identified areas of confusion in data presentation related to graph orientation, terminology and interpretation of T-scores. Patients also highlighted support needs to meaningfully engage with the information in the report. CONCLUSION: Our findings highlight areas of opportunity to further refine this personalized web-based decision report and similar patient-facing PRO applications for routine clinical care. Specific examples include additional tailoring of reports via filterable web-based dashboards and scalable educational supports to facilitate more independent patient understanding and use.

Evaluation of emergency department treat-and-release encounters after major gastrointestinal surgery.

BACKGROUND AND OBJECTIVES: Emergency department (ED) utilization after gastrointestinal cancer operations is poorly characterized. Our study objectives were to determine the incidence of, reasons for, and predictors of ED treat-and-release encounters after gastrointestinal cancer operations. METHODS: Patients who underwent elective esophageal, hepatobiliary, gastric, pancreatic, small intestinal, or colorectal operations for cancer were identified in the 2015-2017 Healthcare Cost and Utilization Project State Inpatient and State Emergency Department Databases for New York, Maryland, and Florida. The primary outcomes were the incidence of ED treat-and-release encounters and readmissions within 30 days of discharge. RESULTS: Among 51 527 patients at 406 hospitals, 4047 (7.9%) had an ED treat-and-release encounter, and 5573 (10.8%) had an ED encounter with readmission. In total, 40.7% of ED encounters were treat-and-release encounters. ED treat-and-release encounters were most frequently for pain (12.0%), device/ostomy complaints (11.7%), or wound complaints (11.4%). ED treat-and-release encounters predictors included non-Hispanic Black race/ethnicity (odds ratio [OR] 1.24, 95% confidence interval [CI] 1.12-1.37) and Medicare (OR 1.27, 95% CI 1.16-1.40) or Medicaid (OR 1.82, 95% CI 1.62-2.40) coverage. CONCLUSIONS: ED treat-and-release encounters are common after major gastrointestinal operations, making up nearly half of postdischarge ED encounters. The reasons for ED treat-and-release encounters differ from those for ED encounters with readmissions.

Barriers and Facilitators to Implementing Patient-Reported Outcome Monitoring in Gastrointestinal Surgery.

INTRODUCTION: More than 30% of patients experience complications after major gastrointestinal (GI) surgery, many of which occur after discharge when patients and families must assume responsibility for monitoring. Patient-reported outcomes (PROs) have been proposed as a tool for remote monitoring to identify deviations in recovery, and recognize and manage complications earlier. This study's objective was to characterize barriers and facilitators to the use of PROs as a patient monitoring tool following GI surgery. METHODS: We conducted semistructured interviews with GI surgery patients and clinicians (surgeons, nurses, and advanced practitioners). Patients and clinicians were asked to describe their experience using a PRO monitoring system in three surgical oncology clinics. Using a phenomenological approach, research team dyads independently coded the transcripts using an inductively developed codebook and the constant comparative approach with differences reconciled by consensus. RESULTS: Ten patients and five clinicians participated in the interviews. We identified four overarching themes related to functionality, workflow, meaningfulness, and actionability. Functionality refers to barriers faced by clinicians and patients in using the PRO technology. Workflow represents problematic integration of PROs into the clinical workflow and need for setting expectations with patients. Meaningfulness refers to lack of patient and clinician understanding of the impact of PROs on patient care. Finally, actionability reflects barriers to follow-up and practical use of PRO data. CONCLUSIONS: While use of PRO systems for postoperative patient monitoring have expanded, significant barriers persist for both patients and clinicians. Implementation enhancements are needed to optimize functionality, workflow, meaningfulness, and actionability.

How Much Perioperative Pain and Dysfunction Underlie the HOOS JR and KOOS JR?

BACKGROUND: The Hip Disability and Osteoarthritis Outcome Score Joint Replacement (HOOS JR) and Knee Injury and Osteoarthritis Outcome Score Joint Replacement (KOOS JR) scores represent pain and dysfunction as a single number ranging from 0 (extreme pain and dysfunction) to 100 (no pain or functional limitations). However, scores between 0 and 100 lack a simple interpretation because they reflect varying combinations of pain levels and dysfunction. Given that most adverse events and improvement occur within the first 90 days after surgery, a deeper understanding of the level of pain and dysfunction may reveal missed opportunities for patient care. QUESTIONS/PURPOSES: (1) What does a given preoperative or postoperative HOOS JR and KOOS JR score indicate about pain and ability to perform daily activities? (2) How much of a change in score (that is, delta) is needed to indicate significant improvement in pain control and daily functioning? METHODS: The Michigan Arthroplasty Registry Collaborative Quality Initiative contains more than 95% of THAs and TKAs performed in Michigan. Between January 2017 and March 2019, 84,175 people in the registry underwent primary THA or TKA and were potentially eligible for this retrospective, comparative study of the first 90 postoperative days. Eighty-four percent (70,608 of 84,175) were excluded because their surgeons did not attain a target survey collection proportion of 70% and another 6% (5042) were missing covariate information or surveys, leaving 10% (8525) for analysis. The mean age and percentage of women were 65 ± 11 years and 55% (2060 of 3716), respectively, for patients undergoing THA and 67 ± 9 years and 61% (2936 of 4809), respectively, for those undergoing TKA. There were no clinically meaningful differences between patients who were analyzed and those who were excluded except for lower representation of non-White patients in the analyzed group. For interpretation, patient responses to Question 7 (pain) and Question 6 (function) from the Patient-Reported Outcomes Measurement Information System global items (PROMIS-10) were dichotomized into "much pain" (rating of pain 4 to 10 of 10) versus "less pain" (rating of ≤ 3) and "good function" (able to perform most activities) versus "poor function" (not able to perform most activities) and combined into four pain-function categories. We examined the mean preoperative and postoperative HOOS JR and KOOS JR scores for each pain-function category, adjusted for patient characteristics. We calculated the size of the delta associated with an increase to a more favorable category postoperatively (versus staying in the same or worse category) via multivariable logistic regression that controlled for patient characteristics. RESULTS: Patients in the least favorable "much pain, poor function" category preoperatively had adjusted mean scores of 40 (95% confidence interval 39 to 41) for both the HOOS JR and KOOS JR. Those with mixed levels of pain and function had mean scores between 46 and 55. Those in the most favorable "less pain, good function" category had means of 60 (95% CI 58 to 62) and 59 (95% CI 58 to 61) for the HOOS JR and KOOS JR, respectively. The adjusted delta to achieve a pain level of ≤ 3 or the ability to perform most activities was 30 (95% CI 26 to 36) on the HOOS JR and 27 (95% CI 22 to 29) on the KOOS JR scales. CONCLUSION: These adjusted means of the HOOS JR and KOOS JR provide context for understanding the levels of pain and dysfunction for individuals as well for patients reported in other studies. Potential quality improvement efforts could include tracking the proportion of patients with THA or TKA who achieved a sufficient delta to attain pain levels of ≤ 3 or the ability to perform most activities. Future studies are needed to understand pain and function represented by the HOOS JR and KOOS JR at 1 to 2 years, how these may differ by patient subgroups, and whether scores can be improved through quality improvement efforts. LEVEL OF EVIDENCE: Level III, therapeutic study.

Equity-Driven Implementation of Patient-Reported Outcome Measures in Musculoskeletal Care: Advancing Value for All.

ABSTRACT: The clinical use of patient-reported outcome measures (PROMs) in musculoskeletal care is expanding, encompassing both individual patient management and population-level applications. However, without thoughtful implementation, we risk introducing or exacerbating disparities in care processes or outcomes. We outline examples of opportunities, challenges, and priorities throughout PROM implementation to equitably advance value-based care at both the patient and population level. Balancing standardization with tailored strategies can enable the large-scale implementation of PROMs while optimizing care processes and outcomes for all patients.

One report, multiple aims: orthopedic surgeons vary how they use patient-reported outcomes with patients.

PURPOSE: We conducted semi-structured qualitative interviews with surgeons to assess their goals for incorporating a patient-reported outcome measure (PROM)-based shared decision report into discussions around surgical and non-surgical treatment options for osteoarthritis of the knee and hip. METHODS: Surgeons actively enrolling patients into a study incorporating a standardized PROM-based shared decision report were invited to participate in a semi-structured interview lasting 30 min. Open-ended questions explored how the surgeon used report content, features that were helpful, confusing, or could be improved, and how use of the report fit into the surgeon's workflow. We used a conventional content analysis approach. RESULTS: Of the 16 eligible surgeons, 11 agreed to participate with 9 completing the interview and 2 withdrawing due to work demands. We identified 8 themes related to PROM-based report use: Acceptability, Patient Characteristics, Communication Goals, Useful Content, Not Useful Content, Challenges, Training Needs, and Recommended Improvements. Additional sub-themes emerged for Communication Goals (7) and Challenges (8). All surgeons shared positive feedback about using the report as part of clinical care. Whereas surgeons described the use of the report to achieve different goals, the most common uses related to setting expectations for post-surgical outcomes (89%) and educating patients (100%). CONCLUSION: Surgeons tailor their use of a PROM-based report with individual patients to achieve a range of aims. This study suggests multiple opportunities to further our understanding of the ways PROMs can be used in clinical practice. The way PROM information is visually displayed and multi-component reports are assembled can facilitate diverse aims.

Incorporating Expected Outcomes Into Clinical Decision-Making for Total Knee Arthroplasty.

OBJECTIVE: Expected outcomes (e.g., expected survivorship after a cancer treatment) have improved decision-making around treatment options in many clinical fields. Our objective was to evaluate the effect of expected values of 3 widely available total knee arthroplasty (TKA) outcomes (risk of serious complications, time to revision, and improvement in pain and function at 2 years after surgery) on clinical recommendation of TKA. METHODS: The RAND/University of California Los Angeles appropriateness criteria method was used to evaluate the role of the 3 expected outcomes in clinical recommendation of TKA. The expected outcomes were added to 5 established preoperative factors from the modified Escobar appropriateness criteria. The 8 indication factors were used to develop 279 clinical scenarios, and a panel of 9 clinicians rated the appropriateness of TKA for each scenario as inappropriate, inconclusive, and appropriate. Classification tree analysis was applied to these ratings to identify the most influential of the 8 factors in discriminating TKA appropriateness classifications. RESULTS: Ratings for the 279 appropriateness scenarios deemed 34.4% of the scenarios as appropriate, 40.1% as inconclusive, and 25.5% as inappropriate. Classification tree analyses showed that expected improvement in pain and function and expected time to revision were the most influential factors that discriminated among the TKA appropriateness classification categories. CONCLUSION: Our results showed that clinicians would use expected postoperative outcome factors in determining appropriateness for TKA. These results call for further work in this area to incorporate estimates of expected pain/function and revision outcomes into clinical practice to improve decision-making for TKA.

Development of a learning health system science competency assessment to guide training and proficiency assessment.

Introduction: Learning health systems (LHS) science is fundamentally a transdisciplinary field. To capture the breadth of the competencies of an LHS scientist, AHRQ and national experts defined a series of 42 competencies across seven domains that support success. Clinicians, researchers, and leaders who are new to the LHS field can identify and prioritize proficiency development among these domains. In addition, existing leaders and researchers will assemble teams of experts who together represent the LHS science domains. To serve LHS workforce development and proficiency assessment, the AHRQ-funded ACCELERAT K12 training program recruited domain experts and trainees to define and operationalize items to include in an LHS Competency Assessment to support emerging and existing LHS scientists in prioritizing and monitoring proficiency development. Methods: Sequential interviews with 18 experts iteratively defined skills and tasks to illustrate stage in proficiency, and its progression, for each of 42 competencies in the seven LHS expertise domains: systems science; research questions and standards of scientific evidence; research methods; informatics; ethics of research and implementation in health systems; improvement and implementation science; and engagement, leadership, and research management. An educational assessment expert and LHS scientist refined the assessment criteria at each stage to use parallel language across domains. Last, current trainees reviewed and pilot tested the assessment and the LHS Competency Assessment was further refined using their feedback. The assessment framework was informed by Bloom's revised taxonomy of educational objectives (Anderson and Krathwohl, A taxonomy for learning, teaching, and assessing: A revision of Bloom's taxonomy of educational objectives, 2001) where learning progresses from recalling, defining, understanding, and awareness at the lower levels of the taxonomy, to applying and adopting and finally to creating, designing, and critiquing at the upper levels of the taxonomy. We also developed assessment criteria that could be used for longer term assessment of direct performance. Van der Vleuten et al. (Best Pract Res Clin Obstetr Gynaecol. 2010;24(6):703-719) define longer term direct assessment methods as assessment that occurs over a period ranging from weeks to even years and involves multiple assessment methods and exposure to the learner's work over an extended period. Results: This experience report describes the content of the LHS Competency Assessment. For each domain and competency, the assessment lists examples of evidence to support expertise at each level of proficiency: no exposure; foundational (awareness/understanding); emerging (early application); and proficient (application with a high level of skill). Trainees begin with baseline standard assessment tables, where they can indicate no exposure or mark the foundational and emerging skills with which they have competence. For domains where foundational and emerging skills have been achieved, users can move on to assessment tables that list evidence of proficiency. Conclusion: The LHS Competency Assessment offers consistent, graded criteria across the seven LHS domains to guide trainees and mentors to evaluate progress from no experience to foundational knowledge, emerging proficiency, and proficiency. The assessment can also be used to design training and mentoring for those newly exposed to LHS science and for those with key expertise who wish to expand LHS expertise.

Training the next generation of learning health system scientists.

Introduction: The learning health system (LHS) aligns science, informatics, incentives, stakeholders, and culture for continuous improvement and innovation. The Agency for Healthcare Research and Quality and the Patient-Centered Outcomes Research Institute designed a K12 initiative to grow the number of LHS scientists. We describe approaches developed by 11 funded centers of excellence (COEs) to promote partnerships between scholars and health system leaders and to provide mentored research training. Methods: Since 2018, the COEs have enlisted faculty, secured institutional resources, partnered with health systems, developed and implemented curricula, recruited scholars, and provided mentored training. Program directors for each COE provided descriptive data on program context, scholar characteristics, stakeholder engagement, scholar experiences with health system partnerships, roles following program completion, and key training challenges. Results: To date, the 11 COEs have partnered with health systems to train 110 scholars. Nine (82%) programs partner with a Veterans Affairs health system and 9 (82%) partner with safety net providers. Clinically trained scholars (n = 87; 79%) include 70 physicians and 17 scholars in other clinical disciplines. Non-clinicians (n = 29; 26%) represent diverse fields, dominated by population health sciences. Stakeholder engagement helps scholars understand health system and patient/family needs and priorities, enabling opportunities to conduct embedded research, improve outcomes, and grow skills in translating research methods and findings into practice. Challenges include supporting scholars through roadblocks that threaten to derail projects during their limited program time, ranging from delays in access to data to COVID-19-related impediments and shifts in organizational priorities. Conclusions: Four years into this novel training program, there is evidence of scholars' accomplishments, both in traditional academic terms and in terms of moving along career trajectories that hold the potential to lead and accelerate transformational health system change. Future LHS training efforts should focus on sustainability, including organizational support for scholar activities.

Web-Based System to Capture Consistent and Complete Real-world Data of Physical Therapy Interventions Following Total Knee Replacement: Design and Evaluation Study.

BACKGROUND: Electronic health records (EHRs) have the potential to facilitate consistent clinical data capture to support excellence in patient care, quality improvement, and knowledge generation. Despite widespread EHR use, the vision to transform health care system and its data to a "learning health care system" generating knowledge from real-world data is limited by the lack of consistent, structured clinical data. OBJECTIVE: The purpose of this paper was to demonstrate the design of a web-based structured clinical intervention data capture system and its evaluation in practice. The use case was ambulatory physical therapy (PT) treatment after total knee replacement (TKR), one of the most common and costly procedures today. METHODS: To identify the PT intervention type and intensity (or dose) used to treat patients with knee arthritis following TKR, an iterative user-centered design process refined an initial list of PT interventions generated during preliminary chart reviews. Input from practicing physical therapists and national and international experts refined and categorized the interventions. Next, a web-based, hierarchical structured system for intervention and intensity documentation was designed and deployed. RESULTS: The PT documentation system was implemented by 114 physical therapists agreeing to record all interventions at patient visits. Data for 161 patients with 2615 PT visits were entered by 83 physical therapists. No technical problems with data entry were reported, and data entry required less than 2 minutes per visit. A total of 42 (2%) interventions could not be categorized and were recorded using free text. CONCLUSIONS: The use of user-centered design principles provides a road map for developing clinically feasible data capture systems that employ structured collection of uniform data for use by multiple practitioners across institutions to complement and augment existing EHRs. Secondarily, these data can be analyzed to define best practices and disseminate knowledge to practice.

Patients' perspectives on the benefits of feedback on patient-reported outcome measures in a web-based personalized decision report for hip and knee osteoarthritis.

BACKGROUND: Applications of patient-reported outcome measures (PROMs) for individual patient management are expanding with the support of digital tools. Providing PROM-based information to patients can potentially improve care experiences and outcomes through informing and activating patients. This study explored patients' perspectives on the benefits of receiving feedback on PROMs in the context of a web-based personalized decision report to guide care for their hip or knee osteoarthritis. METHODS: This qualitative descriptive interview study was nested in a pragmatic clinical trial of a personalized report, which includes descriptive PROM scores and predicted postoperative PROM scores. Patients completed a semi-structured interview within 6 weeks of an office visit with an orthopaedic surgeon. Only patients who reported receiving the report and reviewing it with the surgeon and/or a health educator were included. Data were iteratively analyzed using a combination of deductive and inductive coding strategies. RESULTS: Twenty-five patients aged 49-82 years (60% female, 72% surgical treatment decision) participated and described three primary benefits of the PROM feedback within the report: 1. Gaining Information About My Health Status, including data teaching new information, confirming what was known, or providing a frame of reference; 2. Fostering Communication Between Patient and Surgeon, encompassing use of the data to set expectations, ask and answer questions, and facilitate shared understanding; and 3. Increasing My Confidence and Trust, relating to the treatment outcomes, treatment decision, and surgeon. CONCLUSIONS: Patients identified actual and hypothetical benefits of receiving feedback on PROM scores in the context of a web-based decision report, including advantages for those who had already made a treatment decision before seeing the surgeon. Findings provide insight into patients' perspectives on how digital PROM data can promote patient-centered care. Results should be considered in the context of the homogeneous sample and complex trial. While participants perceived value in this personalized report, questions remain regarding best practices in patient-facing data presentation and engagement. TRIAL REGISTRATION: ClinicalTrials.gov, NCT03102580. Registered on 5 April 2017.