Improving Efficiency of Patient-Reported Outcome Collection: Application of Computerized Adaptive Testing to DASH and QuickDASH Outcome Scores.

PURPOSE: Patient-reported outcome measures assess health status and treatment outcomes in orthopedic care, but they may burden patients with lengthy questionnaires. Predictive models using machine learning, known as computerized adaptive testing (CAT), offer a potential solution. This study evaluates the ability of CAT to improve efficiency of the 30-item Disabilities of the Arm, Shoulder, and Hand (DASH) and 11-item QuickDASH questionnaires. METHODS: A total of 2,860 DASH and 27,355 QuickDASH respondents were included in the analysis. The CAT system was retrospectively applied to each set of patient responses stored on the instrument to calculate a CAT-specific score for all DASH and QuickDASH entries. The accuracy of the CAT scores, viewed in the context of the minimal clinically important difference for both patient-reported outcome measures (DASH, 12; QuickDASH, 9), was determined through descriptive statistics, Pearson correlation coefficient, intraclass correlation coefficient, and distribution of scores and score differences. RESULTS: The CAT model required an average of 15.3 questions to be answered for the DASH and 5.8 questions for the QuickDASH, representing a 49% and 47% decrease in question burden, respectively. Mean CAT score was the same for DASH and 0.1 points lower for QuickDASH with similar SDs (DASH, 12.9 ± 19.8 vs 12.9 ± 19.9; QuickDASH, 32.7 ± 24.7 vs 32.6 ± 24.6). Pearson coefficients (DASH, 0.99; QuickDASH, 0.98) and intraclass correlation coefficients (DASH, 1.0; QuickDASH, 0.98) indicated strong agreement between scores. The difference between the CAT and full score was less than the minimal clinically important difference in 99% of cases for DASH and approximately 95% of cases for QuickDASH. CONCLUSIONS: The application of CAT to DASH and QuickDASH surveys demonstrated an ability to lessen the response burden with negligible effect on score integrity. CLINICAL RELEVANCE: In the case of DASH and QuickDASH, CAT is an appropriate alternative to full questionnaire implementation for patient outcome score collection.

Accuracy and Validity of Computer Adaptive Testing for Outcome Assessment in Patients Undergoing Total Knee Arthroplasty.

BACKGROUND: Computerized adaptive test (CAT) questionnaires may allow standardization of patient-reported outcome measures and reductions in questionnaire burden. We evaluated the validity, accuracy, and efficacy of a CAT system in patients with end-stage osteoarthritis undergoing total knee arthroplasty. METHODS: CAT Knee Osteoarthritis Outcome Scores (KOOS) and CAT KOOS-JR questionnaires were applied to 1871 standard form KOOS and 1493 KOOS-JR patient responses, respectively. Mean, standard deviations, Pearson's correlation coefficients, interclass correlation coefficients (ICCs), frequency distribution plots, and Bland-Altman plots were used to compare the precision, validity, and accuracy between CAT scores and full-form scores. RESULTS: There was a mean reduction of 14 questions (33%) in the CAT KOOS and 1.4 questions (20%) with the CAT KOOS-JR version, compared with the standard KOOS and KOOS-JR surveys, respectively. There were no significant differences between KOOS and CAT KOOS scores with respect to pain (P = .66), symptoms (P = .43), quality of life (P = .99), activities of daily living (P = .68), and sports (P = .84). Similarly, there were no significant differences between the standard form KOOS-JR and CAT KOOS-JR scores (P = .94). There were strong correlations with minimal variability between the CAT KOOS and standard KOOS questionnaires for pain (r = 0.98, ICC: 0.98), symptoms (r = 0.97, ICC: 0.97), quality of life scores (r = 0.99, ICC: 0.99), activities of daily living scores (r = 0.99, ICC: 0.99), and sports scores (r = 0.99, ICC: 0.99). Similarly, there were strong correlations between the KOOS-JR and the CAT KOOS-JR scores (r = 0.99, ICC: 0.99). CONCLUSION: CAT KOOS and the CAT KOOS-JR versions are accurate and reduce questionnaire burden up to one-third compared with standard surveys. CAT versions may improve patient compliance and decrease fatigue.

Accuracy and Validity of Computer Adaptive Testing for Outcome Assessment in Patients Undergoing Total Hip Arthroplasty.

BACKGROUND: Probability-based computer algorithms that reduce patient burden are currently in high demand. These computer adaptive testing (CAT) methods improve workflow and reduce patient frustration, while achieving high measurement precision. In this study, we evaluated the accuracy and validity of the CAT Hip Disability and Osteoarthritis Outcome Score (HOOS) and the Hip Disability and Osteoarthritis Outcome Score Joint Replacement (HOOS-JR) by comparing them to the full version of these scoring systems in a subset of patients who had undergone total hip arthroplasties. METHODS: A previously developed CAT HOOS and HOOS-JR was applied to 354 and 1547 HOOS and HOOS-JR patient responses, respectively. Mean, standard deviations, Pearson's correlation coefficients, interclass correlation coefficients, frequency distribution plots, and Bland-Altman plots were used to compare the precision, validity, and accuracy between CAT scores and full-form scores. RESULTS: By modifying the questions to past responses, the CAT HOOS demonstrated a mean reduction of 30% of questions (28 vs 40 questions). There were no significant differences between the full HOOS and CAT HOOS with respect to pain (P = .73), symptoms (P = .94), quality of life (P = .99), activities of daily living (P = .82), and sports (P = .99). There were strong linear relationships between the CAT versions and the standard questionnaires (r > 0.99). The Bland-Altman plot showed that differences between CAT HOOS and full HOOS were independent of the overall scores. CONCLUSION: The CAT HOOS and HOOS-JR have high correlation and require fewer questions to finish compared to the standard full-form questionnaires. This may represent a reliable and practical alternative that may be less burdensome to patients and may help improve compliance for reporting outcome metrics.

A concise shoulder outcome measure: application of computerized adaptive testing to the American Shoulder and Elbow Surgeons Shoulder Assessment.

BACKGROUND: Patient-reported outcome measures enable quantitative and patient-centric assessment of orthopedic interventions; however, increased use of these forms has an associated burden for patients and practices. We examined the utility of a computerized adaptive testing (CAT) method to reduce the number of questions on the American Shoulder and Elbow Surgeons (ASES) instrument. METHODS: A previously developed ASES CAT system was applied to the responses of 2763 patients who underwent shoulder evaluation and treatment and had answered all questions on the full ASES instrument. Analyses to assess the accuracy of the CAT score in replicating the full-form score included the mean and standard deviation of both groups of scores, frequency distributions of the 2 sets of scores and score differences, Pearson and intraclass correlation coefficients, and Bland-Altman assessment of patterns in score differences. RESULTS: By tailoring questions according to prior responses, CAT reduced the question burden by 40%. The mean difference between CAT and full ASES scores was -0.14, and the scores were within 5 points in 95% of cases (a 12-point difference is considered the threshold for clinical significance) and were clustered around zero. The correlation coefficients were 0.99, and the frequency distributions of the CAT and full ASES scores were nearly identical. The differences between scores were independent of the overall score, and no significant bias for CAT scores was found in either a positive or negative direction. CONCLUSION: The ASES CAT system lessens respondent burden with a negligible effect on score integrity.

Efficiency and Accuracy of Computerized Adaptive Testing for the Oswestry Disability Index and Neck Disability Index.

This study aimed to determine the efficiency and accuracy of computerized adaptive testing (CAT) models of the Oswestry Disability Index (ODI) and Neck Disability Index (NDI). Methods: The study involved simulation using retrospectively collected real-world data. Previously developed CAT models of the ODI and NDI were applied to the responses from 52,551 and 18,196 patients with spinal conditions, respectively. Efficiency was evaluated by the reduction in the number of questions administered. Accuracy was evaluated by comparing means and standard deviations, calculating Pearson r and intraclass correlation coefficient (ICC) values, plotting the frequency distributions of CAT and full questionnaire scores, plotting the frequency distributions of differences between paired scores, and Bland-Altman plotting. Score changes, calculated as the postoperative ODI or NDI scores minus the preoperative scores, were compared between the CAT and full versions in patients for whom both preoperative and postoperative ODI or NDI questionnaires were available. Results: CAT models of the ODI and NDI required an average of 4.47 and 4.03 fewer questions per patient, respectively. The mean CAT ODI score was 0.7 point lower than the full ODI score (35.4 ± 19.0 versus 36.1 ± 19.3), and the mean CAT NDI score was 1.0 point lower than the full NDI score (34.7 ± 19.3 versus 33.8 ± 18.5). The Pearson r was 0.97 for both the ODI and NDI, and the ICC was 0.97 for both. The frequency distributions of the CAT and full scores showed marked overlap for the ODI and NDI. Differences between paired scores were less than the minimum clinically important difference in 98.9% of cases for the ODI and 98.5% for the NDI. Bland-Altman plots showed no proportional bias. The ODI and NDI score changes could be calculated in a subgroup of 6,044 and 4,775 patients, respectively; the distributions of the ODI and NDI score changes were near identical between the CAT and full versions. Conclusions: CAT models were able to reduce the question burden of the ODI and NDI. Scores obtained from the CAT models were faithful to those from the full questionnaires, both on the population level and on the individual patient level. Level of Evidence: Prognostic Level III. See Instructions for Authors for a complete description of levels of evidence.

Computerized adaptive testing for the Oxford Hip, Knee, Shoulder, and Elbow scores : accurate measurement from fewer, and more patient-focused, questions.

AIMS: The aim of this study was to develop and evaluate machine-learning-based computerized adaptive tests (CATs) for the Oxford Hip Score (OHS), Oxford Knee Score (OKS), Oxford Shoulder Score (OSS), and the Oxford Elbow Score (OES) and its subscales. METHODS: We developed CAT algorithms for the OHS, OKS, OSS, overall OES, and each of the OES subscales, using responses to the full-length questionnaires and a machine-learning technique called regression tree learning. The algorithms were evaluated through a series of simulation studies, in which they aimed to predict respondents' full-length questionnaire scores from only a selection of their item responses. In each case, the total number of items used by the CAT algorithm was recorded and CAT scores were compared to full-length questionnaire scores by mean, SD, score distribution plots, Pearson's correlation coefficient, intraclass correlation (ICC), and the Bland-Altman method. Differences between CAT scores and full-length questionnaire scores were contextualized through comparison to the instruments' minimal clinically important difference (MCID). RESULTS: The CAT algorithms accurately estimated 12-item questionnaire scores from between four and nine items. Scores followed a very similar distribution between CAT and full-length assessments, with the mean score difference ranging from 0.03 to 0.26 out of 48 points. Pearson's correlation coefficient and ICC were 0.98 for each 12-item scale and 0.95 or higher for the OES subscales. In over 95% of cases, a patient's CAT score was within five points of the full-length questionnaire score for each 12-item questionnaire. CONCLUSION: Oxford Hip Score, Oxford Knee Score, Oxford Shoulder Score, and Oxford Elbow Score (including separate subscale scores) CATs all markedly reduce the burden of items to be completed without sacrificing score accuracy.Cite this article: Bone Jt Open 2022;3(10):786-794.

Application of Computerized Adaptive Testing to the Foot and Ankle Ability Measure.

BACKGROUND: Patient-reported outcome measures are an increasingly important tool for assessing the impact of treatments orthopedic surgeons render. Despite their importance, they can present a burden. We examined the validity and utility of a computerized adaptive testing (CAT) method to reduce the number of questions on the Foot and Ankle Ability Measure (FAAM), a validated anatomy-specific outcome measure. METHODS: A previously developed FAAM CAT system was applied to the responses of patients undergoing foot and ankle evaluation and treatment over a 3-year period (2017-2019). A total of 15 902 responses for the Activities of Daily Living (ADL) subscale and a total of 14 344 responses for the Sports subscale were analyzed. The accuracy of the CAT to replicate the full-form score was assessed. RESULTS: The CAT system required 11 questions to be answered for the ADL subscale in 85.1% of cases (range, 11-12). The number of questions answered on the Sports subscale was 6 (range, 5-6) in 66.4% of cases. The mean difference between the full FAAM ADL subscale and CAT was 0.63 of a point. The mean difference between the FAAM Sports subscale and CAT was 0.65 of a point. CONCLUSION: The FAAM CAT was able to reduce the number of responses a patient would need to answer by nearly 50%, while still providing a valid outcome score. This measure can therefore be directly correlated with previously obtained full FAAM scores in addition to providing a foot/ankle-specific measure, which previously reported CAT systems are not able to do. LEVEL OF EVIDENCE: Level IV, case series.

Machine Learning Can Predict Level of Improvement in Shoulder Arthroplasty.

The ability to accurately predict postoperative outcomes is of considerable interest in the field of orthopaedic surgery. Machine learning has been used as a form of predictive modeling in multiple health-care settings. The purpose of the current study was to determine whether machine learning algorithms using preoperative data can predict improvement in American Shoulder and Elbow Surgeons (ASES) scores for patients with glenohumeral osteoarthritis (OA) at a minimum of 2 years after shoulder arthroplasty. METHODS: This was a retrospective cohort study that included 472 patients (472 shoulders) diagnosed with primary glenohumeral OA (mean age, 68 years; 56% male) treated with shoulder arthroplasty (431 anatomic total shoulder arthroplasty and 41 reverse total shoulder arthroplasty). Preoperative computed tomography (CT) scans were used to classify patients on the basis of glenoid and rotator cuff morphology. Preoperative and final postoperative ASES scores were used to assess the level of improvement. Patients were separated into 3 improvement ranges of approximately equal size. Machine learning methods that related patterns of these variables to outcome ranges were employed. Three modeling approaches were compared: a model with the use of all baseline variables (Model 1), a model omitting morphological variables (Model 2), and a model omitting ASES variables (Model 3). RESULTS: Improvement ranges of ≤28 points (class A), 29 to 55 points (class B), and >55 points (class C) were established. Using all follow-up time intervals, Model 1 gave the most accurate predictions, with probability values of 0.94, 0.95, and 0.94 for classes A, B, and C, respectively. This was followed by Model 2 (0.93, 0.80, and 0.73) and Model 3 (0.77, 0.72, and 0.71). CONCLUSIONS: Machine learning can accurately predict the level of improvement after shoulder arthroplasty for glenohumeral OA. This may allow physicians to improve patient satisfaction by better managing expectations. These predictions were most accurate when latent variables were combined with morphological variables, suggesting that both patients' perceptions and structural pathology are critical to optimizing outcomes in shoulder arthroplasty. LEVEL OF EVIDENCE: Therapeutic Level IV. See Instructions for Authors for a complete description of levels of evidence.

The Application of Computerized Adaptive Testing to the International Knee Documentation Committee Subjective Knee Evaluation Form.

BACKGROUND: Patient-reported outcome measures (PROMs) are commonly used to monitor functional outcomes for clinical and research purposes; unfortunately, many PROMs include redundant, burdensome questions for patients. The use of predictive models to implement computerized adaptive testing (CAT) offer a potential solution to reduce question burden in outcomes research. PURPOSE: To validate the usage of an appropriate CAT system to improve the efficiency of the International Knee Documentation Committee (IKDC) Subjective Knee Form. STUDY DESIGN: Cohort study (Diagnosis); Level of evidence, 2. METHODS: Validation was based on electronically collected patient responses from 2 separate orthopaedic sports medicine clinics. Diagnoses included, but were not limited to, meniscal lesions, ligamentous injuries, and chondral defects. The CAT system was previously developed through analysis of an electronic knee PROM database that did not contain any of these cases. RESULTS: A total of 2173 patient responses (1229 patients) were collected. The CAT model was able to reduce the question burden by a mean of 9.33 questions (45.1%). Higher CAT-predicted scores correlated strongly with higher actual scores (r = 0.99; intraclass correlation coefficient = 0.99). The mean difference between the CAT-predicted score and the actual PROM score was 0.48 of a point on a scale of 0 to 100. CONCLUSION: The use of CAT systems, in conjunction with electronic PROMs, can accurately predict outcome scores for IKDC PROMs, while dramatically decreasing the number of questionnaire items needed for any given patient. By decreasing questionnaire burden, clinicians and researchers can potentially increase patient participation and follow-up in both clinical assessments and research trials.

Crosswalks Between Knee and Hip Arthroplasty Short Forms: HOOS/KOOS JR and Oxford.

BACKGROUND: The Oxford Knee Score (OKS); Oxford Hip Score (OHS); Knee injury and Osteoarthritis Outcome Score, Joint Replacement (KOOS JR); and Hip disability and Osteoarthritis Outcome Score, Joint Replacement (HOOS JR) are well-validated and widely used short-form patient-reported outcome measures (PROMs) for assessing outcomes after total knee arthroplasty (TKA) and total hip arthroplasty (THA). We are not aware of the existence of any crosswalks to convert scores between these PROMs. We aimed to develop and validate crosswalks that will permit the comparison of scores between studies using different PROMs and the pooling of results for meta-analyses. METHODS: We retrospectively analyzed scores from patients (486 in the knee cohort and 340 in the hip cohort) from the Syracuse Orthopedic Specialists Joint Registry who had completed the appropriate PROMs (OKS and KOOS JR in the knee cohort and OHS and HOOS JR in the hip cohort) as the standard of care before undergoing primary TKA or unicompartmental knee arthroplasty (UKA) between January 9, 2016, and June 19, 2017, or primary THA or hip resurfacing between November 29, 2010, and October 30, 2017, or when returning for postoperative care. Using the equipercentile equating method, we created 4 crosswalks: OKS to KOOS JR, KOOS JR to OKS, OHS to HOOS JR, and HOOS JR to OHS. To assess validity, Spearman coefficients were calculated using bootstrapping methods, and means for actual and crosswalk-derived scores were compared. RESULTS: There were minimal differences between the means of the known and crosswalk-derived scores. As calculated with the use of bootstrapping methods, Spearman coefficients between the actual and derived scores were strong and positive for both knee arthroplasty crosswalks (0.888 to 0.889; 95% confidence interval [CI], 0.887 to 0.891) and hip arthroplasty crosswalks (0.916 to 0.918; 95% CI, 0.914 to 0.919). CONCLUSIONS: We successfully created 4 crosswalks that allow conversion of Oxford scores to KOOS and HOOS JR scores and vice versa. These crosswalks will allow harmonization of PROMs assessment regardless of which of the short forms are used, which may facilitate multicenter collaboration or allow sites to switch PROMs without loss of historic comparison data. LEVEL OF EVIDENCE: Level III. See Instructions for Authors for a complete description of levels of evidence.

Use of Computerized Adaptive Testing to Develop More Concise Patient-Reported Outcome Measures.

BACKGROUND: Patient-reported outcome measures (PROMs) are essential tools that are used to assess health status and treatment outcomes in orthopaedic care. Use of PROMs can burden patients with lengthy and cumbersome questionnaires. Predictive models using machine learning known as computerized adaptive testing (CAT) offer a potential solution. The purpose of this study was to evaluate the ability of CAT to improve efficiency of the Veterans RAND 12 Item Health Survey (VR-12) by decreasing the question burden while maintaining the accuracy of the outcome score. METHODS: A previously developed CAT model was applied to the responses of 19,523 patients who had completed a full VR-12 survey while presenting to 1 of 5 subspecialty orthopaedic clinics. This resulted in the calculation of both a full-survey and CAT-model physical component summary score (PCS) and mental component summary score (MCS). Several analyses compared the accuracy of the CAT model scores with that of the full scores by comparing the means and standard deviations, calculating a Pearson correlation coefficient and intraclass correlation coefficient, plotting the frequency distributions of the 2 score sets and the score differences, and performing a Bland-Altman assessment of scoring patterns. RESULTS: The CAT model required 4 fewer questions to be answered by each subject (33% decrease in question burden). The mean PCS was 1.3 points lower in the CAT model than with the full VR-12 (41.5 ± 11.0 versus 42.8 ± 10.4), and the mean MCS was 0.3 point higher (57.3 ± 9.4 versus 57.0 ± 9.6). The Pearson correlation coefficients were 0.97 for PCS and 0.98 for MCS, and the intraclass correlation coefficients were 0.96 and 0.97, respectively. The frequency distribution of the CAT and full scores showed significant overlap for both the PCS and the MCS. The difference between the CAT and full scores was less than the minimum clinically important difference (MCID) in >95% of cases for the PCS and MCS. CONCLUSIONS: The application of CAT to the VR-12 survey demonstrated an ability to lessen the response burden for patients with a negligible effect on score integrity.