How Prominent Are Gender Bias, Racial Bias, and Score Inflation in Orthopaedic Surgery Residency Recommendation Letters? A Systematic Review.

INTRODUCTION: Letters of recommendation are considered an important factor in the holistic ranking of orthopaedic residency applications. The standardized letter of recommendation (SLOR) was introduced in 2017 in response to the inherent bias and limited comparative value of traditional LORs. It includes standardized questions about an applicant's orthopaedic qualifications, designed to enable fair comparisons among candidates. However, disparate and inconsistent findings have made it difficult to draw meaningful conclusions from individual studies on the SLOR and narrative letter of recommendation. QUESTION/PURPOSES: In this systematic review we asked: (1) What is the distribution of applicant SLOR rating among nine domains and summative scores? (2) Are there applicant characteristics associated with SLOR ratings? (3) Is there gender and racial bias in the SLOR and narrative letter of recommendation? METHODS: PubMed, EBSCO, and Google Scholar electronic databases were queried on March 20, 2023, to identify all studies evaluating SLOR and narrative letter of recommendations between January 1, 2017, and March 20, 2023. Articles that evaluated orthopaedic SLOR or narrative LORs were included. Systematic reviews, case reports, duplicate studies between databases, and grey literature such as abstracts and articles on preprint servers were excluded. Eight studies including 6179 applicants and 18,987 letters of recommendation were included in the final analysis. The applicant classes ranged from years 2014 to 2020. Two reviewers independently evaluated the quality of the included studies using the Joanna Briggs Institute (JBI) tool for cross-sectional studies. The mean JBI score of included studies was 7.4 out of a maximum of 8, with higher scores indicating better quality. The primary outcome was to determine the distribution of applicant SLOR rankings in their summative score. Summative scores were separated into ranked to match, top one-third, middle one-third, lower one-third, and not a fit. Four studies reported the summative SLOR scores of applicants. Our secondary study goal was to assess applicant characteristics associated with SLOR summative scores and assess any bias present in the SLOR or narrative recommendation. Five studies compared SLOR ratings across applicant characteristics including gender, race, USMLE Step 1 score, USMLE Step 2 score, Alpha Omega Alpha (AΩA) membership, clerkship grades, and publications. Gender and racial bias were also assessed across five studies. Four studies utilized a linguistic analysis software to search letters of recommendation for differences in word category use by gender and race. RESULTS: Studies consistently found that a higher percentage of candidates were identified in the top percentile blocks than is mathematically possible. For example, the two studies that tallied the proportion of candidates ranking in the top 10% of applicants found that 36% and 42% were rated as being in the top 10%. Similarly, articles found more than 87% of applicants scored in the top one-third. Studies had mixed results, but in general, they suggested that AΩA status, higher Step 1 scores, and more research publications have a slightly positive association with increased SLOR scores. We found no evidence of gender bias against women, and in fact, most studies evaluating word choices found word choices for women candidates generally were stronger. Similarly, no consistent disparities were identified in word categories utilized in SLORs based on applicant race. CONCLUSION: Most notably, a mathematically impossible proportion of applicants were counted in the top percentiles in letters of recommendation. This compromises readers' abilities to differentiate candidates. Factors like AOA status and research publications displayed a modest positive correlation with SLOR scores. Gender bias against women or candidates from racial minority groups was not evident. CLINICAL RELEVANCE: We suggest that group SLOR authorship, with a consensus assessment by clerkship or residency directors, should be considered. We also recommend the incorporation of mean and SD scores for each letter writer (or group) alongside their letters. Furthermore, studies indicate that submitting all four SLOR letters can substantially improve an applicant's chances of success. We advise students to choose strategically and submit the maximum allowable number of SLORs.

Should my recommendation letter be written by artificial intelligence?

SummaryLetters of recommendation are increasingly important for the residency match. We assessed whether an artificial intelligence (AI) tool could help in writing letters of recommendation by analyzing recommendation letters written by 3 academic staff and AI duplicate versions for 13 applicants. The preferred letters were selected by 3 blinded orthopedic program directors based on a pre-determined set of criteria. The first orthopedic program director selected the AI letter for 31% of applicants, and the 2 remaining program directors selected the AI letter for 38% of applicants, with the staff-written versions selected more often by all of the program directors (p < 0.05). The first program director recognized only 15% of the AI-written letters, the second was able to identify 92%, and the third director identified 77% of AI-written letters (p < 0.05).

Establishing a consensus on research priorities in orthopaedic trauma within South Africa.

BACKGROUND: Musculoskeletal (MSK) injuries are one of the leading causes of disability worldwide. Despite improvements in trauma-related morbidity and mortality in high-income countries over recent years, outcomes following MSK injuries in low- and middle-income countries, such as South Africa (SA), have not. Despite governmental recognition that this is required, funding and research into this significant health burden are limited within SA. This study aims to identify research priorities within MSK trauma care using a consensus-based approach amongst MSK healthcare practitioners within SA. METHOD: Members from the Orthopaedic Research Collaboration in Africa (ORCA), based in SA, collaborated using a two round modified Delphi technique to form a consensus on research priorities within orthopaedic trauma care. Members involved in the process were orthopaedic healthcare practitioners within SA. RESULTS: Participants from the ORCA network, working within SA, scored research priorities across two Delphi rounds from low to high priority. We have published the overall top 10 research priorities for this Delphi process. Questions were focused on two broad groups-clinical effectiveness in trauma care and general trauma public health care. Both groups were represented by the top two priorities, with the highest ranked question regarding the overall impact of trauma in SA and the second regarding the clinical treatment of open fractures. CONCLUSION: This study has defined research priorities within orthopaedic trauma in South Africa. Our vision is that by establishing consensus on these research priorities, policy and research funding will be directed into these areas. This should ultimately improve musculoskeletal trauma care across South Africa and its significant health and socioeconomic impacts.

How Long Will It Take to Reach Gender Parity in Orthopaedic Surgery in the United States? An Analysis of the National Provider Identifier Registry.

BACKGROUND: Although previous studies have evaluated how the proportion of women in orthopaedic surgery has changed over time, these analyses have been limited by small sample sizes, have primarily used data on residents, and have not included information on growth across subspecialties and geographic regions. QUESTION/PURPOSE: We used the National Provider Identifier registry to ask: How have the (1) overall, (2) regional, and (3) subspecialty percentages of women among all currently practicing orthopaedic providers changed over time in the United States? METHODS: The National Provider Identifier Registry of the Centers for Medicare and Medicaid Services (CMS) was queried for all active providers with taxonomy codes pertaining to orthopaedic subspecialties as of April 2020. Women orthopaedic surgeons were identified among all physicians with subspecialty taxonomy codes. As all providers are required to provide a gender when applying for an NPI, all providers with queried taxonomy codes additionally had gender classification. Our final cohort consisted of 31,296 practicing orthopaedic surgeons, of whom 8% (2363 of 31,296) were women. A total of 11,714 (37%) surgeons possessed taxonomy codes corresponding with a specific orthopaedic subspecialty. A univariate linear regression analysis was used to analyze trends in the annual proportions of women who are active orthopaedic surgeons based on NPI enumeration dates. Specifically, annual proportions were defined using cross-sections of the NPI registry on December 31 of each year. Linear regression was similarly used to evaluate changes in the annual proportion of women orthopaedic surgeons across United States Census regions and divisions, as well as orthopaedic subspecialties. The national growth rate was then projected forward to determine the year at which the representation of women orthopaedic surgeons would achieve parity with the proportion of all women physicians (36.3% or 340,018 of 936,254, as determined by the 2019 American Medical Association Physician Masterfile) and the proportion of all women in the United States (50.8% or 166,650,550 of 328,239,523 as determined by 2019 American Community Survey from the United States Census Bureau). Gender parity projections along with corresponding 95% confidence intervals were calculated using the Holt-Winters forecasting algorithm. The proportions of women physicians and women in the United States were assumed to remain fixed at 2019 values of 36.3% and 50.8%, respectively. RESULTS: There was a national increase in the proportion of women orthopaedic surgeons between 2010 and 2019 (r2 = 0.98; p < 0.001) at a compound annual growth rate of 2%. Specifically, the national proportion of orthopaedic surgeons who were women increased from 6% (1670 of 26,186) to 8% (2350 of 30,647). Assuming constant growth at this rate following 2019, the time to achieve gender parity with the overall medical profession (that is, to achieve 36.3% women in orthopaedic surgery) is projected to be 217 years, or by the year 2236. Likewise, the time to achieve gender parity with the overall US population (which is 50.8% women) is projected to be 326 years, or by the year 2354. During our study period, there were increases in the proportion of women orthopaedic surgeons across US Census regions. The lowest growth was in the West (17%) and the South (19%). Similar growth was demonstrated across census divisions. In each orthopaedic subspecialty, we found increases in the proportion of women surgeons throughout the study period. Adult reconstruction (0%) and spine surgery (1%) had the lowest growth. CONCLUSION: We calculate that at the current rate of change, it will take more than 200 years for orthopaedic surgery to achieve gender parity with the overall medical profession. Although some regions and subspecialties have grown at comparably higher rates, collectively, there has been minimal growth across all domains. CLINICAL RELEVANCE: Given this meager growth, we believe that substantive changes must be made across all levels of orthopaedic education and leadership to steepen the current curve. These include mandating that all medical school curricula include dedicated exposure to orthopaedic surgery to increase the number of women coming through the orthopaedic pipeline. Additionally, we believe the Accreditation Council for Graduate Medical Education and individual programs should require specific benchmarks for the proportion of orthopaedic faculty and fellowship program directors, as well as for the proportion of incoming trainees, who are women. Furthermore, we believe there should be a national effort led by American Academy of Orthopaedic Surgeons and orthopaedic subspecialty societies to foster the academic development of women in orthopaedic surgery while recruiting more women into leadership positions. Future analyses should evaluate the efficacy of diversity efforts among other surgical specialties that have achieved or made greater strides toward gender parity, as well as how these programs can be implemented into orthopaedic surgery.

Validation and Implementation of 4-domain Patient-reported Outcome Measures (PROMs) Tailored for Orthopedic Sports Medicine.

The validation of a 4-domain PROM tailored to orthopedic sports medicine was performed through item generation, item scaling, validity and reliability testing, statistical analysis, as well as item reduction. Conbrach's alpha was used to verify item homogeneity, i. e. their accuracy or consistency. This PROM showed acceptable statistical accuracy and clinical applicability for a variety of surgical treatments, regardless of the anatomical injury sites. Moreover, this PROM considers the athletes' primary physical demands in an non-injured baseline condition, their motivation to continue sports practice and participation, and the influence of sports practice on their quality of life. This 4-domain PROM tailored for orthopedic sports medicine appears to be a valid tool to assess athletes and high-performing practitioners with sports injuries, recording their perception of injury, expectations of treatment; evaluation of postoperative care and treatment received, and perceived outcomes compared to their pre-injury status of physical demands in sports activity. The tool is unique, allowing direct comparisons between athletes' perception of pre-injury baseline, injury, treatment, and outcome. It will be a welcome adjunct to the sports medicine professional's tool box when assessing athlete's status and outcome after injury and intervention.

"Greek KOOS-Child: a valid, disease specific, diagnostically accurate and responsive PROM in children with knee-related pathology".

PURPOSE: The Knee Injury Osteoarthritis Outcome Score (KOOS)-Child questionnaire is one of the frequently child-friendly measures used in pediatric studies. The aim of this study was to transculturally adapt the Greek version of KOOS-Child and evaluate its clinimetric properties in children with knee disorders. METHODS: Children visiting the Outpatients Orthopaedic Clinic of a Greek Paediatric General Hospital were considered eligible if they were aged 8-14 years, had a knee soft tissue injury and associated physical limitations. The transcultural adaptation was based on a multistage backward translation approach. Participants completed the KOOS-Child at their first visit to the orthopedic specialist (baseline), 2 weeks and 3 months after baseline. Content validity of the KOOS-Child was evaluated using general QoL measures (KIDSCREEN and Kid-KINDL) and construct validity was explored by correlating relevant items. Responsiveness was evaluated according to the children's response on the given orthopeadic treatment. RESULTS: Sample consisted of 59 children (30 males), aged: 11 ± 1.8 years. The KOOS-Child showed high internal consistency (Cronbach's a: 0.80-0.96). Adequate convergent validity with > 75% relevant a priori hypotheses was confirmed. Construct validity was moderate to strong (Pearson's r correlations between related KOOS and Kid-KINDL subdimensions: 0.54-0.62). KOOS and KIDSCREEN subdimensions correlations were fair (Pearson's r correlations: 0.32-0.65). KOOS-Child's diagnostic accuracy was high. Factor analysis extracted height factors accounting for 76.15% of the total variance, confirmed by the scree plot. Responsiveness was moderate to high with Cohen's d from 0.6 to 1.4. CONCLUSION: The Greek version of the KOOS-Child demonstrated excellent internal consistency, good construct validity, diagnostic accuracy and interpretability as well as good responsiveness. The measure could be used across Greek children with orthopaedic knee problems. Generalisability of findings is limited due to the relatively limited cohort. LEVEL OF EVIDENCE: II.

The Majority of Patient-reported Outcome Measures in Pediatric Orthopaedic Research Are Used Without Validation.

BACKGROUND: Given that patient-reported outcome measures (PROMs) are increasingly used to inform clinical decision-making, it is vital that they are validated, reliable, responsive, and appropriate for the population under study. The purpose of this systematic review was to assess the rate of PROM use in the pediatric orthopaedic literature, characterize whether each use was in the PROM-validated demographic, and analyze the association between bibliometric factors and the use of PROMs with incomplete validation. METHODS: The Institute for Scientific Information (ISI) Web of Science database was queried for all clinical pediatric orthopaedic studies from 2014 to 2017. All PROMs were recorded for each study independently by 2 reviewers and cross-referenced with the published orthopaedic literature as of 2017 to determine if the PROM had been validated for the study population. PROMs that had not been validated, had been shown to be invalid, or had conflicting validity studies for use in the population of interest were designated as incompletely validated or used without complete validation. The following covariates were recorded for each study: subspecialty, inclusion of a statistician coauthor, sample size, journal, and the journal impact factor. χ analysis was used to evaluate the association between categorical variables and the use of at least 1 incompletely validated PROM. RESULTS: In total, 1000 articles were screened, yielding 653 studies that met our inclusion criteria. A total of 104/653 (16%) publications-reported PROMs. PROMs were used without complete validation 120/165 (73%) times, and 77/104 (74%) studies used at least 1 PROM without complete validation. The most frequent reasons for incomplete validation were (1) using PROMs that had been validated in adults, but not pediatrics (n=47; 28.5%), and (2) using PROMs that had been designated in the literature as invalid for pediatrics (n=27; 16.4%). The incomplete validation of at least 1 PROM was associated with smaller sample size (P=0.01) and subspecialty (P<0.01). Overall rates of PROM use and the proportion of PROMs used without complete validation were both found to be higher than those previously reported. CONCLUSION: The majority of pediatric orthopaedic studies reporting PROMs used at least 1 PROM without complete validation for their study population. LEVEL OF EVIDENCE: Level IV-systematic review of level I, II, III, and IV studies.

Radiation Reduction Strategies in Pediatric Orthopaedics.

INTRODUCTION: Pediatric orthopaedic patients have the potential for significant radiation exposure from the use of imaging studies, such as computed tomography and bone scintigraphy. With the potential for long-term treatment, such as is required for scoliosis or osteogenesis imperfecta, patients are at even greater risk of radiation-induced carcinogenesis. DISCUSSION: Although an association between radiation and cancer risk is evident, causation is difficult to prove because comorbidities or genetic predispositions may play a role in the higher baseline rates of malignancy later in life. Efforts have been made over the years to reduce exposure using more modern imaging techniques and simple radiation reduction strategies. Educational efforts and clinical practice guidelines are decreasing the rate of computed tomography scan use in pediatrics. Although considerable work is being done on the development of radiation-free imaging modalities, imaging that uses ionizing radiation will, in the near term, be necessary in specific circumstances to provide optimal care to pediatric orthopaedic patients. CONCLUSION: Knowledge of the ionizing radiation exposure associated with commonly used tests as well as radiation-reduction strategies is essential for the optimal and safe care of pediatric orthopaedic patients.

Inter-rater and Intrarater Reliabilities of the Identification of a "Gothic Arch" in the Acetabulum of Children With Cerebral Palsy.

BACKGROUND: Progressive hip displacement in children with cerebral palsy (CP) is monitored by measuring migration percentage (MP) on anteroposterior (AP) pelvis radiographs. Accurate measurement of MP requires the lateral margin of the ossified acetabulum to be identified for the placement of Perkin's line. It has been suggested that when there is an erosion of the acetabular rim, described as a gothic arch, the midpoint of the arch be used for the placement of Perkin's line. However, this requires that there be agreement on what constitutes a gothic arch. The purpose of this study was to evaluate the inter-rater and intrarater reliabilities of identifying a gothic arch on pelvis radiographs. METHODS: An online survey with 100 AP pelvis images (200 hips) of children with CP was sent to international experts. Participants were asked to identify which hip(s) had a gothic arch (left, right, both, and neither). The Fleiss κ statistic for inter-rater reliability was calculated. Eight weeks later, the images were shuffled and redistributed to calculate intrarater reliability. RESULTS: The initial survey was completed by 10 participants with 9 participants completing the second survey. The average inter-rater κ value was 0.18 [95% confidence interval (CI), 0.14-0.23] and 0.19 (95% CI, 0.14-0.24) for the 2 surveys, respectively. Among the pediatric orthopaedic surgeons subgroup, the κ values were 0.06 (95% CI, 0.02-0.1) and 0.08 (95% CI, 0.03-0.13). The average intrarater reliability κ value was 0.61 (95% CI, 0.2-1), ranging from 0.32 to 0.86. CONCLUSIONS: There were poor inter-rater and moderate intrarater reliabilities in identifying a gothic arch on AP pelvis radiographs in children with CP. Further characterization and clarification of what constitutes a gothic arch are required. The lack of agreement on the identification of a gothic arch may negatively impact the measurement of MP and referrals to a pediatric orthopaedic surgeon.

Standardizing statistics and data reporting in orthopaedic research.

As the demand for research within orthopaedic surgery continues to grow, it is important to emphasize appropriate data reporting. Insufficient statistical reporting confounds data interpretation and makes it difficult to compare results. Currently, there are no guidelines for reporting results within the orthopaedic literature. This article discusses the importance of appropriate data reporting and proposes guidelines for presenting orthopaedic data to highlight clinical relevance rather than statistical significance.