Inadequate Performance of ChatGPT on Orthopedic Board-Style Written Exams.

BACKGROUND: Chat Generative Pre-Trained Transformer (ChatGPT) is an artificial intelligence (AI) chatbot capable of delivering human-like responses to a seemingly infinite number of inquiries. For the technology to perform certain healthcare-related tasks or act as a study aid, the technology should have up-to-date knowledge and the ability to reason through medical information. The purpose of this study was to assess the orthopedic knowledge and reasoning ability of ChatGPT by querying it with orthopedic board-style questions. METHODOLOGY: We queried ChatGPT (GPT-3.5) with a total of 472 questions from the Orthobullets dataset (n = 239), the 2022 Orthopaedic In-Training Examination (OITE) (n = 124), and the 2021 OITE (n = 109). The importance, difficulty, and category were recorded for questions from the Orthobullets question bank. Responses were assessed for answer choice correctness if the explanation given matched that of the dataset, answer integrity, and reason for incorrectness. RESULTS: ChatGPT correctly answered 55.9% (264/472) of questions and, of those answered correctly, gave an explanation that matched that of the dataset for 92.8% (245/264) of the questions. The chatbot used information internal to the question in all responses (100%) and used information external to the question (98.3%) as well as logical reasoning (96.4%) in most responses. There was no significant difference in the proportion of questions answered correctly between the datasets (P = 0.62). There was no significant difference in the proportion of questions answered correctly by question category (P = 0.67), importance (P = 0.95), or difficulty (P = 0.87) within the Orthobullets dataset questions. ChatGPT mostly got questions incorrect due to information error (i.e., failure to identify the information required to answer the question) (81.7% of incorrect responses). CONCLUSIONS: ChatGPT performs below a threshold likely to pass the American Board of Orthopedic Surgery (ABOS) Part I written exam. The chatbot's performance on the 2022 and 2021 OITEs was between the average performance of an intern and to second-year resident. A major limitation of the current model is the failure to identify the information required to correctly answer the questions.

ChatGPT Is Moderately Accurate in Providing a General Overview of Orthopaedic Conditions.

Background: ChatGPT is an artificial intelligence chatbot capable of providing human-like responses for virtually every possible inquiry. This advancement has provoked public interest regarding the use of ChatGPT, including in health care. The purpose of the present study was to investigate the quantity and accuracy of ChatGPT outputs for general patient-focused inquiries regarding 40 orthopaedic conditions. Methods: For each of the 40 conditions, ChatGPT (GPT-3.5) was prompted with the text "I have been diagnosed with [condition]. Can you tell me more about it?" The numbers of treatment options, risk factors, and symptoms given for each condition were compared with the number in the corresponding American Academy of Orthopaedic Surgeons (AAOS) OrthoInfo website article for information quantity assessment. For accuracy assessment, an attending orthopaedic surgeon ranked the outputs in the categories of <50%, 50% to 74%, 75% to 99%, and 100% accurate. An orthopaedics sports medicine fellow also independently ranked output accuracy. Results: Compared with the AAOS OrthoInfo website, ChatGPT provided significantly fewer treatment options (mean difference, -2.5; p < 0.001) and risk factors (mean difference, -1.1; p = 0.02) but did not differ in the number of symptoms given (mean difference, -0.5; p = 0.31). The surgical treatment options given by ChatGPT were often nondescript (n = 20 outputs), such as "surgery" as the only operative treatment option. Regarding accuracy, most conditions (26 of 40; 65%) were ranked as mostly (75% to 99%) accurate, with the others (14 of 40; 35%) ranked as moderately (50% to 74%) accurate, by an attending surgeon. Neither surgeon ranked any condition as mostly inaccurate (<50% accurate). Interobserver agreement between accuracy ratings was poor (κ = 0.03; p = 0.30). Conclusions: ChatGPT provides at least moderately accurate outputs for general inquiries of orthopaedic conditions but is lacking in the quantity of information it provides for risk factors and treatment options. Professional organizations, such as the AAOS, are the preferred source of musculoskeletal information when compared with ChatGPT. Clinical Relevance: ChatGPT is an emerging technology with potential roles and limitations in patient education that are still being explored.

Mouse Lumbar Vertebra Uniaxial Compression Testing with Embedding of the Loading Surface.

There is increasing awareness that cortical and cancellous bone differ in regulating and responding to pharmaceutical therapies, hormone therapies, and other treatments for age-related bone loss. Three-point bending is a common method used to assess the influence of a treatment on the mid-diaphysis region of long bones, which is rich in cortical bone. Uniaxial compression testing of mouse vertebrae, though capable of assessing bones rich in cancellous bone, is less commonly performed due to technical challenges. Even less commonly performed is the pairing of three-point bending and compression testing to determine how a treatment may influence a long bone's mid-diaphysis region and a vertebral centrum similarly or differently. Here, we describe two procedures to make compression testing of mouse lumbar vertebrae a less challenging method to perform in parallel with three-point bending: first, a procedure to convert a three-point bending machine into a compression testing machine, and second, an embedding method for preparing a mouse lumbar vertebra loading surface.

Dietary Hempseed Decreases Femur Maximum Load in a Young Female C57BL/6 Mouse Model but Does Not Influence Bone Mineral Density or Micro-Architecture.

Numerous seed and seed extract diets have been investigated as a means of combating age-related bone loss, with many findings suggesting that the seeds/extracts confer positive effects on bone. Recently, there has been rising interest in the use of dietary hempseed in human and animal diets due to a perceived health benefit from the seed. Despite this, there has been a lack of research investigating the physiologic effects of dietary hempseed on bone. Previous studies have suggested that hempseed may enhance bone strength. However, a complete understanding of the effects of hempseed on bone mineralization, bone micro-architecture, and bone biomechanical properties is lacking. Using a young and developing female C57BL/6 mouse model, we aimed to fill these gaps in knowledge. From five to twenty-nine weeks of age, the mice were raised on either a control (0%), 50 g/kg (5%), or 150 g/kg (15%) hempseed diet (n = 8 per group). It was found that the diet did not influence the bone mineral density or micro-architecture of either the right femur or L5 vertebrae. Furthermore, it did not influence the stiffness, yield load, post-yield displacement, or work-to-fracture of the right femur. Interestingly, it reduced the maximum load of the right femur in the 15% hempseed group compared to the control group. This finding suggests that a hempseed-enriched diet provides no benefit to bone in young, developing C57BL/6 mice and may even reduce bone strength.

The Impact of Hempseed Consumption on Bone Parameters and Body Composition in Growing Female C57BL/6 Mice.

Optimizing peak bone mass is critical to healthy aging. Beyond the established roles of dietary minerals and protein on bone integrity, fatty acids and polyphenols modify bone structure. This study investigated the effect of a diet containing hempseeds (HS), which are rich in polyunsaturated fatty acids and polyphenols, on bone mineral density, bone cell populations and body composition. Groups (n = 8 each) of female C57BL/6 mice were fed one of three diets (15% HS by weight; 5% HS; 0% HS (control)) from age 5 to 30 weeks. In vivo whole-body composition and bone mineral density and content were measured every 4 weeks using dual-energy X-ray absorptiometry. Ex vivo humeri cell populations in the epiphyseal plate region were determined by sectioning the bone longitudinally, mounting the sections on slides and staining with tartrate-resistant acid phosphatase and alkaline phosphatase stain to identify osteoclasts and osteoblasts, respectively. Mixed models with repeated measures across experimental weeks showed that neither body weight nor body weight gain across weeks differed among groups yet mice fed the 15% HS diet consumed significantly more food and more kilocalories per g body weight gained than those fed the 5% HS and control diets (p < 0.0001). Across weeks, fat mass was significantly higher in the 5% HS versus the control group (p = 0.02). At the end point, whole-body bone mineral content was significantly higher in the control compared to the 5% HS group (p = 0.02). Humeri from both HS groups displayed significantly lower osteoblast densities compared to the control group (p < 0.0001). No relationship was seen between osteoblast density and body composition measurements. These data invite closer examination of bone cell activity and microarchitecture to determine the effect of habitual HS consumption on bone integrity.