RESUMO
BACKGROUND: Chat Generative Pre-Trained Transformer (ChatGPT) is an artificial intelligence (AI) chatbot capable of delivering human-like responses to a seemingly infinite number of inquiries. For the technology to perform certain healthcare-related tasks or act as a study aid, the technology should have up-to-date knowledge and the ability to reason through medical information. The purpose of this study was to assess the orthopedic knowledge and reasoning ability of ChatGPT by querying it with orthopedic board-style questions. METHODOLOGY: We queried ChatGPT (GPT-3.5) with a total of 472 questions from the Orthobullets dataset (n = 239), the 2022 Orthopaedic In-Training Examination (OITE) (n = 124), and the 2021 OITE (n = 109). The importance, difficulty, and category were recorded for questions from the Orthobullets question bank. Responses were assessed for answer choice correctness if the explanation given matched that of the dataset, answer integrity, and reason for incorrectness. RESULTS: ChatGPT correctly answered 55.9% (264/472) of questions and, of those answered correctly, gave an explanation that matched that of the dataset for 92.8% (245/264) of the questions. The chatbot used information internal to the question in all responses (100%) and used information external to the question (98.3%) as well as logical reasoning (96.4%) in most responses. There was no significant difference in the proportion of questions answered correctly between the datasets (P = 0.62). There was no significant difference in the proportion of questions answered correctly by question category (P = 0.67), importance (P = 0.95), or difficulty (P = 0.87) within the Orthobullets dataset questions. ChatGPT mostly got questions incorrect due to information error (i.e., failure to identify the information required to answer the question) (81.7% of incorrect responses). CONCLUSIONS: ChatGPT performs below a threshold likely to pass the American Board of Orthopedic Surgery (ABOS) Part I written exam. The chatbot's performance on the 2022 and 2021 OITEs was between the average performance of an intern and to second-year resident. A major limitation of the current model is the failure to identify the information required to correctly answer the questions.
RESUMO
Background: ChatGPT is an artificial intelligence chatbot capable of providing human-like responses for virtually every possible inquiry. This advancement has provoked public interest regarding the use of ChatGPT, including in health care. The purpose of the present study was to investigate the quantity and accuracy of ChatGPT outputs for general patient-focused inquiries regarding 40 orthopaedic conditions. Methods: For each of the 40 conditions, ChatGPT (GPT-3.5) was prompted with the text "I have been diagnosed with [condition]. Can you tell me more about it?" The numbers of treatment options, risk factors, and symptoms given for each condition were compared with the number in the corresponding American Academy of Orthopaedic Surgeons (AAOS) OrthoInfo website article for information quantity assessment. For accuracy assessment, an attending orthopaedic surgeon ranked the outputs in the categories of <50%, 50% to 74%, 75% to 99%, and 100% accurate. An orthopaedics sports medicine fellow also independently ranked output accuracy. Results: Compared with the AAOS OrthoInfo website, ChatGPT provided significantly fewer treatment options (mean difference, -2.5; p < 0.001) and risk factors (mean difference, -1.1; p = 0.02) but did not differ in the number of symptoms given (mean difference, -0.5; p = 0.31). The surgical treatment options given by ChatGPT were often nondescript (n = 20 outputs), such as "surgery" as the only operative treatment option. Regarding accuracy, most conditions (26 of 40; 65%) were ranked as mostly (75% to 99%) accurate, with the others (14 of 40; 35%) ranked as moderately (50% to 74%) accurate, by an attending surgeon. Neither surgeon ranked any condition as mostly inaccurate (<50% accurate). Interobserver agreement between accuracy ratings was poor (κ = 0.03; p = 0.30). Conclusions: ChatGPT provides at least moderately accurate outputs for general inquiries of orthopaedic conditions but is lacking in the quantity of information it provides for risk factors and treatment options. Professional organizations, such as the AAOS, are the preferred source of musculoskeletal information when compared with ChatGPT. Clinical Relevance: ChatGPT is an emerging technology with potential roles and limitations in patient education that are still being explored.
RESUMO
Protein adsorption to solid carbohydrate interfaces is critical to many biological processes, particularly in biomass deconstruction. To engineer more-efficient enzymes for biomass deconstruction into sugars, it is necessary to characterize the complex protein-carbohydrate interfacial interactions. A carbohydrate-binding module (CBM) is often associated with microbial surface-tethered cellulosomes or secreted cellulase enzymes to enhance substrate accessibility. However, it is not well known how CBMs recognize, bind, and dissociate from polysaccharides to facilitate efficient cellulolytic activity, due to the lack of mechanistic understanding and a suitable toolkit to study CBM-substrate interactions. Our work outlines a general approach to study the unbinding behavior of CBMs from polysaccharide surfaces using a highly multiplexed single-molecule force spectroscopy assay. Here, we apply acoustic force spectroscopy (AFS) to probe a Clostridium thermocellum cellulosomal scaffoldin protein (CBM3a) and measure its dissociation from nanocellulose surfaces at physiologically relevant, low force loading rates. An automated microfluidic setup and method for uniform deposition of insoluble polysaccharides on the AFS chip surfaces are demonstrated. The rupture forces of wild-type CBM3a, and its Y67A mutant, unbinding from nanocellulose surfaces suggests distinct multimodal CBM binding conformations, with structural mechanisms further explored using molecular dynamics simulations. Applying classical dynamic force spectroscopy theory, the single-molecule unbinding rate at zero force is extrapolated and found to agree with bulk equilibrium unbinding rates estimated independently using quartz crystal microbalance with dissipation monitoring. However, our results also highlight critical limitations of applying classical theory to explain the highly multivalent binding interactions for cellulose-CBM bond rupture forces exceeding 15 pN.