Evaluating ChatGPT on Orbital and Oculofacial Disorders: Accuracy and Readability Insights.

PURPOSE: To assess the accuracy and readability of responses generated by the artificial intelligence model, ChatGPT (version 4.0), to questions related to 10 essential domains of orbital and oculofacial disease. METHODS: A set of 100 questions related to the diagnosis, treatment, and interpretation of orbital and oculofacial diseases was posed to ChatGPT 4.0. Responses were evaluated by a panel of 7 experts based on appropriateness and accuracy, with performance scores measured on a 7-item Likert scale. Inter-rater reliability was determined via the intraclass correlation coefficient. RESULTS: The artificial intelligence model demonstrated accurate and consistent performance across all 10 domains of orbital and oculofacial disease, with an average appropriateness score of 5.3/6.0 ("mostly appropriate" to "completely appropriate"). Domains of cavernous sinus fistula, retrobulbar hemorrhage, and blepharospasm had the highest domain scores (average scores of 5.5 to 5.6), while the proptosis domain had the lowest (average score of 5.0/6.0). The intraclass correlation coefficient was 0.64 (95% CI: 0.52 to 0.74), reflecting moderate inter-rater reliability. The responses exhibited a high reading-level complexity, representing the comprehension levels of a college or graduate education. CONCLUSIONS: This study demonstrates the potential of ChatGPT 4.0 to provide accurate information in the field of ophthalmology, specifically orbital and oculofacial disease. However, challenges remain in ensuring accurate and comprehensive responses across all disease domains. Future improvements should focus on refining the model's correctness and eventually expanding the scope to visual data interpretation. Our results highlight the vast potential for artificial intelligence in educational and clinical ophthalmology contexts.

Exploring the potential utility of AI large language models for medical ethics: an expert panel evaluation of GPT-4.

Integrating large language models (LLMs) like GPT-4 into medical ethics is a novel concept, and understanding the effectiveness of these models in aiding ethicists with decision-making can have significant implications for the healthcare sector. Thus, the objective of this study was to evaluate the performance of GPT-4 in responding to complex medical ethical vignettes and to gauge its utility and limitations for aiding medical ethicists. Using a mixed-methods, cross-sectional survey approach, a panel of six ethicists assessed LLM-generated responses to eight ethical vignettes.The main outcomes measured were relevance, reasoning, depth, technical and non-technical clarity, as well as acceptability of GPT-4's responses. The readability of the responses was also assessed. Of the six metrics evaluating the effectiveness of GPT-4's responses, the overall mean score was 4.1/5. GPT-4 was rated highest in providing technical (4.7/5) and non-technical clarity (4.4/5), whereas the lowest rated metrics were depth (3.8/5) and acceptability (3.8/5). There was poor-to-moderate inter-rater reliability characterised by an intraclass coefficient of 0.54 (95% CI: 0.30 to 0.71). Based on panellist feedback, GPT-4 was able to identify and articulate key ethical issues but struggled to appreciate the nuanced aspects of ethical dilemmas and misapplied certain moral principles.This study reveals limitations in the ability of GPT-4 to appreciate the depth and nuanced acceptability of real-world ethical dilemmas, particularly those that require a thorough understanding of relational complexities and context-specific values. Ongoing evaluation of LLM capabilities within medical ethics remains paramount, and further refinement is needed before it can be used effectively in clinical settings.

The Isabel Differential Diagnosis Generator for Orbital Diagnosis.

PURPOSE: The Isabel differential diagnosis generator is one of the most widely known electronic diagnosis decision support tools. The authors prospectively evaluated the utility of Isabel for orbital disease differential diagnosis. METHODS: The terms "proptosis," "lid retraction," "orbit inflammation," "orbit tumour," "orbit tumor, infiltrative" and "orbital tumor, well-circumscribed" were separately input into Isabel and the results were tabulated. Then the clinical details (patient age, gender, signs, symptoms, and imaging findings) of 25 orbital cases from a textbook of orbital surgery were entered into Isabel. The top 10 differential diagnoses generated by Isabel were compared with the correct diagnosis. RESULTS: Isabel identified hyperthyroidism and Graves ophthalmopathy as the leading causes of lid retraction, but many common causes of proptosis and orbital tumors were not correctly elucidated. Of the textbook cases, Isabel correctly identified 4/25 (16%) of orbital cases as one of its top 10 differential diagnoses, and the median rank of the correct diagnosis was 6/10. Thirty-two percent of the output diagnoses were unlikely to cause orbital disease. CONCLUSION: Isabel is currently of limited value in the mainstream orbital differential diagnosis. The incorporation of anatomic localizations and imaging findings may help increase the accuracy of orbital diagnosis.

A technique for excisional optic nerve biopsy.

PURPOSE: To describe a transconjunctival technique for full-thickness (excisional) optic nerve biopsy. METHOD: A medial transconjunctival approach to the optic nerve with disinsertion of the medial rectus is used. A small right-angle Mixter forcep is used to clamp the optic nerve far posteriorly, and then a microscalpel is directed metal-on-metal to cut the posterior optic nerve. The cut nerve is then rotated anteriorly to complete the proximal nerve cut. RESULT: A full-thickness specimen of 11 mm of more can be obtained without undue traction on the globe. The globe remains viable. CONCLUSION: A long length, excisional optic nerve biopsy can be readily and safely performed without endoscopic techniques.

The use of an analytic hierarchy process to promote equity, diversity and inclusion.

Equity, diversity and inclusion (EDI) are increasingly important directives in medicine that add further complexity to adjudications. The analytic hierarchy process is proposed as a tool for multicriteria decision-making that can facilitate the incorporation of EDI directives, especially for collective, group determinations.

Gender and editorship in oculoplastics societal publications.

PURPOSE: The editorship of medical journals is a leadership role that can affect recognition and career advancement. We determine the gender representation of the editorial boards of oculoplastic surgery journals in comparison to the proportion of women in oculoplastics societies. METHODS: The gender composition of the American, European and Asia-Pacific societies of oculoplastic and reconstructive surgery and the editorial boards of their respective society journals were determined with online searches in March 2021. Statistical tests for the equality of proportions were performed. RESULTS: Excluding 44 individuals with missing gender data, the three combined oculoplastics societies comprised 1,230 distinct members, with 29% women. The editorial review boards of the three official society publications comprised 59 medical editors, 22% of which were women. There was no statistically significant difference in the proportion of women editors versus women OPRS members (p = .201) but the study is underpowered to detect a 7% difference. A sensitivity analysis with the missing data did not alter the conclusions. The mean h-index/m-quotient of the women editors was 20.50/0.87 and for the men 21.05/0.84, with no statistically significant difference (p = .903/0.851). CONCLUSION: Women are underrepresented on the editorial boards of oculoplastic journals. Possible methods to improve gender balance include multicriteria objective decision-making criteria for editor nominations, mentoring peer reviewers that are women, and appointing a journal editor for equity, diversity and inclusion.

A Survey-Weighted Analytic Hierarchy Process to Quantify Authorship.

BACKGROUND: Authorship is a pinnacle activity in academic medicine that often involves collaboration and a mentor-mentee relationship. The International Committee of Medical Journal Editors criteria for authorship (ICMJEc) are intended to prevent abuses of authorship and are used by more than 5500 medical journals. However, the binary ICMJEc have not yet been quantified. AIM: To develop a numeric scoring rubric for the ICMJEc to corroborate the authenticity of authorship claims. METHODS: The four ICMJEc were separated into the nine authorship components of conception, design, data acquisition, data analysis, interpretation of data, draft, revision, final approval and accountability. In spring 2021, members of an international association of medical editors rated the importance of each authorship component using an 11-point Likert scale ranging from 0 (no importance) to 10 (most important). The median component scores were used to calibrate the pairwise comparisons in an analytic hierarchy process (AHP). The AHP priority weights were multiplied against a four-level perceived effort/capability grade to calculate an authorship score. RESULTS: Sixty-six decision-making medical editors completed the survey. The components had the median scores/AHP weights: conception 7.5/5.3%; design 8/8.9%; data acquisition 7/3.6%; data analysis 7/3.6%; interpretation of data 8/8.9%; draft 8/8.9%; revision 8/8.9%; final approval 9/20.1%; and accountability 10/31.8%, with Kruskal-Wallis Chi2 = 65.11, p < 0.001. CONCLUSION: The editors rated accountability as the most important component of authorship, followed by the final approval of the manuscript; data acquisition had the lowest median importance score for authorship. The scoring rubric (https://tinyurl.com/eyu86y96) transforms the binary tetrad ICMJEc into 9 quantifiable components of authorship, providing a transparent method to objectively assess authorship contributions, determine authorship order and potentially decrease the abuse of authorship. If desired, individual journals can survey their editorial boards and use the AHP method to derive customized weightings for an ICMJEc-based authorship index.