Assessment of Radiology Artificial Intelligence Software: A Validation and Evaluation Framework.

Artificial intelligence (AI) software in radiology is becoming increasingly prevalent and performance is improving rapidly with new applications for given use cases being developed continuously, oftentimes with development and validation occurring in parallel. Several guidelines have provided reporting standards for publications of AI-based research in medicine and radiology. Yet, there is an unmet need for recommendations on the assessment of AI software before adoption and after commercialization. As the radiology AI ecosystem continues to grow and mature, a formalization of system assessment and evaluation is paramount to ensure patient safety, relevance and support to clinical workflows, and optimal allocation of limited AI development and validation resources before broader implementation into clinical practice. To fulfil these needs, we provide a glossary for AI software types, use cases and roles within the clinical workflow; list healthcare needs, key performance indicators and required information about software prior to assessment; and lay out examples of software performance metrics per software category. This conceptual framework is intended to streamline communication with the AI software industry and provide healthcare decision makers and radiologists with tools to assess the potential use of these software. The proposed software evaluation framework lays the foundation for a radiologist-led prospective validation network of radiology AI software. Learning Points: The rapid expansion of AI applications in radiology requires standardization of AI software specification, classification, and evaluation. The Canadian Association of Radiologists' AI Tech & Apps Working Group Proposes an AI Specification document format and supports the implementation of a clinical expert evaluation process for Radiology AI software.

Maternal pre-pregnancy obesity and health care utilization and costs in the offspring.

BACKGROUND/OBJECTIVE: The association between maternal pre-pregnancy obesity and adverse child health outcomes is well described, but there are few data on the relationship with offspring health service use. We examined the influence of maternal pre-pregnancy obesity on offspring health care utilization and costs over the first 18 years of life. METHODS: This was a population-based retrospective cohort study of children (n = 35,090) born between 1989 and 1993 and their mothers, who were identified using the Nova Scotia Atlee Perinatal Database and linked to provincial administrative health data from birth through 2014. The primary outcome was health care utilization as determined by the number and cost of physician visits, hospital admissions and days, and high utilizer status (>95th percentile of physician visits). The secondary outcome was health care utilization by ICD chapter. Maternal pre-pregnancy weight was categorized as normal weight, overweight, or obese. Multivariable-adjusted regression models were used to examine the association between maternal weight status and offspring health care use. RESULTS: Children of mothers with pre-pregnancy obesity had more physician visits (10%), hospital admissions (16%), and hospital days (10%) than children from mothers of normal weight over the first 18 years of life. Offspring of mothers with obesity had C$356 higher physician costs and C$1415 hospital costs over 18 years than offspring of normal weight mothers. Children of mothers with obesity were 1.74 times more likely to be a high utilizer of health care and had higher rates of physician visits and hospital stays for nervous system and sense organ disorders, respiratory disorders, and gastrointestinal disorders compared to children of normal weight mothers. CONCLUSION: Our findings suggest that maternal pre-pregnancy overweight and obesity are associated with slightly higher offspring health care utilization and costs in the first 18 years of life.

Clinical epidemiological analysis of the mortality rate of liver transplant candidates living in rural areas.

MELD score has been used to predict 90-day mortality of subjects listed for liver transplantation (OLT). Validation of MELD score for patients on the waiting list in transplant programmes serving rural areas in North America is lacking. A retrospective cohort of patients affected by end-stage liver disease was studied to assess the mortality rate within 90 days after being listed at our transplant centre. Secondary aims were to identify differences between predicted and observed 90-days mortality using MELD and MELDNa scores at the time of listing. Among 126 patients included in this study, waiting list mortality was 35.0%. Ninety-day mortality was 21.1%, which was significantly greater than the mortality estimated by the MELD (9.1%, 95% CI: 6.6-11.5) and MELDNa (9.3%, 95%CI: 6.0-12.5). Despite this underestimation, AUC for MELD and MELDNa was 0.80 and 0.78 respectively. In our study, independent predictors of waiting list mortality were age, diagnosis of cholestatic disease and residence over 500 km from our transplant centre. MELD and MELDNa underestimated the 90-day mortality in patients with liver failure living in rural areas. Validation of these models should be performed in other transplant centres serving patients with limited access to specialized services.

Outcome of penetrating keratoplasty performed by cornea fellows compared with that of an experienced staff surgeon.

PURPOSE: To compare the outcome of penetrating keratoplasty performed by supervised cornea fellows and an experienced staff surgeon. METHODS: Retrospective review of cases of penetrating keratoplasty performed over a 4-year period with corneal topography performed on a clear graft. Overall, 202 cases from 184 patients were included: 108 performed by an experienced staff surgeon and 94 cases by 9 cornea fellows. Charts were reviewed for topographic astigmatism, other topographic indices, refractive astigmatism and best-corrected visual acuity (BCVA), all recorded after removal of the 10-0 sutures while the 11-0 suture remained in place. Graft clarity was assessed at final follow-up. RESULTS: Mean postoperative topographic astigmatism was 4.60 +/- 2.7 diopters in the fellow group and 4.08 +/- 2.41 diopters in the staff group. The mean postoperative time at which corneal topography was obtained was 9.53 +/- 4.4 months in the fellow group and 9.22 +/- 4.6 months in the staff group. Mean refractive astigmatism was 3.77 +/- 1.97 diopters (fellows) and 3.49 +/- 1.93 diopters (staff). Mean surface asymmetry index (SAI) was 1.89 +/- 1.42 (fellows) and 1.61 +/- 1.08 (staff). Mean surface regularity index (SRI) was 1.43 +/- 0.72 (fellows) and 1.45 +/- 0.61 (staff). Mean BCVA was 0.49 logMAR (20/61) +/- 4.3 lines (fellows) and 0.47 logMAR (20/59) +/- 3.9 lines (staff). In the fellow group, 85 (90%) cases remained clear over a mean follow-up of 32 months and in the staff group 100 (93%) grafts remained clear over a mean follow-up of 38 months. Simple and multiple regression analyses indicated no difference between the 2 groups in all outcome measures. CONCLUSIONS: Properly supervised cornea fellows in training achieved favorable penetrating keratoplasty results that were comparable to those of an experienced staff surgeon.

Visual acuity assessment: a comparison of two tests for measuring children's vision.

PURPOSE: To compare the Kay Picture Test and the logMAR Crowded Test. METHODS: Monocular visual acuity measurement was attempted on 103 paediatric subjects (aged between 2.5 and 16 years) attending eye clinics, using each of the visual acuity tests. RESULTS: The results obtained with the two tests were found to be comparable as confirmed with the Intra-class Correlation Coefficient, which revealed good agreement between the two tests. There was significant correlation between the visual acuity results measured and high conformity in the results from the weaker eye. There was a statistical difference in acuity scores between the tests; the Kay Picture Test gave a lower logMAR score with a mean difference of 0.08 logMAR. This 0.08 difference represents an actual difference of less than one line of acuity measure which can be considered to be clinically insignificant. There was a significant difference in the ability to perform each of the tests under binocular conditions, with more of the 50 children, aged 5 and under, able to perform the Kay Picture Test than the logMAR Crowded Test. CONCLUSIONS: The results obtained with the two tests are comparable. Both tests can be considered to be appropriate for amblyopia detection and the use of picture-based logMAR tests should be considered for younger children.