Interaction between clinicians and artificial intelligence to detect fetal atrioventricular septal defects on ultrasound: how can we optimize collaborative performance?

OBJECTIVES: Artificial intelligence (AI) has shown promise in improving the performance of fetal ultrasound screening in detecting congenital heart disease (CHD). The effect of giving AI advice to human operators has not been studied in this context. Giving additional information about AI model workings, such as confidence scores for AI predictions, may be a way of improving performance further. Our aims were to investigate whether AI advice improved overall diagnostic accuracy (using a single CHD lesion as an exemplar), and to see what, if any, additional information given to clinicians optimized the overall performance of the clinician-AI team. METHODS: An AI model was trained to classify a single fetal CHD lesion (atrioventricular septal defect, AVSD), using a retrospective cohort of 121,130 cardiac four chamber images extracted from 173 ultrasound scan videos (98 with normal hearts, 75 with AVSD). A ResNet50 model architecture was used. Temperature scaling of model prediction probability was performed on a validation set, and gradient-weighted class activation maps (grad-CAMs) produced. Ten clinicians (two consultant fetal cardiologists, three trainees in pediatric cardiology, and five fetal cardiac sonographers) were recruited from a center of fetal cardiology to participate. Each participant was shown 2000 fetal four chamber images in a random order (1,000 normal and 1,000 AVSD). The dataset was comprised of 500 images, each shown in four conditions: 1) image alone without AI output; 2) image with binary AI classification; 3) image with AI model confidence; 4) image with gradient-weighted class activation map image overlays. The clinicians were asked to classify each image as normal or AVSD. RESULTS: 20,000 image classifications were recorded from 10 clinicians. The AI model alone achieved an accuracy of 0.798 (95% CI 0.760 - 0.832), sensitivity of 0.868 (95% CI 0.834 - 0.902) and specificity of 0.728 (95% CI 0.702 - 0.754, and the clinicians without AI achieved an accuracy of 0.844 (95% CI 0.834 - 0.854), sensitivity of 0.827 (95% CI 0.795 - 0.858) and specificity of 0.861 (95% CI 0.828 - 0.895). Showing a binary (normal or AVSD) AI model output resulted in significant improvement in accuracy to 0.865 (p <0.001). This effect was seen in both experienced and less experienced participants. Giving incorrect AI advice resulted in significant deterioration in overall accuracy from 0.761 to 0.693 (p <0.001), which was driven by an increase in both type I and type II error by the clinicians. This effect was worsened by showing model confidence (accuracy 0.649, p <0.001) or grad-CAM (accuracy 0.644, p <0.001). CONCLUSIONS: AI has the potential to improve performance when used in collaboration with clinicians, even if the model performance does not reach expert level. Giving additional information about model workings such as model confidence and class activation map image overlays did not improve overall performance, and actually worsened performance for images where the AI model was incorrect. This article is protected by copyright. All rights reserved.

Unusual Vascular Ring in the Fetus.

We present the clinical course and echocardiographic and genetic findings of two fetuses with an unusual vascular ring, created by a left aortic arch with a right arterial duct and an aberrant right subclavian artery. One fetus was diagnosed with 22q11.2 microdeletion and the other became symptomatic in infancy. It is important to consider the position of the arterial ductal ligament in patients who present with tracheoesophageal compressive symptoms in the presence of a left aortic arch. These cases also highlight that a vascular ring formed from a left arch may have similar associations to a vascular ring formed by a right aortic arch.

Impact of prospective measurement of outflow tracts in prediction of coarctation of the aorta.

OBJECTIVES: Prenatal diagnosis of coarctation of the aorta (CoA) is associated with reduced mortality and morbidity, however, accurate prenatal prediction remains challenging. To date, studies have used retrospective measurements of the outflow tracts to evaluate their potential to predict CoA. Our primary objective was to evaluate prospectively acquired measurements of the outflow tracts in fetuses with prenatally suspected CoA. A secondary aim was to report the postnatal prevalence of bicuspid aortic valve in this cohort. METHODS: Pregnancies with suspicion of isolated CoA and with a minimum of 6 months' postnatal follow-up available were identified from the cardiac database of a tertiary fetal cardiology center in the UK, between January 2002 and December 2017. Measurement of the aortic valve, pulmonary valve, distal transverse aortic arch (DTAA) and arterial duct (AD) diameters were undertaken routinely in fetuses with suspected CoA during the study period. Z-scores were computed using published reference ranges based on > 7000 fetuses from our own unit. RESULTS: Of 149 pregnancies with prenatally suspected CoA included in the study, CoA was confirmed within 6 months after birth in 77/149 (51.7%) cases. DTAA diameter Z-score and the Z-score of second-trimester DTAA/AD diameter ratio were smaller in fetuses with postnatally confirmed CoA than those in false-positive cases (-2.8 vs -1.9; P = 0.039 and -3.13 vs -2.61; P = 0.005, respectively). Multiple regression analysis demonstrated that the Z-scores of DTAA and AD diameters were the only significant predictors of postnatal CoA (P = 0.001). Bicuspid aortic valve was identified in 30% of the false-positive cases. CONCLUSIONS: Measurement of DTAA and AD diameter Z-scores can be used to ascertain risk for postnatal CoA in a selected cohort. The high incidence of bicuspid aortic valve in false-positive cases merits further study with respect to both etiology and longer-term significance. Copyright © 2019 ISUOG. Published by John Wiley & Sons Ltd.

Assessment of cardiac angle in fetuses with congenital heart disease at risk of 22q11.2 deletion.

OBJECTIVES: To evaluate fetal cardiac angle as a screening tool for 22q11.2 deletion among cases with cardiac anomalies known to be associated with this genetic condition, to examine the correlation of fetal cardiac angle with thymic-thoracic (TT)-ratio, and to assess the performance of TT ratio as a covariate in screening for 22q11.2 deletion. METHODS: This was a retrospective cohort study that reviewed the records of 74 cases with cardiac anomalies known to be associated with 22q11.2 deletion (tetralogy of Fallot, common arterial trunk, interrupted aortic arch and right aortic arch) that were diagnosed between 2007 and 2013. The karyotype was known in all cases. The fetal cardiac angle and TT-ratio were measured using stored three-dimensional spatiotemporal image correlation volume datasets and compared in those with del.22q11.2 and those without. RESULTS: Of the 74 cases reviewed, 16 had 22q11.2 deletion. The mean cardiac angle was larger in the cases with 22q11.2 deletion than in those without (68.6° vs 58.7°, respectively; P = 0.02). Multivariate regression analysis showed an association between cardiac angle and TT-ratio in fetuses with 22q11.2 deletion (r(2) = 0.33; P = 0.02) but not in those with a normal karyotype (P = 0.4). Logistic regression analysis demonstrated that fetal cardiac angle, but not TT-ratio, is an independent predictor of 22q11.2 deletion among fetuses with 22q11.2 deletion-associated cardiac anomalies (P = 0.02; area under the receiver-operating characteristics curve = 0.69). CONCLUSIONS: An enlarged fetal cardiac angle is an independent predictor of 22q11.2 deletion among fetuses with 22q11.2 deletion-associated cardiac anomalies. However, its performance as a single variable in a screening model is not sufficient to guide management decisions regarding invasive testing.

Multi-method evaluation of a paediatric ambulatory care unit (PACU): impact on families and staff.

AIM: To assess the impact of a purpose-built, short stay paediatric ambulatory care unit (PACU) on the patient journey and perceptions of parents, staff and referrers. METHODS: Multi-method evaluation, including a parent survey (n = 104), patient journey mapping (n = 10), staff interviews (n = 10), a referrer survey (n = 16), routine activity analysis, and a comparison with the A&E service (A&E parent survey: n = 41). RESULTS: Almost all parents attending PACU (94%) were satisfied with the service and significantly more likely to feel "very" satisfied than parents attending A&E (PACU: 51%, A&E: 31%; p = 0.03). Further, over three quarters (77%) of PACU parents preferred the new model to traditional A&E services. They reported receiving sufficient information (93%), reduced anxiety (55% anxious before service, 13% anxious after; p<0.001), "quick" waiting times (median: 35 min), and enhanced confidence (87%) and understanding (89%) in dealing with their ill child. The number of stages in the patient journey was reduced from six ("traditional" A&E pathway) to four (PACU pathway). Staff and referrers reported this was a "superior" model to A&E, but that improvements were required around appropriate referrals and the need for more multi-disciplinary protocols and liaison. CONCLUSION: Our study suggests that the PACU model is perceived to be an effective alternative to standard A&E services for the assessment and early management of acutely ill children and their families attending a hospital. It is highly valued by users, staff and referrers and enhances the patient journey. Lessons learnt include the need to enhance multi-disciplinary processes and clarify the role of this form of acute care provision in the wider healthcare system.