Radiologists in the loop: the roles of radiologists in the development of AI applications.

OBJECTIVES: To examine the various roles of radiologists in different steps of developing artificial intelligence (AI) applications. MATERIALS AND METHODS: Through the case study of eight companies active in developing AI applications for radiology, in different regions (Europe, Asia, and North America), we conducted 17 semi-structured interviews and collected data from documents. Based on systematic thematic analysis, we identified various roles of radiologists. We describe how each role happens across the companies and what factors impact how and when these roles emerge. RESULTS: We identified 9 roles that radiologists play in different steps of developing AI applications: (1) problem finder (in 4 companies); (2) problem shaper (in 3 companies); (3) problem dominator (in 1 company); (4) data researcher (in 2 companies); (5) data labeler (in 3 companies); (6) data quality controller (in 2 companies); (7) algorithm shaper (in 3 companies); (8) algorithm tester (in 6 companies); and (9) AI researcher (in 1 company). CONCLUSIONS: Radiologists can play a wide range of roles in the development of AI applications. How actively they are engaged and the way they are interacting with the development teams significantly vary across the cases. Radiologists need to become proactive in engaging in the development process and embrace new roles. KEY POINTS: • Radiologists can play a wide range of roles during the development of AI applications. • Both radiologists and developers need to be open to new roles and ways of interacting during the development process. • The availability of resources, time, expertise, and trust are key factors that impact how actively radiologists play roles in the development process.

Training opportunities of artificial intelligence (AI) in radiology: a systematic review.

OBJECTIVES: The aim is to offer an overview of the existing training programs and critically examine them and suggest avenues for further development of AI training programs for radiologists. METHODS: Deductive thematic analysis of 100 training programs offered in 2019 and 2020 (until June 30). We analyze the public data about the training programs based on their "contents," "target audience," "instructors and offering agents," and "legitimization strategies." RESULTS: There are many AI training programs offered to radiologists, yet most of them (80%) are short, stand-alone sessions, which are not part of a longer-term learning trajectory. The training programs mainly (around 85%) focus on the basic concepts of AI and are offered in passive mode. Professional institutions and commercial companies are active in offering the programs (91%), though academic institutes are limitedly involved. CONCLUSIONS: There is a need to further develop systematic training programs that are pedagogically integrated into radiology curriculum. Future training programs need to further focus on learning how to work with AI at work and be further specialized and customized to the contexts of radiology work. KEY POINTS: • Most of AI training programs are short, stand-alone sessions, which focus on the basics of AI. • The content of training programs focuses on medical and technical topics; managerial, legal, and ethical topics are marginally addressed. • Professional institutions and commercial companies are active in offering AI training; academic institutes are limitedly involved.

Promises of artificial intelligence in neuroradiology: a systematic technographic review.

PURPOSE: To conduct a systematic review of the possibilities of artificial intelligence (AI) in neuroradiology by performing an objective, systematic assessment of available applications. To analyse the potential impacts of AI applications on the work of neuroradiologists. METHODS: We identified AI applications offered on the market during the period 2017-2019. We systematically collected and structured information in a relational database and coded for the characteristics of the applications, their functionalities for the radiology workflow and their potential impacts in terms of 'supporting', 'extending' and 'replacing' radiology tasks. RESULTS: We identified 37 AI applications in the domain of neuroradiology from 27 vendors, together offering 111 functionalities. The majority of functionalities 'support' radiologists, especially for the detection and interpretation of image findings. The second-largest group of functionalities 'extends' the possibilities of radiologists by providing quantitative information about pathological findings. A small but noticeable portion of functionalities seek to 'replace' certain radiology tasks. CONCLUSION: Artificial intelligence in neuroradiology is not only in the stage of development and testing but also available for clinical practice. The majority of functionalities support radiologists or extend their tasks. None of the applications can replace the entire radiology profession, but a few applications can do so for a limited set of tasks. Scientific validation of the AI products is more limited than the regulatory approval.

The impact of AI suggestions on radiologists' decisions: a pilot study of explainability and attitudinal priming interventions in mammography examination.

Various studies have shown that medical professionals are prone to follow the incorrect suggestions offered by algorithms, especially when they have limited inputs to interrogate and interpret such suggestions and when they have an attitude of relying on them. We examine the effect of correct and incorrect algorithmic suggestions on the diagnosis performance of radiologists when (1) they have no, partial, and extensive informational inputs for explaining the suggestions (study 1) and (2) they are primed to hold a positive, negative, ambivalent, or neutral attitude towards AI (study 2). Our analysis of 2760 decisions made by 92 radiologists conducting 15 mammography examinations shows that radiologists' diagnoses follow both incorrect and correct suggestions, despite variations in the explainability inputs and attitudinal priming interventions. We identify and explain various pathways through which radiologists navigate through the decision process and arrive at correct or incorrect decisions. Overall, the findings of both studies show the limited effect of using explainability inputs and attitudinal priming for overcoming the influence of (incorrect) algorithmic suggestions.