AI and machine learning in medical imaging: key points from development to translation.

Innovation in medical imaging artificial intelligence (AI)/machine learning (ML) demands extensive data collection, algorithmic advancements, and rigorous performance assessments encompassing aspects such as generalizability, uncertainty, bias, fairness, trustworthiness, and interpretability. Achieving widespread integration of AI/ML algorithms into diverse clinical tasks will demand a steadfast commitment to overcoming issues in model design, development, and performance assessment. The complexities of AI/ML clinical translation present substantial challenges, requiring engagement with relevant stakeholders, assessment of cost-effectiveness for user and patient benefit, timely dissemination of information relevant to robust functioning throughout the AI/ML lifecycle, consideration of regulatory compliance, and feedback loops for real-world performance evidence. This commentary addresses several hurdles for the development and adoption of AI/ML technologies in medical imaging. Comprehensive attention to these underlying and often subtle factors is critical not only for tackling the challenges but also for exploring novel opportunities for the advancement of AI in radiology.

Artificial intelligence in medicine: mitigating risks and maximizing benefits via quality assurance, quality control, and acceptance testing.

The adoption of artificial intelligence (AI) tools in medicine poses challenges to existing clinical workflows. This commentary discusses the necessity of context-specific quality assurance (QA), emphasizing the need for robust QA measures with quality control (QC) procedures that encompass (1) acceptance testing (AT) before clinical use, (2) continuous QC monitoring, and (3) adequate user training. The discussion also covers essential components of AT and QA, illustrated with real-world examples. We also highlight what we see as the shared responsibility of manufacturers or vendors, regulators, healthcare systems, medical physicists, and clinicians to enact appropriate testing and oversight to ensure a safe and equitable transformation of medicine through AI.

AAPM task group report 273: Recommendations on best practices for AI and machine learning for computer-aided diagnosis in medical imaging.

Rapid advances in artificial intelligence (AI) and machine learning, and specifically in deep learning (DL) techniques, have enabled broad application of these methods in health care. The promise of the DL approach has spurred further interest in computer-aided diagnosis (CAD) development and applications using both "traditional" machine learning methods and newer DL-based methods. We use the term CAD-AI to refer to this expanded clinical decision support environment that uses traditional and DL-based AI methods. Numerous studies have been published to date on the development of machine learning tools for computer-aided, or AI-assisted, clinical tasks. However, most of these machine learning models are not ready for clinical deployment. It is of paramount importance to ensure that a clinical decision support tool undergoes proper training and rigorous validation of its generalizability and robustness before adoption for patient care in the clinic. To address these important issues, the American Association of Physicists in Medicine (AAPM) Computer-Aided Image Analysis Subcommittee (CADSC) is charged, in part, to develop recommendations on practices and standards for the development and performance assessment of computer-aided decision support systems. The committee has previously published two opinion papers on the evaluation of CAD systems and issues associated with user training and quality assurance of these systems in the clinic. With machine learning techniques continuing to evolve and CAD applications expanding to new stages of the patient care process, the current task group report considers the broader issues common to the development of most, if not all, CAD-AI applications and their translation from the bench to the clinic. The goal is to bring attention to the proper training and validation of machine learning algorithms that may improve their generalizability and reliability and accelerate the adoption of CAD-AI systems for clinical decision support.

Evaluation of computer-aided detection and diagnosis systems.

Computer-aided detection and diagnosis (CAD) systems are increasingly being used as an aid by clinicians for detection and interpretation of diseases. Computer-aided detection systems mark regions of an image that may reveal specific abnormalities and are used to alert clinicians to these regions during image interpretation. Computer-aided diagnosis systems provide an assessment of a disease using image-based information alone or in combination with other relevant diagnostic data and are used by clinicians as a decision support in developing their diagnoses. While CAD systems are commercially available, standardized approaches for evaluating and reporting their performance have not yet been fully formalized in the literature or in a standardization effort. This deficiency has led to difficulty in the comparison of CAD devices and in understanding how the reported performance might translate into clinical practice. To address these important issues, the American Association of Physicists in Medicine (AAPM) formed the Computer Aided Detection in Diagnostic Imaging Subcommittee (CADSC), in part, to develop recommendations on approaches for assessing CAD system performance. The purpose of this paper is to convey the opinions of the AAPM CADSC members and to stimulate the development of consensus approaches and "best practices" for evaluating CAD systems. Both the assessment of a standalone CAD system and the evaluation of the impact of CAD on end-users are discussed. It is hoped that awareness of these important evaluation elements and the CADSC recommendations will lead to further development of structured guidelines for CAD performance assessment. Proper assessment of CAD system performance is expected to increase the understanding of a CAD system's effectiveness and limitations, which is expected to stimulate further research and development efforts on CAD technologies, reduce problems due to improper use, and eventually improve the utility and efficacy of CAD in clinical practice.

Quality assurance and training procedures for computer-aided detection and diagnosis systems in clinical use.

Computer-aided detection/diagnosis (CAD) is increasingly used for decision support by clinicians for detection and interpretation of diseases. However, there are no quality assurance (QA) requirements for CAD in clinical use at present. QA of CAD is important so that end users can be made aware of changes in CAD performance both due to intentional or unintentional causes. In addition, end-user training is critical to prevent improper use of CAD, which could potentially result in lower overall clinical performance. Research on QA of CAD and user training are limited to date. The purpose of this paper is to bring attention to these issues, inform the readers of the opinions of the members of the American Association of Physicists in Medicine (AAPM) CAD subcommittee, and thus stimulate further discussion in the CAD community on these topics. The recommendations in this paper are intended to be work items for AAPM task groups that will be formed to address QA and user training issues on CAD in the future. The work items may serve as a framework for the discussion and eventual design of detailed QA and training procedures for physicists and users of CAD. Some of the recommendations are considered by the subcommittee to be reasonably easy and practical and can be implemented immediately by the end users; others are considered to be "best practice" approaches, which may require significant effort, additional tools, and proper training to implement. The eventual standardization of the requirements of QA procedures for CAD will have to be determined through consensus from members of the CAD community, and user training may require support of professional societies. It is expected that high-quality CAD and proper use of CAD could allow these systems to achieve their true potential, thus benefiting both the patients and the clinicians, and may bring about more widespread clinical use of CAD for many other diseases and applications. It is hoped that the awareness of the need for appropriate CAD QA and user training will stimulate new ideas and approaches for implementing such procedures efficiently and effectively as well as funding opportunities to fulfill such critical efforts.

Computerized Classification Method for Differentiating Between Benign and Malignant Lesions on Breast MR Images.

Contrast-enhanced breast MRI has been shown to have very high sensitivity in the detection of breast cancers. A new computerized classification method for differentiating between benign and malignant lesions on breast MRIs was developed. This method was based on temporal feature analysis. We experimented with a set of thresholds of the contrast uptake and washout speed to automatically determine suspicious malignant areas. An angiogenesis map was generated to indicate suspicious malignant areas by color. The results obtained from the retrospective analysis on 64 malignant and 29 benign breast lesions showed that our method achieved 90.5% (57/63) sensitivity in detecting malignant lesions, and it correctly classified 55% (16/29) benign lesions as benign. The study results demonstrated the effectiveness of this temporal feature analysis method for the detection of malignant lesions and its performance in delineating malignant lesions from benign lesions.

Automated segmentation of breast lesions in ultrasound images.

Breast cancer is one of the leading causes of death in women. As a convenient and safe diagnosis method, ultrasound is most commonly used second to mammography for early detection and diagnosis of breast cancer. Here we proposed an automatic method to segment lesions in ultrasound images. The images are first filtered with anisotropic diffusion algorithm to remove speckle noise. The edge is enhanced to emphasize the lesion regions. Normalized cut is a graph theoretic that admits combination of different features for image segmentation, and has been successfully used in object parsing and grouping. In this paper we combine normalized cut with region merging method for the segmentation. The merging criteria are derived from the empirical rules used by radiologists when they interpret breast images. In the performance evaluation, we compared the computer-detected lesion boundaries with manually delineated borders. The experimental results show that the algorithm has efficient and robust performance for different kinds of lesions.

Computerized analysis of mammographic parenchymal patterns for assessing breast cancer risk: effect of ROI size and location.

The long-term goal of our research is to develop computerized radiographic markers for assessing breast density and parenchymal patterns that may be used together with clinical measures for determining the risk of breast cancer and assessing the response to preventive treatment. In our earlier studies, we found that women at high risk tended to have dense breasts with mammographic patterns that were coarse and low in contrast. With our method, computerized texture analysis is performed on a region of interest (ROI) within the mammographic image. In our current study, we investigate the effect of ROI size and ROI location on the computerized texture features obtained from 90 subjects (30 BRCA1/BRCA2 gene-mutation carriers and 60 age-matched women deemed to be at low risk for breast cancer). Mammograms were digitized at 0.1 mm pixel size and various ROI sizes were extracted from different breast regions in the craniocaudal (CC) view. Seventeen features, which characterize the density and texture of the parenchymal patterns, were extracted from the ROIs on these digitized mammograms. Stepwise feature selection and linear discriminant analysis were applied to identify features that differentiate between the low-risk women and the BRCA1/BRCA2 gene-mutation carriers. ROC analysis was used to assess the performance of the features in the task of distinguishing between these two groups. Our results show that there was a statistically significant decrease in the performance of the computerized texture features, as the ROI location was varied from the central region behind the nipple. However, we failed to show a statistically significant decrease in the performance of the computerized texture features with decreasing ROI size for the range studied.

Computerized analysis of digitized mammograms of BRCA1 and BRCA2 gene mutation carriers.

PURPOSE: To evaluate, by using computer image analysis, the mammographic density patterns of women with germ-line mutations in BRCA1 and BRCA2 genes in comparison with those of women at low risk of developing breast cancer. MATERIALS AND METHODS: Mammograms from 30 carriers of BRCA1 and BRCA2 mutations and from 142 low-risk women were collected retrospectively and digitized. In addition, 60 of the 142 low-risk women were randomly selected and age matched at 5-year intervals with the 30 mutation carriers. Mammographic features were extracted from the central regions of the breast images to characterize the mammographic density and heterogeneity of dense portions of the breast. These features were then merged into a single value related to the risk of breast cancer by using linear discriminant analysis. The applicability of these computer-extracted features and the output from linear discriminant analysis to differentiate between the carriers of BRCA1 and BRCA2 mutations and the low-risk women in the entire database and in an age-matched group were evaluated by using receiver operating characteristic analysis. RESULTS: Quantitative analysis of mammograms demonstrated that carriers of BRCA1 and BRCA2 mutations tended to have dense breast tissue, and their mammographic patterns tended to be low in contrast, with a coarse texture. Linear discriminant analysis resulted in values of the areas under the receiver operating characteristic curve of 0.91 and 0.92 in distinguishing between the BRCA1 and BRCA2 mutation carriers and the low-risk women in the entire database and the age-matched group, respectively. CONCLUSION: The computerized analysis of mammograms suggests that mammographic patterns in carriers of BRCA1 and BRCA2 mutations differ from those of women at low risk for breast cancer. Our computer-extracted features may be useful as radiographic markers for identifying women at high risk for breast cancer.

Breast cancer: effectiveness of computer-aided diagnosis observer study with independent database of mammograms.

PURPOSE: To evaluate the effectiveness of a computerized classification method as an aid to radiologists reviewing clinical mammograms for which the diagnoses were unknown to both the radiologists and the computer. MATERIALS AND METHODS: Six mammographers and six community radiologists participated in an observer study. These 12 radiologists interpreted, with and without the computer aid, 110 cases that were unknown to both the 12 radiologist observers and the trained computer classification scheme. The radiologists' performances in differentiating between benign and malignant masses without and with the computer aid were evaluated with receiver operating characteristic (ROC) analysis. Two-tailed P values were calculated for the Student t test to indicate the statistical significance of the differences in performances with and without the computer aid. RESULTS: When the computer aid was used, the average performance of the 12 radiologists improved, as indicated by an increase in the area under the ROC curve (A(z)) from 0.93 to 0.96 (P <.001), by an increase in partial area under the ROC curve ((0.90)A(')(z)) from 0.56 to 0.72 (P <.001), and by an increase in sensitivity from 94% to 98% (P =.022). No statistically significant difference in specificity was found between readings with and those without computer aid (Delta = -0.014; P =.46; 95% CI: -0.054, 0.026), where Delta is difference in specificity. When we analyzed results from the mammographers and community radiologists as separate groups, a larger improvement was demonstrated for the community radiologists. CONCLUSION: Computer-aided diagnosis can potentially help radiologists improve their diagnostic accuracy in the task of differentiating between benign and malignant masses seen on mammograms.