National Cancer Institute Imaging Data Commons: Toward Transparency, Reproducibility, and Scalability in Imaging Artificial Intelligence.

The remarkable advances of artificial intelligence (AI) technology are revolutionizing established approaches to the acquisition, interpretation, and analysis of biomedical imaging data. Development, validation, and continuous refinement of AI tools requires easy access to large high-quality annotated datasets, which are both representative and diverse. The National Cancer Institute (NCI) Imaging Data Commons (IDC) hosts large and diverse publicly available cancer image data collections. By harmonizing all data based on industry standards and colocalizing it with analysis and exploration resources, the IDC aims to facilitate the development, validation, and clinical translation of AI tools and address the well-documented challenges of establishing reproducible and transparent AI processing pipelines. Balanced use of established commercial products with open-source solutions, interconnected by standard interfaces, provides value and performance, while preserving sufficient agility to address the evolving needs of the research community. Emphasis on the development of tools, use cases to demonstrate the utility of uniform data representation, and cloud-based analysis aim to ease adoption and help define best practices. Integration with other data in the broader NCI Cancer Research Data Commons infrastructure opens opportunities for multiomics studies incorporating imaging data to further empower the research community to accelerate breakthroughs in cancer detection, diagnosis, and treatment. Published under a CC BY 4.0 license.

Highdicom: a Python Library for Standardized Encoding of Image Annotations and Machine Learning Model Outputs in Pathology and Radiology.

Machine learning (ML) is revolutionizing image-based diagnostics in pathology and radiology. ML models have shown promising results in research settings, but the lack of interoperability between ML systems and enterprise medical imaging systems has been a major barrier for clinical integration and evaluation. The DICOM® standard specifies information object definitions (IODs) and services for the representation and communication of digital images and related information, including image-derived annotations and analysis results. However, the complexity of the standard represents an obstacle for its adoption in the ML community and creates a need for software libraries and tools that simplify working with datasets in DICOM format. Here we present the highdicom library, which provides a high-level application programming interface (API) for the Python programming language that abstracts low-level details of the standard and enables encoding and decoding of image-derived information in DICOM format in a few lines of Python code. The highdicom library leverages NumPy arrays for efficient data representation and ties into the extensive Python ecosystem for image processing and machine learning. Simultaneously, by simplifying creation and parsing of DICOM-compliant files, highdicom achieves interoperability with the medical imaging systems that hold the data used to train and run ML models, and ultimately communicate and store model outputs for clinical use. We demonstrate through experiments with slide microscopy and computed tomography imaging, that, by bridging these two ecosystems, highdicom enables developers and researchers to train and evaluate state-of-the-art ML models in pathology and radiology while remaining compliant with the DICOM standard and interoperable with clinical systems at all stages. To promote standardization of ML research and streamline the ML model development and deployment process, we made the library available free and open-source at https://github.com/herrmannlab/highdicom .

Interactive Multimedia Reporting Technical Considerations: HIMSS-SIIM Collaborative White Paper.

Despite technological advances in the analysis of digital images for medical consultations, many health information systems lack the ability to correlate textual descriptions of image findings linked to the actual images. Images and reports often reside in separate silos in the medical record throughout the process of image viewing, report authoring, and report consumption. Forward-thinking centers and early adopters have created interactive reports with multimedia elements and embedded hyperlinks in reports that connect the narrative text with the related source images and measurements. Most of these solutions rely on proprietary single-vendor systems for viewing and reporting in the absence of any encompassing industry standards to facilitate interoperability with the electronic health record (EHR) and other systems. International standards have enabled the digitization of image acquisition, storage, viewing, and structured reporting. These provide the foundation to discuss enhanced reporting. Lessons learned in the digital transformation of radiology and pathology can serve as a basis for interactive multimedia reporting (IMR) across image-centric medical specialties. This paper describes the standard-based infrastructure and communications to fulfill recently defined clinical requirements through a consensus from an international workgroup of multidisciplinary medical specialists, informaticists, and industry participants. These efforts have led toward the development of an Integrating the Healthcare Enterprise (IHE) profile that will serve as a foundation for interoperable interactive multimedia reporting.

What is the abdomen? Rationalising clinical and anatomical perspectives using formal semantics.

The meaning of the term 'abdomen' has become increasingly ambiguous, as it has to satisfy the contemporary requirements of natural language discourse, literature, gross and radiological anatomy and its role in ontologies supporting electronic records and data modelling. It is critical that there is an agreed understanding of the semantics of the abdominopelvic cavity, its component volumes including the abdomen proper, true and false pelvic cavities, and its boundaries and regional contents. The expression of part-whole (meronymic) relationships is essential for inferences to be drawn by computer algorithms, but unless these are rigorously reviewed and tested incorrect assumptions are drawn. The SNOMED CT terminology descriptions and hierarchy of anatomical concepts relating to the trunk were scrutinised for ambiguity and sub-optimal relationships using a panel of reference sources. Any identified errors were corrected and the impact of any changes reviewed iteratively by evaluating their effect on dependant hierarchies (modelled with the associated anatomical concepts). Anatomical concepts are generally structured according to a traditional gross standpoint, but in clinical practice covert complex regional notions are frequently used and during the evaluation process a new viewpoint relating to projectional (transmissive) or emissive radiological perspective was identified. The subtle but important differences in the boundaries, volumes and contents of these distinctive perspectives of the 'abdomen' are presented. Three significant complex variants have been identified which relate to the most common uses of the word 'abdomen'. The merits and disadvantages of using 'abdomen' as common synonym to more than one concept (polysemy) are briefly discussed and the solution adopted by SNOMED International described. The review of existing ontologies and academic literature confirmed the frequent varied use of the word 'abdomen', which raises concerns when derived data are increasingly being used remotely from the point of clinical contact, potentially leading to incorrect inferences. The documented regional truncal volumes from an anatomical regional, segmental and cross-sectional perspective have been integrated into a logical and comprehensive model suitable for computer processing. The robust modelling of meronymic hierarchies has to be rigorous to avoid systematic errors and it is thus timely that a proposed standard description of these subtly related volumes and structures is made available for discussion and comment.

DICOM Format and Protocol Standardization-A Core Requirement for Digital Pathology Success.

As the use of digital techniques in toxicologic pathology expands, challenges of scalability and interoperability come to the fore. Proprietary formats and closed single-vendor platforms prevail but depend on the availability and maintenance of multiformat conversion libraries. Expedient for small deployments, this is not sustainable at an industrial scale. Primarily known as a standard for radiology, the Digital Imaging and Communications in Medicine (DICOM) standard has been evolving to support other specialties since its inception, to become the single ubiquitous standard throughout medical imaging. The adoption of DICOM for whole slide imaging (WSI) has been sluggish. Prospects for widespread commercially viable clinical use of digital pathology change the incentives. Connectathons using DICOM have demonstrated its feasibility for WSI and virtual microscopy. Adoption of DICOM for digital and computational pathology will allow the reuse of enterprise-wide infrastructure for storage, security, and business continuity. The DICOM embedded metadata allows detached files to remain useful. Bright-field and multichannel fluorescence, Z-stacks, cytology, and sparse and fully tiled encoding are supported. External terminologies and standard compression schemes are supported. Color consistency is defined using International Color Consortium profiles. The DICOM files can be dual personality Tagged Image File Format (TIFF) for legacy support. Annotations for computational pathology results can be encoded.

The Importance of Body Part Labeling to Enable Enterprise Imaging: A HIMSS-SIIM Enterprise Imaging Community Collaborative White Paper.

In order for enterprise imaging to be successful across a multitude of specialties, systems, and sites, standards are essential to categorize and classify imaging data. The HIMSS-SIIM Enterprise Imaging Community believes that the Digital Imaging Communications in Medicine (DICOM) Anatomic Region Sequence, or its equivalent in other data standards, is a vital data element for this role, when populated with standard coded values. We believe that labeling images with standard Anatomic Region Sequence codes will enhance the user's ability to consume data, facilitate interoperability, and allow greater control of privacy. Image consumption-when a user views a patient's images, he or she often wants to see relevant comparison images of the same lesion or anatomic region for the same patient automatically presented. Relevant comparison images may have been acquired from a variety of modalities and specialties. The Anatomic Region Sequence data element provides a basis to allow for efficient comparison in both instances. Interoperability-as patients move between health care systems, it is important to minimize friction for data transfer. Health care providers and facilities need to be able to consume and review the increasingly large and complex volume of data efficiently. The use of Anatomic Region Sequence, or its equivalent, populated with standard values enables seamless interoperability of imaging data regardless of whether images are used within a site or across different sites and systems. Privacy-as more visible light photographs are integrated into electronic systems, it becomes apparent that some images may need to be sequestered. Although additional work is needed to protect sensitive images, standard coded values in Anatomic Region Sequence support the identification of potentially sensitive images, enable facilities to create access control policies, and can be used as an interim surrogate for more sophisticated rule-based or attribute-based access control mechanisms. To satisfy such use cases, the HIMSS-SIIM Enterprise Imaging Community encourages the use of a pre-existing body part ontology. Through this white paper, we will identify potential challenges in employing this standard and provide potential solutions for these challenges.

Multispecialty Enterprise Imaging Workgroup Consensus on Interactive Multimedia Reporting Current State and Road to the Future: HIMSS-SIIM Collaborative White Paper.

Diagnostic and evidential static image, video clip, and sound multimedia are captured during routine clinical care in cardiology, dermatology, ophthalmology, pathology, physiatry, radiation oncology, radiology, endoscopic procedural specialties, and other medical disciplines. Providers typically describe the multimedia findings in contemporaneous electronic health record clinical notes or associate a textual interpretative report. Visual communication aids commonly used to connect, synthesize, and supplement multimedia and descriptive text outside medicine remain technically challenging to integrate into patient care. Such beneficial interactive elements may include hyperlinks between text, multimedia elements, alphanumeric and geometric annotations, tables, graphs, timelines, diagrams, anatomic maps, and hyperlinks to external educational references that patients or provider consumers may find valuable. This HIMSS-SIIM Enterprise Imaging Community workgroup white paper outlines the current and desired clinical future state of interactive multimedia reporting (IMR). The workgroup adopted a consensus definition of IMR as "interactive medical documentation that combines clinical images, videos, sound, imaging metadata, and/or image annotations with text, typographic emphases, tables, graphs, event timelines, anatomic maps, hyperlinks, and/or educational resources to optimize communication between medical professionals, and between medical professionals and their patients." This white paper also serves as a precursor for future efforts toward solving technical issues impeding routine interactive multimedia report creation and ingestion into electronic health records.

Transforming Dermatologic Imaging for the Digital Era: Metadata and Standards.

Imaging is increasingly being used in dermatology for documentation, diagnosis, and management of cutaneous disease. The lack of standards for dermatologic imaging is an impediment to clinical uptake. Standardization can occur in image acquisition, terminology, interoperability, and metadata. This paper presents the International Skin Imaging Collaboration position on standardization of metadata for dermatologic imaging. Metadata is essential to ensure that dermatologic images are properly managed and interpreted. There are two standards-based approaches to recording and storing metadata in dermatologic imaging. The first uses standard consumer image file formats, and the second is the file format and metadata model developed for the Digital Imaging and Communication in Medicine (DICOM) standard. DICOM would appear to provide an advantage over using consumer image file formats for metadata as it includes all the patient, study, and technical metadata necessary to use images clinically. Whereas, consumer image file formats only include technical metadata and need to be used in conjunction with another actor-for example, an electronic medical record-to supply the patient and study metadata. The use of DICOM may have some ancillary benefits in dermatologic imaging including leveraging DICOM network and workflow services, interoperability of images and metadata, leveraging existing enterprise imaging infrastructure, greater patient safety, and better compliance to legislative requirements for image retention.

Technical Challenges of Enterprise Imaging: HIMSS-SIIM Collaborative White Paper.

This white paper explores the technical challenges and solutions for acquiring (capturing) and managing enterprise images, particularly those involving visible light applications. The types of acquisition devices used for various general-purpose photography and specialized applications including dermatology, endoscopy, and anatomic pathology are reviewed. The formats and standards used, and the associated metadata requirements and communication protocols for transfer and workflow are considered. Particular emphasis is placed on the importance of metadata capture in both order- and encounter-based workflow. The benefits of using DICOM to provide a standard means of recording and accessing both metadata and image and video data are considered, as is the role of IHE and FHIR.

A Digital Reference Object to Analyze Calculation Accuracy of PET Standardized Uptake Value.

PURPOSE: To determine the extent of variations in computing standardized uptake value (SUV) by body weight (SUV(BW)) among different software packages and to propose a Digital Imaging and Communications in Medicine (DICOM) reference test object to ensure the standardization of SUV computation between medical image viewing workstations. MATERIALS AND METHODS: Research ethics board approval was not necessary because this study only evaluated images of a phantom. A synthetic set of positron emission tomographic (PET)/computed tomographic (CT) image data, called a digital reference object (DRO), with known SUV was created. The DRO was sent to 16 sites and evaluated on 21 different PET/CT display software packages. Users were asked to draw various regions of interest (ROIs) on specific features and report the maximum, minimum, mean, and standard deviation of the SUVs for each ROI. Numerical tolerances were defined for each metric, and the fraction of reported values within the tolerance was recorded, as was the mean, standard deviation, and range of the metrics. RESULTS: The errors in reported maximum SUV ranged from -37.8% to 0% for an isolated voxel with 4.11:1 target-to-background activity level, and errors in the reported mean SUV ranged from -1.6% to 100% for a region with controlled noise. There was also a range of errors in the less commonly used metrics of minimum SUV and standard deviation SUV. CONCLUSION: The variability of computed SUV(BW) between different software packages is substantial enough to warrant the introduction of a reference standard for medical image viewing workstations.

Consistency and standardization of color in medical imaging: a consensus report.

This article summarizes the consensus reached at the Summit on Color in Medical Imaging held at the Food and Drug Administration (FDA) on May 8-9, 2013, co-sponsored by the FDA and ICC (International Color Consortium). The purpose of the meeting was to gather information on how color is currently handled by medical imaging systems to identify areas where there is a need for improvement, to define objective requirements, and to facilitate consensus development of best practices. Participants were asked to identify areas of concern and unmet needs. This summary documents the topics that were discussed at the meeting and recommendations that were made by the participants. Key areas identified where improvements in color would provide immediate tangible benefits were those of digital microscopy, telemedicine, medical photography (particularly ophthalmic and dental photography), and display calibration. Work in these and other related areas has been started within several professional groups, including the creation of the ICC Medical Imaging Working Group.

Cloud-based large-scale curation of medical imaging data using AI segmentation.

Rapid advances in medical imaging Artificial Intelligence (AI) offer unprecedented opportunities for automatic analysis and extraction of data from large imaging collections. Computational demands of such modern AI tools may be difficult to satisfy with the capabilities available on premises. Cloud computing offers the promise of economical access and extreme scalability. Few studies examine the price/performance tradeoffs of using the cloud, in particular for medical image analysis tasks. We investigate the use of cloud-provisioned compute resources for AI-based curation of the National Lung Screening Trial (NLST) Computed Tomography (CT) images available from the National Cancer Institute (NCI) Imaging Data Commons (IDC). We evaluated NCI Cancer Research Data Commons (CRDC) Cloud Resources - Terra (FireCloud) and Seven Bridges-Cancer Genomics Cloud (SB-CGC) platforms - to perform automatic image segmentation with TotalSegmentator and pyradiomics feature extraction for a large cohort containing >126,000 CT volumes from >26,000 patients. Utilizing >21,000 Virtual Machines (VMs) over the course of the computation we completed analysis in under 9 hours, as compared to the estimated 522 days that would be needed on a single workstation. The total cost of utilizing the cloud for this analysis was $1,011.05. Our contributions include: 1) an evaluation of the numerous tradeoffs towards optimizing the use of cloud resources for large-scale image analysis; 2) CloudSegmentator, an open source reproducible implementation of the developed workflows, which can be reused and extended; 3) practical recommendations for utilizing the cloud for large-scale medical image computing tasks. We also share the results of the analysis: the total of 9,565,554 segmentations of the anatomic structures and the accompanying radiomics features in IDC as of release v18.

Enrichment of lung cancer computed tomography collections with AI-derived annotations.

Public imaging datasets are critical for the development and evaluation of automated tools in cancer imaging. Unfortunately, many do not include annotations or image-derived features, complicating downstream analysis. Artificial intelligence-based annotation tools have been shown to achieve acceptable performance and can be used to automatically annotate large datasets. As part of the effort to enrich public data available within NCI Imaging Data Commons (IDC), here we introduce AI-generated annotations for two collections containing computed tomography images of the chest, NSCLC-Radiomics, and a subset of the National Lung Screening Trial. Using publicly available AI algorithms, we derived volumetric annotations of thoracic organs-at-risk, their corresponding radiomics features, and slice-level annotations of anatomical landmarks and regions. The resulting annotations are publicly available within IDC, where the DICOM format is used to harmonize the data and achieve FAIR (Findable, Accessible, Interoperable, Reusable) data principles. The annotations are accompanied by cloud-enabled notebooks demonstrating their use. This study reinforces the need for large, publicly accessible curated datasets and demonstrates how AI can aid in cancer imaging.

Interoperable slide microscopy viewer and annotation tool for imaging data science and computational pathology.

The exchange of large and complex slide microscopy imaging data in biomedical research and pathology practice is impeded by a lack of data standardization and interoperability, which is detrimental to the reproducibility of scientific findings and clinical integration of technological innovations. We introduce Slim, an open-source, web-based slide microscopy viewer that implements the internationally accepted Digital Imaging and Communications in Medicine (DICOM) standard to achieve interoperability with a multitude of existing medical imaging systems. We showcase the capabilities of Slim as the slide microscopy viewer of the NCI Imaging Data Commons and demonstrate how the viewer enables interactive visualization of traditional brightfield microscopy and highly-multiplexed immunofluorescence microscopy images from The Cancer Genome Atlas and Human Tissue Atlas Network, respectively, using standard DICOMweb services. We further show how Slim enables the collection of standardized image annotations for the development or validation of machine learning models and the visual interpretation of model inference results in the form of segmentation masks, spatial heat maps, or image-derived measurements.

The NCI Imaging Data Commons as a platform for reproducible research in computational pathology.

BACKGROUND AND OBJECTIVES: Reproducibility is a major challenge in developing machine learning (ML)-based solutions in computational pathology (CompPath). The NCI Imaging Data Commons (IDC) provides >120 cancer image collections according to the FAIR principles and is designed to be used with cloud ML services. Here, we explore its potential to facilitate reproducibility in CompPath research. METHODS: Using the IDC, we implemented two experiments in which a representative ML-based method for classifying lung tumor tissue was trained and/or evaluated on different datasets. To assess reproducibility, the experiments were run multiple times with separate but identically configured instances of common ML services. RESULTS: The results of different runs of the same experiment were reproducible to a large extent. However, we observed occasional, small variations in AUC values, indicating a practical limit to reproducibility. CONCLUSIONS: We conclude that the IDC facilitates approaching the reproducibility limit of CompPath research (i) by enabling researchers to reuse exactly the same datasets and (ii) by integrating with cloud ML services so that experiments can be run in identically configured computing environments.

Image data sharing for biomedical research--meeting HIPAA requirements for De-identification.

Data sharing is increasingly recognized as critical to cross-disciplinary research and to assuring scientific validity. Despite National Institutes of Health and National Science Foundation policies encouraging data sharing by grantees, little data sharing of clinical data has in fact occurred. A principal reason often given is the potential of inadvertent violation of the Health Insurance Portability and Accountability Act privacy regulations. While regulations specify the components of private health information that should be protected, there are no commonly accepted methods to de-identify clinical data objects such as images. This leads institutions to take conservative risk-averse positions on data sharing. In imaging trials, where images are coded according to the Digital Imaging and Communications in Medicine (DICOM) standard, the complexity of the data objects and the flexibility of the DICOM standard have made it especially difficult to meet privacy protection objectives. The recent release of DICOM Supplement 142 on image de-identification has removed much of this impediment. This article describes the development of an open-source software suite that implements DICOM Supplement 142 as part of the National Biomedical Imaging Archive (NBIA). It also describes the lessons learned by the authors as NBIA has acquired more than 20 image collections encompassing over 30 million images.

Design and Implementation of the Pre-Clinical DICOM Standard in Multi-Cohort Murine Studies.

The small animal imaging Digital Imaging and Communications in Medicine (DICOM) acquisition context structured report (SR) was developed to incorporate pre-clinical data in an established DICOM format for rapid queries and comparison of clinical and non-clinical datasets. Established terminologies (i.e., anesthesia, mouse model nomenclature, veterinary definitions, NCI Metathesaurus) were utilized to assist in defining terms implemented in pre-clinical imaging and new codes were added to integrate the specific small animal procedures and handling processes, such as housing, biosafety level, and pre-imaging rodent preparation. In addition to the standard DICOM fields, the small animal SR includes fields specific to small animal imaging such as tumor graft (i.e., melanoma), tissue of origin, mouse strain, and exogenous material, including the date and site of injection. Additionally, the mapping and harmonization developed by the Mouse-Human Anatomy Project were implemented to assist co-clinical research by providing cross-reference human-to-mouse anatomies. Furthermore, since small animal imaging performs multi-mouse imaging for high throughput, and queries for co-clinical research requires a one-to-one relation, an imaging splitting routine was developed, new Unique Identifiers (UID's) were created, and the original patient name and ID were saved for reference to the original dataset. We report the implementation of the small animal SR using MRI datasets (as an example) of patient-derived xenograft mouse models and uploaded to The Cancer Imaging Archive (TCIA) for public dissemination, and also implemented this on PET/CT datasets. The small animal SR enhancement provides researchers the ability to query any DICOM modality pre-clinical and clinical datasets using standard vocabularies and enhances co-clinical studies.

Standardization of neurophysiology signal data into the DICOM® standard.

A standard format for neurophysiology data is urgently needed to improve clinical care and promote research data exchange. Previous neurophysiology format standardization projects have provided valuable insights into how to accomplish the project. In medical imaging, the Digital Imaging and Communication in Medicine (DICOM) standard is widely adopted. DICOM offers a unique environment to accomplish neurophysiology format standardization because neurophysiology data can be easily integrated with existing DICOM-supported elements such as video, ECG, and images and also because it provides easy integration into hospital Picture Archiving and Communication Systems (PACS) long-term storage systems. Through the support of the International Federation of Clinical Neurophysiology (IFCN) and partners in industry, DICOM Working Group 32 (WG-32) has created an initial set of standards for routine electroencephalography (EEG), polysomnography (PSG), electromyography (EMG), and electrooculography (EOG). Longer and more complex neurophysiology data types such as high-definition EEG, long-term monitoring EEG, intracranial EEG, magnetoencephalography, advanced EMG, and evoked potentials will be added later. In order to provide for efficient data compression, a DICOM neurophysiology codec design competition will be held by the IFCN and this is currently being planned. We look forward to a future when a common DICOM neurophysiology data format makes data sharing and storage much simpler and more efficient.