Systematic Parameterization, Storage, and Representation of Volumetric DICOM Data.

Tomographic medical imaging systems produce hundreds to thousands of slices, enabling three-dimensional (3D) analysis. Radiologists process these images through various tools and techniques in order to generate 3D renderings for various applications, such as surgical planning, medical education, and volumetric measurements. To save and store these visualizations, current systems use snapshots or video exporting, which prevents further optimizations and requires the storage of significant additional data. The Grayscale Softcopy Presentation State extension of the Digital Imaging and Communications in Medicine (DICOM) standard resolves this issue for two-dimensional (2D) data by introducing an extensive set of parameters, namely 2D Presentation States (2DPR), that describe how an image should be displayed. 2DPR allows storing these parameters instead of storing parameter applied images, which cause unnecessary duplication of the image data. Since there is currently no corresponding extension for 3D data, in this study, a DICOM-compliant object called 3D presentation states (3DPR) is proposed for the parameterization and storage of 3D medical volumes. To accomplish this, the 3D medical visualization process is divided into four tasks, namely pre-processing, segmentation, post-processing, and rendering. The important parameters of each task are determined. Special focus is given to the compression of segmented data, parameterization of the rendering process, and DICOM-compliant implementation of the 3DPR object. The use of 3DPR was tested in a radiology department on three clinical cases, which require multiple segmentations and visualizations during the workflow of radiologists. The results show that 3DPR can effectively simplify the workload of physicians by directly regenerating 3D renderings without repeating intermediate tasks, increase efficiency by preserving all user interactions, and provide efficient storage as well as transfer of visualized data.

Voxel level dense prediction of acute stroke territory in DWI using deep learning segmentation models and image enhancement strategies.

PURPOSE: To build a stroke territory classifier model in DWI by designing the problem as a multiclass segmentation task by defining each stroke territory as distinct segmentation targets and leveraging the guidance of voxel wise dense predictions. MATERIALS AND METHODS: Retrospective analysis of DWI images of 218 consecutive acute anterior or posterior ischemic stroke patients examined between January 2017 to April 2020 in a single center was carried out. Each stroke area was defined as distinct segmentation target with different class labels. U-Net based network was trained followed by majority voting of the voxel wise predictions of the model to transform them into patient level stroke territory classes. Effects of bias field correction and registration to a common space were explored. RESULTS: Of the 218 patients included in this study, 141 (65%) were anterior stroke, and 77 were posterior stroke (35%) whereas 117 (53%) were male and 101 (47%) were female. The model built with original images reached 0.77 accuracy, while the model built with N4 bias corrected images reached 0.80 and the model built with images which were N4 bias corrected and then registered into a common space reached 0.83 accuracy values. CONCLUSION: Voxel wise dense prediction coupled with bias field correction to eliminate artificial signal increase and registration to a common space help models for better performance than using original images. Knowing the properties of target domain while designing deep learning models is important for the overall success of these models.

Thorax computed tomography (CTX) guided ground truth annotation of CHEST radiographs (CXR) for improved classification and detection of COVID-19.

Several data sets have been collected and various artificial intelligence models have been developed for COVID-19 classification and detection from both chest radiography (CXR) and thorax computed tomography (CTX) images. However, the pitfalls and shortcomings of these systems significantly limit their clinical use. In this respect, improving the weaknesses of advanced models can be very effective besides developing new ones. The inability to diagnose ground-glass opacities by conventional CXR has limited the use of this modality in the diagnostic work-up of COVID-19. In our study, we investigated whether we could increase the diagnostic efficiency by collecting a novel CXR data set, which contains pneumonic regions that are not visible to the experts and can only be annotated under CTX guidance. We develop an ensemble methodology of well-established deep CXR models for this new data set and develop a machine learning-based non-maximum suppression strategy to boost the performance for challenging CXR images. CTX and CXR images of 379 patients who applied to our hospital with suspected COVID-19 were evaluated with consensus by seven radiologists. Among these, CXR images of 161 patients who also have had a CTX examination on the same day or until the day before or after and whose CTX findings are compatible with COVID-19 pneumonia, are selected for annotating. CTX images are arranged in the main section passing through the anterior, middle, and posterior according to the sagittal plane with the reformed maximum intensity projection (MIP) method in the coronal plane. Based on the analysis of coronal MIP reconstructed CTX images, the regions corresponding to the pneumonia foci are annotated manually in CXR images. Radiologically classified posterior to anterior (PA) CXR of 218 patients with negative thorax CTX imaging were classified as COVID-19 pneumonia negative group. Accordingly, we have collected a new data set using anonymized CXR (JPEG) and CT (DICOM) images, where the PA CXRs contain pneumonic regions that are hidden or not easily recognized and annotated under CTX guidance. The reference finding was the presence of pneumonic infiltration consistent with COVID-19 on chest CTX examination. COVID-Net, a specially designed convolutional neural network, was used to detect cases of COVID-19 among CXRs. Diagnostic performances were evaluated by ROC analysis by applying six COVID-Net variants (COVIDNet-CXR3-A, -B, -C/COVIDNet-CXR4-A, -B, -C) to the defined data set and combining these models in various ways via ensemble strategies. Finally, a convex optimization strategy is carried out to find the outperforming weighted ensemble of individual models. The mean age of 161 patients with pneumonia was 49.31 ± 15.12, and the median age was 48 years. The mean age of 218 patients without signs of pneumonia in thorax CTX examination was 40.04 ± 14.46, and the median was 38. When working with different combinations of COVID-Net's six variants, the area under the curve (AUC) using the ensemble COVID-Net CXR 4A-4B-3C was .78, sensitivity 67%, specificity 95%; COVID-Net CXR 4a-3b-3c was .79, sensitivity 69% and specificity 94%. When diverse and complementary COVID-Net models are used together through an ensemble, it has been determined that the AUC values are close to other studies, and the specificity is significantly higher than other studies in the literature.

Abdominal multi-organ segmentation with cascaded convolutional and adversarial deep networks.

Abdominal anatomy segmentation is crucial for numerous applications from computer-assisted diagnosis to image-guided surgery. In this context, we address fully-automated multi-organ segmentation from abdominal CT and MR images using deep learning. The proposed model extends standard conditional generative adversarial networks. Additionally to the discriminator which enforces the model to create realistic organ delineations, it embeds cascaded partially pre-trained convolutional encoder-decoders as generator. Encoder fine-tuning from a large amount of non-medical images alleviates data scarcity limitations. The network is trained end-to-end to benefit from simultaneous multi-level segmentation refinements using auto-context. Employed for healthy liver, kidneys and spleen segmentation, our pipeline provides promising results by outperforming state-of-the-art encoder-decoder schemes. Followed for the Combined Healthy Abdominal Organ Segmentation (CHAOS) challenge organized in conjunction with the IEEE International Symposium on Biomedical Imaging 2019, it gave us the first rank for three competition categories: liver CT, liver MR and multi-organ MR segmentation. Combining cascaded convolutional and adversarial networks strengthens the ability of deep learning pipelines to automatically delineate multiple abdominal organs, with good generalization capability. The comprehensive evaluation provided suggests that better guidance could be achieved to help clinicians in abdominal image interpretation and clinical decision making.

CHAOS Challenge - combined (CT-MR) healthy abdominal organ segmentation.

Segmentation of abdominal organs has been a comprehensive, yet unresolved, research field for many years. In the last decade, intensive developments in deep learning (DL) introduced new state-of-the-art segmentation systems. Despite outperforming the overall accuracy of existing systems, the effects of DL model properties and parameters on the performance are hard to interpret. This makes comparative analysis a necessary tool towards interpretable studies and systems. Moreover, the performance of DL for emerging learning approaches such as cross-modality and multi-modal semantic segmentation tasks has been rarely discussed. In order to expand the knowledge on these topics, the CHAOS - Combined (CT-MR) Healthy Abdominal Organ Segmentation challenge was organized in conjunction with the IEEE International Symposium on Biomedical Imaging (ISBI), 2019, in Venice, Italy. Abdominal organ segmentation from routine acquisitions plays an important role in several clinical applications, such as pre-surgical planning or morphological and volumetric follow-ups for various diseases. These applications require a certain level of performance on a diverse set of metrics such as maximum symmetric surface distance (MSSD) to determine surgical error-margin or overlap errors for tracking size and shape differences. Previous abdomen related challenges are mainly focused on tumor/lesion detection and/or classification with a single modality. Conversely, CHAOS provides both abdominal CT and MR data from healthy subjects for single and multiple abdominal organ segmentation. Five different but complementary tasks were designed to analyze the capabilities of participating approaches from multiple perspectives. The results were investigated thoroughly, compared with manual annotations and interactive methods. The analysis shows that the performance of DL models for single modality (CT / MR) can show reliable volumetric analysis performance (DICE: 0.98 ± 0.00 / 0.95 ± 0.01), but the best MSSD performance remains limited (21.89 ± 13.94 / 20.85 ± 10.63 mm). The performances of participating models decrease dramatically for cross-modality tasks both for the liver (DICE: 0.88 ± 0.15 MSSD: 36.33 ± 21.97 mm). Despite contrary examples on different applications, multi-tasking DL models designed to segment all organs are observed to perform worse compared to organ-specific ones (performance drop around 5%). Nevertheless, some of the successful models show better performance with their multi-organ versions. We conclude that the exploration of those pros and cons in both single vs multi-organ and cross-modality segmentations is poised to have an impact on further research for developing effective algorithms that would support real-world clinical applications. Finally, having more than 1500 participants and receiving more than 550 submissions, another important contribution of this study is the analysis on shortcomings of challenge organizations such as the effects of multiple submissions and peeking phenomenon.

Reversible 3D compression of segmented medical volumes: usability analysis for teleradiology and storage.

BACKGROUND: DICOM standard does not have modules that provide the possibilities of two-dimensional Presentation States to three-dimensional (3D). Once the final 3D rendering is obtained, only video/image exporting or snapshots can be used. To increase the utility of 3D Presentation States in clinical practice and teleradiology, the storing and transferring the segmentation results, obtained after tedious procedures, can be very effective. PURPOSE: To propose a strategy for preserving interaction and mobility of visualizations for teleradiology by storing and transferring only binary segmented data, which is effectively compressed by modern adaptive and context-based reversible methods. MATERIAL AND METHODS: A diverse set of segmented data, which include four abdominal organs (liver, spleen, right, and left kidneys) from 20 T1-DUAL and 20 T2-SPIR MRI, liver from 20 CT, and abdominal aorta with aneurysms (AAA) from 19 computed tomography-angiography datasets, are collected. Each organ is segmented manually by expert physicians, and binary volumes are created. The well-established reversible binary compression methods PNG, JPEG-LS, JPEG-XR, CCITT-G4, LZW, JBIG2, and ZIP are applied to medical datasets. Recently proposed context-based (3D-RLE) and adaptive (ABIC) algorithms are also employed. The performance assessment has been presented in terms of the compression ratio that is a universal compression metric. RESULTS: Reversible compression of binary volumes results with substantial decreases in file size such as 254 to 2.14 MB for CT-AAA, 56.7 to 0.3 MB for CT-liver. Moreover, compared to the performance of well-established methods (i.e., mean 76.14%), CR is observed to be increased significantly for all segmented organs from both CT and MRI datasets when ABIC (95.49%) and 3D-RLE (94.98%) are utilized. The hypothesis is that morphological coherence of scanning procedure and adaptation between the segmented organs, that is, bi-level images, contributes to compression performance. Although the performance of well-established techniques is satisfactory, the sensitivity of ABIC to modality type and the advantage of 3D-RLE when the spatial coherence between the adjacent slices are high results with up to 10 times more CR performance. CONCLUSION: Adaptive and context-based compression strategies allow effective storage and transfer of segmented binary data, which can be used to re-produce visualizations for better teleradiology practices preserving all interaction mechanisms.

Comparison of semi-automatic and deep learning-based automatic methods for liver segmentation in living liver transplant donors.

PURPOSE: To compare the accuracy and repeatability of emerging machine learning based (i.e. deep) automatic segmentation algorithms with those of well-established semi-automatic (interactive) methods for determining liver volume in living liver transplant donors at computerized tomography (CT) imaging. METHODS: A total of 12 (6 semi-, 6 full-automatic) methods are evaluated. The semi-automatic segmentation algorithms are based on both traditional iterative models including watershed, fast marching, region growing, active contours and modern techniques including robust statistical segmenter and super-pixels. These methods entail some sort of interaction mechanism such as placing initialization seeds on images or determining a parameter range. The automatic methods are based on deep learning and they include three framework templates (DeepMedic, NiftyNet and U-Net) the first two of which are applied with default parameter sets and the last two involve adapted novel model designs. For 20 living donors (6 training and 12 test datasets), a group of imaging scientists and radiologists created ground truths by performing manual segmentations on contrast material-enhanced CT images. Each segmentation is evaluated using five metrics (i.e. volume overlap and relative volume errors, average/RMS/maximum symmetrical surface distances). The results are mapped to a scoring system and a final grade is calculated by taking their average. Accuracy and repeatability were evaluated using slice by slice comparisons and volumetric analysis. Diversity and complementarity are observed through heatmaps. Majority voting and Simultaneous Truth and Performance Level Estimation (STAPLE) algorithms are utilized to obtain the fusion of the individual results. RESULTS: The top four methods are determined to be automatic deep models having 79.63, 79.46 and 77.15 and 74.50 scores. Intra-user score is determined as 95.14. Overall, deep automatic segmentation outperformed interactive techniques on all metrics. The mean volume of liver of ground truth is found to be 1409.93 mL ± 271.28 mL, while it is calculated as 1342.21 mL ± 231.24 mL using automatic and 1201.26 mL ± 258.13 mL using interactive methods, showing higher accuracy and less variation on behalf of automatic methods. The qualitative analysis of segmentation results showed significant diversity and complementarity enabling the idea of using ensembles to obtain superior results. The fusion of automatic methods reached 83.87 with majority voting and 86.20 using STAPLE that are only slightly less than fusion of all methods that achieved 86.70 (majority voting) and 88.74 (STAPLE). CONCLUSION: Use of the new deep learning based automatic segmentation algorithms substantially increases the accuracy and repeatability for segmentation and volumetric measurements of liver. Fusion of automatic methods based on ensemble approaches exhibits best results almost without any additional time cost due to potential parallel execution of multiple models.

Semiautomatic transfer function initialization for abdominal visualization using self-generating hierarchical radial basis function networks.

As being a tool that assigns optical parameters used in interactive visualization, Transfer Functions (TF) have important effects on the quality of volume rendered medical images. Unfortunately, finding accurate TFs is a tedious and time consuming task because of the trade off between using extensive search spaces and fulfilling the physician's expectations with interactive data exploration tools and interfaces. By addressing this problem, we introduce a semi-automatic method for initial generation of TFs. The proposed method uses a Self Generating Hierarchical Radial Basis Function Network to determine the lobes of a Volume Histogram Stack (VHS) which is introduced as a new domain by aligning the histograms of slices of a image series. The new self generating hierarchical design strategy allows the recognition of suppressed lobes corresponding to suppressed tissues and representation of the overlapping regions which are parts of the lobes but can not be represented by the Gaussian bases in VHS. Moreover, approximation with a minimum set of basis functions provides the possibility of selecting and adjusting suitable units to optimize the TF. Applications on different CT and MR data sets show enhanced rendering quality and reduced optimization time in abdominal studies.

Patient oriented and robust automatic liver segmentation for pre-evaluation of liver transplantation.

Identifying liver region from abdominal computed tomography-angiography (CTA) data sets is one of the essential steps in evaluation of transplantation donors prior to the hepatic surgery. However, due to gray level similarity of adjacent organs, injection of contrast media and partial volume effects; robust segmentation of the liver is a very difficult task. Moreover, high variations in liver margins, different image characteristics with different CT scanners and atypical liver shapes make the segmentation process even harder. In this paper, we propose a three stage (i.e. pre-processing, classification, post-processing); automatic liver segmentation algorithm that adapts its parameters according to each patient by learning the data set characteristics in parallel to segmentation process to address all the challenging aspects mentioned above. The efficiency in terms of the time requirement and the overall segmentation performance is achieved by introducing a novel modular classification system consisting of a K-Means based simple classification system and an MLP based complex one which are combined with a data-dependent and automated switching mechanism that decides to apply one of them. Proposed approach also makes the design of the overall classification system fully unsupervised that depends on the given CTA series only without requiring any given training set of CTA series. The segmentation results are evaluated by using area error rate and volume calculations and the success rate is calculated as 94.91% over a data set of diverse CTA series of 20 patients according to the evaluation of the expert radiologist. The results show that, the proposed algorithm gives better results especially for atypical liver shapes and low contrast studies where several algorithms fail.

Implementation and use of 3D pairwise geodesic distance fields for seeding abdominal aortic vessels.

PURPOSE: Precise extraction of aorta and the vessels departing from it (i.e. coeliac, renal, and iliac) is vital for correct positioning of a graft prior to abdominal aortic surgery. To perform this task, most of the segmentation algorithms rely on seed points, and better-located seed points provide better initial positions for cross-sectional methods. Under non-optimal acquisition characteristics of daily clinical routine and complex morphology of these vessels, inserting seed points to all these small, but critically important vessels is a tedious, time-consuming, and error-prone task. Thus, in this paper, a novel strategy is developed to generate pathways between user-inserted seed points in order to initialize segmentation methods effectively. METHOD: The proposed method requires only a single user-inserted seed for each vessel of interest for initializations. Starting from these initial seeds, it automatically generates pathways that span all vessels in between. To accomplish this, first, a geodesic mask is generated by adaptive thresholding, which reinforces the initial seeds to be kept in the vascular tree. Then, a novel implementation of 3D pairwise geodesic distance field (3D-PGDF) is utilized. It is shown that the minimal-valued geodesic of 3D-PGDF successfully defines a path linking the initial seeds as being the shortest geodesic. Moreover, the robustness of the minimum level set of the 3D-PGDF to local variations and regions of high curvature is increased by a region classification strategy, which adds partial geodesics to these critical regions. RESULTS: The proposed method was applied to 19 challenging CT data sets obtained from four different scanners and compared to two benchmark methods. The first method is a high-precision technique with very long processing time (subvoxel precise multi-stencil fast marching-MSFM), while the second is a very fast method with lower accuracy (3D fast marching). The results, which are obtained using various measures, show that the pathways generated by the developed technique enable significantly higher segmentation performance than 3D fast marching and require much less computational power and time than MSFM. CONCLUSION: The developed technique offers a useful tool for generating pathways between seed points with minimal user interaction. It guarantees to include all important vessels in a computationally effective manner and thus, it can be used to initialize segmentation methods for abdominal aortic tree.

Segmentation of abdominal organs from CT using a multi-level, hierarchical neural network strategy.

Precise measurements on abdominal organs are vital prior to the important clinical procedures. Such measurements require accurate segmentation of these organs, which is a very challenging task due to countless anatomical variations and technical difficulties. Although, several features with various classifiers have been designed to overcome these challenges, abdominal organ segmentation via classification is still an emerging field in order to reach desired precision. Recent studies on multiple feature-classifier combinations show that hierarchical systems outperform composite feature-single classifier models. In this study, how hierarchical formations can translate to improved accuracy, when large size feature spaces are involved, is explored for the problem of abdominal organ segmentation. As a result, a semi-automatic, slice-by-slice segmentation method is developed using a novel multi-level and hierarchical neural network (MHNN). MHNN is designed to collect complementary information about organs at each level of the hierarchy via different feature-classifier combinations. Moreover, each level of MHNN receives residual data from the previous level. The residual data is constructed to preserve zero false positive error until the last level of the hierarchy, where only most challenging samples remain. The algorithm mimics analysis behaviour of a radiologist by using the slice-by-slice iteration, which is supported with adjacent slice similarity features. This enables adaptive determination of system parameters and turns into the advantage of online training, which is done in parallel to the segmentation process. Proposed design can perform robust and accurate segmentation of abdominal organs as validated by using diverse data sets with various challenges.

Semi-automatic segmentation methods for 3-D visualization and analysis of the liver.

Pre-evaluation of donors prior to surgery of living donated liver transplantation is one of the challenging applications that computer aided systems are needed. The precise measurement of liver volume requires effective segmentation procedures, while three dimensional rendering of the segmented data provides demonstrative information to radiologists and surgeons before surgery. The Insight Toolkit provides effective algorithms for segmentation, which are also optimized for high computational performance and processing time. Furthermore, An ITK pipeline can be combined with a VTK pipeline, so that the result of segmentation can be represented directly in 3-D using VTK. Therefore, there is an on-going trend for developing ITK/VTK based systems. This study presents quantitative and qualitative performance evaluation of two effective ITK algorithms on segmentation of liver from CTA data sets.

Performance comparison of compression algorithms for archiving segmented volumetric binary medical data.

Archiving result of a segmentation task allows the representation of the segmented volume at a later time. The segmented volume can be stored in a binary format, which can be restored by a simple combination of the original data with this binary information. Since, the sizes of the segmented binary data have high memory requirements; a lossless compression method should be employed for efficient archiving. Thus, this study examines different approaches for compression and their suitability for restoring binary segmentation results. To evaluate the compressive properties, multiple test cases with diverse spatial structures and acquired with different modalities from clinical practice have been used. The results show that best performance is achieved with JBIG2 method both in terms of compression ratio and processing time.

Integrating segmentation methods from different tools into a visualization program using an object-based plug-in interface.

In medical visualization, segmentation is an important step prior to rendering. However, it is also a difficult procedure because of the restrictions imposed by variations in image characteristics, human anatomy, and pathology. Moreover, what is interesting from clinical point of view is usually not only an organ or a tissue itself, but also its properties together with adjacent organs or related vessel systems that are going in and coming out. For an informative rendering, these necessitate the usage of different segmentation methods in a single application, and combining/representing the results together in a proper way. This paper describes the implementation of an interface, which can be used to plug-in and then apply a segmentation method to a medical image series. The design is based on handling each segmentation procedure as an object where all parameters of each object can be specified individually. Thus, it is possible to use different plug-ins with different interfaces and parameters for the segmentation of different tissues in the same dataset while rendering all of the results together is still possible. The design allows access to insight registration and segmentation toolkit, Java, and MATLAB functionality together, eases sharing and comparing segmentation techniques, and serves as a visual debugger for algorithm developers.