Recognizability bias in citizen science photographs.

Citizen science and automated collection methods increasingly depend on image recognition to provide the amounts of observational data research and management needs. Recognition models, meanwhile, also require large amounts of data from these sources, creating a feedback loop between the methods and tools. Species that are harder to recognize, both for humans and machine learning algorithms, are likely to be under-reported, and thus be less prevalent in the training data. As a result, the feedback loop may hamper training mostly for species that already pose the greatest challenge. In this study, we trained recognition models for various taxa, and found evidence for a 'recognizability bias', where species that are more readily identified by humans and recognition models alike are more prevalent in the available image data. This pattern is present across multiple taxa, and does not appear to relate to differences in picture quality, biological traits or data collection metrics other than recognizability. This has implications for the expected performance of future models trained with more data, including such challenging species.

Maximizing citizen scientists' contribution to automated species recognition.

Technological advances and data availability have enabled artificial intelligence-driven tools that can increasingly successfully assist in identifying species from images. Especially within citizen science, an emerging source of information filling the knowledge gaps needed to solve the biodiversity crisis, such tools can allow participants to recognize and report more poorly known species. This can be an important tool in addressing the substantial taxonomic bias in biodiversity data, where broadly recognized, charismatic species are highly over-represented. Meanwhile, the recognition models are trained using the same biased data, so it is important to consider what additional images are needed to improve recognition models. In this study, we investigated how the amount of training data influenced the performance of species recognition models for various taxa. We utilized a large citizen science dataset collected in Norway, where images are added independently from identification. We demonstrate that while adding images of currently under-represented taxa will generally improve recognition models more, there are important deviations from this general pattern. Thus, a more focused prioritization of data collection beyond the basic paradigm that "more is better" is likely to significantly improve species recognition models and advance the representativeness of biodiversity data.

Adversarial attack vulnerability of medical image analysis systems: Unexplored factors.

Adversarial attacks are considered a potentially serious security threat for machine learning systems. Medical image analysis (MedIA) systems have recently been argued to be vulnerable to adversarial attacks due to strong financial incentives and the associated technological infrastructure. In this paper, we study previously unexplored factors affecting adversarial attack vulnerability of deep learning MedIA systems in three medical domains: ophthalmology, radiology, and pathology. We focus on adversarial black-box settings, in which the attacker does not have full access to the target model and usually uses another model, commonly referred to as surrogate model, to craft adversarial examples that are then transferred to the target model. We consider this to be the most realistic scenario for MedIA systems. Firstly, we study the effect of weight initialization (pre-training on ImageNet or random initialization) on the transferability of adversarial attacks from the surrogate model to the target model, i.e., how effective attacks crafted using the surrogate model are on the target model. Secondly, we study the influence of differences in development (training and validation) data between target and surrogate models. We further study the interaction of weight initialization and data differences with differences in model architecture. All experiments were done with a perturbation degree tuned to ensure maximal transferability at minimal visual perceptibility of the attacks. Our experiments show that pre-training may dramatically increase the transferability of adversarial examples, even when the target and surrogate's architectures are different: the larger the performance gain using pre-training, the larger the transferability. Differences in the development data between target and surrogate models considerably decrease the performance of the attack; this decrease is further amplified by difference in the model architecture. We believe these factors should be considered when developing security-critical MedIA systems planned to be deployed in clinical practice. We recommend avoiding using only standard components, such as pre-trained architectures and publicly available datasets, as well as disclosure of design specifications, in addition to using adversarial defense methods. When evaluating the vulnerability of MedIA systems to adversarial attacks, various attack scenarios and target-surrogate differences should be simulated to achieve realistic robustness estimates. The code and all trained models used in our experiments are publicly available.3.

Fast and effective quantification of symmetry in medical images for pathology detection: Application to chest radiography.

PURPOSE: Symmetry is an important feature of human anatomy and the absence of symmetry in medical images can indicate the presence of pathology. Quantification of image symmetry can then be used to improve the automatic analysis of medical images. METHODS: A method is presented that computes both local and global symmetry in 2D medical images. A symmetry axis is determined to define for each position p in the image a mirrored position p' on the contralateral side of the axis. In the neighborhood of p', an optimally corresponding position ps is determined by minimizing a cost function d that combines intensity differences in a patch around p and the mirrored patch around ps and the spatial distance between p' and ps. The optimal value of d is used as a measure of local symmetry s. The average of all values of s, indicated as S, quantifies global symmetry. Starting from an initial approximation of the symmetry axis, the optimal orientation and position of the axis is determined by greedy minimization of S. RESULTS: The method was evaluated in three experiments concerning abnormality detection in frontal chest radiographs. In the first experiment, global symmetry S was used to discriminate between 174 normal images and 174 images containing diffuse textural abnormalities from the publicly available CRASS database of tuberculosis suspects. Performance, measured as area under the receiver operating characteristic curve Az was 0.838. The second experiment investigated whether adding the local symmetry s as an additional feature to a set of 106 texture features resulted in improvements in classifying local patches in the same image database. We found that Az increased from 0.878 to 0.891 (P = 0.001). In the third experiment, it was shown that the contrast of pulmonary nodules, obtained from the publicly available JSRT database, increased significantly in the local symmetry map compared to the original image. CONCLUSIONS: We conclude that the proposed algorithm for symmetry computation provides informative features which can be used to improve abnormality detection in medical images both at a local and a global level.

Automatic Detection of Tuberculosis in Chest Radiographs Using a Combination of Textural, Focal, and Shape Abnormality Analysis.

Tuberculosis (TB) is a common disease with high mortality and morbidity rates worldwide. Automatic systems to detect TB on chest radiographs (CXRs) can improve the efficiency of diagnostic algorithms for pulmonary TB. The diverse manifestation of TB on CXRs from different populations requires a system that can be adapted to deal with different types of abnormalities. A computer aided detection (CAD) system was developed which combines several subscores of supervised subsystems detecting textural, shape, and focal abnormalities into one TB score. A general framework was developed to combine an arbitrary number of subscores: subscores were normalized, collected in a feature vector and then combined using a supervised classifier into one combined score. The method was evaluated on two databases, both consisting of 200 digital CXRs, from: (A) Western high-risk group screening, (B) TB suspect screening in Africa. The subscores and combined score were compared to (1) an external, non-radiological, reference and (2) a radiological reference determined by a human expert. Performance was measured using Receiver Operator Characteristic (ROC) analysis. Different subscores performed best in the two databases. The combined TB score performed better than the individual subscores, except for the external reference in database B. The performances of the independent observer were slightly higher than the combined TB score. Compared to the external reference, differences in performance between the combined TB score and the independent observer were not significant in both databases. Supervised combination to compute an overall TB score allows for a necessary adaptation of the CAD system to different settings or different operational requirements.

The sensitivity and specificity of using a computer aided diagnosis program for automatically scoring chest X-rays of presumptive TB patients compared with Xpert MTB/RIF in Lusaka Zambia.

OBJECTIVE: To determine the sensitivity and specificity of a Computer Aided Diagnosis (CAD) program for scoring chest x-rays (CXRs) of presumptive tuberculosis (TB) patients compared to Xpert MTB/RIF (Xpert). METHOD: Consecutive presumptive TB patients with a cough of any duration were offered digital CXR, and opt out HIV testing. CXRs were electronically scored as normal (CAD score ≤ 60) or abnormal (CAD score > 60) using a CAD program. All patients regardless of CAD score were requested to submit a spot sputum sample for testing with Xpert and a spot and morning sample for testing with LED Fluorescence Microscopy-(FM). RESULTS: Of 350 patients with evaluable data, 291 (83.1%) had an abnormal CXR score by CAD. The sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) of CXR compared to Xpert were 100% (95%CI 96.2-100), 23.2% (95%CI 18.2-28.9), 33.0% (95%CI 27.6-38.7) and 100% (95% 93.9-100), respectively. The area under the receiver operator curve (AUC) for CAD was 0.71 (95%CI 0.66-0.77). CXR abnormality correlated with smear grade (r = 0.30, p<0.0001) and with Xpert CT(r = 0.37, p<0.0001). CONCLUSIONS: To our knowledge this is the first time that a CAD program for TB has been successfully tested in a real world setting. The study shows that the CAD program had high sensitivity but low specificity and PPV. The use of CAD with digital CXR has the potential to increase the use and availability of chest radiography in screening for TB where trained human resources are scarce.

Cavity contour segmentation in chest radiographs using supervised learning and dynamic programming.

PURPOSE: Efficacy of tuberculosis (TB) treatment is often monitored using chest radiography. Monitoring size of cavities in pulmonary tuberculosis is important as the size predicts severity of the disease and its persistence under therapy predicts relapse. The authors present a method for automatic cavity segmentation in chest radiographs. METHODS: A two stage method is proposed to segment the cavity borders, given a user defined seed point close to the center of the cavity. First, a supervised learning approach is employed to train a pixel classifier using texture and radial features to identify the border pixels of the cavity. A likelihood value of belonging to the cavity border is assigned to each pixel by the classifier. The authors experimented with four different classifiers:k-nearest neighbor (kNN), linear discriminant analysis (LDA), GentleBoost (GB), and random forest (RF). Next, the constructed likelihood map was used as an input cost image in the polar transformed image space for dynamic programming to trace the optimal maximum cost path. This constructed path corresponds to the segmented cavity contour in image space. RESULTS: The method was evaluated on 100 chest radiographs (CXRs) containing 126 cavities. The reference segmentation was manually delineated by an experienced chest radiologist. An independent observer (a chest radiologist) also delineated all cavities to estimate interobserver variability. Jaccard overlap measure Ω was computed between the reference segmentation and the automatic segmentation; and between the reference segmentation and the independent observer's segmentation for all cavities. A median overlap Ω of 0.81 (0.76 ± 0.16), and 0.85 (0.82 ± 0.11) was achieved between the reference segmentation and the automatic segmentation, and between the segmentations by the two radiologists, respectively. The best reported mean contour distance and Hausdorff distance between the reference and the automatic segmentation were, respectively, 2.48 ± 2.19 and 8.32 ± 5.66 mm, whereas these distances were 1.66 ± 1.29 and 5.75 ± 4.88 mm between the segmentations by the reference reader and the independent observer, respectively. The automatic segmentations were also visually assessed by two trained CXR readers as "excellent," "adequate," or "insufficient." The readers had good agreement in assessing the cavity outlines and 84% of the segmentations were rated as "excellent" or "adequate" by both readers. CONCLUSIONS: The proposed cavity segmentation technique produced results with a good degree of overlap with manual expert segmentations. The evaluation measures demonstrated that the results approached the results of the experienced chest radiologists, in terms of overlap measure and contour distance measures. Automatic cavity segmentation can be employed in TB clinics for treatment monitoring, especially in resource limited settings where radiologists are not available.

Suppression of translucent elongated structures: applications in chest radiography.

Projection images, such as those routinely acquired in radiological practice, are difficult to analyze because multiple 3-D structures superimpose at a single point in the 2-D image. Removal of particular superimposed structures may improve interpretation of these images, both by humans and by computers. This work therefore presents a general method to isolate and suppress structures in 2-D projection images. The focus is on elongated structures, which allows an intensity model of a structure of interest to be extracted using local information only. The model is created from profiles sampled perpendicular to the structure. Profiles containing other structures are detected and removed to reduce the influence on the model. Subspace filtering, using blind source separation techniques, is applied to separate the structure to be suppressed from other structures. By subtracting the modeled structure from the original image a structure suppressed image is created. The method is evaluated in four experiments. In the first experiment ribs are suppressed in 20 artificial radiographs simulated from 3-D lung computed tomography (CT) images. The proposed method with blind source separation and outlier detection shows superior suppression of ribs in simulated radiographs, compared to a simplified approach without these techniques. Additionally, the ability of three observers to discriminate between patches containing ribs and containing no ribs, as measured by the area under the receiver operating characteristic curve (AUC), reduced from 0.99-1.00 on original images to 0.75-0.84 on suppressed images. In the second experiment clavicles are suppressed in 253 chest radiographs. The effect of suppression on clavicle visibility is evaluated using the clavicle contrast and border response, showing a reduction of 78% and 34%, respectively. In the third experiment nodules extracted from CT were simulated close to the clavicles in 100 chest radiographs. It was found that after suppression contrast of the nodules was higher than of the clavicles (1.35 and 0.55, respectively) than on original images (1.83 and 2.46, respectively). In the fourth experiment catheters were suppressed in chest radiographs. The ability of three observers to discriminate between patches originating from 36 images with and 21 images without catheters, as measured by the AUC, reduced from 0.98-0.99 on original images to 0.64-0.74 on suppressed images. We conclude that the presented method can markedly reduce the visibility of elongated structures in chest radiographs and shows potential to enhance diagnosis.

Foreign object detection and removal to improve automated analysis of chest radiographs.

PURPOSE: Chest radiographs commonly contain projections of foreign objects, such as buttons, brassier clips, jewellery, or pacemakers and wires. The presence of these structures can substantially affect the output of computer analysis of these images. An automated method is presented to detect, segment, and remove foreign objects from chest radiographs. METHODS: Detection is performed using supervised pixel classification with a kNN classifier, resulting in a probability estimate per pixel to belong to a projected foreign object. Segmentation is performed by grouping and post-processing pixels with a probability above a certain threshold. Next, the objects are replaced by texture inpainting. RESULTS: The method is evaluated in experiments on 257 chest radiographs. The detection at pixel level is evaluated with receiver operating characteristic analysis on pixels within the unobscured lung fields and an Az value of 0.949 is achieved. Free response operator characteristic analysis is performed at the object level, and 95.6% of objects are detected with on average 0.25 false positive detections per image. To investigate the effect of removing the detected objects through inpainting, a texture analysis system for tuberculosis detection is applied to images with and without pathology and with and without foreign object removal. Unprocessed, the texture analysis abnormality score of normal images with foreign objects is comparable to those with pathology. After removing foreign objects, the texture score of normal images with and without foreign objects is similar, while abnormal images, whether they contain foreign objects or not, achieve on average higher scores. CONCLUSIONS: The authors conclude that removal of foreign objects from chest radiographs is feasible and beneficial for automated image analysis.

Clavicle segmentation in chest radiographs.

Automated delineation of anatomical structures in chest radiographs is difficult due to superimposition of multiple structures. In this work an automated technique to segment the clavicles in posterior-anterior chest radiographs is presented in which three methods are combined. Pixel classification is applied in two stages and separately for the interior, the border and the head of the clavicle. This is used as input for active shape model segmentation. Finally dynamic programming is employed with an optimized cost function that combines appearance information of the interior of the clavicle, the border, the head and shape information derived from the active shape model. The method is compared with a number of previously described methods and with independent human observers on a large database. This database contains both normal and abnormal images and will be made publicly available. The mean contour distance of the proposed method on 249 test images is 1.1±1.6mm and the intersection over union is 0.86±0.10.

Fusion of local and global detection systems to detect tuberculosis in chest radiographs.

Automatic detection of tuberculosis (TB) on chest radiographs is a difficult problem because of the diverse presentation of the disease. A combination of detection systems for abnormalities and normal anatomy is used to improve detection performance. A textural abnormality detection system operating at the pixel level is combined with a clavicle detection system to suppress false positive responses. The output of a shape abnormality detection system operating at the image level is combined in a next step to further improve performance by reducing false negatives. Strategies for combining systems based on serial and parallel configurations were evaluated using the minimum, maximum, product, and mean probability combination rules. The performance of TB detection increased, as measured using the area under the ROC curve, from 0.67 for the textural abnormality detection system alone to 0.86 when the three systems were combined. The best result was achieved using the sum and product rule in a parallel combination of outputs.

Quantifying local radiation-induced lung damage from computed tomography.

PURPOSE: Optimal implementation of new radiotherapy techniques requires accurate predictive models for normal tissue complications. Since clinically used dose distributions are nonuniform, local tissue damage needs to be measured and related to local tissue dose. In lung, radiation-induced damage results in density changes that have been measured by computed tomography (CT) imaging noninvasively, but not yet on a localized scale. Therefore, the aim of the present study was to develop a method for quantification of local radiation-induced lung tissue damage using CT. METHODS AND MATERIALS: CT images of the thorax were made 8 and 26 weeks after irradiation of 100%, 75%, 50%, and 25% lung volume of rats. Local lung tissue structure (S(L)) was quantified from local mean and local standard deviation of the CT density in Hounsfield units in 1-mm(3) subvolumes. The relation of changes in S(L) (DeltaS(L)) to histologic changes and breathing rate was investigated. Feasibility for clinical application was tested by applying the method to CT images of a patient with non-small-cell lung carcinoma and investigating the local dose-effect relationship of DeltaS(L). RESULTS: In rats, a clear dose-response relationship of DeltaS(L) was observed at different time points after radiation. Furthermore, DeltaS(L) correlated strongly to histologic endpoints (infiltrates and inflammatory cells) and breathing rate. In the patient, progressive local dose-dependent increases in DeltaS(L) were observed. CONCLUSION: We developed a method to quantify local radiation-induced tissue damage in the lung using CT. This method can be used in the development of more accurate predictive models for normal tissue complications.

Computer-aided diagnosis in chest radiography: beyond nodules.

Chest radiographs are the most common exam in radiology. They are essential for the management of various diseases associated with high mortality and morbidity and display a wide range of findings, many of them subtle. In this survey we identify a number of areas beyond pulmonary nodules that could benefit from computer-aided detection and diagnosis (CAD) in chest radiography. These include interstitial infiltrates, catheter tip detection, size measurements, detection of pneumothorax and detection and quantification of emphysema. Recent work in these areas is surveyed, but we conclude that the amount of research devoted to these topics is modest. Reasons for the slow pace of CAD development in chest radiography beyond nodules are discussed.

Global and local multi-valued dissimilarity-based classification: application to computer-aided detection of tuberculosis.

In many applications of computer-aided detection (CAD) it is not possible to precisely localize lesions or affected areas in images that are known to be abnormal. In this paper a novel approach to computer-aided detection is presented that can deal effectively with such weakly labeled data. Our approach is based on multi-valued dissimilarity measures that retain more information about underlying local image features than single-valued dissimilarities. We show how this approach can be extended by applying it locally as well as globally, and by merging the local and global classification results into an overall opinion about the image to be classified. The framework is applied to the detection of tuberculosis (TB) in chest radiographs. This is the first study to apply a CAD system to a large database of digital chest radiographs obtained from a TB screening program, including normal cases, suspect cases and cases with proven TB. The global dissimilarity approach achieved an area under the ROC curve of 0.81. The combination of local and global classifications increased this value to 0.83.