Enhanced Out-of-Stock Detection in Retail Shelf Images Based on Deep Learning.

The term out-of-stock (OOS) describes a problem that occurs when shoppers come to a store and the product they are seeking is not present on its designated shelf. Missing products generate huge sales losses and may lead to a declining reputation or the loss of loyal customers. In this paper, we propose a novel deep-learning (DL)-based OOS-detection method that utilizes a two-stage training process and a post-processing technique designed for the removal of inaccurate detections. To develop the method, we utilized an OOS detection dataset that contains a commonly used fully empty OOS class and a novel class that represents the frontal OOS. We present a new image augmentation procedure in which some existing OOS instances are enlarged by duplicating and mirroring themselves over nearby products. An object-detection model is first pre-trained using only augmented shelf images and, then, fine-tuned on the original data. During the inference, the detected OOS instances are post-processed based on their aspect ratio. In particular, the detected instances are discarded if their aspect ratio is higher than the maximum or lower than the minimum instance aspect ratio found in the dataset. The experimental results showed that the proposed method outperforms the existing DL-based OOS-detection methods and detects fully empty and frontal OOS instances with 86.3% and 83.7% of the average precision, respectively.

Interpretable machine learning approach for neuron-centric analysis of human cortical cytoarchitecture.

The complexity of the cerebral cortex underlies its function and distinguishes us as humans. Here, we present a principled veridical data science methodology for quantitative histology that shifts focus from image-level investigations towards neuron-level representations of cortical regions, with the neurons in the image as a subject of study, rather than pixel-wise image content. Our methodology relies on the automatic segmentation of neurons across whole histological sections and an extensive set of engineered features, which reflect the neuronal phenotype of individual neurons and the properties of neurons' neighborhoods. The neuron-level representations are used in an interpretable machine learning pipeline for mapping the phenotype to cortical layers. To validate our approach, we created a unique dataset of cortical layers manually annotated by three experts in neuroanatomy and histology. The presented methodology offers high interpretability of the results, providing a deeper understanding of human cortex organization, which may help formulate new scientific hypotheses, as well as to cope with systematic uncertainty in data and model predictions.

Autoencoder-based training for multi-illuminant color constancy.

Color constancy is an essential component of the human visual system. It enables us to discern the color of objects invariant to the illumination that is present. This ability is difficult to reproduce in software, as the underlying problem is ill posed, i.e., for each pixel in the image, we know only the RGB values, which are a product of the spectral characteristics of the illumination and the reflectance of objects, as well as the sensitivity of the sensor. To combat this, additional assumptions about the scene have to be made. These assumptions can be either handcrafted or learned using some deep learning technique. Nonetheless, they mostly work only for single illuminant images. In this work, we propose a method for learning these assumptions for multi-illuminant scenes using an autoencoder trained to reconstruct the original image by splitting it into its illumination and reflectance components. We then show that the estimation can be used as is or can be used alongside a clustering method to create a segmentation map of illuminations. We show that our method performs the best out of all tested methods in multi-illuminant scenes while being completely invariant to the number of illuminants.

Deep learning-based anomaly detection from ultrasonic images.

Non-destructive testing is a group of methods for evaluating the integrity of components. Among them, ultrasonic inspection stands out due to its ability to visualize both shallow and deep sections of the material in the search for flaws. Testing of the critical components can be a tiring and time-consuming task. Therefore, human experts in analyzing inspection data could use a hand in discarding anomaly-free data and reviewing only suspicious data. Using such a tool, errors would be less common, inspection times would shorten and non-destructive testing would be more efficient. In this work, we evaluate multiple state-of-the-art deep-learning anomaly detection methods on the ultrasonic non-destructive testing dataset. We achieved an average performance of almost 82% of ROC AUC. We discuss in detail the advantages and disadvantages of the presented methods.

Generating ultrasonic images indistinguishable from real images using Generative Adversarial Networks.

Ultrasonic imaging is widely used for non-destructive evaluation in various industry applications. Early detection of defects in materials is the key to keeping the integrity of inspected structures. Currently, there have been some attempts to develop models for automated defect detection on ultrasonic data. To push the performance of these models even further more data is needed to train deep convolutional neural networks. A lot of data is also needed for training human experts. However, gathering a sufficient amount of data for training is a challenge due to the rare occurrence of defects in real inspection scenarios. This is why inspection results heavily depend on the inspector's previous experience. To overcome these challenges, we propose the use of Generative Adversarial Networks for generating realistic ultrasonic images. To the best of our knowledge, this work is the first one to show that a Generative Adversarial Network is able to generate images indistinguishable from real ultrasonic images. The most thorough statistical quality analysis to date of generated ultrasonic images has been conducted with the participation of human expert inspectors. The experimental results show that images generated using our Generative Adversarial Network provide the highest quality images compared to other published methods.

Automated Defect Detection From Ultrasonic Images Using Deep Learning.

Nondestructive evaluation (NDE) is a set of techniques used for material inspection and defect detection without causing damage to the inspected component. One of the commonly used nondestructive techniques is called ultrasonic inspection. The acquisition of ultrasonic data was mostly automated in recent years, but the analysis of the collected data is still performed manually. This process is thus very expensive, inconsistent, and prone to human errors. An automated system would significantly increase the efficiency of analysis, but the methods presented so far fail to generalize well on new cases and are not used in real-life inspection. Many of the similar data analysis problems were recently tackled by deep learning methods. This approach outperforms classical methods but requires lots of training data, which is difficult to obtain in the NDE domain. In this work, we train a deep learning architecture EfficientDet to automatically detect defects from ultrasonic images. We showed how some of the hyperparameters can be tweaked in order to improve the detection of defects with extreme aspect ratios that are common in ultrasonic images. The proposed object detector was trained on the largest dataset of ultrasonic images that was so far seen in the literature. In order to collect the dataset, six steel blocks containing 68 defects were scanned with a phased-array probe. More than 4000 VC-B-scans were acquired and used for training and evaluation of EfficientDet. The proposed model achieved 89.6% of mean average precision (mAP) during fivefold cross validation, which is a significant improvement compared to some similar methods that were previously used for this task. A detailed performance overview for each of the folds revealed that EfficientDet-D0 successfully detects all of the defects present in the inspected material.

Microvasculature Segmentation and Intercapillary Area Quantification of the Deep Vascular Complex Using Transfer Learning.

Purpose: Optical coherence tomography angiography (OCT-A) permits visualization of the changes to the retinal circulation due to diabetic retinopathy (DR), a microvascular complication of diabetes. We demonstrate accurate segmentation of the vascular morphology for the superficial capillary plexus (SCP) and deep vascular complex (DVC) using a convolutional neural network (CNN) for quantitative analysis. Methods: The main CNN training dataset consisted of retinal OCT-A with a 6 × 6-mm field of view (FOV), acquired using a Zeiss PlexElite. Multiple-volume acquisition and averaging enhanced the vasculature contrast used for constructing the ground truth for neural network training. We used transfer learning from a CNN trained on smaller FOVs of the SCP acquired using different OCT instruments. Quantitative analysis of perfusion was performed on the resulting automated vasculature segmentations in representative patients with DR. Results: The automated segmentations of the OCT-A images maintained the distinct morphologies of the SCP and DVC. The network segmented the SCP with an accuracy and Dice index of 0.8599 and 0.8618, respectively, and 0.7986 and 0.8139, respectively, for the DVC. The inter-rater comparisons for the SCP had an accuracy and Dice index of 0.8300 and 0.6700, respectively, and 0.6874 and 0.7416, respectively, for the DVC. Conclusions: Transfer learning reduces the amount of manually annotated images required while producing high-quality automatic segmentations of the SCP and DVC that exceed inter-rater comparisons. The resulting intercapillary area quantification provides a tool for in-depth clinical analysis of retinal perfusion. Translational Relevance: Accurate retinal microvasculature segmentation with the CNN results in improved perfusion analysis in diabetic retinopathy.

Complex Congenital Heart Disease Associated With Disordered Myocardial Architecture in a Midtrimester Human Fetus.

BACKGROUND: In the era of increasingly successful corrective interventions in patients with congenital heart disease (CHD), global and regional myocardial remodeling are emerging as important sources of long-term morbidity/mortality. Changes in organization of the myocardium in CHD, and in its mechanical properties, conduction, and blood supply, result in altered myocardial function both before and after surgery. To gain a better understanding and develop appropriate and individualized treatment strategies, the microscopic organization of cardiomyocytes, and their integration at a macroscopic level, needs to be completely understood. The aim of this study is to describe, for the first time, in 3 dimensions and nondestructively the detailed remodeling of cardiac microstructure present in a human fetal heart with complex CHD. METHODS AND RESULTS: Synchrotron X-ray phase-contrast imaging was used to image an archival midgestation formalin-fixed fetal heart with right isomerism and complex CHD and compare with a control fetal heart. Analysis of myocyte aggregates, at detail not accessible with other techniques, was performed. Macroanatomic and conduction system changes specific to the disease were clearly observable, together with disordered myocyte organization in the morphologically right ventricle myocardium. Electrical activation simulations suggested altered synchronicity of the morphologically right ventricle. CONCLUSIONS: We have shown the potential of X-ray phase-contrast imaging for studying cardiac microstructure in the developing human fetal heart at high resolution providing novel insight while preserving valuable archival material for future study. This is the first study to show myocardial alterations occur in complex CHD as early as midgestation.

A two dimensional electromechanical model of a cardiomyocyte to assess intra-cellular regional mechanical heterogeneities.

Experimental studies on isolated cardiomyocytes from different animal species and human hearts have demonstrated that there are regional differences in the Ca2+ release, Ca2+ decay and sarcomere deformation. Local deformation heterogeneities can occur due to a combination of factors: regional/local differences in Ca2+ release and/or re-uptake, intra-cellular material properties, sarcomere proteins and distribution of the intracellular organelles. To investigate the possible causes of these heterogeneities, we developed a two-dimensional finite-element electromechanical model of a cardiomyocyte that takes into account the experimentally measured local deformation and cytosolic [Ca2+] to locally define the different variables of the constitutive equations describing the electro/mechanical behaviour of the cell. Then, the model was individualised to three different rat cardiac cells. The local [Ca2+] transients were used to define the [Ca2+]-dependent activation functions. The cell-specific local Young's moduli were estimated by solving an inverse problem, minimizing the error between the measured and simulated local deformations along the longitudinal axis of the cell. We found that heterogeneities in the deformation during contraction were determined mainly by the local elasticity rather than the local amount of Ca2+, while in the relaxation phase deformation was mainly influenced by Ca2+ re-uptake. Our electromechanical model was able to successfully estimate the local elasticity along the longitudinal direction in three different cells. In conclusion, our proposed model seems to be a good approximation to assess the heterogeneous intracellular mechanical properties to help in the understanding of the underlying mechanisms of cardiomyocyte dysfunction.

Whole heart detailed and quantitative anatomy, myofibre structure and vasculature from X-ray phase-contrast synchrotron radiation-based micro computed tomography.

BACKGROUND: While individual cardiac myocytes only have a limited ability to shorten, the heart efficiently pumps a large volume-fraction thanks to a cell organization in a complex 3D fibre structure. Subclinical subtle cardiac structural remodelling is often present before symptoms arise. Understanding and early detection of these subtle changes is crucial for diagnosis and prevention. Additionally, personalized computational modelling requires knowledge on the multi-scale structure of the whole heart and vessels. METHODS AND RESULTS: We developed a rapid acquisition together with visualization and quantification methods of the integrated microstructure of whole in-vitro rodents hearts using synchrotron based X-ray phase-contrast tomography. These images are formed not only by X-ray absorption by the tissue but also by wave propagation phenomena, enhancing structural information, thus allowing to raise tissue contrast to an unprecedented level. We used a (ex-vivo) normal rat heart and fetal rabbit hearts suffering intrauterine growth restriction as a model of subclinical cardiac remodelling to illustrate the strengths and potential of the technique. For comparison, histology and diffusion tensor magnetic resonance imaging was performed. CONCLUSIONS: We have developed a novel, high resolution, image acquisition, and quantification approach to study a whole in-vitro heart at myofibre resolution, providing integrated 3D structural information at microscopic level without any need of tissue slicing and processing. This superior imaging approach opens up new possibilities for a systems approach towards analysing cardiac structure and function, providing rapid acquisition of quantitative microstructure of the heart in a near native state.

New Algorithm for Microstructural Information Determination from the Overlapping X-ray Diffraction Profiles.

Knowledge about the microstructure is crucial in targeted synthesis of novel nanomaterials. The microstructural parameters, crystallite size and crystallite strain play a major role in physical and chemical properties of the material. X-ray diffraction (XRD) is a very suitable method for this task, since it is non-destructive and it enables a very quick and precise determination of these parameters. The main problem lies in the case where the two neighboring diffraction profiles overlap each other. Here we present a new method for the separation of the overlapping profiles based on the differentiation of the profiles. Further, this method is appropriate for non-crystallographers working in the field of material science since it does not require any crystallographic experience and the full knowledge about the structure of the sample investigated. The microstructural results obtained by the proposed method are very accurate.

Segmentation of the foveal microvasculature using deep learning networks.

Accurate segmentation of the retinal microvasculature is a critical step in the quantitative analysis of the retinal circulation, which can be an important marker in evaluating the severity of retinal diseases. As manual segmentation remains the gold standard for segmentation of optical coherence tomography angiography (OCT-A) images, we present a method for automating the segmentation of OCT-A images using deep neural networks (DNNs). Eighty OCT-A images of the foveal region in 12 eyes from 6 healthy volunteers were acquired using a prototype OCT-A system and subsequently manually segmented. The automated segmentation of the blood vessels in the OCT-A images was then performed by classifying each pixel into vessel or nonvessel class using deep convolutional neural networks. When the automated results were compared against the manual segmentation results, a maximum mean accuracy of 0.83 was obtained. When the automated results were compared with inter and intrarater accuracies, the automated results were shown to be comparable to the human raters suggesting that segmentation using DNNs is comparable to a second manual rater. As manually segmenting the retinal microvasculature is a tedious task, having a reliable automated output such as automated segmentation by DNNs, is an important step in creating an automated output.

Detection of exudates in fundus photographs using deep neural networks and anatomical landmark detection fusion.

BACKGROUND AND OBJECTIVE: Diabetic retinopathy is one of the leading disabling chronic diseases and one of the leading causes of preventable blindness in developed world. Early diagnosis of diabetic retinopathy enables timely treatment and in order to achieve it a major effort will have to be invested into automated population screening programs. Detection of exudates in color fundus photographs is very important for early diagnosis of diabetic retinopathy. METHODS: We use deep convolutional neural networks for exudate detection. In order to incorporate high level anatomical knowledge about potential exudate locations, output of the convolutional neural network is combined with the output of the optic disc detection and vessel detection procedures. RESULTS: In the validation step using a manually segmented image database we obtain a maximum F1 measure of 0.78. CONCLUSIONS: As manually segmenting and counting exudate areas is a tedious task, having a reliable automated output, such as automated segmentation using convolutional neural networks in combination with other landmark detectors, is an important step in creating automated screening programs for early detection of diabetic retinopathy.

Smart light random memory sprays Retinex: a fast Retinex implementation for high-quality brightness adjustment and color correction.

Removing the influence of illumination on image colors and adjusting the brightness across the scene are important image enhancement problems. This is achieved by applying adequate color constancy and brightness adjustment methods. One of the earliest models to deal with both of these problems was the Retinex theory. Some of the Retinex implementations tend to give high-quality results by performing local operations, but they are computationally relatively slow. One of the recent Retinex implementations is light random sprays Retinex (LRSR). In this paper, a new method is proposed for brightness adjustment and color correction that overcomes the main disadvantages of LRSR. There are three main contributions of this paper. First, a concept of memory sprays is proposed to reduce the number of LRSR's per-pixel operations to a constant regardless of the parameter values, thereby enabling a fast Retinex-based local image enhancement. Second, an effective remapping of image intensities is proposed that results in significantly higher quality. Third, the problem of LRSR's halo effect is significantly reduced by using an alternative illumination processing method. The proposed method enables a fast Retinex-based image enhancement by processing Retinex paths in a constant number of steps regardless of the path size. Due to the halo effect removal and remapping of the resulting intensities, the method outperforms many of the well-known image enhancement methods in terms of resulting image quality. The results are presented and discussed. It is shown that the proposed method outperforms most of the tested methods in terms of image brightness adjustment, color correction, and computational speed.

Weighted ensemble based automatic detection of exudates in fundus photographs.

Diabetic retinopathy (DR) is a visual complication of diabetes, which has become one of the leading causes of preventable blindness in the world. Exudate detection is an important problem in automatic screening systems for detection of diabetic retinopathy using color fundus photographs. In this paper, we present a method for detection of exudates in color fundus photographs, which combines several preprocessing and candidate extraction algorithms to increase the exudate detection accuracy. The first stage of the method consists of an ensemble of several exudate candidate extraction algorithms. In the learning phase, simulated annealing is used to determine weights for combining the results of the ensemble candidate extraction algorithms. The second stage of the method uses a machine learning-based classification for detection of exudate regions. The experimental validation was performed using the DRiDB color fundus image set. The validation has demonstrated that the proposed method achieved higher accuracy in comparison to state-of-the art methods.

Image registration and atlas-based segmentation of cardiac outflow velocity profiles.

Cardiovascular disease is the leading cause of death worldwide and for this reason computer-based diagnosis of cardiac diseases is a very important task. In this article, a method for segmentation of aortic outflow velocity profiles from cardiac Doppler ultrasound images is presented. The proposed method is based on the statistical image atlas derived from ultrasound images of healthy volunteers. The ultrasound image segmentation is done by registration of the input image to the atlas, followed by a propagation of the segmentation result from the atlas onto the input image. In the registration process, the normalized mutual information is used as an image similarity measure, while optimization is performed using a multiresolution gradient ascent method. The registration method is evaluated using an in-silico phantom, real data from 30 volunteers, and an inverse consistency test. The segmentation method is evaluated using 59 images from healthy volunteers and 89 images from patients, and using cardiac parameters extracted from the segmented image. Experimental validation is conducted using a set of healthy volunteers and patients and has shown excellent results. Cardiac parameter segmentation evaluation showed that the variability of the automated segmentation relative to the manual is comparable to the intra-observer variability. The proposed method is useful for computer aided diagnosis and extraction of cardiac parameters.

The shape of the aortic outflow velocity profile revisited: is there a relation between its asymmetry and ventricular function in coronary artery disease?

AIMS: Myocardium contracts in the beginning of ejection causing outflow acceleration, resulting in asymmetric outflow velocity profiles peaking around one-third of ejection and declining when force development declines. This article aimed to demonstrate that decreased contractility in coronary artery disease (CAD) changes outflow timing and profile symmetry. METHODS AND RESULTS: Seventy-nine patients undergoing routine full dose dobutamine stress-echo (DSE) were divided into two groups based on resting wall motion and DSE response: DSE negative (DSE(neg)) (35 of 79 patients) and positive (DSE(pos)) (44 of 79 patients) which were compared with 32 healthy volunteers. Aortic CW-Doppler traces at rest were analysed semi-automatically; time-to-peak (T(mod)), ejection-time (ET(mod)), rise-time (t(rise)), and fall-time (t(fall)) were quantified. Asymmetry (asymm) was calculated as the normalized difference of left and right half of the spectrum. Normal curves were triangular, early-peaking, whereas patients showed more rounded shapes and later peaks. T(rise) was longest in DSE(pos). T(fall) was shortest in DSE(pos), followed by controls and DSE(neg). Asymm was lowest in DSE(pos), followed by controls and DSE(neg). Abnormally symmetric profiles (asymm <0.25) were found in none of the controls, 2.9% DSE(neg), and 27.3% DSE(pos). A good correlation was found between assym and ejection fraction (EF) and T(mod)/ET(mod) and EF. Notably, an LV dynamic gradient was induced in 71.4% DSE(neg) and in 18.2% DSE(pos), associated with LV hypertrophy and supernormal (very asymmetric) traces. CONCLUSION: Decreased myocardial function results in a more symmetrical outflow, while very asymmetrical traces suggest increased contractility, potentially inducing intra-cavity gradients during DSE. Therefore, including outflow symmetry as a clinical measurement provides additional information on patients with CAD.

Blood flow and velocity estimation based on vessel transit time by combining 2D and 3D X-ray angiography.

The X-ray imaging equipment could be used to measure hemodynamic function in addition to visualizing the morphology. The parameters of specific interest are arterial blood flow and velocity. Current monoplane X-ray systems can perform 3D reconstruction of the arterial tree as well as to capture the propagation of the injected contrast agent on a sequence of 2D angiograms. We combine the 2D digital subtraction angiography sequence with the mechanically registered 3D volume of the vessel tree. From 3D vessel tree we extract each vessel and obtain its centerline and cross-section area. We get our velocity estimation from 2D sequence by comparing time-density signals measured at different ends of the projected vessel. From the average velocity and cross-section area we get the average blood flow estimate for each vessel. The algorithm described here is applied to datasets from real neuroradiological studies.

Application of artificial neural networks for gradient elution retention modelling in ion chromatography.

Gradient elution in ion chromatography (IC) offers several advantages: total analysis time can be significantly reduced, overall resolution of a mixture can be increased, peak shape can be improved (less tailing) and effective sensitivity can be increased (because there is little variation in peak shape). More importantly, it provides the maximum resolution per time unit. The aim of this work was the development of a suitable artificial neural network (ANN) gradient elution retention model that can be used in a variety of applications for method development and retention modelling of inorganic anions in IC. Multilayer perceptron ANNs were used to model the retention behaviour of fluoride, chloride, nitrite, sulphate, bromide, nitrate and phosphate in relation to the starting time of gradient elution and the slope of the linear gradient elution curve. The advantage of the developed model is the application of an optimized two-phase training algorithm that enables the researcher to make use of the advantages of first- and second-order training algorithms in one training procedure. This results in better predictive ability, with less time required for the calculations. The number of hidden layer neurons and experimental data points used for the training set were optimized in terms of obtaining a precise and accurate retention model with respect to minimization of unnecessary experimentation and time needed for the calculation procedures. This study shows that developed, ANNs are the method of first choice for retention modelling of inorganic anions in IC.

Model-based quantitative AAA image analysis using a priori knowledge.

Abdominal aortic aneurysm (AAA) is a serious vascular disease which may have a fatal outcome. AAA shape and size is important for diagnostics and intervention planning. In this paper, we present a new method for segmentation of AAA from computed tomography (CT) angiography images. The method works by segmenting the inner and the outer aortic border. Segmentation of AAA is a challenging problem because of low contrast of the outer aortic border. In our method, the inner aortic border is segmented using a geometric deformable model (GDM) and morphological postprocessing. The GDM is implemented using the level-set algorithm. The outer aortic border is segmented by a preprocessing method utilizing a priori knowledge about the aorta shape, followed by the GDM-based method, and morphological postprocessing. The preprocessing algorithm operates on a slice-by-slice basis with some information flow among neighboring slices. The GDM performs three-dimensional (3D) segmentation, reducing possible errors in the previous step. The proposed method is automatic and requires minimal user assistance. The method was statistically validated on 12 patient scans having a total number of 497 image slices. Statistical analysis has confirmed high correlation between the results obtained by the proposed method and the gold standard obtained by manual segmentation by an expert radiologist.