A deep learning approach for Direct Immunofluorescence pattern recognition of Autoimmune Bullous Diseases.

BACKGROUND: Artificial intelligence (AI) is reshaping healthcare, using machine and deep learning to enhance disease management. Dermatology has seen improved diagnostics, particularly in skin cancer detection, through the integration of AI. However, the potential of AI in automating immunofluorescence imaging for autoimmune bullous skin diseases remains untapped. While direct immunofluorescence (DIF) supports diagnosis, its manual interpretation can hinder efficiency. The use of deep learning to automatically classify DIF patterns, including the Intercellular Pattern (ICP) and the Linear Pattern (LP), holds promise for improving the diagnosis of autoimmune bullous skin diseases. OBJECTIVES: The objectives of this study are to develop AI algorithms for automated classification of autoimmune bullous skin disease DIF patterns, such as ICP and LP. This aims to enhance diagnostic accuracy, streamline disease management, and improve patient outcomes through deep learning-driven immunofluorescence interpretation. METHODS: We collected immunofluorescence images from skin biopsies of patients suspected of AIBD between January 2022 and January 2024. Skin tissue was obtained via 5-mm punch biopsy, prepared for direct immunofluorescence. Experienced dermatologists classified the images into three classes: ICP, LP, and negative. To evaluate our deep learning approach, we divided the images into training (436) and test sets (93). We employed transfer learning with pre-trained deep neural networks and conducted 5-fold cross-validation to assess model performance. Our dataset's class imbalance was addressed using weighted loss and data augmentation strategies. The models were trained for 50 epochs using Pytorch, achieving an image size of 224x224 for both CNNs and the Swin Transformer. RESULTS: Our study compared six CNNs and the Swin transformer for AIBDs image classification, with the Swin transformer achieving the highest average validation accuracy of 98.5%. On a separate test set, the best model attained an accuracy of 94.6%, demonstrating 95.3% sensitivity and 97.5% specificity across AIBDs classes. Visualization with Grad-CAM highlighted the model's reliance on characteristic patterns for accurate classification. CONCLUSIONS: The study highlighted CNN's accuracy in identifying DIF features. This approach aids automated analysis and reporting, offering reproducibility, speed, data handling, and cost-efficiency. Integrating deep learning in skin immunofluorescence promises precise diagnostics and streamlined reporting in this branch of dermatology.

Knowledge distillation for efficient standard scanplane detection of fetal ultrasound.

In clinical practice, ultrasound standard planes (SPs) selection is experience-dependent and it suffers from inter-observer and intra-observer variability. Automatic recognition of SPs can help improve the quality of examinations and make the evaluations more objective. In this paper, we propose a method for the automatic identification of SPs, to be installed onboard a portable ultrasound system with limited computational power. The deep Learning methodology we design is based on the concept of Knowledge Distillation, transferring knowledge from a large and well-performing teacher to a smaller student architecture. To this purpose, we evaluate a set of different potential teachers and students, as well as alternative knowledge distillation techniques, to balance a trade-off between performances and architectural complexity. We report a thorough analysis of fetal ultrasound data, focusing on a benchmark dataset, to the best of our knowledge the only one available to date.

An efficient deep learning approach to identify dynamics in in vitro neural networks.

Understanding and discriminating the spatiotemporal patterns of activity generated by in vitro and in vivo neuronal networks is a fundamental task in neuroscience and neuroengineering. The state-of-the-art algorithms to describe the neuronal activity mostly rely on global and local well-established spike and burst-related parameters. However, they are not able to capture slight differences in the activity patterns. In this work, we introduce a deep-learning-based algorithm to automatically infer the dynamics exhibited by different neuronal populations. Specifically, we demonstrate that our algorithm is able to discriminate with high accuracy the dynamics of five different populations of in vitro human-derived neural networks with an increasing inhibitory to excitatory neurons ratio.

In-domain versus out-of-domain transfer learning in plankton image classification.

Plankton microorganisms play a huge role in the aquatic food web. Recently, it has been proposed to use plankton as a biosensor, since they can react to even minimal perturbations of the aquatic environment with specific physiological changes, which may lead to alterations in morphology and behavior. Nowadays, the development of high-resolution in-situ automatic acquisition systems allows the research community to obtain a large amount of plankton image data. Fundamental examples are the ZooScan and Woods Hole Oceanographic Institution (WHOI) datasets, comprising up to millions of plankton images. However, obtaining unbiased annotations is expensive both in terms of time and resources, and in-situ acquired datasets generally suffer from severe imbalance, with only a few images available for several species. Transfer learning is a popular solution to these challenges, with ImageNet1K being the most-used source dataset for pre-training. On the other hand, datasets like the ZooScan and the WHOI may represent a valuable opportunity to compare out-of-domain and large-scale plankton in-domain source datasets, in terms of performance for the task at hand.In this paper, we design three transfer learning pipelines for plankton image classification, with the aim of comparing in-domain and out-of-domain transfer learning on three popular benchmark plankton datasets. The general framework consists in fine-tuning a pre-trained model on a plankton target dataset. In the first pipeline, the model is pre-trained from scratch on a large-scale plankton dataset, in the second, it is pre-trained on large-scale natural image datasets (ImageNet1K or ImageNet22K), while in the third, a two-stage fine-tuning is implemented (ImageNet [Formula: see text] large-scale plankton dataset [Formula: see text] target plankton dataset). Our results show that an out-of-domain ImageNet22K pre-training outperforms the plankton in-domain ones, with an average boost in test accuracy of around 6%. In the next part of this work, we adopt three ImageNet22k pre-trained Vision Transformers and one ConvNeXt, obtaining results on par (or slightly superior) with the state-of-the-art, corresponding to the usage of CNN models ensembles, with a single model. Finally, we design and test an ensemble of our Vision Transformers and the ConvNeXt, outperforming the state-of-the-art existing works on plankton image classification on the three target datasets. To support scientific community contribution and further research, our implemented code is open-source and available at https://github.com/Malga-Vision/plankton_transfer .

Uncertainty-Aware Gaze Tracking for Assisted Living Environments.

Effective assisted living environments must be able to infer how their occupants interact in a variety of scenarios. Gaze direction provides strong indications of how a person engages with the environment and its occupants. In this paper, we investigate the problem of gaze tracking in multi-camera assisted living environments. We propose a gaze tracking method based on predictions generated by a neural network regressor that relies only on the relative positions of facial keypoints to estimate gaze. For each gaze prediction, our regressor also provides an estimate of its own uncertainty, which is used to weigh the contribution of previously estimated gazes within a tracking framework based on an angular Kalman filter. Our gaze estimation neural network uses confidence gated units to alleviate keypoint prediction uncertainties in scenarios involving partial occlusions or unfavorable views of the subjects. We evaluate our method using videos from the MoDiPro dataset, which we acquired in a real assisted living facility, and on the publicly available MPIIFaceGaze, GazeFollow, and Gaze360 datasets. Experimental results show that our gaze estimation network outperforms sophisticated state-of-the-art methods, while additionally providing uncertainty predictions that are highly correlated with the actual angular error of the corresponding estimates. Finally, an analysis of the temporal integration performance of our method demonstrates that it generates accurate and temporally stable gaze predictions.

Automatic video analysis and classification of sleep-related hypermotor seizures and disorders of arousal.

OBJECTIVE: Sleep-related hypermotor epilepsy (SHE) is a focal epilepsy with seizures occurring mostly during sleep. SHE seizures present different motor characteristics ranging from dystonic posturing to hyperkinetic motor patterns, sometimes associated with affective symptoms and complex behaviors. Disorders of arousal (DOA) are sleep disorders with paroxysmal episodes that may present analogies with SHE seizures. Accurate interpretation of the different SHE patterns and their differentiation from DOA manifestations can be difficult and expensive, and can require highly skilled personnel not always available. Furthermore, it is operator dependent. METHODS: Common techniques for human motion analysis, such as wearable sensors (e.g., accelerometers) and motion capture systems, have been considered to overcome these problems. Unfortunately, these systems are cumbersome and they require trained personnel for marker and sensor positioning, limiting their use in the epilepsy domain. To overcome these problems, recently significant effort has been spent in studying automatic methods based on video analysis for the characterization of human motion. Systems based on computer vision and deep learning have been exploited in many fields, but epilepsy has received limited attention. RESULTS: In this paper, we present a pipeline composed of a set of three-dimensional convolutional neural networks that, starting from video recordings, reached an overall accuracy of 80% in the classification of different SHE semiology patterns and DOA. SIGNIFICANCE: The preliminary results obtained in this study highlight that our deep learning pipeline could be used by physicians as a tool to support them in the differential diagnosis of the different patterns of SHE and DOA, and encourage further investigation.

A fully automated deep learning pipeline for micro-CT-imaging-based densitometry of lung fibrosis murine models.

Idiopathic pulmonary fibrosis, the archetype of pulmonary fibrosis (PF), is a chronic lung disease of a poor prognosis, characterized by progressively worsening of lung function. Although histology is still the gold standard for PF assessment in preclinical practice, histological data typically involve less than 1% of total lung volume and are not amenable to longitudinal studies. A miniaturized version of computed tomography (µCT) has been introduced to radiologically examine lung in preclinical murine models of PF. The linear relationship between X-ray attenuation and tissue density allows lung densitometry on total lung volume. However, the huge density changes caused by PF usually require manual segmentation by trained operators, limiting µCT deployment in preclinical routine. Deep learning approaches have achieved state-of-the-art performance in medical image segmentation. In this work, we propose a fully automated deep learning approach to segment right and left lung on µCT imaging and subsequently derive lung densitometry. Our pipeline first employs a convolutional network (CNN) for pre-processing at low-resolution and then a 2.5D CNN for higher-resolution segmentation, combining computational advantage of 2D and ability to address 3D spatial coherence without compromising accuracy. Finally, lungs are divided into compartments based on air content assessed by density. We validated this pipeline on 72 mice with different grades of PF, achieving a Dice score of 0.967 on test set. Our tests demonstrate that this automated tool allows for rapid and comprehensive analysis of µCT scans of PF murine models, thus laying the ground for its wider exploitation in preclinical settings.

A markerless pipeline to analyze spontaneous movements of preterm infants.

BACKGROUND AND OBJECTIVE: The analysis of spontaneous movements of preterm infants is important because anomalous motion patterns can be a sign of neurological disorders caused by lesions in the developing brain. A diagnosis in the first weeks of child's life is crucial to plan timely and appropriate rehabilitative interventions. An accurate visual assessment of infants' spontaneous movements requires highly specialized personnel, not always available, and it is operator dependent. Motion capture systems, markers and wearable sensors are commonly used for human motion analysis, but they can be cumbersome, limiting their use in the study of infants' movements. METHODS: In this paper we propose a computer-aided pipeline to characterize and classify infants' motion from 2D video recordings. The final goal is detecting anomalous motion patterns. The implemented pipeline is based on computer vision and machine learning algorithms and includes a specific step to increase the interpretability of the results. Specifically, it can be summarized by the following steps: (i) body keypoints detection: we rely on a deep learning-based semantic features detector to localize the positions of meaningful landmark points on infants' bodies; (ii) parameters extraction: starting from the trajectories of the detected landmark points, we extract quantitative parameters describing infants motion patterns; (iii) classification: we implement different classifiers (Support Vector Machines, Random Forest, fully connected Neural Network, Long Short Term Memory) that, starting from the motion parameters, classify between normal or abnormal motion patterns. RESULTS: We tested the proposed pipeline on a dataset, recorded at the 40th gestational week, of 142 infants, 59 with evidence of neuromotor disorders according to a medical assessment carried out a posteriori. Our procedure successfully discriminates normal and anomalous motion patterns with a maximum accuracy of 85.7%. CONCLUSIONS: In conclusion, our pipeline has the potential to be adopted as a tool to support the early detection of abnormal motion patterns in preterm infants.

A semi-automatic toolbox for markerless effective semantic feature extraction.

VisionTool is an open-source python toolbox for semantic features extraction, capable to provide accurate features detectors for different applications, including motion analysis, markerless pose estimation, face recognition and biological cell tracking. VisionTool leverages transfer-learning with a large variety of deep neural networks allowing high-accuracy features detection with few training data. The toolbox offers a friendly graphical user interface, efficiently guiding the user through the entire process of features extraction. To facilitate broad usage and scientific community contribution, the code and a user guide are available at https://github.com/Malga-Vision/VisionTool.git .

Markerless vs. Marker-Based Gait Analysis: A Proof of Concept Study.

The analysis of human gait is an important tool in medicine and rehabilitation to evaluate the effects and the progression of neurological diseases resulting in neuromotor disorders. In these fields, the gold standard techniques adopted to perform gait analysis rely on motion capture systems and markers. However, these systems present drawbacks: they are expensive, time consuming and they can affect the naturalness of the motion. For these reasons, in the last few years, considerable effort has been spent to study and implement markerless systems based on videography for gait analysis. Unfortunately, only few studies quantitatively compare the differences between markerless and marker-based systems in 3D settings. This work presented a new RGB video-based markerless system leveraging computer vision and deep learning to perform 3D gait analysis. These results were compared with those obtained by a marker-based motion capture system. To this end, we acquired simultaneously with the two systems a multimodal dataset of 16 people repeatedly walking in an indoor environment. With the two methods we obtained similar spatio-temporal parameters. The joint angles were comparable, except for a slight underestimation of the maximum flexion for ankle and knee angles. Taking together these results highlighted the possibility to adopt markerless technique for gait analysis.

The MoCA dataset, kinematic and multi-view visual streams of fine-grained cooking actions.

MoCA is a bi-modal dataset in which we collect Motion Capture data and video sequences acquired from multiple views, including an ego-like viewpoint, of upper body actions in a cooking scenario. It has been collected with the specific purpose of investigating view-invariant action properties in both biological and artificial systems. Besides that, it represents an ideal test bed for research in a number of fields - including cognitive science and artificial vision - and application domains - as motor control and robotics. Compared to other benchmarks available, MoCA provides a unique compromise for research communities leveraging very different approaches to data gathering: from one extreme of action recognition in the wild - the standard practice nowadays in the fields of Computer Vision and Machine Learning - to motion analysis in very controlled scenarios - as for motor control in biomedical applications. In this work we introduce the dataset and its peculiarities, and discuss a baseline analysis as well as examples of applications for which the dataset is well suited.

Scale Invariant and Noise Robust Interest Points With Shearlets.

Shearlets are a relatively new directional multi-scale framework for signal analysis, which have been shown effective to enhance signal discontinuities, such as edges and corners at multiple scales even in the presence of a large quantity of noise. In this paper, we consider blob-like features in the shearlets framework. We derive a measure, which is very effective for blob detection, and, based on this measure, we propose a blob detector and a keypoint description, whose combination outperforms the state-of-the-art algorithms with noisy and compressed images. We also demonstrate that the measure satisfies the perfect scale invariance property in the continuous case. We evaluate the robustness of our algorithm to different types of noise, including blur, compression artifacts, and Gaussian noise. Furthermore, we carry on a comparative analysis on benchmark data, referring, in particular, to tolerance to noise and image compression.

Edges and Corners With Shearlets.

Shearlets are a relatively new and very effective multi-scale framework for signal analysis. Contrary to the traditional wavelets, shearlets are capable to efficiently capture the anisotropic information in multivariate problem classes. Therefore, shearlets can be seen as the valid choice for multi-scale analysis and detection of directional sensitive visual features like edges and corners. In this paper, we start by reviewing the main properties of shearlets that are important for edge and corner detection. Then, we study algorithms for multi-scale edge and corner detection based on the shearlet representation. We provide an extensive experimental assessment on benchmark data sets which empirically confirms the potential of shearlets feature detection.

Online space-variant background modeling with sparse coding.

In this paper, we propose a sparse coding approach to background modeling. The obtained model is based on dictionaries which we learn and keep up to date as new data are provided by a video camera. We observe that, without dynamic events, video frames may be seen as noisy data belonging to the background. Over time, such background is subject to local and global changes due to variable illumination conditions, camera jitter, stable scene changes, and intermittent motion of background objects. To capture the locality of some changes, we propose a space-variant analysis where we learn a dictionary of atoms for each image patch, the size of which depends on the background variability. At run time, each patch is represented by a linear combination of the atoms learnt online. A change is detected when the atoms are not sufficient to provide an appropriate representation, and stable changes over time trigger an update of the current dictionary. Even if the overall procedure is carried out at a coarse level, a pixel-wise segmentation can be obtained by comparing the atoms with the patch corresponding to the dynamic event. Experiments on benchmarks indicate that the proposed method achieves very good performances on a variety of scenarios. An assessment on long video streams confirms our method incorporates periodical changes, as the ones caused by variations in natural illumination. The model, fully data driven, is suitable as a main component of a change detection system.

A sparsity-enforcing method for learning face features.

In this paper, we propose a new trainable system for selecting face features from over-complete dictionaries of image measurements. The starting point is an iterative thresholding algorithm which provides sparse solutions to linear systems of equations. Although the proposed methodology is quite general and could be applied to various image classification tasks, we focus here on the case study of face and eyes detection. For our initial representation, we adopt rectangular features in order to allow straightforward comparisons with existing techniques. For computational efficiency and memory saving requirements, instead of implementing the full optimization scheme on tenths of thousands of features, we propose a three-stage architecture which consists of finding first intermediate solutions to smaller size optimization problems, then merging the obtained results, and next applying further selection procedures. The devised system requires the solution of a number of independent problems, and, hence, the necessary computations could be implemented in parallel. Experimental results obtained on both benchmark and newly acquired face and eyes images indicate that our method is a serious competitor to other feature selection schemes recently popularized in computer vision for dealing with problems of real-time object detection. A major advantage of the proposed system is that it performs well even with relatively small training sets.

Building kernels from binary strings for image matching.

In the statistical learning framework, the use of appropriate kernels may be the key for substantial improvement in solving a given problem. In essence, a kernel is a similarity measure between input points satisfying some mathematical requirements and possibly capturing the domain knowledge. In this paper, we focus on kernels for images: we represent the image information content with binary strings and discuss various bitwise manipulations obtained using logical operators and convolution with nonbinary stencils. In the theoretical contribution of our work, we show that histogram intersection is a Mercer's kernel and we determine the modifications under which a similarity measure based on the notion of Hausdorff distance is also a Mercer's kernel. In both cases, we determine explicitly the mapping from input to feature space. The presented experimental results support the relevance of our analysis for developing effective trainable systems.