Face-mask-aware Facial Expression Recognition based on Face Parsing and Vision Transformer.

As wearing face masks is becoming an embedded practice due to the COVID-19 pandemic, facial expression recognition (FER) that takes face masks into account is now a problem that needs to be solved. In this paper, we propose a face parsing and vision Transformer-based method to improve the accuracy of face-mask-aware FER. First, in order to improve the precision of distinguishing the unobstructed facial region as well as those parts of the face covered by a mask, we re-train a face-mask-aware face parsing model, based on the existing face parsing dataset automatically relabeled with a face mask and pixel label. Second, we propose a vision Transformer with a cross attention mechanism-based FER classifier, capable of taking both occluded and non-occluded facial regions into account and reweigh these two parts automatically to get the best facial expression recognition performance. The proposed method outperforms existing state-of-the-art face-mask-aware FER methods, as well as other occlusion-aware FER methods, on two datasets that contain three kinds of emotions (M-LFW-FER and M-KDDI-FER datasets) and two datasets that contain seven kinds of emotions (M-FER-2013 and M-CK+ datasets).

Enhancing classification of cells procured from bone marrow aspirate smears using generative adversarial networks and sequential convolutional neural network.

BACKGROUND AND OBJECTIVE: Leukemia represents 30% of all pediatric cancers and is considered the most common malignancy affecting adults and children. Cell differential count obtained from bone marrow aspirate smears is crucial for diagnosing hematologic diseases. Classification of these cell types is an essential task towards analyzing the disease, but it is time-consuming and requires intensive manual intervention. While machine learning has shown excellent outcomes in automating medical diagnosis, it needs ample data to build an efficient model for real-world tasks. This paper aims to generate synthetic data to enhance the classification accuracy of cells obtained from bone marrow aspirate smears. METHODS: A three-stage architecture has been proposed. We first collaborate with experts from the medical domain to prepare a dataset that consolidates microscopic cell images obtained from bone marrow aspirate smears from three different sources. The second stage involves a generative adversarial networks (GAN) model to generate synthetic microscopic cell images. We propose a GAN model consisting of three networks; generator discriminator and classifier. We train the GAN model with the loss function of Wasserstein GAN with gradient penalty (WGAN-GP). Since our GAN has an additional classifier and was trained using WGAN-GP, we named our model C-WGAN-GP. In the third stage, we propose a sequential convolutional neural network (CNN) to classify cells in the original and synthetic dataset to demonstrate how generating synthetic data and utilizing a simple sequential CNN model can enhance the accuracy of cell classification. RESULTS: We validated the proposed C-WGAN-GP and sequential CNN model with various evaluation metrics and achieved a classification accuracy of 96.98% using the synthetic dataset. We have presented each cell type's accuracy, specificity, and sensitivity results. The sequential CNN model achieves the highest accuracy for neutrophils with an accuracy rate of 97.5%. The highest value for sensitivity and specificity are 97.1% and 97%. Our proposed GAN model achieved an inception score of 14.52 ± 0.10, significantly better than the existing GAN models. CONCLUSIONS: Using three network GAN architecture produced more realistic synthetic data than existing models. Sequential CNN model with the synthetic data achieved higher classification accuracy than the original data.

Conditional generation of medical images via disentangled adversarial inference.

Synthetic medical image generation has a huge potential for improving healthcare through many applications, from data augmentation for training machine learning systems to preserving patient privacy. Conditional Adversarial Generative Networks (cGANs) use a conditioning factor to generate images and have shown great success in recent years. Intuitively, the information in an image can be divided into two parts: 1) content which is presented through the conditioning vector and 2) style which is the undiscovered information missing from the conditioning vector. Current practices in using cGANs for medical image generation, only use a single variable for image generation (i.e., content) and therefore, do not provide much flexibility nor control over the generated image. In this work we propose DRAI-a dual adversarial inference framework with augmented disentanglement constraints-to learn from the image itself, disentangled representations of style and content, and use this information to impose control over the generation process. In this framework, style is learned in a fully unsupervised manner, while content is learned through both supervised learning (using the conditioning vector) and unsupervised learning (with the inference mechanism). We undergo two novel regularization steps to ensure content-style disentanglement. First, we minimize the shared information between content and style by introducing a novel application of the gradient reverse layer (GRL); second, we introduce a self-supervised regularization method to further separate information in the content and style variables. For evaluation, we consider two types of baselines: single latent variable models that infer a single variable, and double latent variable models that infer two variables (style and content). We conduct extensive qualitative and quantitative assessments on two publicly available medical imaging datasets (LIDC and HAM10000) and test for conditional image generation, image retrieval and style-content disentanglement. We show that in general, two latent variable models achieve better performance and give more control over the generated image. We also show that our proposed model (DRAI) achieves the best disentanglement score and has the best overall performance.

Trends in IoT based solutions for health care: Moving AI to the edge.

In recent times, we assist to an ever growing diffusion of smart medical sensors and Internet of things devices that are heavily changing the way healthcare is approached worldwide. In this context, a combination of Cloud and IoT architectures is often exploited to make smart healthcare systems capable of supporting near realtime applications when processing and performing Artificial Intelligence on the huge amount of data produced by wearable sensor networks. Anyway, the response time and the availability of cloud based systems, together with security and privacy, still represent critical issues that prevents Internet of Medical Things (IoMT) devices and architectures from being a reliable and effective solution to the aim. Lately, there is a growing interest towards architectures and approaches that exploit Edge and Fog computing as an answer to compensate the weaknesses of the cloud. In this paper, we propose a short review about the general use of IoT solutions in health care, starting from early health monitoring solutions from wearable sensors up to a discussion about the latest trends in fog/edge computing for smart health.

Multi-part segmentation for porcine offal inspection with auto-context and adaptive atlases.

Extensions to auto-context segmentation are proposed and applied to segmentation of multiple organs in porcine offal as a component of an envisaged system for post-mortem inspection at abbatoir. In common with multi-part segmentation of many biological objects, challenges include variations in configuration, orientation, shape, and appearance, as well as inter-part occlusion and missing parts. Auto-context uses context information about inferred class labels and can be effective in such settings. Whereas auto-context uses a fixed prior atlas, we describe an adaptive atlas method better suited to represent the multimodal distribution of segmentation maps. We also design integral context features to enhance context representation. These methods are evaluated on a dataset captured at abbatoir and compared to a method based on conditional random fields. Results demonstrate the appropriateness of auto-context and the beneficial effects of the proposed extensions for this application.