Facial expression recognition in videos using hybrid CNN & ConvLSTM.

The three-dimensional convolutional neural network (3D-CNN) and long short-term memory (LSTM) have consistently outperformed many approaches in video-based facial expression recognition (VFER). The image is unrolled to a one-dimensional vector by the vanilla version of the fully-connected LSTM (FC-LSTM), which leads to the loss of crucial spatial information. Convolutional LSTM (ConvLSTM) overcomes this limitation by performing LSTM operations in convolutions without unrolling, thus retaining useful spatial information. Motivated by this, in this paper, we propose a neural network architecture that consists of a blend of 3D-CNN and ConvLSTM for VFER. The proposed hybrid architecture captures spatiotemporal information from the video sequences of emotions and attains competitive accuracy on three FER datasets open to the public, namely the SAVEE, CK + , and AFEW. The experimental results demonstrate excellent performance without external emotional data with the added advantage of having a simple model with fewer parameters. Moreover, unlike the state-of-the-art deep learning models, our designed FER pipeline improves execution speed by many factors while achieving competitive recognition accuracy. Hence, the proposed FER pipeline is an appropriate candidate for recognizing facial expressions on resource-limited embedded platforms for real-time applications.

A non-invasive ultrasensitive diagnostic approach for COVID-19 infection using salivary label-free SERS fingerprinting and artificial intelligence.

Clinical diagnostics for SARS-CoV-2 infection usually comprises the sampling of throat or nasopharyngeal swabs that are invasive and create patient discomfort. Hence, saliva is attempted as a sample of choice for the management of COVID-19 outbreaks that cripples the global healthcare system. Although limited by the risk of eliciting false-negative and positive results, tedious test procedures, requirement of specialized laboratories, and expensive reagents, nucleic acid-based tests remain the gold standard for COVID-19 diagnostics. However, genetic diversity of the virus due to rapid mutations limits the efficiency of nucleic acid-based tests. Herein, we have demonstrated the simplest screening modality based on label-free surface enhanced Raman scattering (LF-SERS) for scrutinizing the SARS-CoV-2-mediated molecular-level changes of the saliva samples among healthy, COVID-19 infected and COVID-19 recovered subjects. Moreover, our LF-SERS technique enabled to differentiate the three classes of corona virus spike protein derived from SARS-CoV-2, SARS-CoV and MERS-CoV. Raman spectral data was further decoded, segregated and effectively managed with the aid of machine learning algorithms. The classification models built upon biochemical signature-based discrimination method of the COVID-19 condition from the patient saliva ensured high accuracy, specificity, and sensitivity. The trained support vector machine (SVM) classifier achieved a prediction accuracy of 95% and F1-score of 94.73%, and 95.28% for healthy and COVID-19 infected patients respectively. The current approach not only differentiate SARS-CoV-2 infection with healthy controls but also predicted a distinct fingerprint for different stages of patient recovery. Employing portable hand-held Raman spectrophotometer as the instrument and saliva as the sample of choice will guarantee a rapid and non-invasive diagnostic strategy to warrant or assure patient comfort and large-scale population screening for SARS-CoV-2 infection and monitoring the recovery process.

Three-dimensional DenseNet self-attention neural network for automatic detection of student's engagement.

Today, due to the widespread outbreak of the deadly coronavirus, popularly known as COVID-19, the traditional classroom education has been shifted to computer-based learning. Students of various cognitive and psychological abilities participate in the learning process. However, most students are hesitant to provide regular and honest feedback on the comprehensiveness of the course, making it difficult for the instructor to ensure that all students are grasping the information at the same rate. The students' understanding of the course and their emotional engagement, as indicated via facial expressions, are intertwined. This paper attempts to present a three-dimensional DenseNet self-attention neural network (DenseAttNet) used to identify and evaluate student participation in modern and traditional educational programs. With the Dataset for Affective States in E-Environments (DAiSEE), the proposed DenseAttNet model outperformed all other existing methods, achieving baseline accuracy of 63.59% for engagement classification and 54.27% for boredom classification, respectively. Besides, DenseAttNet trained on all four multi-labels, namely boredom, engagement, confusion, and frustration has registered an accuracy of 81.17%, 94.85%, 90.96%, and 95.85%, respectively. In addition, we performed a regression experiment on DAiSEE and obtained the lowest Mean Square Error (MSE) value of 0.0347. Finally, the proposed approach achieves a competitive MSE of 0.0877 when validated on the Emotion Recognition in the Wild Engagement Prediction (EmotiW-EP) dataset.

Validation of expert system enhanced deep learning algorithm for automated screening for COVID-Pneumonia on chest X-rays.

SARS-CoV2 pandemic exposed the limitations of artificial intelligence based medical imaging systems. Earlier in the pandemic, the absence of sufficient training data prevented effective deep learning (DL) solutions for the diagnosis of COVID-19 based on X-Ray data. Here, addressing the lacunae in existing literature and algorithms with the paucity of initial training data; we describe CovBaseAI, an explainable tool using an ensemble of three DL models and an expert decision system (EDS) for COVID-Pneumonia diagnosis, trained entirely on pre-COVID-19 datasets. The performance and explainability of CovBaseAI was primarily validated on two independent datasets. Firstly, 1401 randomly selected CxR from an Indian quarantine center to assess effectiveness in excluding radiological COVID-Pneumonia requiring higher care. Second, curated dataset; 434 RT-PCR positive cases and 471 non-COVID/Normal historical scans, to assess performance in advanced medical settings. CovBaseAI had an accuracy of 87% with a negative predictive value of 98% in the quarantine-center data. However, sensitivity was 0.66-0.90 taking RT-PCR/radiologist opinion as ground truth. This work provides new insights on the usage of EDS with DL methods and the ability of algorithms to confidently predict COVID-Pneumonia while reinforcing the established learning; that benchmarking based on RT-PCR may not serve as reliable ground truth in radiological diagnosis. Such tools can pave the path for multi-modal high throughput detection of COVID-Pneumonia in screening and referral.