Facial thermal and blood perfusion patterns of human emotions: Proof-of-Concept.

In this work, a preliminary study of proof-of-concept was conducted to evaluate the performance of the thermographic and blood perfusion data when emotions of positive and negative valence are applied, where the blood perfusion data are obtained from the thermographic data. The images were obtained for baseline, positive, and negative valence according to the protocol of the Geneva Affective Picture Database. Absolute and percentage differences of average values of the data between the valences and the baseline were calculated for different regions of interest (forehead, periorbital eyes, cheeks, nose and upper lips). For negative valence, a decrease in temperature and blood perfusion was observed in the regions of interest, and the effect was greater on the left side than on the right side. In positive valence, the temperature and blood perfusion increased in some cases, showing a complex pattern. The temperature and perfusion of the nose was reduced for both valences, which is indicative of the arousal dimension. The blood perfusion images were found to be greater contrast; the percentage differences in the blood perfusion images are greater than those obtained in thermographic images. Moreover, the blood perfusion images, and vasomotor answer are consistent, therefore, they can be a better biomarker than thermographic analysis in identifying emotions.

Improved Thermal Infrared Image Super-Resolution Reconstruction Method Base on Multimodal Sensor Fusion.

When traditional super-resolution reconstruction methods are applied to infrared thermal images, they often ignore the problem of poor image quality caused by the imaging mechanism, which makes it difficult to obtain high-quality reconstruction results even with the training of simulated degraded inverse processes. To address these issues, we proposed a thermal infrared image super-resolution reconstruction method based on multimodal sensor fusion, aiming to enhance the resolution of thermal infrared images and rely on multimodal sensor information to reconstruct high-frequency details in the images, thereby overcoming the limitations of imaging mechanisms. First, we designed a novel super-resolution reconstruction network, which consisted of primary feature encoding, super-resolution reconstruction, and high-frequency detail fusion subnetwork, to enhance the resolution of thermal infrared images and rely on multimodal sensor information to reconstruct high-frequency details in the images, thereby overcoming limitations of imaging mechanisms. We designed hierarchical dilated distillation modules and a cross-attention transformation module to extract and transmit image features, enhancing the network's ability to express complex patterns. Then, we proposed a hybrid loss function to guide the network in extracting salient features from thermal infrared images and reference images while maintaining accurate thermal information. Finally, we proposed a learning strategy to ensure the high-quality super-resolution reconstruction performance of the network, even in the absence of reference images. Extensive experimental results show that the proposed method exhibits superior reconstruction image quality compared to other contrastive methods, demonstrating its effectiveness.

Multisensorial Assessment of Laser Effects on Shellac Applied on Wall Paintings.

The assessment of five different laser treatments in the conservation of wall paintings was devised on the basis of the surface temperature monitoring by infrared thermography (IRT), ultraviolet-induced fluorescence-visible (UV-VIS) imaging, and optical coherence tomography (OCT). A series of yttrium-aluminum-garnet (YAG) lasers were tested for removal of shellac layers from wall painting mock-ups. The mock-ups were realized as buon fresco with different mineral based pigments (earths and iron oxide) on a lime- and sand-based mortar. After the carbonatation process, all the samples were treated with shellac (5% in ethanol). The effects of neodymium (Nd):YAG, holmium (Ho):YAG, and erbium (Er):YAG laser sources, in different operative modes, on average temperature of the surface, color, and morphology were inspected with complementary sensors. The results show the necessity to adopt a combined approach in establishing safe laser operating conditions to avoid any undesired effects induced on the artefacts by the laser treatments. We demonstrate, for the first time, the performance of the Ho:YAG laser in the removal of a conservation treatment.

Thermal infrared imaging based facial temperature in comparison to ear temperature during a real-driving scenario.

Motion Sickness is associated with a variety of symptoms, which differ in occurrence rate and intensity between individuals. In order to research the cause of car sickness and develop countermeasures, it is important to determine symptoms and their severity objectively. A tool for this purpose could be the assessment of physiological reactions due to motion sickness. This paper describes and discusses a methodology to identify changes in facial skin temperatures in a real-driving study. Common techniques had to be adjusted in order to meet the requirements given by the challenges of in-car-recording. The examined data was generated in a previous study, which was designed to research motion sickness in a driving environment. A pre-processing technique had to be developed to magnify features on the face and subsequently improve the tracking in thermal imagery. After the pre-processing, regions of interest (ROI) were manually marked and tracked in thermal images. The thereby assessed facial skin temperatures were compared to tympanic temperatures. Derived temperatures from the forehead as well as from the 20 hottest pixels within the face indicated a better tracking, while the nose tip was more affected by detection errors. The correlation of the three features with the tympanic temperature showed remarkable differences between a baseline measurement and the actual driving. Less than 10% of the data derived during the driving and up to 30% of the data during the baseline measurement correlated highly. It is concluded that detecting changes in facial skin temperature using thermal infrared imaging in a moving car is challenging and results are hardly comparable to tympanic temperatures. Future research should aim at the different influencing factors of skin and tympanic temperature, while enhancing tracking or detection of ROI could be achieved by reducing the passengers' movements or choosing the target area more carefully.

Determining Mancozeb Deposition Benchmark Values on Apple Leaves for the Management of Venturia inaequalis.

Apple scab, caused by Venturia inaequalis, is the most common fruit and foliar disease in commercial apple production worldwide. Early in the production season, preventative contact fungicide sprays are essential for protecting highly susceptible continuously unfolding and expanding young leaves. In South Africa, mancozeb is a key contact fungicide used for controlling apple scab early in the season. The current study developed deposition benchmarks indicative of the biological efficacy of mancozeb against apple scab, using a laboratory-based apple seedling model system. The model system employed a yellow fluorescent pigment that is known to be an effective tracer of mancozeb deposition. A concentration range of mancozeb (0.15 to 1 times the registered dosage) and fluorescent pigment concentrations was sprayed onto seedling leaves, which yielded various fluorescent particle coverage (FPC%) levels. Modeling of the FPC% values versus percent disease control yielded different benchmark values when disease quantification was conducted using two different methods. Thermal infrared imaging (TIRI) disease quantification resulted in a benchmark model where 0.40%, 0.79%, and 1.35 FPC% yielded 50, 75, and 90% apple scab control, respectively. These FPC% values were higher than the benchmarks (0.10, 0.20, and 0.34 FPC%, respectively) obtained with quantitative real-time PCR (qPCR) disease quantification. The qPCR benchmark model is recommended as a guideline for evaluating the efficacy of mancozeb sprays on leaves in apple orchards since the TIRI benchmark model underestimated disease control. The TIRI benchmark model yielded 68% disease control at the lowest mancozeb dosage, yet no visible lesion developed at this dosage. Both benchmark models showed that mancozeb yielded high levels of disease control at very low concentrations; for the qPCR benchmark model the FPC% value of the FPC90 (90% control) corresponded to 0.15 times that of the registered mancozeb concentration in South Africa, i.e., 85% lower than the registered dosage.

A Modular System for Detection, Tracking and Analysis of Human Faces in Thermal Infrared Recordings.

We present a system that utilizes a range of image processing algorithms to allow fully automated thermal face analysis under both laboratory and real-world conditions. We implement methods for face detection, facial landmark detection, face frontalization and analysis, combining all of these into a fully automated workflow. The system is fully modular and allows implementing own additional algorithms for improved performance or specialized tasks. Our suggested pipeline contains a histogtam of oriented gradients support vector machine (HOG-SVM) based face detector and different landmark detecion methods implemented using feature-based active appearance models, deep alignment networks and a deep shape regression network. Face frontalization is achieved by utilizing piecewise affine transformations. For the final analysis, we present an emotion recognition system that utilizes HOG features and a random forest classifier and a respiratory rate analysis module that computes average temperatures from an automatically detected region of interest. Results show that our combined system achieves a performance which is comparable to current stand-alone state-of-the-art methods for thermal face and landmark datection and a classification accuracy of 65.75% for four basic emotions.

Feasibility Study on S-Band Microwave Radiation and 3D-Thermal Infrared Imaging Sensor-Aided Recognition of Polymer Materials from End-of-Life Vehicles.

With the increase the worldwide consumption of vehicles, end-of-life vehicles (ELVs) have kept rapidly increasing in the last two decades. Metallic parts and materials of ELVs can be easily reused and recycled, but the automobile shredder residues (ASRs), of which elastomer and plastic materials make up the vast majority, are difficult to recycle. ASRs are classified as hazardous materials in the main industrial countries, and are required to be materially recycled up to 85⁻95% by mass until 2020. However, there is neither sufficient theoretical nor practical experience for sorting ASR polymers. In this research, we provide a novel method by using S-Band microwave irradiation together with 3D scanning as well as infrared thermal imaging sensors for the recognition and sorting of typical plastics and elastomers from the ASR mixture. In this study, an industrial magnetron array with 2.45 GHz irradiation was utilized as the microwave source. Seven kinds of ELV polymer (PVC, ABS, PP, EPDM, NBR, CR, and SBR) crushed scrap residues were tested. After specific power microwave irradiation for a certain time, the tested polymer materials were heated up to different extents corresponding to their respective sensitivities to microwave irradiation. Due to the variations in polymer chemical structure and additive agents, polymers have different sensitivities to microwave radiation, which leads to differences in temperature rises. The differences of temperature increase were obtained by a thermal infrared sensor, and the position and geometrical features of the tested scraps were acquired by a 3D imaging sensor. With this information, the scrap material could be recognized and then sorted. The results showed that this method was effective when the tested polymer materials were heated up to more than 30 °C. For full recognition of the tested polymer scraps, the minimum temperature variations of 5 °C and 10.5 °C for plastics and elastomers were needed, respectively. The sorting efficiency was independent of particle sizes but depended on the power and time of the microwave irradiation. Generally, more than 75% (mass) of the tested polymer materials could be successfully recognized and sorted under an irradiation power of 3 kW. Plastics were much more insensitive to microwave irradiation than elastomers. With this method, the tested mixture of the plastic group (PVC, ABS, PP) and the mixture of elastomer group (EPDM, NBR, CR, and SBR) could be fully separated with an efficiency of 100%.

Thermal facial image analyses reveal quantitative hallmarks of aging and metabolic diseases.

Although human core body temperature is known to decrease with age, the age dependency of facial temperature and its potential to indicate aging rate or aging-related diseases remains uncertain. Here, we collected thermal facial images of 2,811 Han Chinese individuals 20-90 years old, developed the ThermoFace method to automatically process and analyze images, and then generated thermal age and disease prediction models. The ThermoFace deep learning model for thermal facial age has a mean absolute deviation of about 5 years in cross-validation and 5.18 years in an independent cohort. The difference between predicted and chronological age is highly associated with metabolic parameters, sleep time, and gene expression pathways like DNA repair, lipolysis, and ATPase in the blood transcriptome, and it is modifiable by exercise. Consistently, ThermoFace disease predictors forecast metabolic diseases like fatty liver with high accuracy (AUC > 0.80), with predicted disease probability correlated with metabolic parameters.

Intraoperative thermal infrared imaging in neurosurgery: machine learning approaches for advanced segmentation of tumors.

Surgical resection is one of the most relevant practices in neurosurgery. Finding the correct surgical extent of the tumor is a key question and so far several techniques have been employed to assist the neurosurgeon in preserving the maximum amount of healthy tissue. Some of these methods are invasive for patients, not always allowing high precision in the detection of the tumor area. The aim of this study is to overcome these limitations, developing machine learning based models, relying on features obtained from a contactless and non-invasive technique, the thermal infrared (IR) imaging. The thermal IR videos of thirteen patients with heterogeneous tumors were recorded in the intraoperative context. Time (TD)- and frequency (FD)-domain features were extracted and fed different machine learning models. Models relying on FD features have proven to be the best solutions for the optimal detection of the tumor area (Average Accuracy = 90.45%; Average Sensitivity = 84.64%; Average Specificity = 93,74%). The obtained results highlight the possibility to accurately detect the tumor lesion boundary with a completely non-invasive, contactless, and portable technology, revealing thermal IR imaging as a very promising tool for the neurosurgeon.

Facial functional networks during resting state revealed by thermal infrared imaging.

In recent decades, an increasing number of studies on psychophysiology and, in general, on clinical medicine has employed the technique of facial thermal infrared imaging (IRI), which allows to obtain information about the emotional and physical states of the subjects in a completely non-invasive and contactless fashion. Several regions of interest (ROIs) have been reported in literature as salient areas for the psychophysiological characterization of a subject (i.e. nose tip and glabella ROIs). There is however a lack of studies focusing on the functional correlation among these ROIs and about the physiological basis of the relation existing between thermal IRI and vital signals, such as the electrodermal activity, i.e. the galvanic skin response (GSR). The present study offers a new methodology able to assess the functional connection between salient seed ROIs of thermal IRI and all the pixel of the face. The same approach was also applied considering as seed signal the GSR and its phasic and tonic components. Seed correlation analysis on 63 healthy volunteers demonstrated the presence of a common pathway regulating the facial thermal functionality and the electrodermal activity. The procedure was also tested on a pathological case study, finding a completely different pattern compared to the healthy cases. The method represents a promising tool in neurology, physiology and applied neurosciences.

Children with facial paralysis due to Moebius syndrome exhibit reduced autonomic modulation during emotion processing.

BACKGROUND: Facial mimicry is crucial in the recognition of others' emotional state. Thus, the observation of others' facial expressions activates the same neural representation of that affective state in the observer, along with related autonomic and somatic responses. What happens, therefore, when someone cannot mimic others' facial expressions? METHODS: We investigated whether psychophysiological emotional responses to others' facial expressions were impaired in 13 children (9 years) with Moebius syndrome (MBS), an extremely rare neurological disorder (1/250,000 live births) characterized by congenital facial paralysis. We inspected autonomic responses and vagal regulation through facial cutaneous thermal variations and by the computation of respiratory sinus arrhythmia (RSA). These parameters provide measures of emotional arousal and show the autonomic adaptation to others' social cues. Physiological responses in children with MBS were recorded during dynamic facial expression observation and were compared to those of a control group (16 non-affected children, 9 years). RESULTS: There were significant group effects on thermal patterns and RSA, with lower values in children with MBS. We also observed a mild deficit in emotion recognition in these patients. CONCLUSION: Results support "embodied" theory, whereby the congenital inability to produce facial expressions induces alterations in the processing of facial expression of emotions. Such alterations may constitute a risk for emotion dysregulation.

Phenotyping viral infection in sweetpotato using a high-throughput chlorophyll fluorescence and thermal imaging platform.

BACKGROUND: Virus diseases caused by co-infection with Sweet potato feathery mottle virus (SPFMV) and Sweetpotato chlorotic stunt virus (SPCSV) are a severe problem in the production of sweetpotato (Ipomoea batatas L.). Traditional molecular virus detection methods include nucleic acid-based and serological tests. In this study, we aimed to validate the use of a non-destructive imaging-based plant phenotype platform to study plant-virus synergism in sweetpotato by comparing four virus treatments with two healthy controls. RESULTS: By monitoring physiological and morphological effects of viral infection in sweetpotato over 29 days, we quantified photosynthetic performance from chlorophyll fluorescence (ChlF) imaging and leaf thermography from thermal infrared (TIR) imaging among sweetpotatoes. Moreover, the differences among different treatments observed from ChlF and TIR imaging were related to virus accumulation and distribution in sweetpotato. These findings were further validated at the molecular level by related gene expression in both photosynthesis and carbon fixation pathways. CONCLUSION: Our study validated for the first time the use of ChlF- and TIR-based imaging systems to distinguish the severity of virus diseases related to SPFMV and SPCSV in sweetpotato. In addition, we demonstrated that the operating efficiency of PSII and photochemical quenching were the most sensitive parameters for the quantification of virus effects compared with maximum quantum efficiency, non-photochemical quenching, and leaf temperature.

Lateral Sinus Floor Elevation Performed with Trapezoidal and Modified Triangular Flap Designs: A Randomized Pilot Study of Post-Operative Pain Using Thermal Infrared Imaging.

Purpose: Post-operative pain and swelling are frequently observed after sinus lift procedures. The aim of the present study was the clinical evaluation of swelling and pain of two different sinus flap lift techniques using a visual analogue scale (VAS), verbal rating scale (VRS), and infrared thermal imaging (i.e., thermography). Materials Methods: A randomized controlled trial was conducted with 15 patients (30 sinuses in total) randomly allocated into two groups. For the sinuses of Group I a trapezoidal flap was used, while for Group II a modified triangular flap without anterior release was utilized. Postoperative pain was scored by means of a 100-mm VAS ranging from 0 (no pain) to 100 (worst pain imaginable), and was recorded at 2, 4, 6 and 14 days after surgery. Swelling was recorded by a verbal rating scale (VRS) and was classified into four categories: a score of 1 referred the absence of swelling, patients with intra-oral swelling in the surgical zone scored 2, any extra-oral swelling in the surgical zone scored 3, and intense swelling exhibited by extra-oral swelling extending beyond the surgical zone scored 4. The facial temperature was recorded before and after sinus augmentation, and at 2, 4, 6, and 14 days post-surgery to check the course of healing. Results: In Group I pain intensity was recorded at 2 days after surgery with a mean score of 38.67 ± 6.4 mm. Swelling was greater at 2 and 4 days, and was absent at day 6. The facial temperature difference before and after the procedure was 4.737 °C ± 0.37. In Group II the pain score were lower than in Group I (p < 0.05). The score for swelling was 2 on the first and second days, and was reduced on day 4. After the second day the difference in temperature was significantly reduced as compared to the day of surgery (0.77 °C); at 2 and 4 days no difference was registered. Conclusions: The results of this clinical study show the significant effectiveness of the modified triangular flap in the sinus lift procedure for reducing pain and swelling.

"The face of ostracism": The impact of the social categorization on the thermal facial responses of the target and the observer.

Ostracism has been shown to elicit pain in both the target and the observers. Two experiments investigated the autonomic thermal signature associated with an ostracism experience and assessed whether and how social categorization impacts the autonomic arousal of both the target and the observer. Autonomic response was assessed using thermal infrared imaging, recording facial temperature variation during an online game of ball toss (i.e., Cyberball). Social categorization was manipulated using a minimal group paradigm. The results show a more intense autonomic response during ostracism (vs. inclusion), marked by an increase in facial temperature in the nose and the perioral area. This autonomic response is stronger when individuals are ostracized by ingroup (vs. outgroup) members. Similar pattern of temperature variations emerge when individuals observe an ostracism episode involving ingroup members. Our findings advance the understanding of psycho-physiological mechanisms underlying the ostracism experience and emphasize the impact of social categorization in such mechanisms.

Behavioral and facial thermal variations in 3-to 4-month-old infants during the Still-Face Paradigm.

Behavioral and facial thermal responses were recorded in twelve 3- to 4-month-old infants during the Still-Face Paradigm (SFP). As in the usual procedure, infants were observed in a three-step, face-to-face interaction: a normal interaction episode (3 min); the "still-face" episode in which the mother became unresponsive and assumed a neutral expression (1 min); a reunion episode in which the mother resumed the interaction (3 min). A fourth step that consisted of a toy play episode (5 min) was added for our own research interest. We coded the behavioral responses through the Infant and Caregiver Engagement Phases system, and recorded facial skin temperature via thermal infrared (IR) imaging. Comparing still-face episode to play episode, the infants' communicative engagement decreased, their engagement with the environment increased, and no differences emerged in self-regulatory and protest behaviors. We also found that facial skin temperature increased. For the behavioral results, infants recognized the interruption of the interactional reciprocity caused by the still-face presentation, without showing upset behaviors. According to autonomic results, the parasympathetic system was more active than the sympathetic, as usually happens in aroused but not distressed situations. With respect to the debate about the causal factor of the still-face effect, thermal data were consistent with behavioral data in showing this effect as related to the infants' expectations of the nature of the social interactions being violated. Moreover, as these are associated to the infants' subsequent interest in the environment, they indicate the thermal IR imaging as a reliable technique for the detection of physiological variations not only in the emotional system, as indicated by research to date, but also in the attention system. Using this technique for the first time during the SFP allowed us to record autonomic data in a more ecological manner than in previous studies.

Thermal infrared imaging in psychophysiology: potentialities and limits.

Functional infrared thermal imaging (fITI) is considered an upcoming, promising methodology in the emotional arena. Driven by sympathetic nerves, observations of affective nature derive from muscular activity subcutaneous blood flow as well as perspiration patterns in specific body parts. A review of 23 experimental procedures that employed fITI for investigations of affective nature is provided, along with the adopted experimental protocol and the thermal changes that took place on selected regions of interest in human and nonhuman subjects. Discussion is provided regarding the selection of an appropriate baseline, the autonomic nature of the thermal print, the experimental setup, methodological issues, limitations, and considerations, as well as future directions.

Thermal expression of intersubjectivity offers new possibilities to human-machine and technologically mediated interactions.

The evaluation of the psychophysiological state of the interlocutor is an important element of interpersonal relationships and communication. Thermal infrared (IR) imaging has proved to be a reliable tool for non-invasive and contact-less evaluation of vital signs, psychophysiological responses, and emotional states. This technique is quickly spreading in many fields, from psychometrics to social and developmental psychology; and from the touch-less monitoring of vital signs and stress, up to the human-machine interaction. In particular, thermal IR imaging promises to be of use for gathering information about affective states in social situations. This paper presents the state of the art of thermal IR imaging in psychophysiology and in the assessment of affective states. The goal is to provide insights about its potentialities and limits for its use in human-artificial agent interaction in order to contribute to a major issue in the field: the perception by an artificial agent of human psychophysiological and affective states.