Attentional Generative Multimodal Network for Neonatal Postoperative Pain Estimation.

Artificial Intelligence (AI)-based methods allow for automatic assessment of pain intensity based on continuous monitoring and processing of subtle changes in sensory signals, including facial expression, body movements, and crying frequency. Currently, there is a large and growing need for expanding current AI-based approaches to the assessment of postoperative pain in the neonatal intensive care unit (NICU). In contrast to acute procedural pain in the clinic, the NICU has neonates emerging from postoperative sedation, usually intubated, and with variable energy reserves for manifesting forceful pain responses. Here, we present a novel multi-modal approach designed, developed, and validated for assessment of neonatal postoperative pain in the challenging NICU setting. Our approach includes a robust network capable of efficient reconstruction of missing modalities (e.g., obscured facial expression due to intubation) using an unsupervised spatio-temporal feature learning with a generative model for learning the joint features. Our approach generates the final pain score along with the intensity using an attentional cross-modal feature fusion. Using experimental dataset from postoperative neonates in the NICU, our pain assessment approach achieves superior performance (AUC 0.906, accuracy 0.820) as compared to the state-of-the-art approaches.

Multimodal spatio-temporal deep learning approach for neonatal postoperative pain assessment.

The current practice for assessing neonatal postoperative pain relies on bedside caregivers. This practice is subjective, inconsistent, slow, and discontinuous. To develop a reliable medical interpretation, several automated approaches have been proposed to enhance the current practice. These approaches are unimodal and focus mainly on assessing neonatal procedural (acute) pain. As pain is a multimodal emotion that is often expressed through multiple modalities, the multimodal assessment of pain is necessary especially in case of postoperative (acute prolonged) pain. Additionally, spatio-temporal analysis is more stable over time and has been proven to be highly effective at minimizing misclassification errors. In this paper, we present a novel multimodal spatio-temporal approach that integrates visual and vocal signals and uses them for assessing neonatal postoperative pain. We conduct comprehensive experiments to investigate the effectiveness of the proposed approach. We compare the performance of the multimodal and unimodal postoperative pain assessment, and measure the impact of temporal information integration. The experimental results, on a real-world dataset, show that the proposed multimodal spatio-temporal approach achieves the highest AUC (0.87) and accuracy (79%), which are on average 6.67% and 6.33% higher than unimodal approaches. The results also show that the integration of temporal information markedly improves the performance as compared to the non-temporal approach as it captures changes in the pain dynamic. These results demonstrate that the proposed approach can be used as a viable alternative to manual assessment, which would tread a path toward fully automated pain monitoring in clinical settings, point-of-care testing, and homes.

Pattern Recognition in Vital Signs Using Spectrograms.

Spectrograms visualize the frequency components of a given signal which may be an audio signal or even a time-series signal. Audio signals have higher sampling rate and high variability of frequency with time. Spectrograms can capture such variations well. But, vital signs which are time-series signals have less sampling frequency and low-frequency variability due to which, spectrograms fail to express variations and patterns. In this paper, we propose a novel solution to introduce frequency variability using frequency modulation on vital signs. Then we apply spectrograms on frequency modulated signals to capture the patterns. The proposed approach has been evaluated on 4 different medical datasets across both prediction and classification tasks. Significant results are found showing the efficacy of the approach for vital sign signals. The results from the proposed approach are promising with an accuracy of 91.55% and 91.67% in prediction and classification tasks respectively.

Multimodal neonatal procedural and postoperative pain assessment dataset.

This paper presents the first multimodal neonatal pain dataset that contains visual, vocal, and physiological responses following clinically required procedural and postoperative painful procedures. It was collected from 58 neonates (27-41 gestational age) during their hospitalization in the neonatal intensive care unit. The visual and vocal data were recorded using an inexpensive RGB camera while the physiological responses (vital signs and cortical activity) were recorded using portable bedside monitors. The recorded behavioral and physiological responses were scored by expert nurses using two validated pain scales to obtain the ground truth labels. In addition to behavioral and physiological responses, our dataset contains clinical information such as the neonate's age, gender, weight, pharmacological and non-pharmacological interventions, and previous painful procedures. The presented multimodal dataset can be used to develop artificial intelligence systems that monitor, assess, and predict neonatal pain based on the analysis of behavioral and physiological responses. It can also be used to advance the understanding of neonatal pain, which can lead to the development of effective pain prevention and treatment.

Future roles of artificial intelligence in early pain management of newborns.

The advent of increasingly sophisticated medical technology, surgical interventions, and supportive healthcare measures is raising survival probabilities for babies born premature and/or with life-threatening health conditions. In the United States, this trend is associated with greater numbers of neonatal surgeries and higher admission rates into neonatal intensive care units (NICU) for newborns at all birth weights. Following surgery, current pain management in NICU relies primarily on narcotics (opioids) such as morphine and fentanyl (about 100 times more potent than morphine) that lead to a number of complications, including prolonged stays in NICU for opioid withdrawal. In this paper, we review current practices and challenges for pain assessment and treatment in NICU and outline ongoing efforts using Artificial Intelligence (AI) to support pain- and opioid-sparing approaches for newborns in the future. A major focus for these next-generation approaches to NICU-based pain management is proactive pain mitigation (avoidance) aimed at preventing harm to neonates from both postsurgical pain and opioid withdrawal. AI-based frameworks can use single or multiple combinations of continuous objective variables, that is, facial and body movements, crying frequencies, and physiological data (vital signs), to make high-confidence predictions about time-to-pain onset following postsurgical sedation. Such predictions would create a therapeutic window prior to pain onset for mitigation with non-narcotic pharmaceutical and nonpharmaceutical interventions. These emerging AI-based strategies have the potential to minimize or avoid damage to the neonate's body and psyche from postsurgical pain and opioid withdrawal.