A Systematic Evaluation of Feature Encoding Techniques for Gait Analysis Using Multimodal Sensory Data.

This paper addresses the problem of feature encoding for gait analysis using multimodal time series sensory data. In recent years, the dramatic increase in the use of numerous sensors, e.g., inertial measurement unit (IMU), in our daily wearable devices has gained the interest of the research community to collect kinematic and kinetic data to analyze the gait. The most crucial step for gait analysis is to find the set of appropriate features from continuous time series data to accurately represent human locomotion. This paper presents a systematic assessment of numerous feature extraction techniques. In particular, three different feature encoding techniques are presented to encode multimodal time series sensory data. In the first technique, we utilized eighteen different handcrafted features which are extracted directly from the raw sensory data. The second technique follows the Bag-of-Visual-Words model; the raw sensory data are encoded using a pre-computed codebook and a locality-constrained linear encoding (LLC)-based feature encoding technique. We evaluated two different machine learning algorithms to assess the effectiveness of the proposed features in the encoding of raw sensory data. In the third feature encoding technique, we proposed two end-to-end deep learning models to automatically extract the features from raw sensory data. A thorough experimental evaluation is conducted on four large sensory datasets and their outcomes are compared. A comparison of the recognition results with current state-of-the-art methods demonstrates the computational efficiency and high efficacy of the proposed feature encoding method. The robustness of the proposed feature encoding technique is also evaluated to recognize human daily activities. Additionally, this paper also presents a new dataset consisting of the gait patterns of 42 individuals, gathered using IMU sensors.

Review of Learning-Based Robotic Manipulation in Cluttered Environments.

Robotic manipulation refers to how robots intelligently interact with the objects in their surroundings, such as grasping and carrying an object from one place to another. Dexterous manipulating skills enable robots to assist humans in accomplishing various tasks that might be too dangerous or difficult to do. This requires robots to intelligently plan and control the actions of their hands and arms. Object manipulation is a vital skill in several robotic tasks. However, it poses a challenge to robotics. The motivation behind this review paper is to review and analyze the most relevant studies on learning-based object manipulation in clutter. Unlike other reviews, this review paper provides valuable insights into the manipulation of objects using deep reinforcement learning (deep RL) in dense clutter. Various studies are examined by surveying existing literature and investigating various aspects, namely, the intended applications, the techniques applied, the challenges faced by researchers, and the recommendations adopted to overcome these obstacles. In this review, we divide deep RL-based robotic manipulation tasks in cluttered environments into three categories, namely, object removal, assembly and rearrangement, and object retrieval and singulation tasks. We then discuss the challenges and potential prospects of object manipulation in clutter. The findings of this review are intended to assist in establishing important guidelines and directions for academics and researchers in the future.

A Review of Emotion Recognition Methods Based on Data Acquired via Smartphone Sensors.

In recent years, emotion recognition algorithms have achieved high efficiency, allowing the development of various affective and affect-aware applications. This advancement has taken place mainly in the environment of personal computers offering the appropriate hardware and sufficient power to process complex data from video, audio, and other channels. However, the increase in computing and communication capabilities of smartphones, the variety of their built-in sensors, as well as the availability of cloud computing services have made them an environment in which the task of recognising emotions can be performed at least as effectively. This is possible and particularly important due to the fact that smartphones and other mobile devices have become the main computer devices used by most people. This article provides a systematic overview of publications from the last 10 years related to emotion recognition methods using smartphone sensors. The characteristics of the most important sensors in this respect are presented, and the methods applied to extract informative features on the basis of data read from these input channels. Then, various machine learning approaches implemented to recognise emotional states are described.

Ping-Pong Free Advanced and Energy Efficient Sensor Relocation for IoT-Sensory Network.

With the growing interest in big data technology, mobile IoT devices play an essential role in data collection. Generally, IoT sensor nodes are randomly distributed to areas where data cannot be easily collected. Subsequently, when data collection is impossible (i.e., sensing holes occurrence situation) due to improper placement of sensors or energy exhaustion of sensors, the sensors should be relocated. The cluster header in the sensing hole sends requests to neighboring cluster headers for the sensors to be relocated. However, it can be possible that sensors in the specific cluster zones near the sensing hole are continuously requested to move. With this knowledge, there can be a ping-pong problem, where the cluster headers in the neighboring sensing holes repeatedly request the movement of the sensors in the counterpart sensing hole. In this paper, we first proposed the near-uniform selection and movement scheme of the sensors to be relocated. By this scheme, the energy consumption of the sensors can be equalized, and the sensing capability can be extended. Thus the network lifetime can be extended. Next, the proposed relocation protocol resolves a ping-pong problem using queues with request scheduling. Another crucial contribution of this paper is that performance was analyzed using the fully-customed OMNeT++ simulator to reflect actual environmental conditions, not under over-simplified artificial network conditions. The proposed relocation protocol demonstrates a uniform and energy-efficient movement with ping-pong free capability.

Equipment Anomaly Detection for Semiconductor Manufacturing by Exploiting Unsupervised Learning from Sensory Data.

In-line anomaly detection (AD) not only identifies the needs for semiconductor equipment maintenance but also indicates potential line yield problems. Prompt AD based on available equipment sensory data (ESD) facilitates proactive yield and operations management. However, ESD items are highly diversified and drastically scale up along with the increased use of sensors. Even veteran engineers lack knowledge about ESD items for automated AD. This paper presents a novel Spectral and Time Autoencoder Learning for Anomaly Detection (STALAD) framework. The design consists of four innovations: (1) identification of cycle series and spectral transformation (CSST) from ESD, (2) unsupervised learning from CSST of ESD by exploiting Stacked AutoEncoders, (3) hypothesis test for AD based on the difference between the learned normal data and the tested sample data, (4) dynamic procedure control enabling periodic and parallel learning and testing. Applications to ESD of an HDP-CVD tool demonstrate that STALAD learns normality without engineers' prior knowledge, is tolerant to some abnormal data in training input, performs correct AD, and is efficient and adaptive for fab applications. Complementary to the current practice of using control wafer monitoring for AD, STALAD may facilitate early detection of equipment anomaly and assessment of impacts to process quality.

Towards Efficient Data Collection in Space-Based Internet of Things.

Due to the strong anti-destructive ability, global coverage, and independent infrastructure of the space-based Internet of Things (S-IoT), it is one of the most important ways to achieve a real interconnection of all things. In S-IoT, a single satellite can often achieve thousands of kilometers of coverage and needs to provide data transmission services for massive ground nodes. However, satellite bandwidth is usually low and the uplink and downlink bandwidth is extremely asymmetric. Therefore, exact data collection is not affordable for S-IoT. In this paper, an approximate data collection algorithm is proposed for S-IoT; that is, the sampling-reconstruction (SR) algorithm. Since the uplink bandwidth is very limited, the SR algorithm samples only the sensory data of some nodes and then reconstructs the unacquired data based on the spatiotemporal correlation between the sensory data. In order to obtain higher data collection precision under a certain data collection ratio, the SR algorithm optimizes the sampling node selection by leveraging the curvature characteristics of the sensory data in time and space dimensions. Moreover, the SR algorithm innovatively applies spatiotemporal compressive sensing (ST-CS) technology to accurately reconstruct unacquired sensory data by making full use of the spatiotemporal correlation between the sensory data. We used a real-weather data set to evaluate the performance of the SR algorithm and compared it with two existing representative approximate data collection algorithms. The experimental results show that the SR algorithm is well-suited for S-IoT and can achieve efficient data collection under the condition that the uplink bandwidth is extremely limited.

Approximate Sensory Data Collection: A Survey.

With the rapid development of the Internet of Things (IoTs), wireless sensor networks (WSNs) and related techniques, the amount of sensory data manifests an explosive growth. In some applications of IoTs and WSNs, the size of sensory data has already exceeded several petabytes annually, which brings too many troubles and challenges for the data collection, which is a primary operation in IoTs and WSNs. Since the exact data collection is not affordable for many WSN and IoT systems due to the limitations on bandwidth and energy, many approximate data collection algorithms have been proposed in the last decade. This survey reviews the state of the art of approximatedatacollectionalgorithms. Weclassifythemintothreecategories: themodel-basedones, the compressive sensing based ones, and the query-driven ones. For each category of algorithms, the advantages and disadvantages are elaborated, some challenges and unsolved problems are pointed out, and the research prospects are forecasted.

On the Prediction of Flickr Image Popularity by Analyzing Heterogeneous Social Sensory Data.

The increase in the popularity of social media has shattered the gap between the physical and virtual worlds. The content generated by people or social sensors on social media provides information about users and their living surroundings, which allows us to access a user's preferences, opinions, and interactions. This provides an opportunity for us to understand human behavior and enhance the services provided for both the real and virtual worlds. In this paper, we will focus on the popularity prediction of social images on Flickr, a popular social photo-sharing site, and promote the research on utilizing social sensory data in the context of assisting people to improve their life on the Web. Social data are different from the data collected from physical sensors; in the fact that they exhibit special characteristics that pose new challenges. In addition to their huge quantity, social data are noisy, unstructured, and heterogeneous. Moreover, they involve human semantics and contextual data that require analysis and interpretation based on human behavior. Accordingly, we address the problem of popularity prediction for an image by exploiting three main factors that are important for making an image popular. In particular, we investigate the impact of the image's visual content, where the semantic and sentiment information extracted from the image show an impact on its popularity, as well as the textual information associated with the image, which has a fundamental role in boosting the visibility of the image in the keyword search results. Additionally, we explore social context, such as an image owner's popularity and how it positively influences the image popularity. With a comprehensive study on the effect of the three aspects, we further propose to jointly consider the heterogeneous social sensory data. Experimental results obtained from real-world data demonstrate that the three factors utilized complement each other in obtaining promising results in the prediction of image popularity on social photo-sharing site.

On Curating Multimodal Sensory Data for Health and Wellness Platforms.

In recent years, the focus of healthcare and wellness technologies has shown a significant shift towards personal vital signs devices. The technology has evolved from smartphone-based wellness applications to fitness bands and smartwatches. The novelty of these devices is the accumulation of activity data as their users go about their daily life routine. However, these implementations are device specific and lack the ability to incorporate multimodal data sources. Data accumulated in their usage does not offer rich contextual information that is adequate for providing a holistic view of a user's lifelog. As a result, making decisions and generating recommendations based on this data are single dimensional. In this paper, we present our Data Curation Framework (DCF) which is device independent and accumulates a user's sensory data from multimodal data sources in real time. DCF curates the context of this accumulated data over the user's lifelog. DCF provides rule-based anomaly detection over this context-rich lifelog in real time. To provide computation and persistence over the large volume of sensory data, DCF utilizes the distributed and ubiquitous environment of the cloud platform. DCF has been evaluated for its performance, correctness, ability to detect complex anomalies, and management support for a large volume of sensory data.

Multi-modal remote perception learning for object sensory data.

Introduction: When it comes to interpreting visual input, intelligent systems make use of contextual scene learning, which significantly improves both resilience and context awareness. The management of enormous amounts of data is a driving force behind the growing interest in computational frameworks, particularly in the context of autonomous cars. Method: The purpose of this study is to introduce a novel approach known as Deep Fused Networks (DFN), which improves contextual scene comprehension by merging multi-object detection and semantic analysis. Results: To enhance accuracy and comprehension in complex situations, DFN makes use of a combination of deep learning and fusion techniques. With a minimum gain of 6.4% in accuracy for the SUN-RGB-D dataset and 3.6% for the NYU-Dv2 dataset. Discussion: Findings demonstrate considerable enhancements in object detection and semantic analysis when compared to the methodologies that are currently being utilized.

Future of ADHD Care: Evaluating the Efficacy of ChatGPT in Therapy Enhancement.

This study explores the integration of large language models (LLMs), like ChatGPT, to improve attention deficit hyperactivity disorder (ADHD) treatments. Utilizing the Delphi method for its systematic forecasting capabilities, we gathered a panel of child ADHD therapy experts. These experts interacted with our custom ChatGPT through a specialized interface, thus engaging in simulated therapy scenarios with behavioral prompts and commands. Using empirical tests and expert feedback, we aimed to rigorously evaluate ChatGPT's effectiveness in therapy settings to integrate AI into healthcare responsibly. We sought to ensure that AI contributes positively and ethically to therapy and patient care, thus filling a gap in ADHD treatment methods. Findings show ChatGPT's empathy, adaptability, and communication strengths, thereby highlighting its potential to significantly improve ADHD care. The study points to ChatGPT's capacity to transform therapy practices through personalized and responsive patient care. However, it also notes the need for enhancements in privacy, cultural sensitivity, and interpreting nonverbal cues for ChatGPT's effective healthcare integration. Our research advocates for merging technological innovation with a comprehensive understanding of patient needs and ethical considerations, thereby aiming to pioneer a new era of AI-assisted therapy. We emphasize the ongoing refinement of AI tools like ChatGPT to meet ADHD therapy and patient care requirements more effectively.

A Guide to Best Practice in Sensory Analysis of Pharmaceutical Formulations.

It is well established that treatment regime compliance is linked to the acceptability of a pharmaceutical formulation, and hence also to therapeutic outcomes. To that end, acceptability must be assessed during the development of all pharmaceutical products and especially for those intended for paediatric patients. Although acceptability is a multifaceted concept, poor sensory characteristics often contribute to poor patient acceptability. In particular, poor taste is often cited as a major reason for many patients, especially children, to refuse to take their medicine. It is thus important to understand and, as far as possible, optimise the sensory characteristics and, in particular, the taste/flavour/mouthfeel of the formulation throughout the development of the product. Sensory analysis has been widely practiced, providing objective data concerning the sensory aspects of food and cosmetic products. In this paper, we present proposals concerning how the well-established principles of sensory analysis can best be applied to pharmaceutical product development, allowing objective, scientifically valid, sensory data to be obtained safely. We briefly discuss methodologies that may be helpful in reducing the number of samples that may need to be assessed by human volunteers. However, it is only possible to be sure whether or not the sensory characteristics of a pharmaceutical product are non-aversive to potential users by undertaking sensory assessments in human volunteers. Testing is also required during formulation assessment and to ensure that the sensory characteristics remain acceptable throughout the product shelf life. We provide a risk assessment procedure to aid developers to define where studies are low risk, the results of a survey of European regulators on their views concerning such studies, and detailed guidance concerning the types of sensory studies that can be undertaken at each phase of product development, along with guidance about the practicalities of performing such sensory studies. We hope that this guidance will also lead to the development of internationally agreed standards between industry and regulators concerning how these aspects should be measured and assessed throughout the development process and when writing and evaluating regulatory submissions. Finally, we hope that the guidance herein will help formulators as they seek to develop better medicines for all patients and, in particular, paediatric patients.

An Overview of Sensory Characterization Techniques: From Classical Descriptive Analysis to the Emergence of Novel Profiling Methods.

Sensory science provides objective information about the consumer understanding of a product, the acceptance or rejection of stimuli, and the description of the emotions evoked. It is possible to answer how consumers perceive a product through discriminative and descriptive techniques. However, perception can change over time, and these fluctuations can be measured with time-intensity methods. Instrumental sensory devices and immersive techniques are gaining headway as sensory profiling techniques. The authors of this paper critically review sensory techniques from classical descriptive analysis to the emergence of novel profiling methods. Though research has been done in the creation of new sensory methods and comparison of those methods, little attention has been given to the timeline approach and its advantages and challenges. This study aimed to gather, explain, simplify, and discuss the evolution of sensory techniques.

Chromatographic Fingerprinting Strategy to Delineate Chemical Patterns Correlated to Coffee Odor and Taste Attributes.

Coffee cupping includes both aroma and taste, and its evaluation considers several different attributes simultaneously to define flavor quality and therefore requires complementary data from aroma and taste. This study investigates the potential and limits of a data-driven approach to describe the sensory quality of coffee using complementary analytical techniques usually available in routine quality control laboratories. Coffee flavor chemical data from 155 samples were obtained by analyzing volatile (headspace-solid-phase microextraction-gas chromatography-mass spectrometry (HS-SPME-GC-MS)) and nonvolatile (liquid chromatography-ultraviolet/diode array detector (LC-UV/DAD)) fractions, as well as from sensory data. Chemometric tools were used to explore the data sets, select relevant features, predict sensory scores, and investigate the networks between features. A comparison of the Q model parameter and root-mean-squared error prediction (RMSEP) highlights the variable influence that the nonvolatile fraction has on prediction, showing that it has a higher impact on describing acid, bitter, and woody notes than on flowery and fruity. The data fusion emphasized the aroma contribution to driving sensory perceptions, although the correlative networks highlighted from the volatile and nonvolatile data deserve a thorough investigation to verify the potential of odor-taste integration.

Optimization of Machine Learning in Various Situations Using ICT-Based TVOC Sensors.

A computational framework using artificial intelligence (AI) has been suggested in numerous fields, such as medicine, robotics, meteorology, and chemistry. The specificity of each AI model and the relationship between data characteristics and ground truth, allowing their guidance according to each situation, has not been given. Since TVOCs (total volatile organic compounds) cause serious harm to human health and plants, the prevention of such damages with a reduction in their occurrence frequency becomes not an optional process but an essential one in manufacturing, as well as for chemical industries and laboratories. In this study, with consideration of the characteristics of the machine learning technique and ICT (information and communications technology), TVOC sensors are explored as a function of grounded data analysis and the selection of machine learning models, determining their performance in real situations. For representative scenarios, considering features from an ICT semiconductor sensor and one targeting TVOC gas, we investigated suitable analysis methods and machine learning models such as LSTM (long short-term memory), GRU (gated recurrent unit), and RNN (recurrent neural network). Detailed factors for these machine learning models with respect to the concentration of TVOC gas in the atmosphere are compared with original sensory data to obtain their accuracy. From this work, we expect to significantly minimize risk in empirical applications, i.e., maintaining homeostasis or predicting abnormal situations to construct an opportune response.

Tracking and Predicting Depressive Symptoms of Adolescents Using Smartphone-Based Self-Reports, Parental Evaluations, and Passive Phone Sensor Data: Development and Usability Study.

BACKGROUND: Depression carries significant financial, medical, and emotional burden on modern society. Various proof-of-concept studies have highlighted how apps can link dynamic mental health status changes to fluctuations in smartphone usage in adult patients with major depressive disorder (MDD). However, the use of such apps to monitor adolescents remains a challenge. OBJECTIVE: This study aimed to investigate whether smartphone apps are useful in evaluating and monitoring depression symptoms in a clinically depressed adolescent population compared with the following gold-standard clinical psychometric instruments: Patient Health Questionnaire (PHQ-9), Hamilton Rating Scale for Depression (HAM-D), and Hamilton Anxiety Rating Scale (HAM-A). METHODS: We recruited 13 families with adolescent patients diagnosed with MDD with or without comorbid anxiety disorder. Over an 8-week period, daily self-reported moods and smartphone sensor data were collected by using the Smartphone- and OnLine usage-based eValuation for Depression (SOLVD) app. The evaluations from teens' parents were also collected. Baseline depression and anxiety symptoms were measured biweekly using PHQ-9, HAM-D, and HAM-A. RESULTS: We observed a significant correlation between the self-evaluated mood averaged over a 2-week period and the biweekly psychometric scores from PHQ-9, HAM-D, and HAM-A (0.45≤|r|≤0.63; P=.009, P=.01, and P=.003, respectively). The daily steps taken, SMS frequency, and average call duration were also highly correlated with clinical scores (0.44≤|r|≤0.72; all P<.05). By combining self-evaluations and smartphone sensor data of the teens, we could predict the PHQ-9 score with an accuracy of 88% (23.77/27). When adding the evaluations from the teens' parents, the prediction accuracy was further increased to 90% (24.35/27). CONCLUSIONS: Smartphone apps such as SOLVD represent a useful way to monitor depressive symptoms in clinically depressed adolescents, and these apps correlate well with current gold-standard psychometric instruments. This is a first study of its kind that was conducted on the adolescent population, and it included inputs from both teens and their parents as observers. The results are preliminary because of the small sample size, and we plan to expand the study to a larger population.