Recovery Gestures Classification Using KNN and LDA Models.

Hand and joint mobility recovery involve performing a set of exercises. Gestures are often used in the hand mobility recovery process. This paper discusses the selection and the use of 2 models of neural networks for the classification of data that describe Leap Motion gestures. The gestures are: the hand opening and closing gesture and the palm rotation gesture. The purpose is the optimal selection of the neural network model to be used in the classification of the data describing the recovery gestures. The models chosen for the classification of the two gestures were: Linear Discriminant Analysis (LDA) and K-neighbors Classifier (KNN). The accuracies achieved in the classification of the gestures for each model are: 0.91 - LDA and 0.98 - KNN.

Comparison of Data Classification Results in Serious Gaming for Rehabilitation of Rheumatoid Arthritis.

Rheumatoid arthritis is a common disease which affects the joints of the wrist, fingers, feet and in the end the daily activities. Nowadays, gestures and virtual reality are used in many activities supporting recovery, games, learning as technology is present more and more in different fields. This paper presents results related to the grip movement detected by a Leap Motion device using binary classification and machine learning algorithms. We used 2 models to compare the results: Naïve Bayes and Random Forest Classifier. The metrics for comparison were: accuracy, precision, recall and f1-score. Also, we create a confusion matrix for a clear visualization of the results. We used 5000 data to train the algorithm and 1500 data to test. The accuracy and the precision were bigger than 97% in all the cases.

Decision Tree Versus Linear Support Vector Machine Classifier in the Screening of Medial Speech Sounds: A Quest for a Sound Rationale.

This paper describes the latest development in the classification stage of our Speech Sound Disorder (SSD) Screening algorithm and presents the results achieved by using two classifier models: the Classification and Regression Tree (CART)-based model versus the Single Decision Hyperplane-based Linear Support Vector Machine (SVM) model. For every single speech sound in medial position, 10 features extracted from the audio samples along with an 11th feature representing the validation of the (mis)pronunciation by the Speech Language Pathologist (SLP) were fed into the 2 classifiers to compare and discuss their performance. The accuracy achieved by the two classifiers on a data test size of 30% of the analyzed samples was 98.2% for the Linear SVM classifier, and 100% for the Decision Tree classifier, which are optimal results that encourage our quest for a sound rationale.

A Custom Image Recognition Model in the Cloud for Skin Diseases Identification.

In the context of global warming and increasing exposure to UV radiation, skin diseases are becoming more prevalent. Some of the most widespread skin conditions are solar lentigo and actinic keratosis. In this paper, we propose a technical approach related to the use of Azure Custom Vision services to classify these two conditions. The main advantage of using this service is the computational power offered by Azure. Additionally, generating a convolutional neural network model does not require a large dataset to achieve a good performance. For training the model, we used a dataset of 600 images from the ISIC database. The limitations of these approaches are imposed by the manual image labeling part that needs to be performed. As a result, we provide a trained model on a series of images that can be used for classifying images related to these two conditions. The performance of our neural network on the pre-trained images is 94%.

Comparison of Data Classification Results for Leap Motion Recovery Gestures.

Static and dynamic gestures are frequently used in activities supporting learning, recovery healthcare, engineering, and 3D games to increase the interactivity between man and machine. The gestures are detected via hardware devices and data is processed using different software methods. This paper presents the manner of detection and interpretation of two gestures, a hand rotation gesture and a palm closing and opening gesture, using the Leap Motion device. These two dynamic gestures are very often used in hand recovery exercises. For comparing the two gestures we use data classification methods, Support Vector Machine (SVM) and Multilayer Perceptron (MLP). The data for the gesture classification were 80% training data and 20% testing data. The metrics for comparison are precision, recall, F1-score, and the total number of testing cases (support). The SVM classifier gives an accuracy of 99.4% and the MLP classifier a 96.2%. We built two confusion matrices for better visualizing the results.

Serious Gaming and Artificial Intelligence in Rehabilitation of Rheumatoid Arthritis.

This paper presents a current situation of studies and applications which are using serious games and artificial intelligence (AI) in rehabilitation of rheumatoid arthritis, and possible future directions. The objectives of this paper are: to highlight the technologies used for recovery of patients with rheumatoid arthritis (RA), to summarize the state of the art of existing applications and to present the authors work, a software application that aims contributing to the recovery of the specific patients. At this stage the application was tested by a group of 10 patients from Medical Centre Sf. Mary of Timisoara. Most of the patients reported that the physical and psychical effort were between easy-very easy. The patients confirmed that they would use the application in their rehabilitation process and are very excited about this type of rehabilitation that stimulates curiosity and a state of wellbeing. The application works based on the leap motion device that proved to be the most suitable device in terms of precision and the manner of interaction in virtual reality games.

Support-Vector Machine-Based Classifier of Cross-Correlated Phoneme Segments for Speech Sound Disorder Screening.

This paper presents a Support-Vector Machine (SVM) based method of classification of cross-correlated phoneme segments as part of the development of an automated Speech Sound Disorder (SSD) Screening tool. The pre-processing stage of the algorithm uses cross-correlation to segment the target phoneme and extracts data from the new homogeneously trimmed audio samples. Such data is then fed into the SVM-based classification script which currently achieves an accuracy of 97.5% on a dataset of 132 rows. Given the global context of an increasing trend in the incidence of Speech Sound Disorders (SSDs) amongst early-school aged children (5-6 years old), the constraints imposed by the new Corona virus pandemic, and the (consequent) shortage of professionally trained specialists, an automated screening tool would be of much assistance to Speech-Language Pathologists (SLPs).

Sharing the IT Educational Experience of Developing 3D Applications for Medical Students Training.

The importance of using the new technologies to develop educational and training applications for medical students is given by the times we live in and also by the continued development of the IT industry. New concepts used in 3D applications such as gamification bring added value to the use of learning applications. The introduction of technologies-based on virtual and augmented reality contributes to increasing of the interactivity. This paper presents an overview of the students' experience in developing a complex 3D application for medical students. The application development methodology is explained by splitting the design and development of the 3D learning application into steps. The main users' benefits provided by such applications are increased interactivity and learning benefits, and the continuous availability of the application wherein virtual laboratories within a medical clinic are simulated.

Serious Gaming and AI Supporting Treatment in Rheumatoid Arthritis.

This paper presents a complex application for rehabilitation of patients with first and second stage rheumatoid arthritis (RA). The application contains a module for the doctor, for the kinetotherapist, and a module as a game matching the symptoms for each stage of RA. The purpose of this application is to achieve the rehabilitation of the RA hand with support of digital technology and multimodal interaction: leap motion, serious gaming, and neuronal networks. The neural network offers support for patients to perform the exercises at home classifying the correct movement with accuracy of 95%. During the development of the application, various challenges were encountered in terms of populating the database, raising the cubes within the game related to second stage of RA, and the implementation of the neural network. The application was tested by a group of students, resulting in the fact that the degree of mental stress, fatigue in the fingers, wrists and physical exertion were insignificant in most cases.

Word-Final Phoneme Segmentation Using Cross-Correlation.

The goal of this paper is to present a word-final target phoneme automated segmentation method based on cross-correlation coefficients computed between a reference sound wave and a sample sound wave. Most existing Speech Sound Disorder (SSD) Screening solutions require human intervention to a greater or lesser extent and use segmentation methods based on hard-coded time frames. Moreover, existing solutions extract features from the frequency domain, which entails large amounts of computational power to the detriment of real-time feedback. The pre-processing algorithm proposed in this paper, implemented in a Python version 3.7 script, automatically generates 2 new .wav files corresponding to the phonemes found in word-final position in the initial sound waves. The newly-generated .wav files are meant to be used as valid and homogeneous input in a subsequent classification stage aimed at rigorously discriminating mispronunciations of the target phoneme and assist Speech-Language Pathologists (SLPs) with the SSD screening.

Evaluating Interactivity in VR Healthcare Applications Using Analytics.

Currently, Virtual Reality-based 3D applications are more and more in use. Education is a field that makes use of this technology providing the users with a pleasant way of learning. Thus, the human-computer/smart phone interactivity increases. To measure the human-computer interactivity in a Virtual Reality-based 3D application, certain metrics may be considered (e.g.: application usage time, speed of execution of certain tasks). In this paper we use Google Analytics to observe and measure the interactivity in a medical application created for educational purposes. The application is based on Virtual Reality, and the user may observe and handle the bones of the human skeleton. The main actions monitored via Google Analytics to observe interactivity are: skeleton parts selection, zooming in/zooming out, rotation. An 80% usage percentage (measure of interaction) was achieved for the skeleton selection (foot bones), while for the other actions and selections the achieved percentage was of 100%.

Cross-Correlation Based Automated Segmentation of Audio Samples.

This paper presents an audio file segmentation method in an attempt to mitigate the issue of variable durations of the same utterance by different individuals, e.g.: Speech-Language Pathologist (SLP) and dyslalic subjects. The Method section describes the manner of determination of the maximum cross-correlation value between the 2 audio files and the subsequent automated segmentation thereof in order to extract 2 valid pronunciation samples of the target consonant. The method is aimed at pre-processing audio files and supplying homogeneously-trimmed audio samples to a computerized SSD Screening system. The results obtained on a batch of 30 pronunciations are presented and briefly discussed in the third section while the last section is reserved for conclusions and perspectives.

Gesture Classification for a Hand Controller Device Using Neural Networks.

This paper presents a convolutional neural network-based classification of the hand flexion and extension gestures used in wrist recovery after injury. The hand gesture recognition device used in our study is the Leap Motion controller. The Leap Motion device's inability to accurately differentiate the left hand from the right hand when performing hand rotation gestures was eliminated by introducing hand and thumb direction vectors into the database used to train the neural network. A 3D environment was created for the introduction of the data describing the gesture into the database. A classification accuracy of 95% was achieved for the hand flexion and extension gesture divided into three levels for each hand. The populated database may also be used to classify other gestures involving hand rotation.

Enhancing Precision in Gesture Detection for Hand Recovery After Injury Using Leap Motion and Machine Learning.

This paper presents an improved solution for detecting gestures with a better precision using the Leap Motion sensor and Machine Learning support. A neural network is trained to recognize a hand rotation gesture expressing the grade of recovery, with a supination and pronation exercise. The supination-pronation movement is divided into 4 levels because the users are not usually able to perform a complete rotation gesture in hand recovery after injury. The neural network is trained with data representing the hand rotation angle measurements on the x, y and z axes. The Neural Network training is based on the Tensorflow library. 3 tests were carried out to test the network and eventually a 96% gesture-detection accuracy was achieved.

Mixed Reality Supporting Modern Medical Education.

The paper presents new learning support for medical students exemplifying with several 3D applications for training on specific topics in medicine and investigates the impact on medical students. The applications were built using new concepts: Virtual Reality, Augmented Reality, as environments agreed by young people, and gamification to make learning easy and fun. Leap Motion and the VR headset are the devices to control the applications and provide a better human-computer/mobile phone interaction as compared to the current ones. The concepts and the new technologies to display/visualize the applications are the core of the Mixed Reality concept resulting from combining the 4 applications implemented for medical education.

A Computer-Based Speech Sound Disorder Screening System Architecture.

This paper reviews several architectures of Computer-Based Speech Therapy (CBST) systems and solutions and describes an architecture for an Entropy-Based Sound Speech Disorder (SSD) Screening System aimed at by our research project. The proposed architecture and data flow scenario aim to provide a fully-automated Entropy-based SSD Screening System, to be connected with CBSTs and to be used as a research infrastructure for further refinement of the objectives of our research project.

Hand Rehabilitation Using a 3D Environment and Leap Motion Device.

The paper presents a 3D healthcare informatics support for Hand Rehabilitation after injury. As a novelty, the application uses the Leap Motion sensor for patient's gestures recognition, and videos to illustrate to the users the hand exercise to perform. The implemented application provides feedback to users regarding the correctness of the performed recovery gestures/exercises. The data related to hand rehabilitation is saved in a database and offers to the Medical Rehabilitation Experts the possibility to monitor the patients in a more consistent manner. To assess the efficiency and accuracy of the application the application will be tested following a usability plan.

DNA Encoding Training Using 3D Gesture Interaction.

The work described in this paper summarizes the development process and presents the results of a human genetics training application, studying the 20 amino acids formed by the combination of the 3 nucleotides of DNA targeting mainly medical and bioinformatics students. Currently, the domain applications using recognized human gestures of the Leap Motion sensor are used in molecules controlling and learning from Mendeleev table or in visualizing the animated reactions of specific molecules with water. The novelty in the current application consists in using the Leap Motion sensor creating new gestures for the application control and creating a tag based algorithm corresponding to each amino acid, depending on the position in the 3D virtual space of the 4 nucleotides of DNA and their type. The team proposes a 3D application based on Unity editor and on Leap Motion sensor where the user has the liberty of forming different combinations of the 20 amino acids. The results confirm that this new type of study of medicine/biochemistry using the Leap Motion sensor for handling amino acids is suitable for students. The application is original and interactive and the users can create their own amino acid structures in a 3D-like environment which they could not do otherwise using traditional pen-and-paper.

VR Medical Gamification for Training and Education.

The new virtual reality based medical applications is providing a better understanding of healthcare related subjects for both medical students and physicians. The work presented in this paper underlines gamification as a concept and uses VR as a new modality to study the human skeleton. The team proposes a mobile Android platform application based on Unity 5.4 editor and Google VR SDK. The results confirmed that the approach provides a more intuitive user experience during the learning process, concluding that the gamification of classical medical software provides an increased interactivity level for medical students during the study of the human skeleton.