Design of Digital-Twin Human-Machine Interface Sensor with Intelligent Finger Gesture Recognition.

In this study, the design of a Digital-twin human-machine interface sensor (DT-HMIS) is proposed. This is a digital-twin sensor (DT-Sensor) that can meet the demands of human-machine automation collaboration in Industry 5.0. The DT-HMIS allows users/patients to add, modify, delete, query, and restore their previously memorized DT finger gesture mapping model and programmable logic controller (PLC) logic program, enabling the operation or access of the programmable controller input-output (I/O) interface and achieving the extended limb collaboration capability of users/patients. The system has two main functions: the first is gesture-encoded virtual manipulation, which indirectly accesses the PLC through the DT mapping model to complete control of electronic peripherals for extension-limbs ability by executing logic control program instructions. The second is gesture-based virtual manipulation to help non-verbal individuals create special verbal sentences through gesture commands to improve their expression ability. The design method uses primitive image processing and eight-way dual-bit signal processing algorithms to capture the movement of human finger gestures and convert them into digital signals. The system service maps control instructions by observing the digital signals of the DT-HMIS and drives motion control through mechatronics integration or speech synthesis feedback to express the operation requirements of inconvenient work or complex handheld physical tools. Based on the human-machine interface sensor of DT computer vision, it can reflect the user's command status without the need for additional wearable devices and promote interaction with the virtual world. When used for patients, the system ensures that the user's virtual control is mapped to physical device control, providing the convenience of independent operation while reducing caregiver fatigue. This study shows that the recognition accuracy can reach 99%, demonstrating practicality and application prospects. In future applications, users/patients can interact virtually with other peripheral devices through the DT-HMIS to meet their own interaction needs and promote industry progress.

Crack Detection on a Retaining Wall with an Innovative, Ensemble Learning Method in a Dynamic Imaging System.

In this study, an innovative, ensemble learning method in a dynamic imaging system of an unmanned vehicle is presented. The feasibility of the system was tested in the crack detection of a retaining wall in a climbing area or a mountain road. The unmanned vehicle can provide a lightweight and remote cruise routine with a Geographic Information System sensor, a Gyro sensor, and a charge-coupled device camera. The crack was the target to be tested, and the retaining wall was patrolled through the drone flight path setting, and then the horizontal image was instantly returned by using the wireless transmission of the system. That is based on the cascade classifier, and the feature comparison classifier was designed further, and then the machine vision correlation algorithm was used to analyze the target type information. First, the system collects the target image and background to establish the samples database, and then uses the Local Binary Patterns feature extraction algorithm to extract the feature values for classification. When the first stage classification is completed, the classification results are target features, and edge feature comparisons. The innovative ensemble learning classifier was used to analyze the image and determine the location of the crack for risk assessment.

Automatic microfluidic fluorescence-array measurement system for detecting organic phosphate.

In this study, an automatic microfluidic fluorescence-array measurement system is developed to detect the concentration of organic phosphate based on the luminol-hydrogen peroxide catalytic fluorescent mechanism. Not only sample quantity and cost can be reduced, but also detection time, accuracy and precision can be improved in the system. The system is composed of a CCD image module, a stepper motor with driver, a microfluidic fluorescence array, a background light elimination module, and a dynamic image-analyzed interface. The pesticides of chlorpyrifos and fenitrothion of organic phosphate are chosen as experimental samples. Only a 2.5 µ l quantity of sample is required to have a fast response time of 1.4 second. Experimental results show that the sensitivities of chlorpyrifos and fenitrothion are 1.88 V/ppm in the range of 0.166 ∼ 10 ppm with averaged error of 1.66% and 0.32 V/ppm in the range of 0.03 ∼ 10 ppm with averaged error of 1.68% respectively. The organophosphorus effective detection range of the developed system covers the legal prescription for pesticide residues.

The designs and applications of a scanning interface with electrical signal detection on the scalp for the severely disabled.

This study discussed a computer-aided program development that meets the requirements of people with physical disabilities. A number of control modes, such as electrode signal recorded on the scalp and blink control, were combined with the scanning human-machine interface to improve the external input/output device. Moreover, a novel and precise algorithm, which filters noise and reduces misrecognition of the system, was proposed. A convenient assistive device can assist people with physical disabilities to meet their requirements for independent living and communication with the outside. The traditional scanning keyboard is changed, and only the phonetic notations are typed instead of characters, thus the time of tone and function selection could be saved, and the typing time could be also reduced. Barrier-free computer assistive devices and interface for people with physical disabilities in typing or speech could allow them to use a scanning keyboard to select phonetic symbols instead of Chinese characters to express their thoughts. The human-machine interface controls can obtain more reliable results as 99.8% connection success rate and 95% typing success rate.

Measurement of surface roughness of thin films by a hybrid interference microscope with different phase algorithms.

We propose a hybrid and flexible interference microscope combined with different phase algorithms to measure the surface roughness of thin films. Two phase measurement algorithms of the fast Fourier transform method and the five-step phase-shifting interferometry are used to evaluate the surface contour of aluminum-doped zinc oxide thin films coated using varying radio-frequency sputtering powers. The experimental results show that the proposed approach is feasible in determining the 3D deformation and surface roughness of thin films.

A novel method for concentration evaluation of reading behaviors with electrical activity recorded on the scalp.

In this paper, a concentration evaluation of reading behaviors with electrical signal detection on the head is presented. The electrode signal is extracted by brain-computer-interface (BCI) to monitor the user's degree of concentration, where the user is reminded by sound to concentrate, or teaching staffs are reminded to help users improve reading habits, in order to facilitate the user's ability to concentrate. The digital signal processing methods, such as the Kalman Filter, Fast Fourier Transform, the Hamming window, the average value of the total energy of a frame, correlation coefficient, and novel judgment algorithm are used to obtain the corresponding parameters of concentration evaluation. Users can correct their manner of reading with reminders. The repeated test results may be expected to lie with a probability of 95%. Such model training results in better learning effect.

Evaluating the image quality of Closed Circuit Television magnification systems versus a head-mounted display for people with low vision. .

In this research, image analysis was used to optimize the visual output of a traditional Closed Circuit Television (CCTV) magnifying system and a head-mounted display (HMD) for people with low vision. There were two purposes: (1) To determine the benefit of using an image analysis system to customize image quality for a person with low vision, and (2) to have people with low vision evaluate a traditional CCTV magnifier and an HMD, each customized to the user's needs and preferences. A CCTV system can electronically alter images by increasing the contrast, brightness, and magnification for the visually disabled when they are reading texts and pictures. The test methods was developed to evaluate and customize a magnification system for persons with low vision. The head-mounted display with CCTV was used to obtain better depth of field and a higher modulation transfer function from the video camera. By sensing the parameters of the environment (e.g., ambient light level, etc.) and collecting the user's specific characteristics, the system could make adjustments according to the user's needs, thus allowing the visually disabled to read more efficiently.

Design of virtual keyboard using blink control method for the severely disabled.

In this paper, a human-machine interface with the concept of "blink control" is proposed. The human-machine interface is applied to an assistive device, namely "blink scanning keyboard", which is designed specifically for the severely physical disabled and people suffering from motor neuron diseases or severe cerebral palsy. The pseudo electromyography (EMG) signal of blinking eyes could be acquired by wearing a Bluetooth headset with one sensor on the forehead and the other three on the left ear of the user. Through a conscious blink, a clear and immediate variation will be formed in the pseudo EMG signal from the users' forehead. The occurrence of this variation in pseudo EMG signal could be detected and filtered by the algorithms proposed in this paper, acting like a trigger to activate the functions integrated in the scanning keyboard. The severely physical and visual disabled then can operate the proposed design by simply blinking their eyes, thus communicating with outside world.

A musical composition assistant for the people with severe physical and visual disabilities.

Current musical composition software on the commercial market is well-functioned for most users. However, there remains no program suitable for people with severe disabilities. In this paper a musical composition assistant is designed for individuals that require assistance using a musical composition tool, with special consideration for the people with physical and visual disabilities. In the single-switch mode a musical composition assistant first initiates a cursor, which moves across the top of each row (referred to as row scanning) for pitch, beat, octave, and halftone selection. To select a note option, the users have to press the mini switch connected to the left key when the cursor is over the row containing the desired entry. In the double-switch mode, the users can control two mini switches connected to the left and right key, with the right key to select and the left key to verify. The additional tools; scanning keyboard and scanning mouse were provided for some users with severe disabilities to write lyrics and share their composition ideas with others. After a 90-day experiment, 10 participants carried out the composition learning task with 110 notes/hr composing speed.

Design and evaluation of a public installation of an eye-gaze system.

This article evaluates an eye-tracking device ("I Can Speak With My Eyes") designed for a museum exhibition. An eye-tracking system is designed not only for able individuals, but also for bedridden patients or physically handicapped people who intend to use computers. With one video CCD camera and frame grabber analyzing a series of images taken of the human pupil gazing at a screen, an auto-range-finding algorithm can be used to obtain the location of the pupil in real time. The computers will produce speech according to the location of where the eyes are gazing. In essence, the participants can speak with their eyes to achieve the advantages of learning and communicating. After the system was implemented, an experimental evaluation with innovation diffusion theory was conducted to survey users' feelings regarding the system. The implications of the evaluation findings are discussed to aid further research and education.