Online estimating weight of white Pekin duck carcass by computer vision.

The increasing consumption of ducks and chickens in China demands characterizing carcasses of domestic birds efficiently. Most existing methods, however, were developed for characterizing carcasses of pigs or cattle. Here, we developed a noncontact and automated weighing method for duck carcasses hanging on a production line. A 2D camera with its facilitating parts recorded the moving duck carcasses on the production line. To estimate the weight of carcasses, the images in the acquired dataset were modeled by a convolution neuron network (CNN). This model was trained and evaluated using 10-fold cross-validation. The model estimated the weight of duck carcasses precisely with a mean abstract deviation (MAD) of 58.8 grams and a mean relative error (MRE) of 2.15% in the testing dataset. Compared with 2 widely used methods, pixel area linear regression and the artificial neural network (ANN) model, our model decreases the estimation error MAD by 64.7 grams (52.4%) and 48.2 grams (45.0%). We release the dataset and code at https://github.com/RuoyuChen10/Image_weighing.

Glycated Hemoglobin Electrochemical Immunosensor Based on Screen-Printed Electrode.

An electrochemical HbA1c sensor with high sensitivity and good specificity is proposed based on the electrochemical immune principle. The reproducibility and conductivity of the electrode are improved by depositing gold nanoparticles (AuNPs) on the surface of the screen-printed electrode (SPE). The HbA1c antibodies are immobilized on the surface of the modified electrode by adsorption to capture the HbA1c in the sample. The hindering effect of HbA1c on the electrode transfer reaction was exploited as the HbA1c detection mechanism. The electrode's properties were characterized by electrochemical impedance spectroscopy (EIS), and the measurement properties of the electrode were analyzed using differential pulse voltammetry (DPV) and cyclic voltammetry (CV). The experimental results show that the peak current signal of the electrochemical immunosensor produced a linear response to HbA1c in the concentration range of 20-200 µg/mL, a linear relationship coefficient of 0.9812, a detection limit of 15.5 µg/mL, and a sensitivity of 0.0938 µA/µg·mL-1. The sensor delivered satisfactory repeatability, stability, and anti-interference performance. Due to its small size, high sensitivity, and wide linear detection range, it is expected to play a significant role in managing diabetes at home.

Edge-to-Cloud IIoT for Condition Monitoring in Manufacturing Systems with Ubiquitous Smart Sensors.

The Industrial Internet of Things (IIoT) connects industrial assets to ubiquitous smart sensors and actuators to enhance manufacturing and industrial processes. Data-driven condition monitoring is an essential technology for intelligent manufacturing systems to identify anomalies from malfunctioning equipment, prevent unplanned downtime, and reduce the operation costs by predictive maintenance without interrupting normal machine operations. However, data-driven condition monitoring requires massive data collected from smart sensors to be transmitted to the cloud for further processing, thereby contributing to network congestion and affecting the network performance. Furthermore, unbalanced training data with very few labelled anomalies limit supervised learning models because of the lack of sufficient fault data for the training process in anomaly detection algorithms. To address these issues, we proposed an IIoT-based condition monitoring system with an edge-to-cloud architecture and computed the relative wavelet energy as feature vectors on the edge layer to reduce the network traffic overhead. We also proposed an unsupervised deep long short-term memory (LSTM) network module for anomaly detection. We implemented the proposed IIoT condition monitoring system for a manufacturing machine in a real shop site to evaluate our proposed solution. Our experimental results verify the effectiveness of our approach which can not only reduce the network traffic overhead for the IIoT but also detect anomalies accurately.

Wide-Bandwidth Nanocomposite-Sensor Integrated Smart Mask for Tracking Multiphase Respiratory Activities.

Wearing masks has been a recommended protective measure due to the risks of coronavirus disease 2019 (COVID-19) even in its coming endemic phase. Therefore, deploying a "smart mask" to monitor human physiological signals is highly beneficial for personal and public health. This work presents a smart mask integrating an ultrathin nanocomposite sponge structure-based soundwave sensor (≈400 µm), which allows the high sensitivity in a wide-bandwidth dynamic pressure range, i.e., capable of detecting various respiratory sounds of breathing, speaking, and coughing. Thirty-one subjects test the smart mask in recording their respiratory activities. Machine/deep learning methods, i.e., support vector machine and convolutional neural networks, are used to recognize these activities, which show average macro-recalls of ≈95% in both individual and generalized models. With rich high-frequency (≈4000 Hz) information recorded, the two-/tri-phase coughs can be mapped while speaking words can be identified, demonstrating that the smart mask can be applicable as a daily wearable Internet of Things (IoT) device for respiratory disease identification, voice interaction tool, etc. in the future. This work bridges the technological gap between ultra-lightweight but high-frequency response sensor material fabrication, signal transduction and processing, and machining/deep learning to demonstrate a wearable device for potential applications in continual health monitoring in daily life.

Development of a Wearable Glove System with Multiple Sensors for Hand Kinematics Assessment.

In traditional hand function assessment, patients and physicians always need to accomplish complex activities and rating tasks. This paper proposes a novel wearable glove system for hand function assessment. A sensing system consisting of 12 nine-axis inertial and magnetic unit (IMMU) sensors is used to obtain the acceleration, angular velocity, and geomagnetic orientation of human hand movements. A complementary filter algorithm is applied to calculate the angles of joints after sensor calibration. A virtual hand model is also developed to map with the glove system in the Unity platform. The experimental results show that this glove system can capture and reproduce human hand motions with high accuracy. This smart glove system is expected to reduce the complexity and time consumption of hand kinematics assessment.

Optimal strategy of sEMG feature and measurement position for grasp force estimation.

Grasp force estimation based on surface electromyography (sEMG) is essential for the dexterous control of a prosthetic hand. Nowadays, although increasing the number of sEMG measurement positions and extracting more features are common methods to increase the accuracy of grasp force estimation, it will increase the computational burden. In this paper, an approach based on analysis of variance (ANOVA) and generalized regression neural network (GRNN) for optimal measurement positions and features is proposed, with the purpose of using fewer measurement positions or features to achieve higher estimation accuracy. Firstly, we captured six channels of sEMG from subjects' forearm and grasp force synchronously. Then, four kinds of features in time domain are extracted from each channel of sEMG. By combining different measurement position sets (MPSs) and feature set (FSs), we construct 945 data sets. These data sets are fed to GRNN to realize grasp force estimation. Normalized root mean square error (NRMS), normalized mean of absolute error (NMAE), and correlation coefficient (CC) between estimated grasp force and actual force are introduced to evaluate the performance of grasp force estimation. Finally, ANOVA and Tukey HSD testing are introduced to analyze grasp force estimation results so as to obtain the optimal measurement positions and features. We obtain the optimal MPSs for grasp force estimation when different FSs are employed, and the optimal FSs when different MPSs are utilized.

In situ printing of liquid superlenses for subdiffraction-limited color imaging of nanobiostructures in nature.

The nanostructures and patterns that exist in nature have inspired researchers to develop revolutionary components for use in modern technologies and our daily lives. The nanoscale imaging of biological samples with sophisticated analytical tools, such as scanning electron microscopy (SEM) and transmission electron microscopy (TEM), has afforded a precise understanding of structures and has helped reveal the mechanisms contributing to the behaviors of the samples but has done so with the loss of photonic properties. Here, we present a new method for printing biocompatible "superlenses" directly on biological objects to observe subdiffraction-limited features under an optical microscope in color. We demonstrate the nanoscale imaging of butterfly wing scales with a super-resolution and larger field-of-view (FOV) than those of previous dielectric microsphere techniques. Our approach creates a fast and flexible path for the direct color observation of nanoscale biological features in the visible range and enables potential optical measurements at the subdiffraction-limited scale.

Model Order Identification for Cable Force Estimation Using a Markov Chain Monte Carlo-Based Bayesian Approach.

The tensile force on the hanger cables of a suspension bridge is an important indicator of the structural health of the bridge. Tensile force estimation methods based on the measured frequency of the hanger cable have been widely used. These methods empirically pre-determinate the corresponding model order of the measured frequency. However, because of the uncertain flexural rigidity, this empirical order determination method not only plays a limited role in high-order frequencies, but also hinders the online cable force estimation. Therefore, we propose a new method to automatically identify the corresponding model order of the measured frequency, which is based on a Markov chain Monte Carlo (MCMC)-based Bayesian approach. It solves the limitation of empirical determination in the case of large flexural rigidity. The tensile force and the flexural rigidity of cables can be calculated simultaneously using the proposed method. The feasibility of the proposed method is validated via a numerical study involving a finite element model that considers the flexural rigidity and via field application to a suspension bridge.

Determination of the Three-Dimensional Rate of Cancer Cell Rotation in an Optically-Induced Electrokinetics Chip Using an Optical Flow Algorithm.

Our group has reported that Melan-A cells and lymphocytes undergo self-rotation in a homogeneous AC electric field, and found that the rotation velocity of these cells is a key indicator to characterize their physical properties. However, the determination of the rotation properties of a cell by human eyes is both gruesome and time consuming, and not always accurate. In this paper, a method is presented to more accurately determine the 3D cell rotation velocity and axis from a 2D image sequence captured by a single camera. Using the optical flow method, we obtained the 2D motion field data from the image sequence and back-project it onto a 3D sphere model, and then the rotation axis and velocity of the cell were calculated. After testing the algorithm on animated image sequences, experiments were also performed on image sequences of real rotating cells. All of these results indicate that this method is accurate, practical, and useful. Furthermore, the method presented there can also be used to determine the 3D rotation velocity of other types of spherical objects that are commonly used in microfluidic applications, such as beads and microparticles.

Thermometry of photosensitive and optically induced electrokinetics chips.

Optically induced electrokinetics (OEK)-based technologies, which integrate the high-resolution dynamic addressability of optical tweezers and the high-throughput capability of electrokinetic forces, have been widely used to manipulate, assemble, and separate biological and non-biological entities in parallel on scales ranging from micrometers to nanometers. However, simultaneously introducing optical and electrical energy into an OEK chip may induce a problematic temperature increase, which poses the potential risk of exceeding physiological conditions and thus inducing variations in cell behavior or activity or even irreversible cell damage during bio-manipulation. Here, we systematically measure the temperature distribution and changes in an OEK chip arising from the projected images and applied alternating current (AC) voltage using an infrared camera. We have found that the average temperature of a projected area is influenced by the light color, total illumination area, ratio of lighted regions to the total controlled areas, and amplitude of the AC voltage. As an example, optically induced thermocapillary flow is triggered by the light image-induced temperature gradient on a photosensitive substrate to realize fluidic hydrogel patterning. Our studies show that the projected light pattern needs to be properly designed to satisfy specific application requirements, especially for applications related to cell manipulation and assembly.

Calibration method for a large-scale structured light measurement system.

The structured light method is an effective non-contact measurement approach. The calibration greatly affects the measurement precision of structured light systems. To construct a large-scale structured light system with high accuracy, a large-scale and precise calibration gauge is always required, which leads to an increased cost. To this end, in this paper, a calibration method with a planar mirror is proposed to reduce the calibration gauge size and cost. An out-of-focus camera calibration method is also proposed to overcome the defocusing problem caused by the shortened distance during the calibration procedure. The experimental results verify the accuracy of the proposed calibration method.

Three-dimensional panoramic sensor based on a variable-frequency fringe pattern.

This paper aims to develop a 3D panoramic sensor with a catadioptric imaging system. First, the fringe pattern is designed in a polar coordinate system to match the dimensions of a hyperbolic mirror. Second, a variable-frequency fringe pattern is proposed to obtain the constant-frequency reconstruction in real space. Experiments are conducted to verify system accuracy and feasibility, and the experimental results show the effectiveness of the proposed 3D panoramic sensor.

Accurate Extraction of the Self-Rotational Speed for Cells in an Electrokinetics Force Field by an Image Matching Algorithm.

We present an image-matching-based automated algorithm capable of accurately determining the self-rotational speed of cancer cells in an optically-induced electrokinetics-based microfluidic chip. To automatically track a specific cell in a video featuring more than one cell, a background subtraction technique was used. To determine the rotational speeds of cells, a reference frame was automatically selected and curve fitting was performed to improve the stability and accuracy. Results show that the algorithm was able to accurately calculate the self-rotational speeds of cells up to ~150 rpm. In addition, the algorithm could be used to determine the motion trajectories of the cells. Potential applications for the developed algorithm include the differentiation of cell morphology and characterization of cell electrical properties.

Fabrication of High-Aspect-Ratio 3D Hydrogel Microstructures Using Optically Induced Electrokinetics.

We present a rapid hydrogel polymerization and prototyping microfabrication technique using an optically induced electrokinetics (OEK) chip, which is based on a non-UV hydrogel curing principle. Using this technique, micro-scale high-aspect-ratio three-dimensional polymer features with different geometric sizes can be fabricated within 1⁻10 min by projecting pre-defined visible light image patterns onto the OEK chip. This method eliminates the need for traditional photolithography masks used for patterning and fabricating polymer microstructures and simplifies the fabrication processes. This technique uses cross-link hydrogels, such as poly(ethylene glycol) (PEG)-diacrylate (PEGDA), as fabrication materials. We demonstrated that hydrogel micropillar arrays rapidly fabricated using this technique can be used as molds to create micron-scale cavities in PDMS (polydimethylsiloxane) substrates. Furthermore, hollow, circular tubes with controllable wall thicknesses and high-aspect ratios can also be fabricated. These results show the potential of this technique to become a rapid prototyping technology for producing microfluidic devices. In addition, we show that rapid prototyping of three-dimensional suspended polymer structures is possible without any sacrificial etching process.

Measurement of single leukemia cell's density and mass using optically induced electric field in a microfluidics chip.

We present a method capable of rapidly (∼20 s) determining the density and mass of a single leukemic cell using an optically induced electrokinetics (OEK) platform. Our team had reported recently on a technique that combines sedimentation theory, computer vision, and micro particle manipulation techniques on an OEK microfluidic platform to determine the mass and density of micron-scale entities in a fluidic medium; the mass and density of yeast cells were accurately determined in that prior work. In the work reported in this paper, we further refined the technique by performing significantly more experiments to determine a universal correction factor to Stokes' equation in expressing the drag force on a microparticle as it falls towards an infinite plane. Specifically, a theoretical model for micron-sized spheres settling towards an infinite plane in a microfluidic environment is presented, and which was validated experimentally using five different sizes of micro polystyrene beads. The same sedimentation process was applied to two kinds of leukemic cancer cells with similar sizes in an OEK platform, and their density and mass were determined accordingly. Our tests on mouse lymphocytic leukemia cells (L1210) and human leukemic cells (HL-60) have verified the practical viability of this method. Potentially, this new method provides a new way of measuring the volume, density, and mass of a single cell in an accurate, selective, and repeatable manner.

Rapid determination of cell mass and density using digitally controlled electric field in a microfluidic chip.

The density of a single cell is a fundamental property of cells. Cells in the same cycle phase have similar volume, but the differences in their mass and density could elucidate each cell's physiological state. Here we report a novel technique to rapidly measure the density and mass of a single cell using an optically induced electrokinetics (OEK) microfluidic platform. Presently, single cellular mass and density measurement devices require a complicated fabrication process and their output is not scalable, i.e., it is extremely difficult to measure the mass and density of a large quantity of cells rapidly. The technique reported here operates on a principle combining sedimentation theory, computer vision, and microparticle manipulation techniques in an OEK microfluidic platform. We will show in this paper that this technique enables the measurement of single-cell volume, density, and mass rapidly and accurately in a repeatable manner. The technique is also scalable - it allows simultaneous measurement of volume, density, and mass of multiple cells. Essentially, a simple time-controlled projected light pattern is used to illuminate the selected area on the OEK microfluidic chip that contains cells to lift the cells to a particular height above the chip's surface. Then, the cells are allowed to "free fall" to the chip's surface, with competing buoyancy, gravitational, and fluidic drag forces acting on the cells. By using a computer vision algorithm to accurately track the motion of the cells and then relate the cells' motion trajectory to sedimentation theory, the volume, mass, and density of each cell can be rapidly determined. A theoretical model of micro-sized spheres settling towards an infinite plane in a microfluidic environment is first derived and validated experimentally using standard micropolystyrene beads to demonstrate the viability and accuracy of this new technique. Next, we show that the yeast cell volume, mass, and density could be rapidly determined using this technology, with results comparable to those using the existing method suspended microchannel resonator.

Hand-writing motion tracking with vision-inertial sensor fusion: calibration and error correction.

The purpose of this study was to improve the accuracy of real-time ego-motion tracking through inertial sensor and vision sensor fusion. Due to low sampling rates supported by web-based vision sensor and accumulation of errors in inertial sensors, ego-motion tracking with vision sensors is commonly afflicted by slow updating rates, while motion tracking with inertial sensor suffers from rapid deterioration in accuracy with time. This paper starts with a discussion of developed algorithms for calibrating two relative rotations of the system using only one reference image. Next, stochastic noises associated with the inertial sensor are identified using Allan Variance analysis, and modeled according to their characteristics. Finally, the proposed models are incorporated into an extended Kalman filter for inertial sensor and vision sensor fusion. Compared with results from conventional sensor fusion models, we have shown that ego-motion tracking can be greatly enhanced using the proposed error correction model.

Automated rotation rate tracking of pigmented cells by a customized block-matching algorithm.

This article describes an automated rotation rate tracking algorithm for pigmented cells that undergo rotation in a dielectrophoretic (DEP) force field. In a completely automated process, we preprocess each frame of a video sequence, then analyze the sequence frame by frame using a rotating-circle template with a block-matching algorithm, and finally estimate the rotation rate of the pigmented cells using a pixel-patch correlation. The algorithm has been demonstrated to accurately calculate the DEP-induced rotation rate of the cell up to 250 rpm. Cell rotation rates in various DEP force fields (i.e., by varying the applied voltages, frequencies, and waveforms to induce different force fields) were analyzed using this automated algorithm and reported in this article. Most importantly, the algorithm is accurate even when the cells have simultaneous translational and rotational motions across the video image sequence. Also, the algorithm is capable of tracking changes in rotation speed over a long period of time (90 s) by stably analyzing a massive data set of video image frames.