Human body 3D posture estimation using significant points and two cameras.

This paper proposes a three-dimensional (3D) human posture estimation system that locates 3D significant body points based on 2D body contours extracted from two cameras without using any depth sensors. The 3D significant body points that are located by this system include the head, the center of the body, the tips of the feet, the tips of the hands, the elbows, and the knees. First, a linear support vector machine- (SVM-) based segmentation method is proposed to distinguish the human body from the background in red, green, and blue (RGB) color space. The SVM-based segmentation method uses not only normalized color differences but also included angle between pixels in the current frame and the background in order to reduce shadow influence. After segmentation, 2D significant points in each of the two extracted images are located. A significant point volume matching (SPVM) method is then proposed to reconstruct the 3D significant body point locations by using 2D posture estimation results. Experimental results show that the proposed SVM-based segmentation method shows better performance than other gray level- and RGB-based segmentation approaches. This paper also shows the effectiveness of the 3D posture estimation results in different postures.

Deep learning-based glomerulus detection and classification with generative morphology augmentation in renal pathology images.

Glomerulus morphology on renal pathology images provides valuable diagnosis and outcome prediction information. To provide better care, an efficient, standardized, and scalable method is urgently needed to optimize the time-consuming and labor-intensive interpretation process by renal pathologists. This paper proposes a deep convolutional neural network (CNN)-based approach to automatically detect and classify glomeruli with different stains in renal pathology images. In the glomerulus detection stage, this paper proposes a flattened Xception with a feature pyramid network (FX-FPN). The FX-FPN is employed as a backbone in the framework of faster region-based CNN to improve glomerulus detection performance. In the classification stage, this paper considers classifications of five glomerulus morphologies using a flattened Xception classifier. To endow the classifier with higher discriminability, this paper proposes a generative data augmentation approach for patch-based glomerulus morphology augmentation. New glomerulus patches of different morphologies are generated for data augmentation through the cycle-consistent generative adversarial network (CycleGAN). The single detection model shows the F1 score up to 0.9524 in H&E and PAS stains. The classification result shows that the average sensitivity and specificity are 0.7077 and 0.9316, respectively, by using the flattened Xception with the original training data. The sensitivity and specificity increase to 0.7623 and 0.9443, respectively, by using the generative data augmentation. Comparisons with different deep CNN models show the effectiveness and superiority of the proposed approach.

Navigation of Three Cooperative Object-Transportation Robots Using a Multistage Evolutionary Fuzzy Control Approach.

This article proposes a new multistage evolutionary fuzzy control configuration and navigation of three-wheeled robots cooperatively carrying an overhead object in unknown environments. Based on the divide-and-conquer technique, this article proposes a stage-by-stage evolutionary obstacle boundary following (OBF) fuzzy control of each of the three robots through multiobjective continuous ant colony optimization. In the first stage, a set of evolutionary nondominated fuzzy controllers (FCs) for a single robot (a leader robot) in the execution of the OBF behavior is learned. In the second stage, a follower robot is controlled by two evolutionary FCs in combination with a switched compensation FC so that the leader and follower robots can cooperatively transport an object while executing the OBF behavior along obstacles containing corners with right angles. In the third stage, the third robot functions as an accompanying robot and is learned to enter into a predicted triangular formation with the leader-follower robots to transport a larger object while executing the OBF behavior. In the navigation of the three object-transportation robots, a new cooperative behavior supervisor is proposed to coordinate the learned OBF behavior and a target seeking behavior. Successful navigations in simulations and experiments verify the effectiveness of the multistage evolutionary fuzzy control approach and navigation scheme.

Navigation of a Fuzzy-Controlled Wheeled Robot Through the Combination of Expert Knowledge and Data-Driven Multiobjective Evolutionary Learning.

This article proposes a navigation scheme for a wheeled robot in unknown environments. The navigation scheme consists of obstacle boundary following (OBF), target seeking (TS), and vertex point seeking (VPS) behaviors and a behavior supervisor. The OBF behavior is achieved by a fuzzy controller (FC). This article formulates the FC design problem as a new constrained multiobjective optimization problem and finds a set of nondominated FC solutions through the combination of expert knowledge and data-driven multiobjective ant colony optimization. The TS behavior is achieved by new fuzzy proportional-integral-derivative (PID) and proportional-derivative (PD) controllers that control the orientation and speed of the robot, respectively. The VPS behavior is proposed to shorten the navigation route by controlling the robot to move toward a new subgoal determined from the vertex point of an obstacle. A new behavior supervisor that manages the switching among the OBF, TS, and VPS behaviors in unknown environments is proposed. In the navigation of a real robot, a new robot localization method through the fusion of encoders and an infrared localization sensor using a particle filter is proposed. Finally, this article presents simulations and experiments to verify the feasibility and advantages of the navigation scheme.

A Fuzzy Neural Network Model for Rapid Prediction of Optimal Positive Airway Pressures in OSAS Patients.

Manual titration of positive airway pressure (PAP) is a gold standard to provide an optimal pressure for the treatment of obstructive sleep apnea-hypopnea syndrome (OSAS). Since manual titration studies were costly and time-consuming, many statistical models for predicting effective PAPs were reported. However, the prediction accuracies of the models associated with nocturnal parameters still remain low. This study proposes a fuzzy neural prediction network (FNPN) with input candidate variables, selected among easily available measurements (e.g., body mass index (BMI), waist circumstance (WC), and body composition) and OSAS related questionnaires, to rapidly predict an optimal PAP. The FNPN comprises fuzzy rules and is characterized with the ability of automatic rule growing and pruning from training data. A total of 147 participants from April 2018 to April 2019 were enrolled in Taichung Veterans General Hospital, Taiwan. After two selection processes for feature extraction, WC and BMI were the significant variables for entering the FNPN to predict optimal PAP. Experimental results showed that the average successful prediction rate of the proposed method was 71.8%. This study also found that Epworth sleepiness scales (ESS) and body composition, such as visceral fat area and percent body fat, were excluded in the final prediction model. Compared with existing models, the proposed prediction approach provided a rapid prediction of optimal PAP with higher accuracy.

Explainable fuzzy neural network with easy-to-obtain physiological features for screening obstructive sleep apnea-hypopnea syndrome.

OBJECTIVE/BACKGROUND: Recently, several tools for screening obstructive sleep apnea-hypopnea syndrome (OSAHS) have been devised with varied shortcomings. To overcome these drawbacks, we aimed to propose a self-estimation method using an explainable prediction model with easy-to-obtain variables and evaluate its performance for predicting OSAHS. PATIENTS/METHODS: This retrospective, cross-sectional study selected significant easy-to-obtain variables from patients, suspected of having OSAHS by regression analysis, and fed these variables into the proposed explainable fuzzy neural network (EFNN), a back propagation neural network (BPNN) and a stepwise regression model to compare the screening performance for OSAHS. RESULTS: Of the 300 participants, three easily available features, such as waist circumference, mean blood pressure (BP) at the end of polysomnography and the difference in systolic BP between the end and start of polysomnography, were obtained from regression analysis with a five-fold cross-validation scheme. Feeding these three variables into the prediction models showed that the average prediction differences for apnea-hypopnea index (AHI) when using the EFNN, BPNN, and regression model were respectively 1.5 ± 18.2, 3.5 ± 19.1 and 0.1 ± 19.3, indicating none of the tested methods had good efficacy to predict the AHI values. The performance as determined by the sensitivity + specificity-1 value for screening moderate-to-severe OSAHS of the EFNN, BPNN and regression model were respectively 0.440, 0.414 and 0.380. CONCLUSIONS: When fed with easy-to-obtain physiological features, the understandable EFNN should be the preferred method to predict moderate-to-severe OSAHS.

Multiobjective Rule-Based Cooperative Continuous Ant Colony Optimized Fuzzy Systems With a Robot Control Application.

This paper proposes a new rule-based cooperative framework for multiobjective evolutionary fuzzy systems (FSs). Based on the framework, a multiobjective rule-based cooperative continuous ant-colony optimization (MO-RCCACO) algorithm is proposed to optimize all of the free parameters in FSs. Instead of optimization using a single colony of FSs (solutions), the MO-RCCACO consists of r subcolonies of size N cooperatively optimizing an FS that consists of r rules, with a subcolony optimizing only a single fuzzy rule. In addition, an auxiliary colony is created to store all of the fuzzy rules in the best-so-far N FSs to enhance the optimization ability of MO-RCCACO. The performance ranking of different fuzzy rules in the same subcolony is performed based on the multiobjective function values of their participating FSs by using Pareto nondominated sorting and the crowding distance. The MO-RCCACO is applied to find the Pareto-optimal fuzzy controllers (FCs) of a mobile robot for wall following with multiple control objectives. The optimization ability of the MO-RCCACO is verified through comparisons with various multiobjective population-based optimization algorithms in the robot wall-following control problem. Experimental results verify the effectiveness of the MO-RCCACO-based FCs for the boundary following control of a real robot.

Multiobjective Evolution of Biped Robot Gaits Using Advanced Continuous Ant-Colony Optimized Recurrent Neural Networks.

This paper proposes the optimization of a fully connected recurrent neural network (FCRNN) using advanced multiobjective continuous ant colony optimization (AMO-CACO) for the multiobjective gait generation of a biped robot (the NAO). The FCRNN functions as a central pattern generator and is optimized to generate angles of the hip roll and pitch, the knee pitch, and the ankle pitch and roll. The performance of the FCRNN-generated gait is evaluated according to the walking speed, trajectory straightness, oscillations of the body in the pitch and yaw directions, and walking posture, subject to the basic constraints that the robot cannot fall down and must walk forward. This paper formulates this gait generation task as a constrained multiobjective optimization problem and solves this problem through an AMO-CACO-based evolutionary learning approach. The AMO-CACO finds Pareto optimal solutions through ant-path selection and sampling operations by introducing an accumulated rank for the solutions in each single-objective function into solution sorting to improve learning performance. Simulations are conducted to verify the AMO-CACO-based FCRNN gait generation performance through comparisons with different multiobjective optimization algorithms. Selected software-designed Pareto optimal FCRNNs are then applied to control the gait of a real NAO robot.

Hierarchical singleton-type recurrent neural fuzzy networks for noisy speech recognition.

This paper proposes noisy speech recognition using hierarchical singleton-type recurrent neural fuzzy networks (HSRNFNs). The proposed HSRNFN is a hierarchical connection of two singleton-type recurrent neural fuzzy networks (SRNFNs), where one is used for noise filtering and the other for recognition. The SRNFN is constructed by recurrent fuzzy if-then rules with fuzzy singletons in the consequences, and their recurrent properties make them suitable for processing speech patterns with temporal characteristics. In n words recognition, n SRNFNs are created for modeling n words, where each SRNFN receives the current frame feature and predicts the next one of its modeling word. The prediction error of each SRNFN is used as recognition criterion. In filtering, one SRNFN is created, and each SRNFN recognizer is connected to the same SRNFN filter, which filters noisy speech patterns in the feature domain before feeding them to the SRNFN recognizer. Experiments with Mandarin word recognition under different types of noise are performed. Other recognizers, including multilayer perceptron (MLP), time-delay neural networks (TDNNs), and hidden Markov models (HMMs), are also tested and compared. These experiments and comparisons demonstrate good results with HSRNFN for noisy speech recognition tasks.

A New Method for Self-Estimation of the Severity of Obstructive Sleep Apnea Using Easily Available Measurements and Neural Fuzzy Evaluation System.

This paper proposes a neural fuzzy evaluation system (NFES) with significant variables selected from stepwise regression to predict apnea-hypopnea index (AHI) for evaluating obstructive sleep apnea (OSA). The variables considered are the change statuses of blood pressure (BP) before going to sleep and early in the morning as well as other five easily available measurements (age, body mass index (BMI), etc.) so that users can use the system for self-evaluation of OSA. A total of 150 subjects are reviewed retrospectively and categorized as training (120 subjects) and validation (30 subjects) sets by a fivefold cross-validation scheme with stratified sampling based on the OSA severity. Among the eight variables, the stepwise regression shows that BMI, the difference of systolic BP, and Epworth Sleepiness Scale were the significant factors to predict AHI. The three variables are fed as inputs to the NFES with interpretable fuzzy rules automatically generated from the training set. The average accuracy, sensitivity (Sn), specificity (Sp), and Sn+Sp-1 of the NFES were 75.6%, 77.2%, 75.0%, and 0.552, respectively, in distinguishing the OSA level of normal-mild (AHI <15) from moderate-severe (AHI ≱ 15), and outperformed the stepwise regression, back-propagation neural network, and support vector machine models. In addition to personal self-estimation, physicians could differentiate the two OSA levels by means of the fast-screening system for both outpatients and inpatients.

An Interpretable Fuzzy System Learned Through Online Rule Generation and Multiobjective ACO With a Mobile Robot Control Application.

This paper proposes a new multiobjective optimization approach to designing a fuzzy logic system (FLS) using process data and applies it to the wall-following control of a mobile robot. The objectives considered include both the interpretability and control performance of the FLS. It is assumed that no off-line training data are available in advance, and the rule base is initially empty. All rules are generated through an online clustering and fuzzy set merging (OCFM) algorithm using data generated online during the FLS evaluation process. The OCFM builds a reference rule base that flexibly partitions the input space with distinguishable fuzzy sets (FSs). Based on the reference rule base, a new multiobjective front-guided continuous ant-colony optimization (MO-FCACO) algorithm is proposed to optimize the FLS structure and parameters. In addition to the objective functions defined to evaluate the FLS control performance, a transparency-oriented objective function is defined with constraints imposed on the FS parameters to obtain an interpretable FLS with transparent FSs. The MO-FCACO solves the constrained multiobjective optimization problem by optimizing all of the free parameters in an FLS through ant-path selection, sampling operation, and front-guided optimization processes. The multiobjective FLS design approach is applied to control the orientation and moving speed of a mobile robot in performing the wall-following task. Optimization performance of the MO-FCACO is verified through comparisons with various multiobjective population-based optimization algorithms. Experimental results verify the effectiveness of the designed FLSs in controlling a real robot.

A recurrent fuzzy network for fuzzy temporal sequence processing and gesture recognition.

A fuzzified Takagi-Sugeno-Kang (TSK)-type recurrent fuzzy network (FTRFN) for handling fuzzy temporal information is proposed in this paper. The FTRFN extends our previously proposed network, TRFN, to deal with fuzzy temporal signals represented by Gaussian or triangular fuzzy numbers. In the precondition part of FTRFN, matching degrees between input fuzzy variables and fuzzy antecedent sets is performed by similarity measure. In the TSK-type consequence, a linear combination of fuzzy variables is computed, where two sets of combination coefficients, one for the center and the other for the width of each fuzzy number, are used. Derivation of the linear combination results and final network output is based on left-right fuzzy number operation. There are no rules in FTRFN initially; they are constructed online by concurrent structure and parameter learning, where all free parameters in the precondition/consequence of FTRFN are all tunable. FTRFN can be applied on a variety of domains related to fuzzy temporal information processing. In this paper, it has been applied on one-dimensional and two-dimensional fuzzy temporal sequence prediction and CCD-based temporal gesture recognition. The performance of FTRFN is verified from these examples.

Evolutionary Fuzzy Control and Navigation for Two Wheeled Robots Cooperatively Carrying an Object in Unknown Environments.

This paper presents a method that allows two wheeled, mobile robots to navigate unknown environments while cooperatively carrying an object. In the navigation method, a leader robot and a follower robot cooperatively perform either obstacle boundary following (OBF) or target seeking (TS) to reach a destination. The two robots are controlled by fuzzy controllers (FC) whose rules are learned through an adaptive fusion of continuous ant colony optimization and particle swarm optimization (AF-CACPSO), which avoids the time-consuming task of manually designing the controllers. The AF-CACPSO-based evolutionary fuzzy control approach is first applied to the control of a single robot to perform OBF. The learning approach is then applied to achieve cooperative OBF with two robots, where an auxiliary FC designed with the AF-CACPSO is used to control the follower robot. For cooperative TS, a rule for coordination of the two robots is developed. To navigate cooperatively, a cooperative behavior supervisor is introduced to select between cooperative OBF and cooperative TS. The performance of the AF-CACPSO is verified through comparisons with various population-based optimization algorithms for the OBF learning problem. Simulations and experiments verify the effectiveness of the approach for cooperative navigation of two robots.

A hybrid of genetic algorithm and particle swarm optimization for recurrent network design.

An evolutionary recurrent network which automates the design of recurrent neural/fuzzy networks using a new evolutionary learning algorithm is proposed in this paper. This new evolutionary learning algorithm is based on a hybrid of genetic algorithm (GA) and particle swarm optimization (PSO), and is thus called HGAPSO. In HGAPSO, individuals in a new generation are created, not only by crossover and mutation operation as in GA, but also by PSO. The concept of elite strategy is adopted in HGAPSO, where the upper-half of the best-performing individuals in a population are regarded as elites. However, instead of being reproduced directly to the next generation, these elites are first enhanced. The group constituted by the elites is regarded as a swarm, and each elite corresponds to a particle within it. In this regard, the elites are enhanced by PSO, an operation which mimics the maturing phenomenon in nature. These enhanced elites constitute half of the population in the new generation, whereas the other half is generated by performing crossover and mutation operation on these enhanced elites. HGAPSO is applied to recurrent neural/fuzzy network design as follows. For recurrent neural network, a fully connected recurrent neural network is designed and applied to a temporal sequence production problem. For recurrent fuzzy network design, a Takagi-Sugeno-Kang-type recurrent fuzzy network is designed and applied to dynamic plant control. The performance of HGAPSO is compared to both GA and PSO in these recurrent networks design problems, demonstrating its superiority.

An interval type-2 neural fuzzy chip with on-chip incremental learning ability for time-varying data sequence prediction and system control.

This paper proposes a new circuit to implement a Mamdani-type interval type-2 neural fuzzy chip with on-chip incremental learning ability (IT2NFC-OL) for applications in changing environments. Traditional interval type-2 fuzzy systems use an iterative procedure to find the system outputs, which is computationally expensive, especially for hardware implementation. To address this problem, the IT2NFC-OL uses a simplified type reduction operation to reduce the hardware implementation cost without degrading the learning performance. The software-implemented IT2NFC-OL is characterized by online structure learning and parameter learning using a gradient descent algorithm. The learned fuzzy model is then implemented in a field-programmable gate array (FPGA) chip. The FPGA-implemented IT2NFC-OL performs not only fuzzy inference but also online consequent parameter learning for applications in changing environments. Novel circuits for the computation of system outputs and the update of interval consequent values are proposed. The learning performance of the software-implemented IT2NFC-OL and the on-chip learning ability are verified with applications to time-varying data sequence prediction and system control problems and by comparisons with different software-implemented type-1 and type-2 neural fuzzy systems and interval type-2 fuzzy chips.

Data-driven interval type-2 neural fuzzy system with high learning accuracy and improved model interpretability.

Current studies of type-2 neural fuzzy systems (FSs) (NFSs) primarily focus on building a fuzzy model with high accuracy and disregard the interpretability of fuzzy rules. This paper proposes a data-driven interval type-2 (IT2) NFS with improved model interpretability (DIT2NFS-IP). The DIT2NFS-IP uses IT2 fuzzy sets in its antecedent part and intervals in its zero-order Takagi-Sugeno-Kang-type consequent part for rule form simplicity. The initial rule base is generated by a self-splitting clustering algorithm in the input-output space. The DIT2NFS-IP uses a two-phase parameter-learning algorithm to design an accurate model with improved rule interpretability. In the first phase, a new cost function that considers both accuracy and transparent fuzzy set partition is defined. The antecedent and consequent parameters are learned through gradient descent and rule-ordered recursive least squares algorithms, respectively, to achieve cost function minimization. The second phase performs a fuzzy set reduction, followed by consequent parameter learning to improve accuracy. Comparisons with different type-1 and type-2 FSs in five databased modeling and prediction problems verify the performance of the DIT2NFS-IP in both model accuracy and interpretability.

Reinforcement interval type-2 fuzzy controller design by online rule generation and q-value-aided ant colony optimization.

This paper proposes a new reinforcement-learning method using online rule generation and Q-value-aided ant colony optimization (ORGQACO) for fuzzy controller design. The fuzzy controller is based on an interval type-2 fuzzy system (IT2FS). The antecedent part in the designed IT2FS uses interval type-2 fuzzy sets to improve controller robustness to noise. There are initially no fuzzy rules in the IT2FS. The ORGQACO concurrently designs both the structure and parameters of an IT2FS. We propose an online interval type-2 rule generation method for the evolution of system structure and flexible partitioning of the input space. Consequent part parameters in an IT2FS are designed using Q -values and the reinforcement local-global ant colony optimization algorithm. This algorithm selects the consequent part from a set of candidate actions according to ant pheromone trails and Q-values, both of which are updated using reinforcement signals. The ORGQACO design method is applied to the following three control problems: 1) truck-backing control; 2) magnetic-levitation control; and 3) chaotic-system control. The ORGQACO is compared with other reinforcement-learning methods to verify its efficiency and effectiveness. Comparisons with type-1 fuzzy systems verify the noise robustness property of using an IT2FS.

A type-2 self-organizing neural fuzzy system and its FPGA implementation.

This paper proposes a type-2 self-organizing neural fuzzy system (T2SONFS) and its hardware implementation. The antecedent parts in each T2SONFS fuzzy rule are interval type-2 fuzzy sets, and the consequent part is of Mamdani type. Using interval type-2 fuzzy sets in T2SONFS enables it to be more robust than type-1 fuzzy systems. T2SONFS learning consists of structure and parameter identification. For structure identification, an online clustering algorithm is proposed to generate rules automatically and flexibly distribute them in the input space. For parameter identification, a rule-ordered Kalman filter algorithm is proposed to tune the consequent-part parameters. The learned T2SONFS is hardware implemented, and implementation techniques are proposed to simplify the complex computation process of a type-2 fuzzy system. The T2SONFS is applied to nonlinear system identification and truck backing control problems with clean and noisy training data. Comparisons between type-1 and type-2 neural fuzzy systems verify the learning ability and robustness of the T2SONFS. The learned T2SONFS is hardware implemented in a field-programmable gate array chip to verify functionality of the designed circuits.