Divide-and-conquer learning and modular perceptron networks.

A novel modular perceptron network (MPN) and divide-and-conquer learning (DCL) schemes for the design of modular neural networks are proposed. When a training process in a multilayer perceptron falls into a local minimum or stalls in a flat region, the proposed DCL scheme is applied to divide the current training data region into two easier to be learned regions. The learning process continues when a self-growing perceptron network and its initial weight estimation are constructed for one of the newly partitioned regions. Another partitioned region will resume the training process on the original perceptron network. Data region partitioning, weight estimating and learning are iteratively repeated until all the training data are completely learned by the MPN. We evaluated and compared the proposed MPN with several representative neural networks on the two-spirals problem and real-world dataset. The MPN achieved better weight learning performance by requiring much less data presentations during the network training phases, and better generalization performance, and less processing time during the retrieving phase.

User adaptive handwriting recognition by self-growing probabilistic decision-based neural networks.

It is generally agreed that, for a given handwriting recognition task, a user dependent system usually outperforms a user independent system, as long as a sufficient amount of training data is available. When the amount of user training data is limited, however, such a performance gain is not guaranteed. One way to improve the performance is to make use of existing knowledge, contained in a rich multiuser data base, so that a minimum amount of training data is sufficient to initialize a model for the new user.We mainly address the user adaption issues for a handwriting recognition system. Based on self-growing probabilistic decision-based neural networks (SPDNNs), user adaptation of the parameters of SPDNN is formulated as incremental reinforced and antireinforced learning procedures, which are easily integrated into the batched training procedures of the SPDNN. In this study, we developed 1) an SPDNN based handwriting recognition system; 2) a two-stage recognition structure; and 3) a three-phase training methodology for a) a global coarse classifier (stage 1); b) a user independent hand written character recognizer (stage 2); and c) a user adaptation module on a personal computer. With training and testing on a 600-word commonly used Chinese character set, the recognition results indicate that the user adaptation module significantly improved the recognition accuracy. The average recognition rate increased from 44.2% to 82.4% in five adapting cycles, and the performance could finally increase up to 90.2% in ten adapting cycles.