The Exact VC Dimension of the WiSARD n -Tuple Classifier.

ABSTRACT

The Wilkie, Stonham, and Aleksander recognition device (WiSARD) n -tuple classifier is a multiclass weightless neural network capable of learning a given pattern in a single step. Its architecture is determined by the number of classes it should discriminate. A target class is represented by a structure called a discriminator, which is composed of N RAM nodes, each of them addressed by an n -tuple. Previous studies were carried out in order to mitigate an important problem of the WiSARD n -tuple classifier having its RAM nodes saturated when trained by a large data set. Finding the VC dimension of the WiSARD n -tuple classifier was one of those studies. Although no exact value was found, tight bounds were discovered. Later, the bleaching technique was proposed as a means to avoid saturation. Recent empirical results with the bleaching extension showed that the WiSARD n -tuple classifier can achieve high accuracies with low variance in a great range of tasks. Theoretical studies had not been conducted with that extension previously. This work presents the exact VC dimension of the basic two-class WiSARD n -tuple classifier, which is linearly proportional to the number of RAM nodes belonging to a discriminator, and exponentially to their addressing tuple length, precisely N(2n-1)+1 . The exact VC dimension of the bleaching extension to the WiSARD n -tuple classifier, whose value is the same as that of the basic model, is also produced. Such a result confirms that the bleaching technique is indeed an enhancement to the basic WiSARD n -tuple classifier as it does no harm to the generalization capability of the original paradigm.