Hyperparameter Control Using Fuzzy Logic: Evolving Policies for Adaptive Fuzzy Particle Swarm Optimization Algorithm.

ABSTRACT

Heuristic optimization methods such as Particle Swarm Optimization depend on their parameters to achieve optimal performance on a given class of problems. Some modifications of heuristic algorithms aim at adapting those parameters during the optimization process. We present a novel approach to design such adaptation strategies using continuous fuzzy feedback control. Fuzzy feedback provides a simple interface where probes are sampled in the optimization process and parameters are fed back to the optimizer. The probes are turned into parameters by a fuzzy process optimized beforehand to maximize performance on a training benchmark. Utilizing this framework, we systematically established 127 different Fuzzy Particle Swarm Optimization algorithms featuring a maximum of 7 parameters under fuzzy control. These newly devised algorithms exhibit superior performance compared to both traditional PSO and some of its best parameter control variants. The performance is reported in the single-objective bound-constrained numerical optimization competition of CEC 2020. Additionally, two specific controls, highlighted for their efficacy and dependability, demonstrated commendable performance in real-world scenarios from CEC 2011.