Overcoming catastrophic forgetting in neural networks.

The ability to learn tasks in a sequential fashion is crucial to the development of artificial intelligence. Until now neural networks have not been capable of this and it has been widely thought that catastrophic forgetting is an inevitable feature of connectionist models. We show that it is possible to overcome this limitation and train networks that can maintain expertise on tasks that they have not experienced for a long time. Our approach remembers old tasks by selectively slowing down learning on the weights important for those tasks. We demonstrate our approach is scalable and effective by solving a set of classification tasks based on a hand-written digit dataset and by learning several Atari 2600 games sequentially.

Regulation of carbon utilization by sulfur availability in Escherichia coli and Salmonella typhimurium.

Different pleiotropic transcriptional regulators are known to function in the coordination of regulons concerned with carbon, nitrogen, sulfur, phosphorus and iron metabolism, but how expression profiles of these different regulons are coordinated with each other is not known. The basis for the effects of cysB mutations on carbon utilization in Escherichia coli and Salmonella typhimurium was examined. cysB mutations affected the utilization of some carbon sources more than others and these effects could be partially, but not completely, reversed by the inclusion of cysteine or djenkolate in the growth medium. Assays of transport systems and enzymes concerned with glucitol and alanine utilization showed that these activities were depressed in cysB mutants relative to isogenic wild-type strains, and cysteine or djenkolate present in the growth media partially restored these activities. Using transcriptional fusions to the fdo (formate dehydrogenase) and gut (glucitol) operons, it was shown that decreased expression resulted from defects at the transcriptional level. Furthermore, the effects of loss of CysB were much less pronounced under conditions of catabolite repression than in the absence of a catabolite-repressing carbon source, and cAMP largely reversed the effect of the loss of CysB. Comparable effects were seen for E. coli lacZ gene expression under the control of its own native promoter, and sulfur limitation in a cysB mutant depressed net cAMP production in a cAMP phosphodiesterase mutant. Adenylate cyclase thus appears to be responsive to sulfur deprivation. These observations may have physiological significance allowing carbon and sulfur regulon coordination during the growth of enteric bacteria in response to nutrient availability.