A framework to deal with interference in connectionist systems

Article type: Research Article

Authors: Ruiz de Angulo, Vicente | Torras, Carme

Affiliations: Institut de Robòtica i Informàtica Industrial, (CSIC‐UPC), Edifici NEXUS, Gran Capità 2‐4, 08034‐Barcelona, Spain E‐mail: {ruiz,torras}@iri.upc.es

Note: [] Corresponding author.

Abstract: We analyze the conditions under which a memory system is prone to interference between new and old items. Essentially, these are the distributedness of the representation and the lack of retraining. Both are, however, desirable features providing compactness and speed. Thus, a two‐stage framework to palliate interference in this type of systems is proposed based on exploiting the information available at each moment. The two stages are separated by the instant at which a new item becomes known: (a) interference prevention, prior to that instant, consists in preparing the system to minimize the impact of learning new items and (b) retroactive interference minimization, posterior to that instant, seeks to learn the new item while minimizing the damages inflicted on the old items. The subproblems addressed at the two stages are stated rigorously and possible methods to solve each of them are presented.

Keywords: Interference, catastrophic forgetting, associative memory, neural networks

Journal: AI Communications, vol. 13, no. 4, pp. 259-274, 2000