Keynotes | K&Ts | GACs | Talks | Posters | Search

Poster F in Poster Session F: Thursday, August 6, 1:45 – 3:30 pm, Kimmel Center, Shorin & Rosenthal Rooms

How do Deep Learning Models and Humans Continually Learn?

Anirudh Doppalapudi¹, Gaurav Malhotra¹; ¹State University of New York at Albany

Presenter: Anirudh Doppalapudi

When deep learning models are trained on a new task, there is a sharp drop in performance on the previously learned task, a phenomenon termed catastrophic forgetting. The present study examines how the internal representations of networks change when they undergo catastrophic forgetting. We compared the internal representations of each layer of a network before and after it learned a new task, and found that even when a model showed catastrophic forgetting, it retained a significant proportion of its internal representations in the early layers. We also found that these results were robust across training datasets, across model architectures, and across task complexities. These results challenge the assumption that neural networks forget all internal representations during catastrophic forgetting and pave the way for not only solving this problem for the network, but also provide insight into human memory.

Topic Area: Memory, Learning & Knowledge Structures