Does the Adam Optimizer Amplify Catastrophic Forgetting?
SMRTR summary
Researchers investigated whether the choice of optimization algorithm affects catastrophic forgetting in neural networks, finding that modern optimizers like Adam actually cause more forgetting than simpler methods like vanilla SGD. The study also revealed that different metrics for measuring catastrophic forgetting can lead to dramatically different conclusions about algorithm performance.
SMRTR provides this summary for quick context. The original article belongs to Hacker Noon.
Read the original article