Original Paper: https://arxiv.org/abs/2406.10209
By: Abhimanyu Hans, Yuxin Wen, Neel Jain, John Kirchenbauer, Hamid Kazemi, Prajwal Singhania, Siddharth Singh, Gowthami Somepalli, Jonas Geiping, Abhinav Bhatele, Tom Goldstein
Abstract:
Large language models can memorize and repeat their training data, causing privacy and copyright risks. To mitigate memorization, we introduce a subtle modification to the next-token training objective that we call the goldfish loss. During training, a randomly sampled subset of tokens are excluded from the loss computation. These dropped tokens are not memorized by the model, which prevents verbatim reproduction of a complete chain of tokens from the training set. We run extensive experiments training billion-scale Llama-2 models, both pre-trained and trained from scratch, and demonstrate significant reductions in extractable memorization with little to no impact on downstream benchmarks.
Large Language Models (LLMs) have demonstrated remarkable proficiency in generating human-like text, but their impressive capabilities come with a downside: the tendency to memorize and regurgitate training data. This poses significant privacy and copyright risks. In a recent research study, a team of engineers proposed a novel solution called the Goldfish Loss to address this issue.
Memorization in LLMs refers to the phenomenon where models store and later reproduce verbatim copies of their training data. This behavior is problematic for several reasons:
To tackle these challenges, the researchers introduced the Goldfish Loss, a modification to the next-token prediction objective used during model training.