Leaking training data from GPT-2. How is this possible?

Ms. Coffee Bean explains how a huge collaboration of researchers managed to extract training data from large language models like GPT-2. Why is this even possible and what does this mean for even larger models like GPT-3?
Discussed paper:
" Extracting Training Data from Large Language Models" explained. Paper by Carlini et al. 2020.
➡️ AI Coffee Break Merch! 🛍️ aicoffeebreak.creator-spring....

▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
🔥 Optionally, pay us a coffee to boost our Coffee Bean production! ☕
Patreon: www.patreon.com/AICoffeeBreak
Ko-fi: ko-fi.com/aicoffeebreak
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀

📺 GPT-3 – what you need to know:    • GPT-3 explained with examples. Possibiliti...  

Outline:
00:00 Large Language Models
01:55 GPT-2
02:33 Why is it possible?
03:38 k-eidetic memorization
04:21 How does the attack work?
05:49 How bad is it?
08:12 What to do?

📄 Paper explained: Carlini, N., Tramer, F., Wallace, E., Jagielski, M., Herbert-Voss, A., Lee, K., ... & Oprea, A. (2020). Extracting Training Data from Large Language Models. arXiv preprint arXiv:2012.07805. arxiv.org/pdf/2012.07805.pdf

-------------------------------------------------------------
🔗 Links:
YouTube: youtube.com/AICoffeeBreak
Twitter: twitter.com/AICoffeeBreak
Reddit: www.reddit.com/r/AICoffeeBreak/

#AICoffeeBreak #MsCoffeeBean #GPT #MachineLearning #AI #research