Bag of Tricks for Training Data Extraction from Language Models
abs:https://arxiv.org/abs/2302.04460v1
 https://i.imgur.com/MT3TcVP.png

This paper proposes tricks for improving training data extraction from language models, such as sampling strategies and token-level criteria. Experiments show that these tricks can significantly outperform the baseline in most cases, providing a stronger baseline for future research.

Keywords: training data extraction | language models | privacy protection | text generation | text ranking | token-level criteria