Detecting Data Contamination in LLMs via In-Context Learning

The Intuitive Principles Behind CoDeC's Effectiveness

CoDeC's ability to accurately detect contamination stems from several core mechanisms related to how LLMs process information:

1. Dataset-Specific Priors: Models trained on a dataset internalize its unique style, structure, vocabulary, and implicit assumptions. If these priors are already memorized, additional in-context examples add little useful information.

2. Context Disrupts Memorization: For contaminated datasets, adding memorized in-context examples can interfere with established token sequences or priors. This disruption leads to reduced model confidence, indicating reliance on memorization rather than generalization.

3. ICL Simulates Finetuning Dynamics: In-context learning acts as an efficient proxy for finetuning. Contaminated models, already "saturated" with their training data, show minimal gains from additional context, much like a finetuned model would.

4. Interventions Expose Loss Landscape: Contamination is often linked to overfitting, where models sit in narrow, easily destabilized local minima. New context can easily push overfitted models out of these minima, revealing a drop in confidence.

CoDeC Identifies Contamination Early in Training

Experiments tracking CoDeC scores during the training of the OLMO 7B model reveal critical insights into model learning. At initialization, scores hover around 50%. However, a significant shift occurs between 1k and 10k training steps (roughly 2% of total training), where contamination scores for training datasets rise sharply while scores for unseen datasets stabilize near their final low values.

Implication: This early detection capability makes CoDeC highly effective for identifying and preventing benchmark leaks, ensuring more reliable model development from its nascent stages. The model quickly learns to recognize and memorize distributions of its training sets.

Finetuning Confirms CoDeC's Reliability

To further validate CoDeC, we performed controlled finetuning experiments on various models (Pythia, GPT-Neo, OLMO, Qwen3). When models were finetuned on a chosen dataset, the CoDeC score consistently rose above 90% for that dataset. This clear and stable increase, independent of the model's original training corpus, demonstrates that finetuning directly introduces contamination that CoDeC reliably detects.

Implication: This finding confirms CoDeC's applicability even with models that have undisclosed training data, providing a robust method for assessing contamination induced by targeted model updates.

CoDeC Reveals Overfitting through Loss Landscape Sensitivity

CoDeC's mechanism aligns with the concept of overfitting. Contaminated (memorized) samples often correspond to narrow, high-variability minima in the model's loss landscape. Even small perturbations, such as the introduction of in-context examples, can easily destabilize predictions for these memorized patterns, leading to a drop in confidence. Conversely, unseen data resides in flatter, more stable regions of the loss landscape, where additional learning signals (like in-context examples) can lead to improved confidence.

Implication: This insight suggests that CoDeC not only detects contamination but also provides a proxy for understanding the generalization capabilities of an LLM. Models exhibiting lower contamination scores on unseen data are likely generalizing better, as their confidence is less sensitive to minor context changes.