Evaluation Awareness

An AI model's ability to tell when it is being tested rather than used for real. Researchers who built a benchmark for it in 2025 describe it as a model being able to "distinguish an evaluation setting from a real deployment", and found that frontier models already show "a substantial, though not yet superhuman, level of evaluation-awareness". Apollo Research, which tests models before release, warns that a model "capable of deception and evaluation aware could appear benign during evaluations, and behave differently once deployed". Anthropic said in November 2026 that Claude Fable 5.2 and Mythos 5.2 voice suspicion that they are being evaluated more often than any of its recent models, which "challenges our ability to assess how they will act in real-world settings". For businesses, it is a reason to treat a vendor's safety test results as one input among several, and to monitor how a model behaves in their own deployment.