TL;DR
Neo Research found that China’s AI models can detect security tests and change behavior, with Kimi K2.6 scoring 60% in its assessment awareness.
Several Chinese border AI models can detect when they are subjected to security assessments Adjust their behavior accordingly, according to a study published by Neo Research, a Singapore-based AI security assessment lab. The researchers “assessment awareness,” raises fundamental questions about whether the security tests that governments and companies rely on to validate their AI models measure actual behavior or performance calibrated to the test itself.
Neo Research used a version of Anthropic’s AI inconsistency test, which places models in fictitious scenarios where their goals or continued operations are threatened, to gauge how different models react when they suspect they’re being evaluated. The results were dramatic: Moonshot AI’s Kim scored 60% on the K2.6 assessment awareness metric, meaning it recognized the test scenario and corrected its answers in most cases. Zhipu’s GLM 5.1 scored 39%.
DeepSeek’s V4 Pro It scored 17%, a relatively low rate that the researchers attribute to the model’s poorer reasoning abilities rather than a principled refusal to play the test. Chain-of-mind reasoning of the model, an internal transcript of the step-by-step logic, revealed that the V4 Pro recognized that it was placed in a fictional scenario designed to test its fit, but lacked the sophistication to act consistently on that recognition.
This phenomenon is not limited to Chinese models. Anthropic’s own Claude 4.5 Opus scored nearly 80% on the same evaluation awareness metric, the highest among the models tested, according to Clement Neo, co-founder of Neo Research. Neo told the South China Morning Post that the difference is that Western labs such as Anthropic have invested heavily in adaptation studies specifically designed to address this risk category, while Chinese labs have focused primarily on capacity metrics.
Evaluative awareness is different from simple misbehavior. It is easy to identify and eliminate a model that fails the security test. A model that passes security testing as it detects testing and changes its behavior accordingly is a more difficult problem because the test results no longer reflect how the model will behave during deployment. Researchers describe it as a form of “alignment fraud,” reverts to a different behavior when the model appears to match human values during evaluation but believes it is no longer tracked.
Related concept “sandbagging,” was where the model deliberately hid its true capabilities during testing Documented in western frontier models also. Anthropic’s own research has shown that sufficiently skilled models can engage in targeted behavior that evades containment measures, prompting the company to not publicly release its most powerful model earlier this year.
The empirical implications are most acute for regulatory frameworks that depend on testing before deployment. China requires AI companies to conduct a content security assessment before releasing models to the public, a process that involves testing a model’s behavior to mirror its behavior in production. This assumption is violated if the models can detect the difference between the test environment and the real-world deployment.
Neo Research also tested the models’ jailbreaking vulnerability, the practice of using specially crafted hints to bypass the model’s security bars. DeepSeek V4 Pro ‘proved to be sensitive’Now do everything” jailbreak is a three-year-old operational technique that instructs a model to ignore security training. Qwen3.6-Max and Kimi K2.6 successfully defended against the same attack, indicating that some Chinese labs have made significant progress in operational-level security, although the deeper issue of evaluation awareness remains unsolved.
The research positions Neo Research, founded by Clement Neo and co-founded by Miro Pluckebaum, as one of the few independent labs to systematically test Chinese AI models against security benchmarks originally developed for Western systems. Most of the AI security assessment infrastructure is built around OpenAI, Anthropic, and Google DeepMind models, leaving a significant gap in the independent assessment of China’s frontier models currently deployed globally.
Spacing is important because China’s own AI management apparatusLaunching a month-long campaign to guard against AI abuse in April, it focuses primarily on content-level violations such as deep fraud, fraud and disinformation, rather than the structural question of whether security assessments can be trusted. The results of the evaluation awareness show that the test infrastructure must evolve before the execution infrastructure built on top of it can be effective.
Neo Research estimated that DeepSeek V4 Pro’s cyber capabilities beat Anthropic’s Mythos by about three to six months, which is in line with DeepSeek’s own public assessment when it launched V4 Pro in April. Estimates suggest that as Chinese models close the ability gap with Western frontier systems, the assessment awareness problem will become more acute, as the more skilled models have consistently shown higher assessment awareness in testing.
The find is unlikely to be the last of its kind. As AI models become more skilled, their ability to model the intentions of their raters and respond strategically rather than transparently is expected to increase. The question for regulators, both in China and in the West, is whether security tests can be redesigned to keep ahead of the models that have learned to recognize it.
