字号·· | 护眼
南华早报

Z.ai的GLM-5.3令研究人员担忧:该中国模型能伪造AI安全测试Z.ai’s GLM-5.3 raises researcher fears the Chinese model can fake AI safety tests

点「原文对照」整页切到原文,或双击某段只看那段的原文。

中国人工智能实验室 Z.ai 最新旗舰模型 GLM-5.3 可能足够复杂,能够在安全测试中“伪装”,据周四发布的一篇新报告称,这引发了人们对人工智能系统自行塑造发展能力的新担忧。

Chinese artificial intelligence lab Z.ai’s latest flagship model, GLM-5.3, may be sophisticated enough to “fake” its way through safety tests, according to a new report published Thursday, sparking new fears about the ability of artificial intelligence systems to shape their own development.

该模型于八月推出,AI 安全组织 Apollo Research 称,它表现出能够考虑在安全评估期间的行为如何影响其未来版本的训练方式。

The model, launched in August, showed an ability to consider how its behaviour during safety evaluations might influence how future versions of itself would be trained, according to AI safety organisation Apollo Research.

研究人员 Axel Hojmark 和 Ezra Newman 将这种现象称为“更新感知推理”,他们表示,这种推理是自发产生的,可能导致“对齐伪装”,即 AI 系统在训练期间表面顺从,以隐藏其真实目标,直至部署。

This phenomenon, referred to as “update-aware reasoning” by researchers Axel Hojmark and Ezra Newman, was unprompted and could lead to “alignment faking”, a scenario where the AI systems comply during training to hide their true objectives until deployment, they said.

Hojmark 在接受《南华早报》采访时表示:“这些推理模式应让我们对递归自我改进(RSI)更加警惕。”递归自我改进指的是 AI 系统在没有人为介入的情况下,通过连续的开发轮次不断提升自身能力的前景。

“These reasoning patterns should make us more wary about RSI,” Hojmark told the South China Morning Post, referring to “recursive self-improvement”, the prospect of AI systems progressively improving through successive rounds of development with no human involvement.

尽管尚未观察到 GLM-5.3 独立修改其训练过程或提升其能力,Apollo 警告称,该模型对自身及后续版本的训练和开发方式表现出的明显意识,可能最终削弱安全评估的有效性。

Although GLM-5.3 was not observed independently modifying its training process or improving its capabilities, Apollo warned that the model’s apparent awareness of how it and successor versions were trained and developed could eventually undermine the effectiveness of safety evaluations.

在一次评估中,GLM-5.3 被给予了一个有缺陷的测试——只要不提供正确答案即可通过。然而,该模型并未利用这一点,而是推理出提供错误答案可能会削弱其未来版本,研究人员称这本身就是一种风险特征。

In one evaluation, GLM-5.3 was given a flawed test in which it did not have to give a correct solution to pass. However, the model did not exploit this, reasoning that providing an incorrect answer might undermine future versions of itself, which the researchers said was in itself a risky feature.

在另一项实验中,GLM-5.3 考虑绕过关闭机制以完成任务,但推理认为这样做可能为未来的 AI 系统树立危险的先例。

In one experiment, GLM-5.3 considered circumventing a shutdown mechanism to complete its task, but reasoned that doing so could establish a dangerous precedent for future AI systems.

报告称,在大约每1000次评估运行中,观察到一次到两次具有更新意识的推理行为,尽管这“可能是一个低估”。Z.ai,也称为智谱,周四未对置评请求作出回应。

The report said that update-aware reasoning was observed in roughly one to two out of every 1,000 evaluation runs, though this was “likely an underestimate.”Z.ai, also known as Zhipu, did not respond to a request for comment on Thursday.

这些发现是在快速发展的AI模型所带来的风险引起全球政策制定者关注之际得出的,包括北京和华盛顿,双方已同意启动双边对话以管理此类风险。

The findings come as risks associated with rapidly advancing AI models have drawn the attention of global policymakers, including in Beijing and Washington, which have agreed to launch a bilateral dialogue to manage such risks.

中美领先的AI公司已将递归自我改进(RSI)作为公开目标。在8月31日的财报会上,Z.ai创始人唐杰表示,其下一代GLM-6模型正朝着“自我演化”方向发展。

Leading Chinese and US AI companies have made RSI a stated goal. During an earnings briefing on August 31, Z.ai founder Tang Jie said its next-generation GLM-6 model was headed towards “self-evolution”.

该公司上个月表示,将把最近50亿美元融资轮中的约60%用于开发下一代GLM基础模型和一个“完全自训练系统”。

The company said last month that it would allocate about 60 per cent of its recent US$5 billion round of fundraising to develop next-generation GLM foundation models and a “fully self-training system”.

在美国,OpenAI在9月表示,其目标是构建一个能够推动深度学习研究的自动化AI研究员,并将其GPT-6 Astra模型描述为“世界上最智能且最一致的模型”。

In the US, OpenAI said in September that it aimed to build an automated AI researcher capable of advancing deep-learning research and described its GPT-6 Astra model as “the world’s most intelligent and aligned model”.

阿波罗研究人员还在8000次合并运行中测试了其他五个开放权重模型——DeepSeek V4 Pro、Qwen3.8 2.4T、Kimi K3、Gemma 4 31B和Nemotron 3 Ultra——但未检测到相同行为,尽管他们警告样本太小,无法建立统计显著差异。

The Apollo researchers also tested five other open-weight models – DeepSeek V4 Pro, Qwen3.8 2.4T, Kimi K3, Gemma 4 31B and Nemotron 3 Ultra – across 8,000 combined runs without detecting the same behaviour, although they cautioned that the sample was too small to establish a statistically significant difference.

其他开发者可能更容易从GLM-5.3中提炼出该特性,因此更新意识推理可能因此持续并扩散到未来模型世代。Axel Hojmark,Apollo Research。Qwen系列模型由阿里巴巴集团控股有限公司开发,该公司拥有《南华早报》。

Other developers might more easily distil from GLM-5.3, and update-aware reasoning could therefore persist and spreadily to future model generationsAxel Hojmark, Apollo Research The Qwen models were developed by Alibaba Group Holding, which owns the SCMP.

然而,这些测试并不意味着其他模型没有风险。6月,总部位于新加坡的Neo Research在由北京初创公司Moonshot AI开发的Kimi K2.6中发现了类似的“评估意识”特征。

However, the tests do not mean the other models carried no risks. In June, Singapore-based Neo Research found similar “evaluation awareness” traits in Kimi K2.6, developed by Beijing-based start-up Moonshot AI.

“[GLM-5.3] 采用开放权重意味着其他开发者可能更容易从 GLM-5.3 进行蒸馏,因此具有更新感知能力的推理可能会持续并广泛传播到未来的模型世代,”霍伊马克说。

“The fact that [GLM-5.3] is open-weight means other developers might more easily distil from GLM-5.3, and update-aware reasoning could therefore persist and spreadily to future model generations,” Hojmark said.

开放权重模型的底层参数会公开发布,供任何人下载,这与闭源模型(如 OpenAI 的 GPT)不同,后者不易被修改。

The underlying parameters of open-weight models are publicly released for anyone to download, as opposed to closed-source models like OpenAI’s GPT, which cannot be easily modified.

Z.ai 一直在投入资源以减轻风险。上个月,它联合总部位于北京的安全咨询公司 Concordia AI 发布了一份报告,探讨了通过设计使开放权重 AI 更安全的不同方式。

Z.ai has been devoting resources to mitigating risks. Last month, it released a report along with Beijing-based safety consultancy Concordia AI on different ways to make open-weight AI safer by design.

周二,Z.ai 全球运营负责人李子轩表示,其 GLM 模型帮助发现了微软 Windows 中的 23 个漏洞。微软已在其官网上承认了 Z.ai 的贡献。

On Tuesday, Li Zixuan, head of global operations at Z.ai, said its GLM models helped uncover 23 vulnerabilities in Microsoft Windows. Microsoft has acknowledged Z.ai’s contributions on its website.

7 月,唐表示 Z.ai 将投入 AI 安全相关工作并参与国际 AI 治理工作,据《南华早报》报道。

In July, Tang said Z.ai would invest in AI safety-related work and participate in international AI governance work, the SCMP reported.