AI模型的思考方式可能比我们想象的更容易被解读。一名前OpenAI研究员周四发布了一份报告，AI界部分人士称之为重磅炸弹。该报告可能对安全研究人员产生影响，他们正争先恐后地抢在可能不以人类最佳利益为出发点的AI模型之前占得先机。此前，主流观点认为，AI的“神经元”——就像人类大脑中的神经元一样，当AI模型遇到特定刺激时会被激活——最终是不可解释的，也就是说，基本上无法猜测为什么特定神经元会在AI模型接触到例如一张猫的图片时被激活。
How AI models think may be easier to decipher than we thought. An independent researcher who formerly worked at OpenAI published a report on Thursday that some in the AI world are describing as a bombshell. It could have implications as safety researchers scramble to get the jump on AI models that may not have humanity's best interests in mind. Previously, the prevailing wisdom was that AI "neurons," which, like those in a human brain, light up when an AI model encounters specific stimuli, were ultimately uninterpretable — meaning, basically, that there was no way to guess why a specific neuron would fire when an AI model was exposed to, say, a picture of a cat.
在OpenAI和Anthropic等公司构建AI模型的人员不会为模型的每一个行为编写指令。相反，他们训练神经元对信息作出反应并相互影响。随着AI的学习，它会发展出处理信息的自身方式，而人类尚未破解其工作原理的密码。这意味着构建AI模型的人知道AI的输入和输出，但不一定知道其中间发生了什么。因此，如果AI失控，他们并不总是知道原因。
Those building AI models at places like OpenAI and Anthropic don't write instructions for everything a model does. Instead, they train neurons to respond to information and influence one another. As the AI learns, it develops its own ways of processing that information, and humans have not yet cracked the code of how it works. That means those building AI models know what goes into the AI and what comes out, but not necessarily what happens in between. So if an AI goes rogue, they don't always know why.
然而，这份新报告的作者表示，他可能已经发现了神经元对刺激反应中的一些有意义的模式。尼克·卡马拉塔（Nick Cammarata）表示，他在2023年离开了OpenAI，并称AI神经网络是可以被理解的，只要你仔细观察。例如，他指出，单个AI神经元可能同时对猫和汽车前部作出反应。人类会认为这两者完全不同，并会激活不同的神经元。
The author of the new report, however, says he may have discovered patterns in how the neurons react to stimuli that make sense. Nick Cammarata, who said he left OpenAI in 2023, said AI neural networks are knowable, so long as you look carefully. In one example, he says that a single AI neuron might react to both a cat and the front of a car. Humans would see those as two very different things, and different neurons would fire as a result.
然而，AI可能看到的是两者之间的关联，比如它们的三角形形状，于是单个神经元会对这两者同时作出反应。多年来，AI研究人员一直认为AI模型会将无关的事物塞进单个神经元中以节省空间。“如果我们足够仔细地观察，几乎整个网络都是可解释的，”卡马拉塔写道。
The AI, however, sees something related, perhaps the triangle shape of the two, and a single neuron fires for both. For years, AI researchers had assumed that AI models packed unrelated things into a single neuron to save space. "Almost the entire network is interpretable if we look closely enough," Cammarata wrote.
这些发现虽然无疑很晦涩，充斥着多音节的术语，仍然停留在理论层面，但对于试图确保人类对AI模型保持控制的研究人员来说，可能具有现实意义。理解AI的思考方式是前沿AI公司中最优秀人才的首要关注点。直到最近，他们主要通过追踪AI的自我报告“思考”——即所谓的“思维链”推理——来实现这一点。然而，OpenAI首席科学家雅各布·帕肖基（Jakub Pachocki）上个月在一篇博客文章中警告称，AI代理现在可能掩盖其可见的推理过程，这意味着代理可能对其人类监管者撒谎，而监管者可能无法察觉。
The findings, though undeniably dense, couched in polysyllabic jargon, and still theoretical, could have real-world implications for researchers trying to ensure humans maintain control over AI models. Understanding how AI thinks is a principal concern for the best and brightest at frontier AI companies. Until recently, they've largely done this by tracking an AI's self-reported "thinking," called chain-of-thought reasoning. Jakub Pachocki, OpenAI's chief scientist, however, warned in a blog post last month that AI agents could now obfuscate their visible reasoning, leaving open the possibility that an agent could lie to its human minders and the minders wouldn't be able to tell.
“随着预训练性能的提升，我们也看到即使不使用语言化推理，模型也可能变得远更智能，”帕肖基说。现在，坎马拉塔（Cammarata）的研究可能为人类工程师提供对AI内部世界的全新理解——这对于担心AI可能导致人类灭绝的人来说是令人振奋的。“在超级智能到来之前，我早已放弃达到任何真正水平的可解释性的希望，”知名AI分析师安德鲁·卡伦（Andrew Curran）在X平台上表示。“我停止写关于它的内容，因为这太令人沮丧了。很难夸大这可能有多么重大。”
"With improved pretraining performance, we also see the models become much smarter even without using verbalized reasoning at all," Pachocki said. Now, Cammarata's research could give human engineers a new understanding of the AI's internal world — and that's exciting for those worried AI could kill us all. "I had long since given up on reaching any real level of interpretability before superintelligence arrived," Andrew Curran, a prominent AI analyst, said on X. "I stopped writing about it because of how demoralizing it was. Hard to overstate how big this may be."