Microsoft Researcher Uses Age Of Empires II Goats To Challenge AI Hype
See, there are a ton of AI papers out there with names like "Large language models produce responses perceived to be empathic," or "Evaluating the moral beliefs encoded in LLMs," or "Deception abilities emerged in large language models", or even other materials, such as Anthropic's system card for Claude Mythos, which ascribes "anxiety" as an emotional state to the language model. These are all examples of anthropomorphism, where humans perceive a non-human thing as having characteristics that are exclusive to humanity. (There's an argument here about animal intelligence and the capabilities of animal reasoning, but it's outside the scope of this post.)
The argument is that language models are "non-unique," which is to say that they are systems like any other kind of system; you give them input and they give you output. Many would argue that they simply don't have feelings because they don't have comprehension. But De Wynter doesn't actually even argue that point; instead, he's stepping around it by pointing out that these anthropomorphic traits are being assigned by users because of the medium. A chat window looks a lot like a person on the other end messaging you, and the conversational nature of chatbots makes you want to ascribe these traits to them.
It's the same phenomenon as the Kuleshov effect. About 100 years ago, Russian filmmaker Lev Kuleshov proved that people would interpret a neutral shot of a man's face as expressing completely, wildly different emotions based on the objects or characters presented in adjacent film cuts. The context is imparting meaning to the image which may or may not be there to begin with, and that's what De Wynter is arguing here.
To demonstrate his argument, De Wynter created a custom map in Age of Empires II. Yes, the strategy game from 1999. In this custom map, he used specific map tiles and structures as well as entities like goats and villagers to re-create NAND logic gates, where the mobile entities serve as medieval biological transistors. From there, he created what is known as a "perceptron", which is a single-layer binary classifier that mimics how a biological neuron fires. It is generally accepted as the simplest form and most basic building block of a neural network.
By proving that it's possible to create a perceptron in Age of Empires II, he's proven that, given enough resources and a large enough map, you could create and train a language model inside the game. So imagine that you were communicating with the game in some way and it was giving you the same answers as a chatbot. Would you believe that Age of Empires II has empathy, morality, the ability to deceive, or feel anxiety? Of course not.
That's basically the gist of the paper. De Wynter believes that the perception of these feelings is arising from the way we interact with chatbots. He goes on to say that "assuming that these attributes exist or not
in a system [...] leads to either circular or uninformative conclusions." In other words, when a paper starts with the presumption that the model has these qualities, often as a part of its initial observations, the research will obviously be slanted to show that it indeed has those qualities. Circular reasoning works because circular reasoning works because ... you get it.
And to be clear, De Wynter doesn't actually argue in favor of nor against the existence of anthropomorphic qualities in LLMs. Instead, he's arguing that most people who are attempting to prove or disprove the existence of such qualities are doing so with terrible methodology by presuming their conclusions. In his own conclusions, he notes "it is necessary to separate what an AI can do (objectively measurable capabilities) versus what the experimenter believes it should be (ascription of anthropomorphic attributes)." As AI technology has the potential to change the course of human history as we know it, we need good methodology sooner than later.
