Critique and legacy

Demo or die

The famous transcript, reproduced on the dialogue page, reads with an uncanny smoothness. Each question sits in exactly the register the system can handle; each answer arrives polished and apt; the ambiguities resolved are the ambiguities the program was built to resolve. That smoothness is not an accident of selection. By Winograd's own later testimony the dialogue was, in his words, "very carefully worked through, line by line." It is not a recording of a conversation that happened to go well. It is really a composition or an assemblage.

SHRDLU was given as an example in textbooks and popular accounts as evidence that the problem of natural language understanding had, in some meaningful local sense, been solved. Winograd is unusually candid about the distance between what the demo showed and what the system could do. If you sat down and asked a question that was not in the dialogue, he recalled, "there was some probability it would answer it." Some probability. The reckoning is not that SHRDLU could converse and the transcript merely sampled that capacity, but that the transcript and the capacity were close to coextensive.

Winograd notes the amount of institutional pressure on him and others working on prototypes and software in the demand for "something you could demo." Indeed, the Media Lab's later slogan characterises the whole mode of work that became a norm at MIT:

The danger is not that the facade was built, every research prototype is in some measure a facade, but that the facade was mistaken for the town, and that was where the field believed its successes could be demonstrated. A demonstration optimised to demo to your colleagues is a poor way of measuring how far computer science has actually developed.

The scaling problem

Why could mastering the blocks world not be extended, step by step, into mastering the world? SHRDLU's competence rested on the fact that its world was closed and hand-axiomatised. Every object, property, and relation that mattered, every cube, pyramid, box, and colour, had been enumerated in advance and given an explicit representation. Within such a world, reasoning is solvable because the space of what can be meant is bounded. The program does not have to decide what is relevant; relevance has been settled by the design of the world. More detail is on the blocks world page.

To understand an ordinary sentence is to bring to bear an indefinitely large and unbounded body of background knowledge, and, harder still, to know which fragment of it is relevant here and now. The microworld could not address this fundamental problem of the space of meaning is in many ways impossible to formally define; it also could not learn and expand its knowledge, even about the limited world it had access to. Some thought that adding a second microworld might help, but in reality this just produced two microworlds rather than a larger cohesive world.

When the microworld results refused to generalise and the funding agencies noticed the distance between promise and delivery, optimism from the funders dried up rapidly. If understanding required an enormous body of common-sense knowledge, then perhaps that knowledge should simply be encoded by hand, all of it. Doug Lenat's Cyc, begun in 1984, set out to do exactly that, axiomatising millions of everyday facts in formal logic. Cyc is the microworld strategy taken to its limit and its reductio: the recognition that the background cannot be ignored, paired with the idea that if only it can be written down the problem might be solved. Decades on, this remains a cautionary tale about the cost of believing that formalising knowledge is an answer to the problem of being in the world, or indeed of understanding it in any real way.

Dreyfus, and the body

Hubert Dreyfus, in What Computers Can't Do (1972; revised 1979; reissued as What Computers Still Can't Do, MIT Press, 1992), argued that artificial intelligence succeeds only in artificial micro-worlds, and that this success is not a stepping stone towards general intelligence because intelligence does not rest on formal representations at all. Drawing on Heidegger and Merleau-Ponty, Dreyfus held that human competence is grounded in an embodied, holistic background of skilled coping, a familiarity with situations that is lived rather than represented, and which cannot in principle be reduced to a set of explicit rules and facts.

SHRDLU only worked because someone had supplied, in advance and by hand, the relevances that a human being inhabits without representing. Its world had no horizon, no body, no involvement, only a finite lattice of declared objects and rules and was therefore 'world-poor'. That is why it could explain its own actions with such precision was a sign that something was mechanical, not that it had understanding: a world transparent enough to be fully narrated by the agent within it is a world from which the hard part of understanding has already been removed. Dreyfus was widely resented in AI circles at the time, not because formal AI achieved nothing, but because it had a tendency to overclaim and over-formalise the world.

SHRDLU's self-explanation is interesting in light of this and for subsequent attempts to build causality into world models. Asked "why did you do it", the program walks one level up its own goal stack and reports the subgoal the action served: it gives a reason, a purpose, in the sense, perhaps, Anscombe (1957) and Davidson (1963) drew out of ordinary action. What it never gives is a cause. In Judea Pearl's terms it cannot climb past the lowest rung of causal reasoning: it holds no model of the blocks world that would survive the removal of its own act, and so cannot answer the counterfactual that a causal "why" demands, what would have become of the tower had it not moved the cube (Pearl and Mackenzie, 2018). The account it gives is a property of its designed and engineered world, not a sign that the program grasps how that world hangs together. But this also can throw light on contemporary large language models (LLMs), where SHRDLU could at least narrate the reasons for its acts, a large model cannot give the reason or the cause of what it says. This is a problem that has begun to be researched in the area of explainability (or mechanistic interpretability at Anthropic).

The builder turns critic

What gives SHRDLU's story its peculiar force is that its author followed the argument to its conclusion and changed his mind. Winograd moved to Stanford in 1973 intending to build out a full multi-volume framework for language understanding. However, only the first volume, Language as a Cognitive Process, Vol. 1: Syntax, appeared, in 1983, and the larger edifice was never completed. Having engaged seriously with Dreyfus's critique, and having met the Chilean philosopher and politician Fernando Flores, Winograd came to doubt the foundations of the programme he had helped to define.

The result was Understanding Computers and Cognition: A New Foundation for Design (Ablex, 1986), written with Flores. It is a Heideggerian, phenomenological critique of the rationalist tradition in artificial intelligence, arguing that cognition is not the manipulation of internal representations of a pre-given world but is constituted in language, practice, and being-in-the-world. Winograd's later career moved decisively toward human-computer interaction and design, toward building tools that augment situated human activity rather than machines that purport to replace human understanding.

It is not common for a member of a discipline to repudiate the paradigm they are working within, drawing on philosophers whom his own field had not only dismissed but also ignored. The same person who produced the most persuasive demonstration of symbolic understanding produced one of its most considered refutations, and he did so because he took the difficulty seriously.

SHRDLU and the language models

SHRDLU is symbolic, the paradigm now called GOFAI, good old-fashioned AI. Its words are tied to procedures that act over an inspectable world; when it manipulates "the red pyramid" there is a representation in the system that the term denotes, and the program can trace and report the reasoning that connects word to deed. It is grounded and self-explaining. What is outside this limited world it has nothing to say about, and actually no means of getting from the limited to the open world we inhabit.

Large language models are different. They are statistical and neural, trained to predict text rather than to reason over a declared world, and they are general in a way SHRDLU never approached, fluent across open domains, responsive to almost any prompt. But the grounding that SHRDLU had by design is exactly what they lack – they are also world-poor but more worryingly have no ability to discern truth from falsehood. This is the symbol-grounding problem in Stevan Harnad's formulation: tokens manipulated only by their statistical relations to other tokens have no anchorage in anything they are about, and meaning threatens to circulate endlessly within the system without ever touching the world. And where SHRDLU could narrate its goal stack, the workings of a large model are opaque even to those who build it. The readability (what we would now call explainability) was one of symbolic AI's achievements and has proven to be much harder to implement with the connectionist systems that dominate today.

This suggests that the two great strategies, build the world and let understanding follow, or absorb the corpus and let competence emerge, have each secured one half of what understanding seemed to require, and that neither half has yet been able to unify with the other.

This is why SHRDLU remains worth a close reading half a century on. The first dream of machine understanding was specific and demanding: a world the machine could grasp, words it could mean, actions it could account for. That dream was realised, in miniature, and the limited world it implied remained just that, limited. We have since built machines that talk about everything and ground nothing, and machines that ground everything and talk about almost nothing, and we still do not have one that does both.