Programming is building a theory
In 1985 Peter Naur, who had helped design ALGOL 60 and would later win the Turing Award, published a short paper called “Programming as Theory Building.” What a programmer makes, he said, is not the text of the program but the understanding that grows in the programmer while the program is being written; the text sets down only a part of it. Naur called that understanding a theory.
He had seen what this means. A good team was hired to extend a compiler that another group had written. They had the source, well commented, the design papers, and the authors themselves to ask. Even so, what they added were patches, stuck on the outside of the design instead of growing out of it, and when the same requirements were put to the original authors, the authors could point at once to simpler ways of doing it inside the structure that was already there. After ten years of patching, the old design could still be seen in the code, but it no longer did anything. The new team had everything that could be written down, and it was not enough.
The reason lies in the word theory, which Naur took from the philosopher Gilbert Ryle. Ryle’s point was that no skill can be reduced to rules. To follow a rule well takes judgment; a rule for the judgment would take judgment in its turn; and so on without end. A theory is knowledge of that kind. In a programmer it shows as the power to look at a new requirement and see at once where it belongs in the program and what it will upset.
From this Naur drew a hard conclusion. A program is alive only so long as a team that holds its theory goes on working on it. When the team breaks up the program dies, though it may go on running for years. A new team, working from the code, builds a different theory, and Naur thought such a program was usually better rewritten than revived. It followed that programmers should be treated as the owners of what they build, for as long as it is in use, and not as hands that can be replaced once the task is done.
The problem with language models
The picture Naur was arguing against was of programming as the turning out of text by interchangeable workers following rules. Language models have made that picture come true. A model writes the code it is asked for and does as it is told, and whatever it comes to understand about the program while it works is gone when the session ends. Notes can be kept, but notes are documentation, and documentation holds only part of a theory. The person who asked for the code did not write it, and more and more often does not read it either, since a model can turn out several thousand lines in an afternoon. What is left is a program whose theory nobody holds.
And it happens again at every change. Before a model can alter a program it has to get the program’s theory back out of the text, and Naur’s whole point is that only part of it is there to be got. So the change is made with half a view of the design, and even that half is thrown away when the change is done. All this is paid for in tokens, and the bill goes up as the program grows, because there is more text to read through each time. To write the same feature twice, in two projects, is a familiar waste, but it is paid once. To relearn a program is paid at every change, for as long as the program is in use.
The two troubles have one cause. The theory of a program is kept nowhere a model can reach, and so it has to be built up again from the text every time.
The 1968 proposal
In October 1968 NATO brought some fifty of the leading people in computing to Garmisch, in Bavaria, because large software projects were failing, late and over budget, and nobody knew why. The report of the meeting, edited by Naur and the British computer scientist Brian Randell, says that the phrase “software engineering,” which was the name of the conference and of the discipline it called for, was “deliberately chosen as being provocative.” The provocation lay in the suggestion that software needed the theoretical foundations and the practical discipline that the older branches of engineering already had.
Douglas McIlroy of Bell Labs, who later gave Unix its pipes, told the meeting that software ought to be built as the older engineers build everything else: out of standard parts. He put part of the industry’s weakness down to “the absence of a software components subindustry,” and he pictured firms selling routines out of catalogs as suppliers sell screws and resistors, so that a programmer would order the routine a program needed instead of writing it yet again.
Put in Naur’s terms, the proposal cuts the theory in two. The theory of each part stays with the people who made it; whoever uses the part needs only its specification; and what a builder has to understand shrinks to the way the parts fit together. That was the promise. It was not kept, and for most of what programs are made of it has not been kept yet.
The promise asks a good deal. A builder must be able to find a part that does the job. The part must fit the rest of the system, including the code the builder writes, for no real system is parts all the way down. It must be trusted without being read. It must be replaceable when needs change. And the whole business must cost less than writing the part oneself. Hardware meets every one of these conditions, with catalogs, standard connections, published ratings, interchangeable parts, and prices below what it would cost to make one’s own. Software met none of them.
Why now
It met none of them, and under all five the reason was much the same. Code was expensive to write. A part that did the job cost more to find, fit and check than to write again, so every builder wrote everything, carried the whole theory of it alone, and the trade in parts that McIlroy asked for never grew up. Language models have changed the price. A model can write in an afternoon more code than anyone will read, and, as the last section showed, the understanding does not come with it. Code is cheap now and understanding is not, and when the scarce thing changes, the sensible way to build changes with it. It no longer pays to hold the theory of every line. It pays to hold the theory of the design and to leave the theory of each part with whoever made it, which is what McIlroy proposed. The costs that beat him in 1968 have all come down. To make a part, to make it again when it stops fitting, to look inside it when its abstraction leaks, as every abstraction does in the end, once took a programmer days and now takes a machine a small fraction of that. Even trust has got cheaper. A part that is cheap to make is cheap to test, and a part whose description has been tested can be used without being read and without knowing who wrote it. The upshot is that a program built of such parts has a theory small enough to hold again, since it comes down to the design and no more; and that, in Naur’s sense, is the difference between a program that is alive and one that merely runs.
References
- Dijkstra, E. W. (1969). Structured Programming (EWD268). E. W. Dijkstra Archive, University of Texas at Austin.
- Garlan, D., Allen, R., & Ockerbloom, J. (1995). Architectural Mismatch: Why Reuse Is So Hard. IEEE Software, 12(6), 17–26.
- McIlroy, M. D. (1968). Mass Produced Software Components. In Naur & Randell (Eds.), Software Engineering (see below).
- Naur, P. (1985). Programming as Theory Building. Microprocessing and Microprogramming, 15(5), 253–261.
- Naur, P., & Randell, B. (Eds.). (1969). Software Engineering: Report on a Conference Sponsored by the NATO Science Committee, Garmisch, Germany, 7–11 October 1968. NATO Scientific Affairs Division.
- Ryle, G. (1949). The Concept of Mind. Hutchinson.
- Spolsky, J. (2002). The Law of Leaky Abstractions. Joel on Software.
- Winters, T., Manshreck, T., & Wright, H. (2020). Software Engineering at Google. O’Reilly Media.
Photographs: a GIER console, 1974, public domain; below the Zugspitze, by PtrQs, CC BY-SA 4.0. Both via Wikimedia Commons.