The Heretic's Execution

In the summer of 1951, a graduate student at Princeton spent a few thousand dollars of Navy money on vacuum tubes, surplus motors, and clutches, and assembled them into a machine of forty artificial neurons. He called it the SNARC, the Stochastic Neural Analog Reinforcement Calculator, and when he switched it on it did something genuinely strange for a tangle of tubes and gears: it learned. Set loose on the abstract problem of a rat finding its way through a maze to food, the contraption adjusted itself, reinforced the paths that worked, and got better. It is remembered today as the first artificial neural network machine ever built. The young man who built it, with his friend Dean Edmonds, was Marvin Minsky. Eighteen years later, that same man would put his name to the book that history remembers as the one that strangled neural networks in their crib. This is the first thing to understand about the execution of the connectionist dream: the executioner began as one of its inventors.

The man cast as the victim, Frank Rosenblatt, had known Minsky since adolescence. They were near-contemporaries at the Bronx High School of Science, Minsky a grade ahead, and they would spend their careers circling each other like duelists who had grown up on the same street. Rosenblatt was a psychologist by training, which mattered more than it should have, because to a mathematically minded crowd it marked him as a soft-science interloper poaching on hard territory. He was also, by every account, a man of extravagant range: an astronomer who built his own observatory and dreamed up techniques for detecting planets around other stars, a musician, a sailor, a campaigner for Eugene McCarthy. And in 1958 he published, in the Psychological Review, a model he called the perceptron, the first artificial neural network ever realized in hardware. It was a machine that could be shown examples and, without being explicitly programmed, learn to tell them apart.

What happened next was a three-way collision of ambition that has been flattened, ever since, into a simple morality tale. The Office of Naval Research called a press conference. The wire services carried it, and on July 8 a newspaper reader could open the paper to learn that the Navy had unveiled the embryo of a machine it expected would one day walk, talk, see, write, reproduce itself, and be conscious of its own existence. That breathless inventory belonged to the Navy's hopes and the reporters' appetites, not to any equation Rosenblatt had written. He had made bold claims of his own, to be sure, calling the perceptron the first machine capable of having an original idea, but he was also appalled by what the press did with it. Years later he wrote, with real bitterness, that the popular coverage had been handled with all the exuberance and discretion of a pack of happy bloodhounds, and he singled out one headline in particular that announced a Frankenstein monster designed by the Navy. The familiar story in which a vainglorious Rosenblatt oversold his invention and earned a comeuppance simply does not survive contact with the record. He hyped and he objected to the hype, often in the same breath.

The hardware itself, the Mark I Perceptron, arrived a little later than the legend suggests, demonstrated in 1960: a grid of four hundred photocells standing in for an eye, wired more or less at random to a layer of five hundred and twelve association units, with the machine's hard-won knowledge stored as physical settings in motor-driven dials. It worked. And it had a real and stubborn limitation, one that the eventual critique would make famous. A single layer of these units, however large, cannot learn certain patterns; the textbook example is the simple logical operation called exclusive-or, which cannot be separated by a single straight line no matter how the line is drawn. Rosenblatt knew about the limitation. He had written about deeper, multi-layered systems himself. What he lacked was a method to train them, and that absence would define the next two decades.

Into this gap stepped Minsky and his collaborator Seymour Papert, whose 1969 book Perceptrons is the supposed murder weapon. The book is a work of real mathematics, and on its own terms it is correct and uncontested to this day. It proved, with rigor, that a particular restricted class of single-layer perceptrons could not compute certain functions, and that the resources required for others grew impossibly with the size of the problem. The trouble has never been with what the book proved. The trouble is with one sentence about what it did not prove. When Minsky and Papert turned to the multi-layered networks that Rosenblatt had gestured toward, they did not demonstrate that such networks were doomed. They wrote, instead, that they considered it an important research problem to elucidate or reject their intuitive judgment that extending the work in that direction would be sterile. They flagged it explicitly as a hunch. They even invited the field to prove them wrong, speculating that some powerful theorem might yet be found. A conjecture, an honest one, dressed in tentative language, was read by a generation as a verdict.

And so the myth hardened: that this one book killed neural networks until their resurrection in the mid-1980s. The historian Mikel Olazaran, who interviewed the survivors, tells a more honest and more interesting story. The official version, he argues, was a retrospective construction, a tidy narrative assembled after the fact to justify the triumph of a rival approach. The truth is that the decline had many fathers. The deepest problem was technical and predated the book entirely: nobody yet knew how to train the deeper networks, so the field had stalled on its own. Symbolic artificial intelligence, the camp that worked with logic and rules rather than brain-inspired learning, had meanwhile captured the lion's share of defense funding, and a 1969 amendment redirecting military money toward immediately useful research squeezed the whole exploratory enterprise. Most of the researchers had already drifted away before Perceptrons appeared. The book did not empty the room. It walked into a room that was already nearly empty, and it explained, persuasively, why the people who had left had been right to leave.

Nor did it destroy Frank Rosenblatt, the other durable myth. By the time the book came out he had moved on years earlier, into neurobiology, chasing the chemistry of memory itself by injecting brain extracts from trained rats into untrained ones to see whether knowledge could be transferred as a substance. A colleague who watched him lose much of his government funding said he never seemed bitter. He was simply somewhere else, reaching, in another colleague's phrase, for the biggest problems he could see. The real pathos of his story is not that a book broke him. It is that he died before he could be vindicated. On a Sunday afternoon in July 1971, sailing on the Chesapeake Bay, he drowned. It was his forty-third birthday; he had been born and would die on the same July day.

He did not live to see the rest. In 1986, the method he had lacked finally arrived, the technique now called backpropagation that made it possible to train the deep, multi-layered networks he had only been able to imagine. The sterile direction turned out to be the only one that mattered; nearly every machine that now writes and sees and talks descends from it. Minsky's intuitive judgment had been, in a later generation's gentle phrase, too hasty. The two men had never been personal enemies, whatever the spectacle of their public clashes suggested, and there is a quiet coda buried in the bibliography of the very book that supposedly buried Rosenblatt. When a later printing of Perceptrons went to press, it carried a new line on its dedication page, three words offered by the man history calls the executioner to the man it calls the victim: in memory of Frank Rosenblatt.

=> The Reports