Supporters of Marcus Endicott’s Patreon can access weekly or monthly consultations on this topic.
The phone call that woke Geoffrey Hinton came in the small hours of October 8, 2024, and reached him in a cheap California hotel room with no internet and a phone line that kept threatening to drop. He had been planning to get an MRI scan that morning and supposed now he would have to cancel it. He had no idea he had even been nominated. It took the caller's heavy Swedish accent to convince him the whole thing was not a prank. By the time the news had settled, the man who twelve years earlier had bet his career on the unfashionable proposition that neural networks might one day learn to see and speak had been awarded the Nobel Prize in Physics — a prize in a discipline he had abandoned, by his own admission, after his first year of university because he could not do the math.
The irony was almost too neat for nonfiction. The establishment was crowning him at the precise moment he had grown most afraid of what he had helped to build. To understand the dread you have to rewind eighteen months, to a spring morning in a dining room in Toronto.
In April 2023, Hinton, then seventy-five, told Google he was leaving. He had been there a decade, a vice president and engineering fellow, ever since the company acquired the tiny startup he had formed with two of his students after their network shocked the field by winning the ImageNet competition in 2012. The news broke in The New York Times on May 1 under a headline that fixed the story in the public mind: the godfather of AI was quitting and sounding the alarm. The framing was that he had left in order to criticize his employer. Hinton corrected it the same day, and the correction matters because the myth has proven stubborn. He had not quit in protest, he wrote; he had left so that he could speak about the dangers of the technology without weighing how his words might land on Google's business, and he added, pointedly, that Google had acted very responsibly. The distinction was not corporate politeness. It was the whole point. He wanted to warn the world without his warnings being read as the position of a company that paid his salary, and a man cannot do that, he said, while the company is paying his salary.
What he wanted to warn about, he had only recently come to believe. For most of his life Hinton had assumed that machines surpassing human intelligence were a problem for some distant century — thirty to fifty years off, he had thought, or longer. The new generation of large language models, and GPT-4 in particular, had changed his mind. He found himself reasoning about the arithmetic of it: the human brain carries something like a hundred trillion connections, while these models had at most a trillion, and yet GPT-4 already seemed to know hundreds of times more than any single person. The conclusion he drew was unsettling — that the machine might simply have a better way of learning than we do. Worse, he came to think, digital minds enjoyed an advantage no biological one could match. A thousand copies of the same model, sharing identical weights, learn as one; whatever any of them discovers, all of them instantly know. He took to describing the situation with a line that has stuck: it was as if aliens had landed and nobody had noticed, because they happened to speak very good English. He could imagine, he said, that humanity was just a passing phase in the evolution of intelligence.
At the emotional center of all this sat a single sentence, and it carried an old echo. Asked how he reconciled himself to his own work, Hinton fell back on what he called the normal excuse: if he had not done it, somebody else would have. It was nearly word for word the consolation that scientists have reached for since Los Alamos, and Hinton knew it; he had Oppenheimer on his mind, the technically sweet problem you solve first and argue about afterward.
He was not alone in his unease, but the three men who had shared the field's highest honor did not move together. Five years earlier, Hinton, Yann LeCun, and Yoshua Bengio had stood on the same stage to accept the Turing Award, the closest thing computing has to a Nobel, for dragging neural networks from the wilderness into the center of the discipline. By 2023 that trio was pulling apart in three directions. Bengio's turn was real but quieter than the legend suggests; he did not resign a post or follow Hinton out any door. He stayed in Montreal and turned to advocacy, signing the open letter that called for a six-month pause on the largest experiments, signing the one-sentence statement that ranked extinction from AI alongside pandemics and nuclear war, and writing essays about how rogue systems might arise and about the personal toll of changing his mind. When the BBC ran a headline saying he felt lost over his life's work, he objected that he had never said any such thing — a caution worth honoring, since the phrase has outrun the man.
LeCun went the other way entirely, and his dissent keeps the chapter from collapsing into one note. He stayed at Meta and called the existential fears preposterous, and later, blunter still, complete nonsense. Intelligence, he argued, has nothing to do with a desire to dominate; it is not even true of humans. Before fretting about a machine smarter than us, he liked to say, we would need the beginning of a hint of a design for something smarter than a house cat. It is worth being precise about where he drew his line: he did sign the call for a pause, but not the statement about extinction. His quarrel was with the apocalypse, not with caution as such.
The deeper irony is that the work the Physics committee honored was the very seed these gardeners had kept alive through the long winter. The prize went jointly to Hinton and to the physicist John Hopfield, whose associative-memory network in the early 1980s borrowed the mathematics of magnetized systems, and on whose foundation Hinton and Terrence Sejnowski had built the Boltzmann machine, named for the physicist of statistical mechanics. The committee praised Hinton for keeping at it even as the scientific community lost interest. Not everyone applauded the category. Many physicists and computer scientists complained that none of this was physics, and Jürgen Schmidhuber, ever the field's conscience on questions of credit, called the award a reward for misattribution, arguing that earlier pioneers had gone unrecognized. The objection deserves an airing, and so does Hinton's own bemusement at receiving the highest honor in a subject he had flunked out of.
The week made the establishment's surrender total. The morning after the Physics announcement, the Chemistry prize went half to David Baker for designing proteins from scratch, and the other half jointly to Demis Hassabis and John Jumper of Google DeepMind for teaching a machine to predict how proteins fold — a fifty-year problem dissolved. The committee chairs insisted the timing was coincidence, that no one had coordinated honoring artificial intelligence twice in two days. The denial was the most convincing proof of all that AI had conquered the citadel.
Hinton accepted the crown with a barbed grace. Among the students he was proud of, he noted at his celebration, was one who had fired Sam Altman — a rhetorical flourish more than a fact, since Ilya Sutskever was one of four board members who briefly ousted the OpenAI chief in late 2023, an ouster reversed within days and one Sutskever soon regretted. The line landed because of what it implied: that safety and profit had parted ways, and Hinton knew which side he was on. By the end of that year, asked whether his estimate of catastrophe had shifted, he put the odds of AI causing human extinction within three decades at ten to twenty percent, and offered the only comparison he could find. We have never before had to deal, he said, with anything more intelligent than ourselves.