The Translation Memo

In the summer of 1949, a mathematician on vacation in Carlsbad, New Mexico, sat down to commit a strange idea to paper. Warren Weaver was no obscure crank; he ran the natural sciences program at the Rockefeller Foundation, had directed a sprawling wartime mathematics effort, and would later coin the phrase "molecular biology." But the notion he wanted to set down was one that most serious people of his era considered faintly absurd. He believed a machine could translate between human languages, and he believed it because, two years earlier, he had described to a friend exactly how the trick might be played. "When I look at an article in Russian," he had written, "I say, 'This is really written in English, but it has been coded in some strange symbols. I will now proceed to decode.'"

That single sentence is the hinge on which an entire history turns. The friend Weaver had written to was Norbert Wiener, the founder of cybernetics and one of the most formidable minds of the century, and Wiener's response had been a polite door closing in Weaver's face. The boundaries between words in different languages were too vague, Wiener replied, the emotional and cultural connotations too sprawling, to make any mechanical translation scheme hopeful. The whole enterprise was, he judged, premature. Weaver pushed back gently — even a vocabulary of two thousand words combined in pairs yielded only four million possibilities, hardly a frightening number for a modern computer — but the great man was unmoved. Weaver later called the exchange "exceedingly discouraging," precisely because if anyone should have seen the possibilities, it was Wiener. The amateur administrator turned out to be right, and the genius wrong, and that quiet inversion is the kind of irony history rarely arranges so cleanly.

What gave Weaver the nerve to disagree was the war he had just lived through. Running the Applied Mathematics Panel had thrown him among code-breakers, and code-breaking suggested a tantalizing analogy. A wartime story he had heard, and could never quite verify, had lodged itself in his imagination. A mathematician he would only call "P" had written out a hundred words of Turkish, stripped of its special letters, and enciphered them into columns of five-digit numbers. A colleague who knew no Turkish, and did not even know the message was Turkish, decoded it the next day using nothing but the statistical fingerprints of the text — the frequencies of letters, the gaps between them, the recurring patterns. If the deep structure of a message could survive being scrambled into numbers, Weaver reasoned, then perhaps something invariant lay beneath all human languages, a common signal that no particular tongue could fully obscure. He was building on more than war stories. He had read Claude Shannon's new theory of communication, even Shannon's still-classified work on the mathematics of secrecy, and he saw in them a way to make the intuition rigorous. Translation, he insisted, was at bottom a statistical problem. Perfect translation was almost surely impossible; translation that erred only some measurable fraction of the time was almost surely within reach. Statistical studies of meaning, he wrote, were not a luxury but a necessary first step.

He laid out the case in a short memorandum he titled, with no flourish, simply "Translation," and circulated it to a few dozen acquaintances across the sciences — twenty or thirty, by his own later recollection, though the figure has a way of swelling to two hundred in the retellings. The memo offered several lines of attack, and two of its images have outlived everything technical in it. The first was a meditation on context. Imagine reading a book one word at a time through a hole in an opaque mask, Weaver wrote. The word "fast" floats there, meaning nothing in particular: rapid, or fixed in place, with no way to choose. Widen the slit to take in a few words on either side, and the ambiguity dissolves. Meaning lives not in the isolated word but in its neighborhood — an idea that, decades on, would become the context window of every statistical language model and, eventually, the attention span of the systems that write fluent prose today.

The second image was grander and more lyrical. Picture people living in tall closed towers raised over a shared foundation. When they shout to one another from the tops of their towers, the sound barely carries, and communication limps along; this is what it is to translate directly from one language to another. But descend the tower, Weaver wrote, and you reach a great open basement common to all of them, the real if undiscovered universal language beneath human speech. The way to get from Chinese to Arabic was not to shout across the rooftops but to climb down into that shared basement and back up by another stair. The towers metaphor seeded the dream of an "interlingua," a neutral intermediate representation, and it gestures, more than half a century early, at the shared multilingual spaces inside modern neural networks, where sentences in dozens of languages are pressed into the same geometry of meaning.

The memo did its work. Weaver's institutional weight, as much as his arguments, turned an eccentric idea into a funded field; within a few years the United States had its first full-time machine-translation researcher, its first conference, and a splashy 1954 demonstration in which an IBM machine rendered a few dozen Russian sentences about chemistry into English and prompted confident predictions of fluent translation within five years. The hype was wildly premature, as it so often is. And here the story bends away from its author. The discipline Weaver launched promptly abandoned his statistical instinct and went the other way — toward hand-built dictionaries, hand-written grammars, the painstaking codification of rules. The cryptography analogy, taken literally, was simply wrong; a Russian text is not encrypted English, and no English ever passed through the mind of the person who wrote it. Researchers seized on that flaw, declared the decode idea a category error, and moved on. Weaver's deeper point — that the analogy was wrong in the letter but right in the mathematics — went unheard. The statistical seed lay in the ground, unwatered, for forty years.

When it finally germinated, the people holding the watering can said so themselves. Around 1990, a group at IBM led by Fred Jelinek revived exactly the approach Weaver had sketched, formalizing translation as a noisy channel: model the source language, model the corruption that turns it into the foreign string, then run the corruption backward to recover the original. The bilingual transcripts of the Canadian parliament gave them millions of matched sentence pairs to learn from, and the computers Wiener had implicitly doubted were finally fast enough to count. The team opened their landmark paper by naming Weaver outright. Machine translation was almost as old as the digital computer, they wrote; in 1949 Weaver had proposed attacking it with statistics and information theory, researchers had quickly abandoned that path on theoretical grounds, but the true obstacles had not been theoretical at all — they had been the weakness of the machines and the scarcity of digital text. It is a rare thing to watch a field reopen a door its founder had been pushed away from, and credit him by name as they walk through.

There is a final grace note, and it belongs to the man rather than the method. Weaver was a lifelong devotee of Lewis Carroll, and he assembled one of the world's great collections of Alice in Wonderland in translation — roughly a hundred and sixty editions across more than forty languages. Ever the quantifier, he could not simply admire them; he devised a scheme to grade how each version handled Carroll's puns and nonsense and parodied verse, recruiting an improbable jury that included the anthropologist Margaret Mead and a Nobel-winning biochemist. The same mind that proposed decoding Russian as scrambled English spent its leisure cataloguing exactly how Alice survives the passage from one tongue to another, and where she is lost in the descent. He called the work building a Tower of Anti-Babel — not to charm or delight, he wrote, but to carry the essential content of a text across the gulf between languages. He never saw the field prove him right. But the gulf he spent his life staring into is the one our machines now cross a hundred times a second, and they cross it, more or less, by the route he drew.

=> The Mirror