Machine Intelligence is not Artificial - Part 4

Cybernetics and Norbert Wiener

If you asked me a few years ago if I knew what cybernetics was, I would have said "sure" and then rambled off something vague about computers and automata before diverting into discussing William Gibson, cyberspace, cybernetic organisms (cyborgs) and other cyber derivations. I didn't really know what it is, or more particularly, what it was. I suspect many of you readers are in a similar situation (though some are not, given the likely bias in readership on this topic). Given the importance of cybernetics on the development of the concept of machine intelligence, and given the gaps in our current version of artificial intelligence that has left many important aspects behind (see Part 2 for details), I felt it necessary to devote some time to dive into cybernetics and its early history (more recent history and current state will only briefly be touched upon here, though revisited in more detail in the future).

Cybernetics is, as Norbert Wiener's seminal 1948 book Cybernetics describes in its subtitle, the science of "Control and Communication in the Animal and the Machine". The complexity packed into this idea, and the book that first brought it to the world, is enormous. Like the proverbial blind men describing an elephant, many variations of description of the complexity of cybernetics exist. This figure from Novikov's CYBERNETICS: From Past to Future (2015) gathers a reasonable consensus on the key elements cybernetics initially drew from (Control Theory - 1868, Operations Research - 1943, Data Analysis & Decision Making - 1944, Mathematical Communications Theory - 1948) and spawned (Information Theory - 1948, Artificial Intelligence - 1956, General Systems Theory & Systems Analysis - 1968). Patricia Churchland and Terry Sejnowski have described it has the "precocious ancestor" of computational neuroscience in their textbook The Computational Brain, which is a succinct bridge to the role it may serve in the biological sciences in addition to the practices of mathematics, management, logic and computer engineering. I'm neither the authority on the topic, nor am I trying to sort out all of the nuance of it here. There are a number of great books that can give an overview which I will list in the bibliography in Part 6. Wiener's 1948 original Cybernetics being a great place to start.

My goals in this essay are 1) to note that something quite distinct and impactful preceded and influenced the field of AI in the decade or so leading up to the 1956 Dartmouth AI conference, 2) to show that the name cybernetics was for a time more so a formal field of inquiry than the sci-fi adjective meaning "computer-related" that has been its most popular common usage in recent decades; and 3) to dive into a bit of the history (and pre-history) of cybernetics to understand what aspects may have been lost as we moved into a more narrow exploration of AI as the primary focus of machine intelligence in the US and Britain in the late 1950s and early 60s. Cybernetics did gain traction and eventually take prominence in the Soviet Union following the death of Stalin in 1953. It also began to be of interest in China, India, Australia and various parts of Europe in the 1960s and later parts of South and Central America by the 1970s (notably, one of the early pioneers who worked closely with Wiener was a Mexican physician scientist Arturo Rosenbleuth, who moved between Harvard University and the University of Mexico in the 1940s, but I have not yet seen evidence of broader traction of the ideas in Mexico until later). This later era also saw the blending of cybernetics and broader social sciences and social movements worldwide by the 1960s into what has become known as cybernetics 2.0, at a time when the Cold War was impacting decision-making and funding on cybernetics and AI. We'll revisit that time period in more depth at a later time.

What is of particular note is not simply the existence of cybernetics as a discipline dating back to the 1940s, but the broad popularity Wiener's book and the topic received not only from scientists and engineers, but also the general public. His 1948 book went through four printings in the first 6 months and was a best seller, rivaling the contemporary Kinsey Report in popularity over the following decade. This was despite this book, ostensibly for a lay audience, being extremely dense with many chapters full of mathematical equations. One of the chapters in the original version, "The Computing Machines and the Nervous System," makes it clear that Wiener's mention of humans AND machines in the book's subtitle was pointing towards the comparison of these systems, making this book one of the early popularizers of the idea of "thinking machines" (but by no means the only one). This was two years before Alan Turing's 1950 paper "Computing Machinery and Intelligence," which gave us the Turing test for AI.

"It appears impossible for anyone seriously interested in our civilization to ignore this book." - Saturday Review on Cybernetics

What's in a name?

Cybernetics was not the first name given to this science of control and communication that captured the attention of some of the leading scientists, physicians, psychologists, engineers, and mathematicians in the US and the UK in the later days of World War II. Its origins were in the pairing of Wiener and engineer Julian Bigelow in 1940 to contribute to the US preparation for the war and aiding their allies (prior to this, Wiener had attempted to assist the war effort by sharing his ideas for an advanced computing device with Vannevar Bush, who was overseeing the war-related research effort; this was deemed not useful at the time but may have contributed to later advances in this area). They were tasked with solving the problem of how an anti-aircraft gunner could shoot down enemy aircraft, taking into account maneuvers by the pilot of that aircraft in the time it took the anti-aircraft shell to reach it. What resulted may have been the first computerized algorithmic prediction system in existence, though it was only demonstrated and never used in a real-world setting. The schematics for this "anti-aircraft predictor" were widely circulated amongst the military and its civilian support in 1942, but remained classified until several years later when it was published as a book, Extrapolation, Interpolation, and Smoothing of Stationary Time Series: With Engineering Applications, by Wiener in 1949. The third chapter of the 1948 Cybernetics, entitled "Time Series, Information and Communication," does seem to have some general similarities (notably, this chapter is also credited by some as having co-created the foundation of Information Theory and the idea of information as entropy, including by Claude Shannon, a former student of Wiener's, whose 1948 "A Mathematical Theory of Communication" has garnered him most of the credit for Information Theory. But that is a whole other story).

Wiener and Bigelow next partnered with Wiener's physician collaborator at Harvard, Arturo Rosenbleuth, on a 1943 paper, "Behavior, Purpose and Teleology," which would serve as one of the foundations for cybernetics according to Wiener, along with giving the topic its first name (teleology). Around the same time, Warren McCulloch and Walter Pitts, then at the University of Chicago, published their 1943 paper, "A Logical Calculus of the Ideas Immanent in Nervous Activity," which introduced the concept of an artificial neuron and the was the first description of an artificial neural network. Rosenbleuth had introduced Wiener to McCulloch at the Macy Conference on Cerebral Inhibition in 1942, kicking off a nearly decade-long collaboration between the two and their students at MIT until a personal falling out severed the relationship (see Part 1 for detail on this). Following some correspondence Wiener had with computing pioneer John Von Neumann at Princeton in 1944 on this topic of the science of communications and control in humans and machines, by then being referred to as "teleology" based on the 1943 paper, Wiener, Von Neumann, and Howard Aiken (computing pioneer from Harvard) hosted a small meeting on teleology at Princeton in January of 1945 along with the available co-authors from the two 1943 papers and several others. While notes from this meeting were limited due to the classified nature of some of the topics and the ongoing war, follow up letters between Wiener and Von Neumann discuss further meetings and work on the topic for this group, though they also indicate that the name teleology would be dropped due to its use in other contexts (the word has a religious connotation as well).

The group was able to get funding from the Macy Foundation, which funded a variety of conferences on cutting edge topics at the time. They next convened in March of 1946 with added partners including some from the social sciences (notably anthropologists Margaret Mead and Greg Bateson) to begin what was called from 1946-48 the Macy Conferences on "Circular Causal and Feedback Mechanisms in Biological and Social Systems." Following the publication of Wiener's book, Cybernetics, in 1948, the name was changed to the Macy Conferences on Cybernetics in 1949 and remained that until the last one in 1953. Retrospectively the 10 conferences held from 1946-1953 (two each in '46 and '47) are referred to as the Macy Cybernetic Conferences or sometimes the Macy Conferences (though the Macy Foundation also hosted conferences on other topics at the time, "Neuropharmacology" as well as "Group Processes", the latter of which included some overlap in topics and attendees with the Cybernetics conferences).

It was Norbert Wiener who came up with the new name, cybernetics, for this budding science of communication and control. It is a derivation of the Greek kybernetes meaning steersman or governor ("governor" is also a Latinized version of the same root). It appears to be an homage to physicist James Clerk Maxwell, whose 1868 paper "On Governors" is a classic of feedback control theory. Side notes: the similarly derived term cybernetique was used in 1834 by French physicist André-Marie Ampère to refer to "the science of the government of men." There is no indication that this political usage was an influence on Wiener's use of the term. The open-source (and Google originated) Kubernetes is also derived from the term cybernetics, along with the William Gibson usage of the term cyberspace in 1982 and a myriad of other cyber- using terms. For all of this, we have Wiener to thank.

"Kybernetes saves the souls, bodies, and material possessions from the gravest dangers." - Socrates (from Plato)

Norbert Wiener - A Strange Experiment

Norbert Wiener was a childhood prodigy turned philosopher and mathematician, he coined the term cybernetics and led an academic and engineering movement that spawned many aspects of AI, information theory, and even cognitive science. He also annoyed or fought with most everyone including the Soviets and the FBI, plus most of his colleagues. Nearly as famous as Einstein in his day, his contributions and his fame have been somewhat lost in history.

Wiener was a publicly noted child prodigy who published his first paper at 10, finished high school at 11, his Bachelors in mathematics at 14 from Tufts, his Masters in philosophy from Cornell at 17 (this was after briefly focusing on graduate studies in zoology at Harvard), and his PhD in mathematics from Harvard at 19. His thesis work, published in 1914 contributed to advancing Set Theory. In his youth he had been largely home schooled by his father, Leo Wiener, a professor of Slavic literature at Harvard who spoke 30 languages and was the first to translate many of Leo Tolstoy's works to English. Leo was convinced that learning was strictly a matter of the proper nurture and sought to emphasize this by regularly publicizing his prodigious son's early accomplishments while noting to the media that all of the credit for this was to go to his own unique (and intensive) teaching style. In his two (not one, but two) later autobiographies, including one entitled Ex-Prodigy, Norbert would go on to describe this upbringing in more detail, noting while all of his early success were supposedly attributable to his father's teaching approach, any early failures were his own. This strange upbringing as a public experiment most certainly contributed to his course in life in ways both obvious and non-obvious.

After his PhD, Wiener travelled to Europe and studied with mathematics legend Bertrand Russell at Cambridge amongst others before returning to the US. He guest lectured at Harvard but struggled to find a permanent position there. In 1919, after a brief period in the US military during World War I, he settled into teaching and then a tenure track role in the mathematics department of what then was a small, backwater technical school in Cambridge, MA called Massachusetts Institute of Technology (MIT). He would spend the rest of his illustrious career there, growing the department and the school with him.

Wiener was an accomplished mathematician even prior to his work on prediction algorithms and cybernetics described above. Some of his most notable advances were in the mathematics of Brownian motion. He was also a patriot, attempting to enlist in the US Army three times during WWI (twice rejected for physical reasons), before succeeding towards the end of the war. While his period in the military was brief, his assignment to study ballistics at Aberdeen Proving Ground gave him early exposure to the mathematics that would play a role in the anti-aircraft predictor he developed with Bigelow during WWII. This foundation set the stage for the rapid rise in popularity he and his ideas would see decades later in the 1940s and into the 1950s.

Walks like a duck

Two other factors besides his life's work contributed to Wiener's notoriety during his lifetime - he was a peculiar character by all accounts, and he was also very outspoken as an anti-military critic following WWII. He was a somewhat short and rounded man, who was frequently compared to a waddling duck when he walked around the MIT campus, which he did often. He was known for long, daily "Wiener Walks" which had him stroll seemingly aimless around the campus, puffing on his ubiquitous cigar while leaning back as he walk looking to stare off into the sky. The lean was actually him using slightly shortened glasses as ad-hoc bifocals, he would lean back and look under them into the distance as he walked. But the effect, along with him randomly giving audience to those that might be with him or gather around him during the walks, was that of a deeply ponderous professor. Sometimes these walks included brief but brilliant visits to colleagues, as in one story from another MIT professor who recalled Wiener walking into his office, standing silent for a few moments puffing on his cigar, declaring "information is entropy!", and then walking out the door again without another word spoken between them.

On top of that, he also gained a probably deserved reputation for being a stereotypical absent-minded professor. One story has him standing in the coatroom of the faculty lounge after lunch, where people simply thought he was thinking about something. Turns out he had forgotten which jacket was his and was waiting to see which one remained when the rest of the faculty left. Another incident, partially confirmed by his daughter, had him returning home only to find his keys did not work in the door to his house. He asked some kids playing nearby if this was where the Wieners lived, only to have one of the kids reply that she was his daughter and they had moved, but her mother had told her to wait for him and let him know where to go. His daughter has verified this occurred, though noted that unlike some versions, he did at least recognize her as his daughter.

The brilliant and comedic character aspects aside, his outspokenness likely contributed even more to his fame and notoriety in his lifetime, though this and his stubborn quick temper may have also led to his posthumous fall into a much lesser level of prominence than others he worked with and influenced. Despite his significant war time efforts in WWII (and significant effort to be able to contribute in WWI), he became extremely wary if not hostile to the idea of scientific collaboration with the military after the bombing of Hiroshima and Nagasaki at the end of WWII. This first took the form of conflicted dialogue with his colleague Von Neumann, who had worked on the Manhattan Project and wanted the atomic bomb to be considered against the Soviets pre-emptively (and was later parodied as Dr. Strangelove in the movie of the same name). This conflict didn't end their relationship (the continued working together on the Macy Conferences) but may have contributed to Von Neumann passing on an offer to join Wiener as a professor at MIT. Wiener also began refusing Department of Defense (DoD) funding and later began criticizing any federally funded research (though he did still continue to get non-DoD federal funding). His concerns became public in 1947 when after declining to share research papers with an aircraft company that had requested them in the context of defense research. He shared the letter with the Atlantic, which published it under the title, "A Scientist Rebels".

This did not go unnoticed, and Wiener ended up on an FBI watchlist for un-American activity. It did not help his case that he was also meeting with Soviet colleagues while travelling overseas, just as the Cold War was heating up. Meanwhile in the late 40's and early 50's, the Soviets viewed cybernetics as Western propaganda (this would change after the death of Stalin in 1953), and Wiener criticized the Communists for this along with their overall authoritarianism, such as never letting their scientists meet with him without handlers. Wiener was making enemies, foreign and domestic, at the same time his fame was going to the moon.

In 1950, Wiener published "The Human Use of Human Beings: Cybernetics and Society" (1950), a cautionary book about the negative impact technology might have on society. It's final chapter in the original edition, "Voices of Rigidity" - which was scrubbed from all later versions - outlines his concerns with groups using technology for societal surveillance and manipulation (he had these particular concerns about the FBI, the Soviets, the Catholic Church, and corporations - in no particular order). His foresight was remarkable, but this only ramped up the scrutiny he was under.

It was under this pressure, along with struggles to get his first autobiography published, when Wiener made the snap decision to cut off all contact with McCulloch, Pitts and the others at MIT (more detail on this in Part 1). He continued to collaborate with Rosenbleuth in Mexico and spent more time speaking on cybernetics in Europe and Asia until his death in 1964. He stopped attending the last of the Macy Conferences - though he was invoked there by his former colleagues. And despite having worked with or influenced many of the attendees, he was notably not invited to the Dartmouth meeting on AI in 1956. Was this because the organizers saw his work outside of their narrowed, engineering view of machine intelligence? Was it because they wanted to avoid the shine and hype of cybernetics? Or was it simply that some of them didn't like him? (Pamela McCorduck in her book Machines Who Think hints that John McCarthy may have wanted to avoid cybernetics or Wiener himself. I am still looking for more detail on this in her interviews with McCarthy from 1974-75. You'll find more detail on McCorduck and her book in Part 3).

It isn't clear what compounding riffs may have occurred in the broader machine intelligence/information processing community in the 1950s, but it is clear that the outcome in the US was that the (mostly DoD) federal funding in this area began to go primarily to AI rather than cybernetics, at least directly. The word "intelligence" seems to have resonated with those in the US defense and intelligence community more so than cybernetics did by the late 1950s. Meanwhile the Soviets began going all in on cybernetics, not only as the science of control and communication in animals and machines, but also society - this latter piece becoming known as Cybernetics 2.0. This social aspect also took off in England in the 60s, along with the counter-culture movement in the US. This seems to have created its own negative feedback loop between the two countries, furthering the AI and cybernetics divide, at least until the late 60s when the US realized it was falling behind in global cybernetic supremacy and the CIA appears to have helped found the American Cybernetics Society in response. There's a lot more to this period than I'm going to go into now. Suffice to say, the 60's got a little weird.

Next up: Part 5 - The Ratio Club and British Cybernetics

(Thank you to the CMU Archives staff for helping me explore the Pamela McCorduck archives of her interviews and notes from her 1979 book Machines Who Think. Thanks also to the MIT Archives for making the Norbert Wiener collection of his papers and correspondence available, along with Ethan Vodovotz for helping to transcribe and make searchable some of the handwritten letters in the MIT Wiener collection).