Machine Intelligence is not Artificial - Part 7

Interlude 2, Electric Boogaloo

As I mentioned in the Introduction, the first 6 parts of this essay series, “Machine Intelligence is not Artificial” (MIINA) have been slightly modified versions of similarly numbered parts that first appeared as articles on my LinkedIn. Given that I used portion of the original LI version of the Part 7: Interlude essay in the Introduction to this series on Substack, this is where we begin to diverge into new territory. Moving forward, new material for the MIINA series as well as other items that fall under Unjournaling related topics will be based here and sometimes mirrored or linked at LI and elsewhere.

To summarize some of the key things on our journey so far:

My hypotheses - With respect to machine intelligence, 1) we forgot some things, 2) those things will likely stay forgotten for a while, and 3) those things could help us now.

1)We forgot some things. There were broader interdisciplinary elements relating to machine intelligence (and its cybernetics precursor) that got left behind or underexplored when the field and funding primarily narrowed to the engineering focused artificial intelligence John McCarthy envisioned in the 1956 Dartmouth meeting and its aftermath.

2) Those things will likely stay forgotten for a while. Because of a series of unfortunate events in the short term (4 Funerals and a Divorce - see Part 1) and long term (funding and political pressures touched on briefly and to be explored more later), this narrowed vision became the norm for the field and funding, which combined with the tech/engineering focus forward not back, made the loss unlikely to be corrected in the near future without deliberation.

3) Those things could help us now. Revisiting these older interdisciplinary approaches to machine intelligence may yield component, concepts, and approaches that can be utilized for better machine intelligence/complex information processing/augmented human problem solving than we can achieve with only the one-dimensional approach we are primarily using today (note, some seem to be confused that this is an either-or situation, or that me saying we can do better means I’m denying current progress - this is not the case and is your own hallucination if that is what you are concluding).

The categories of machine intelligence - what Allen Newell referred to as “information-processing research” was broken into five approaches in his 1957 doctoral dissertation (with his examples):

1. "...practical workers who deny any connection between their work and a science of human behavior, but who take an engineering approach to some particular task that needs mechanizing." Here he mentions those working on problems like "mechanical translation of languages" and "machine literature searching" as examples.

2. "...a group concerned with pure artificial intelligence. They too prefer to disclaim any immediate relation to behavioral science, but are working directly to synthesize systems that will show as much of the higher human intellectual functions as possible." Here he mentions Alan Turing, "Computing Machines and Intelligence" (1950) and Claude Shannon (a Dartmouth meeting organizer), "Programming for a Computer to Play Chess" as two good examples.

3. "A third, rather diverse, group may fairly be called the cybernetic group. Here the fundamental object is to construct a science of human behavior. The point of departure is physiology..." Here he mentions the Homeostat of R. W. Ashby and the mechanical turtles of Grey Walter as some of the direct work on automata that belong to this group, while also noting, "the more directly physiological work of [Warren] McCulloch and [Walter] Pitts" as well.

4. "A small group can be separated from the cybernetics group by its attitudes towards digital computers. These investigators share in common with the cyberneticists a concern with the science of human behavior at the level of physiology and its first behavioral correlates. However, they use the digital computer as an analytical device for discovering the consequences of various theories, formulated as sets of interacting mechanisms." Here he mentions Nathaniel Rochester (another Dartmouth meeting organizer) for his Hebbian models of the nervous system, "Tests in a Cell Assembly Theory of the Action of the Brain, Using a Large Digital Computer" along with Oliver Selfridge and Gerald Din[n]een for their, "Pattern Recognition and Modern Computers".

5. "A group which I would call the information processing group...concerned with the science of human behavior, but the point of departure is at the social and cognitive level. Also, the computer is viewed as a consequences-generating device, and not as a model of human behavior." Here he mentions his group, with Simon and Shaw, as the main active participants.

Or more simply:

1. Engineering

2. Artificial Intelligence (Alan Turing and Claude Shannon)

3. Cybernetics (Ross Ashby, Grey Walter, Warren McCulloch, and Walter Pitts)

4. Cybernetics + digital computing (Nathaniel Rochester, Oliver Selfridge, and Gerald Dinneen)

5. Information processing (Allen Newell, Herb Simon, and Cliff Shaw or NSS as their group was commonly known)

And as I noted in Part 2, these can be mapped to today with some splitting and bundling (this is a work in progress - refinement will be presented in the future):

Mapping these groups across time could be the work of an entirely separate thesis, but we can trace the broad strokes. The 1) engineers and 2) AI engineers have largely merged while adopting a cartoon simple version of the physiology represented in "neural network" of McCulloch and Pitts (3) that now underlies the impressive fill-in-the-blank feats of current large language models. While the automata portions of the 3) cyberneticists have lineage to today's robust robotics efforts, the more detailed exploration of the physiology of 3) and 4) has largely been forgotten except for in the relatively small (compared to AI) realm of neuromorphic computing. The neuronal and synaptic complexity that has been known for more than a century… has still to be considered by much of the AI and artificial neural net community, nor has the deeper functional complexity Von Neumann warned about and the realization of the analog/digital hybridization of the brain that devastated Pitts (see Part 1) really been addressed. Add to this the neurotransmitter and channel complexity and genomic and epigenomic variance in their expression in a single neuron across a lifetime, and you get a brain architecture and physiology complexity (roughly 10^40 different brain states) that has barely been scratched by even the most advanced neuromorphic architectures.

John McCarthy and Marvin Minsky, the two junior organizers of the Dartmouth AI conference only a year before his dissertation where Newell and Simon presented the only functional AI program, are not mentioned in his taxonomy. The two more senior organizers of the conference, Shannon and Rochester, are mentioned and categorized. From the influences and reactions that led McCarthy, who was the main organizer, to scope and put forth the Dartmouth AI conference (which I went into detail about in Part 6) - I would argue that he was largely in category 1 with influences from 2 while trying to reject or marginalize 3 and 4 to a lesser degree. Recall the description he put in the proposal about his, “…conjecture that every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it.”

No one has yet done a very good job of developing these requirements.

McCarthy’s focus on AI was not seeking to emulate intelligence, merely individual features of it. He was expressly looking for low-hanging fruit in the form of tasks related to features of intelligence that could be simulate by machines - an engineer with aspirations of only a subset of Turing’s vision for machine intelligence. This is what largely took hold, and mostly what was funded, in the AI early days of late 50’s through the early 70’s. Some cybernetics approaches continued, but these were limited and dried up and died off as the original practitioners (mainly Norbert Wiener, Warren McCulloch and Heinz von Foerster) passed away or retired. The NSS group continued as well, focused on specific features of intelligence but considering the broader psychological, philosophical, and business administration context of human problem solving and complex information processing. But while Newell tried to “precisely describe” the integrated functions of cognition by the early 90s with SOAR and his book Unified Theories of Cognition, his integrative approach was distinctly in the minority of machine intelligence research.

By the 90’s, the decades long primary focus on symbolic AI was shifting instead to the newly resurgent connectionist approach that has largely dominated since and given us the deep learning and large language models of today. While computing speed advances played a significant part in this rise in the neural nets of today, there is still a lesson to be learned in having let these valuable approaches sit mostly on the sidelines from a funding and focus perspective for 30 years. Conversely, the near abandonment of focus on symbolic AI today may also be regretted as lost time (and lost knowledge needing to be regained) at some point in the future. Figuring out how to both diversify our overall research portfolio into approaches, while also maintain a systems thinking perspective on integration of multiple features of machine intelligence is something that could use more attention. I head up a new section, “Systems Concepts, Theory and Policy in Biology and Medicine” at Frontiers in Systems Biology where these ideas are going to be entertained and explored in a more rigorous way.

All of this, and the factors, events, and people that drove these various directions, are topics we will be getting to in more detail. At present, this series is meant to sweep broadly across a lot of material with brief dives into some things that I’m finding that don’t seem to have gotten much attention. One of these, the story of the Simulmatics Corporation, is so strange and seemingly underreported for its impact, that I’m going to give it its own spin-off mini-series next before we return to this MIINA series.

Next: The Simulmatics Corporation, The Vietnam War and the First AI Winter (teaser here)

References and Recommendations

All of the books below contributed in some way, even if it was small or contextual, to my research in this series and are worth reading. Those books that have been most critical to the details I have covered are marked with **. If curious to learn more, start there. This list is not exhaustive and will grow as the topics expand.

I would also like to note the resources at the CMU Archives (McCorduck, Traub, Simon, and Newell collections) and the MIT Archives (Wiener collection) that have been critical to this ongoing work.

Ashby, W. Ross. Design for a Brain. 1952, 1960.

Brockman, John (Editor). Possible Minds: 25 Ways of Looking at AI. 2019.

Churchland, Patricia and Sejnowski, Terrence J. The Computational Brain. 1992.

** Conway, Flo and Siegelman, Jim. Dark Hero of the Information Age: In Search of Norbert Weiner the Father of Cybernetics. 2005.

Feigenbaum, Edward A. and Feldman, Julian. Computers and Thought. 1963

Fuller, R. Buckminster. Critical Path. 1981.

George, F.H. Cybernetics. 1971.

Gleick, James. The Information: A History, a Theory, a Flood. 2011.

** Heims, Steve. John Von Neumann and Norbert Wiener: From Mathematics to the Technologies of Life and Death. 1980.

** Heims, Steve. The Cybernetics Group. 1991.

** Husbands, Philip, Holland, Owen, and Wheeler, Michael (Eds.). The Mechanical Mind in History. 2008.

Kelly, Kevin. Out of Control: The New Biology of Machines, Social Systems, and the Economic World. 1994.

** Kline, Ronald R. The Cybernetics Moment: Or Why We Call Our Age the Information Age. 2015.

Levy, Joel. Frankenstein, Mary Shelley and the Birth of Science. 2018.

Malapi-Nelson, Alcibiades. The Nature of the Machine and the Collapse of Cybernetics: A Transhumanist Lesson for Emerging Technologies. 2017.

** McCorduck, Pamela. Machines Who Think. 1979.

** McCorduck, Pamela. This Could Be Important: My Life and Times with the Artificial Intelligentsia. 2019.

McLuhan, Marshall. Understanding Media: The Extensions of Man. 1964.

McNeely, Ian F. with Wolverton, Lisa. Reinventing Knowledge: From Alexandria to the Internet. 2008.

Morrison, Philip and Morrison, Emily (Editors and Introduction). Charles Babbage and his Calculating Engines: Selected Writings by Charles Babbage and Others. 1961.

** Newell, Allen and Simon, Herbert. Human Problem Solving. 1976.

Newell, Allen. Unified Theories of Cognition. 1990.

Nilsson, Nils. Principles of Artificial Intelligence. 1980.

Novikov, D.A. CYBERNETICS: From Past to Future. 2015.

Pias, Claus. Cybernetics - The Macy Conferences 1946-1953: The Complete Transactions. 2015.

** Pickering, Andrew. The Cybernetic Brain. 2010.

Pierce, John R. An Introduction to Information Theory: Symbols, Signals and Noise. 1980.

Singh, Jagit. Great Ideas in Information Theory, Language, and Cybernetics. 1966.

** von Neumann, John. Computer and the Brain, 2nd Edition. 2000.

Weizenbaum, Joseph. Computer Power and Human Reason. 1976.

** Wiener, Norbert. Cybernetics: Or control and Communication in the Animal and Machine. 1948, 1961, 1965, 1985.

** Wiener, Norbert. The Human Use of Human Beings: Cybernetics and Society. 1950 - Note: Reprinted in 1954 and 1989 without final chapter “Voices of Rigidity”.

Wiener, Norbert. A Life in Cybernetics - Ex-Prodigy: My Childhood and Youth & I Am a Mathematician: The Later Life of a Prodigy. 2017 (1953 & 1956).

Wiener, Norbert. God & Golem: A Comment on Certain Points Where Cybernetics Impinges on Religion. 1963.