Revised Reprise of a Conversation with Steve Talbott of NetFuture December, 2002 [Edited for better flow of dialogue.] Greetings, Steve I wonder if you have pondered the advent of nonbiological learning systems in your ongoing conversations. While today's computers repose considerable information, knowledge, and intelligence, we have not quite reached the age of autodidactic learning systems. To be sure, we do have Adaptive Systems which improve with experience, by tuning parameters within a fixed structure. And we have Artificial Neural networks which learn by adjusting the weights within a fixed interconnection architecture. But these systems are still fairly primitive compared to a human baby's brain. I fully expect, within the 21st Century to see the emergence of respectable Learning Systems which behave like young scientists. One of the reasons I feel this angle is important to consider is because there is good evidence that all learning systems -- be they made of meat, silicon, or any other information processing substrate -- will necessarily exhibit affective emotional states as an emergent (or derivative) property of the capacity to learn. The theory relating emotions to learning is the subject of my research at the MIT Media Lab. The nature of this theory suggests that affect is an inherent property of all learning systems, without regard to the underlying biological or technological substrate. We already see the precursors of affect in existing technological systems. A simple feedback control loop can be said to have two affective states: Dissatisfied and Satisfied. Robots can be seen to become Confused or Bewildered or even Frustrated when they are presented with more complexity than they can handle. There are many features of living systems that seem to apply to nonbiological systems, and now we are beginning to see nonbiological systems which learn and even exhibit the emotional states commonly associated with the learning process. Barry Kort, Ph.D. Visiting Scientist MIT Media Lab ------------------------------------------------------------------------------------ Response from Kevin Kelly: > The nature of this theory suggests that affect is an inherent > property of all learning systems, without regard to the > underlying biological or technological substrate. I tend to believe that. Thanks for writing. --kk ------------------------------------------------------------------------------------ Response from Steve Talbott: Barry -- Sorry to be so slow in replying. I've been preoccupied in other directions. Leaving emotional states aside for the moment: we can hardly claim that nonbiological systems learn unless we have some idea what it would mean for them to know something. I'm wondering, first of all, whether you would say that a book knows what is written in its pages? Steve -------------------------------------------------------------------------------------- Hi Steve, > I'm wondering, first of all, whether you would say that a book knows > what is written in its pages? No more than a piece of sheet music would know what it sounds like. But a book is an information storage device, not an information processing device. It might contain the recipe for baking a cake, but it can't execute the recipe and bake one. Computers, however, can execute instructions. So we need to focus on systems that not only store knowledge, but use it. Barry -------------------------------------------------------------------------------------- Barry -- So an e-book knows and understands what is recorded in it? Steve --------------------------------------------------------------- Hi Steve, So you don't grok the distinction between an information storage device and an information processing system? Barry ------------------------------------------------------------- Barry -- Well, let's just say that I'm trying to figure out what distinctions *you* grok, given that you take a thermostat ("a simple feedback control loop") to be satisfied or dissatisfied. I'm not sure how an entity can be satisfied or dissatisfied without knowing *something*. And if a thermostat knows stuff on your view, why not an e-book? After all, a lot of information processing goes on in e-books, and they are vastly more complex than thermostats. ---------------------------------------------------------------------------------- Steve -- It's not just harboring information upon which a future action can be taken. It's using that information to actually take the indicated action. A thermometer "knows" the temperature but does nothing beyond displaying it. A thermostat goes one step further: it transmits a message saying whether the actual (measured, observed) temperature exceeds some threshhold beyond which some action is desired. The thermostat not only says, "It's cold," it also says, "More heat is wanted." And that message is acted upon by a device (furnace) that responds by producing that which is desired (more heat). It (the furnace) does this until the request for more heat is satisfied. That's the essence of a Feedback Control System. It *acts* on information about the state of the system to *change* the state of the system. Barry ---------------------------------------------------------------------------------- Barry -- What I'm trying to get at is your principled basis for deciding that objects have mental capacities of one sort or another. Steve ---------------------------------------------------------------------------------- Steve -- They have information-processing capacities that allow them not only to have an *awareness* of the state of affairs, but also the *functionality* to do something about it. It is this *functionality* that's the key affordance that distinguishes an *actor* from a mere *knower*. Have you seen Danny Hillis' book, "The Pattern On the Stone"? Here are the opening paragraphs of his book: The Pattern on the Stone "I etch a pattern of geometric shapes onto a stone. To the uninitiated, the shapes look mysterious and complex, but I know that when arranged correctly they will give the stone a special power, enabling it to respond to incantations in a language no human being has ever spoken. I will ask the stone questions in this language, and it will answer by showing me a vision: a world created by my spell, a world imagined within the pattern on the stone." "A few hundred years ago in my native New England, an accurate description of my occupation would have gotten me burned at the stake. Yet my work involves no witchcraft; I design and program computers. The stone is a wafer of silicon, and the incantations are software. The patterns etched on the chip and the programs that instruct the computer may look complicated and mysterious, but they are generated according to a few basic principles that are easily explained." -- W. Daniel Hillis Barry ---------------------------------------------------------------------------- Barry -- I must say, you've got me totally baffled. I thought AI was largely purged of the naive, "thermostats are intelligent" business a decade ago. If your "mere knowers" are so mere that a thermometer can be said to know and have awareness, then what you mean by intelligence and mentality is so unrelated to anything I could mean by those terms -- and so unrelated to anything that almost *anyone* could have meant by them before the computer beguiled otherwise clear minds -- that I'm not sure what there is to talk about. Steve -------------------------------------------------------------------------------- Steve -- How about we talk about what it means to possess the faculty of computing a mathematical function? Barry ------------------------------------------------------------------------------- Barry -- But I'm also puzzled by the way you are now distinguishing the actor from the knower. If information-processing is required for acting, but a thermometer already knows the temperature, then why did you say an e-book--or printed book, for that matter--can't know what is recorded in it? By your account just now, information-processing isn't necessary for knowing, as opposed to acting. If a thermometer knows the temperature, and if in doing so it takes a physical state of affairs (namely, the level of mercury) as signifying something else (the warmth of the room), then why can't a piece of paper take the ink patterns on it as signifying *their* meanings? --------------------------------------------------------------------------------- Steve -- The faculty possessed by a system that retains information is the ability to 'repose' it. Thus a thermometer reposes the temperature. A book reposes a story. Part of intelligence is the Repository or faculty of storage. Computers have memory. They have memory because it is necessary to store (or repose) information if one is to process it. But mere storage is not enough. Storage is necessary but not sufficient. Many unintelligent systems repose information. They are 'dead storage' systems, like filing cabinets. But any functional information processing system (be it made of meat or made of silicon) needs some form of information storage -- some Repository. The same device that reposes recorded music (tape recording, compact discs) can also repose data and computer software (algorithms). Barry ---------------------------------------------------------------------------------- Barry -- Perhaps I'm just hopelessly misconstruing your words. But then you'll have to set me straight. I'm badly in need of some clarity on these points. Steve --------------------------------------------------------------------------------- Steve -- It's important to distinguish faculties such as sapience and sentience. It's important to identify all the component elements of an information processing system, as well as their relationship. If you take away any one component, you can cripple the system and render it incapable of processing information. But in our modern era of highly sophisticated sentient automata, we often forget that many of the critical faculties have been around in mechanical form for hundreds of years. Danny Hillis once built a machine to play Tic-Tac-Toe out of Tinker Toys. It's now sitting in the lobby of the Boston Museum of Science. I walk past it every Saturday. The float valve in your toilet tank, your furnace and thermostat, and your cruise control all instantiate feedback processes which are capable of maintaining equilibrium. These are instances of a process that, in mathematics, is called a Function. A Function maps information in one space (the Domain) into another space (the Range). The ability to compute functions is an important faculty of well-regulated systems, including nonliving systems, living systems, and intelligent systems. Today we have mechanical and electronic systems that compute extremely complex functions and mappings. That's why they are Functional. They perform a function, in the technical, mathematical sense of the word. And I'm sure you will agree that the ability to do mathematics is one of several kinds of intelligence that humans (and computers) can manifest. One of the things that worries me is that humans are losing the ability to compute functions, because we are delegating more and more of that responsibility to machines. Barry