Monday, March 16, 2009

How Advanced Artificial Intelligence (AI) Will Learn

Advanced AI will depend as much on experience as raw intelligence in order to fulfill its design objectives well.
How AI will learn
I recently received a question whose answer I thought worth sharing:
"Is sentience like my own as a 27 year old man something that can exist in an AI as soon as it is turned on for the first time?

Will it always take trial and error manipulating one's enviroment (to include interacting with other sentient beings) to learn the sentience of an average 27 year old male?"


You pose interesting questions.

I recently read a quote by Hans Moravec that I found interesting, and have been considering at length. I'm paraphrasing here, but it ran along the lines of "robots will eventually have exceedingly detailed models of human psychology.".

Which I agree with. Models and algorithms that allow it to not only to interact with human beings successfully, but of course have awareness of its environment so that it can navigate that environment successfully.

However, the way I heard it, there was something missing from Moravec's description - that is, data. Models and algorithms need data, of course, to do anything useful.

Here's the thing about human intelligence - every single human's intelligence is different from every other's - intelligence, in fact, is far more variable than fingerprints. So one psychological "model" can not instantly know every human mind out of the box, because a successful interaction with one human might be unsuccessful with another. However, I believe that there are good, high-level psychological models of the human mind that do in fact successfully embrace all of humanity. My favored one is the Freudian model of id, superego, and ego. However, I was re-listening to a Teaching Company course over the last few weeks, "Great Minds of the Western Intellectual Tradition", which in 48 hours of lectures covers from the Pre-Socratics to the 20th century, and I was struck by something. Essentially all of them, every last one, struggled with these same 3 components of the human mind. They use a wide variety of terms, and their focus and assumptions vary, but that fundamental similarity across the board is striking.

So I think we have a good start for a psychological model of the human mind - the animal passions, the moral and "instant judgement" portion, and the more or less objectively rational, reasoning portion.

However, the degree, expression, and interaction of these three, especially when coupled with external factors such as education, culture, etc, form a near-infinite variety of specific human minds. I don't use the word "infinite" often, but given that the human brain is the most complex object in the universe that we yet know of, if it is justified anywhere, it would be in this case.

Therefore, when you say, "turned on for the first time", I assume you mean the basic software and hardware are in place, but the experiential data are not yet acquired. If this is the case, I would say no, sentience like your own would not exist in an AI as soon as it is turned on for the first time.

However, the true test of the power of the software and hardware in this device is not its instantaneousness, I would suggest, but rather, how well does it learn.

So, how would it learn?

Depends on what we're talking about. Some things can be learned from books, some things require face-to-face interaction, etc.

I tend to think in terms of the value proposition of one these still hypothetical devices not so much in terms of how it could exactly mimic you (although that could be useful at times), but rather, in terms of how good it is at understanding, communicating with, and interacting with you. Remember, the most likely role these devices will serve is not to replace you, but to serve or assist you.

I discuss at length in my blog why I believe these devices, in all their forms, will be rationally controlled at all times, though capable of simulating emotions, primarily for the benefit of the human(s) with which it interacts.

Here's how I see the nature of their value proposition, as described in more detail here:
"The primary directive of these droids will be to maximize its owner's weal and happiness. Because it is a consumer product, there are very real limitations to how overtly this droid can fulfill that directive. For example, telling its owner what to do, even if it's for his own good? No, not acceptable, a droid can't do that. The droid must have consulting skills, really - it must steer its owner in the best direction, while at all times having the owner believe it's his own idea. This is not a nice to have, it is absolutely critical.

Also, part of the weal maximization mission is to help that owner be the best he can be, in all respects. So the droid will have to be a mentor, able to communicate in the very best way for that particular person. No matter how more intelligent than its owner the droid may be, it will never talk down to him, because that is a manifestation of arrogance, an id-thing. It will also be infinitely patient if that's required, because impatience is also an id-thing.

Over time, the droid's "personality" will mold into a custom fit for its owner's personality, maximizing compatibility while maintaining respect."


I then go on to describe in some detail the droid "superpowers" of hyper-observancy, super-subtlety, and ultra-coordination, which seem like a reasonable model for achieving the ends above.

Because each person is different, initially yes, I would expect there to be some trial-and-error. In fact, early in the relationship, I would suggest that it is perfectly alright as part of the "training" process for the droid to ask clarifying questions, and of course for the human to explicitly give insights into his personality and/or general preferences such as, "I like this, and don't like that."

Over time, there will still be trial-and-error, there will always be that, because not only are no two humans the same, no one person stays the same over time. Sometimes we're cranky, sometimes we're happy, and overall, can be quite unpredictable. However, the level of the trial-and-error, its "delta", as it were, will become steadily less, as the droid gathers more and more experiential data from that human or humans with which it interacts. In initial models, this process may take a while, weeks or months to get to a point where the human feels like, "yeah, this droid knows me pretty well."

However, in more advanced models, this process could become faster, perhaps quite a bit faster, as well as more successful. And of course, unlike us, the experience "database" of these droids will have characteristics similar to computer data today - ie, it will be downloadable, storable, reloadable, etc. As these devices become widespread, and if the experiential data of these devices can be combined into a large database, with proper protections for the consumer identity of course, ie, behaviors with certain demographic attributes, but not their actual name (much like CRM data is handled today), then we could get to a point where advanced versions of these droids could in fact arrive "pre-loaded" with appropriately distilled insights from this larger database that could make them much more psychologically insightful "out of the box", if you will.

I like the idea of the "personal droid", and believe it will converge as rapidly as technology allows on a hyper-realistically human form factor for a number of reasons I describe in my blog.

However, the exact same approach could be imagined for any number of professional applications, such as a droid as an assistant to a scientist. Combined with knowledge gained from books or wherever on the domain knowledge, say, particle physics, and working with a scientist, observing how he frames his hypotheses, questions, etc, while of course providing tremendous value as an advanced AI liaison to the often vast datasets that the scientist utilizes to test his ideas, that droid over time could become quite a useful scientist. And if you go further and imagine a droid that works not with one scientist but with several, and can flexibly mix and match their various approaches to get the best results overall, now you're talking a very useful scientist.

But, no matter how smart the droid, observation and analysis of both the humans around it and the world at large will always be essential. This is true for many reasons. Things simply change from day to day. But also, no matter how great the amount of experiential data, no matter how sophisticated the psychological algorithms running the droidian mind, you can never really know a specific human mind 100%, able to guess it's every thought, what it will say next, what its response will be in every conceivable situation. However, I expect that at some point, in very advanced versions of these devices, the "mistakes" the droid makes will fall below the human's ability to detect in most cases, which might qualify as near "perfection" from the human perspective, though I use that word very guardedly.

What you call trial-and-error I would phrase slightly differently, in terms of hypothesis formulation, experimentation, and testing, with essentially every single human interaction being an opportunity for these. For example, in a given interaction, did the human register:
- Annoyance?
- Seem to understand what the droid just said, or was there confusion there? If so, why? How could the droid better phrase its language to have the human better understand him?
- Approbation, indicating the droid did something especially well?
- etc (there are many more possible examples)

In other words, continually refining its internal models of that human's psychological profile, to achieve higher and higher "interaction success" rates. Interaction success can be defined in various ways, optimization of respect being one. If it's a professional relationship, such as a scientist assistant, in addition to optimization of respect, an additional criteria might be the productivity of the collaboration in terms of science achieved.

The future will tell, I could be wrong. But in my view this seems a probable trajectory for advanced AI that actually understands human behavior and can productively and successfully interact with said humans.

No comments: