Ten questions for, and about, AI
A ten-question test for AI
I have had it with the stream of articles about what artificial intelligence (AI) can do. Yes, machine learning works. It is possible to analyse key words, correlate them with other key words, do a massive amount of statistics, and find out some stuff. People cannot do that and computers can.
Is this AI? It sure isn’t anything people can do, and it also doesn’t correspond to anything I understand about what it means to be intelligent.
So, to make clear what AI is really about I propose the ten-question test. Here are ten questions that any person could easily answer and that no computer can answer well.
Since I consider myself an AI person I would not say that no computer will ever be able to answer these. I hope we can figure it out. But AI isn’t here yet, despite what venture capitalists, giant companies, and the media keep saying.
These questions can help explain why.
1. What would be the first question you would ask Bob Dylan if you were to meet him?
I am using this one because IBM’s Watson “met” Bob Dylan and told him that his songs were about “love fading” (you can read more about it here).
First, I might point out that that conversation is insane. If you were to meet Bob Dylan you might have some things you’d want to know. I’d like to know if he feels that his songs are “literature”. I’d also like to know if he thinks he helped a generation feel stronger about protesting injustice and war.
I would not count all the words he used and tell him which word appears most often. Watson does not behave as intelligent entities do. Intelligent entities are curious. They have things they want to know and can recognise who can answer questions that come to their minds about different arenas of life.
2. Your friend told you, after you invited him for dinner, that he had just ordered pizza. What will he eat? Will he use a knife and fork? Why won’t he change his plans?
You will notice that eating is not mentioned in question two. Neither are utensils. So how could an AI understand these questions?
It would have to know about how people function in daily life. It would have to know that we eat what we order, and that when we say we ordered food it means that we intend to eat it, and it also means that we don’t want to waste it.
It would also have to know that pizza is typically eaten with one’s hands.
It might also know that Donald Trump famously eats pizza with a knife and fork and might mention that when asked.
3. I am thinking of driving to New York from my home in Florida next week. What do you think?
To answer the above question, one would need a model of why people ask questions like that one. It is hard to answer if you don’t know the person who is asking.
If you do know that person you would also know something about what he is really asking. Does he/she have a car that is too old to make the trip? Maybe they have a brand new car and asking your advice about whether a long trip is a good way to break it in. Maybe they know you live in New York and might have an idea whether the roads are icy there.
Real conversation involves people who make assessments about each other and know what to say to whom based on their previous relationship and what they know about each other.
Maybe the asker is really asking about a place to stay along the way (if the person being asked lives in Virginia, for example). Sorry, but no AI is anywhere near being able to have such a conversation because modern AI is not building complex models of what we know about each other.
4. Who do you love more: your parents, your spouse or your dog?
What does this question mean and why would anyone ask it? Maybe the person being asked is hugging their dog all the time. Maybe the person being asked is constantly talking about his/her parents.
People ask questions as well as answer them. Is there an AI that is observing the world and getting curious enough to ask a question about the inner feelings of someone whom it is interacting with? People do this all the time. AIs do not.
5. My friend’s son wants to drop out of high school and learn car repair. I told her to send him over. What advice do you think I gave him?
If you know me, you would know how I feel about kids being able to follow their own interests despite what school wants to teach. So an intelligent entity that I told this to would probably be able to guess what I said. Can you? No AI could.
6. I just saw an ad for IBM Watson. It says it can help me make smarter decisions. Can it?
My guess is that this is something Watson can do. It can analyse data, and with more information a person can make better decisions.
Could Watson make the decision? Of course not.
Decision-making involves prioritising goals and being able to anticipate the consequences of actions. Watson can do none of that.
7. Suppose you wanted to write a novel and you met Stephen King. What would you ask him?
If I wanted advice on writing a novel I doubt I would ask Stephen King, but here is one thing that is sure. AI, such as IBM’s Watson, wouldn’t understand anything he said about writing a novel and Watson wouldn’t be writing any novels any time soon.
Now as it happens my AI group frequently worked on getting computers to write stories of one sort or another. We learned a lot from doing that.
I am quite sure that IBM hasn’t even thought about what is involved in getting a computer to write novels. Having something the computer wants to say? Having had an experience that the computer is bursting to describe to people? That would be AI.
8. Is there anything else I need to know?
When might you ask such a question? You might have had a conversation with a chatbot and found out how to get somewhere you were trying go.
Then you might (if you were talking to a person) ask if there is anything else you needed to know.
Answering that question involves knowing whom you are talking to.
(Oh, yeah, there is a great Ethiopian Restaurant nearby and watch out for speed traps.)
Let’s see the chatbot that can answer that question.
9. I can’t figure out how to grow my business. Got any ideas?
It is obvious why this is a difficult question. But, in business, people have conversations like that all the time. They use their prior experiences to predict future experiences. They make suggestions based on stuff they themselves have done. They give advice based on cases in their own lives and they usually tell personal stories to illustrate their points.
That is what intelligent conversation sounds like. Can AI do that? Not today, but it is possible.
Unfortunately, there is no one that I know of who is working on that. Instead they are working on counting words and matching syntactic phrases.
They are also working on AI document checkers that will help Word with spell check, or grammar check.
Apparently, “NeuroGrammar uses its advanced neural-network AI algorithms in order to analyse every noun phrase and verb phrase in every sentence for syntactic and semantic errors”.
How marvellous. So here is my last question:
10. Does what I am writing make sense?
Amazingly, this is hard. Why? Because to understand my points you need to match them to things you already think and see if I have helped you think about things better or decide that you disagree with what I am saying here based on your own beliefs.
You already have an opinion on whether my writing style was comprehensible and whether the points I made also made sense to you. You can do that. AI cannot.
Do I think we could do that someday in AI? Maybe. We would have to have a complete model of the world and an understanding of what kinds of ideas people argue for and what counterarguments are reasonable. Intelligent people all do this. AIs do not. An AI that understood documents would not be a grammar checker.
It would be nice if people stopped pushing AI that is based on statistics and word counts and AI people tried to do the hard work that making AI happen would require.
By Roger Schank, founder and CEO at Socratic Arts
This article is featured in the September 2017 issue of the Banking Technology magazine. Click here to read the digital edition (it is free!).
