Why Genetic Programming cannot solve the most important AI problem

In a comment to my last post, Upgrade Zero One A asks:

I was wondering what your opinion was on John Koza’s work specifically (Invention Machine)and Genetic Programming in general.
Is there any hope that GP could help solve or improve upon existing approaches any of the three areas mentioned?

1. Natural language understanding.
2. Vision, Image/Scene understanding.
3. Creating “consciousness”.

Since this excellent question really made me think beyond the scope of the original post, I decided to create a separate post for the answer, which follows.

This is really asking me to predict the future of mankind here..but I’ll give my best shot.
(I’ve changed the order of the areas)

1. Creating consciousness: No, I don’t think this is an area needing more direct interest. When we have more intelligent algorithms for processing input and turning it into output, the arising consciousness will automatically seem more real. Just program in a way for it to “wail” whenever a specific combination of sensory inputs happen, and I guarantee you, it will *be* real. That’s when the “AI Rights” bill will be presented in the house. One of the clauses will be to deem the “Artificial” in Artificial Intelligence as a politically incorrect/insensitive word.

2. Vision,Image/Scene understanding: Yes. Not so much for the “understanding” part, but very much so for the segmentation part. A perfect or even near-perfect image segmentation algorithm is yet to be found/invented/discovered, even though the task happens in the lowest level of processing in most beings. Separating a tiger or a zebra from the background of a forest is trivial even for tiny animals, but it stumps the best of today’s generic algorithms (they might work ONLY if they have been specially constructed to solve this specific problem itself, or if they are run in a “supervised” manner). The final breakthrough might very well come from using a GP approach.

3. Natural Language Understanding: I highly doubt it. Look at it from “real” evolution’s point of view. It took a gazillion number of species living concurrently, some new ones coming up, some going extinct, for billions of years, for ultimately *one* of them to be able to process natural language. The level of *concurrency* and the *interaction* between the species is enormous. Furthermore, “real” evolution is able to transcend all “local” maxima problems, because it, in effect, has an unlimited time scale. Dinosaurs evolved to be the dominant species 65 million years ago, but the next dominant species(us) was not a descendant of dinosaurs, it came from a totally different branch. And after all these years, this one species developed natural language.
One might say that GP is ideally suited to exactly this kind of task, but I think the scale is way off.

The task was to build an algorithm for understanding natural language, but I equated it to almost having to emulate the human species itself, or in essence, a human level intelligence.
Since GP tries to emulate evolution, and saying that current or somewhat futuristic GP could accomplish this task(NLU), is saying that GP can compress into a few days/weeks what real evolution did in billions of years. Give it a few years to run, maybe. But then we would never know if it will really ever converge or not. That’s the curse of evolution, presented to GP.

Therefore, I think the solution to this problem will come not from GP, but from other traditional directions, where a spark in the mind of some genius will be able to “utilize” the inherent knowledge built into the human brain and bootstrap an algorithm with a human-like capacity to learn.


9 Comments on “Why Genetic Programming cannot solve the most important AI problem

  1. Thank you so much for taking the time create a post on this question. Now I will take the time to think about what you wrote.
    Your answer helped to shed some more light on why natural language *understanding* is such a hard problem.

    It occurred to me that both consciousness and *understanding* are not things that either something has or it doesn’t. There are degrees of consciousness and degrees of understanding.

    I tend to agree with Marvin Minsky point about words “like intuition or consciousness” are examples of “suitcase” words that: “we all use to encapsulate our jumbled ideas about our minds. We use those words as suitcases in which to contain all sorts of mysteries that we can’t yet explain.”

    The word “understanding” might be another example of a suitcase word. Understanding a scene is certainly a lot different than understanding natural language, and in both instances, the understanding can be broken down still further…

    Natural Language understanding is a very very hard problem even to define. The concept of “Divide and Conquer” comes to mind.

    Thanks again!

  2. Pingback: Intelligent Machines

  3. Jide, At first glance, I thought this was a spam comment and almost hit the delete button, but then I saw your feeble attempt at addressing me by name (you could at least copy-paste to get the spelling right), and also you dont have any link back to any spam site, so I figured its someone in flesh and blood. Ok, but since you think just 2 words are enough to rebut my “nice argument”, I won’t grace you with the benefit of a response.

  4. Ragh,

    Those problems are indeed difficult, I’d like to add a side note. GA are “just” clever global optimizer of a fitness function, and the real difficulty is to find the “good” fitness function. GA are really far from what’s happening in real evolution. So I agree with you, I don’t think it’ll be so easy to solve those pbs withs GA (or with whatever algorithm).

    ->uPgRaD3 Z3R0 0n3 A : I’m not sure that scene understanding and language understanding are so different, and I guess that they share a lot, for instance the notion of compression, finding regularities, patterns in row data.


  5. Hmm..I am not sure if determination or implementation of the fitness function is the significant problem here. For NLU, off the top of my head, one check for a demo system could be to take a set of sentences as input that specify which cell to highlight in a 9×9 grid. Besides the simple sentences, some will of course be highly convoluted, as in “Take the third black cell from the left, go up a couple, then hop over three diagonally towards the top right, and stop where you have two white cells above it”.

    A harder problem, in building this for the GP world would be, at what state should the system be bootstrapped? How much knowledge should we implicitly build into the system even before the “learning” is started? Should the sentences be input as ascii characters, or as bitmap images? Should the algorithm be pre-programmed with the instructions needed to turn cells on or off, or should it even be made to learn that on its own? Should it be given an identity, or a name, so it knows the meaning of “you” when it reads a sentence, or should we allow it to evolve its “id”?
    Should it know that a space character separates the words in an ascii sentence? What about punctuation? What about humor? For an entity which only knows of a 9×9 grid as its world, is it even possible for it ever learn to decode humor, even if it were left to “evolve” for millenia? Or a better question: If its sensory inputs and outputs were limited to operations in the 9×9 grid only, would it ever be able to learn it fully? Doesn’t this mean that the agent can fully manipulate its environment, as in a god-like status within its world? Maybe to evolve something that can fully control the 9×9 grid, we need to evolve it in an environment which only contains the target 9×9 grid as a constituent, besides others. But what should the others be? Is there a critical mass, or a critical ratio?

    On the philosophical side: Can we ever divorce the NLU problem from the human aspect? My view is that we can’t. The only reason we want to build NLU capabilities is because it will make things easier for *us*, not the gadgets.

    If we had two highly *learning* capable robots, and we programmed them with the means to hear sound and make sounds to communicate, would they evolve something like human language?

    I think not. (Or actually, I am not really sure. Sounds like material for a new post…)

  6. Regardless of the techniques used to artificially process language, I think it is going to be a really long time before a machine can truly understand the meaning of the word “cute”. At a certain level, a mother seal must have an innate “idea” of cuteness with its baby, and it of course does not have any ability to process natural language.

    It seems there are many words like “cute” and “love” that must have extremely complex algorithms behind them, which includes chemical reactions related to the emotions which feedback into our minds and add to our knowledge of the meanings of these emotionally-based words.



  7. Steven Pinker has an interesting book:

    “The Stuff of Thought; Language as a Window into Human Nature”

    I think it is important to understand language at the level presented here, before attempting to drill down…and there are many many layers below this level before thinking about software.

    It is not an easy read, partly because of my lack of knowledge in linguistics, but already lights are turning on in my head.

  8. Hello Raghavgupta. I recently read a fictional book mentioning a biochemical computer as artifical intelligence. I was like what the hell? That’s like a revolutionary idea. Wonder if it’s already out there or just some figment of imagination. Sparked an interest in Artifical Intelligence and i’ve been doing a little research. I have no other knowledge other than my recent findings.

    You raise a good argument about understanding language as intelligence. I always wonder how I can just randomly say something without priorly thinking about what I am going to say. This is truly /A/ quality of “intelligence,” but i’m not sure if it’s the /ONLY/ quality. How do emotions come into this? Someone who is mute for example might not display this language quality, even more so if deafness was added as a variable, and yet this person would very well be conscious and could relate to the world around them and think and reason.

    It leads to the whole pilosophical thing…something about “I think therefore I am.” I don’t think anyone can really know when our robots are trully AI. Look at the movie I, Robot. I wonder when that robot developed his own independent consicousness. A sense of right and wrong?! I’m not sure where I read it, but somebody made the point that if you can teach a robot to “wail” as the word was used when certain receptive sensors were combined, you would start to have activists all over the place about AI Rights. I am remembering the movie Artifical Intelligence with the boy robot that was crying. Crazy ending in that movie…hope it doesn’t happen 😛

    Anyways, take a look at this site (not spam! 😀 ) seems like they’re getting closer to AI.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: