Machine Learning (Theory)

4/2/2006

Mad (Neuro)science

One of the questions facing machine learning as a field is “Can we produce a generalized learning system that can solve a wide array of standard learning problems?” The answer is trivial: “yes, just have children”.

Of course, that wasn’t really the question. The refined question is “Are there simple-to-implement generalized learning systems that can solve a wide array of standard learning problems?” The answer to this is less clear. The ability of animals (and people ) to learn might be due to megabytes encoded in the DNA. If this algorithmic complexity is necessary to solve machine learning, the field faces a daunting task in replicating it on a computer.

This observation suggests a possibility: if you can show that few bits of DNA are needed for learning in animals, then this provides evidence that machine learning (as a field) has a hope of big success with relatively little effort.

It is well known that specific portions of the brain have specific functionality across individuals. There are two ways this observation can be explained:

  1. Maybe the specific functionality areas are encoded in the DNA.
  2. Maybe the specific functionality areas arise from the learning process of the brain. This is the answer that machine learning would like to hear because it agrees with the hypothesis that a simple general learning system exists.

It’s important to realize that these choices actually specify a spectrum rather than a dichotomy. There are surely some problem-specific learning hacks in the brain and there is surely some generalized learning ability. The question is: to what degree is learning encoded by genetic heritage vs personal experience?

It is anecdotally well known that people (especially children) can recover from fairly severe brain damage, but of course we would prefer to avoid anecdotal evidence.

There are also neuroscience experiments addressing this question. This paper by Jitendra Sharma, Alessandra Angelucci, and Mriganka Sur provides some evidence. In a nutshell, they rewire the optic nerve of ferrets into the auditory region of the brain. They observe that structures similar to the visual specific region of the brain arise in the auditory region after rewiring (although the new regions may be less capable).

There are doubtless many other experiments addressing this question, but my knowledge of Neuroscience is lacking. (Thanks to Maneesh for pointing this one out.)

3 Comments to “Mad (Neuro)science”
  1. Andrej Bauer says:

    If you are interested in this sort of thing, I recommend “The Symbolic Species: The Co-Evolution of Language and the Brain” by Terrence W. Deacon. I read it a while ago and it has a wealth of information about how scientists think the brain works. Apparently, not much of it is hard-wired.

  2. Mark says:

    Another interesting book looking at the intersection of these fields is Leslie Valiant’s (of PAC learning fame) “Circuits of the Mind”. He proposes and analyses several simple “neuriodal” algorithms for memorisation, association and relational learning based on data others have collected about the structure of the brain. He shows that the timed network structures that are required to implement each of those algorithms are fairly simple and therefore probable even in a random network of synapses.

    Worth a look.

  3. Elizabeth Bates also did some nice work, looking at language learning in children with brain damage. Bottom line: damage early enough and they learn despite having no “language area” to learn in.

Leave a Reply


Powered by WordPress