Building Intelligence from the Bottom-Up
Today, the question for Humanoid Robotics is how best to impart these primitive behaviors to robots. Many researchers find it ineffective to directly hard-code such low-level behavior with imperative languages like C or C++ and instead use a more biologically motivated technique such as artificial neural networks. Artificial neural networks allow a 'supervised' learning approach where a designer trains a system's response to stimulation by adjusting weights between nodes of a network. The rise of artificial neural networks (ANNs) brought much optimism. Researchers believed that they could use ANNs to simulate the distributed, parallel nature of computation in the brain, allowing skills and knowledge to be conditioned as implicit generalizations of repeated experience.
As it turns out, ANNs fail to capture the recursive power of the human brain. Unlike an ANN where the structure of the network is usually fixed, the brain's highly integrated, well-ordered structure emerges through competition between separately evolving collectives of neurons. Critics argue that ANNs' lack of such an architecture prohibits meta-level learning -- the ability to not only generalize, but also extend acquired knowledge beyond the frontiers of experience. Although ANNs do not accurately model cognitive capacities of the human cortex, they do offer a truly unique and effective way to encode motor-skills and low-level behavior. It may be that, like the cerebellum and other, older structures of the brain, ANNs can provide a foundation on which high-level learning can be built. In any case, they have provided powerful insight into understanding both machine and biological learning.
Other learning techniques such as reinforcement learning and genetic algorithms have also played a role in modeling various levels of learning. Reinforcement learning can be used as an 'unsupervised,' learning-with-a-critic approach where mappings from percepts to actions are learned inductively through trial and error. Other approaches use evolutionary methods that begin with an initial pool of program elements and use genetic operators such as recombination and mutation to generate successive generations of increasingly 'fit' controllers. Using these approaches and others, robots can learn by adjusting parameters, exploiting patterns, evolving rule sets, generating entire behaviors, devising new strategies, predicting environmental changes, and recognizing the strategies of opponents or exchanging knowledge with other robots. Such robots have the potential to acquire new knowledge at a variety of levels and to adapt existing knowledge to new purposes. Robots now learn to solve problems in ways that humans can scarcely understand. In fact, one of the side effects of these learning methods is systems that are anything but 'explainable.' Emergent behavior is no longer suppressed by careful design, but instead, encouraged by similarly careful design.
Those working with humanoid robots often code learning mechanisms directly into their design environments and use them to hone existing behaviors, to develop new behaviors and even to string behaviors together. For instance, a designer can use a neural network to implicitly encode low-level motor control for an arm-reaching behavior and then use reinforcement learning to train the humanoid when to reach and when to grasp. If the humanoid still struggles, the designer might, for instance, optimize behavior using a genetic algorithm to tweak parameters controlling rotational torque.
Although these capabilities have proved invaluable, the devastating complexity of humanoids has required specialization. The goal of human-like versatility has often bowed to the goal of engineering specific human-like behaviors. The result is humanoids that can exhibit impressive functionality within a restricted domain or task. The next step is for an increasing number of capabilities to reside on general-purpose machines, capable of many tasks because they are engineered for none in particular.
Learning by Doing: Automated Development through Real-World Interaction
The mechanical sophistication of a full-fledged humanoid body poses a devastating challenge to even the most robust learning technique. The more complex a humanoid body, the harder it is to place constraints necessary for productive learning. If too few constraints are employed, learning becomes intractable. Too many constraints on the other hand, and we may curtail the ability of learning to scale. Ultimately, the conventional learning techniques described above are limited by the fact that they are tools wielded by human designers, rather than self-directed capabilities of the robot. This may not need to be the case. Although robots will always require an initial program, this fact does not preclude them from building indefinitely, willfully and creatively upon it. After all, humans also begin with a program encoded in our DNA. The key is that in humans the majority of this genetic code is devoted not to mere behavior, but to laying a foundation necessary for future development.
Researchers at MIT believe that the key to creating human-like behavior is to create a robot that can learn from natural interactions with a human (much as a human infant learns).
A growing number of humanoid researchers believe it is this ability to appropriately 'seed' development that will make learning tractable for humanoids. The goal is no longer for robots to merely learn (acquire knowledge and skill in a particular area), but to also develop (enrich cognitive ability to learn and extend physical ability to apply learning). Truly autonomous humanoids must ultimately play some role as arbiters of their own development, able to channel and structure learning across layers of control. This will require generalized learning starting from the ground up and continuing throughout the life of the humanoid, affecting what the robot is, rather than merely what the robot does.
Before we can transform a cognitive architecture into a developing mind, there are a host of difficult questions to be answered. How do we give humanoids the ability to impress their own meaning onto the world? How can humanoids direct their own development? How do we motivate this development? How much a priori skill and knowledge do we build in? Using what level of representation? What, if any, bounds should be imposed?
While there may never be definitive answers to these questions, a learning approach is emerging that provides a unique, functional balance of human input, self-development and real-world interaction. This technique, which we will call imitative learning, allows the robot to learn continuously through multi-modal interactions with a human trainer and the environment. The robot can pose questions, ask for actions to be repeatedly demonstrated, and use emotional states to communicate frustration, exhaustion or boredom to the human trainer. Advocates of imitative learning see it as the cornerstone in a developmental foundation that can enable self-directed, future learning.