Cerebellar learning for motor control
The cerebellum is responsible for timing, fine-tuning and coordinating the motor system. By learning in a self-supervised fashion from error signals generated by other parts of the brain and body, the cerebellum is able to rapidly execute and accurately time motor actions in response to external stimuli.
The learning algorithm executed by the cerebellum is at the same time powerful and simple: if a reflex is triggered in response to an event, the system will associate the action of the reflex with the states that preceded the event. The next time a similar state is observed, the system will anticipate the reflex by performing the reflex action beforehand. With suitably chosen reflexes, the cerebellum learns to be a stable controller that can, for instance, keep a dynamically balanced robot upright.
One of the main attractions of the cerebellar model of control is its robustness: the system can quickly learn to respond to new conditions, and can learn to anticipate changes in the external world that place demands on the motor system (for example, knowing that a heavy weight will shortly be placed in one's hands, a person will automatically prepare by assuming a more solid posture). The cerebellar algorithm is also able to make use of any contextual data from the rest of the brain that happens to be available.
Our work concerns the application of the cerebellar control model into robotics using a simulation environment; the ability of the cerebellar controller to take advantage of extraneous inputs for adaptation and the mathematical aspects of the cerebellar controller itself.
For example, Figure 1 shows a simple system where the reflex only uses the current state of the robot (including velocities) and where the goal of the reflex is to keep the robot upright. Occasionally, footballs are launched towards the robot and it is given some inputs to be able to tell when a football is coming. Due to the reflex, the robot learns to anticipate the incoming football in advance and to lean towards it, in order to regain upright posture as soon as possible (the robot does not have sufficient power to recover from a hit of the football if it only stays upright and reacts after the football has hit it). Having such lifelike behaviour (anticipating an impact and leaning towards it) emerges from a simple rule and a simple learning system is a good demonstration of the potential of cerebellar control. A more complicated example where the system has learned to use visual information is shown in Figure 2.
The animations shown in the right-hand margin demonstrate this experiment: in the topmost animation, the robot has not yet learned to respond to the ball and falls over when the ball hits. In the middle animation, the robot has learned to respond to the ball by leaning towards it and is able to stay upright. In the bottom animation, the ball has been made massless: expecting a collision, the robot leans towards the ball but when the ball has no mass, it stumbles.
In ongoing development at the end of the year were systems such as the ones shown in 3. The goal of the system is to keep the multi-jointed robot arm at a given position. With a sensory delay, this is not an easy task: the different degrees of freedom interact and righting one segment will cause the others to experience more force and can easily lead to chaotic and unstable states. The cerebellar controller is able to coordinate the arm by taking into account the positions of all joints and anticipating the motion of the other joints and compensating for it proactively.
Other documents of this project
Master's thesis of Heikki Joensuu (pdf): Adaptive control inspired by the cerebellar system