Computational Neuroscience

Research Group at Laboratory of Computational Engineering

 

Cerebellar learning for motor control

The cerebellum is responsible for timing, fine-tuning and coordinating the motor system. By learning in a self-supervised fashion from error signals generated by other parts of the brain and body, the cerebellum is able to rapidly execute and accurately time motor actions in response to external stimuli.

The learning algorithm executed by the cerebellum is at the same time powerful and simple: if a reflex is triggered in response to an event, the system will associate the action of the reflex with the states that preceded the event. The next time a similar state is observed, the system will anticipate the reflex by performing the reflex action beforehand. With suitably chosen reflexes, the cerebellum learns to be a stable controller that can, for instance, keep a dynamically balanced robot upright.

One of the main attractions of the cerebellar model of control is its robustness: the system can quickly learn to respond to new conditions, and can learn to anticipate changes in the external world that place demands on the motor system (for example, knowing that a heavy weight will shortly be placed in one's hands, a person will automatically prepare by assuming a more solid posture). The cerebellar algorithm is also able to make use of any contextual data from the rest of the brain that happens to be available.

Our work concerns the application of the cerebellar control model into robotics using a simulation environment; the ability of the cerebellar controller to take advantage of extraneous inputs for adaptation and the mathematical aspects of the cerebellar controller itself.

For example, Figure 1 shows a simple system where the reflex only uses the current state of the robot (including velocities) and where the goal of the reflex is to keep the robot upright. Occasionally, footballs are launched towards the robot and it is given some inputs to be able to tell when a football is coming. Due to the reflex, the robot learns to anticipate the incoming football in advance and to lean towards it, in order to regain upright posture as soon as possible (the robot does not have sufficient power to recover from a hit of the football if it only stays upright and reacts after the football has hit it). Having such lifelike behaviour (anticipating an impact and leaning towards it) emerges from a simple rule and a simple learning system is a good demonstration of the potential of cerebellar control. A more complicated example where the system has learned to use visual information is shown in Figure 2.


Figure 1. A simple reflex that tries to keep the robot upright but that knows nothing about the environment has taught the robot to respond to an environmental stimulus. Since the cerebellar algorithm predicts from the inputs giving the position of the ball that the reflex would soon (after the impact) cause it to try to lean towards the left, it leans towards the left already in preparation and manages to avoid falling down.


Figure 2. A robot that learns to make use of visual cues from a virtual camera in order to anticipate and compensate for the roughness of the terrain, modelled as a kind of drag on the robot's wheel.

The animations shown in the right-hand margin demonstrate this experiment: in the topmost animation, the robot has not yet learned to respond to the ball and falls over when the ball hits. In the middle animation, the robot has learned to respond to the ball by leaning towards it and is able to stay upright. In the bottom animation, the ball has been made massless: expecting a collision, the robot leans towards the ball but when the ball has no mass, it stumbles.

In ongoing development at the end of the year were systems such as the ones shown in 3. The goal of the system is to keep the multi-jointed robot arm at a given position. With a sensory delay, this is not an easy task: the different degrees of freedom interact and righting one segment will cause the others to experience more force and can easily lead to chaotic and unstable states. The cerebellar controller is able to coordinate the arm by taking into account the positions of all joints and anticipating the motion of the other joints and compensating for it proactively.

(a)

(b)


Figure 3. a) A variation of the basic pole balancing problem where the robot balances in two dimensions using a ball, just like the real-life Ballbot robot of CMU. b) A three-joint arm where the joints have learned to coordinate with each other by predicting each other's effects on themselves. In both a) and b), the extra boxes are animated bar graphs visualizing various dynamic variables.

Other documents of this project

Synopsis of master's thesis of Heikki Joensuu

Master's thesis of Heikki Joensuu (pdf): Adaptive control inspired by the cerebellar system