Implementation of neural plasticity mechanisms on reconfigurable hardware for robot learning

Abstract

It is often assumed that insects are “primitive” animals, without the ability to exhibit complex learning behaviour. Fortunately, their tiny brains quite often surprise us with their performance. This thesis investigates the plasticity mechanisms of the insect brain through the research method of neurorobotics, i.e., the development of a physical agent, equipped with a silicon brain. In order to implement such a brain, we have chosen to model it directly onto hardware. Not only does this allow us to take advantage of the inherent hardware parallelism, but the robot can also behave in a completely autonomous mode, without having to communicate with the software simulator of a remote machine. FPGAs offer both the option for such a lowlevel design approach and the flexibility required in computational studies of biological neural networks. With the use of VHDL (a hardware description language), we develop a simulator for neural networks, designed as a series of computational modules, running in parallel and solving the differential equations which describe neural processes. It has the ability to simulate networks with spiking neurons that follow a phenomenological model, proposed by Izhikevich, which requires only 13 operations per 1 ms of simulation. The synaptic plasticity mechanism can be either that of spike timing-dependent plasticity (STDP) or a modified version of STDP which is also affected by neuromodulators. There are no constraints, as far as the connectivity pattern is concerned. The hardware simulator is then added as a peripheral to an embedded system so that it can be more easily controlled through software and connected to a robot. We show that this hardware system is able to model networks with hundreds of neurons and with a speed performance that is better than real-time. With some slight modifications, it could also scale up to thousands of neurons, starting to approach the size of the insect brain. Subsequently, we use the simulator in order to model a neural network with an architecture inspired by the insect brain, representing the connectivity of the antennal lobe, the mushroom body and the lateral horn, structures which are part of the insect’s olfactory pathway. Our silicon brain is then attached to a robot and its limits and capabilities are tested in a series of experiments. The experiments involve tasks of associative learning inside an arena which is based on a T-maze set-up usually employed in behavioural experiments with flies. The robot is trained to associate different stimuli (or combinations of stimuli) with a punishment, as indicated by the presence of a light source. We observe that the robot can solve most of the tasks, including elemental learning, discrimination learning, biconditional discrimination and negative patterning but fails to solve the problem of positive patterning. It is concluded that the architecture of the insect’s olfactory pathway has the computational efficiency to solve even non-elemental learning tasks. However, this pattern of results does not precisely match the fly, suggesting we have not fully understood the learning mechanisms involved. Moreover, embedding the learning circuit in robot behaviour reveals that the simple version of STDP is not the appropriate mechanism which can link neural plasticity to learning behaviour. Although the modified version of STDP is more suitable, it remains problematic as well as sensitive to timing issues. Therefore, we propose that STDP might function more as a “priming” process rather than as the basic learning mechanism.