Edinburgh Research Archive >
Informatics, School of >
Informatics thesis and dissertation collection >
Please use this identifier to cite or link to this item:
Files in This Item:
||Size||Format||thesis_original_submission.zip||18.64 MB||Unknown||thesis_resubmission.zip||25.92 MB||Unknown|
|Kyle2012.pdf||4.28 MB||Adobe PDF||View/Open|
|Title: ||Models and metaphors in neuroscience : the role of dopamine in reinforcement learning as a case study|
|Authors: ||Kyle, Robert|
|Supervisor(s): ||Willshaw, David|
|Issue Date: ||25-Jun-2012|
|Publisher: ||The University of Edinburgh|
|Abstract: ||Neuroscience makes use of many metaphors in its attempt to explain the relationship
between our brain and our behaviour. In this thesis I contrast the most commonly used
metaphor - that of computation driven by neuron action potentials - with an alternative
view which seeks to understand the brain in terms of an agent learning from the reward
signalled by neuromodulators.
To explore this reinforcement learning model I construct computational models to
assess one of its key claims — that the neurotransmitter dopamine signals unexpected
reward, and that this signal is used by the brain to learn control of our movements and
drive goal-directed behaviour.
In this thesis I develop a selection of computational models that are motivated by
either theoretical concepts or experimental data relating to the effects of dopamine.
The first model implements a published dopamine-modulated spike timing-dependent
plasticity mechanism but is unable to correctly solve the distal reward problem. I analyse
why this model fails and suggest solutions.
The second model, more closely linked to the empirical data attempts to investigate
the relative contributions of firing rate and synaptic conductances to synaptic
plasticity. I use experimental data to estimate how model neurons will be affected by
dopamine modulation, and use the resulting computational model to predict the effect
of dopamine on synaptic plasticity. The results suggest that dopamine modulation of
synaptic conductances is more significant than modulation of excitability.
The third model demonstrates how simple assumptions about the anatomy of the
basal ganglia, and the electrophysiological effects of dopamine modulation can lead
to reinforcement learning like behaviour. The model makes the novel prediction that
working memory is an emergent feature of a reinforcement learning process.
In the course of producing these models I find that both theoretically and empirically
based models suffer from methodological problems that make it difficult to adequately
support such fundamental claims as the reinforcement learning hypothesis.
The conclusion that I draw from the modelling work is that it is neither possible,
nor desirable to falsify the theoretical models used in neuroscience. Instead I argue
that models and metaphors can be valued by how useful they are, independently of
As a result I suggest that we ought to encourage a plurality of models and metaphors
in neuroscience. In Chapter 7 I attempt to put this into practice by reviewing the other transmitter
systems that modulate dopamine release, and use this as a basis for exploring the context
of dopamine modulation and reward-driven behaviour. I draw on evidence to suggest
that dopamine modulation can be seen as part of an extended stress response, and
that the function of dopamine is to encourage the individual to engage in behaviours
that take it away from homeostasis. I also propose that the function of dopamine can
be interpreted in terms of behaviourally defining self and non-self, much in the same
way as inflammation and antibody responses are said to do in immunology.|
|Sponsor(s): ||Economic and Social Research Council (ESRC)|
|Appears in Collections:||Informatics thesis and dissertation collection|
This item is licensed under a Creative Commons License
Items in ERA are protected by copyright, with all rights reserved, unless otherwise indicated.