Parameter exploration of a neural model of state transition probabilities in model-based reinforcement learning

Proceedings of the 15th International Conference on Cognitive Modelling, 2017

Mariah Martin Shein, Terrence C. Stewart, Chris Eliasmith


We explore the effects of parameters in our novel model of model-based reinforcement learning. In this model, spiking neurons are used to represent state-action pairs, learn state transition probabilities, and compute the resulting Q-values needed for action selection. All other aspects of model-based reinforcement learning are computed normally, without neurons. We test our model on a two-stage decision task, and compare its behaviour to ideal model-based behaviour. While some of these parameters have expected effects, such as increasing the learning rate and the number of neurons, we find that the model is surprisingly sensitive to variations in the distribution of neural tuning curves and the length of the time interval between state transitions.

Full text links


Conference Proceedings

Proceedings of the 15th International Conference on Cognitive Modelling


Plain text