A COMPUTATIONAL MODEL OF REINFORCEMENT
LEARNING AT BASAL GANGLIA
D Hasan, Ö Karabacak, NS Şengör
Istanbul Technical University, Faculty of Electrical Electronics Eng. Istanbul Turkey
Once the basal ganglia were thought to take part only in movement control, but now the
discussion about this structure is focused more on how they take part in ‘habit’ learning and
their role during working memory tasks. While performing a cognitive task, there is need for
on-line maintenance and manipulation of information and one view of working memory
perceives it as the collection of necessary mechanisms to fulfill these these needs and basal
ganglia’s proposed role is to update working memory representations. Thus, it is thought that
basal ganglia provide a gating signal to select working memory representations and maintain
them over delays.This gating process is based on reinforcement learning mechanism.
Moreover, the possibility of explaining all different functions of basal ganglia in the same
context is a new argument. In this work, a biologically inspired model which mimics the
behavior of cortico-basal ganglia-thalamo-cortical closed loop connections for stimulusresponse task is regenerated numerically. The model is generalized to consider reinforcement
learning paradigm where basal ganglia structure corresponds to actor-critic subsystems and
remainder of loop corresponds environment. The effect of basal ganglia is to provide a signal
based on reinforcement signal and knowledge about the state of environment, and this signal
triggers the generation and storage of temporal sequence during stimulus response task
composed of three movements: push, pull and turn. Simulation results for the task considered
are obtained and effect of reinforcement learning at basal ganglia on the temporal sequence
generation and storage is investigated.