Guiding Questions for Paper Presentations
Version: 09/04/12
2012 Fall Course on Action-Perception-Learning Cycles
(a.k.a. Artificial Neural Networks)
Prof. Byoung-Tak Zhang
Dear Presenters, please attempt to address the following questions when you prepare a paper presentation.
Keep in mind that the presentation time is limited, typically less than 1 hour shared by two or three presenters,
and we also need ample time for discussion which is very important. The issues related with the questions can
also be asked in the exams. So you are also helping other students to prepare their exams.
Note: These questions are minimal and incomplete, giving only a rough guideline for your reading and
presentation. It is highly encouraged for you to ask other interesting questions.
[2-1]
Gordon, G., Kaplan, D. M., Lankow, B., Little, D. Y.-J., Sherwin, J., Suter, B. A., and Thaler, L., Toward an
integrated approach to perception and action: conference report and future directions, Frontiers in Systems
Neuroscience, 5(20):1-6, 2011.
Q1: Traditional approaches to neuroscience are to investigate sensory function of the brain in isolation from
motor function. Why sensory and motor processing cannot be fully separated?
Q2: What is the biological evidence for a strong integration of perception and action in the brain? What are the
implications of closed action-perception loops for computational modeling? What are the theoretical benefits
of directly incorporating actions into computational models of perception?
Q3: While some researchers argue that actions are necessary for perception, others argue that perception can
be understood independently of action. Give the arguments of both sides. What’s your opinion?
Q4: Explain the model of hierarchical action-perception loops in Figure 1. What are forward models and
inverse models? What do the (four) different types of arrows represent? Explain the whole meaning of this
model.
Q5: Is the relationship between action and perception invariant? Which timescale is relevant for the
integration of action and perception? Is prediction central to the action-perception framework? What are the
implications of embodiment on action and perception?
[2-2-1]
Fuster, J. M., The prefrontal cortex - an update: Time is of the essence, Neuron, 30:319-333, 2001
Q1: Explain the general organization of cognitive representations of the human cortex in Figure 3. Is it
hierarchically organized? What are the three major regions of the prefrontal cortex (PFC) and what are their
functions?
Q2: How are the actions temporally sequenced or integrated? Explain Figure 4. How does the cortex process a
novel and complex sequence?
Q3: Explain Figure 7. What is the experimental paradigm? What is the objective of the experiment? What is
the conclusion of the experimental results? How sound and color are cross-temporally integrated in frontal
cortex?
Q4: Explain the cortical dynamics in the perception-action cycle in Figure 10.
Q5: Answer also the questions addressed in [2-2-2] below.
[2-2-2]
Fuster, J. M., The cognit: A network model of cortical representation, International Journal of
Psychophysiology, 60:125-132, 2006.
Q1: Compare the modular models and network models of the cortex. How do they view the cortex differently?
What can they explain and what they cannot?
Q2: What are the cognits? Explain the global architecture of the brain in terms of the cognits.
Q3: Explain the main cognitive functions of the brain in the following regions: posterior and frontal cortex,
parasensory and premotor and prefrontal cortex?
Q4: What brain connections are responsible for the perception-action cycle in sequential behavior, speech, and
reasoning?
Q5: What is a relation code? Why is it important? How is it different from other coding mechanisms known to
be used in the brain?
[3-1]
Pickering, M. and Garrod, S., An integrated theory of language production and comprehension, Behavioral
and Brain Sciences, 2012 (in press)
Q1: Give the evidence for how language production and comprehension are tightly interwoven. How does this
relate to the perception-action cycle theory of cognitive systems?
Q2: Explain and give the evidence for how action, action perception, and joint action are interwoven. Explain
how the authors use this to develop accounts of production, comprehension, and interactive language.
Q3: Give examples of what behavioral and neuroscientific data on language processing can be explained by
the integrated theory of language production and comprehension explains, while modular theory does not.
[3-2]
Clark, A., Whatever next? Predictive brains, situated agents, and the future of cognitive science, Behavioral
and Brain Sciences, 2012 (in press)
Q1: Brains are essentially prediction machines. What does this mean? What are key elements of this approach?
What form of structure does the predictive machine take? What’s the aim of this machine?
Q2: How can the predictive machine account of the brain offer a unifying model of perception and action?
How does it illuminate the functional role of attention? How may this capture the contribution of cortical
processing to adaptive success?
Q3: What impacts does the hierarchical prediction machine approach have on the general vision of mind,
experience, and agency?
[4-1-1]
Fry, R. L., Neural statics and dynamics, Neurocomputing, 65-66:455-462, 2005.
Q1: Compare the communications system and the single-neuron system. What is the design objective of a
communication system? What is the computational goal of a single-neuron system? How are they similar and
how different?
Q2: Describe the model of single-neuron computation in Figure 1. What is the input and what is the output?
How does the neuron compute the output from the input? Explain the neural decisioning process. How is it
related to Bayes theorem?
Q3: In Figure 1, what are the parameters of the single-neuron system? How are they adapted? Explain the
rules for gain adaptation, delay equalization, and threshold adaptation. What is the Hebbian gating? Why is it
necessary?
Q4: How is the function of the single-neuron system related to the principle of maximum entropy (ME)?
Derive the neural model from the ME functional.
[4-1-2]
Fry, R. L., Computation by neural and cortical systems, BMC Neuroscience, 9(Suppl 1):P66, 2008
Q1: What is dual matching in information theory? (you may refer to another paper of the same author). What
is the Carnot cycle in thermodynamics? How does the dynamical processing of pyramidal neurons correspond
to that of a Carnot cycle?
Q2: Explain the neural Carnot cycle of a rectangle in Figure 1 that describes neural computation. Explain the
four phases of the cycle. Compare the neural Carnot cycle with the Carnot engine in thermodynamics.
===========================================================================
[5-1]
Friston, K., The free-energy principle: a rough guide to the brain?, Trends in Cognitive Sciences, 2009
Q1: Explain the following terms: KL divergence, entropy, ergodic, free energy, Bayesian surprise, generative
model, recognition density, sufficient statistics.
Q2: Explain the free-energy principle of the brain, i.e. the fact that self-organizing biological agents resist a
tendency to disorder and therefore minimize the entropy of their sensory states. Give various forms of free
energy.
Q3: How can action reduce free energy? How can perception reduce free energy? How can active sampling of
the sensorium contribute to the free energy reduction?
Q4: Explain the neurobiological architecture for implementing the free-energy principle in Figure 1 in Box 1.
Describe each of the modules in the figure and their functions as well as the quantities that define the free
energy.
Q5: Describe the sufficient statistics representing a hierarchical dynamic model of the world in the brain in
Figure 1 in Box 2. How are they related with each other? How are the changes in synaptic activity,
connectivity, and gain involved with perceptual inference, learning and attention?
Q6: Formulate and describe the neuronal architecture for the hierarchical dynamic model in Figure 1 in Box 3.
How are the forward prediction errors computed? How are the backward predictions made? What are the
sources of the forward and backward connections in terms of brain anatomy?
Q7: A key implementational issue is how the brain encodes the recognition density. There are two forms of
probabilistic neuronal codes: free forms and fixed forms. Give examples of each form and explain them.
Q8: What kinds of optimization schemes does the brain use? Does it use deterministic search on free energy to
optimize action and perception? Or, does it use stochastic search? What is your opinion?
==========================================================================
[5-2]
Dayan, P. & Daw, N. D., Connections between computational and neurobiological perspectives on decision
making, Cognitive, Affective, & Behavioral Neuroscience, 8(4):429-453, 2008
Q1: Explain the experimental tasks in Figure 1. What should the subject do in each task? What is the
underlying computational theory for each task?
Q2: Explain the following terms: Markov decision problem (MDP), signal detection theory, exploration and
exploitation tradeoff.
Q3: Marr distinguished the three levels of describing cognitive systems: computational level,
algorithmic/representational level, and implementational level. Use the concrete task given in Figure 2 to
explain the issues related to each of these levels. What is the task? What are the main issues in terms of the
three description levels?
Q4: Consider the prediction task in Figure 1A and Figure 3A. Describe the MDP problem using the specific
task in Figure 3A. What are the computational issues? What are the algorithmic issues? What are the
implementational issues, i.e. in terms of psychology and neuroscience? Use equations, if applicable, to explain
the specific issues.
[6-1]
Zahedi, K., Ay,N., and Der, R., Higher coordination with less control-A result of information maximization in
the sensorimotor loop, Adaptive Behavior, 18(3-4):338-355, 2010.
Q1: Explain the concept of the perception-action cycle in Figure 1. What are the essential characteristics of
this concept? How is this concept distinguished from traditional symbolic AI approach?
Q2: Explain the simplified version of the perception-action-cycle in Figure 2. What are their differences from
the full version of Figure 1? How reasonable is this simplification? When it will work and when it does not?
Q3: Define the mutual information. Define the predictive information. Give a learning rule that maximizes the
predictive information. Derive the learning rules.
Q4: Explain the experimental tasks that are designed by the authors to evaluate the learning rule for predictive
information maximization. What’s the setup? What is the task? What has been measured in simulation
experiments? Summarize the results. What’s the conclusion of the experiments?
[6-2]
Tishby, N. & Polani, D., Information theory of decisions and actions, In: Perception-Reason-Action Cycle:
Models, Algorithms and Systems, 2010.

Q1: How is Fuster’s perception-action-cycle and Shannon’s information theory related? How is this analogy
related with reinforcement learning?
Q2: What is value-to-go? What is information-to-go? How do we trade-off between these two terms? Give a
formulation that can make this trade-off. Hint: Free-energy principle. How can we find the optimal policy, i.e.
the one minimizing its information-to-go under a constraint on the attained value-to-go.
Q3: Define the entropy. Define the relative entropy or Kullback-Leibler divergence. Define the Markov
decision process (MDP). Define the value function of the MDP. How is the value function optimized? What is
Bellman equation and how is this related with the MDP problem? What’s the relationship between
reinforcement learning, the MDP, and the Bellman equation?
Q4: Use a Bayesian network or graphical model (see Figure in page 12) to describe the perception-action
cycle of an agent with sensors and memory. What are the characteristics of this agent?