Resource-Bounded Machines are
Motivated to be Effective, Efficient,
and Curious
Bas R. Steunebrink, Jan Koutník, Kristinn R.
Thórisson, Eric Nivel, Jürgen Schmidhuber
The Swiss AI Lab IDSIA, USI & SUPSI,
Reykjavik University
The Main Argument
1. Explicitly acknowledge resource constraints
2. Identify the constrained resources
3. Design AI system to be driven by better and
better resource utilization
4. Order activities around resource utilization
5. Emergent result: effectiveness, efficiency &
curiosity
Fundamental Resources
Resource
Energy
Input
Time
Compression
Efficiency
Learning
Effectiveness
Drive
(unnamed)
Curiosity
(unnamed)
To improve
Work
Play
Dream
Resource Compression: Why?
• Less time & energy spent  more reward
(resources shared with other agents)
• Less time & energy spent  more left for
future tasks
• Compression of input = learning  better
prepared for unknown future
Driven by Resource Compression Progress
Minimize resource consumption through:
1. Knowledge
– Learn new more effective & efficient routines
– Curious exploration to “fill knowledge gaps”
2. Architecture
– Re-encode known routines for more effective &
efficient execution
– Example: self-compilation
Work—Play—Dream Framework
• Utilize the kinds of activities afforded by the
patterns of interaction with human teachers /
supervisors / users
• Work: fulfill main purpose, no exploration,
store interesting / unexpected events
• Play: curiosity-driven exploration, perform
experiments, may still requires supervision
• Dream: analyze unprocessed events, selfcompilation, task invention for Play
WPD Framework, cont.
• Work—Play—Dream are processes, not states
• Can run in parallel, but sometimes not possible
due to resource/situational constraints
• Combination of Work & Play leads to creativity
• Dreaming may be a necessary side-effect for any
system constrained in computational power and
memory
– “Tired” = buffers reach capacity
– “Dream” = process input history backlog
AERA: An Explicitly Resource-bounded Architecture
•
•
•
•
•
Knowledge is operationally constructive
Model-based & model-driven, hierarchically
Simulation through forward chaining
Planning through backward chaining
Compilation of useful & reliable chains
– Originally for scalability
– Now realized to satisfy the architectural way of
achieving resource compression: re-encoding
How AERA does resource compression
• Consider 3 resources: Time, RAM, HDD
• Time more precious than memory
• Self-compilation leads to better resource
utilization
• Thus AERA must be motivated to self-compile
• Simple analysis of control values yields goals that
give rise to empirical testing of unstable models
• Crux: scheduling needed  Work, Play, Dream
Conclusion
• AERA is being developed as a cognitive
architecture towards AGI
–
–
–
–
Based on many firm principles, but not curiosity
But all ingredients for curiosity are present
Learning & re-encoding both possible!
Thanks to self-compilation ability
• Resource usage compression = principled middle
ground
– No twisting of AERA or Theory of Curiosity
– AERA still based on solid principles
– Curiosity generalized to resources-bounded view