Intrinsic Goals for Autonomous Agents (Reece Keller)
Why do we do things? Standard RL says: to maximize rewards, generously handed to us by the environment.
Sensorimotor AI Journal Club
Next session: Monday, May 18, 2026
Topic: AI in a vat: Fundamental limits of efficient world modelling for agent sandboxing and interpretability
Presenter: Fernando Rosas
Paper: Rosas et al., arXiv
Recent Posts
The Affective Gradient Hypothesis (Amitai Shenhav)
Why did the chicken cross the road?
Plasticity as the Mirror of Empowerment (David Abel)
Two weeks ago, David Abel argued that RL’s central concepts need precise mathematical definitions. Last week, Mani Hamidi responded through the lens of evolu...
Illuminating the Three Dogmas of RL under Evolutionary Light (Mani Hamidi)
Last week, David Abel argued that RL’s central concepts (agent, learning, reward) need more precise definitions. He identified three dogmas limiting the fiel...
Three Dogmas of Reinforcement Learning (David Abel)
Newton gave force, inertia, and motion precise mathematical definitions. This unlocked centuries of progress in mechanics. Carnot formalized heat engines as ...
Nested Learning: The Illusion of Deep Learning Architecture (Ali Behrouz)
Ali Behrouz, PhD student at Cornell and student researcher at Google, presented a sweeping unification of deep learning architectures and optimization algori...
Continuous Thought Machines and how to think about thought (Luke Darlow)
We kicked off the spring 2026 semester with Luke Darlow from Sakana AI, who presented the Continuous Thought Machine (CTM)—a recurrent architecture built on ...
RL Debates: Finale and Synthesis
For the final session of the RL Debate Series, we brought all six contenders back, not to declare a winner, but to attempt a synthesis. What actually happene...
RL Debates 6: Thomas “no reward for you” Ringstrom
In our 6th and final RL Debates presentation, Tom introduced a rigorous framework for compositional planning that replaces the standard scalar reward with pr...
The neuron as a direct data-driven controller (DD-DC) by Moore et al., 2024
We explored DD-DC, a provocative normative theory proposing that neurons are not just information processors, but active feedback controllers that steer thei...
RL Debates 5: Anne “not everything is RL” Collins
In our 5th RL Debates presentation, Anne argued that reward-based learning is not always driven by RL computations. Sometimes it’s working memory combined wi...
RL Debates 4: Adam “I literally measured value in the brain” Lowet
In our 4th RL Debates presentation, Adam presented broad topics on RL and the brain, including his distributional RL paper.
RL Debates 3: Niels “you have 1000 brains in your brain” Leadholm
In our 3rd RL Debates presentation, Niels from the Thousand Brains Project shared how their team is reverse-engineering the cortex to build sensorimotor syst...
RL Debates 2: Fritz “learning for the sake of learning” Sommer
In our 2nd RL Debates presentation, Fritz introduced a first-principles, information-theoretic approach to modeling exploration through the maximization of ‘...
RL Debates 1: Eli “abolish the value function” Sennesh
In our 3rd meeting (the 1st in the RL Debate Series), we kicked off a contentious discussion on the role of the value function in RL, and why Eli believes it...
Bayesian Online learning in Non-stationary Environments (BONE) by Duran-Martin et al., 2025
In our sceond meeting, we covered BONE, a unifying framework for online Bayesian learning when the environment is changing.
Bayesian Online Natural Gradient (BONG) by Jones et al., 2024
In our inaugural meeting, we covered BONG, which shows us how to do online variational inference in a principled manner.