In our 3rd meeting (the 1st in the RL Debate Series), we kicked off a contentious discussion on the role of the value function in RL, and why Eli believes it should be abolished.

We began with a roundtable of introductions from the debate’s contenders, each giving a brief overview of their position. Eli started by introducing the provocative phrase he coined, “abolish the value function”, [00:56] and arguing for a more complete understanding of the brain that separates decision-making and control problems [02:14]. The discussion also touched on the limitations of reward being an environmental given, with Eli pointing out that to model animal behavior accurately, the internal state of the agent must be considered [39:46]. He proposed that instead of maximizing a “substance” like rewards, we should think in terms of minimizing the “distance” to an optimal trajectory [01:02:49].

Watch the full meeting here:

Updated: