Paper: arXiv 2310.15082
Authors: Michele Vodret
Abstract
Time irreversibility in neuronal dynamics has recently been demonstrated to correlate with various indicators of cognitive effort in living systems. Using Landauer’s principle, which posits that time-irreversible information processing consumes energy, we establish a thermodynamically consistent measure of cognitive energy cost associated with belief dynamics. We utilize this concept to analyze a two-armed bandit game, a standard decision-making framework under uncertainty, considering exploitation, finite memory, and concurrent allocation to both game options or arms. Through exploitative, prediction-error-based belief dynamics, the decision maker incurs a cognitive energy cost. Initially, we observe the rise of dissipative structures in the steady state of the belief space due to time-reversal symmetry breaking at intermediate exploitative levels. To delve deeper into the belief dynamics, we liken it to the behavior of an active particle subjected to state-dependent noise. This analogy enables us to relate emergent risk aversion to standard thermophoresis, connecting two apparently unrelated concepts. Finally, we numerically compute the time irreversibility of belief dynamics in the steady state, revealing a strong correlation between elevated - yet optimized - cognitive energy cost and optimal decision-making outcomes. This correlation suggests a mechanism for the evolution of living systems towards maximally out-of-equilibrium structures.
Complexity vs Empirical Score
- Math Complexity: 8.5/10
- Empirical Rigor: 3.0/10
- Quadrant: Lab Rats — high math complexity, limited empirical testing
Why this score: The paper relies heavily on advanced mathematical concepts from thermodynamics and stochastic processes (Landauer’s principle, time irreversibility, active particle models), but it lacks any backtesting, real-world datasets, or implementation-focused code, focusing instead on theoretical modeling and numerical simulations within a controlled setup.
Research Flowchart
flowchart TD A["Research Goal:<br>Quantify Cognitive Energy Cost of Decisions"] --> B["Methodology:<br>Landauer's Principle &<br>Two-Armed Bandit Game"] B --> C["Input Data:<br>Exploitative Belief Dynamics<br>Finite Memory"] C --> D["Computational Process:<br>Numerical Simulation of<br>Time-Irreversibility & Thermodynamics"] D --> E["Key Findings:<br>1. Time-Irreversibility & Cost Correlation<br>2. Emergent Risk Aversion (Thermophoresis)<br>3. Optimization via Maximal Out-of-Equilibrium Structures"]
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper proposes a thermodynamically consistent measure of cognitive energy cost based on Landauer's principle and applies it to a two‑armed bandit decision‑making model with exploitation, finite memory, and concurrent allocation to both arms.
Source sentence
Using Landauer's principle, which posits that time-irreversible information processing consumes energy, we establish a thermodynamically consistent measure of cognitive energy cost associated with belief dynamics. We utilize this concept to analyze a two-armed bandit game, a standard decision-making framework under uncertainty, considering exploitation, finite memory, and concurrent allocation to both game options
Assumes 6
- The decision maker invests a single unit of endowment each period between two arms, with investment fraction a_t ∈ [0,1]. model
Source sentence
e scenario where, at every time step t, a decision maker has to invest a single unit of endowment between two ‘arms’ of a slot machine, denoted as A and B. at ∈[0, 1] signifies the investment fraction at time step t on bandit A, while 1 −at does so for bandit B. The rewards yielded by the arms at each time step are RA t at and RB t (1 −at), respectively.
- Rewards from each arm are drawn from time‑independent Gaussian distributions with unknown means and variances. data
Source sentence
Both RA t and RB t are drawn by time-independent Gaussian distributions, chosen arbitrarily such that the support is mostly in the interval [0, 1]. We indicate the mean and variance of RA t respectively as ⟨RA⟩and σ2 A, with analogous notation for RB t ; these pieces of information are unknown to the decision maker.
- The investment fraction a_t is treated as a deterministic continuous variable rather than a Bernoulli random variable. model
Source sentence
First, we consider at to be a deterministic variable, while customarily it is distributed accordingly to a Bernoulli distribution; this is not a crucial characteris-tic of the model and, since it introduces a third source of noise, we neglect fluctuations of this variable. In doing so, we let at vary continuously, incorporating a notion of confidence [30] in the model.
- The forgetting Q‑learning model is modified by using Gaussian rewards instead of Bernoulli rewards. model
Source sentence
3 Second, using Gaussian rewards (RA t and RB t ) diverges from typical cognitive neuroscience models, where these are drawn from Bernoulli distributions [17, 18]. This choice might be favored as it allows point estimates in the update equation to capture all noise statistics.
Show 2 more
- For passive learning (Γ = 0) the belief dynamics reduce to two independent first‑order AR processes that are not time‑reversible in the long run. model
Source sentence
How-ever, we posit that Gaussian noise is more appropriate for the following reason: in passive learning scenarios (Γ = 0), where the investment is split equally on both arms (at = 1/2), the belief dynamics given by Eqs. (2) re-sult in two independent first-order Auto-Regressive (AR) processes. These are not time-reversible in the long run [31] if Bernoulli rewards are considered.
- The noise variance η is set much smaller than the reward variances (σ^2_η ≪ σ^2_A, σ^2_B). statistical
Source sentence
I set the variances of ηA t and ηB t so that var[ηA] = var(ηB) = σ2 η ≪σ2 A, σ2 B. The derivation of the Fokker-Planck equation (see Sec. IV) leads to: Ft = β −ˆRA t + at⟨RA⟩ −ˆRB t + (1 −at)⟨RB⟩ (B.2) and Dt = (β/2)2 σ2 Aa2 t + σ2 η, 0 0, σ2 B(1 −at)2 + σ2 η (B.3) These are the expressions of F and D we use to quantify the irreversibility rate from Monte Carlo simulations by means of Eq. (13) in Fig. 5.
Data
No dataset named in the text.
Costs
Not applicable
Finds 4
- Numerical computation shows a strong correlation between elevated but optimized cognitive energy cost and optimal decision‑making outcomes in the steady‑state belief dynamics. theoretical
Source sentence
Finally, we numerically compute the time irreversibility of belief dynamics in the steady state, revealing a strong correlation between elevated - yet optimized - cognitive energy cost and optimal decision-making outcomes.
- Risk‑aversion emerges as an effective thermophoresis phenomenon in the belief dynamics, consistent with Fokker‑Planck analysis. theoretical
Source sentence
C. Emergent risk-aversion as termophoresis At first glance, the use of point estimates in the up-date equation for the beliefs given by Eqs. (2) appears overly simplistic, especially when considering its lack of direct reference to well-documented human behavioral tendencies, such as risk aversion. However, early numerical analysis on related models revealed that the use of point estimates in the update equation of
- The irreversibility rate peaks at intermediate exploitation levels (moderate Γ), indicating maximal cognitive energy dissipation when decisions are neither fully passive nor fully trapped. statistical
Source sentence
Displayed metrics include (from top to bottom) average difference in beliefs, average earned reward minus the one related to Γ = 0, and irreversibility rate. Irreversibility rate: each point corresponds to the aver-age irreversibility rate across the trajectories.
- For large Γ the belief dynamics become effectively one‑dimensional and recover time‑reversibility, reducing the irreversibility rate. theoretical
Source sentence
The belief dynamics for large Γs is therefore analogous to a single first-order auto-regressive process with Gaussian noise, recovering time reversibility in the steady state. Refer to the panel a) in Fig. 2 for an indicative example of belief dynamics ˆRt = ( ˆRA t , ˆRB t ) in these three regions.
Checked by others
No other paper in the archive has been found to support or contradict these findings (compared with 11 related papers). Coverage is partial, so this is neither replication nor refutation.
Code
No repository by the authors found in the paper.
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.