Paper: arXiv 2312.05169

Authors: Gabriel Turinici, Pierre Brugiere

Abstract

We introduce Onflow, a reinforcement learning technique that enables online optimization of portfolio allocation policies based on gradient flows. We devise dynamic allocations of an investment portfolio to maximize its expected log return while taking into account transaction fees. The portfolio allocation is parameterized through a softmax function, and at each time step, the gradient flow method leads to an ordinary differential equation whose solutions correspond to the updated allocations. This algorithm belongs to the large class of stochastic optimization procedures; we measure its efficiency by comparing our results to the mathematical theoretical values in a log-normal framework and to standard benchmarks from the ‘old NYSE’ dataset. For log-normal assets, the strategy learned by Onflow, with transaction costs at zero, mimics Markowitz’s optimal portfolio and thus the best possible asset allocation strategy. Numerical experiments from the ‘old NYSE’ dataset show that Onflow leads to dynamic asset allocation strategies whose performances are: a) comparable to benchmark strategies such as Cover’s Universal Portfolio or Helmbold et al. “multiplicative updates” approach when transaction costs are zero, and b) better than previous procedures when transaction costs are high. Onflow can even remain efficient in regimes where other dynamical allocation techniques do not work anymore. Therefore, as far as tested, Onflow appears to be a promising dynamic portfolio management strategy based on observed prices only and without any assumption on the laws of distributions of the underlying assets’ returns. In particular it could avoid model risk when building a trading strategy.

Complexity vs Empirical Score

  • Math Complexity: 7.0/10
  • Empirical Rigor: 6.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper presents a sophisticated reinforcement learning algorithm based on gradient flows and stochastic optimization with theoretical analysis, indicating high mathematical complexity. It is tested on both theoretical (log-normal framework) and empirical (NYSE) data, with results compared to established benchmarks, though the summary does not provide detailed implementation code or full backtest metrics.

Research Flowchart

  flowchart TD
  Goal["<b>Research Goal</b><br>Optimize online portfolio allocation<br>with transaction costs"] --> Method["<b>Methodology</b><br>Onflow: Gradient Flow Reinforcement Learning<br>Softmax Parameterization & ODE Updates"]
  Method --> Inputs["<b>Data & Inputs</b><br>1. Log-Normal Assets (Theory)<br>2. 'old NYSE' Dataset (Empirical)"]
  Inputs --> Comp["<b>Computational Process</b><br>Stochastic Optimization via Gradient Flows<br>Maximize Expected Log Return<br>Minimize Transaction Fees"]
  Comp --> Results["<b>Key Findings & Outcomes</b><br>1. Matches Markowitz Optimal (Zero Cost)<br>2. Outperforms Benchmarks (High Cost)<br>3. Robust & Model-Free"]

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper proposes Onflow, a reinforcement‑learning portfolio allocation algorithm based on gradient flows and a softmax parameterisation.

Source sentence
We introduce Onflow, a reinforcement learning method for optimizing portfolio allocation via gradient flows.

Assumes 6

  • Zero transaction costs (ξ = 0) in the theoretical convergence analysis. execution
    Source sentence
    Proposition 1 assumes zero transaction costs.
  • Asset dynamics are log‑normal in the continuous‑time limit. model
    Source sentence
    we will consider the continuous limit i.e., T = R+, no transaction fees (ξ = 0) and assume that the asset dynamic is log-normal.
  • No short selling; portfolio weights are positive and sum to one. market
    Source sentence
    We suppose that each π(k) is positive, i.e., no short selling is allowed.
  • Softmax parameterisation of the portfolio ensures all weights are positive. model
    Source sentence
    The portfolio allocation is parameterized through a softmax function.
Show 2 more
  • Transaction costs are small relative to other ODE terms and can be treated as a perturbation. execution
    Source sentence
    such costs exist but are typically an order of magnitude smaller than the other terms in the ODE; they can therefore be incorporated through a perturbation analysis
  • The ODE solver (odeint) default settings are sufficient for implementation. other
    Source sentence
    The ODE resolution is very robust with respect to the method used; in practice we employed the odeint routine from the SciPy Python package

Data

  • Old NYSE market-prices public
    1965-1987 · 36 NYSE stocks
    Source sentence
    For the numerical tests we use the ”Old NYSE” database, a benchmark from the literature listing the prices of 36 stocks quoted on the New York Stock Exchange from 1965 to 1987 (5651 daily prices i.e., T = 5650)
  • High‑volatility tech stocks (NVDA, TSLA) market-prices public
    2020-2025 · NVDA and TSLA
    Source sentence
    we considered two cryptocurrencies : the Bitcoin (BTC-USD ticker) and Ethereum (ETH-USD) and two impactful tech stocks Nvidia (NVDA), Tesla (TSLA). The dataset is constructed with end of day quotes for the period Jan 1st 2020 to Nov 28th 2025.

Costs

Modeled Costs are part of the reported headline results.

A flat fee level ξ is used (e.g., ξ = 2%); results are reported with and without this fee.

Source sentence
the … fee level ξ, specify the parameter τ and solve ODE (11).

Finds 5

  • Onflow replicates the Markowitz optimal portfolio for log‑normal assets with zero transaction costs. theoretical
    Source sentence
    For log-normal assets with zero transaction costs, Onflow replicates Markowitz optimal portfolio, achieving the best possible allocation.
  • With zero transaction costs, Onflow’s performance is comparable to Cover’s Universal Portfolio and Helmbold et al.’s multiplicative updates. performance
    Source sentence
    a) comparable to benchmark strategies such as Cover's Universal Portfolio or Helmbold et al. “multiplicative updates” approach when transaction costs are zero
  • When transaction costs are high (e.g., ξ = 2%), Onflow outperforms previous procedures. performance
    Source sentence
    b) better than previous procedures when transaction costs are high.
  • On high‑volatility assets, Onflow achieves Sharpe ratios of 1.21 (vs 1.19 for EG) for tech stocks and 0.86 (vs 0.82 for EG) for cryptocurrencies. performance
    Source sentence
    Sharpe ratio for ’Onflow’ is 0.86, for ’EG’ 0.82, for ’BTC-USD’ 0.83 and for ’ETH-USD’ 0.87.
  • Across 1000 random 10‑asset samples, Onflow’s Sharpe ratio is statistically superior (p‑value < 10⁻³) to EG, uniform CRP and Split‑and‑Forget. performance
    Source sentence
    a standard Wilcoxon test reveals that the Sharpe ratio of ’Onflow’ is superior to each of the other strategies and this difference is statistically significant (p-value < 10−3).

Checked by others

No other paper in the archive has been found to support or contradict these findings (compared with 30 related papers). Coverage is partial, so this is neither replication nor refutation.

Code

No repository by the authors found in the paper.

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.

Cited References (48)

  1. Universal Portfolios (1991) — link
  2. Note on the Derivatives with Respect to a Parameter of the Solutions of a System of Differential Equations (1919)
  3. SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python (2020) — link
  4. A Deep Deterministic Policy Gradient-based Strategy for Stocks Portfolio Management (2021) — link
  5. Reinforcement Learning for Quantitative Trading (2023) — link
  6. On adaptive control processes (1959) — link
  7. Dynamic time warping (2007)
  8. Does reinforcement learning outperform deep learning and traditional portfolio optimization models in frontier and developed financial markets? (2023) — link
and 40 more
  1. It’s All in the Timing : Simple Active Portfolio Strategies that Outperform Naïve Diversification (2012) — link
  2. On-Line Portfolio Selection Using Multiplicative Updates (1998) — link
  3. Natural Gradient Works Efficiently in Learning (1998) — link
  4. Machine Learning for Financial Engineering (2012) — link
  5. Estimating the Global Minimum Variance Portfolio (2006) — link
  6. Positively Weighted Minimum-Variance Portfolios and the Structure of Asset Expected Returns (1992)
  7. ASYMPTOTICALLY OPTIMAL PORTFOLIOS (1992) — link
  8. Moving average reversion strategy for on-line portfolio selection (2015) — link
  9. Universal portfolios with and without transaction costs (1997)
  10. Efficient algorithms for universal portfolios (2002)
  11. Online Universal Dirichlet Factor Portfolios — → 2308.07763
  12. Universal portfolio selection (1998)
  13. Universal portfolios with side information (1996)
  14. Online portfolio selection: A survey (2014)
  15. Proposed Algorithms with Risk Management (2016) — link
  16. The Cost of Achieving the Best Portfolio in Hindsight (1998) — link
  17. Portfolio Selection (1952)
  18. OLPS: a toolbox for on-line portfolio selection (2016)
  19. Prediction, learning, and games (2006)
  20. Asset allocation: combining investor views with market equilibrium (1990)
  21. A New Interpretation of Information Rate (1956) — link
  22. Universal Portfolios (2013) — link
  23. Gradient flows: in metric spaces and in the space of probability measures (2005)
  24. Deterministic Edge-Preserving Regularization in Computed Imaging (1997) — link
  25. Radon– Sobolev Variational Auto - Encoders (2021) — link
  26. Combining expert weights for online portfolio selection based on the gradient descent algorithm (2021) — link
  27. Model-Free Reinforcement Learning for Financial Portfolios: A Brief Survey (2019) — link
  28. The convergence of the Stochastic Gradient Descent ( SGD ) : a self-contained proof (2023) — link
  29. A Stochastic Approximation Method (1951) — link
  30. Optimality and Approximation with Policy Gradient Methods in Markov Decision Processes (2020) — link
  31. On the Theory of Policy Gradient Methods : Optimality , Approximation , and Distribution Shift (2021) — link
  32. On the Global Convergence Rates of Softmax Policy Gradient Methods (2020) — link
  33. Can We Learn to Beat the Best Stock (2003)
  34. A New Boosting Algorithm for Online Portfolio Selection Based on dynamic Time Warping and Anti -correlation (2024) — link
  35. Reinforcement learning. An introduction (2018)
  36. Deep learning with long short-term memory networks for financial market predictions (2018) — link
  37. Stock market’s price movement prediction with LSTM neural networks (2017) — link
  38. olpsR: On-line Portfolio Selection with R (2019) — link
  39. Predicting Stock Market Index Using LSTM (2022) — link
  40. S& P -500 vs. Nasdaq -100 price movement prediction with LSTM for different daily periods (2025) — link

1 of 48 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.