Onflow: an online portfolio allocation algorithm

ArXiv ID: 2312.05169 “View on arXiv”

Authors: Unknown

Abstract

We introduce Onflow, a reinforcement learning technique that enables online optimization of portfolio allocation policies based on gradient flows. We devise dynamic allocations of an investment portfolio to maximize its expected log return while taking into account transaction fees. The portfolio allocation is parameterized through a softmax function, and at each time step, the gradient flow method leads to an ordinary differential equation whose solutions correspond to the updated allocations. This algorithm belongs to the large class of stochastic optimization procedures; we measure its efficiency by comparing our results to the mathematical theoretical values in a log-normal framework and to standard benchmarks from the ‘old NYSE’ dataset. For log-normal assets, the strategy learned by Onflow, with transaction costs at zero, mimics Markowitz’s optimal portfolio and thus the best possible asset allocation strategy. Numerical experiments from the ‘old NYSE’ dataset show that Onflow leads to dynamic asset allocation strategies whose performances are: a) comparable to benchmark strategies such as Cover’s Universal Portfolio or Helmbold et al. “multiplicative updates” approach when transaction costs are zero, and b) better than previous procedures when transaction costs are high. Onflow can even remain efficient in regimes where other dynamical allocation techniques do not work anymore. Therefore, as far as tested, Onflow appears to be a promising dynamic portfolio management strategy based on observed prices only and without any assumption on the laws of distributions of the underlying assets’ returns. In particular it could avoid model risk when building a trading strategy.

Keywords: Gradient flows, Reinforcement learning, Portfolio optimization, Transaction costs, Stochastic optimization, Multi-Asset

Complexity vs Empirical Score

• Math Complexity: 7.0/10
• Empirical Rigor: 6.0/10
• Quadrant: Holy Grail
• Why: The paper presents a sophisticated reinforcement learning algorithm based on gradient flows and stochastic optimization with theoretical analysis, indicating high mathematical complexity. It is tested on both theoretical (log-normal framework) and empirical (NYSE) data, with results compared to established benchmarks, though the summary does not provide detailed implementation code or full backtest metrics.

  flowchart TD
    Goal["<b>Research Goal</b><br>Optimize online portfolio allocation<br>with transaction costs"] --> Method["<b>Methodology</b><br>Onflow: Gradient Flow Reinforcement Learning<br>Softmax Parameterization & ODE Updates"]
    Method --> Inputs["<b>Data & Inputs</b><br>1. Log-Normal Assets (Theory)<br>2. 'old NYSE' Dataset (Empirical)"]
    Inputs --> Comp["<b>Computational Process</b><br>Stochastic Optimization via Gradient Flows<br>Maximize Expected Log Return<br>Minimize Transaction Fees"]
    Comp --> Results["<b>Key Findings & Outcomes</b><br>1. Matches Markowitz Optimal (Zero Cost)<br>2. Outperforms Benchmarks (High Cost)<br>3. Robust & Model-Free"]