Paper: arXiv 2408.05382

Authors: Ali Habibnia, Mahdi Soltanzadeh

Abstract

This study presents a Reinforcement Learning (RL)-based portfolio management model tailored for high-risk environments, addressing the limitations of traditional RL models and exploiting market opportunities through two-sided transactions and lending. Our approach integrates a new environmental formulation with a Profit and Loss (PnL)-based reward function, enhancing the RL agent’s ability in downside risk management and capital optimization. We implemented the model using the Soft Actor-Critic (SAC) agent with a Convolutional Neural Network with Multi-Head Attention (CNN-MHA). This setup effectively manages a diversified 12-crypto asset portfolio in the Binance perpetual futures market, leveraging USDT for both granting and receiving loans and rebalancing every 4 hours, utilizing market data from the preceding 48 hours. Tested over two 16-month periods of varying market volatility, the model significantly outperformed benchmarks, particularly in high-volatility scenarios, achieving higher return-to-risk ratios and demonstrating robust profitability. These results confirm the model’s effectiveness in leveraging market dynamics and managing risks in volatile environments like the cryptocurrency market.

Complexity vs Empirical Score

  • Math Complexity: 8.5/10
  • Empirical Rigor: 9.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper employs advanced mathematics, including reinforcement learning (SAC with CNN-MHA), dynamic programming, and stochastic optimization frameworks. It demonstrates high empirical rigor with specific implementation details (Binance perpetual futures, 12-asset portfolio, 4-hour rebalancing), two backtested periods (16 months each), and comparative performance metrics.

Research Flowchart

  flowchart TD
  A["Research Goal: High-Risk Portfolio Optimization via RL"] --> B["Methodology: SAC with CNN-MHA Architecture"]
  B --> C["Data: 12 Crypto Assets, 48h Window, 4h Rebalance"]
  C --> D["Computations: PnL-based Reward & Two-Sided Transactions"]
  D --> E["Outcome: Superior Risk-Adjusted Returns in High Volatility"]

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper proposes a reinforcement‑learning based portfolio management model that uses two‑sided transactions and lending to improve performance in high‑risk environments.

Source sentence
This study presents a Reinforcement Learning (RL)-based portfolio management model tailored for high-risk environments, addressing the limitations of traditional RL models and exploiting market opportunities through two-sided transactions and lending.

Assumes 3

  • Full Liquidity: transactions are executed instantly upon decision. execution
    Source sentence
    Full Liquidity: Transactions are assumed to be executed instantly upon decision.
  • Market Neutrality: transactions do not affect future asset prices. market
    Source sentence
    Market Neutrality: Transactions are assumed not to affect future asset prices.
  • Collateral is unnecessary for loans in the model. model
    Source sentence
    we assume it is unnecessary in our context.

Data

Backtest sample: 12-crypto asset portfolio (Binance perpetual futures) · crypto · intraday

  • Binance perpetual futures data · Binance market-prices public
    May 1, 2021 – September 1, 2022 (high‑vo · 12 crypto assets
    Source sentence
    ... Each period lasts 16 months, divided into 12 months (2178 steps) for training and 4 months (732 steps) for testing, featuring 4-hour rebalancing intervals. Additionally, hourly historical price data for cryptocurrencies is sourced from www.cryptodatadownload.com.

Recognised sources: Binance

Costs

Modeled Costs are part of the reported headline results.

A transaction fee (δ) is deducted when opening or closing positions; loans have no fees, only an interest rate (ζ) applies.

Source sentence
When opening or closing positions in exchanges and brokers, a transaction fee (δ) is deducted. However, typically, there are no fees associated with either granting or receiving loans, and only the interest rate (ζ) applies.

Finds 3

Reported performance (RL(Ours)): Sharpe 10.47 · total return 575.6%

  • The model achieved total returns of 575.5% and 145.6% for the high‑volatility and low‑volatility portfolios, respectively, far exceeding benchmark models. performance
    Source sentence
    testing results reveal our model outperformed benchmark models, achieving total returns of 575.5% and 145.6% in Portfolios A and B, respectively
  • The model delivered superior return‑to‑risk ratios, especially higher Sortino and Calmar ratios compared to the Sharpe ratio. performance
    Source sentence
    our model demonstrated superior return-to-risk ratios, particularly in the Sortino and Kalmar ratios compared to the Sharpe ratio.
  • The model excelled during the high‑volatility period, capitalizing on market fluctuations for higher profit while efficiently managing risk. performance
    Source sentence
    the model excelled during the high‑volatility period, illustrating its ability to capitalize on market fluctuations for higher profit while efficiently managing risk.

Checked by others

No other paper in the archive has been found to support or contradict these findings (compared with 27 related papers). Coverage is partial, so this is neither replication nor refutation.

Cited by 1 paper in the archive (1 by independent authors)

Code

No repository by the authors found in the paper.

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.