Paper: arXiv 2303.00356

Authors: Boian Lazov

Abstract

In this paper we pursue the question of a fully online trading algorithm (i.e. one that does not need offline training on previously gathered data). For this task we use Double Deep $Q$-learning in the episodic setting with Fast Learning Networks approximating the expected reward $Q$. Additionally, we define the possible terminal states of an episode in such a way as to introduce a mechanism to conserve some of the money in the trading pool when market conditions are seen as unfavourable. Some of these money are taken as profit and some are reused at a later time according to certain criteria. After describing the algorithm, we test it using the 1-minute-tick data for Cardano’s price on Binance. We see that the agent performs better than trading with randomly chosen actions on each timestep. And it does so when tested on the whole dataset as well as on different subsets, capturing different market trends.

Complexity vs Empirical Score

  • Math Complexity: 4.5/10
  • Empirical Rigor: 3.5/10
  • Quadrant: Philosophers — conceptual discussion, limited math and data

Why this score: The paper introduces an interesting online Double Deep Q-learning framework with Fast Learning Networks that avoids offline pre-training. However, the empirical evaluation is extremely limited, relying on a single cryptocurrency asset (Cardano) compared primarily against a random baseline rather than standard quantitative benchmarks.

Research Flowchart

  flowchart TD
    A["Research Goal: Fully Online Trading Agent<br/>(No Offline Training)"] --> B["Data Input: 1-Minute Cardano (ADA) Data from Binance"]
    B --> C["Methodology: Episodic Double Deep Q-Learning<br/>+ Fast Learning Networks (FLN)"]
    C --> D["Computational Process: Online Learning & Dynamic Capital Preservation"]
    D --> E["Evaluation: Tested Against Random Action Baseline Across Market Trends"]
    E --> F["Key Findings: Outperforms Random Trading & Preserves Capital"]