Paper: arXiv 2311.13326
Authors: Woosung Koh, Insu Choi, Yuntae Jang, Gimin Kang, Woo Chang Kim
Abstract
Curriculum learning and imitation learning have been leveraged extensively in the robotics domain. However, minimal research has been done on leveraging these ideas on control tasks over highly stochastic time-series data. Here, we theoretically and empirically explore these approaches in a representative control task over complex time-series data. We implement the fundamental ideas of curriculum learning via data augmentation, while imitation learning is implemented via policy distillation from an oracle. Our findings reveal that curriculum learning should be considered a novel direction in improving control-task performance over complex time-series. Our ample random-seed out-sample empirics and ablation studies are highly encouraging for curriculum learning for time-series control. These findings are especially encouraging as we tune all overlapping hyperparameters on the baseline – giving an advantage to the baseline. On the other hand, we find that imitation learning should be used with caution.
Complexity vs Empirical Score
- Math Complexity: 6.5/10
- Empirical Rigor: 6.0/10 (capped from 8.0: transaction costs not modelled; see penalty flags)
- Quadrant: Holy Grail — high math complexity, high empirical rigor (effective empirical rigor capped from 8.0)
Why this score: The paper utilizes advanced mathematical frameworks including Markov Decision Processes, signal/noise decomposition, and reinforcement learning theory, while also featuring substantial empirical validation through out-of-sample testing, ablation studies, and hyperparameter tuning on multiple datasets.
Research Flowchart
flowchart TD A["Research Goal<br>Model-Free Control on Financial Time-Series<br>Using Curriculum & Imitation Learning"] --> B["Methodology<br>Data Augmentation (Curriculum)<br>Policy Distillation (Imitation)"] B --> C["Input Data<br>Complex Financial Time-Series"] C --> D["Computational Process<br>Model-Free Control via RL"] D --> E["Comparative Analysis<br>Curriculum vs. Imitation vs. Baseline"] E --> F["Key Findings<br>Curriculum Learning improves performance<br>Imitation Learning requires caution"] F --> G["Outcome<br>Novel Direction for Time-Series Control"]
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper proposes applying curriculum learning via data augmentation and imitation learning via policy distillation from an oracle to improve control tasks on financial time-series.
Source sentence
We implement the fundamental ideas of curriculum learning via data augmentation, while imitation learning is implemented via policy distillation from an oracle.
Assumes 3
- The training data is a fixed sampled approximation to the data-generating process and cannot be further sampled without the passage of time. data
Source sentence
the training data ˆpdata, which is a sampled approximation to pdata, is fixed and can not be further sampled without the passage of time.
- Portfolio exposure is constrained: max aggregate gross exposure of 1 (net [-1,1]) for Data Set 1 and 2 (net [-2,2]) for Data Set 2. execution
Source sentence
Constraint 1 (Data Set 1): The portfolio’s maximum aggregate gross exposure is 1. Equivalently, the net exposure is bound by [-1, 1]. Constraint 2 (Data Set 2): The portfolio’s maximum aggregate gross exposure is 2. Equivalently, the net exposure is bound by [-2, 2].
- All overlapping hyperparameters are tuned for the baseline algorithm, giving the baseline an advantage. model
Source sentence
These findings are especially encouraging as we tune all overlapping hyperparameters on the baseline -- giving an advantage to the baseline.
Data
- Data Set 1 macro unknown
Source sentence
We hereforth refer to Data Sets 1 and 2 as representing the Macro ETFs and Commodity Futures environments in Table 1, respectively.
- Data Set 2 other unknown
Source sentence
We hereforth refer to Data Sets 1 and 2 as representing the Macro ETFs and Commodity Futures environments in Table 1, respectively.
Costs
Not modeled
Finds 2
- Curriculum learning methods (EMA5, IS8, TIS) achieve the best performance across all empirical environments, with EMA5 attaining statistical significance (<0.05 p‑value) in all six cases. performance
Source sentence
each best result is the EMA-based CL we propose—{EMA5, IS8, TIS}. Consistent with expectations, the naïve rounding approach’s performance significantly lags the EMA-based approaches.
- Imitation learning approaches (A2C‑DPD, A2C‑DPD‑LGN, etc.) dramatically worsen performance, often failing to take any position and yielding near‑flat test‑set results. negative
Source sentence
The most extreme case can be viewed in Figure 11 where A2C-DPD and A2C-DPD-LGN refuses to take any position at all, resulting in a nearly flat test set performance.
Checked by others
No other paper in the archive has been found to support or contradict these findings (compared with 16 related papers). Coverage is partial, so this is neither replication nor refutation.
Code
No repository by the authors found in the paper.
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.
Cited References (37)
- SAR: Generalization of Physiological Dexterity via Synergistic Action Representation (2023) — link
- Goal-Conditioned Imitation Learning using Score-based Diffusion Policies (2023) — link
- Teach a Robot to FISH: Versatile Imitation from One Minute of Demonstrations (2023) — link
- Learning Reward Functions from a Combination of Demonstration and Evaluative Feedback (2022) — link
- Recent advances in reinforcement learning in finance (2021) — link
- Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning (2021) — link
- A Survey on Curriculum Learning (2021) — link
- Portfolio optimization with return prediction using deep learning and machine learning (2021) — link
and 29 more
- Universal Trading for Order Execution with Oracle Policy Distillation (2021) — link
- Follow the Object: Curriculum Learning for Manipulation Tasks with Imagined Goals (2020) — link
- Learning perturbation sets for robust machine learning (2020) — link
- Collective Decision Making under Incomplete Knowledge: Possible and Necessary Solutions (2020) — link
- Recent Advances in Robot Learning from Demonstration (2020) — link
- Adaptive Quantitative Trading: An Imitative Deep Reinforcement Learning Approach (2020) — link
- Curriculum Learning for Reinforcement Learning Domains: A Framework and Survey (2020) — link
- Universal Adversarial Perturbation via Prior Driven Uncertainty Approximation (2019) — link
- Curriculum Learning in Deep Neural Networks for Financial Forecasting (2019) — link
- On The Power of Curriculum Learning in Training Deep Networks (2019) — link
- Tuning Hyperparameters without Grad Students: Scalable and Robust Bayesian Optimisation with Dragonfly (2019) — link
- Algorithmic financial trading with deep convolutional neural networks: Time series to image conversion approach (2018) — link
- QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation (2018) — link
- A deep learning based stock trading model with 2-D CNN trend detection (2017) — link
- Proximal Policy Optimization Algorithms (2017) — link
- Deep Direct Reinforcement Learning for Financial Signal Representation and Trading (2017) — link
- Asynchronous Methods for Deep Reinforcement Learning (2016) — link
- Implementing deep neural networks for financial market prediction on the Intel Xeon Phi (2015) — link
- Trust Region Policy Optimization (2015) — link
- Long Short-Term Memory (1997) — link
- Random Walks in Stock Market Prices (1965) — link
- Scaling Up Multi-Task Robotic Reinforcement Learning (2021)
- The $100 trillion machine (2021)
- A Hybrid Machine Learning and Dynamic Nonlinear Framework for Determination of Optimum Portfolio Structure (2019) — link
- Author manuscript, published in “American Control Conference (2012)” Model-Free Reinforcement Learning with Continuous Action in Practice (2012)
- Universal Portfolios (1996) — link
- Machine learning port-folio allocation
- Expert Systems With Applications
- International conference on machine learning , 1928–1937
0 of 37 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.