Paper: arXiv 2403.14695

Authors: Denis Levchenko, Efstratios Rappos, Shabnam Ataee, Biagio Nigro, Stephan Robert-Nicoud

Abstract

Neural architecture search (NAS) emerged as a way to automatically optimize neural networks for a specific task and dataset. Despite an abundance of research on NAS for images and natural language applications, similar studies for time series data are lacking. Among NAS search spaces, chain-structured are the simplest and most applicable to small datasets like time series. We compare three popular NAS strategies on chain-structured search spaces: Bayesian optimization (specifically Tree-structured Parzen Estimator), the hyperband method, and reinforcement learning in the context of financial time series forecasting. These strategies were employed to optimize simple well-understood neural architectures like the MLP, 1D CNN, and RNN, with more complex temporal fusion transformers (TFT) and their own optimizers included for comparison. We find Bayesian optimization and the hyperband method performing best among the strategies, and RNN and 1D CNN best among the architectures, but all methods were very close to each other with a high variance due to the difficulty of working with financial datasets. We discuss our approach to overcome the variance and provide implementation recommendations for future users and researchers.

Complexity vs Empirical Score

  • Math Complexity: 4.5/10
  • Empirical Rigor: 7.0/10
  • Quadrant: Street Traders — empirical and applied, lower math complexity

Why this score: The paper involves advanced hyperparameter optimization techniques and neural network architecture search, but focuses on practical implementation with real-world financial datasets, backtesting, and specific performance metrics like F1 and AUC.

Research Flowchart

  flowchart TD
  A["Research Goal: Optimize Neural Networks<br>for Financial Time Series Forecasting"] --> B["Methodology: Chain-Structured NAS<br>with 3 Strategies & 5 Architectures"]
  B --> C["Data Inputs: Financial Datasets<br>(High Variance, Small Size)"]
  C --> D{"Computational Process"}
  D --> E["Bayesian Optimization<br>(Tree-structured Parzen Estimator)"]
  D --> F["Hyperband Method<br>(Resource-based Pruning)"]
  D --> G["Reinforcement Learning<br>(Policy Gradient)"]
  E & F & G --> H["Key Outcomes"]
  H --> I["Best Strategies: Bayesian & Hyperband"]
  H --> J["Best Architectures: RNN & 1D CNN"]

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper evaluates three NAS strategies on chain-structured search spaces for financial time series forecasting and reports that Bayesian optimization and hyperband perform best, with RNN and 1D CNN yielding the top architectures.

Source sentence
We find Bayesian optimization and the hyperband method performing best among the strategies, and RNN and 1D CNN best among the architectures, but all methods were very close to each other with a high variance due to the difficulty of working with financial datasets.

Assumes 3

  • Data are split chronologically with 70% training, 20% validation, and 10% test. data
    Source sentence
    The data was split into (in historical order): first 70% for training, the following 20% for validation used in the evaluation during optimization runs, and the last 10% for final testing on unseen data.
  • Financial time series are noisy and non‑stationary, leading to low signal‑to‑noise ratios. market
    Source sentence
    The problem is challenging due to the inherently noisy and non-stationary nature of financial time series.
  • Binary classification performance is evaluated primarily with AUC, balanced accuracy and F1, assuming these metrics reflect predictive usefulness. statistical
    Source sentence
    We mostly focused on AUC as we found it to be the hardest metric to optimize, but we paid close attention to bACC and F1 too and report…

Data

  • Germany dataset market-prices unknown
    Source sentence
    Our best results were obtained on the Germany dataset, with time-derived redundant features removed but no PCA applied.
  • Japan dataset market-prices unknown
    Source sentence
    For the Japan dataset, the redundant time-derived features removed and PCA applied retaining most of the information in the dataset while further cutting the number of features approximately in half gave the best results.
  • US dataset market-prices unknown
    Source sentence
    The US dataset was most challenging for us. Although tuned 1d CNNs gave an AUC score of 0.6 ± 0.02 on validation data (not used for training)

Costs

Not applicable The paper reports no trading or execution result.

Finds 5

  • On the Germany dataset, an LSTM tuned by hyperband achieved an average AUC of 0.56 ± 0.05 and balanced accuracy of 0.54 ± 0.04 over 50 test runs. performance
    Source sentence
    The LSTM with parameters selected by the hyperband method applied to the unseen test data achieved an AUC score of 0.56 on average over 50 test runs starting from different random seeds (see Section 5), with 0.05 standard deviation. The same architecture showed balanced accuracy score of 0.54 ± 0.04.
  • A 1D CNN found by Bayesian optimization on the Germany dataset obtained AUC 0.54 ± 0.05 and a high F1 of 0.6 ± 0.06. performance
    Source sentence
    1D CNNs were also able to achieve good performance on this dataset, with the best architecture provided by Bayesian optimization giving an AUC score of 0.54 ± 0.05 and a high F1 score of 0.6 ± 0.06.
  • For the Japan dataset, the best model was a 1D CNN from Bayesian optimization with AUC 0.54 ± 0.03 and F1 0.65 ± 0.02. performance
    Source sentence
    The best performing architecture was a 1D CNN coming from Bayesian optimization, achieving an AUC score of 0.54 ± 0.03 over 50 test runs, with a high F1 score of 0.65 ± 0.02.
  • On the US dataset, no architecture achieved average test AUC above 0.5, indicating poor predictive performance. negative
    Source sentence
    on average no architecture could achieve AUC over 0.5 on the test dataset.
  • Overall, Bayesian optimization and hyperband outperformed reinforcement learning, while LSTMs and 1D CNNs consistently beat simple feed‑forward networks across datasets. performance
    Source sentence
    For search strategy, overall the hyperband method and Bayesian optimization showed better performance than the reinforcement learning based approach, but all three were very close to each other.

Checked by others

No other paper in the archive has been found to support or contradict these findings (compared with 25 related papers). Coverage is partial, so this is neither replication nor refutation.

Code

No repository by the authors found in the paper.

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.

Cited References (88)

  1. CNNpred: CNN-based stock market prediction using a diverse set of variables (2019) — link
  2. Neural Architecture Search: A Survey (2019) — link
  3. Neural Architecture Search with Reinforcement Learning (2017)
  4. Algorithms for Hyper-Parameter Optimization (2011) — link
  5. Practical Bayesian Optimization of Machine Learning Algorithms (2012) — link
  6. Gaussian processes for machine learning. (2006)
  7. Taking the Human Out of the Loop: A Review of Bayesian Optimization (2016) — link
  8. A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning (2010)
and 80 more
  1. Towards an Empirical Foundation for Assessing Bayesian Optimization of Hyperparameters (2013) — link
  2. Optuna: A Next-generation Hyperparameter Optimization Framework (2019)
  3. KerasTuner (2019) — link
  4. Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization (2018)
  5. Automated Machine Learning in Action (2022)
  6. Human-level control through deep reinforcement learning (2015)
  7. Q-learning (1992)
  8. On-line Q-learning using connectionist systems (1994)
  9. Mastering the game of Go with deep neural networks and tree search (2016)
  10. A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play (2018)
  11. Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates (2017)
  12. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor (2018)
  13. Informing sequential clinical decision-making through reinforcement learning: an empirical study (2011)
  14. Supervised reinforcement learning with recurrent neural network for dynamic treatment recommendation (2018)
  15. Extremely randomized trees (2006)
  16. Deep reinforcement learning with double q-learning (2016)
  17. Addressing function approximation error in actor-critic methods (2018)
  18. The sharpe ratio (1998)
  19. Principal component analysis (1987)
  20. Long short-term memory (1997)
  21. Temporal Fusion Transformers for interpretable multi-horizon time series forecasting (2021) — link
  22. Deep reinforcement learning that matters (2018)
  23. Sunrise: A simple unified framework for ensemble learning in deep reinforcement learning (2021)
  24. A distributional perspective on reinforcement learning (2017)
  25. Implicit quantile networks for distributional reinforcement learning (2018)
  26. On calibration of modern neural networks (2017)
  27. Neural Architecture Search: Insights from 1000 papers (2023) — link
  28. The Transformer Network for the Traveling Salesman Problem (2021) — link
  29. Combining Reinforcement Learning and Optimal Transport for the Traveling Salesman Problem (2022) — link
  30. Pointerformer: Deep Reinforced Multi-Pointer Transformer for the Traveling Salesman Problem (2023) — link
  31. A Lightweight CNN-Transformer Model for Learning Traveling Salesman Problems (2023) — link
  32. Reinforcement Learning-based Non-Autoregressive Solver for Traveling Salesman Problems (2023) — link
  33. Solving Quadratic Traveling Salesman Problem with Deep Reinforcement Learning (2022) — link
  34. Learning Heuristics for the TSP by Policy Gradient (2018)
  35. Revised Note on Learning Algorithms for Quadratic Assignment with Graph Neural Networks (2018) — link
  36. Solving the Quadratic Assignment Problem using Deep Reinforcement Learning (2023) — link
  37. RL 4 CO : a Unified Reinforcement Learning for Combinatorial Optimization Library (2023) — link
  38. Attention-based Reinforcement Learning for Combinatorial Optimization: Application to Job Shop Scheduling Problem (2024) — link
  39. Accelerating Quadratic Optimization with Reinforcement Learning (2021) — link
  40. Pointer Networks (2015) — link
  41. Neural Combinatorial Optimization with Reinforcement Learning (2017) — link
  42. Unsupervised Machine Learning for the Quadratic Assignment Problem (2023) — link
  43. PyTorch Forecasting (2023) — link
  44. Deep neural networks for quantum circuit mapping (2021) — link
  45. neural-architecture-search (2018)
  46. Reinforcement Learning for Solving the Vehicle Routing Problem (2018) — link
  47. Vehicle Routing Problem Using Reinforcement Learning: Recent Advancements (2022)
  48. The Traveling Salesman Problem: A Computational Study (2006)
  49. Proximal Policy Optimization Algorithms (2017) — link
  50. POPMUSIC for the Travelling Salesman Problem (2019) — link
  51. A Linearithmic Heuristic for the Travelling Salesman Problem (2022) — link
  52. \textbf Design of Heuristic Algorithms for Hard Optimization (2023) — link
  53. On the Emerging Potential of Quantum Annealing Hardware for Combinatorial Optimization (2022) — link
  54. MQT QMAP: Efficient Quantum Circuit Mapping (2023) — link
  55. Machine Learning Optimization of Quantum Circuit Layouts (2023) — link
  56. Optimization of quantum circuit mapping using gate transformation and commutation (2020) — link
  57. A dataset for quantum circuit mapping (2021) — link
  58. The complexity of quantum circuit mapping with fixed parameters (2022) — link
  59. A Dynamic Look-Ahead Heuristic for the Qubit Mapping Problem of NISQ Computers (2020) — link
  60. Reinforcement Learning and DEAR Framework for Solving the Qubit Mapping Problem (2022)
  61. Resource Bounds for Quantum Circuit Mapping via Quantum Circuit Complexity (2024) — link
  62. The symmetric quadratic traveling salesman problem (2013) — link
  63. Exact algorithms and heuristics for the Quadratic Traveling Salesman Problem with an application in bioinformatics (2014) — link
  64. A Polyhedral Study of the Quadratic Traveling Salesman Problem (2016)
  65. An Efficient Hybrid Genetic Algorithm for the Quadratic Traveling Salesman Problem (2023) — link
  66. Algorithms and Experimental Study for the Traveling Salesman Problem of Second Order (2008)
  67. A class of exponential neighbourhoods for the quadratic travelling salesman problem (2020) — link
  68. Geometric and LP-based heuristics for angular travelling salesman problems in the plane (2019) — link
  69. Nearest Neighbor Transformation of Quantum Circuits in 2D Architecture (2020) — link
  70. Machine Learning into Metaheuristics: A Survey and Taxonomy (2021) — link
  71. Machine learning enhancing metaheuristics: a systematic review (2023) — link
  72. The ant system: Optimization by a colony of cooperating agents (1996)
  73. Tree-Structured Parzen Estimator: Understanding Its Algorithm Components and Their Roles for Better Empirical Performance (2023) — link
  74. Particle swarm optimization-based automatic parameter selection for deep neural networks and its applications in large-scale and high-dimensional data (2017) — link
  75. Neural Predictor for Neural Architecture Search (2020)
  76. Neural Architecture Search without Training (2021) — link
  77. Deep learning with long short-term memory networks for financial market predictions (2018) — link
  78. Classification-based financial markets prediction using deep neural networks (2017) — link
  79. Evolving Deep Convolutional Neural Networks for Image Classification (2017) — link
  80. Pattern Recognition and Machine Learning (Information Science and Statistics) (2006)

0 of 88 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.