Multi-Armed Bandit

Improving Portfolio Optimization Results with Bandit Networks ArXiv ID: 2410.04217 “View on arXiv” Authors: Unknown Abstract In Reinforcement Learning (RL), multi-armed Bandit (MAB) problems have found applications across diverse domains such as recommender systems, healthcare, and finance. Traditional MAB algorithms typically assume stationary reward distributions, which limits their effectiveness in real-world scenarios characterized by non-stationary dynamics. This paper addresses this limitation by introducing and evaluating novel Bandit algorithms designed for non-stationary environments. First, we present the Adaptive Discounted Thompson Sampling (ADTS) algorithm, which enhances adaptability through relaxed discounting and sliding window mechanisms to better respond to changes in reward distributions. We then extend this approach to the Portfolio Optimization problem by introducing the Combinatorial Adaptive Discounted Thompson Sampling (CADTS) algorithm, which addresses computational challenges within Combinatorial Bandits and improves dynamic asset allocation. Additionally, we propose a novel architecture called Bandit Networks, which integrates the outputs of ADTS and CADTS, thereby mitigating computational limitations in stock selection. Through extensive experiments using real financial market data, we demonstrate the potential of these algorithms and architectures in adapting to dynamic environments and optimizing decision-making processes. For instance, the proposed bandit network instances present superior performance when compared to classic portfolio optimization approaches, such as capital asset pricing model, equal weights, risk parity, and Markovitz, with the best network presenting an out-of-sample Sharpe Ratio 20% higher than the best performing classical model. ...

A General Framework for Portfolio Construction Based on Generative Models of Asset Returns ArXiv ID: 2312.03294 “View on arXiv” Authors: Unknown Abstract In this paper, we present an integrated approach to portfolio construction and optimization, leveraging high-performance computing capabilities. We first explore diverse pairings of generative model forecasts and objective functions used for portfolio optimization, which are evaluated using performance-attribution models based on LASSO. We illustrate our approach using extensive simulations of crypto-currency portfolios, and we show that the portfolios constructed using the vine-copula generative model and the Sharpe-ratio objective function consistently outperform. To accommodate a wide array of investment strategies, we further investigate portfolio blending and propose a general framework for evaluating and combining investment strategies. We employ an extension of the multi-armed bandit framework and use value models and policy models to construct eclectic blended portfolios based on past performance. We consider similarity and optimality measures for value models and employ probability-matching (“blending”) and a greedy algorithm (“switching”) for policy models. The eclectic portfolios are also evaluated using LASSO models. We show that the value model utilizing cosine similarity and logit optimality consistently delivers robust superior performances. The extent of outperformance by eclectic portfolios over their benchmarks significantly surpasses that achieved by individual generative model-based portfolios over their respective benchmarks. ...