Paper: arXiv 2402.18485
Authors: Wentao Zhang, Lingxuan Zhao, Haochong Xia, Shuo Sun, Jiaze Sun, Molei Qin, Xinyi Li, Yuqing Zhao, Yilei Zhao, Xinyu Cai, Longtao Zheng, Xinrun Wang, Bo An
Abstract
Financial trading is a crucial component of the markets, informed by a multimodal information landscape encompassing news, prices, and Kline charts, and encompasses diverse tasks such as quantitative trading and high-frequency trading with various assets. While advanced AI techniques like deep learning and reinforcement learning are extensively utilized in finance, their application in financial trading tasks often faces challenges due to inadequate handling of multimodal data and limited generalizability across various tasks. To address these challenges, we present FinAgent, a multimodal foundational agent with tool augmentation for financial trading. FinAgent’s market intelligence module processes a diverse range of data-numerical, textual, and visual-to accurately analyze the financial market. Its unique dual-level reflection module not only enables rapid adaptation to market dynamics but also incorporates a diversified memory retrieval system, enhancing the agent’s ability to learn from historical data and improve decision-making processes. The agent’s emphasis on reasoning for actions fosters trust in its financial decisions. Moreover, FinAgent integrates established trading strategies and expert insights, ensuring that its trading approaches are both data-driven and rooted in sound financial principles. With comprehensive experiments on 6 financial datasets, including stocks and Crypto, FinAgent significantly outperforms 9 state-of-the-art baselines in terms of 6 financial metrics with over 36% average improvement on profit. Specifically, a 92.27% return (a 84.39% relative improvement) is achieved on one dataset. Notably, FinAgent is the first advanced multimodal foundation agent designed for financial trading tasks.
Complexity vs Empirical Score
- Math Complexity: 4.0/10
- Empirical Rigor: 8.0/10
- Quadrant: Street Traders — empirical and applied, lower math complexity
Why this score: The paper focuses on system architecture and multimodal data integration with advanced AI (LLMs, RL, vision), but lacks dense mathematical derivations or novel proofs; it is highly empirical, with specific backtest results, multiple datasets, and clear implementation details for a deployable agent.
Research Flowchart
flowchart TD Start["Research Goal: Create Multimodal Foundation Agent for Financial Trading"] Data["Market Data: News, Prices, Kline Charts (Stocks & Crypto)"] Method["Methodology: FinAgent Architecture"] Process["Computational Processes: Market Intelligence & Dual-Level Reflection"] Tool["Tool-Augmented Strategy Integration"] Out["Outcome: 36% Avg Profit Improvement, 92.27% Return on Specific Dataset"] Start --> Data Data --> Method Method --> Process Process --> Tool Tool --> Out
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper proposes FinAgent, a multimodal foundational trading agent with tool augmentation that processes numerical, textual, and visual data for financial trading.
Source sentence
To address these challenges, we present FinAgent, a multimodal foundational agent with tool augmentation for financial trading.
Assumes 2
- The study focuses on trading a single asset (stock or cryptocurrency) at a time. market
Source sentence
Specifically, we focus on single asset (e.g., stock or Crypto) trading.
- The reward function includes the change of market capital and accounts for commission fees. execution
Source sentence
The reward function leverages the change of market capitals (earned/lost money) [15] with consideration of commission fee [37, 44].
Data
- Six real‑world datasets (five US stock markets and one cryptocurrency) market-prices unknown
Source sentence
To conduct a thorough evaluation of FinAgent, we evaluate it across 6 real‑world datasets. These included five datasets from the US stock markets and one is the cryptocurrency.
- Asset price data (open, high, low, close, adj close) market-prices unknown
Source sentence
i) Asset Price at the day‑level, including price data for open, high, low, close, and adj close;
- Visual data (historical Kline and trading charts) alternative unknown
Source sentence
ii) Visual Data consists of historical Kline charts and trading charts, which are visual representations of asset market data and trading process on a daily basis;
- Asset news coverage (Bloomberg Technology, Seeking Alpha, CNBC, etc.) text-news unknown
Source sentence
iii) Asset News coverage with daily updates from various esteemed sources, including Bloomberg Technology, Seeking Alpha, CNBC Television, and more,
- Expert Guidance (financial experts) other unknown
Source sentence
iv) Expert Guidance provided by financial experts as the auxiliary information,
Costs
Modeled
Commission fees are incorporated into the reward function as part of the trading cost model.
Source sentence
The reward function leverages the change of market capitals (earned/lost money) [15] with consideration of commission fee [37, 44].
Finds 4
total return 92.3%
- FinAgent achieves over 36% average improvement on profit compared to 9 (or 12) state‑of‑the‑art baselines across six datasets. performance
Source sentence
FinAgent significantly outperforms 9 state‑of‑the‑art baselines in terms of 6 financial metrics with over 36% average improvement on profit.
- FinAgent attains a 92.27% return, representing an 84.39% relative improvement on one dataset. performance
Source sentence
Specifically, a 92.27% return (a 84.39% relative improvement) is achieved on one dataset.
- Ablation shows adding low‑level reflection increases ARR% by up to 101% and reduces risk by up to 44% for TSLA and ETHUSD. performance
Source sentence
When compared to using solely M and ML, the integration of the low‑level reflection module leads to an impressive increase in ARR% by 45% to 101% for TSLA, and ETHUSD, and cutting risk by 14% to 44%.
- Adding high‑level reflection further improves ARR% and Sharpe Ratio while slightly increasing maximum drawdown. performance
Source sentence
When comparing the ML and MLH, the addition of the high‑level reflection module significantly enhances the ARR% and SR, while notably reducing risk. This improvement comes with a minor trade‑off: a slight 7% rise in MDD% for TSLA.
Checked by others
No other paper in the archive has been found to support or contradict these findings (compared with 39 related papers). Coverage is partial, so this is neither replication nor refutation.
52 papers in the archive : MM-DREX: Multimodal-Driven Dynamic Routing of LLM Experts … ; FinVision: A Multi-Agent Framework for Stock Market … ; Forecasting Commodity Price Shocks Using Temporal and … ; FinBloom: Knowledge Grounding Large Language Model with … ; Can LLM-based Financial Investing Strategies Outperform the …
Code
No repository by the authors found in the paper.
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.
Cited References (140)
- Soft actor-critic algorithms and applications (2018) — link
- Training verifiers to solve math word problems (2021) — link
- Webgpt: Browser-assisted question-answering with human feedback (2021) — link
- Lamda: Language models for dialog applications (2022) — link
- Llama: Open and efficient foundation language models (2023) — link
- Proximal policy optimization algorithms (2017) — link
- Playing atari with deep reinforcement learning (2013) — link
- FinGPT: Open-Source Financial Large Language Models — → 2306.06031
and 132 more
- FinMem: A performance-enhanced LLM trading agent with layered memory and character design — → 2311.13743
- Efficient capital markets: A review of theory and empirical work (1970)
- Stock market volatility and learning (2016)
- Prune and tune ensembles: low-cost ensemble learning with sparse independent subnetworks (2022)
- No One Representation to Rule Them All: Overlapping Features of Training Methods (2022)
- An adaptive master-slave regularized model for unexpected revenue prediction enhanced with alternative data (2020)
- Computational intelligence and financial markets: A survey and future directions (2016)
- Uncertainty-aware lookahead factor models for quantitative investing (2020)
- Listening to chaotic whispers: A deep learning framework for news-oriented stock trend prediction (2018)
- The theory of the trading firm revisited (1978)
- Stock movement prediction from tweets and historical prices (2018)
- Investment behaviors can tell what inside: Exploring stock intrinsic properties for stock trend prediction (2019)
- Deep ensembles: A loss landscape perspective (2019) — link
- Stock selection via spatiotemporal hypergraph attention network: A learning to rank approach (2021)
- Stock market’s price movement prediction with LSTM neural networks (2017)
- CLVSA: A convolutional LSTM based variational sequence-to-sequence model with attention for predicting trends of financial markets (2019)
- Enhancing stock movement prediction with adversarial training (2018) — link
- Hierarchical multi-scale Gaussian transformer for stock movement prediction. (2020)
- Temporal relational ranking for stock prediction (2019)
- Deep attentive learning for stock movement prediction from social media text and company correlations (2020)
- Qlib: An AI-oriented quantitative investment platform (2020) — link
- Accurate multivariate stock movement prediction via data-axis transformer with multi-level contexts (2021)
- Learning multiple stock trading patterns with temporal routing adaptor and optimal transport (2021)
- The Sharpe ratio (1994)
- When quantitative trading meets machine learning: A pilot survey (2016)
- Algorithmic trading (2011)
- A comprehensive evaluation of ensemble learning for stock-market prediction (2020)
- Stock price prediction via discovering multi-frequency trading patterns (2017)
- Rest: Relational event-driven stock trend forecasting (2021)
- Long-tailed recognition by routing diverse distribution-aware experts (2020)
- Stock investment decision support using an ensemble of L-GEM based on RBFNN diverse trained from different years (2012)
- Predicting the stock market using multiple models (2006)
- Hybrid forecasting model research on stock data mining (2010)
- Bagging predictors (1996)
- DoubleEnsemble: A new ensemble method based on sample reweighting and feature selection for financial data analysis (2020)
- Random classification noise defeats all convex potential boosters (2010)
- Data arameters: A new family of parameters for learning a differentiable curriculum (2019)
- CatBoost: Unbiased boosting with categorical features (2018)
- Stock market forecast using multivariate analysis with bidirectional and stacked (LSTM, GRU) (2018)
- Multi-scale two-way deep neural network for stock trend prediction. (2020)
- Alphastock: A buying-winners-and-selling-losers investment strategy using interpretable deep reinforcement attention networks (2019)
- VolTAGE: Volatility forecasting via text-audio fusion with graph convolution networks for earnings calls (2020)
- Adaptive mixtures of local experts (1991)
- Learning factored representations in a deep mixture of experts (2013) — link
- Outrageously large neural networks: The sparsely-gated mixture-of-experts layer (2017) — link
- Modeling task relationships in multi-task learning with multi-gate mixture-of-experts (2018)
- Scaling vision with sparse mixture of experts (2021)
- Deep mixture of experts via shallow embedding (2020)
- GLaM: Efficient scaling of language models with mixture-of-experts (2021) — link
- A dual-stage attention-based recurrent neural network for time series prediction (2017)
- Designing financial strategies based on artificial neural networks ensembles for stock markets (2018)
- Stock market index prediction using deep neural network ensemble (2017)
- Evaluating multiple classifiers for stock price direction prediction (2015)
- Performance of moving average trading strategies over varying stock market conditions: The Finnish evidence (2014)
- Deep learning with gated recurrent unit networks for financial sequence predictions (2018)
- A LSTM-based method for stock returns prediction: A case study of China stock market (2015)
- Stock market value prediction using neural networks (2010)
- Coherent measures of risk (1999)
- Conditional value-at-risk for general loss distributions (2002)
- Measuring systemic risk (2017)
- Measuring uncertainty (2015)
- Modeling financial uncertainty with multivariate temporal entropy-based curriculums (2021)
- Uncertainty in deep learning (2016)
- Selective Ensembles for Consistent Predictions (2021)
- BatchEnsemble: an Alternative Approach to Efficient Ensemble and Lifelong Learning (2019)
- Hyperparameter ensembles for robustness and uncertainty quantification (2020)
- Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time (2022) — link
- Film: Visual reasoning with a general conditioning layer (2018)
- Flipout: Efficient Pseudo-Independent Weight Perturbations on Mini-Batches (2018)
- Simple and scalable predictive uncertainty estimation using deep ensembles (2017)
- Snapshot ensembles: Train 1, get m for free (2017) — link
- Loss surfaces, mode connectivity, and fast ensembling of dnns (2018)
- Neural network ensembles (1990)
- Scalable bayesian optimization using deep neural networks (2015)
- A survey of uncertainty in deep neural networks (2021) — link
- Neural network ensembles, cross validation, and active learning (1994)
- Efficient and robust automated machine learning (2015)
- Hyperparameter optimization (2019)
- Popular ensemble methods: An empirical study (1999)
- Deep learning (2015)
- Ensemble selection from libraries of models (2004)
- What is being transferred in transfer learning? (2020)
- Averaging weights leads to wider optima and better generalization (2018)
- Cognitive architectures for language agents (2023) — link
- Generative Agents: Interactive simulacra of human behavior (2023) — link
- Toolformer: Language models can teach themselves to use tools (2023) — link
- Palm: Scaling language modeling with pathways (2023)
- Bloomberggpt: A large language model for finance (2023) — link
- Voyager: An open-ended embodied agent with large language models (2023)
- PRUDEX -Compass: Towards systematic evaluation of reinforcement learning in financial markets — → 2302.00586
- Reinforcement learning for quantitative trading (2023)
- TradeMaster: A holistic quantitative trading platform empowered by reinforcement learning (2023)
- Mastering stock markets with efficient mixture of diversified trading experts (2023)
- Random search for hyper-parameter optimization. (2012)
- Quantitative trading: how to build your own algorithmic trading business (2021)
- Deep reinforcement learning for quantitative trading: Challenges and opportunities (2022)
- FinRL : A deep reinforcement learning library for automated stock trading in quantitative finance (2020)
- Quantitative day trading from natural language using reinforcement learning (2021)
- Reinforcement-learning based portfolio management with augmented asset movement prediction states (2020)
- Investor-imitator: A framework for trading knowledge extraction (2018)
- DeepTrader: a deep reinforcement learning approach for risk-return balanced portfolio management with market conditions Embedding (2021)
- Commission fee is not enough: A hierarchical reinforced framework for portfolio management (2021)
- Earnhft: Efficient hierarchical reinforcement learning for high frequency trading — → 2309.12891
- DeepScalper: A risk-aware reinforcement learning framework to capture fleeting intraday trading opportunities (2022)
- Universal trading for order execution with oracle policy distillation (2021)
- Deep reinforcement learning for automated stock trading: An ensemble strategy (2020)
- Deep direct reinforcement learning for financial signal representation and trading (2016)
- Adaptive quantitative trading: An imitative deep reinforcement learning approach (2020)
- Auto-GPT for Online Decision Making: Benchmarks and Additional Opinions (2023) — link
- MetaGPT: Meta programming for a multi-agent collaborative framework (2023) — link
- AppAgent: Multimodal agents as smartphone users (2023) — link
- MM-REACT: Prompting ChatGPT for multimodal reasoning and action (2023) — link
- Chameleon: Plug-and-Play compositional reasoning with large language models (2023) — link
- GPT-4 Technical Report (2023) — link
- Chatgpt (2021) — [link](https://openai.com/ research/chatgpt)
- GPT-4V(ision) system card (2023) — link
- Pangu-Agent: A fine-tunable generalist agent with structured reasoning (2023) — link
- Chain-of-thought prompting elicits reasoning in large language models (2022)
- Why Can Large Language Models Generate Correct Chain-of-Thoughts? (2023) — link
- Talm: Tool augmented language models (2022) — link
- Learn to explain: Multimodal reasoning via thought chains for science question answering (2022)
- Dynamic prompt learning via policy gradient for semi-structured mathematical reasoning (2022) — link
- Theoremqa: A theorem-driven question answering dataset (2023) — link
- Scibench: Evaluating college-level scientific problem-solving abilities of large language models (2023) — link
- Scieval: A multi-level large language model evaluation benchmark for scientific research (2023) — link
- Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts (2023) — link
- Generate rather than retrieve: Large language models are strong context generators (2022) — link
- Billion-scale similarity search with gpus (2019)
- O ptuna: A Next-Generation Hyperparameter Optimization Framework (2019)
- Technical analysis of stock trends (2018)
- Synapse: Trajectory-as-Exemplar Prompting with Memory for Computer Control (2023)
- Lightgbm: A highly efficient gradient boosting decision tree (2017)
4 of 140 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.