Leveraging Vision-Language Models for Granular Market Change Prediction

Paper: arXiv 2301.10166

Authors: Christopher Wimmer, Navid Rekabsaz

Abstract

Predicting future direction of stock markets using the historical data has been a fundamental component in financial forecasting. This historical data contains the information of a stock in each specific time span, such as the opening, closing, lowest, and highest price. Leveraging this data, the future direction of the market is commonly predicted using various time-series models such as Long-Short Term Memory networks. This work proposes modeling and predicting market movements with a fundamentally new approach, namely by utilizing image and byte-based number representation of the stock data processed with the recently introduced Vision-Language models. We conduct a large set of experiments on the hourly stock data of the German share index and evaluate various architectures on stock price prediction using historical stock data. We conduct a comprehensive evaluation of the results with various metrics to accurately depict the actual performance of various approaches. Our evaluation results show that our novel approach based on representation of stock data as text (bytes) and image significantly outperforms strong deep learning-based baselines.

Complexity vs Empirical Score

• Math Complexity: 3.0/10
• Empirical Rigor: 6.5/10
• Quadrant: Street Traders — practical and empirical, lighter on theory

Why this score: The paper introduces an innovative approach by casting time-series stock data into image and byte-level representations for Vision-Language Models (VLMs). While mathematically simple, the empirical evaluation against traditional deep learning baselines on DAX hourly data offers strong practical interest, though broader cross-market validation is needed.

Research Flowchart

  flowchart TD
    A["Research Question: Can Vision-Language Models (VLMs) improve granular stock market prediction over traditional time-series models?"] --> B["Data Input: Hourly stock data from German Share Index (DAX)"]
    B --> C["Data Representation: Convert numerical data into Images & Bytes (Text)"]
    C --> D["Modeling: Train & Evaluate VLMs alongside Deep Learning Baselines (e.g., LSTM)"]
    D --> E["Evaluation: Assess performance using multi-metric financial & predictive benchmarks"]
    E --> F["Key Outcome: VLM approach with byte and image representations significantly outperforms strong DL baselines"]