Paper: arXiv 2404.02053

Authors: Enmin Zhu, Jerome Yen

Abstract

This paper explores the intersection of Natural Language Processing (NLP) and financial analysis, focusing on the impact of sentiment analysis in stock price prediction. We employ BERTopic, an advanced NLP technique, to analyze the sentiment of topics derived from stock market comments. Our methodology integrates this sentiment analysis with various deep learning models, renowned for their effectiveness in time series and stock prediction tasks. Through comprehensive experiments, we demonstrate that incorporating topic sentiment notably enhances the performance of these models. The results indicate that topics in stock market comments provide implicit, valuable insights into stock market volatility and price trends. This study contributes to the field by showcasing the potential of NLP in enriching financial analysis and opens up avenues for further research into real-time sentiment analysis and the exploration of emotional and contextual aspects of market sentiment. The integration of advanced NLP techniques like BERTopic with traditional financial analysis methods marks a step forward in developing more sophisticated tools for understanding and predicting market behaviors.

Complexity vs Empirical Score

  • Math Complexity: 3.5/10
  • Empirical Rigor: 6.0/10
  • Quadrant: Street Traders — empirical and applied, lower math complexity

Why this score: The paper demonstrates strong empirical rigor with a comprehensive backtesting methodology using deep learning models (LSTM, CNN) and real-world stock market data, but the math complexity is modest, focusing on applied NLP techniques rather than novel theoretical derivations.

Research Flowchart

  flowchart TD
  A["Research Goal<br>Predict Stock Prices using<br>Sentiment from Market Comments"] --> B["Data Collection<br>Stock Market Comments"]
  B --> C["Topic Modeling<br>BERTopic Analysis"]
  C --> D["Sentiment Extraction<br>Per Topic Analysis"]
  D --> E{"Model Integration<br>Deep Learning Models"}
  E --> F["Prediction Output<br>Stock Price Trends"]
  F --> G["Key Findings<br>Enhanced Accuracy<br>NLP Improves Financial Analysis"]
  G --> H["Future Work<br>Real-time Analysis &<br>Contextual Insights"]

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper proposes applying BERTopic for topic-level sentiment analysis to improve stock price prediction models.

Source sentence
This paper is the first to employ the BERTopic model for sentiment analysis in the context of the stock market.

Assumes 3

  • The stock‑related comments collected from Reddit and Seeking Alpha are representative of market sentiment. data
    Source sentence
    Given that the data set is constructed of stock-related comments from platforms such as Reddit or Seeking Alpha without utilizing predefined topic tags, it becomes imperative to employ clustering algorithms to categorize these comments into different topics.
  • Daily closing prices from Yahoo Finance accurately reflect the true market price for evaluation. market
    Source sentence
    Finally, we compare these topic-sentiment-based predictions with the actual daily stock prices sourced from Yahoo Finance.
  • Sentiment scores derived from BERTopic can be meaningfully combined with topic information to enhance model inputs. model
    Source sentence
    Because our sentiment scores don’t cover the topic,we will try to use topic sentiment for improving our model.

Data

Recognised sources: Yahoo Finance

Costs

Not applicable The paper reports no trading or execution result.

Finds 1

  • Incorporating BERTopic‑derived topic sentiment improves key error metrics by roughly 10% across models. performance
    Source sentence
    Quantitatively, the key evaluation metrics of RMSE, MAE, R-squared, and MAPE improved by about 10%.

Checked by others

Contradicted by

  • Impact of LLMs news Sentiment Analysis on Stock Price Movement Prediction (2026)
    On the question: Does incorporating sentiment features improve stock prediction error?
    Compare the findings

    This paper: Incorporating BERTopic‑derived topic sentiment improves key error metrics by roughly 10% across models.

    That paper: LSTM and tPatchGNN exhibit minimal sensitivity to sentiment integration, achieving their lowest errors without sentiment features.

    A shows ~10% error reduction with BERTopic sentiment; B shows LSTM and tPatchGNN have lowest errors without sentiment, indicating minimal sensitivity.

Cited by 2 papers in the archive (2 by independent authors)

Code

No repository by the authors found in the paper.

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.

Cited References (31)

  1. Web media and stock markets: A survey and future directions from a big data perspective (2017)
  2. Combining the wisdom of crowds and technical analysis for financial market prediction using deep random subspace ensembles (2018)
  3. Can investor sentiment be used to predict the stock price? Dynamic analysis based on China stock market (2017)
  4. BERTopic: Neural topic modeling with a class-based TF-IDF procedure (2022) — link
  5. Do travelers’ reviews depend on the destination? An analysis in coastal and urban peer-to-peer lodgings (2022)
  6. Sentiment analysis on stock social media for stock price movement prediction (2019)
  7. Indexing by latent semantic analysis (1990)
  8. The effect of news and public mood on stock movements (2014)
and 23 more
  1. Incorporating stock prices and news sentiments for stock market prediction: A case of Hong Kong (2020)
  2. Stock market movement forecast: A systematic review (2020)
  3. A comprehensive survey on sentiment analysis: Approaches, challenges and trends (2021)
  4. Sentiment analysis algorithms and applications: A survey (2014)
  5. Latent dirichlet allocation (2003)
  6. Top2vec: Distributed representations of topics (2020) — link
  7. Intelligent asset allocation via market sentiment views (2018)
  8. Integrating multiple data sources for stock prediction (2008)
  9. News impact on stock price return via sentiment analysis (2014)
  10. Indonesian Hoax News Classification with Multilingual Transformer Model and BERTopic (2022)
  11. Intelligent asset allocation via market sentiment views (2018)
  12. Topic modelling of legal documents via legal-bert (2021)
  13. On dynamic topic models for mining social media (2019)
  14. Identifying hidden semantic structures in Instagram data: a topic modelling comparison (2021)
  15. Experimental explorations on short text topic mining between LDA and NMF based Schemes (2019)
  16. BERT for Arabic topic modeling: An experimental study on BERTopic technique (2021)
  17. Topic modelling on consumer financial protection bureau data: An approach using BERT based embeddings (2022)
  18. Deep learning with long short-term memory networks for financial market predictions (2018) — link
  19. Generative Adversarial Nets (2014) — link
  20. Sentiment analysis on social media for stock movement prediction (2015)
  21. Probabilistic latent semantic indexing (1999)
  22. Learning the parts of objects by non-negative matrix factorization (1999)
  23. Media coverage and public perception of distance learning during the COVID-19 pandemic: a topic modeling approach based on BERTopic (2022)

0 of 31 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.