Paper: arXiv 2404.02053
Authors: Enmin Zhu, Jerome Yen
Abstract
This paper explores the intersection of Natural Language Processing (NLP) and financial analysis, focusing on the impact of sentiment analysis in stock price prediction. We employ BERTopic, an advanced NLP technique, to analyze the sentiment of topics derived from stock market comments. Our methodology integrates this sentiment analysis with various deep learning models, renowned for their effectiveness in time series and stock prediction tasks. Through comprehensive experiments, we demonstrate that incorporating topic sentiment notably enhances the performance of these models. The results indicate that topics in stock market comments provide implicit, valuable insights into stock market volatility and price trends. This study contributes to the field by showcasing the potential of NLP in enriching financial analysis and opens up avenues for further research into real-time sentiment analysis and the exploration of emotional and contextual aspects of market sentiment. The integration of advanced NLP techniques like BERTopic with traditional financial analysis methods marks a step forward in developing more sophisticated tools for understanding and predicting market behaviors.
Complexity vs Empirical Score
- Math Complexity: 3.5/10
- Empirical Rigor: 6.0/10
- Quadrant: Street Traders — empirical and applied, lower math complexity
Why this score: The paper demonstrates strong empirical rigor with a comprehensive backtesting methodology using deep learning models (LSTM, CNN) and real-world stock market data, but the math complexity is modest, focusing on applied NLP techniques rather than novel theoretical derivations.
Research Flowchart
flowchart TD
A["Research Goal<br>Predict Stock Prices using<br>Sentiment from Market Comments"] --> B["Data Collection<br>Stock Market Comments"]
B --> C["Topic Modeling<br>BERTopic Analysis"]
C --> D["Sentiment Extraction<br>Per Topic Analysis"]
D --> E{"Model Integration<br>Deep Learning Models"}
E --> F["Prediction Output<br>Stock Price Trends"]
F --> G["Key Findings<br>Enhanced Accuracy<br>NLP Improves Financial Analysis"]
G --> H["Future Work<br>Real-time Analysis &<br>Contextual Insights"]
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper proposes applying BERTopic for topic-level sentiment analysis to improve stock price prediction models.
Source sentence
This paper is the first to employ the BERTopic model for sentiment analysis in the context of the stock market.
Assumes 3
- The stock‑related comments collected from Reddit and Seeking Alpha are representative of market sentiment. data
Source sentence
Given that the data set is constructed of stock-related comments from platforms such as Reddit or Seeking Alpha without utilizing predefined topic tags, it becomes imperative to employ clustering algorithms to categorize these comments into different topics.
- Daily closing prices from Yahoo Finance accurately reflect the true market price for evaluation. market
Source sentence
Finally, we compare these topic-sentiment-based predictions with the actual daily stock prices sourced from Yahoo Finance.
- Sentiment scores derived from BERTopic can be meaningfully combined with topic information to enhance model inputs. model
Source sentence
Because our sentiment scores don’t cover the topic,we will try to use topic sentiment for improving our model.
Data
- Stock comments web‑crawled from Reddit and Seeking Alpha text-news unknown
Source sentence
Our text dataset comprises stock comments obtained by web crawling various internet websites related to the stock market.
- Daily stock prices from Yahoo Finance market-prices public
Source sentence
Finally, we compare these topic-sentiment-based predictions with the actual daily stock prices sourced from Yahoo Finance.
Costs
Not applicable
Finds 1
- Incorporating BERTopic‑derived topic sentiment improves key error metrics by roughly 10% across models. performance
Source sentence
Quantitatively, the key evaluation metrics of RMSE, MAE, R-squared, and MAPE improved by about 10%.
Checked by others
Contradicted by
- Impact of LLMs news Sentiment Analysis on Stock Price Movement Prediction
Compare the findings
This paper: Incorporating BERTopic‑derived topic sentiment improves key error metrics by roughly 10% across models.
That paper: LSTM and tPatchGNN exhibit minimal sensitivity to sentiment integration, achieving their lowest errors without sentiment features.
2 papers in the archive
Code
No repository by the authors found in the paper.
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.
Cited References (31)
- Web media and stock markets: A survey and future directions from a big data perspective (2017)
- Combining the wisdom of crowds and technical analysis for financial market prediction using deep random subspace ensembles (2018)
- Can investor sentiment be used to predict the stock price? Dynamic analysis based on China stock market (2017)
- BERTopic: Neural topic modeling with a class-based TF-IDF procedure (2022) — link
- Do travelers’ reviews depend on the destination? An analysis in coastal and urban peer-to-peer lodgings (2022)
- Sentiment analysis on stock social media for stock price movement prediction (2019)
- Indexing by latent semantic analysis (1990)
- The effect of news and public mood on stock movements (2014)
and 23 more
- Incorporating stock prices and news sentiments for stock market prediction: A case of Hong Kong (2020)
- Stock market movement forecast: A systematic review (2020)
- A comprehensive survey on sentiment analysis: Approaches, challenges and trends (2021)
- Sentiment analysis algorithms and applications: A survey (2014)
- Latent dirichlet allocation (2003)
- Top2vec: Distributed representations of topics (2020) — link
- Intelligent asset allocation via market sentiment views (2018)
- Integrating multiple data sources for stock prediction (2008)
- News impact on stock price return via sentiment analysis (2014)
- Indonesian Hoax News Classification with Multilingual Transformer Model and BERTopic (2022)
- Intelligent asset allocation via market sentiment views (2018)
- Topic modelling of legal documents via legal-bert (2021)
- On dynamic topic models for mining social media (2019)
- Identifying hidden semantic structures in Instagram data: a topic modelling comparison (2021)
- Experimental explorations on short text topic mining between LDA and NMF based Schemes (2019)
- BERT for Arabic topic modeling: An experimental study on BERTopic technique (2021)
- Topic modelling on consumer financial protection bureau data: An approach using BERT based embeddings (2022)
- Deep learning with long short-term memory networks for financial market predictions (2018) — link
- Generative Adversarial Nets (2014) — link
- Sentiment analysis on social media for stock movement prediction (2015)
- Probabilistic latent semantic indexing (1999)
- Learning the parts of objects by non-negative matrix factorization (1999)
- Media coverage and public perception of distance learning during the COVID-19 pandemic: a topic modeling approach based on BERTopic (2022)
0 of 31 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.