Paper: arXiv 2403.00772
Authors: Ziyuan Ma, Conor Ryan, Jim Buckley, Muslim Chochlov
Abstract
Sentiment analysis can be used for stock market prediction. However, existing research has not studied the impact of a user’s financial background on sentiment-based forecasting of the stock market using artificial neural networks. In this work, a novel combination of neural networks is used for the assessment of sentiment-based stock market prediction, based on the financial background of the population that generated the sentiment. The state-of-the-art language processing model Bidirectional Encoder Representations from Transformers (BERT) is used to classify the sentiment and a Long-Short Term Memory (LSTM) model is used for time-series based stock market prediction. For evaluation, the Weibo social networking platform is used as a sentiment data collection source. Weibo users (and their comments respectively) are divided into Authorized Financial Advisor (AFA) and Unauthorized Financial Advisor (UFA) groups according to their background information, as collected by Weibo. The Hong Kong Hang Seng index is used to extract historical stock market change data. The results indicate that stock market prediction learned from the AFA group users is 39.67% more precise than that learned from the UFA group users and shows the highest accuracy (87%) when compared to existing approaches.
Complexity vs Empirical Score
- Math Complexity: 3.5/10
- Empirical Rigor: 7.0/10
- Quadrant: Street Traders — empirical and applied, lower math complexity
Why this score: The paper relies primarily on established deep learning architectures (BERT, LSTM) with minimal novel mathematical formulation, but demonstrates strong empirical rigor through a large-scale, real-world dataset from Weibo, detailed data processing pipelines, public code availability, and clear comparative results against baseline methods.
Research Flowchart
flowchart TD Start["Research Goal: Do financial experts perform better at stock market prediction?"] --> Data["Data: Weibo Comments &<br>Hong Kong Hang Seng Index"] Data --> BERT["Process: BERT Model<br>Classify Sentiment"] BERT --> Groups["Segment Users:<br>Authorized Financial Advisors (AFA)<br>Unauthorized Financial Advisors (UFA)"] Groups --> LSTM["Process: LSTM Model<br>Time-series Prediction"] LSTM --> Compare["Comparison & Evaluation"] Compare --> Outcomes["Outcomes: <br>AFA Predictions 39.67% more precise than UFA<br>Peak Accuracy: 87%"]
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper proposes a novel combination of BERT for sentiment classification and LSTM for time‑series stock prediction, applied to Weibo users grouped by financial advisor status.
Source sentence
A novel combination of natural language processing neural network (BERT) and LSTM neural network towards stock market prediction;
Assumes 2
- Sentiment analysis results have a significant correlation with the stock market after being shifted by T days. statistical
Source sentence
The calculation process is based on the following assumption: the result of sentiment analysis has a significant correlation with the stock market after it is shifted by T days.
- Positive sentiment label is 1 and negative sentiment label is 0. data
Source sentence
the label for positive is 1 and for negative is 0.
Data
- weibo senti 100k text-news public
Source sentence
Fine-tuning dataset of this model comes from ’weibo senti 100k’ data set in SophonPlus [29] and can be accessed and downloaded publicly.
- Hang Seng Index historical data market-prices public
Source sentence
The Hong Kong Hang Seng index is used to extract historical stock market change data.
Costs
Not applicable
Finds 3
- The AFA group achieves a prediction precision of 90.15%, which is 39.67% higher than the UFA group's precision of 64.64%. predictive-accuracy
Source sentence
It can be observed from Table 1 that the prediction precision of the AFA group is 90.15%. The prediction precision of the UFA group is 64.64%. Therefore, the prediction precision of the AFA group for the stock market is 39.67% higher than that of the UFA group.
- Using the BERT+LSTM model, the AFA‑based prediction reaches an accuracy of 87%, outperforming the UFA‑based accuracy of 62% and other reported approaches. performance
Source sentence
This work BERT + LSTM (AFA) Hang Seng Index 87 ... BERT + LSTM (UFA) Hang Seng Index 62
- The mean‑squared error for the AFA model (0.149) is higher than that for the UFA model (0.076). statistical
Source sentence
For the AFA group, the MSE was 0.149 and the valid loss was 0.140. For the UFA group, the MSE was 0.076 and the valid loss was 0.211.
Checked by others
No other paper in the archive has been found to support or contradict these findings (compared with 51 related papers). Coverage is partial, so this is neither replication nor refutation.
Code
No repository by the authors found in the paper.
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.
Cited References (57)
- Predicting the Future with Social Media (2010)
- Stock prediction using twitter sentiment analysis (2012)
- Sentiment analysis on social media for stock movement prediction (2015)
- Exploiting topic based twitter sentiment for stock prediction (2013)
- Stock Market Trend Prediction with Sentiment Analysis based on LSTM Neural Network (2019)
- Twitter mood predicts the stock market (2011)
- Stock market forecasting with financial micro-blog based on sentiment and time series analysis (2017)
- A nonlinear Granger causality test between stock returns and investor sentiment for Chinese stock market: a wavelet-based approach (2016)
and 49 more
- Can investor sentiment be used to predict the stock price? Dynamic analysis based on China stock market (2017)
- News impact on stock price return via sentiment analysis (2014)
- Has microblogging changed stock market behavior? Evidence from China (2016)
- Predicting Stock Price Returns Using Microblog Sentiment for Chinese Stock Market (2017)
- A LSTM-based method for stock returns prediction: A case study of China stock market (2015)
- LSTM Neural Network with Emotional Analysis for prediction of stock price. (2017)
- Bert: Pre-training of deep bidirectional transformers for language understanding (2018) — link
- BERT-based Financial Sentiment Index and LSTM-based Stock Return Predictability (2019) — link
- Long short-term memory (1997)
- Modeling public mood and emotion: Stock market trend prediction with anticipatory computing approach (2019)
- Investor sentiment and the near-term stock market (2004)
- Pre-trained Models for Natural Language Processing: A Survey (20200318)
- A Review of Sentiment Analysis Research in Chinese Language (2017)
- Lexicon-based methods for sentiment analysis (2011)
- Building a Concept-Level Sentiment Dictionary Based on Commonsense Knowledge (2013)
- Efficient Estimation of Word Representations in Vector Space (2013)
- Review of Speech and language processing: an introduction to natural language processing, computational linguistics, and speech recognition by Daniel Jurafsky and James H. Martin. Prentice Hall 2000. (2000)
- More Than Words: Quantifying Language to Measure Firms’ Fundamentals (2008) — link
- NewsCATS: A News Categorization and Trading System (2006)
- Daily stock market forecast from textual web data (1998) — link
- dataabc/weibo-search — link
- tracy-talent/curriculum — [link](https://github.com/tracy-talent/curriculum/tree/master/Data Mining/sentiment)
- google-research/bert — link
- SophonPlus/ChineseNlpCorpus — link
- rguthrie3/DeepLearningForNLPInPytorch — link
- Investing.com_Global financial market information expert: foreign exchange, stocks, futures, bonds, digital currency market and financial news — link
- Stock trend forecasting method based on sentiment analysis and system similarity model (2011)
- Attention-based LSTM for Aspect-level Sentiment Classification (2016)
- Influence of Investor Sentiment on Stock Returns- Analysis of A- shares in China (2018) — link
- Expected stock returns and volatility (1987)
- Stock price synchronicity and stock price crash risk: Based on the mediating effect of herding behavior of QFII (2016)
- An Empirical Study on the Herding Effect in China’s Stock Market (2004)
- Weibo customer service — link
- Building emotional dictionary for sentiment analysis of online news (2014)
- Designing a neural network for forecasting financial and economic time series (1996) — link
- pytorch — link
- Tweets and trades: The information content of stock microblogs (2014)
- How to fine-tune bert for text classification? (2019)
- LSTM can solve hard long time lag problems (1997)
- Financial time series forecasting model based on CEEMDAN and LSTM (2019)
- Impact of Training Dataset Size on Neural Answer Selection Models (2019)
- Sentiment Analysis on Weibo Platform for Stock Prediction (2020)
- Weibo sentiments and stock return: A time-frequency view (2017) — link
- Event-based user classification in weibo media (2014)
- Encyclopedia of machine learning and data mining (2017)
- Adversarial training for aspect-based sentiment analysis with BERT (2020) — link
- Audio albert: A lite bert for self-supervised learning of audio representation (2021)
- Learning to forget: Continual prediction with LSTM (1999)
- Can online emotions predict the stock market in China? (2016)
0 of 57 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.