Paper: arXiv 2412.06794
Authors: Subhasis Dasgupta, Pratik Satpati, Ishika Choudhary, Jaydip Sen
Abstract
In the recent past, there were several works on the prediction of stock price using different methods. Sentiment analysis of news and tweets and relating them to the movement of stock prices have already been explored. But, when we talk about the news, there can be several topics such as politics, markets, sports etc. It was observed that most of the prior analyses dealt with news or comments associated with particular stock prices only or the researchers dealt with overall sentiment scores only. However, it is quite possible that different topics having different levels of impact on the movement of the stock price or an index. The current study focused on bridging this gap by analysing the movement of Nifty 50 index with respect to the sentiments associated with news items related to various different topic such as sports, politics, markets etc. The study established that sentiment scores of news items of different other topics also have a significant impact on the movement of the index.
Complexity vs Empirical Score
- Math Complexity: 2.0/10
- Empirical Rigor: 8.0/10
- Quadrant: Street Traders — empirical and applied, lower math complexity
Why this score: The paper uses relatively simple statistical models like Ridge and Lasso regression with a clear methodology for data scraping and preprocessing, resulting in moderate math complexity. However, it demonstrates high empirical rigor through extensive data collection (400k+ news items over 3 years), implementation of web scraping and sentiment analysis pipelines, and clear backtesting-ready setup for Nifty 50 index prediction.
Research Flowchart
flowchart TD
A["Research Goal:<br>How do news topic sentiments<br>impact Nifty 50 Index movement?"] --> B["Data Collection:<br>Nifty 50 historical data &<br>Multi-topic news articles"]
B --> C["Preprocessing &<br>Topic Modeling LDA"]
C --> D["Sentiment Analysis<br>Calculate scores per topic"]
D --> E["Computational Model<br>Time-series integration & Regression"]
E --> F{"Key Finding"}
F --> G["Sentiment of diverse topics<br>politics, markets, sports<br>significantly predicts index movement"]
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper proposes analyzing Nifty 50 index movements using sentiment scores from news items across multiple topics and evaluating predictive performance with regression models.
Source sentence
The current study focused on bridging this gap by analysing the movement of Nifty 50 index with respect to the sentiments associated with news items related to various different topic such as sports, politics, markets etc.
Assumes 6
- News published on weekends can influence the index movement on Monday. data
Source sentence
But it was quite natural to assume that news of Saturdays and Sundays can easily influence the movement of the Index on Monday.
- Sentiment of a headline reflects the sentiment of the corresponding news article. model
Source sentence
It is quite natural to assume that if the sentiment of the news headline is positive, the sentiment of the corresponding news would also be positive.
- Movement of NIFTY 50 index depends on immediate past news. market
Source sentence
It was assumed that movement of NIFTY 50 index was dependent of immediate past news and hence, these two separate lag values were considered in the analysis.
- Linear regression assumptions (error distribution, multicollinearity) hold for the model. statistical
Source sentence
Linear regression has several important assumptions related to the distribution of errors and multicollinearity.
Show 2 more
- Missing price data can be filled by forward fill (last value carry forward). data
Source sentence
The missing values introduced due to left joining in the variables “Open”, “High”, “Low” and “Close” were filled with forward fill process (i.e., last value carry forward).
- Sentiment scores of topics with frequency below 200 are negligible and can be omitted. data
Source sentence
The threshold frequency was taken as 200 and all the topics which were below this threshold were neglected.
Data
- NIFTY 50 price dataset market-prices unknown
Source sentence
The dataset of NIFTY 50 data from 1st Jan 2021 till 22nd Feb 2024 were also downloaded. This dataset contained “Open”, “High”, “Low” and “Close” values of the NIFTY 50 index date wise.
- News sentiment dataset (VADER) text-news unknown
Source sentence
Next, for news, VADER sentiment analysis was used to extract the overall sentiment of each news item and DistilBERT was used to extract the sentiment score of each headline.
- News headline sentiment dataset (DistilBERT) text-news unknown
Source sentence
DistilBERT gives binary outcome, i.e., POSITIVE and NEGATIVE along with the probability score of the predicted class. For the current study, the requirement was a numeric score.
Costs
Not applicable
Finds 5
- Using VADER sentiment scores with up to 3 lags marginally improves RMSE from 134.85 to 134.15 compared to a model without sentiment. predictive-accuracy
Source sentence
The RMSE observed was 134.85. That way, with VADER sentiment scores on the news data, a marginal improvement in RMSE was observed with up to lag 3 data.
- DistilBERT sentiment scores lead to larger RMSE reductions; Linear Regression RMSE drops to 128.78 (lag 3) from 137.76 (no sentiment). predictive-accuracy
Source sentence
Linear Regression 137.76 128.78 137.45
- Ridge regression RMSE improves from 206.01 (no sentiment, lag 5) to 128.74 (lag 3) when using DistilBERT sentiment scores. predictive-accuracy
Source sentence
Ridge Regression 206.01 128.74 136.98
- Top contributing features in Ridge regression (VADER) include market_lag1 (0.017038) and industry_lag2 (0.006105) for up to 3 lags. statistical
Source sentence
market_lag1 0.017038 market_lag2 0.008143 industry_lag2 0.006105 industry_lag1 0.005057 industry_lag3 0.004473
- Linear regression assumptions are violated (high condition number, non‑normal error distribution), prompting use of regularized models. statistical
Source sentence
Even though the results were impressive, the condition number was very high and the error terms were significantly deviated from the normal distribution.
Checked by others
Supported by
- FNSPID: A Comprehensive Financial News Dataset in Time Series
Compare the findings
This paper: Using VADER sentiment scores with up to 3 lags marginally improves RMSE from 134.85 to 134.15 compared to a model without sentiment.
That paper: Adding sentiment scores modestly enhances performance on the transformer-based model.
Qualified by
- DAM: A Universal Dual Attention Mechanism for Multimodal Timeseries Cryptocurrency Trend Forecasting
Compare the findings
This paper: Using VADER sentiment scores with up to 3 lags marginally improves RMSE from 134.85 to 134.15 compared to a model without sentiment.
That paper: Integrating sentiment data raises TFT model prediction performance by about 20 % and LSTM model by about 5 %.
Code
No repository by the authors found in the paper.
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.