Paper: arXiv 2412.06794

Authors: Subhasis Dasgupta, Pratik Satpati, Ishika Choudhary, Jaydip Sen

Abstract

In the recent past, there were several works on the prediction of stock price using different methods. Sentiment analysis of news and tweets and relating them to the movement of stock prices have already been explored. But, when we talk about the news, there can be several topics such as politics, markets, sports etc. It was observed that most of the prior analyses dealt with news or comments associated with particular stock prices only or the researchers dealt with overall sentiment scores only. However, it is quite possible that different topics having different levels of impact on the movement of the stock price or an index. The current study focused on bridging this gap by analysing the movement of Nifty 50 index with respect to the sentiments associated with news items related to various different topic such as sports, politics, markets etc. The study established that sentiment scores of news items of different other topics also have a significant impact on the movement of the index.

Complexity vs Empirical Score

  • Math Complexity: 2.0/10
  • Empirical Rigor: 8.0/10
  • Quadrant: Street Traders — empirical and applied, lower math complexity

Why this score: The paper uses relatively simple statistical models like Ridge and Lasso regression with a clear methodology for data scraping and preprocessing, resulting in moderate math complexity. However, it demonstrates high empirical rigor through extensive data collection (400k+ news items over 3 years), implementation of web scraping and sentiment analysis pipelines, and clear backtesting-ready setup for Nifty 50 index prediction.

Research Flowchart

  flowchart TD
  A["Research Goal:<br>How do news topic sentiments<br>impact Nifty 50 Index movement?"] --> B["Data Collection:<br>Nifty 50 historical data &<br>Multi-topic news articles"]
  B --> C["Preprocessing &<br>Topic Modeling LDA"]
  C --> D["Sentiment Analysis<br>Calculate scores per topic"]
  D --> E["Computational Model<br>Time-series integration & Regression"]
  E --> F{"Key Finding"}
  F --> G["Sentiment of diverse topics<br>politics, markets, sports<br>significantly predicts index movement"]

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper proposes analyzing Nifty 50 index movements using sentiment scores from news items across multiple topics and evaluating predictive performance with regression models.

Source sentence
The current study focused on bridging this gap by analysing the movement of Nifty 50 index with respect to the sentiments associated with news items related to various different topic such as sports, politics, markets etc.

Assumes 6

  • News published on weekends can influence the index movement on Monday. data
    Source sentence
    But it was quite natural to assume that news of Saturdays and Sundays can easily influence the movement of the Index on Monday.
  • Sentiment of a headline reflects the sentiment of the corresponding news article. model
    Source sentence
    It is quite natural to assume that if the sentiment of the news headline is positive, the sentiment of the corresponding news would also be positive.
  • Movement of NIFTY 50 index depends on immediate past news. market
    Source sentence
    It was assumed that movement of NIFTY 50 index was dependent of immediate past news and hence, these two separate lag values were considered in the analysis.
  • Linear regression assumptions (error distribution, multicollinearity) hold for the model. statistical
    Source sentence
    Linear regression has several important assumptions related to the distribution of errors and multicollinearity.
Show 2 more
  • Missing price data can be filled by forward fill (last value carry forward). data
    Source sentence
    The missing values introduced due to left joining in the variables “Open”, “High”, “Low” and “Close” were filled with forward fill process (i.e., last value carry forward).
  • Sentiment scores of topics with frequency below 200 are negligible and can be omitted. data
    Source sentence
    The threshold frequency was taken as 200 and all the topics which were below this threshold were neglected.

Data

  • NIFTY 50 price dataset market-prices unknown
    1st Jan 2021 to 22nd Feb 2024 · NIFTY 50 index
    Source sentence
    The dataset of NIFTY 50 data from 1st Jan 2021 till 22nd Feb 2024 were also downloaded. This dataset contained “Open”, “High”, “Low” and “Close” values of the NIFTY 50 index date wise.
  • News sentiment dataset (VADER) text-news unknown
    Source sentence
    Next, for news, VADER sentiment analysis was used to extract the overall sentiment of each news item and DistilBERT was used to extract the sentiment score of each headline.
  • News headline sentiment dataset (DistilBERT) text-news unknown
    Source sentence
    DistilBERT gives binary outcome, i.e., POSITIVE and NEGATIVE along with the probability score of the predicted class. For the current study, the requirement was a numeric score.

Costs

Not applicable The paper reports no trading or execution result.

Finds 5

  • Using VADER sentiment scores with up to 3 lags marginally improves RMSE from 134.85 to 134.15 compared to a model without sentiment. predictive-accuracy
    Source sentence
    The RMSE observed was 134.85. That way, with VADER sentiment scores on the news data, a marginal improvement in RMSE was observed with up to lag 3 data.
  • DistilBERT sentiment scores lead to larger RMSE reductions; Linear Regression RMSE drops to 128.78 (lag 3) from 137.76 (no sentiment). predictive-accuracy
    Source sentence
    Linear Regression 137.76 128.78 137.45
  • Ridge regression RMSE improves from 206.01 (no sentiment, lag 5) to 128.74 (lag 3) when using DistilBERT sentiment scores. predictive-accuracy
    Source sentence
    Ridge Regression 206.01 128.74 136.98
  • Top contributing features in Ridge regression (VADER) include market_lag1 (0.017038) and industry_lag2 (0.006105) for up to 3 lags. statistical
    Source sentence
    market_lag1 0.017038 market_lag2 0.008143 industry_lag2 0.006105 industry_lag1 0.005057 industry_lag3 0.004473
  • Linear regression assumptions are violated (high condition number, non‑normal error distribution), prompting use of regularized models. statistical
    Source sentence
    Even though the results were impressive, the condition number was very high and the error terms were significantly deviated from the normal distribution.

Checked by others

Supported by

  • FNSPID: A Comprehensive Financial News Dataset in Time Series (2024)
    On the question: Does adding sentiment scores improve model performance on financial time‑series predictions?
    Compare the findings

    This paper: Using VADER sentiment scores with up to 3 lags marginally improves RMSE from 134.85 to 134.15 compared to a model without sentiment.

    That paper: Adding sentiment scores modestly enhances performance on the transformer-based model.

    Both report modest gains: A notes a modest enhancement, B lowers RMSE from 134.85 to 134.15 (Δ0.70).

Qualified by

  • DAM: A Universal Dual Attention Mechanism for Multimodal Timeseries Cryptocurrency Trend Forecasting (2024)
    On the question: Does adding sentiment data improve prediction accuracy for financial time series?
    Compare the findings

    This paper: Using VADER sentiment scores with up to 3 lags marginally improves RMSE from 134.85 to 134.15 compared to a model without sentiment.

    That paper: Integrating sentiment data raises TFT model prediction performance by about 20 % and LSTM model by about 5 %.

    A reports ~20% gain for TFT and ~5% for LSTM; B shows RMSE drop from 134.85 to 134.15 (~0.5%). Both find sentiment beneficial, though magnitude and metric differ.

Code

No repository by the authors found in the paper.

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.