Paper: arXiv 2511.12129

Authors: Hongyang Yang, Xiao-Yang Liu, Qingwei Wu

Abstract

Stock recommendation is vital to investment companies and investors. However, no single stock selection strategy will always win while analysts may not have enough time to check all S&P 500 stocks (the Standard & Poor’s 500). In this paper, we propose a practical scheme that recommends stocks from S&P 500 using machine learning. Our basic idea is to buy and hold the top 20% stocks dynamically. First, we select representative stock indicators with good explanatory power. Secondly, we take five frequently used machine learning methods, including linear regression, ridge regression, stepwise regression, random forest and generalized boosted regression, to model stock indicators and quarterly log-return in a rolling window. Thirdly, we choose the model with the lowest Mean Square Error in each period to rank stocks. Finally, we test the selected stocks by conducting portfolio allocation methods such as equally weighted, mean-variance, and minimum-variance. Our empirical results show that the proposed scheme outperforms the long-only strategy on the S&P 500 index in terms of Sharpe ratio and cumulative returns. This work is fully open-sourced at \href{“https://github.com/AI4Finance-Foundation/Dynamic-Stock-Recommendation-Machine_Learning-Published-Paper-IEEE"}{“GitHub”}.

Complexity vs Empirical Score

  • Math Complexity: 6.0/10
  • Empirical Rigor: 8.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper employs moderate-to-advanced machine learning methods (ridge regression, random forest, GBM) and rolling-window cross-validation, but its empirical rigor is high due to detailed backtesting on S&P 500 data over 27 years, portfolio allocations, and open-source code.

Research Flowchart

  flowchart TD
  A["<b>Research Goal</b><br>Develop a practical ML scheme for dynamic stock recommendation<br>outperforming the S&P 500 index"] --> B["<b>Data Collection</b><br>S&P 500 Stock Data<br>Quarterly Log-Returns<br>Representative Indicators"]
  
  B --> C["<b>Rolling Window Modeling</b><br>5 ML Methods:<br>Linear Regression, Ridge Regression,<br>Stepwise Regression, Random Forest,<br>Generalized Boosted Regression"]
  
  C --> D["<b>Model Selection</b><br>Select model with lowest<br>Mean Square Error (MSE)<br>per period"]
  
  D --> E["<b>Stock Ranking</b><br>Rank stocks based on<br>predicted returns<br>Top 20% selected dynamically"]
  
  E --> F["<b>Portfolio Allocation Testing</b><br>Equally Weighted<br>Mean-Variance<br>Minimum-Variance"]
  
  F --> G["<b>Key Findings</b><br>• Outperforms S&P 500 index<br>• Better Sharpe Ratio<br>• Higher Cumulative Returns<br>• Fully open-sourced on GitHub"]

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper proposes a practical scheme that dynamically recommends and buys the top 20% of S&P 500 stocks each quarter using five machine‑learning models and selects the best‑performing model each period.

Source sentence
In this paper, we propose a practical scheme that recommends stocks from S&P 500 using machine learning. Our basic idea is to buy and hold the top 20% stocks dynamically.

Assumes 6

  • Transaction costs are assumed to be 0.1% (1/1000) of the trade value. execution
    Source sentence
    we consider our transaction cost to be 1/1000 of the value of that trade.
  • The strategy is long‑only with no leverage and a maximum position size of 5% of portfolio value. model
    Source sentence
    ... limits on position sizes (maximum of position size is 5% of portfolio value)
  • Rolling windows of 4–10 years are used for training and a one‑year window for testing. data
    Source sentence
    Rolling windows for training ranges from 16-quarter (4-year) to a maximum of 40-quarter (10-year). This training rolling window is followed by an one-year window for testing
  • Outlier records with release dates after the trade date (≈0.84% of data) are removed. data
    Source sentence
    we delete outlier records that indicate a release date (rdq) after the trade date, which include about 0.84% of the dataset.
Show 2 more
  • Missing factor data >5% leads to factor deletion; stocks with the most missing data are removed. data
    Source sentence
    if one factor has more than 5% missing data, we delete this factor; if a certain stock generates the most missing data, we delete this stock.
  • Quarterly trade dates are lagged by two months to accommodate non‑standard quarter ends. execution
    Source sentence
    We also extend the trade date by two months lag beyond the standard quarter end date in case some companies have a non-standard quarter end date

Data

Backtest sample: 1995 – 2007 · S&P 500 stocks · equities

  • Compustat database · Wharton Research Data Services (WRDS) fundamentals commercial
    06/01/1990 to 06/01/2017 · S&P 500 component stocks (about 1142 stocks)
    Source sentence
    ... period of 27 years (from 06/01/1990 to 06/01/2017). We use all historical S&P 500 component stocks (about 1142 stocks) as the S&P 500 pool

Recognised sources: WRDS Compustat

Costs

Modeled Costs are part of the reported headline results.

Costs are modeled as 0.1% of trade value (1/1000 of the trade value).

Source sentence
we consider our transaction cost to be 1/1000 of the value of that trade.

Finds 4

Reported performance (Dynamic top‑20% quarterly stock selection with min‑variance allocation): Sharpe 0.46 · annual return 9.9% · total return 1181.5% · max drawdown 46.3%

  • All three portfolio allocation methods (equally weighted, mean‑variance, min‑variance) outperformed the market in cumulative return and annualized return. performance
    Source sentence
    all three portfolio allocation methods outperform the market
  • The min‑variance portfolio delivered the highest overall end value (12.81 million) compared to equally weighted (22.14 million) and the S&P 500 (1.93 million) when starting with 1 million. performance
    Source sentence
    End Value in million 10.9917 22.1383 12.81498 1.933153
  • The strategy’s annualized returns were 8.29% (Mean‑Var), 10.77% (Equally), and 9.87% (Min‑Var) versus 5.22% for the S&P 500. performance
    Source sentence
    Annualized Return 8.29% 10.77% 9.87% 5.22%
  • The proposed machine‑learning scheme consistently outperformed the long‑only S&P 500 benchmark in Sharpe ratio and cumulative returns. performance
    Source sentence
    Our empirical results show that the proposed scheme outperforms the long-only strategy on the S&P 500 index in terms of Sharpe ratio and cumulative returns.

Checked by others

No other paper in the archive has been found to support or contradict these findings (compared with 30 related papers). Coverage is partial, so this is neither replication nor refutation.

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.

Cited References (24)

  1. Value Investing: A Look at the Benjain Graham Approach (1996)
  2. SCREENING FOR STOCKS USING THE JOHN TEMPLETON VALUE APPROACH (2001)
  3. The Use of the P/E Ratio to Stock Valuation (2012)
  4. Ls-decomposition for robust recovery of sensory big data (2017)
  5. Parallel Architectures for Improving the Performance of a GA Based Trading System (2012)
  6. The implied cost of capital: A new approach (2011)
  7. Security Analysis, Fifth Edition (1988)
  8. The Intelligent Investor: a Book of Practical Counsel (1959)
and 16 more
  1. Regression-Based Earnings Forecasts (2013)
  2. Deep Learning for Multivariate Financial Time Series (2015)
  3. A LSTM-based method for stock returns prediction: A case study of China stock market (2015)
  4. Investigation Into The Effectiveness Of Long Short Term Memory Networks For Stock Price Prediction (2016) — link
  5. Improving Factor-Based Quantitative Investing by Forecasting Company Fundamentals (2017) — link
  6. The Elements of Statistical Learning, 2nd Edition (2009)
  7. R: A Language and Environment for Statistical Computing.
  8. An Introduction to Statistical Learning with Applications in R (2013)
  9. Least Angle Regression (2004)
  10. Classification and Regression by randomForest (2002)
  11. Generalized Boosted Models: A guide to the gbm package (2007)
  12. Mean-Variance Investing (August 10, 2012)
  13. MATLAB Financial Toolbox: Portfolio Object (2017)
  14. Digesting Anomalies: An Investment Approach (2014)
  15. Fourth-order tensors with multidimensional discrete transforms (2017) — link
  16. Our codes

0 of 24 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.