{"paper_id":"2510.14156","title":"On Evaluating Loss Functions for Stock Ranking: An Empirical Analysis With Transformer Model","authors":["Jan Kwiatkowski","Jarosław A. Chudziak"],"paper_date":"2025-10-15","paper_type":"empirical","paper_url":"https://arxiv.org/pdf/2510.14156v1","page_url":"https://thequant.space/flowcharts/on-evaluating-loss-functions-for-stock-ranking-an-empirical/","scores":{"empirical_rigor":6,"math_complexity":6,"quadrant":"Holy Grail"},"proposes":{"status":"available","statement":"The paper systematically evaluates advanced loss functions for training Transformer models to rank stocks for portfolio selection.","quote":"This paper addresses this gap by systematically evaluating a diverse set of advanced loss functions including pointwise, pairwise, listwise for daily stock return forecasting to facilitate rank-based portfolio selection on S&P 500 data.","summary":"Quantitative trading strategies rely on accurately ranking stocks to identify profitable investments. Effective portfolio management requires models that can reliably order future stock returns. Transformer models are promising for understanding financial time series, but how different training…","strategy":"PortfolioMASTER"},"assumptions":{"status":"available","items":[{"assumption":"A daily rebalanced, long‑only, top‑5 equal‑weight portfolio is used for evaluation.","type":"execution","stated":"explicit","quote":"We simulate a daily rebalanced, long only, top-k portfolio (k = 5) with equal weighting."},{"assumption":"Input features are daily return and turnover, normalized per stock, with a lookback window of 20 days.","type":"data","stated":"explicit","quote":"The input features (daily return and turnover) for each stock over a lookback window T are first projected to a model dimension D."},{"assumption":"Data are split chronologically: 70% training, 15% validation, 15% testing.","type":"data","stated":"explicit","quote":"The data is split chronologically: 70% for training, 15% for validation, and 15% for testing."},{"assumption":"Models are trained up to 50 epochs with AdamW optimizer, weight decay, early stopping, and a learning‑rate scheduler.","type":"model","stated":"explicit","quote":"Models are trained for up to 50 epochs using the AdamW optimizer with weight decay, employing early stopping based on validation loss and a learning rate scheduler."},{"assumption":"Sharpe ratio calculations use a risk‑free rate of 4.3%.","type":"market","stated":"explicit","quote":"Annualized Sharpe Ratio (SR, with a risk-free rate of 4.3%)"}]},"datasets":{"status":"available","items":[{"name":"Daily stock market data for 110 S&P 500 stocks","provider":null,"kind":"market-prices","access":"public","period":"2015-01-03 to 2024-12-03","universe":"Top 10 by market cap from each of the 11 GICS sectors","quote":"We use daily stock market data for N = 110 S&P 500 stocks, selected as the top 10 by market capitalization from each of the 11 GICS sectors. The data spans from 2015-01-03 to 2024-12-03."}],"known":[],"sample":{"start":"2015-01","end":"2024-12","market":"S&P 500 stocks","asset_class":"equities","frequency":"daily"},"data_resolution":"daily"},"transaction_costs":{"status":"available","modeled":"no","headline_net_of_costs":false,"detail":null,"quote":null,"source":"front_matter","rule":"R1","candidates":{"front_matter":"no","evidence_card":"unclear","evidence_card_quoted":false},"meaning":"a trading or execution result is reported without costs"},"results":{"status":"available","evaluation_type":"trading-pnl","findings":[{"finding":"Margin loss yields the highest annualized return (16.23%) and Sharpe ratio (0.7529) among the tested loss functions.","kind":"performance","quote":"The Margin loss achieved the highest Annualized Return (16.23%) and Sharpe Ratio (0.7529)."},{"finding":"ListNet loss also performs strongly with an annualized return of 16.00%, Sharpe ratio 0.7407, and the lowest annualized volatility (15.79%).","kind":"performance","quote":"ListNet also performed strongly, with an AR of 16.00% and SR of 0.7407, along with the lowest Annualized Volatility (15.79%)."},{"finding":"Strong portfolio performance does not always align with the highest predictive quality metrics.","kind":"other","quote":"We observed that strong portfolio performance does not always directly correlate with the highest scores on standard predictive quality metrics."}],"reported_performance":{"strategy":"PortfolioMASTER","sharpe":0.7529,"sharpe_benchmark":null,"annual_return_pct":16.23,"total_return_pct":89.07,"max_drawdown_pct":18.33},"out_of_sample":{"method":"holdout","headline_out_of_sample":true}},"independent_evidence":{"status":"none_found","supports":[],"contradicts":[],"qualifies":[],"compared_with":24,"citations":{"cited_by_in_corpus":1,"by_independent_authors":1,"how_cited":{"used_as_baseline":0,"uses_method_or_data":0,"extends":0,"discusses_findings":0,"mentions":1,"unclassified":0},"independent_papers":[],"scope":"Citations between papers in this corpus only; citations from outside it are not counted."}},"implementation":{"status":"available","public":false,"code_url":null,"note":"No repository presented by the authors as their own was found in the paper. Absence here is not proof that no code exists."},"replication":{"status":"none_found","card":null,"note":"No replication of this paper has been run by thequant.space yet."},"extraction":{"text_source":"fulltext","model":"openai/gpt-oss-120b","version":1,"extracted_at":"2026-11-14T17:24:50Z"},"provenance":{"claims":"Everything here is what the authors state in the paper. Nothing was re-run or reproduced by the platform.","quotes":"Assumptions, datasets, cost detail and findings were extracted by a language model. Every quote is verbatim paper text. Where an item has `support`, \"full\" means every figure in it appears in the quoted sentences and \"partial\" means a figure is stated elsewhere in the paper; items with a figure found nowhere in the paper were discarded.","datasets_known":"datasets.known lists well-known vendors and public datasets by a stable id, found by a dictionary over the full text where the sentence says the data is used; it is independent of the model and covers only names in the dictionary.","flags":"transaction_costs.modeled, results.evaluation_type and results.out_of_sample.method are platform classifications (structured-decision models); see https://thequant.space/score-guide/#penalty-flags","comparisons":"supports / contradicts / qualifies are a language model's reading of two papers' quoted findings, restricted to closely related papers that share no author. They are pointers to read, not verified replications, and coverage is partial.","implementation":"code_url is a repository the paper itself presents as its own (rule-scored, ambiguous cases judged by a model), checked for liveness when it was recorded."},"docs":"https://thequant.space/agents/"}