{"paper_id":"2505.01575","title":"Asset Pricing in Pre-trained Transformer","authors":["Shanyan Lai"],"paper_date":"2025-05-02","paper_type":"empirical","paper_url":"https://arxiv.org/pdf/2505.01575v2","page_url":"https://thequant.space/flowcharts/asset-pricing-in-pre-trained-transformer/","scores":{"empirical_rigor":6,"math_complexity":7.5,"quadrant":"Holy Grail"},"proposes":{"status":"available","statement":"The paper proposes a new Transformer model called Single-directional representative from Transformer (SERT) for pricing US large-cap stocks.","quote":"This paper proposes an innovative Transformer model, Single-directional representative from Transformer (SERT), for US large capital stock pricing.","summary":"This paper proposes an innovative Transformer model, Single-directional representative from Transformer (SERT), for US large capital stock pricing. It also innovatively applies the pre-trained Transformer models under the stock pricing and factor investment context. They are compared with standard…","strategy":"Trans3 (pre-trained Transformer)"},"assumptions":{"status":"available","items":[{"assumption":"Stocks are selected from the top 15% of market capitalisation and have full data in the testing period with less than 50% missing values during training.","type":"data","stated":"explicit","quote":"They satisfy the conditions of having full available data in the testing period and less than 50% of values missing during the training period for meeting the requirement of the ‘going concern’ and data representability."},{"assumption":"The selected stocks are assumed to be ‘too‑big‑to‑fail’ and thus represent a realistic investment universe.","type":"market","stated":"explicit","quote":"As a benefit of the assumption of ‘too-big-to-fail’ and ‘going concern’, this setting is more realistic for practitioners since it protects the investors from systematic risks such as companies are delisted unexpectedly due to insolvency."},{"assumption":"Risk‑free rate is taken from the Kenneth R. French Data Library for constructing excess returns and evaluating Sharpe and Sortino ratios.","type":"data","stated":"explicit","quote":"The risk-free rate for constructing the excess returns and evaluating the Sharp ratio and Sortino ratio is derived from the Kenneth R. French Data Library."},{"assumption":"The softmax trading signal filter removes the worst 50% of signals each month for 420 stocks.","type":"execution","stated":"explicit","quote":"The softmax trading signal filter utilizes the softmax function on sign signals and filters out the worst 50% of trading signals for 420 stocks each month."},{"assumption":"Model performance is evaluated using out‑of‑sample R² and Sortino ratio as primary metrics.","type":"statistical","stated":"explicit","quote":"The best proposed SERT model achieves the highest out-of-sample R2, 11.2% and 10.91% respectively"}]},"datasets":{"status":"available","items":[{"name":"Monthly sorted portfolio returns as factors","provider":"Andrew Y.; Zimmermann (reconstructed)","kind":"other","access":"unknown","period":null,"universe":null,"quote":"In this research, the monthly sorted portfolio returns as factors reconstructed by Andrew Y.; Zimmermann [52] are deployed as the observable factors (original input)."},{"name":"420 large‑capital US stocks excess returns","provider":"CRSP via Wharton Research Data Services (WRDS)","kind":"market-prices","access":"commercial","period":"January 1957 to December 20…","universe":"US NASDAQ and NYSE stocks filtered from the top 15% market cap","quote":"the 420 large capital US stocks’ excess returns as the labels (original output). These stocks come from the US NASDAQ and NYSE markets which are derived from CRSP of Wharton Research Data Services (WRDS).","known_id":"crsp"}],"known":[],"sample":{"start":"1957-01","end":"2022-12","market":"US large-cap stocks","asset_class":"equities","frequency":null},"data_resolution":"unclear"},"transaction_costs":{"status":"available","modeled":"no","headline_net_of_costs":false,"detail":null,"quote":null,"source":"front_matter","rule":"R1","candidates":{"front_matter":"no","evidence_card":"no","evidence_card_quoted":false},"meaning":"a trading or execution result is reported without costs"},"results":{"status":"available","evaluation_type":"trading-pnl","findings":[{"finding":"The SERT model attains the highest out‑of‑sample R² of 11.2% (and 10.91%) during extreme market fluctuations, outperforming standard and encoder‑only Transformers.","kind":"predictive-accuracy","quote":"The best proposed SERT model achieves the highest out-of-sample R2, 11.2% and 10.91% respectively, when extreme market fluctuation takes place followed by pre-trained Transformer models (10.38% and 9.15%)."},{"finding":"SERT’s Sortino ratio is 47% higher than the buy‑and‑hold benchmark for equal‑weighted portfolios and 28% higher for value‑weighted portfolios during the pandemic period.","kind":"performance","quote":"The proposed SERT model achieves a Sortino ratio 47% higher than the buy-and-hold benchmark in the equal-weighted portfolio and 28% higher in the value-weighted portfolio when the pandemic period is attended."},{"finding":"Pre‑trained Transformer models significantly outperform buy‑and‑hold benchmarks during extreme market periods (2112 and 2212) according to Sortino ratio.","kind":"performance","quote":"the proposed pretrained Transformer models significantly outperform the benchmark models as well as the buy-and-hold strategy (BHE) during the extreme market fluctuation (2112 and 2212) according to the Sortino ratio."},{"finding":"Increasing attention‑head numbers improves model performance only insignificantly; layer‑norm‑first does not boost performance.","kind":"negative","quote":"while increasing attention heads improve the model performance insignificantly and applying the 'layer norm first' method do not boost the model performance in our case."}],"reported_performance":{"strategy":"Trans3 (pre-trained Transformer)","sharpe":0.3113,"sharpe_benchmark":null,"annual_return_pct":8.01,"total_return_pct":null,"max_drawdown_pct":32.6},"out_of_sample":{"method":"holdout","headline_out_of_sample":true}},"independent_evidence":{"status":"none_found","supports":[],"contradicts":[],"qualifies":[],"compared_with":25,"citations":{"cited_by_in_corpus":2,"by_independent_authors":0,"how_cited":{"used_as_baseline":0,"uses_method_or_data":0,"extends":0,"discusses_findings":0,"mentions":0,"unclassified":2},"independent_papers":[],"scope":"Citations between papers in this corpus only; citations from outside it are not counted."}},"implementation":{"status":"available","public":false,"code_url":null,"note":"No repository presented by the authors as their own was found in the paper. Absence here is not proof that no code exists."},"replication":{"status":"none_found","card":null,"note":"No replication of this paper has been run by thequant.space yet."},"extraction":{"text_source":"fulltext","model":"openai/gpt-oss-120b","version":1,"extracted_at":"2026-11-14T17:39:24Z"},"provenance":{"claims":"Everything here is what the authors state in the paper. Nothing was re-run or reproduced by the platform.","quotes":"Assumptions, datasets, cost detail and findings were extracted by a language model. Every quote is verbatim paper text. Where an item has `support`, \"full\" means every figure in it appears in the quoted sentences and \"partial\" means a figure is stated elsewhere in the paper; items with a figure found nowhere in the paper were discarded.","datasets_known":"datasets.known lists well-known vendors and public datasets by a stable id, found by a dictionary over the full text where the sentence says the data is used; it is independent of the model and covers only names in the dictionary.","flags":"transaction_costs.modeled, results.evaluation_type and results.out_of_sample.method are platform classifications (structured-decision models); see https://thequant.space/score-guide/#penalty-flags","comparisons":"supports / contradicts / qualifies are a language model's reading of two papers' quoted findings, restricted to closely related papers that share no author. They are pointers to read, not verified replications, and coverage is partial.","implementation":"code_url is a repository the paper itself presents as its own (rule-scored, ambiguous cases judged by a model), checked for liveness when it was recorded."},"docs":"https://thequant.space/agents/"}