Paper: arXiv 2505.11243

Authors: Elliot L. Epstein, Apaar Sadhwani, Kay Giesecke

Abstract

Many prediction problems across science and engineering, especially in finance and economics, involve large cross-sections of individual time series, where each unit (e.g., a loan, stock, or customer) is driven by unit-level features and latent cross-sectional dynamics. While sequence models have advanced per-unit temporal prediction, capturing cross-sectional effects often still relies on hand-crafted summary features. We propose Set-Sequence, a model that learns cross-sectional structure directly, enhancing expressivity and eliminating manual feature engineering. At each time step, a permutation-invariant Set module summarizes the unit set; a Sequence module then models each unit’s dynamics conditioned on both its features and the learned summary. The architecture accommodates unaligned series, supports varying numbers of units at inference, integrates with standard sequence backbones (e.g., Transformers), and scales linearly in cross-sectional size. Across a synthetic contagion task and two large-scale real-world applications, equity portfolio optimization and loan risk prediction, Set-Sequence significantly outperforms strong baselines, delivering higher Sharpe ratios, improved AUCs, and interpretable cross-sectional summaries.

Complexity vs Empirical Score

  • Math Complexity: 7.0/10
  • Empirical Rigor: 6.0/10 (capped from 8.0: reported Sharpe ratio of 4.82 is implausibly high for daily or lower-frequency data; see penalty flags)
  • Quadrant: Holy Grail — high math complexity, high empirical rigor (effective empirical rigor capped from 8.0)

Why this score: The paper introduces a novel neural architecture with theoretical foundations, involving permutation invariance and complexity analysis, which indicates advanced mathematics. It also demonstrates strong empirical results on large-scale, real-world financial datasets (portfolio optimization and mortgage prediction) with quantitative metrics like Sharpe ratios and AUCs, showing high backtest readiness.

Research Flowchart

  flowchart TD
  A["Research Goal:<br>Cross-Sectional Time Series Prediction"] --> B["Methodology:<br>Set-Sequence Model"]
  B --> C["Permutation-Invariant Set Module<br>Summarizes Unit Set"]
  C --> D["Sequence Module<br>Models Dynamics per Unit"]
  D --> E["Training & Evaluation"]
  E --> F["Key Outcomes:<br>Better Sharpe/AUC, Interpretability"]
  
  B -->|"Inputs"| G["Data: Finance/Econ Cross-Sections<br>(e.g., Loans, Stocks)"]
  
  style F fill:#e1f5e1,stroke:#2e7d32,stroke-width:2px

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper proposes the Set-Sequence model, a neural architecture that learns cross‑sectional structure directly and combines a permutation‑invariant set module with a sequence module.

Source sentence
We propose Set-Sequence, a model that learns cross-sectional structure directly, enhancing expressivity and eliminating manual feature engineering.

Assumes 5

  • The transition matrix and dynamics of the latent factor λt are known in the synthetic contagion task. model
    Source sentence
    We assume that the transition matrix is known, and we also assume that the dynamics of λt, λt+1 = βλt + αNt are known
  • Portfolio weights are L1‑normalized to unit leverage each timestep. execution
    Source sentence
    We L1 normalize the portfolio weights at each timestep to maintain unit leverage (||wt||1 = 1).
  • Transaction costs are subtracted from daily returns using a linear cost model. execution
    Source sentence
    cost(wt, wt−1) = 0.0005 ∥wt −wt−1∥1 + 0.0001 ∥max(−wt, 0)∥1.
  • Only the 500 assets with the largest market cap at the last quarter of the training period are retained for the equity experiment. data
    Source sentence
    We include only the 500 assets with the largest market cap at the last quarter of the training period, mimicking the S&P 500 index.
Show 1 more
  • Missing loan features are handled with a missing‑indicator method. data
    Source sentence
    We deal with missing data in other features with the missing indicator method

Data

Backtest sample: 2002-01 – 2021-12 · U.S. equities · equities · daily

  • CRSP stock dataset · CRSP market-prices commercial
    8‑year training windows, tested on follo · 36,600 stocks, filtered to 500 largest‑cap assets
    Source sentence
    ... We include only the 500 assets with the largest market cap at the last quarter of the training period
  • CoreLogic Loan‑Level Market Analytics Dataset · CoreLogic alternative proprietary
    1988‑2024 · Loans from top 4 ZIP codes in greater Los Angeles area (~117,523 loans)
    Source sentence
    We provide additional details about the CoreLogic Loan-Level Market Analytics Dataset. The dataset
  • Synthetic contagion task synthetic unknown
    1000 units
    Source sentence
    synthetic task with 1000 units in total, the Set-Sequence model’s set summary in layer 3 and layer 5, learns the true joint effect λ0,t

Recognised sources: CRSP

Costs

Modeled Costs are part of the reported headline results.

cost(wt, wt−1) = 0.0005 ∥wt −wt−1∥1 + 0.0001 ∥max(−wt, 0)∥1; net return rnet,t = rt − cost(wt, wt−1)

Source sentence
cost(wt, wt−1) = 0.0005 ∥wt −wt−1∥1 + 0.0001 ∥max(−wt, 0)∥1. The return at time t, can then be expressed as rnet,t = rt −cost(wt, wt−1).

Finds 5

Reported performance (Set-Sequence): Sharpe 4.82 · annual return 13.0%

  • Set-Sequence achieves an annualized Sharpe ratio of 4.82 ± 0.12, outperforming all sequence baselines (e.g., S4 at 3.94) and improving by 22 % over the second‑best baseline. performance
    Source sentence
    The Set-Sequence model outperforms every sequence model baseline. We note a high annualized mean Sharpe ratio over the full period of 4.82, 22% higher than the second best sequence model (S4)
  • Set-Sequence’s Sharpe ratio standard deviation across five random seeds is 0.12, lower than Transformer (0.52) and S4 (0.29). performance
    Source sentence
    with a Sharpe ratio standard deviation over 5 random initialized training runs of 0.12, compared with 0.52 for the Transformer and 0.29 for S4
  • Against a domain‑specific CNN‑Transformer, Set-Sequence improves Sharpe ratio by 42 %. performance
    Source sentence
    the general Set-Sequence model outperforms the CNN-Transformer designed specifically for the equities task by 42 % in terms of the Sharpe ratio
  • In the synthetic task, the set summary in layer five correlates 0.98 with the true joint effect λ0,t. statistical
    Source sentence
    with a high correlation (r = 0.98) between the true joint effect and the set summary in layer five
  • When training with a net‑Sharpe objective that accounts for transaction costs, Set-Sequence still outperforms baselines (e.g., net Sharpe 1.64 for LongConv vs 2.17 for S4). performance
    Source sentence
    Table 9: Out of sample annualized performance metrics net transaction costs 2002 - 2016, trained with a net sharpe ratio objective.

Checked by others

No other paper in the archive has been found to support or contradict these findings (compared with 28 related papers). Coverage is partial, so this is neither replication nor refutation.

Cited by 1 paper in the archive

Code

No repository by the authors found in the paper.

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.

Cited References (68)

  1. Simple hardware-efficient long convolutions for sequence modeling (2023)
  2. Effectively Modeling Time Series with Simple Discrete State Spaces (2023) — link
  3. Efficiently Modeling Long Sequences with Structured State Spaces (2022) — link
  4. Mamba: Linear-Time Sequence Modeling with Selective State Spaces (2024) — link
  5. Is Mamba effective for time series forecasting? (2025) — link
  6. Fredformer: Frequency Debiased Transformer for Time Series Forecasting (2024) — link
  7. iTransformer: Inverted Transformers Are Effective for Time Series Forecasting (2024) — link
  8. Crossformer: Transformer Utilizing Cross-Dimension Dependency for Multivariate Time Series Forecasting (2023) — link
and 60 more
  1. Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting (2021) — link
  2. Pyraformer: Low-Complexity Pyramidal Attention for Long-Range Time Series Modeling and Forecasting (2022) — link
  3. FED former: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting (2022) — link
  4. Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting (2021) — link
  5. TSM ixer: An All- MLP Architecture for Time Series Forecast-ing (2023) — link
  6. Frequency-domain MLPs are More Effective Learners in Time Series Forecasting (2023) — link
  7. Exploring the sources of default clustering (2018) — link
  8. Consumer credit-risk models via machine-learning algorithms (2010) — link
  9. Arbitrage, Factor Structure, and Mean-Variance Analysis on Large Asset Markets (1983)
  10. Macroeconomics and Reality (1980)
  11. Affine Point Processes and Portfolio Credit Risk (2010)
  12. Deep Learning for Mortgage Risk* (2020) — link
  13. Portfolio Transformer for Attention-Based Asset Allocation (2023) — link
  14. Set transformer: A framework for attention-based permutation-invariant neural networks (2019)
  15. Deep Sets (2017) — link
  16. Rep the Set: Neural Networks for Learning Set Representations (2019) — link
  17. DeepSetNet: Predicting Sets with Deep Neural Networks (2016) — link
  18. Joint Learning of Set Cardinality and State Distribution (2017) — link
  19. Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges (2021) — link
  20. Set-based Neural Network Encoding (2023) — link
  21. Set functions for time series (2020)
  22. Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networks (2020) — link
  23. Discrete Graph Structure Learning for Forecasting Multiple Time Series (2021) — link
  24. Graph neural network for traffic forecasting: A survey (2022) — link
  25. Moirai: A Time Series Foundation Model for Universal Forecasting (2024) — link
  26. Self-supervised transformer for sparse and irregularly sampled multivariate clinical time-series (2022)
  27. Time series as images: Vision transformer for irregularly sampled time series (2024)
  28. Warpformer: A multi-scale modeling approach for irregular clinical time series (2023)
  29. DCSF: deep convolutional set functions for classification of asynchronous time series (2022)
  30. Tripletformer for Probabilistic Interpolation of Irregularly sampled Time Series (2023)
  31. Latent Linear ODE s with Neural Kalman Filtering for Irregular Time Series Forecasting (2023) — link
  32. Time-parameterized convolutional neural networks for irregularly sampled time series (2023) — link
  33. Deep Factors for Forecasting (2019) — link
  34. Macroeconomic Forecasting Using Diffusion Indexes (2002) — link
  35. Dynamic Factor Models (2011) — link
  36. Macroeconomic nowcasting and forecasting with big data (2018)
  37. Forecasting using principal components from a large number of predictors (2002)
  38. Are more data always better for factor analysis? (2006)
  39. A comparison of direct and iterated multistep AR methods for forecasting macroeconomic time series (2006) — link
  40. How is machine learning useful for macroeconomic forecasting? (2022)
  41. Large panels with common factors and spatial correlation (2011)
  42. Panel time series (2010)
  43. Confidence Intervals for Diffusion Index Forecasts and Inference for Factor-Augmented Regressions (2006)
  44. Deep learning in characteristics-sorted factor models (2024)
  45. The Dynamic, the Static, and the Weak factor models and the analysis of high-dimensional time series (2025) — link
  46. High-dimensional multivariate forecasting with low-rank Gaussian Copula Processes (2019) — link
  47. Think Globally, Act Locally: A Deep Neural Network Approach to High-Dimensional Time Series Forecasting (2019) — link
  48. Normalizing Kalman Filters for Multivariate Time Series Analysis (2020) — link
  49. Statistical Analysis with Missing Data (2014) — link
  50. A New Approach to Linear Filtering and Prediction Problems (1960) — link
  51. Deep Learning in Asset Pricing (2024) — link
  52. Attention is all you need (2017)
  53. Hyena hierarchy: Towards larger convolutional language models (2023)
  54. Hungry hungry hippos: Towards language modeling with state space models (2022) — link
  55. Deep Learning Statistical Arbitrage (2022)
  56. TimeMixer: Decomposable Multiscale Mixing for Time Series Forecasting (2024) — link
  57. The Graph Neural Network Model (2009) — link
  58. Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networks (2020) — link
  59. TimeXer: Empowering Transformers for Time Series Forecasting with Exogenous Variables (2024) — link
  60. TimeMixer++: A General Time Series Pattern Machine for Universal Predictive Analysis (2025) — link

0 of 68 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.