Paper: arXiv 2609.39261

Authors: Aojie Yuan, Haiyue Zhang, Zijian Su

Abstract

Decision-focused learning (DFL) trains predictors through downstream objectives, but a different loss need not provide an independent parameter-update direction. We characterize this restriction through the predictor Jacobian, using sparse index tracking to distinguish the covariance entries read by the optimizer from the parameter directions available to learning. Rank-one Jacobians make nonzero per-example gradients collinear; a conditional spectral bound describes near-collinearity. A batch-subspace characterization and counterexamples show why these local statements imply neither common minimizers nor collinear batch updates. Experiments examine when geometry translates into decision quality. Across 38 one-parameter equity configurations, DFL gains over MSE remain below 1.8%; a 385-parameter conditional predictor also has pointwise rank one. In validation-tuned shortest-path and knapsack experiments, full-capacity SPO+ reduces mean regret by 11.6% and 10.6%, respectively; only knapsack survives correction across eight comparisons. The capacity contrast persists on fresh datasets across batch orders and training budgets. Holding expressivity fixed, invertible coordinate scaling lowers spectral effective rank and ordinary SGD gains; compensating for the scaling restores the original trajectories. Financial forward-target controls separate forecast accuracy from decision quality; a matched neural comparison finds no aggregate DFL advantage in the tested architecture. These findings distinguish local rank restrictions, coordinate-dependent optimization and predictive accuracy. Predictor geometry helps explain available learning directions, while held-out decision quality remains the test of practical benefit.

Complexity vs Empirical Score

  • Math Complexity: 7.5/10
  • Empirical Rigor: 8.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: This paper presents a rigorous mathematical analysis of Jacobian rank collapse in DFL, backed by extensive empirical validation across financial and synthetic tasks. Its novelty lies in characterizing the limitations of DFL through predictor geometry, offering valuable insights into when and why DFL might underperform. The work is well-structured and provides a clear, albeit dense, explanation of complex concepts.

Research Flowchart

  flowchart TD
    A["Research Goal: Understand Restrictions in Decision-Focused Learning"] --> B{"Key Methodology: Predictor Jacobian & Sparse Index Tracking"};
    B --> C["Data/Inputs: Equity Configurations, Shortest-Path, Knapsack, Financial Forward-Target Data"];
    C --> D{"Computational Processes: DFL, MSE, SPO+ Training, SGD, Coordinate Scaling"};
    D --> E["Key Findings/Outcomes"];
    E --> F{"1. Jacobian Rank Collapse: Collinear Per-Example Gradients (Rank-One)"};
    E --> G{"2. DFL Gains Modest: <1.8% in Equity, 11.6%/10.6% in SPO+"};
    E --> H{"3. Coordinate-Dependent Optimization & No Aggregate DFL Advantage in Tested Arch."};

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper proposes to characterize decision-focused learning restrictions via the predictor Jacobian and to study how Jacobian rank affects learning directions and decision quality.

Source sentence
We characterize this restriction through the predictor Jacobian, using sparse index tracking to distinguish the covariance entries read by the optimizer from the parameter directions available to learning.

Assumes 5

  • Isotropy of the normalized energy of gradient coordinates for a fixed support. statistical
    Source sentence
    For a fixed support, a sufficient assumption is isotropy of the normalized energy: E[g2ij/∥g∥2] = 1/N2 for every coordinate.
  • Hard‑K cardinality constraint (exact K assets) is used instead of ℓ1 relaxation. model
    Source sentence
    All experiments therefore use hard-K.
  • Dynamic stock selection changes the selection rule and can hurt performance. execution
    Source sentence
    Dynamic selection (re‑selecting stocks at each rebalance by tracking score) combined with DFL hurts performance: TE = 0.0346 (vs. 0.0287 for fixed) with turnover 6× higher.
  • Invertible coordinate scaling changes SGD outcomes without changing expressivity. model
    Source sentence
    Holding expressivity fixed, invertible coordinate scaling lowers spectral effective rank and ordinary SGD gains; compensating for the scaling restores the original trajectories.
Show 1 more
  • Transaction‑cost penalty is linear in the ℓ1 norm of weight changes. execution
    Source sentence
    We augment the QP with an explicit transaction cost penalty: minw w⊤ˆΣw −2w⊤σidx + c∥w − wprev∥1

Data

  • 38 one‑parameter equity configurations market-prices public
    Various markets (S&P 500, FTSE 100, Nikkei 225, Euro Stoxx 50, ASX 200, Hang Seng, S&P 150
    Source sentence
    Across 38 one‑parameter equity configurations, DFL gains over MSE remain below 1.8%; a 385‑parameter conditional predictor also has pointwise rank one.
  • Shortest‑path synthetic datasets synthetic public
    Source sentence
    Each task uses ten independently generated datasets, a shared train‑only ridge initializer, and identical validation budgets for MSE and SPO+ at every capacity.
  • Knapsack synthetic datasets synthetic public
    Source sentence
    Each task uses ten independently generated datasets, a shared train‑only ridge initializer, and identical validation budgets for MSE and SPO+ at every capacity.

Costs

Modeled Costs are part of the reported headline results.

Cost coefficient c is set to 100 bps; turnover is 0.030–0.046 vs. 0.002–0.003, adding only 4 bps net cost at K=10 and 3 bps at K=20.

Source sentence
... even at c = 100 bps it adds only 4 bps of net cost at K=10 and 3 bps at K=20, against a TE reduction of 49 and 57 bps respectively.

Finds 5

  • DFL gains over MSE are modest, staying below 1.8% across 38 equity configurations. performance
    Source sentence
    Across 38 one‑parameter equity configurations, DFL gains over MSE remain below 1.8%; a 385‑parameter conditional predictor also has pointwise rank one.
  • Full‑capacity SPO+ reduces mean test regret by 11.59% on shortest‑path and 10.59% on knapsack, with only knapsack passing Holm correction across eight tests. performance
    Source sentence
    Full‑capacity SPO+ reduces mean test regret by 11.59% on shortest path and 10.59% on knapsack.
  • Invertible coordinate scaling lowers spectral effective rank and harms ordinary SGD gains; compensating restores original trajectories. theoretical
    Source sentence
    Holding expressivity fixed, invertible coordinate scaling lowers spectral effective rank and ordinary SGD gains; compensating for the scaling restores the original trajectories.
  • Matched neural forward‑target control finds no aggregate DFL advantage in the tested architecture. negative
    Source sentence
    neural comparison has no aggregate DFL gain in its tested architecture; this is a descriptive result, not an equivalence test.
  • Dynamic selection with BD‑DFL regularization can reduce tracking‑error relative to unregularized DFL, but gains are not universal. performance
    Source sentence
    With dynamic tracking‑score selection at N=100, neural DFL increases mean TE by 8.6% at K=10; BD‑DFL with β = 0.001 reduces it by 8.2% relative to MSE.

Checked by others

No other paper in the archive has been found to support or contradict these findings (compared with 39 related papers). Coverage is partial, so this is neither replication nor refutation.

Code

No repository by the authors found in the paper.

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.

Cited References (34)

  1. Differentiable Convex Optimization Layers (2019)
  2. OptNet : Differentiable Optimization as a Layer in Neural Networks (2017)
  3. Sparse Portfolios for High-Dimensional Financial Index Tracking (2018)
  4. Integrating Prediction in Mean-Variance Portfolio Optimization (2023)
  5. Task-Based End-to-End Model Learning in Stochastic Optimization (2017)
  6. Smart ``Predict, Then Optimize’’ (2022)
  7. A Well-Conditioned Estimator for Large-Dimensional Covariance Matrices (2004)
  8. Melding the Data-Decisions Pipeline: Decision-Focused Learning for Combinatorial Optimization (2019)
and 26 more
  1. Decision-Focused Learning: Foundations, State of the Art, Benchmark and Future Opportunities (2024)
  2. Mixed-Integer Programming Approaches for Index Tracking and Enhanced Indexation (2009)
  3. Decision-Focused Learning without Differentiable Optimization: Learning Locally Optimized Decision Losses (2022)
  4. An Evolutionary Heuristic for the Index Tracking Problem (2003)
  5. An Efficient Optimization Approach for a Cardinality-Constrained Index Tracking Problem (2016)
  6. Efficient and Modular Implicit Differentiation (2022)
  7. Deep Learning for Portfolio Optimization (2020)
  8. 60 Years of Portfolio Optimization: Practical Challenges and Current Trends (2014)
  9. Differentiation of Blackbox Combinatorial Solvers (2020)
  10. Sparse and Stable Markowitz Portfolios (2009)
  11. Large Covariance Estimation by Thresholding Principal Orthogonal Complements (2013)
  12. Sparse Inverse Covariance Estimation with the Graphical Lasso (2008)
  13. Data-Driven Distributionally Robust Optimization Using the Wasserstein Metric: Performance Guarantees and Tractable Reformulations (2018)
  14. Dynamic Conditional Correlation: A Simple Class of Multivariate Generalized Autoregressive Conditional Heteroskedasticity Models (2002)
  15. Quantifying Distributional Model Risk via Optimal Transport (2019)
  16. Near-Optimal Sensor Placements in Gaussian Processes: Theory, Efficient Algorithms and Empirical Studies (2008)
  17. Sensor Selection via Convex Optimization (2009)
  18. Decision-Focused Learning with Directional Gradients (2024)
  19. LPGD : A General Framework for Backpropagation through Embedded Optimization Layers (2024)
  20. Regret Equals Covariance: A Closed-Form Characterization for Stochastic Optimization (2026) — link
  21. Decision-Focused Learning via Tangent-Space Projection of Prediction Error (2026) — link
  22. Decision-Focused Sparse Tangent Portfolio Optimization (2026) — link
  23. Decision-Induced Ranking Explains Prediction Inflation and Excessive Turnover in SPO -Based Portfolio Optimization — → 2605.01176
  24. Estimating Covariance for Global Minimum Variance Portfolio: A Decision-Focused Learning Approach — → 2508.10776
  25. Online Decision-Focused Learning (2026)
  26. PyEPO : a PyTorch -based end-to-end predict-then-optimize library for linear and integer programming (2024) — link

2 of 34 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.