Paper: arXiv 2609.39261
Authors: Aojie Yuan, Haiyue Zhang, Zijian Su
Abstract
Decision-focused learning (DFL) trains predictors through downstream objectives, but a different loss need not provide an independent parameter-update direction. We characterize this restriction through the predictor Jacobian, using sparse index tracking to distinguish the covariance entries read by the optimizer from the parameter directions available to learning. Rank-one Jacobians make nonzero per-example gradients collinear; a conditional spectral bound describes near-collinearity. A batch-subspace characterization and counterexamples show why these local statements imply neither common minimizers nor collinear batch updates. Experiments examine when geometry translates into decision quality. Across 38 one-parameter equity configurations, DFL gains over MSE remain below 1.8%; a 385-parameter conditional predictor also has pointwise rank one. In validation-tuned shortest-path and knapsack experiments, full-capacity SPO+ reduces mean regret by 11.6% and 10.6%, respectively; only knapsack survives correction across eight comparisons. The capacity contrast persists on fresh datasets across batch orders and training budgets. Holding expressivity fixed, invertible coordinate scaling lowers spectral effective rank and ordinary SGD gains; compensating for the scaling restores the original trajectories. Financial forward-target controls separate forecast accuracy from decision quality; a matched neural comparison finds no aggregate DFL advantage in the tested architecture. These findings distinguish local rank restrictions, coordinate-dependent optimization and predictive accuracy. Predictor geometry helps explain available learning directions, while held-out decision quality remains the test of practical benefit.
Complexity vs Empirical Score
- Math Complexity: 7.5/10
- Empirical Rigor: 8.0/10
- Quadrant: Holy Grail — high math complexity, high empirical rigor
Why this score: This paper presents a rigorous mathematical analysis of Jacobian rank collapse in DFL, backed by extensive empirical validation across financial and synthetic tasks. Its novelty lies in characterizing the limitations of DFL through predictor geometry, offering valuable insights into when and why DFL might underperform. The work is well-structured and provides a clear, albeit dense, explanation of complex concepts.
Research Flowchart
flowchart TD
A["Research Goal: Understand Restrictions in Decision-Focused Learning"] --> B{"Key Methodology: Predictor Jacobian & Sparse Index Tracking"};
B --> C["Data/Inputs: Equity Configurations, Shortest-Path, Knapsack, Financial Forward-Target Data"];
C --> D{"Computational Processes: DFL, MSE, SPO+ Training, SGD, Coordinate Scaling"};
D --> E["Key Findings/Outcomes"];
E --> F{"1. Jacobian Rank Collapse: Collinear Per-Example Gradients (Rank-One)"};
E --> G{"2. DFL Gains Modest: <1.8% in Equity, 11.6%/10.6% in SPO+"};
E --> H{"3. Coordinate-Dependent Optimization & No Aggregate DFL Advantage in Tested Arch."};
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper proposes to characterize decision-focused learning restrictions via the predictor Jacobian and to study how Jacobian rank affects learning directions and decision quality.
Source sentence
We characterize this restriction through the predictor Jacobian, using sparse index tracking to distinguish the covariance entries read by the optimizer from the parameter directions available to learning.
Assumes 5
- Isotropy of the normalized energy of gradient coordinates for a fixed support. statistical
Source sentence
For a fixed support, a sufficient assumption is isotropy of the normalized energy: E[g2ij/∥g∥2] = 1/N2 for every coordinate.
- Hard‑K cardinality constraint (exact K assets) is used instead of ℓ1 relaxation. model
Source sentence
All experiments therefore use hard-K.
- Dynamic stock selection changes the selection rule and can hurt performance. execution
Source sentence
Dynamic selection (re‑selecting stocks at each rebalance by tracking score) combined with DFL hurts performance: TE = 0.0346 (vs. 0.0287 for fixed) with turnover 6× higher.
- Invertible coordinate scaling changes SGD outcomes without changing expressivity. model
Source sentence
Holding expressivity fixed, invertible coordinate scaling lowers spectral effective rank and ordinary SGD gains; compensating for the scaling restores the original trajectories.
Show 1 more
- Transaction‑cost penalty is linear in the ℓ1 norm of weight changes. execution
Source sentence
We augment the QP with an explicit transaction cost penalty: minw w⊤ˆΣw −2w⊤σidx + c∥w − wprev∥1
Data
- 38 one‑parameter equity configurations market-prices public
Source sentence
Across 38 one‑parameter equity configurations, DFL gains over MSE remain below 1.8%; a 385‑parameter conditional predictor also has pointwise rank one.
- Shortest‑path synthetic datasets synthetic public
Source sentence
Each task uses ten independently generated datasets, a shared train‑only ridge initializer, and identical validation budgets for MSE and SPO+ at every capacity.
- Knapsack synthetic datasets synthetic public
Source sentence
Each task uses ten independently generated datasets, a shared train‑only ridge initializer, and identical validation budgets for MSE and SPO+ at every capacity.
Costs
Modeled
Cost coefficient c is set to 100 bps; turnover is 0.030–0.046 vs. 0.002–0.003, adding only 4 bps net cost at K=10 and 3 bps at K=20.
Source sentence
... even at c = 100 bps it adds only 4 bps of net cost at K=10 and 3 bps at K=20, against a TE reduction of 49 and 57 bps respectively.
Finds 5
- DFL gains over MSE are modest, staying below 1.8% across 38 equity configurations. performance
Source sentence
Across 38 one‑parameter equity configurations, DFL gains over MSE remain below 1.8%; a 385‑parameter conditional predictor also has pointwise rank one.
- Full‑capacity SPO+ reduces mean test regret by 11.59% on shortest‑path and 10.59% on knapsack, with only knapsack passing Holm correction across eight tests. performance
Source sentence
Full‑capacity SPO+ reduces mean test regret by 11.59% on shortest path and 10.59% on knapsack.
- Invertible coordinate scaling lowers spectral effective rank and harms ordinary SGD gains; compensating restores original trajectories. theoretical
Source sentence
Holding expressivity fixed, invertible coordinate scaling lowers spectral effective rank and ordinary SGD gains; compensating for the scaling restores the original trajectories.
- Matched neural forward‑target control finds no aggregate DFL advantage in the tested architecture. negative
Source sentence
neural comparison has no aggregate DFL gain in its tested architecture; this is a descriptive result, not an equivalence test.
- Dynamic selection with BD‑DFL regularization can reduce tracking‑error relative to unregularized DFL, but gains are not universal. performance
Source sentence
With dynamic tracking‑score selection at N=100, neural DFL increases mean TE by 8.6% at K=10; BD‑DFL with β = 0.001 reduces it by 8.2% relative to MSE.
Checked by others
No other paper in the archive has been found to support or contradict these findings (compared with 39 related papers). Coverage is partial, so this is neither replication nor refutation.
Code
No repository by the authors found in the paper.
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.
Cited References (34)
- Differentiable Convex Optimization Layers (2019)
- OptNet : Differentiable Optimization as a Layer in Neural Networks (2017)
- Sparse Portfolios for High-Dimensional Financial Index Tracking (2018)
- Integrating Prediction in Mean-Variance Portfolio Optimization (2023)
- Task-Based End-to-End Model Learning in Stochastic Optimization (2017)
- Smart ``Predict, Then Optimize’’ (2022)
- A Well-Conditioned Estimator for Large-Dimensional Covariance Matrices (2004)
- Melding the Data-Decisions Pipeline: Decision-Focused Learning for Combinatorial Optimization (2019)
and 26 more
- Decision-Focused Learning: Foundations, State of the Art, Benchmark and Future Opportunities (2024)
- Mixed-Integer Programming Approaches for Index Tracking and Enhanced Indexation (2009)
- Decision-Focused Learning without Differentiable Optimization: Learning Locally Optimized Decision Losses (2022)
- An Evolutionary Heuristic for the Index Tracking Problem (2003)
- An Efficient Optimization Approach for a Cardinality-Constrained Index Tracking Problem (2016)
- Efficient and Modular Implicit Differentiation (2022)
- Deep Learning for Portfolio Optimization (2020)
- 60 Years of Portfolio Optimization: Practical Challenges and Current Trends (2014)
- Differentiation of Blackbox Combinatorial Solvers (2020)
- Sparse and Stable Markowitz Portfolios (2009)
- Large Covariance Estimation by Thresholding Principal Orthogonal Complements (2013)
- Sparse Inverse Covariance Estimation with the Graphical Lasso (2008)
- Data-Driven Distributionally Robust Optimization Using the Wasserstein Metric: Performance Guarantees and Tractable Reformulations (2018)
- Dynamic Conditional Correlation: A Simple Class of Multivariate Generalized Autoregressive Conditional Heteroskedasticity Models (2002)
- Quantifying Distributional Model Risk via Optimal Transport (2019)
- Near-Optimal Sensor Placements in Gaussian Processes: Theory, Efficient Algorithms and Empirical Studies (2008)
- Sensor Selection via Convex Optimization (2009)
- Decision-Focused Learning with Directional Gradients (2024)
- LPGD : A General Framework for Backpropagation through Embedded Optimization Layers (2024)
- Regret Equals Covariance: A Closed-Form Characterization for Stochastic Optimization (2026) — link
- Decision-Focused Learning via Tangent-Space Projection of Prediction Error (2026) — link
- Decision-Focused Sparse Tangent Portfolio Optimization (2026) — link
- Decision-Induced Ranking Explains Prediction Inflation and Excessive Turnover in SPO -Based Portfolio Optimization — → 2605.01176
- Estimating Covariance for Global Minimum Variance Portfolio: A Decision-Focused Learning Approach — → 2508.10776
- Online Decision-Focused Learning (2026)
- PyEPO : a PyTorch -based end-to-end predict-then-optimize library for linear and integer programming (2024) — link
2 of 34 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.