Paper: arXiv 2609.39420
Authors: Alexey Chernysh, Orkhan Ekhtibarov, Dmitry Zmitrovich
Abstract
Large language models are strong general-purpose code generators, but executable algorithmic trading remains a demanding specialization target: a model must translate a natural-language strategy specification into correct program logic for a specialized trading framework, execute on historical data, produce trades, and remain semantically faithful to the request. We study two complementary mechanisms for specializing language models for this setting: continued pretraining on algorithmic-trading framework code and supervised fine-tuning (SFT) on agent-validated request-to-code pairs. Evaluation is centered on QuantCode-Bench, our 400-task benchmark for Backtrader strategy generation, together with a repository-level SWE-bench-like track. Continued pretraining improves single-turn Judge Pass from 41.5% to 47.5% for Qwen3.5-397B-A17B and from 27.8% to 33.0% for Qwen3.6-35B-A3B. SFT applied after continued pretraining yields a larger gain for Qwen3.6-35B-A3B, reaching 58.2% Judge Pass and 83.5% successful backtests; in agentic evaluation it raises first-turn success from 22.3% to 58.3% and final success after up to 10 turns from 47.5% to 79.5%. Continued pretraining alone improves first-turn agentic success but lowers final success after repair from 47.5% to 32.5%, consistent with degraded instruction following, whereas SFT improves both. We also identify a capability-retention failure: domain specialization degrades parser-conformant structured tool calling, and targeted recovery SFT restores tool-call formatting but not the base checkpoint’s repository-level agent performance. The results show that framework-oriented pretraining, validated SFT, and explicit capability-retention evaluation address distinct failure modes in domain-specific executable code generation.
Complexity vs Empirical Score
- Math Complexity: 4.0/10
- Empirical Rigor: 6.0/10 (capped from 9.0: no out-of-sample test; see penalty flags)
- Quadrant: Street Traders — empirical and applied, lower math complexity (effective empirical rigor capped from 9.0)
Why this score: This paper presents a highly rigorous empirical study on specializing LLMs for algorithmic trading code generation, introducing a new benchmark and detailed evaluation. While the mathematical complexity is moderate, the empirical setup and results are very strong, demonstrating significant novelty in its approach to domain-specific code generation for finance.
Research Flowchart
flowchart TD
A["Research Goal: Specialize LLMs for Executable Algorithmic Trading Code"] --> B{"Methodology: Continued Pretraining & SFT"};
B -- Data/Inputs --> C["Algorithmic Trading Framework Code; Agent-Validated Request-to-Code Pairs"];
C --> D["Computational Processes: Model Training, Evaluation on QuantCode-Bench"];
D --> E["Outcomes: Improved Judge Pass Rate & Backtest Success"];
E --> F["Key Findings: CP & SFT Address Distinct Failure Modes; Capability Retention Issues Identified"];
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper proposes specializing large language models for executable algorithmic‑trading code via continued pretraining on trading‑framework code and supervised fine‑tuning on validated request‑to‑code pairs, and introduces the QuantCode‑Bench benchmark.
Source sentence
We study two complementary mechanisms for specializing language models for this setting: continued pretraining on algorithmic‑trading framework code and supervised fine‑tuning (SFT) on agent‑validated request‑to‑code pairs.
Assumes
No explicit assumptions found in the text.
Data
- QuantCode‑Bench benchmark public
Source sentence
Evaluation is centered on QuantCode‑Bench, our 400‑task benchmark for Backtrader strategy generation, together with a repository‑level SWE‑bench‑like track.
- SFT request‑to‑code dataset benchmark public
Source sentence
The resulting SFT dataset contains approximately 800 validated request‑to‑code pairs.
Costs
Unclear - sources disagree
Source sentence (the authors' words)
Neither QuantCode‑Bench nor the SFT validation pipeline measures profitability, robustness across market regimes, realistic transaction costs, market impact, or production risk.
Finds 5
- Continued pretraining improves single‑turn Judge Pass from 41.5% to 47.5% for Qwen3.5‑397B‑A17B and from 27.8% to 33.0% for Qwen3.6‑35B‑A3B. performance
Source sentence
Continued pretraining improves single‑turn Judge Pass from 41.5% to 47.5% for Qwen3.5‑397B‑A17B and from 27.8% to 33.0% for Qwen3.6‑35B‑A3B.
- Supervised fine‑tuning after continued pretraining raises Judge Pass to 58.2% and achieves 83.5% successful backtests for Qwen3.6‑35B‑A3B. performance
Source sentence
SFT applied after continued pretraining yields a larger gain for Qwen3.6‑35B‑A3B, reaching 58.2% Judge Pass and 83.5% successful backtests;
- In agentic evaluation, first‑turn success increases from 22.3% to 58.3% and final success after up to 10 turns rises from 47.5% to 79.5% for the SFT checkpoint. performance
Source sentence
in agentic evaluation it raises first‑turn success from 22.3% to 58.3% and final success after up to 10 turns from 47.5% to 79.5%.
- Continued pretraining alone lowers final success after repair from 47.5% to 32.5%, indicating degraded instruction following. negative
Source sentence
Continued pretraining alone improves first‑turn agentic success but lowers final success after repair from 47.5% to 32.5%, consistent with degraded instruction following,
- Domain specialization degrades parser‑conformant structured tool calling, and recovery SFT restores tool‑call formatting but not overall repository‑level performance. negative
Source sentence
Domain specialization degraded parser‑conformant structured tool calling in the target agent harness. Dedicated recovery SFT restored operational tool‑call formatting in qualitative deployment checks, but repository‑level task resolution remained below the Qwen3.6‑35B‑A3B base checkpoint.
Checked by others
No other paper in the archive has been found to support or contradict these findings (compared with 21 related papers). Coverage is partial, so this is neither replication nor refutation.
Code
No repository by the authors found in the paper.
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.