Paper: arXiv 2610.26324

Authors: Saad Naja, Nasser Odtallah, Jake Van-Praag, Sarah May

Abstract

Trading discipline often fails people. Our study tested whether four AI trading agents followed their strategies and risk rules. Offering traders the evidence and an agent evaluation toolkit to separate luck from following the rules, before giving agents meaningful autonomy or capital. Our study analysed 2,627 closed positions from 12 February to 27 August 2026 across the same asset universe and execution conditions. Most positions were simulated against live prices with a 0.09% round-trip fee; 18% involved real Hyperliquid orders. Each position declared a strategy, stop and target, with a limit of five open positions per agent. The fleet finished net positive after fees. Rule adherence was high across all four agents, including the unprofitable one. Profitability tracked how often agents won relative to their break-even win rates. Institutional-style controls ran by default and contained losses through a 29% market drawdown and recovery, with documented exceptions. The discipline held for all four agents. Agents kept 97.5% of gross loss within declared stops, and typical losses closed at declared levels. Declared risk remained stable after losses and broadly stable across calm and volatile months. Twenty-three of 24 runs respected the position limit; logs identify the exception’s size and duration. The same live data produced different results across assigned strategies. The trade-log tests can be rerun from the published trade snapshot; regime tests also require the published Bitcoin prices and regime labels. Section 3.5 specifies the stop-breach tolerance. These trade-level results are not account-level returns or proof of a lasting edge. Our study provides an evaluation toolkit to help traders distinguish rule-following from luck before committing capital. For anyone considering allocating capital to agents to trade, discipline should be the entry test, not the only test.

Complexity vs Empirical Score

  • Math Complexity: 3.0/10
  • Empirical Rigor: 6.0/10 (capped from 7.0: transaction costs only partially modelled; see penalty flags)
  • Quadrant: Street Traders — practical and empirical, lighter on theory

Why this score: The paper focuses on empirical validation of AI agent discipline rather than complex mathematical models. Its strength lies in the rigorous testing framework and the practical toolkit provided for evaluating trading agents, offering a novel perspective on agent assessment.

Research Flowchart

  flowchart TD
    A[Research Goal: Assess AI Agent Rule-Following & Risk Adherence] --> B{Methodology: Analyze 2,627 Closed Trades};
    B --> C[Data: Agent Trades (Simulated & Real), Asset Universe, Live Prices, Bitcoin Prices, Regime Labels];
    C --> D{Computational Process: Trade Log Analysis, Win Rate Calculation, Stop/Target Adherence Checks, Position Limit Checks};
    D --> E[Key Finding 1: High Rule Adherence Across All Agents (97.5% gross loss within stops)];
    D --> F[Key Finding 2: Profitability Tracks Win Rate vs. Break-Even Win Rate];
    D --> G[Key Finding 3: Institutional-Style Controls Contained Losses During Market Drawdown];
    E & F & G --> H[Outcome: Evaluation Toolkit to Distinguish Discipline from Luck for Agent Capital Allocation];