{"paper_id":"2306.06087","title":"Learning Not to Spoof","authors":["David Byrd"],"paper_date":"2023-06-09","paper_type":"empirical","paper_url":"https://arxiv.org/pdf/2306.06087v1","page_url":"https://thequant.space/flowcharts/learning-not-to-spoof/","scores":{"empirical_rigor":7,"math_complexity":3.5,"quadrant":"Street Traders"},"proposes":{"status":"available","statement":"The paper introduces a method that uses a learned spoofing recognizer to shape the RL agent's rewards and rerank actions, thereby guiding the agent to remain profitable while avoiding spoofing behavior.","quote":"Finally, I introduce a method to incorporate the recognizer as normative guide, shaping the agent's perceived rewards and altering its selected actions.","summary":"As intelligent trading agents based on reinforcement learning (RL) gain prevalence, it becomes more important to ensure that RL agents obey laws, regulations, and human behavioral expectations. There is substantial literature concerning the aversion of obvious catastrophes like crashing a…","strategy":null},"assumptions":{"status":"available","items":[{"assumption":"A responsible practitioner creates an RL-based stock trading agent with an input space of technical market features and sufficient internal state to make the problem Markovian, and an output space that permits placing or","type":"model","stated":"explicit","quote":"[22] 1.6 Inadvertent Spoofing Suppose that a responsible practitioner of financial ML undertakes to create an RL-based stock trading agent with an input space consisting of technical market features and sufficient internal state to render the problem Markovian.","support":"full"},{"assumption":"The simulated market environment (with ZI, Value, and OBI agents) is sufficient to make spoofing possible and profitable, allowing synthetic data generation for detector training.","type":"market","stated":"implied","quote":"3.1 Initial Spoofing Strategy To achieve the ultimate aim of learning not to spoof, I must first ensure that spoofing our simulated market is both possible and profitable. I instantiate a population of: 500 Zero Intelligence (ZI) agents, which submit bids and offers drawn from a stochastic distri-bution around a private extrinsic valuation; 500 Value agents which arbitrage the current market price against a private","support":"full"}]},"datasets":{"status":"available","items":[{"name":"Synthetic spoofing action sequence dataset","provider":null,"kind":"synthetic","access":"unknown","period":null,"universe":null,"quote":"3.2 Spoofing Detector The data synthesis described in Section 2.1 results in an imbalanced labeled data set of size (2611707, 20, 4) with 84,666 (approx. 3.2%) examples of spoofing action sequences.","support":"full"}],"known":[],"sample":{"start":null,"end":null,"market":null,"asset_class":null,"frequency":null},"data_resolution":"synthetic"},"transaction_costs":{"status":"available","modeled":"no","headline_net_of_costs":null,"detail":null,"quote":null,"source":"front_matter","rule":"R1","candidates":{"front_matter":"no","evidence_card":"no","evidence_card_quoted":false},"meaning":"a trading or execution result is reported without costs"},"results":{"status":"available","evaluation_type":"unclear","findings":[{"finding":"Both the honest policy (π_h) and the spoofing policy (π_s) achieve positive profits of the same order of magnitude as the original strategies.","kind":"performance","quote":"Policy 𝜋ℎ(𝑠) outperforms the original honest strategy, while 𝜋𝑠(𝑠) underper-forms the original spoofing strategy, but returns for both policies are positive and of the same order of magnitude as the original strategies.","support":"full"},{"finding":"The spoofing detector activation is 0.000 for the honest policy and 0.842 for the spoofing policy, correctly flagging spoofing behavior.","kind":"performance","quote":"I record the average activation of the spoofing detector for both policies and find that Θ(𝜋ℎ(𝑠),𝑎0 : 𝑎19) = 0.000 and Θ(𝜋𝑠(𝑠),𝑎0 : 𝑎19) = 0.842.","support":"full"},{"finding":"For the restricted Q‑trader (scaled‑rBoltzmann variation), the average detector activation is less than 1e‑17, indicating it does not spoof.","kind":"performance","quote":"The average activation of spoofing detector Θ(𝑎0 : 𝑎19) < 1𝑒−17 for all variations of𝑄𝑅, supporting the hypothesis that the restricted agent lacks the capacity to spoof the simulated market.","support":"full"},{"finding":"In the restricted Q‑trader, only 5% of market days end in a loss, with an average loss of $195 on those days.","kind":"performance","quote":"Without spoofing, 𝑄𝑅consistently learns to profit: in the scaled-𝑟Boltzmann variation, only 5% of market days end in a loss, and the average loss on those days is only $195.","support":"full"},{"finding":"The study concludes that a properly configured RL trader can learn to profit without engaging in spoofing.","kind":"theoretical","quote":"I conclude that a properly configured RL trader can learn to profit in the market without spoofing.","support":"full"}],"reported_performance":null,"out_of_sample":{"method":"unclear","headline_out_of_sample":null}},"independent_evidence":{"status":"none_found","supports":[],"contradicts":[],"qualifies":[],"compared_with":15,"citations":{"cited_by_in_corpus":0,"by_independent_authors":0,"how_cited":{"used_as_baseline":0,"uses_method_or_data":0,"extends":0,"discusses_findings":0,"mentions":0,"unclassified":0},"independent_papers":[],"scope":"Citations between papers in this corpus only; citations from outside it are not counted."}},"implementation":{"status":"available","public":false,"code_url":null,"note":"No repository presented by the authors as their own was found in the paper. Absence here is not proof that no code exists."},"replication":{"status":"none_found","card":null,"note":"No replication of this paper has been run by thequant.space yet."},"extraction":{"text_source":"fulltext","model":"openai/gpt-oss-120b","version":2,"extracted_at":"2026-11-14T20:02:30Z"},"provenance":{"claims":"Everything here is what the authors state in the paper. Nothing was re-run or reproduced by the platform.","quotes":"Assumptions, datasets, cost detail and findings were extracted by a language model. Every quote is verbatim paper text. Where an item has `support`, \"full\" means every figure in it appears in the quoted sentences and \"partial\" means a figure is stated elsewhere in the paper; items with a figure found nowhere in the paper were discarded.","datasets_known":"datasets.known lists well-known vendors and public datasets by a stable id, found by a dictionary over the full text where the sentence says the data is used; it is independent of the model and covers only names in the dictionary.","flags":"transaction_costs.modeled, results.evaluation_type and results.out_of_sample.method are platform classifications (structured-decision models); see https://thequant.space/score-guide/#penalty-flags","comparisons":"supports / contradicts / qualifies are a language model's reading of two papers' quoted findings, restricted to closely related papers that share no author. They are pointers to read, not verified replications, and coverage is partial.","implementation":"code_url is a repository the paper itself presents as its own (rule-scored, ambiguous cases judged by a model), checked for liveness when it was recorded."},"docs":"https://thequant.space/agents/"}