{"paper_id":"2308.11406","title":"Designing an attack-defense game: how to increase robustness of financial transaction models via a competition","authors":["Alexey Zaytsev","Maria Kovaleva","Alex Natekin","Evgeni Vorsin","Valerii Smirnov","Georgii Smirnov","Oleg Sidorshin","Alexander Senin","Alexander Dudin","Dmitry Berestnev"],"paper_date":"2023-08-22","paper_type":"empirical","paper_url":"https://arxiv.org/pdf/2308.11406v3","page_url":"https://thequant.space/flowcharts/designing-an-attack-defense-game-how-to-increase-robustness/","scores":{"empirical_rigor":8.5,"math_complexity":2.5,"quadrant":"Street Traders"},"proposes":{"status":"available","statement":"The paper proposes a novel competition framework to evaluate robustness of financial transaction models and releases a new open dataset of bank transactions with credit default labels.","quote":"a novel type of competition that allows a realistic and detailed investigation of problems in financial transaction data.","summary":"Banks routinely use neural networks to make decisions. While these models offer higher accuracy, they are susceptible to adversarial attacks, a risk often overlooked in the context of event sequences, particularly sequences of financial transactions, as most works consider computer vision and NLP…","strategy":null},"assumptions":{"status":"available","items":[{"assumption":"The competition uses a static model environment in the pre‑tournament phase.","type":"model","stated":"implied","quote":"The pre‑tournament phase allows for detailed study of a static model environment"},{"assumption":"Each customer sequence contains at least 300 transactions.","type":"data","stated":"implied","quote":"Each customer within the dataset has at least 300 associated transactions in a sequence."},{"assumption":"Transaction amounts are anonymized with normalization and small noise.","type":"data","stated":"implied","quote":"all transaction amounts within the dataset have been anonymized with normalization and small noise."},{"assumption":"The credit default positive label rate is 0.04.","type":"data","stated":"implied","quote":"the share of the positive label rate of defaults in data is 0.04."}]},"datasets":{"status":"available","items":[{"name":"Default","provider":null,"kind":"other","access":"public","period":null,"universe":null,"quote":"For this competition, we present a new open dataset of bank transactions named Default, which is described below."},{"name":"Churn","provider":null,"kind":"other","access":"public","period":null,"universe":null,"quote":"Also, we applied a previously existing dataset named Churn for additional testing of the best solutions."}],"known":[],"sample":{"start":null,"end":null,"market":null,"asset_class":null,"frequency":null},"data_resolution":"unclear"},"transaction_costs":{"status":"not_applicable","modeled":"n/a","headline_net_of_costs":null,"detail":null,"quote":null,"source":"front_matter","rule":"R1","candidates":{"front_matter":"n/a","evidence_card":"unclear","evidence_card_quoted":false},"meaning":"the paper reports no trading or execution result"},"results":{"status":"available","evaluation_type":"forecast-accuracy","findings":[{"finding":"The highest defense score achieved was 0.72, exceeding the quality of the initial model, but defenses significantly degrade against stronger attacks.","kind":"performance","quote":"while weaker attacks lead to minor effects, with the highest score of 0.72 being higher than the quality of the initial model, the defenses significantly degrade when put against stronger attacks."},{"finding":"During the black‑box tournament stage, ROC AUC decreased on average by about 0.05 across attack‑defense pairs, with some attacks and defenses showing almost zero decrease.","kind":"performance","quote":"we still see defenses and attacks with almost zero decrease in ROC AUC and about a 0.05 decrease of ROC AUC on average"},{"finding":"Models using only features not susceptible to attack (Boosting clean) achieve competitive ROC AUC on clean data, and gradient‑boosted models distilled from neural networks improve performance.","kind":"predictive-accuracy","quote":"For the sake of comparison, we also present scores for Boosting clean model that uses only features not susceptible to the attack"}],"reported_performance":null,"out_of_sample":{"method":"holdout","headline_out_of_sample":null}},"independent_evidence":{"status":"none_found","supports":[],"contradicts":[],"qualifies":[],"compared_with":11,"citations":{"cited_by_in_corpus":0,"by_independent_authors":0,"how_cited":{"used_as_baseline":0,"uses_method_or_data":0,"extends":0,"discusses_findings":0,"mentions":0,"unclassified":0},"independent_papers":[],"scope":"Citations between papers in this corpus only; citations from outside it are not counted."}},"implementation":{"status":"available","public":false,"code_url":null,"note":"No repository presented by the authors as their own was found in the paper. Absence here is not proof that no code exists."},"replication":{"status":"none_found","card":null,"note":"No replication of this paper has been run by thequant.space yet."},"extraction":{"text_source":"fulltext","model":"openai/gpt-oss-120b","version":1,"extracted_at":"2026-11-14T18:23:07Z"},"provenance":{"claims":"Everything here is what the authors state in the paper. Nothing was re-run or reproduced by the platform.","quotes":"Assumptions, datasets, cost detail and findings were extracted by a language model. Every quote is verbatim paper text. Where an item has `support`, \"full\" means every figure in it appears in the quoted sentences and \"partial\" means a figure is stated elsewhere in the paper; items with a figure found nowhere in the paper were discarded.","datasets_known":"datasets.known lists well-known vendors and public datasets by a stable id, found by a dictionary over the full text where the sentence says the data is used; it is independent of the model and covers only names in the dictionary.","flags":"transaction_costs.modeled, results.evaluation_type and results.out_of_sample.method are platform classifications (structured-decision models); see https://thequant.space/score-guide/#penalty-flags","comparisons":"supports / contradicts / qualifies are a language model's reading of two papers' quoted findings, restricted to closely related papers that share no author. They are pointers to read, not verified replications, and coverage is partial.","implementation":"code_url is a repository the paper itself presents as its own (rule-scored, ambiguous cases judged by a model), checked for liveness when it was recorded."},"docs":"https://thequant.space/agents/"}