{"paper_id":"2508.02702","title":"Evaluating Transfer Learning Methods on Real-World Data Streams: A Case Study in Financial Fraud Detection","authors":["Ricardo Ribeiro Pereira","Jacopo Bono","Hugo Ferreira","Pedro Ribeiro","Carlos Soares","Pedro Bizarro"],"paper_date":"2025-07-29","paper_type":"empirical","paper_url":"https://arxiv.org/pdf/2508.02702v1","page_url":"https://thequant.space/flowcharts/evaluating-transfer-learning-methods-on-real-world-data-stre/","scores":{"empirical_rigor":7,"math_complexity":3,"quadrant":"Street Traders"},"proposes":{"status":"available","statement":"The paper proposes an evaluation framework that simulates dynamic data availability, creates multiple domains via resampling, and applies realistic transformations for transfer learning assessment.","quote":"To address this challenge, we propose an evaluation framework that simulates the dynamic nature of real-world data streams. Our framework provides three key capabilities: (1) creating multiple domains from a given dataset through re‑sampling, enabling systematic TL evaluation even when few datasets are available; (2) a","summary":"When the available data for a target domain is limited, transfer learning (TL) methods can be used to develop models on related data-rich domains, before deploying them on the target domain. However, these TL methods are typically designed with specific, static assumptions on the amount of…","strategy":null},"assumptions":{"status":"available","items":[{"assumption":"Labels become available after a delay (ty_i ≥ tx_i).","type":"data","stated":"explicit","quote":"... tx i is the timestamp when xi is collected, and ty i ≥tx i is the timestamp when yi becomes available."},{"assumption":"A large volume of labeled data is available from the source domains.","type":"data","stated":"explicit","quote":"that a large volume of labeled data is available from the source domains."},{"assumption":"The target domain initially has no data and then data/labels arrive gradually over time.","type":"data","stated":"explicit","quote":"at some time ta, the target domain DT is introduced, initially without any data (DL T (ta) = DU T (ta) = ∅), and target domain data and labels begin to be collected from that point on."},{"assumption":"Multiple domains can be generated from a single dataset via resampling.","type":"model","stated":"explicit","quote":"creating multiple domains from a given dataset through re‑sampling, enabling systematic TL evaluation even when few datasets are available"},{"assumption":"Realistic domain shifts can be simulated by applying controlled transformations.","type":"model","stated":"explicit","quote":"applying transformations to the data, hence reproducing realistic data shifts over time and across domains"}]},"datasets":{"status":"available","items":[{"name":"proprietary card payment dataset","provider":null,"kind":"alternative","access":"proprietary","period":null,"universe":null,"quote":"We perform a case study using our framework on a suite of proprietary real‑world datasets containing payment events from multiple financial institutions."},{"name":"Bank Account Fraud (BAF) dataset","provider":null,"kind":"alternative","access":"public","period":null,"universe":null,"quote":"Given the confidential nature of the case study, we also illustrate the use of the framework on the publicly available Bank Account Fraud (BAF) dataset."}],"known":[],"sample":{"start":null,"end":null,"market":null,"asset_class":null,"frequency":null},"data_resolution":"unclear"},"transaction_costs":{"status":"not_applicable","modeled":"n/a","headline_net_of_costs":null,"detail":null,"quote":null,"source":"front_matter","rule":"R3","candidates":{"front_matter":"n/a","evidence_card":"n/a","evidence_card_quoted":false},"meaning":"the paper reports no trading or execution result"},"results":{"status":"available","evaluation_type":"forecast-accuracy","findings":[{"finding":"MTAE, DANN, BL‑A and BL‑S show similar recall levels and maintain stable performance over time, indicating limited benefit from additional target data.","kind":"performance","quote":"– MTAE, DANN, BL-A and BL-S show similar levels of recall, maintaining a stable distribution of predictive performance over time. This suggests that there is a limited benefit from the additional target domain data and labels."},{"finding":"MAN and BL‑T start worse but improve steadily, gaining about 2 percentage points of recall per model update and eventually match the performance of the other methods.","kind":"performance","quote":"– MAN and BL-T begin to perform significantly worse than the previous group of methods, but they improve steadily over time (gaining on average 2 percentage points of recall per model update) and eventually reaching the same level of performance."},{"finding":"MME maintains relatively stable performance but is consistently outperformed by the other methods.","kind":"performance","quote":"– MME maintains a relatively stable performance throughout, but is consistently surpassed by the other methods."},{"finding":"Methods that do not use any target data (MTAE, BL‑S) perform similarly to BL‑T, suggesting source and target domains are relatively similar and knowledge sharing is feasible.","kind":"predictive-accuracy","quote":"the performance of methods such as MTAE and BL‑S, which do not use any target data during training, is similar to the performance of BL‑T, which follows the traditional ML approach of only using in‑domain data to train. This suggests that the source and target domains in the BAF dataset are relatively similar, which me"}],"reported_performance":null,"out_of_sample":{"method":"walk-forward","headline_out_of_sample":null}},"independent_evidence":{"status":"none_found","supports":[],"contradicts":[],"qualifies":[],"compared_with":33,"citations":{"cited_by_in_corpus":0,"by_independent_authors":0,"how_cited":{"used_as_baseline":0,"uses_method_or_data":0,"extends":0,"discusses_findings":0,"mentions":0,"unclassified":0},"independent_papers":[],"scope":"Citations between papers in this corpus only; citations from outside it are not counted."}},"implementation":{"status":"available","public":true,"code_url":"https://github.com/feedzai/tred","note":null},"replication":{"status":"none_found","card":null,"note":"No replication of this paper has been run by thequant.space yet."},"extraction":{"text_source":"fulltext","model":"openai/gpt-oss-120b","version":1,"extracted_at":"2026-11-14T17:31:02Z"},"provenance":{"claims":"Everything here is what the authors state in the paper. Nothing was re-run or reproduced by the platform.","quotes":"Assumptions, datasets, cost detail and findings were extracted by a language model. Every quote is verbatim paper text. Where an item has `support`, \"full\" means every figure in it appears in the quoted sentences and \"partial\" means a figure is stated elsewhere in the paper; items with a figure found nowhere in the paper were discarded.","datasets_known":"datasets.known lists well-known vendors and public datasets by a stable id, found by a dictionary over the full text where the sentence says the data is used; it is independent of the model and covers only names in the dictionary.","flags":"transaction_costs.modeled, results.evaluation_type and results.out_of_sample.method are platform classifications (structured-decision models); see https://thequant.space/score-guide/#penalty-flags","comparisons":"supports / contradicts / qualifies are a language model's reading of two papers' quoted findings, restricted to closely related papers that share no author. They are pointers to read, not verified replications, and coverage is partial.","implementation":"code_url is a repository the paper itself presents as its own (rule-scored, ambiguous cases judged by a model), checked for liveness when it was recorded."},"docs":"https://thequant.space/agents/"}