Paper: arXiv 2508.20097

Authors: Andrew Blair-Stanek, Nils Holzenberger, Benjamin Van Durme

Abstract

We investigate whether large language models can discover and analyze U.S. tax-minimization strategies. This real-world domain challenges even seasoned human experts, and progress can reduce tax revenue lost from well-advised, wealthy taxpayers. We evaluate the most advanced LLMs on their ability to (1) interpret and verify tax strategies, (2) fill in gaps in partially specified strategies, and (3) generate complete, end-to-end strategies from scratch. This domain should be of particular interest to the LLM reasoning community: unlike synthetic challenge problems or scientific reasoning tasks, U.S. tax law involves navigating hundreds of thousands of pages of statutes, case law, and administrative guidance, all updated regularly. Notably, LLM-based reasoning identified an entirely novel tax strategy, highlighting these models’ potential to revolutionize tax agencies’ fight against tax abuse.

Complexity vs Empirical Score

  • Math Complexity: 2.0/10
  • Empirical Rigor: 7.5/10
  • Quadrant: Street Traders — empirical and applied, lower math complexity

Why this score: The paper’s core math is low-complexity linguistic prompting and legal analysis, but it demonstrates high empirical rigor through a handcrafted dataset (Shelter Check), GitHub code, and expert-graded strategy generation from real-world LLMs.

Research Flowchart

  flowchart TD
  A["Research Goal<br>Can LLMs Identify Tax Abuse?"] --> B["Methodology: Three Evaluation Tasks<br>(1) Interpret & Verify<br>(2) Fill Gaps<br>(3) Generate Strategies"]
  B --> C["Computational Process<br>LLM Reasoning on Tax Law<br>Simulated Expert Interactions"]
  C --> D["Key Findings & Outcomes<br>Novel Strategy Discovery<br>Verification Capability<br>Potential for Tax Agency Use"]

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper introduces the Shelter Check dataset of 36 U.S. tax‑minimization strategies and benchmarks several leading LLMs on interpreting, verifying, and generating such strategies.

Source sentence
We introduce a new domain-expert crafted dataset called Shelter Check, derived from public sources that describe schemes used in the past by taxpayers to minimize taxes.

Assumes 1

  • The tax‑minimization strategies in the study depend on textual tax‑law authorities rather than on asset‑valuation uncertainty. data
    Source sentence
    3 The Shelter Check Dataset The Shelter Check dataset currently contains 36 known U.S. tax-minimization strategies that depend upon applying tex-tual authorities. Why only 36?

Data

  • Shelter Check · public sources (IRS listed transactions, court decisions) text-news public
    Source sentence
    We introduce a new domain-expert crafted dataset called Shelter Check, derived from public sources that describe schemes used in the past by taxpayers to minimize taxes. The dataset includes the text of the tax-law authorities, like tax code sections or court decisions, that enabled the scheme.

Costs

Not applicable The paper reports no trading or execution result.

Finds 3

  • LLM‑based reasoning identified an entirely novel tax strategy, demonstrating potential for tax agencies. other
    Source sentence
    Notably, LLM-based reasoning identified an entirely novel tax strategy, highlighting these models' potential to revolutionize tax agencies' fight against tax abuse.
  • All three reasoning models (o3, claude‑4, gemini‑2.5) outperformed the baseline llama‑3.3‑70B on the step‑cloze and generation tasks. performance figure from elsewhere in the paper
    Source sentence
    The three reasoning models outperformed the baseline llama-3.3-70B. so that replacing a specified correct step with the adversarial step would keep the strategy from working (i.e. achieving all listed goals).
  • When adversarial steps were added, the reasoning models initially showed agreement bias, but after iterative refinement they returned 0/36 correct strategies, indicating resistance to finding non‑viable strategies. negative
    Source sentence
    But the LLMs proved remarkably resistant to finding that a strategy does not work. We improved the adversarial steps and expressly eliminated unwarranted factual assumptions to minimize the number of strategies the models found cor-rect, until all three reasoning models returned 0/36 as being correct.

Checked by others

No other paper in the archive has been found to support or contradict these findings (compared with 8 related papers). Coverage is partial, so this is neither replication nor refutation.

Code

https://github.com/BlairStanek/LLM-Tax-Abuse — the authors' own repository

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.

Cited References (64)

  1. Rethinking the Development of Large Language Models from the Causal Perspective: A Legal Text Prediction Case Study (2024) — link
  2. RR-PU: A Synergistic Two-Stage Positive and Unlabeled Learning Framework for Robust Tax Evasion Detection (2024) — link
  3. GxVAEs: two joint VAEs generate hit molecules from gene expression profiles (2024) — link
  4. Black-Box Optimization with Implicit Constraints for Public Policy (2025) — link
  5. J & H: Evaluating the Robustness of Large Language Models Under Knowledge-Injection Attacks in Legal Domain (2025) — link
  6. Blackboard Systems (1986)
  7. Communication, Simulation, and Intelligent Agents: Implications of Personal Intelligent Machines for Medical Education (1983)
  8. Classification Problem Solving (1984)
and 56 more
  1. New Ways to Make Microcircuits Smaller (1980) — link
  2. New Ways to Make Microcircuits Smaller—Duplicate Entry (1980)
  3. Strategic explanations for a diagnostic consultation system (1984) — link
  4. Strategic Explanations in Consultation—Duplicate (1983)
  5. Poligon: A System for Parallel Problem Solving (1986)
  6. Transfer of Rule-Based Expertise through a Tutorial Dialogue (1979)
  7. The Engineering of Qualitative Models (2021)
  8. Attention Is All You Need (2017) — link
  9. Pluto: The ‘Other’ Red Planet (2015)
  10. Deep Blue (2002) — link
  11. A Survey on Large Language Models for Automated Planning (2025) — link
  12. Overview and Discussion of the Competition on Legal Information, Extraction/Entailment (COLIEE) 2023 (2024) — link
  13. Swiss-Judgment-Prediction: A Multilingual Legal Judgment Prediction Benchmark (2021) — link
  14. JEC-QA: A Legal-Domain Question Answering Dataset (2020) — link
  15. ILDC for CJPE: Indian Legal Documents Corpus for Court Judgment Prediction and Explanation (2021) — link
  16. LexGLUE: A Benchmark Dataset for Legal Language Understanding in English (2022) — link
  17. Computational Discovery of Transaction-Based Financial Crime via Grammatical Evolution: The Case of Ponzi Schemes (2022) — link
  18. Can AI expose tax loopholes? Towards a new generation of legal policy assistants (2025) — link
  19. Attention is All you Need (2017) — link
  20. Accurate structure prediction of biomolecular interactions with AlphaFold 3 (2024)
  21. Machine science (2010)
  22. GPT Gets Its First Law School B-pluses (2024)
  23. Can GPT-3 Perform Statutory Reasoning? (2023) — link
  24. A Dataset for Statutory Reasoning in Tax Law Entailment and Question Answering (2020) — link
  25. LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models (2023) — link
  26. Detecting tax evasion: a co-evolutionary approach (2016) — link
  27. Using LLMs to Discover Legal Factors (2024) — link
  28. Predicting Outcomes of Case-Based Legal Arguments (2003) — link
  29. The Design of a Legal Analysis Program (1983) — link
  30. Catala: a programming language for the law (2021) — link
  31. The British Nationality Act as a logic program (1986) — link
  32. Reflections on ‘‘Taxman": An Experiment in Artificial Intelligence and Legal Reasoning (1977) — link
  33. Crafting Papers on Machine Learning (2000)
  34. The Need for Biases in Learning Generalizations (1980)
  35. Computational Complexity of Machine Learning (1989)
  36. Machine Learning: An Artificial Intelligence Approach, Vol. I (1983)
  37. Pattern Classification (2000)
  38. Suppressed for Anonymity (2021)
  39. Mechanisms of Skill Acquisition and the Law of Practice (1981)
  40. Some Studies in Machine Learning Using the Game of Checkers (1959)
  41. Wiz Research Uncovers Exposed DeepSeek Database Leaking Sensitive Information, Including Chat History (2025) — link
  42. Tax non-compliance detection using co-evolution of tax evasion risk and audit likelihood (2015)
  43. The general theory of tax avoidance (1985)
  44. Federal Taxation of Income, Estates and Gifts \textparagraph 4.3 (2025)
  45. Esmark, I nc. v. C ommissioner (1988)
  46. Notice 2003-54, COMMON TRUST FUND STRADDLE TAX SHELTER (2003)
  47. Common Trust Funds: The Living Fossil of Passthroughs (2010)
  48. Recognized abusive and listed transactions (2025) — link
  49. Large language models as tax attorneys: a case study in legal capabilities emergence (2024)
  50. Tax Gap Projections Methodology: Tax Year 2022 (2024)
  51. Base erosion and profit shifting (2025) — link
  52. Rulearena: A benchmark for rule-guided reasoning with llms in real-world scenarios (2024) — link
  53. The Ideation-Execution Gap: Execution Outcomes of LLM-Generated versus Human Research Ideas (2025) — link
  54. Shelter Check: Proactively Finding Tax Minimization Strategies via AI (2025) — link
  55. Intellectual Property Law Solutions to Tax Avoidance (2015)
  56. AI, Taxation, and Valuation (2023) — link

0 of 64 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.