LemonadeBench: Evaluating the Economic Intuition of Large Language Models in Simple Markets

Paper: arXiv 2602.13209

Authors: Aidan Vyas

Abstract

We introduce LemonadeBench v0.5, a minimal benchmark for evaluating economic intuition, long-term planning, and decision-making under uncertainty in large language models (LLMs) through a simulated lemonade stand business. Models must manage inventory with expiring goods, set prices, choose operating hours, and maximize profit over a 30-day period-tasks that any small business owner faces daily. All models demonstrate meaningful economic agency by achieving profitability, with performance scaling dramatically by sophistication-from basic models earning minimal profits to frontier models capturing 70% of theoretical optimal, a greater than 10x improvement. Yet our decomposition of business efficiency across six dimensions reveals a consistent pattern: models achieve local rather than global optimization, excelling in select areas while exhibiting surprising blind spots elsewhere.

Complexity vs Empirical Score

• Math Complexity: 2.5/10
• Empirical Rigor: 6.5/10
• Quadrant: Street Traders — practical and empirical, lighter on theory

Why this score: The paper introduces a highly novel and practical benchmark (LemonadeBench) for evaluating LLM economic intuition and multi-objective decision-making. While the mathematical modeling is deliberately minimal (basic calculus for static optimal pricing), the empirical evaluation across models is structured and easily reproducible thanks to provided repository links and explicit system prompts.

Research Flowchart

  flowchart TD
    A["Research Goal: Evaluate LLMs' Economic Intuition & Decision-Making"] --> B["LemonadeBench v0.5 Simulation"]
    
    subgraph Inputs ["Data & Task Inputs"]
        C["30-Day Business Dynamics: Inventory Expiration, Pricing, Hours, Weather/Demand Uncertainty"]
    end
    
    Inputs --> B
    
    B --> D["LLM Execution & Performance Scaling"]
    
    D --> E["Multi-Dimensional Efficiency Decomposition"]
    
    E --> F["Key Findings: Frontier LLMs reach 70% optimal profit (>10x vs basic), but show local over global optimization & blind spots"]