$ε$-Policy Gradient for Online Pricing

ArXiv ID: 2405.03624 “View on arXiv”

Authors: Unknown

Abstract

Combining model-based and model-free reinforcement learning approaches, this paper proposes and analyzes an $ε$-policy gradient algorithm for the online pricing learning task. The algorithm extends $ε$-greedy algorithm by replacing greedy exploitation with gradient descent step and facilitates learning via model inference. We optimize the regret of the proposed algorithm by quantifying the exploration cost in terms of the exploration probability $ε$ and the exploitation cost in terms of the gradient descent optimization and gradient estimation errors. The algorithm achieves an expected regret of order $\mathcal{“O”}(\sqrt{“T”})$ (up to a logarithmic factor) over $T$ trials.

Keywords: online learning, pricing, reinforcement learning, ε-greedy, regret minimization

Complexity vs Empirical Score

• Math Complexity: 8.0/10
• Empirical Rigor: 3.0/10
• Quadrant: Lab Rats
• Why: The paper presents advanced mathematical analysis with regret bounds of order O(√T), utilizing parametric models, gradient estimation, and complex error quantification, indicating high mathematical complexity. However, it is purely theoretical with no empirical validation, backtests, or implementation details provided, resulting in low empirical rigor.

  flowchart TD
    A["Research Goal: Minimize Regret in Online Pricing"] --> B["Key Methodology: ε-Policy Gradient Algorithm"]
    
    B --> C["Data/Input: Historical Price-Conversion Data & Model Inference"]
    C --> D{"Computational Process"}
    
    D --> E["Exploration: ε-greedy selection<br>sample market response"]
    D --> F["Exploitation: Gradient descent step<br>optimize price parameter θ"]
    
    E & F --> G["Update Model: Infer demand model<br>estimate gradient of expected reward"]
    
    G --> H["Key Finding: Regret bound of<br>O(√T) up to logarithmic factor"]
    H --> I["Outcome: Efficient learning balance<br>via exploration-exploitation tradeoff"]