Media Trading with Censored Auction Prices: A Stochastic Control and Reinforcement Learning Approach

Paper: arXiv 2611.16992

Authors: Yun-Ithry Gamrani, Huyên Pham

Abstract

In real-time bidding, marketing agencies must balance campaign performance, transaction margins and the use of a finite advertising budget. A key difficulty is that market-clearing prices are typically observed only when bids are successful, so the information available for subsequent decisions depends on the bidding policy. We formulate this problem using a continuous-time stochastic control model with a latent price process. To construct a price signal, we consider parallel exploratory subcampaigns in which bids are randomized subject to an upper bound. Under a stylized cross-sectional sampling model, the resulting price estimator is consistent and satisfies a mean-square error bound of order $N^{-1}$, where $N$ is the number of exploratory observations. We then describe a model-based approach combining forward-equation calibration with Hamilton-Jacobi-Bellman policy iteration and a physics-informed neural network, alongside an entropy-regularized actor-critic approach based on observable information in a model-free setting. Synthetic experiments illustrate the estimator and the computational procedures. The framework clarifies the decision and information trade-offs faced by the agency, while distinguishing full-information benchmarks and estimated-price policies from the genuinely partially observed control problem.

Complexity vs Empirical Score

• Math Complexity: 8.5/10
• Empirical Rigor: 4.0/10
• Quadrant: Lab Rats — theoretically deep, empirically untested

Why this score: The paper presents a highly sophisticated mathematical framework involving stochastic control, HJB equations, and PINNs for a challenging problem. While the theoretical development is strong and novel, the empirical validation is limited to synthetic experiments, lacking real-world data application or extensive backtesting. The methodology is well-articulated, but the absence of code or data makes direct reproducibility challenging.

Research Flowchart

  flowchart TD
    A[Research Goal: Optimize Real-time Bidding under Latent Prices] --> B{Formulate Continuous-time Stochastic Control Model};
    B --> C{Develop Latent Price Estimator};
    C --> D[Data/Inputs: Observed Wins/Losses, Bids, Ad Budget];
    D --> E{Computational Processes};
    E -- Model-Based --> F1[Forward-Equation Calibration + HJB Policy Iteration + PINN];
    E -- Model-Free --> F2[Entropy-Regularized Actor-Critic];
    F1 & F2 --> G[Key Findings: Clarified Decision/Information Trade-offs, Benchmarks vs. Partially Observed Control];

Cited References (65)

1. Optimal Real-Time Bidding Strategies (2017)
2. Consistency of Generalized Finite Difference Schemes for the Stochastic HJB Equation (2003) — link
3. DGM: A deep learning algorithm for solving partial differential equations (2018) — link
4. Neural networks-based algorithms for stochastic control and PDEs in finance (2021) — link
5. An approximation scheme for the optimal control of diffusion processes (1995)
6. Convergence of approximation schemes for fully nonlinear second order equations (1990) — link
7. Deep learning for continuous-time stochastic control with jumps — → 2505.15602
8. Controlled Markov Processes and Viscosity Solutions (2006)

4 of 65 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2570 linked papers.