Analytic Regularity and Approximation Limits of Coefficient-Constrained Shallow Networks

Paper: arXiv 2601.04914

Authors: Jean-Gabriel Attali

Abstract

We study approximation limits of single-hidden-layer neural networks with analytic activation functions under global coefficient constraints. Under uniform $\ell^1$ bounds, or more generally sub-exponential growth of the coefficients, we show that such networks generate model classes with strong quantitative regularity, leading to uniform analyticity of the realized functions. As a consequence, up to an exponentially small residual term, the error of best network approximation on generic target functions is bounded from below by the error of best polynomial approximation. In particular, networks with analytic activation functions with controlled coefficients cannot outperform classical polynomial approximation rates on non-analytic targets. The underlying rigidity phenomenon extends to smoother, non-analytic activations satisfying Gevrey-type regularity assumptions, yielding sub-exponential variants of the approximation barrier. The analysis is entirely deterministic and relies on a comparison argument combined with classical Bernstein-type estimates; extensions to higher dimensions are also discussed.

Complexity vs Empirical Score

• Math Complexity: 8.0/10
• Empirical Rigor: 1.5/10
• Quadrant: Lab Rats — theoretically deep, empirically untested

Why this score: The paper presents a rigorous theoretical analysis of neural network approximation limits using complex analysis (Bernstein ellipses) and functional analysis (Gevrey regularity). However, it is entirely deterministic and theoretical with no empirical evaluation or backtesting datasets included.

Research Flowchart

  flowchart TD
    A[Research Goal: Limits of Shallow Networks with Analytic Activations & Coefficient Bounds] --> B[Inputs: Network Model with L1 / Sub-exponential Coefficient Constraints]
    B --> C[Methodology: Deterministic Comparison & Bernstein-Type Estimates]
    C --> D[Analysis: Establish Uniform Analyticity & Extend to Gevrey-Type Regularity]
    D --> E[Outcome 1: Network Error Bounded Below by Best Polynomial Approximation]
    D --> F[Outcome 2: Controlled Shallow Networks Cannot Outperform Polynomial Rates on Non-Analytic Targets]

Cited References (13)

1. Universal approximation bounds for superpositions of a sigmoidal function (1993)
2. Approximation of functions by a multilayer perceptron: a probabilistic approach (1997)
3. Approximations of Functions by a Multilayer Perceptron: the Random Case (1995)
4. Constructive Approximation (1993)
5. Nonlinear approximation (1998)
6. $\varepsilon$-entropy and $\varepsilon$-capacity of sets in functional spaces (1959)
7. Le \c c ons sur les propri 'e t 'e s extr 'e males et la meilleure approximation des fonctions analytiques d’une variable r 'e elle (1912)
8. Theory of Approximation of Functions of a Real Variable (1963)

0 of 13 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.