← Proof Notes
Counterexample / Optimization & generalization

The KL identity requires more than a finite symmetry set

Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Data

975 citations ↗Semantic Scholar · 2026-09-09

2017 · arXiv:1703.11008v2, 19 October 2017; UAI 2017 supplement · Reviewed 09 September 2026

An explicit example contradicts the selected statement as written.

Paper context

Overview

This paper computes informative numerical generalization bounds for stochastic neural networks, including networks with many more parameters than training examples. It optimizes a PAC-Bayes objective that balances measured training error against the complexity of a distribution over network weights.

Role of the theoretical result

The selected appendix lemma explores whether averaging equivalent parameter settings can further reduce the complexity term. It is an auxiliary route to tighter bounds, not a step used to obtain the paper's reported numerical bounds.

Original paper ↗

01 / Summary

Summary of the result

Lemma B.1 asserts an exact KL reduction for averaging an arbitrary finite set of prior-preserving network symmetries. The valid mixture identity contains the average component-to-mixture divergence; it generally cannot be replaced by the divergence of one distinguished component. The replacement is justified for a complete finite-group average, whose mixture is invariant. A partial cyclic average violates the printed identity and can underestimate the resulting KL complexity.

02 / Statement

Statement under review

Appendix B.1, Lemma B.1, physical p11; official supplement p1. · paraphrased

For any finite set of parameter symmetries preserving the prior, Lemma B.1 assigns an exact KL reduction to the equally weighted mixture of transformed posteriors.

QS=1SσSQσ,KL(QSP)=KL(QP)KL(QQS)Q^S=\frac1{|S|}\sum_{\sigma\in S}Q_\sigma,\qquad \mathrm{KL}(Q^S\|P)=\mathrm{KL}(Q\|P)-\mathrm{KL}(Q\|Q^S)
Relevant assumptions
  • Use a three-parameter network whose output is invariant to permuting three hidden units; for example h_w(x)=sign(Σi ReLU(w_i x)−1).
  • The prior is the full-support Gaussian P=N(0,I3). Let Ei be the sector where coordinate i is largest; ties have probability zero and P(Ei)=1/3.
  • Set dQ/dP=3qi on Ei, with q=(0.6,0.3,0.1). This positive bounded density ratio gives an absolutely continuous Q with finite differential entropy.
  • S contains the identity and one coordinate 3-cycle. It is a finite set of genuine network symmetries, but it is not a group.

03 / Derivation

Counterexample and derivation

6 steps · complete derivation
  1. 01

    Choose the cycle so its sector masses are qσ=(0.1,0.6,0.3). The mixture has masses qS=(0.35,0.45,0.2).

  2. 02

    Because the density ratios are constant on sectors, every continuous KL integral equals its three-point categorical calculation.

  3. 03

    KL(Q||P)=0.20066656381, KL(QS||P)=0.04995849931 and KL(Q||QS)=0.13244364995.

  4. 04

    The claimed right side is 0.06822291386, not the actual 0.04995849931. The exact left-minus-right residual is log(11664/16807)/20.

  5. 05

    A second admissible ordering q=(0.6,0.1,0.3) gives mixture (0.45,0.35,0.2). The actual KL remains 0.04995849931, but the printed right side is 0.03169408476: it can also understate complexity.

  6. 06

    The correct mixture decomposition subtracts the average of KL(Qσ||QS) over all selected transformations. Under full finite-group averaging these terms coincide; an arbitrary subset does not ensure that equality.

Counterexample

For q=(0.6,0.1,0.3), its cyclic transform (0.3,0.6,0.1), and a uniform prior, the mixture KL is approximately 0.04995850. The claimed difference of KL terms is approximately 0.03169408. Equal-probability Gaussian sectors embed this calculation in absolutely continuous distributions.

KL(QSP)=0.049958499310.06822291386=KL(QP)KL(QQS)\mathrm{KL}(Q^S\|P)=0.04995849931\ne0.06822291386=\mathrm{KL}(Q\|P)-\mathrm{KL}(Q\|Q^S)

04 / Implications

Implications and proposed correction

Theoretical implications

Affected result

Refutes the exact identity for arbitrary finite symmetry sets. The weaker claim that such invertible prior-preserving symmetrization does not increase KL still follows from convexity.

Empirical scope

Relation to reported experiments

The reported numerical PAC-Bayes bounds did not use this symmetry reduction, so this finding does not invalidate those reported bounds.

Proposed correction

Sufficient conditions and revised bound

For a finite collection of invertible prior-preserving maps, use KL(QS||P)=KL(Q||P)−|S|⁻¹Σσ KL(Qσ||QS). Alternatively, require S to be a finite group and average over the whole group. Expand the average KL(Qσ||P), insert log(qσ/p)=log(qσ/qS)+log(qS/p), and average. The second integral becomes KL(QS||P). Invariance of a full-group mixture makes the remaining KL terms equal.

Implementation implications

A symmetry-based certificate should evaluate the actual mixture complexity or the average component-to-mixture divergence. Subtracting only KL(Q||QS) after sampling an arbitrary list of permutations is not an exact certificate.

Limits of this review

  • The finite set is intentionally not closed under composition; the printed lemma does not require closure.
  • The Gaussian construction avoids relying on a discrete posterior or a compactly supported prior.
  • The finding concerns an auxiliary identity, not the validity of PAC-Bayes theory.
  • The simplified group formula requires the usual measurable bijection and prior-invariance conditions. No claim is made that full-group averaging is computationally feasible for a large network.

05 / References

Sources and correction history

  1. 01
    Current primary paper

    Appendix B.1, Lemma B.1, p11

  2. 02
    Later invariance analysis

    Group-based function averaging; a distinct operation from arbitrary finite mixtures of parameter distributions

  3. 03
    Later PAC-Bayes symmetry analysis

    Lemma 1.4 and averaging-operator construction; general deterministic-pushforward decomposition

Download the arithmetic witnesses · Python, no dependencies ↓
Correction search · 09 September 2026

2026-09-09 Rechecked the current lemma and its explicit finite-set wording in the rendered source. Revisited the official UAI supplement and the relevant parts of Lyle et al. and Beck–Ochs. Repeated exact-title/arXiv, finite-set, mixture, symmetry, KL and correction searches. No retrieved source explicitly repairs this arbitrary-finite-set identity. The later group/pushforward results concern different operations. This does not prove the mixture identity is unknown; the mathematical repair itself is elementary. The correction search is selective and bounded, not an inventory of all PAC-Bayes literature.

A bounded search is not evidence of priority or proof that no correction exists.

Entirely AI-generated analysis, including cross-checks by separate AI agents; no independent human verification. Authors have not been contacted. Review standard.

Suggest a correction with a source ↗
Next analysisA negative upper bound on the squared BatchNorm weight gradient