← Proof Notes
Counterexample / Generative models

A missing coefficient condition in InfoVAE Proposition 2

InfoVAE: Balancing Learning and Inference in Variational Autoencoders ↗

495 citations ↗Semantic Scholar · 2026-09-09

2019 · arXiv:1706.02262v3, 30 May 2018; published AAAI 2019 version also inspected · Reviewed 09 September 2026

An explicit example contradicts the selected statement as written.

Paper context

Overview

A variational autoencoder learns an encoder from data to hidden variables and a decoder back to data. InfoVAE changes its training objective to balance generation, inference, and the information retained in those hidden variables. It introduces adjustable weights and alternative ways to compare the aggregate latent distribution with the model's prior.

Role of the theoretical result

Proposition 2 supports the use of alternative divergence measures by asserting that exact data and inference matching remain globally optimal at a fixed mutual information. It provides an idealized optimization justification for the broader objective family.

Original paper ↗

01 / Summary

Summary of the result

The assumptions α<1 and λ>0 permit α+λ−1<0. In this region, substituting an arbitrary strict divergence into the marginal regularizer can reverse its intended effect. Set α=0, λ=1/2 and D=3 KL. An independent Gaussian encoder with mean m then has fixed mutual information zero and objective −H(pD)+m²/4. The matched configuration m=0 is not a global maximizer, and the objective is unbounded above along this family.

02 / Statement

Statement under review

arXiv v3, Proposition 2 and Eq. (6), PDF p. 4; proof pp. 11–12. Published AAAI version: Proposition 2 and Eq. (6), printed p. 5888. · paraphrased

Proposition 2 states that, at a fixed encoder mutual information, replacing the marginal KL term by any strict divergence preserves global optimality of a model that matches the data and has exact amortized inference, assuming α<1 and λ>0.

L^=EpD(x)q(z∣x)log⁡pθ(x∣z)−(1−α)EpD(x)DKL(q(z∣x)∥p(z))−(α+λ−1)D(q(z)∥p(z)).\widehat{\mathcal L}=\mathbb E_{p_D(x)q(z\mid x)}\log p_\theta(x\mid z)-(1-\alpha)\mathbb E_{p_D(x)}D_{\mathrm{KL}}(q(z\mid x)\|p(z))-(\alpha+\lambda-1)D(q(z)\|p(z)).
Relevant assumptions ↘
  • Both data and latent spaces are continuous.
  • The proposition permits α<1 and λ>0, without requiring α+λ−1≥0.
  • The marginal divergence is nonnegative and vanishes exactly when its two distributions agree.
  • Mutual information Iq(X;Z) is held fixed; the witness uses Iq=0 and finite, unit encoder variance.
  • Optimization is considered over a family containing the stated Gaussian encoders and an independent Gaussian decoder.

03 / Derivation

Counterexample and derivation

5 steps · complete derivation
  1. 01

    Choose an allowed parameter pair

    Set α=0 and λ=1/2. The coefficient α+λ−1 is −1/2, although the appendix treats it as positive.

    α=0,λ=12,α+λ−1=−12.\alpha=0,\quad\lambda=\tfrac12,\quad\alpha+\lambda-1=-\tfrac12.
  2. 02

    Use ordinary Gaussian distributions

    Let the data and prior both be standard normal. Keep the decoder equal to the data distribution and let the encoder mean m vary independently of x.

    pD(x)=pθ(x∣z)=N(0,1),p(z)=N(0,1),qm(z∣x)=N(m,1).p_D(x)=p_\theta(x\mid z)=\mathcal N(0,1),\quad p(z)=\mathcal N(0,1),\quad q_m(z\mid x)=\mathcal N(m,1).
  3. 03

    Check mutual information and the divergence

    The encoder ignores x, so Iq(X;Z)=0 for every m. Three times KL is still a strict divergence, and the relevant Gaussian KL equals m²/2.

    D=3DKL,Iqm(X;Z)=0,DKL(N(m,1)∥N(0,1))=m22.D=3D_{\mathrm{KL}},\quad I_{q_m}(X;Z)=0,\quad D_{\mathrm{KL}}(\mathcal N(m,1)\|\mathcal N(0,1))=\tfrac{m^2}{2}.
  4. 04

    Evaluate the complete objective

    The reconstruction term stays fixed. The negative KL contribution is outweighed by the positively signed substituted divergence.

    L^(m)=−H(pD)−m22+12(3m22)=−H(pD)+m24.\widehat{\mathcal L}(m)=-H(p_D)-\tfrac{m^2}{2}+\tfrac12\left(3\tfrac{m^2}{2}\right)=-H(p_D)+\tfrac{m^2}{4}.
  5. 05

    Compare with the claimed optimum

    At m=0 the generative model matches the data and its posterior equals the encoder. Nevertheless, every nonzero m increases the objective, and the objective is unbounded above along this family.

    L^(2)−L^(0)=1,L^(m)⟶+∞as ∣m∣→∞.\widehat{\mathcal L}(2)-\widehat{\mathcal L}(0)=1,\qquad \widehat{\mathcal L}(m)\longrightarrow+\infty\quad\text{as }|m|\to\infty.

Counterexample

At α=0, λ=1/2 and D=3 KL, shifting the encoder mean from zero to two increases the objective by one. The decoder, encoder variance and mutual information remain fixed.

α=0, λ=12, D=3DKL⟹L^(m)=−H(N(0,1))+14m2.\alpha=0,\ \lambda=\tfrac12,\ D=3D_{\mathrm{KL}}\quad\Longrightarrow\quad\widehat{\mathcal L}(m)=-H(\mathcal N(0,1))+\tfrac14m^2.
Numerical illustration

InfoVAE objective under a Gaussian mean shift

0.02.04.0
KL from the prior0.500
Objective gain0.250

With α = 0, λ = ½ and D = 3 KL, moving the encoder away from its prior improves the objective. The decoder and mutual information stay unchanged. The displayed gain is relative to the matching configuration m = 0.

Curve: objective gain for m from 0 to 4. The slider evaluates a point on this curve.

04 / Implications

Implications and proposed correction

Theoretical implications

Affected result

The arbitrary-divergence global-optimality statement fails in part of its printed parameter range. The KL-only objective is not refuted by this example.

Empirical scope

Relation to reported experiments

The example does not test the paper's experiments or show failure of its recommended positive-penalty settings.

Proposed correction

Sufficient conditions and revised bound

Adding α+λ−1≥0 removes the displayed reward-for-mismatch mechanism. Under finite entropy/KL quantities and availability of an exact model with the prescribed mutual information, it also gives the following sufficient argument: E_q log pθ(X|Z)≤−H_q(X|Z)=−H(pD)+Iq(X;Z), so the objective is bounded above by −H(pD)+αIq minus two nonnegative marginal penalties; an exact matched joint distribution attains that bound.

1−α>0,α+λ−1≥0.1-\alpha>0,\qquad \alpha+\lambda-1\ge0.
L^≤−H(pD)+αIq−(1−α)DKL(qZ∥pZ)−(α+λ−1)D(qZ∥pZ)≤−H(pD)+αIq.\widehat{\mathcal L}\le-H(p_D)+\alpha I_q-(1-\alpha)D_{\mathrm{KL}}(q_Z\|p_Z)-(\alpha+\lambda-1)D(q_Z\|p_Z)\le-H(p_D)+\alpha I_q.

Implementation implications

A general implementation that exposes arbitrary divergence choices should validate the coefficient range, or document an appropriate divergence-specific condition. Existing configurations with a nonnegative marginal penalty do not need a new training algorithm.

Limits of this review

  • The witness uses a valid rescaling of KL because the proposition permits any strict divergence.
  • All objective values are finite for each finite m; the construction does not rely on a vanishing encoder variance.
  • The repair stated here is sufficient; it is not claimed to characterize every admissible divergence and parameter combination.
  • The sufficient optimality repair assumes the entropy/KL manipulations are well-defined and that the prescribed mutual information is attainable by an exact matched joint model.

05 / References

Sources and correction history

  1. 01
    InfoVAE, exact arXiv version ↗

    Proposition 2; Eq. (6); proof of Proposition 2, pp. 4, 11–12

  2. 02
    Published AAAI paper ↗

    Proposition 2 and Eq. (6), printed p. 5888

  3. 03
    Version history ↗

    Latest listed revision v3, 30 May 2018

  4. 04
    The Information Autoencoding Family: A Lagrangian Perspective ↗

    Eq. (18), Section 6, and the InfoVAE conversion

  5. 05
    Coupled Variational Autoencoder ↗

    Section 2, Eq. (4); experimental parameter footnote on p. 6

Download the arithmetic witnesses · Python, no dependencies ↓
Correction search · 09 September 2026

2026-09-09 The primary proposition and appendix were reread; the published proposition was visually checked. Current arXiv history and targeted web searches for a correction to Proposition 2 and the negative marginal-divergence coefficient were checked. The later Information Autoencoding Family and Coupled Variational Autoencoder formulations were inspected at the relevant equations. "InfoVAE" "proposition" "correction" "InfoVAE" "alpha" "lambda" "negative" divergence InfoVAE alpha lambda divergence coefficient negative condition theorem No source addressing this exact failing parameter range was found in the bounded search. The Information Autoencoding Family uses nonnegative dual multipliers in a different constrained formulation. Coupled Variational Autoencoder restates the KL objective and uses α=0 with λ=2 or 3 in the inspected experiments. These are relevant valid formulations, but neither inspected passage qualifies the original unrestricted arbitrary-divergence proposition. The author-linked InfoVAE repository was unavailable (404). The OpenReview attachment could not be fully inspected because of access/verification failures. This is not an exhaustive search of all literature or private correspondence, and it does not establish novelty.

A bounded search is not evidence of priority or proof that no correction exists.

Entirely AI-generated analysis, including cross-checks by separate AI agents; no independent human verification. Authors have not been contacted. Review standard.

Suggest a correction with a source ↗
Next analysisAn omitted off-support condition in the EBGAN equilibrium characterization