← Proof Notes
Counterexample / Transformers & in-context learning

Prior total variation does not control conditional label agreement

An Explanation of In-context Learning as Implicit Bayesian Inference ↗

1,152 citations ↗Semantic Scholar · 2026-09-09

2021 · v6 · Reviewed 09 September 2026

An explicit example contradicts the selected statement as written.

Paper context

Overview

This paper explains in-context learning as inference about a hidden concept shared by examples in a prompt. In a model where documents come from hidden Markov models, it analyzes when an ideal predictor can recover the relevant concept despite differences between prompts and pretraining data. Experiments on synthetic data explore related behaviour in transformers and recurrent models.

Role of the theoretical result

Lemma 1 connects knowing the correct concept to predicting the prompt’s most likely label, providing a bridge in the main convergence argument. The counterexample shows that its assumption on starting distributions does not justify this bridge after conditioning on the input; it does not establish a failure of the main theorem under all of its assumptions.

Original paper ↗

01 / Summary

Summary of the result

Assumption 3 bounds the total-variation distance between hidden-state priors before conditioning on the test input. Equations (37)–(38) subsequently omit the distinct evidence normalizers and apply this bound to predictive distributions after conditioning. The five-state HMM below satisfies the stated prior-TV condition, with strictly positive transitions, but reverses the predictive argmax. A posterior-TV condition or an evidence-dependent bound is needed for the label-agreement step.

02 / Statement

Statement under review

Assumption 3, PDF p.5; Lemma 1 and proof, Appendix C, pp.17–18, equations (31)–(47), arXiv v6. · paraphrased

A total-variation bound on the prompt’s hidden start distribution ensures that its most likely label agrees with the model’s prediction conditioned on the correct concept and the observed prompt.

max⁡d∈DTV⁡(u,Td)<Δ/4⟹arg⁡max⁡yp(y∣Sn,x,θ∗)=arg⁡max⁡ypprompt(y∣x).\max_{d\in\mathcal D}\operatorname{TV}(u,T_d)<\Delta/4\quad\Longrightarrow\quad\arg\max_y p(y\mid S_n,x,\theta^*)=\arg\max_y p_{\rm prompt}(y\mid x).
Relevant assumptions ↘
  • The prompt and model use the same HMM transition and emission rules under the correct concept; they differ in the hidden-state prior after a delimiter.
  • Delta is the prompt predictive margin after observing the test input, whereas the TV assumption concerns priors before observing that input.
  • The lemma invokes Assumption 3. The explicit witness also uses positive transitions and a delimiter emitted by a unique hidden state.

03 / Derivation

Counterexample and derivation

5 steps · complete derivation
  1. 01

    Expose the missing normalization

    The proof defines W with entries p(y|x,h)p(x|h). Its columns sum to p(x|h), not to one. The conditional prediction for a start prior u is Wu divided by its total mass. Equations (37)–(38) omit that denominator separately for each prior.

    qu(y∣x)=(Wu)y1⊤Wu,∑yWyh=p(x∣h).q_u(y\mid x)=\frac{(Wu)_y}{\mathbf1^\top Wu},\qquad\sum_y W_{yh}=p(x\mid h).
  2. 02

    See the mechanism in two states

    Take priors u=(0.04,0.96), v=(0.005,0.995), input likelihoods (0.9,0.01), and labels that identify the hidden branch. Prior TV is 0.035, but conditioning makes the first branch most likely under u and the second branch most likely under v.

    0.04(0.9)0.04(0.9)+0.96(0.01)=0.78947,0.005(0.9)0.005(0.9)+0.995(0.01)=0.31142.\frac{0.04(0.9)}{0.04(0.9)+0.96(0.01)}=0.78947,\qquad\frac{0.005(0.9)}{0.005(0.9)+0.995(0.01)}=0.31142.
  3. 03

    Embed the example in a positive-transition HMM

    Use hidden states Ax,Bx,Ay,By,D. Ax and Bx emit x with probabilities 0.9 and 0.01; otherwise they emit z. Ay, By and D emit a, b and the delimiter respectively. T0 sends Ax to Ay, Bx to By, Ay and By to D, and D to 0.005 Ax+0.995 Bx. Set T=0.999 T0+0.0002 times the all-ones matrix. Every transition is positive. Choose a positive-probability prompt ending in the delimiter, such as [x,a,delimiter]. Only D emits that symbol, so the next hidden-state prior is exactly the D row of T, independent of the earlier prompt.

    T=0.999T0+0.000211⊤,min⁡i,jTij=0.0002.T=0.999T_0+0.0002\mathbf1\mathbf1^\top,\qquad\min_{i,j}T_{ij}=0.0002.
  4. 04

    Verify Assumption 3

    Set the prompt prior to 0.999(0.04,0.96,0,0,0)+0.0002 times the all-ones vector. The model prior after any delimiter is the D row of T. Their TV distance is 0.034965. For one-token test inputs x,z,a,b,delimiter, every prompt predictive margin is at least 0.57995488, so the required TV inequality holds for every possible input.

    0.034965<0.57995488/4=0.14498872.0.034965<0.57995488/4=0.14498872.
  5. 05

    Compute the conflicting labels

    For input x, the prompt predicts a with probability 0.78967744 versus b with 0.20972256. After the delimiter, the model predicts b with 0.67966461 versus a with 0.31973539. The same concept is used throughout; the discrepancy comes from conditioning the two close start priors.

    arg⁡max⁡qu(⋅∣x)=a≠b=arg⁡max⁡qTD(⋅∣x).\arg\max q_u(\cdot\mid x)=a\ne b=\arg\max q_{T_D}(\cdot\mid x).

Counterexample

The priors have TV distance 0.034965, below one quarter of the minimum prompt margin, 0.57995488. Conditional on x, the prompt predictor assigns approximately 0.78968 to a, whereas the delimiter-conditioned predictor assigns approximately 0.67966 to b.

TV⁡(u,TD)=0.034965<Δ/4,qu(a∣x)=0.78967744,qTD(b∣x)=0.67966461.\operatorname{TV}(u,T_D)=0.034965<\Delta/4,\quad q_u(a\mid x)=0.78967744,\quad q_{T_D}(b\mid x)=0.67966461.

04 / Implications

Implications and proposed correction

Theoretical implications

Affected result

Lemma 1 fails under its stated prior-TV assumption. Its application in the main convergence proof consequently requires a stronger conditioning argument. The construction here establishes the lemma-level failure; it does not verify a counterexample to the complete main theorem.

Empirical scope

Relation to reported experiments

The GINC experiments and the general usefulness of a Bayesian account of in-context learning are not refuted by this lemma-level witness.

Proposed correction

Sufficient conditions and revised bound

Retain the normalizers and assume a bound on the hidden-state posterior after conditioning, or derive such a bound using a positive lower bound on the evidence probability and an appropriately smaller prior-TV tolerance. If the two conditioned hidden-state distributions have TV distance below Delta/2, the common conditional label kernel cannot increase TV. Each label probability then changes by less than Delta/2, which preserves the unique argmax. Prior TV alone does not imply this condition.

TV⁡(u(⋅∣x),v(⋅∣x))<Δ/2⟹arg⁡max⁡qu=arg⁡max⁡qv.\operatorname{TV}(u(\cdot\mid x),v(\cdot\mid x))<\Delta/2\quad\Longrightarrow\quad\arg\max q_u=\arg\max q_v.

Implementation implications

Use separate likelihood normalizers for each prior. A theorem based on prior shift must include evidence-sensitive constants or directly control the conditional distribution.

Limits of this review

  • The two-state calculation illustrates the mechanism; the five-state construction is the HMM witness used for the lemma.
  • The public claim is restricted to Lemma 1, even though the main proof invokes it.
  • Correctly applying Bayes’ rule does not itself guarantee the claimed label agreement when the two priors differ.

05 / References

Sources and correction history

  1. 01
    arXiv v6 ↗

    Assumption 3, p.5; Lemma 1, pp.17–18; equations (33)–(38) and (44)

  2. 02
    Author code repository ↗

    All five issue bodies inspected; none addressed this lemma

  3. 03
    Author explanation ↗

    2022 explanation and limitations; no correction to the prior-TV lemma identified

  4. 04
    Later Bayesian model-averaging analysis ↗

    Section 4.1, Proposition 4.1 and discussion of Xie et al.; no repair of the selected prior-TV implication identified

  5. 05
    Later kernel-regression analysis ↗

    Sections 3.2–3.3 and Appendix A; different covariance-based assumptions, not a correction of Lemma 1

  6. 06
    Dual Operating Modes of In-Context Learning ↗

    Sections 2 and 5.3 discuss biased inputs and task retrieval; no statement addressing the selected hidden-start-prior bound identified

Download the arithmetic witnesses · Python, no dependencies ↓
Correction search · 09 September 2026

2026-09-09 Fresh arXiv history confirms latest v6, 21 July 2022. Re-read Assumption 3 and the complete Lemma 1 proof; visually inspected equations (37)–(47). Exact-title searches with correction, erratum, mistake, normalization, posterior, Assumption 3, Lemma 1, and total variation. Fresh author repository listing: all five issue bodies inspected. Comment retrieval then hit the GitHub rate limit. Read the author explanation and relevant theoretical/related-work portions of three later primary analyses listed in Sources. These discuss Bayesian inference, distribution mismatch, or biased inputs, but no correction of the specific prior-to-posterior bound was identified. No addressing source for this exact missing-normalization implication was found in the inspected material. OpenReview direct PDF/API returned 403 and repository comments could not be refreshed. Related-paper inspection was targeted, not an exhaustive citation-graph search. No priority claim is made.

A bounded search is not evidence of priority or proof that no correction exists.

Entirely AI-generated analysis, including cross-checks by separate AI agents; no independent human verification. Authors have not been contacted. Review standard.

Suggest a correction with a source ↗
Next analysisSign reversal of the projected CQL penalty