{"id":"dbb0e5f6-c2ca-4fbe-a51a-88f8097e85ae","arxiv_id":"2506.01893","paper_version":2,"verdict":"ACCEPT","confidence":"MODERATE","novelty_score":8.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"The paper characterizes the exact regimes where mean-field variational inference is accurate for LDA and MMSB, with tight finite-sample KL bounds.","lead":"This paper proves when mean-field variational inference accurately approximates Bayesian posteriors in latent variable models, and gives sharp finite-sample bounds. For topic models (LDA) it shows MFVI works exactly when the product of the number of documents and topics is much smaller than the number of words; for mixed-membership block models it identifies a partially grouped algorithm that works while the standard fully factorized version fails.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No significant objection identified.","rationale":"The reader's verdict is ACCEPT and I concur. I focused on the strongest claimed results: the exact DK=o(n) regime for LDA and the K=o(n) regime for partially grouped MMSB, plus the suboptimality of fully factorized MMSB VI. The upper bounds hinge on Assumption 2.3 and the E1/E2 bounds; I checked the algebraic scalings in Sections 3 and Appendix C and found them consistent (e.g., for MMSB the n0=n(n−1) local variables and K0=K^2 categories lead to E1=O(nK), and division by n0 gives the claimed (K/n) log(n/K) rate). The lower bounds are specialized to symmetric hyperparameters, which is sufficient to certify the exact threshold; there is no claim of failure for all hyperparameters in the large-DK regime. Theorem 1.5's construction and its sign convention (adding t to off-diagonal entries, which have higher B) are internally consistent. No fitted parameters, no code, and no circular reasoning were found. The only limitation is the proof's length, which prevents a fully independent check, but that is a verification challenge, not an identified error. Hence the verdict should remain unchanged.","tokens_in":82905,"tokens_out":28885,"duration_ms":276343,"concrete_test":"Independently re-derive the smoothness-term bound in Appendix C Step 2 for MMSB, tracking the general-framework parameters n0=n(n−1) and K0=K^2. Specifically, verify that the first product factor in Definition 2.4 is O(nK) as claimed in (C.37), not O(n0 K0)=O(n^2 K^2); if it were the latter, the normalization by n0 would yield a rate O(K^2), breaking the K=o(n) claim in Theorem 1.3.","verdict_should_be":"UNCHANGED","load_bearing_attack":"After a careful read, I cannot identify a load-bearing concern that would change the ACCEPT verdict. The most delicate premise, Assumption 2.3 (twice-continuous smoothness of the collapsed Hamiltonian F(y)), is explicitly verified for both applications: for LDA via the log-gamma expression (3.9) and for MMSB via (C.14), both arising from Dirichlet integrals. The core upper bounds (Theorems 2.1 and 2.2) are internally consistent, and the specialized bounds for LDA and MMSB follow the stated scalings (E1=O(K) for LDA, E1=O(nK) for MMSB, with normalization by n and n(n−1) respectively). The matching lower bounds (Theorems 1.2, 1.4, 1.5) are lengthy but contain no evident circular step or unsupported identity. The main residual risk is the length of the proof, which limits independent line-by-line verification, not a specific identified flaw.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper develops a general non-asymptotic framework for mean-field variational inference in Bayesian latent variable models with categorical local latent variables. Under explicit smoothness assumptions on the collapsed Hamiltonian F (Assumptions 2.1--2.3), Theorems 2.1 and 2.2 bound the log-partition function and the normalized KL divergence between the variational posterior and the true posterior. The framework is applied to two benchmark models. For LDA, Theorem 1.1 gives sup_X (1/n) D_KL(P̂ || P) ≤ C (DK/n) log(n/DK + 2), and Theorem 1.2 gives a matching lower bound in a symmetric example, establishing DK = o(n) as the exact regime of validity. For MMSB, Theorems 1.3 and 1.4 show that partially grouped VI achieves (1/n^2) D_KL ≤ C (K/n) log(n/K + 2) exactly in the regime K = o(n), while Theorem 1.5 exhibits a specific example where fully factorized VI has (1/n^2) D_KL ≥ δ. The proofs extend the nonlinear large deviations framework with local Hessian bounds, two-term Taylor expansions, and first- and second-moment control of the error terms.","tokens_in":83036,"tokens_out":12770,"duration_ms":150151,"significance":"If the results hold, they constitute a substantial contribution: this is the first finite-sample, exact-regime characterization of MFVI accuracy for LDA and MMSB. The upper bounds are uniform over data and hyperparameters, hold under model misspecification, and do not require posterior contraction or identifiability; the lower bounds match the upper rates up to constants. The general meta-theorems extend nonlinear large deviations with explicit constants and a new reduction from full VI to collapsed VI. The paper provides complete proofs in the appendices, including explicit verification of the key smoothness assumption, Assumption 2.3, for both LDA and MMSB. The main caveat is that the general framework's applicability beyond the two examples is conditional on verifying Assumption 2.3, which is a substantive but clearly stated premise.","major_comments":[],"minor_comments":[{"comment":"The phrase 'arbitrary global latent variables' overstates the scope of the general framework: Assumptions 2.1--2.3 require the conditional posterior of θ given Z and X to factorize and require a twice continuously differentiable extension F(y) with explicit gradient and Hessian bounds. Please qualify this wording so readers know the general theorems are conditional on these substantive conditions.","section":"Abstract and §1.1"},{"comment":"'Mixed mebership stochastic blockmodel' appears with a typo; it should be 'Mixed membership stochastic blockmodel'.","section":"Running headers, pages 2 and 7"},{"comment":"The section heading 'V ariational inference for latent Dirichlet allocation' contains an extra space; it should be 'Variational inference for latent Dirichlet allocation'.","section":"Section 3 heading"},{"comment":"The argument that the general D case follows from D = 1 via Jensen's inequality is compressed into a single sentence. A one-line derivation of sum_d n_d log(n_d/K + 2) ≤ n log(n/(DK) + 2) would improve transparency for readers who want to verify the scaling in D.","section":"Section 3, reduction to D = 1"},{"comment":"Theorem 1.5 demonstrates suboptimality of fully factorized VI in a specific two-community, fully observed example, but the abstract's wording could be read as a blanket statement that fully factorized VI is always suboptimal for MMSB. Please clarify that the claim is a counterexample showing a non-vanishing lower bound.","section":"§1.3.2 and abstract"},{"comment":"The reference for Huang, Liu, and Chen appears to be missing the volume number; it should include 'Bayesian Analysis 15, 711--736.'","section":"Reference [30]"}],"recommendation":"accept","confidential_remarks":"The manuscript is long and the appendices are dense; I did not independently verify every algebraic inequality in Appendices B--E, but I could not identify a load-bearing error. The stress-test concern about Assumption 2.3 is legitimate as a general caveat, but it is explicitly and correctly verified for LDA and MMSB, so it does not undermine the paper's central claims. The paper is within the journal's scope and makes a strong theoretical contribution."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Zhong, Mukherjee, and Sen give the first exact characterization of when mean-field variational inference is accurate for latent Dirichlet allocation and the mixed membership stochastic blockmodel. The results are new: the threshold DK=o(n) for LDA and K=o(n) for MMSB with partially grouped VI, with matching upper and lower bounds, is not in Chatterjee–Dembo or Yan. The general framework in Theorems 2.1 and 2.2 is a real extension of nonlinear large deviations: the two-term Taylor error terms, local Hessian bounds, and the reduction from full to collapsed VI via the Gibbs variational principle are all genuinely different from what came before. The proofs are complete in the appendices, with explicit constants. I did not find circular reasoning or fitted parameters; the lower-bound examples are constructed to match the upper rates, which is standard for tightness.\n\nThe main thing to check is the smoothness condition in Assumption 2.3, and the authors verify it for both applications using the explicit form of the collapsed Hamiltonian through log-gamma functions. So that load-bearing assumption is supported. The proof is long, and I have not checked every line, but the structure is coherent and the lemmas are stated with enough precision that a dedicated referee can verify them. The absence of code and data for the numerical simulations is a minor issue; the simulations are only illustrative, and the theoretical results do not depend on them.\n\nMy only hesitation is the sheer length of the proof, which will slow referees down. That is not a flaw in the math, but it is a real cost. I would still send this to a serious referee. The paper will be influential for the theory of variational inference: it gives the right language for when mean-field approximations are valid in high-dimensional latent variable models, and it opens a way to analyze other models. The citation pattern looks appropriate; the self-citations are to directly relevant previous work.\n\nFor a reading group, I would say maybe, because the core proofs are involved, but the introduction and main theorem statements are clear. I would cite this in my own work on VI accuracy if I ever venture into latent variable models.","headline":"A genuinely new and careful piece of theory that pins down when mean-field VI works for LDA and MMSB; worth serious refereeing.","tokens_in":83588,"tokens_out":1819,"would_cite":true,"duration_ms":18290,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62F15","62C10","60F10"],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper establishes exact validity thresholds for mean-field variational inference in LDA and MMSB.","keywords":["variational inference","mean-field approximation","latent Dirichlet allocation","mixed membership stochastic blockmodel","collapsed posterior","nonlinear large deviations","partially grouped variational inference","posterior approximation"],"falsifier":"In the symmetric LDA example of Theorem 1.2, set α=1/2, use a uniform topic-word matrix, take equal document lengths, and choose D and K so that DK = c n for a fixed c>0; if the minimized mean-field KL divergence per word decays to zero as n grows, the claimed exact regime is wrong, since the theorem predicts it stays bounded below by (c/5) log(1/c+2).","tokens_in":82699,"feed_emoji":"🧮","tokens_out":8560,"duration_ms":92605,"temperature":0.7,"pith_summary":"This paper tries to answer a practical question: when can a fast mean-field variational approximation be trusted to stand in for an intractable posterior? It develops a general framework that bounds the KL divergence between the variational posterior and the true posterior by a smoothness term plus a complexity term, then applies it to two benchmark models. For latent Dirichlet allocation, it proves that the per-word approximation error vanishes exactly when DK=o(n), where D is the number of documents, K the number of topics, and n the total number of words, and a matching lower bound shows the condition is necessary. For the mixed membership stochastic blockmodel, it proves that the usual fully factorized approximation can fail, and that a partially grouped version, which pairs the two membership draws attached to each directed edge, is accurate exactly when K=o(n). If these claims are right, practitioners get precise certificates for when variational inference is safe and guaranteed bounds on the ELBO as a proxy for log evidence.","feed_headline":"DK=o(n) is exactly when mean-field VI works","feed_subtitle":"Matching upper and lower bounds for LDA, plus proof that grouped VI beats vanilla MFVI in mixed-membership networks.","key_machinery":"The object that carries the argument is the collapsed Hamiltonian f(Z), the log of the unnormalized posterior after global latent variables are integrated out; its partition function S_{n,K} is the normalizing constant of the collapsed posterior. Lemma 2.1 converts the MFVI gap into the identity D_KL(Q̂ ∥ P(Z|X)) = log S_{n,K} − sup_y {E_{Q_y}[f(Z)] − I(y)}, where Q_y ranges over product distributions on the local variables and I(y) is their KL divergence to the base measure. The proof then approximates this free energy by replacing f with a twice-differentiable extension F(y) on the continuous simplex; the error is split into two terms Δ1(y) and Δ2(y) coming from a two-term Taylor expansion. Their first and second moments are controlled through local Hessian bounds in row-wise and multiplicative neighborhoods, and a new probability lemma converts those moment bounds into a high-probability bound. A separate step uses the Gibbs variational principle to reduce full VI to collapsed VI, which is what lets the LDA and MMSB rates come out sharp.","core_discovery":"The central claim is that, for Bayesian latent variable models with categorical local latent variables, mean-field variational inference approximates the true posterior well exactly when the total number of local categorical states is sublinear in the sample size, and the paper proves this by matching upper and lower bounds. In LDA, with n words, K topics, and vocabulary size V, the normalized KL gap is at most C(DK/n) log(n/DK+2) for every data set, so it vanishes exactly when DK=o(n); Theorem 1.2 shows that when DK is proportional to n, the gap stays bounded below by a constant multiple of the same expression. For the mixed membership stochastic blockmodel, the paper shows that the vanilla fully factorized mean-field posterior fails even in a simple symmetric example, and that partially grouped VI, which treats each directed pair of membership draws as one $K^{2}$-valued variable, achieves normalized KL gap at most C(K/n) log(n/K+2), with K=o(n) as the sharp regime. The general theorems supply the same kind of finite-sample KL and log-partition bounds for the whole class of models, not only for the two examples.","pith_inferences":["Beyond the paper, a natural conjecture is that other latent variable models obey a similar rule of thumb: validity of mean-field VI requires the product of the number of categorical states to be sublinear in sample size, so correlated topic models would plausibly need DK+KV=o(n), matching the paper's passing remark about a related Gaussian topic-model instability.","The MMSB result suggests a design heuristic: group local categorical variables that are strongly correlated under the posterior before applying mean field; the paper's simulations show Z_{i→j} and Z_{i←j} are highly correlated, and this grouping heuristic could be tested in other relational models.","The lower-bound examples use symmetric, exchangeable settings; with identifiable topics or communities, MFVI might remain accurate beyond the proved regime, so a useful stress test is whether the threshold shifts when topic-word matrices are sparse or the signal is strong.","Because the framework controls the log-partition function tightly, it could be repurposed for empirical Bayes estimation of hyperparameters, since the ELBO gap is now under explicit control; the paper lists this as a direction but does not develop it."],"forward_implications":["For LDA, practitioners can trust MFVI's posterior approximation and its ELBO as a proxy for log evidence only when the product DK is o(n); when DK is proportional to n, the per-word gap stays bounded below.","The LDA rates are optimal up to constants: the upper and lower bounds both scale as (DK/n) log(n/DK+2).","For MMSB, the standard fully factorized variational approximation should be replaced by partially grouped VI; treating each directed pair (Z_{i→j}, Z_{i←j}) as one categorical variable yields vanishing normalized KL when K=o(n).","The general framework gives non-asymptotic finite-sample bounds for any latent variable model in the stated class, including under model misspecification and for arbitrarily small hyperparameters, so the analysis covers both sparse and dense regimes.","Because the KL gap equals log p(X) minus the ELBO, the bounds also serve as certified error bounds for evidence approximation and for model-selection criteria based on the ELBO."],"supporting_citations":[{"why":"Defines the latent Dirichlet allocation generative model whose variational posterior the paper analyzes.","marker":"[13]"},{"why":"Introduces the mixed membership stochastic blockmodel and the fully factorized variational algorithm that the paper shows to be suboptimal.","marker":"[2]"},{"why":"Supplies the nonlinear large deviations framework this paper extends; its error bounds are the baseline that the new local-Hessian bounds improve.","marker":"[15]"},{"why":"Extends nonlinear large deviations beyond the hypercube; the paper's smoothness and complexity terms are compared against this work.","marker":"[58]"},{"why":"Previously employed partial grouping of membership variables for a logistic-normal mixed membership blockmodel, the idea generalized here as partially grouped VI.","marker":"[57]"}],"fun_headline_variants":["Mean-field VI works iff DK scales sublinearly in n","Grouped VI beats vanilla MFVI for mixed-membership networks","Sharp finite-sample bounds for mean-field VI","When does MFVI work? Only if DK=o(n)","Exact DK=o(n) threshold for mean-field variational inference"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The general theorems stand on the assumption that the collapsed Hamiltonian extends to a twice-differentiable function on the continuous simplex with bounded gradient and Hessian; without such a smooth extension, the error terms in the proof cannot be controlled and the upper bounds collapse.","fun_headline_variants_meta":{"raw":{"variants":["Mean-field VI works iff DK scales sublinearly in n","Grouped VI beats vanilla MFVI for mixed-membership networks","Sharp finite-sample bounds for mean-field VI","When does MFVI work? Only if DK=o(n)","Exact DK=o(n) threshold for mean-field variational inference"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000644,"raw_usage":{"total_tokens":2958,"prompt_tokens":943,"completion_tokens":2015,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":559,"completion_tokens_details":{"reasoning_tokens":1941}},"tokens_in":559,"tokens_out":2015,"duration_ms":14590,"temperature":1.0,"reasoning_tokens":1941,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T11:31:25.112613+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"In the symmetric LDA example of Theorem 1.2, set α=1/2, use a uniform topic-word matrix, take equal document lengths, and choose D and K so that DK = c n for a fixed c>0; if the minimized mean-field KL divergence per word decays to zero as n grows, the claimed exact regime is wrong, since the theorem predicts it stays bounded below by (c/5) log(1/c+2).","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the latent Dirichlet allocation generative model whose variational posterior the paper analyzes."},{"cited_title":"( 2018 )","cited_arxiv_id":null,"evidence_quote":"Introduces the mixed membership stochastic blockmodel and the fully factorized variational algorithm that the paper shows to be suboptimal."},{"cited_title":"A variational Bayes approach to debiased inference for low-dimensional parameters in high-dimensional linear regression","cited_arxiv_id":"2406.12659","evidence_quote":"Supplies the nonlinear large deviations framework this paper extends; its error bounds are the baseline that the new local-Hessian bounds improve."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Extends nonlinear large deviations beyond the hypercube; the paper's smoothness and complexity terms are compared against this work."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Previously employed partial grouping of membership variables for a logistic-normal mixed membership blockmodel, the idea generalized here as partially grouped VI."}],"review_version":1}