{"id":"e147e2f4-4773-4a9a-82a2-aed70782e292","arxiv_id":"2501.09163","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"Under a minimal-change latent model, the invariant part of a representation is identifiable from one off-support target sample, under bounded dense shifts or arbitrary sparse shifts.","lead":"This paper gives theoretical conditions under which a model can identify the invariant latent causes of a single out-of-support test sample, enabling extrapolation. It distinguishes dense shifts, which must be small, from sparse shifts, which can be arbitrarily large, and uses the results to improve test-time adaptation methods.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 4.2's proof applies the support-only Jacobian bound J_u to off-support paths and to alternative generating functions, so the dense-shift identification claim is not established as stated.","rationale":"The paper's central contribution is the claim that the invariant latent c is identifiable from one off-support target under dense (Theorem 4.2) or sparse (Theorem 4.4) shifts. The sparse theorem's proof is largely self-contained and appears internally consistent given the strong rank assumptions. The dense theorem, however, has a serious gap. In Appendix A2, Eq. (4) expresses the difference between x_tgt and the boundary point g(c_k', s'_src) as an integral of J_g along the segment from s'_src to s'_tgt, but the model g' is only known to coincide with g on C × S_src; off the support, J_{g'} need not equal J_g, and both paths in Eq. (5) leave the support where the bound J_u is stated to hold. The proof of max{∥h∥, ∥h'∥} ≥ D/(2J_u) therefore does not follow. Without a global Jacobian bound on the model class, the distance-minimization term in Eq. (2) fails to exclude alternative models with high slope in the extrapolation region, which can explain the off-support target using a different c with arbitrarily small s-off-support distance. This is a concrete counterexample pattern, not merely a missing proof step. The reliance on Lemma A1 from an unpublished preprint compounds the issue. The reader's verdict of CONDITIONAL is reasonable but understates the risk: the dense-shift identification claim is unsupported as stated. I recommend REJECT unless the authors add an explicit global Jacobian/Lipschitz bound covering the extrapolation region and supply a full proof of Lemma A1.","tokens_in":23846,"tokens_out":11931,"duration_ms":116306,"concrete_test":"Construct a 1D counterexample satisfying Assumption 4.1: let c ∈ {0,1}, source s ∈ [0,1], with g(0,s)=s and g(1,s)=M+s for M ≫ 1, so D ≈ M. Choose target x_tgt = 1.5 with true explanation c=0, s_tgt=1.5, giving ∥h∥=0.5 < M/(2J_u) for J_u ≈ 1. Define g'(1,·) = g(1,·) on [0,1], but off-support let g'(1, s) rise steeply so that g'(1, 1.01) = 1.5 while preserving smoothness and invertibility. If g' is admissible and p'(x)=p(x) on X_src, the distance-minimization term selects g' over the true model, assigning x_tgt to c=1 and violating Definition 3.1. If the construction succeeds, Theorem 4.2 as stated is false; if it fails, the proof still requires an explicit global J_u bound to be valid.","verdict_should_be":"REJECT","load_bearing_attack":"The most load-bearing weakness is in the proof of Theorem 4.2 (Appendix A2). Equation (4) expresses the alternative model's displacement as an integral of J_g, the true Jacobian, even though the alternative model g' is only constrained to equal g on C × S_src; off the support, g' may have an arbitrarily large Jacobian. The bound ∥J_g∥ ≤ J_u is stated only 'on the support' (page 5), but the path integrals in Eqs. (4)–(5) run from the support boundary to off-support points s_tgt and s'_tgt, leaving S_src immediately. Moreover, J_{g'} need not equal J_g on those segments. Consequently, the derivation of max{∥h∥, ∥h'∥} ≥ D(c_k, c_k')/(2J_u) in Eq. (5) is unjustified. The distance-minimization term in objective (2) cannot rule out a high-slope alternative model that reaches x_tgt with s'_tgt arbitrarily close to S_src. Unless the model class is explicitly restricted to a global Jacobian bound over the entire extrapolation region (or at least a neighborhood of S_src), Theorem 4.2 does not follow. This is a gap in the proof of the central dense-shift claim, independent of the additional reliance on the unproved Lemma A1 from [38].","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies extrapolation, i.e., prediction on a target sample lying outside the support of the training distribution. It proposes a latent-variable model embodying a \"minimal change principle\": the target is out-of-support only through a changing latent variable s, while an invariant latent variable c remains inside the source support and the label depends only on c. Extrapolation is then reduced to identifying c. The authors give two identifiability results: Theorem 4.2 for dense shifts (all dimensions of x may be affected by s, but the off-support distance of the target's s is bounded) and Theorem 4.4 for sparse shifts (only few dimensions are affected, with no bound on the off-support distance). They also propose practical test-time adaptation regularizers motivated by the theory and report synthetic and real-world experiments.","tokens_in":1678,"tokens_out":1797,"duration_ms":97312,"significance":"If the two identification theorems were correct, the paper would make a valuable theoretical contribution: it would provide conditions under which latent-variable identification and subsequent classification are possible with a single off-support target sample, without overlapping supports or a full target distribution. The connection between the theory and practical TTA algorithms is interesting, and the empirical gains on CIFAR-C and ImageNet-C, while modest, support the practical relevance. The paper also has the strength of addressing a clearly defined and practically important problem, and it explicitly discusses limitations. However, the central proof of the dense-shift theorem has gaps that are currently load-bearing, and the sparse-shift proof depends on assumptions whose satisfaction by the stated objective is not fully justified. The manuscript is therefore not yet ready for acceptance.","major_comments":[{"comment":"The derivation of the lower bound Ju(||h|| + ||h'||) >= D(ck, ck') is not justified. The bound ||Jg(z)|| <= Ju is introduced in Section 4.1 as holding \"on the support,\" but the path integrals in Eqs. (4)-(5) run from s_src and s'_src, which lie on the boundary of S_src, to stgt and s'_tgt, which lie outside S_src; the paths immediately leave the region where the bound is assumed. Moreover, Eq. (4) applies Jg to the alternative generating function g', even though g' is only assumed to equal g on C times S_src and may have an arbitrarily large Jacobian off the support. Consequently, the inequality leading to max{||h||, ||h'||} >= D(ck, ck')/(2Ju) and the subsequent exclusion of alternative models in Eq. (6) do not follow. This gap is load-bearing for the entire dense-shift theorem; it could be repaired by assuming a uniform global Jacobian bound on both g and all admissible g' over the relevant extrapolation region, but that is not assumed in the paper.","section":"Appendix A2, Eqs. (4)-(5), Theorem 4.2"},{"comment":"The proof restricts the alternative generating function g' to satisfy g' = g over C times S_src, but the optimization in Eq. (2) only enforces the marginal equality p_hat(x) = p(x) on X_src. This does not force g_hat to agree with g on the support; any invertible reparameterization s_hat = phi(s) with g_hat(c, s_hat) = g(c, phi^{-1}(s_hat)) also matches the source distribution. Since Lemma A1 identifies c but not s, the estimated source support S_hat_src and the distance D(s_hat, S_hat_src) depend on the coordinate choice for s. An alternative model can therefore compress the s-coordinate and make its off-support distance arbitrarily small, undermining the \"minimal off-support distance\" selection step. The proof needs an additional condition that fixes the coordinate system or a model class for which distances are comparable across admissible models.","section":"Objective (2) and Appendix A2, proof of Theorem 4.2"},{"comment":"The proof of Theorem 4.2 begins with Lemma A1, which asserts source-support identification of the discrete invariant variable c and is imported verbatim from the authors' earlier preprint [38] without proof. This lemma is the foundation of the on-support identification step, and the paper gives no argument for it. Because the result is self-cited and not available in the present manuscript, the paper is not self-contained. Please include a proof of Lemma A1 in the appendix or state it as an additional assumption.","section":"Appendix A2, Lemma A1 (from [38])"}],"minor_comments":[{"comment":"The statement of Lemma A1 says \"S subset of R^ds,\" but in the proof the object [Jg(z)]_S is used as a submatrix indexed by a set of output dimensions, e.g., Ic\\s(z) subset of [dx]. The statement should be corrected to \"S subset of [dx]\" to match its use.","section":"Appendix A3, Lemma A1 (Brady et al.)"},{"comment":"The definition D(c1, c2) := inf_{s1,s2 in Bd(S_src)} ||g(c1, s1) - g(c2, s2)|| assumes the boundary of the source support is nonempty and the infimum is attained; this is not discussed, though it follows from compactness of the boundary under Assumption 4.1-ii together with continuity of g. A brief comment would help.","section":"Section 4.1, definition of D(c1, c2)"},{"comment":"The improvements reported in Table 2 (e.g., TeSLA-s from 12.1 to 11.7 on CIFAR10-C) are small and the standard deviations are not reported for the +SC variants in the same way as for the baselines; please clarify the number of seeds and whether the differences are statistically significant.","section":"Section 6, Tables 2 and 4"},{"comment":"The synthetic experiments use a VAE-based estimator, which does not satisfy the invertibility assumption (Assumption 4.1-i) exactly. The paper should acknowledge this gap between the theoretical conditions and the experimental implementation, or justify why the VAE approximately respects the condition.","section":"Section 5, synthetic experiments"}],"recommendation":"major_revision","confidential_remarks":"The paper's central contribution depends on Theorem 4.2, whose proof currently has a genuine gap regarding the use of the Jacobian bound off the support and across alternative generating functions. This is repairable by strengthening Assumption 4.1 to include a global uniform smoothness bound over the model class and the extrapolation region, but as written the theorem is not established. The paper also relies heavily on a self-cited unpublished lemma; this should be either proved in an appendix or clearly stated as an assumption. If the authors can close the dense-shift proof and make the model-class restrictions explicit, the paper would be a solid contribution; otherwise, the theoretical claim should be weakened accordingly."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The genuinely new thing here is the identification story for single off-support target samples. Prior work needed multiple distributions or overlapping supports; these authors show that under a minimal-change latent model, one target sample can be enough. The dense-versus-sparse shift distinction is a useful organizing idea, and the sparse-shift theorem with Jacobian-rank mechanistic dependence is a real new condition, not a repackaging. The paper also deserves credit for being honest about its limitations and for shipping synthetic and real experiments that at least illustrate the claims.\n\nWhat worried me most is the proof of Theorem 4.2. The stress-test note is right: the bound on the Jacobian is stated on the support, but the path integrals in Eqs. (4)-(5) go off support, and the alternative generating function g' is only constrained to match g on the source support. Nothing in the assumptions bounds the Jacobian of g' off the support, so the key inequality max{||h||,||h'||} >= D/(2J_u) doesn't follow. The distance-minimization term can't rule out a high-slope alternative model that reaches the target with s' very close to the support. This is in the appendix, it's load-bearing, and it needs a fix—either a global Jacobian bound on the model class or a more careful argument that doesn't rely on the true Jacobian along off-support paths. I also note the dependence on Lemma A1 from the authors' own preprint [38]; that's not automatically a problem, but it does mean the paper isn't self-contained for its starting point.\n\nThe sparse-shift proof (Theorem 4.4) looks more solid to me. The contradiction argument with Jacobian ranks is clever, and it doesn't need the off-support distance bound at all, which matches the empirical story. The connection from theory to the practical algorithm is heuristic—entropy minimization as a surrogate for target likelihood, sparsity as a proxy for sparse influence—but the authors say so themselves, and the empirical gains in Tables 2-4 are consistent with the qualitative story.\n\nMinor quibble: the synthetic experiments use a single distance loss weight for all dense distances; that's fine for a sanity check but not a thorough validation. Also, the real-world experiments are on TTA benchmarks where the \"one target sample\" framing is stretched—they use batches of 32.\n\nWho is this for? Researchers working on causal representation learning, test-time adaptation theory, or identifiability. It deserves a serious referee: the questions it asks are important, the sparse-shift result is novel, and the dense-shift theorem is fixable. My recommendation: send it to review, but make sure the referee checks Appendix A2 carefully and demands a revised proof for Theorem 4.2.","headline":"A real contribution to single-sample extrapolation theory, but the dense-shift theorem has a genuine proof gap that needs fixing before I'd trust it.","tokens_in":24645,"tokens_out":667,"would_cite":false,"duration_ms":9102,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Extrapolation is achievable, under a minimal-change latent model, by identifying an invariant variable that stays on-support.","keywords":["extrapolation","distribution shift","latent variable identification","minimal change principle","test-time adaptation","causal generative model","identifiability","sparse shifts"],"falsifier":"Exhibit one target sample satisfying Assumption 4.1, with a smooth invertible $g$, compact source support, finite $c$, continuous $p(s|c)$, and $s_{\\mathrm{tgt}}$ within the stated distance bound, for which every model solving the objective in Eq. 2 assigns the target to the wrong $c$; such a case would refute Theorem 4.2. Conversely, for the sparse case, a generative process satisfying Assumption 4.3 in which Eq. 3 maps two distinct true $c$ values to the same estimated $c$ would refute Theorem 4.4.","tokens_in":23662,"feed_emoji":"🧩","tokens_out":5557,"duration_ms":57853,"temperature":0.7,"pith_summary":"The paper claims that extrapolation, predicting labels for a target sample outside the training support, is possible when the shift obeys a minimal change principle: only a latent subspace $s$ changes and moves off-support, while an invariant latent subspace $c$ stays on-support and carries the label. It casts extrapolation as identifying $c$, and gives two identification theorems. If the shift is dense, identification holds when $c$ is discrete, the class manifolds are separated, and the target $s$ is within a distance bound set by the manifold separation and the smoothness of the generating map. If the shift is sparse, identification holds with no bound on the off-support distance, under an informativeness and a Jacobian-rank dependence condition. This would let a classifier trained on labeled source data be applied directly to off-support target samples.","feed_headline":"Two proofs show when one off-support sample can be predicted","feed_subtitle":"Label-relevant latents stay identifiable, so source classifiers transfer to off-support targets.","key_machinery":"The load-bearing object is the latent-variable generative model $x=g(c,s)$, $y=g_y(c)$, where $z=[c,s]$ is partitioned into an invariant variable $c$ and a changing variable $s$, with $g$ smooth and invertible. The estimation objective is maximum likelihood on the source density matched to $p(x)$, plus maximization of the target likelihood $p(c_{\\mathrm{tgt}})$; in the dense case an additional term minimizes the off-support distance of the estimated $s$. The proof uses block-wise identifiability of $c$, meaning estimated and true $c$ values correspond bijectively, which is established on the source support and then extended off support either by the distance bound in the dense case or by identifying the $c$-only-influenced dimensions through Jacobian rank conditions in the sparse case.","core_discovery":"Under the generative model $x=g(c,s)$, $y=g_y(c)$, with $c$ the invariant variable and $s$ the changing variable, extrapolation succeeds exactly when the invariant variable $c$ is identifiable for the target sample. Theorem 4.2 shows that under dense influence of $s$, maximizing the target likelihood subject to matching the source density and minimizing the off-support distance of $s$ identifies $c$, provided $c$ is finite-valued, the manifolds $g(c,\\cdot)$ are separated, and $s_{\\mathrm{tgt}}$ is close enough to the source support. Theorem 4.4 shows that when $s$ influences at most $d_s$ dimensions and the $c$-only-influenced dimensions are informative and mechanistically dependent, the same identification holds with no restriction on how far $s_{\\mathrm{tgt}}$ moves. The label then transfers because $c_{\\mathrm{tgt}}$ remains inside the source support, so a classifier $f: c \\mapsto y$ trained on the source applies to the target.","pith_inferences":["The dense-shift bound suggests a practical diagnostic: estimate the between-manifold separation and the Jacobian norm, then predict the maximum safe shift before extrapolation collapses; the paper does not develop this diagnostic.","If the latent decomposition is correct, extrapolation failure should be predictable from whether the shift also moves $c$ or overlaps label-relevant dimensions, so measuring that overlap could indicate when a target prediction can be trusted.","The sparse-shift rank-dependence condition may extend naturally to object-centric and compositional generalization, where $c$ is an object identity and $s$ is a pose or background variable, a connection the paper leaves implicit."],"forward_implications":["With as few as one out-of-support target sample, a classifier trained on $c \\mapsto y$ from the labeled source can transfer without retraining on the target.","Dense global shifts are extrapolable only up to a severity governed by the ratio between class-manifold separation and the generating map's Jacobian norm.","Sparse local shifts are extrapolable at any severity, provided the $c$-only-influenced dimensions remain informative and mechanistically dependent.","The theory motivates practical test-time adaptation: entropy minimization acts as target-likelihood maximization and sparsity constraints act as minimal-change regularization, and the paper reports gains on both.","The trade-off between shift scope and severity is testable: classification error should stay roughly constant across severity levels for sparse shifts but degrade sharply for dense shifts."],"supporting_citations":[{"why":"Supplies the prior partial-disentanglement framework and the iMSDA baseline that the paper extends from overlapping supports to off-support extrapolation.","marker":"[18]"},{"why":"Provides Lemma A1, the source-support discrete latent identification result on which Theorem 4.2 builds.","marker":"[38]"},{"why":"Provides the Jacobian rank-preservation lemma used in the proof of Theorem 4.4 to identify the $c$-only-influenced dimensions.","marker":"[39]"},{"why":"Is the MAE-TTT test-time training method that the paper augments with entropy minimization.","marker":"[20]"},{"why":"Is the TeSLA test-time adaptation method to which the paper adds sparsity constraints.","marker":"[21]"},{"why":"Is the TENT entropy-minimization baseline used in the shift-scope and severity experiment and in comparisons.","marker":"[15]"}],"fun_headline_variants":["Latent identification unlocks off-support prediction from one sample","Causal minimal change enables extrapolation without full target distribution","When can one off-support sample be predicted? Causal conditions","Extrapolation theory: identifiability of invariant latents","Single off-support sample prediction via causal identifiability"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The target sample is out of support only because a separate latent factor $s$ moves outside its training range, while the label-relevant factor $c$ remains inside the training range and the label depends only on $c$.","fun_headline_variants_meta":{"raw":{"variants":["Latent identification unlocks off-support prediction from one sample","Causal minimal change enables extrapolation without full target distribution","When can one off-support sample be predicted? Causal conditions","Extrapolation theory: identifiability of invariant latents","Single off-support sample prediction via causal identifiability"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000234,"raw_usage":{"total_tokens":1489,"prompt_tokens":927,"completion_tokens":562,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":543,"completion_tokens_details":{"reasoning_tokens":481}},"tokens_in":543,"tokens_out":562,"duration_ms":6193,"temperature":1.0,"reasoning_tokens":481,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T20:09:56.225696+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Exhibit one target sample satisfying Assumption 4.1, with a smooth invertible $g$, compact source support, finite $c$, continuous $p(s|c)$, and $s_{\\mathrm{tgt}}$ within the stated distance bound, for which every model solving the objective in Eq. 2 assigns the target to the wrong $c$; such a case would refute Theorem 4.2. Conversely, for the sparse case, a generative process satisfying Assumption 4.3 in which Eq. 3 maps two distinct true $c$ values to the same estimated $c$ would refute Theorem 4.4.","supporting_citations":[{"cited_title":"Partial disentanglement for domain adaptation","cited_arxiv_id":null,"evidence_quote":"Supplies the prior partial-disentanglement framework and the iMSDA baseline that the paper extends from overlapping supports to off-support extrapolation."},{"cited_title":"Learning Discrete Concepts in Latent Hierarchical Models","cited_arxiv_id":"2406.00519","evidence_quote":"Provides Lemma A1, the source-support discrete latent identification result on which Theorem 4.2 builds."},{"cited_title":"Provably learning object-centric representations","cited_arxiv_id":null,"evidence_quote":"Provides the Jacobian rank-preservation lemma used in the proof of Theorem 4.4 to identify the $c$-only-influenced dimensions."},{"cited_title":"Test-time training with masked autoencoders","cited_arxiv_id":null,"evidence_quote":"Is the MAE-TTT test-time training method that the paper augments with entropy minimization."},{"cited_title":"Tesla: Test- time self-learning with automatic adversarial augmentation","cited_arxiv_id":null,"evidence_quote":"Is the TeSLA test-time adaptation method to which the paper adds sparsity constraints."}],"review_version":1}