{"id":"60a14bdb-2643-44e0-a717-0b7b3ef03acd","arxiv_id":"2608.12279","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"CAZO uses curvature-informed, forward-only optimization to adapt models at test time, achieving strong accuracy with about 70% lower memory use than backprop methods.","lead":"A test-time adaptation method called CAZO adapts models to new data using only forward passes, cutting memory use by about 70% compared with backpropagation-based methods. It estimates the loss landscape's curvature to make random search smarter, and claims state-of-the-art accuracy on standard robustness benchmarks.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"CAZO's SOTA claim rests on a composite loss with clean-source feature statistics; entropy-only CAZO collapses to 56.52%, and this dependence is not disclosed as a limitation.","rationale":"The reader identified the clean-statistics dependence as the weakest assumption, and my independent reading of the full text and supplementary confirms it is the load-bearing issue. Supplementary Table 7 is decisive evidence, but it appears only in the appendix and is not disclosed as a limitation; the main text's framing of CAZO as a general SOTA TTA method is therefore misleading in the source-free setting that defines standard TTA protocols. The secondary concern about the diagonal proxy versus non-axis-aligned low-rank Hessian is also legitimate and independently observable: the paper's own Figures 2-3 establish a low-rank subspace, but Eq. 6's diagonal EMA cannot express rotations of that subspace, so the mechanism that supposedly motivates the algorithm is not the mechanism implemented. I do not regard the TENT 2.7% fog entry as a fatal flaw by itself, since it could be a typo or a partially collapsed run, but combined with the clean-statistics dependence it further weakens the unqualified SOTA comparison. The convergence result is a rate guarantee under assumptions that do not certify the practical mechanism, so it does not counteract the empirical dependence. The recommended verdict CONDITIONAL matches the reader's assessment: the method is plausible and the controlled ZO-vs-CAZO comparison shows the curvature-aware sampling helps under the same loss, but the paper should either disclose the clean-statistics requirement prominently and re-scope the claims, or provide strong evidence that the alignment term is removable without losing the SOTA margin. I do not see grounds for REJECT because the method is not disproven, the code is promised, and the controlled comparisons are informative; UNVERDICTED would be too harsh given the extensive experiments, while ACCEPT would overlook the suppressed dependence and the unverified mechanism link.","tokens_in":23149,"tokens_out":3962,"duration_ms":30501,"concrete_test":"Run CAZO on the same ImageNet-C severity-5 protocol and ViT-B/16 with the entropy-only loss L_ent (no L_align), keeping all hyperparameters identical, and verify whether the 56.52% supplementary result reproduces. Then measure the accuracy gap between this entropy-only run and the 69.0% composite-loss run; if the gap exceeds 5 points, the SOTA claim must be re-scoped as dependent on clean-source statistics.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The paper's central SOTA claim, 69.0% average on ImageNet-C severity-5 (Table 1), is obtained with the composite loss L_com = L_ent + L_align, where L_align aligns adapter features to feature statistics from clean source data [40] (Section 4.2). Supplementary Table 7 is decisive: with entropy-only loss under the same protocol, CAZO drops to 56.52%, only +1.0 point over the 55.5% NoAdapt baseline. Virtually all of the claimed 13.5-point gain over NoAdapt therefore comes from the clean-statistics alignment term, not from the curvature-aware ZO sampling that is the paper's methodological contribution. This is load-bearing because standard TTA is source-free: at test time only the pretrained model and unlabeled shifted data are available, so clean-domain statistics are not available. The paper nowhere discloses this as a limitation; Section 6.1 presents CAZO as an unqualified SOTA over both BP-free and BP-based competitors. A secondary structural gap reinforces the concern: the motivation is a non-axis-aligned low-rank Hessian subspace (Section 4.1, Figures 2-3), but the implementation uses a diagonal EMA of squared ZO gradients (Eq. 6), which cannot represent a rotated subspace, so the mechanism claimed to drive the gains is not the mechanism tested. The convergence theorem (Theorem 1) assumes this gap away via Assumption 3. The TENT fog entry of 2.7% in Table 1 (vs 66.5% in Table 2) also indicates the BP-based comparison is unreliable, but even setting that aside, the entropy-only ablation undercuts the unqualified SOTA claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes CAZO, a zeroth-order (ZO) test-time adaptation (TTA) method. Its key empirical claim is that the Hessian of the adaptation loss is persistently low-rank and slowly varying; based on this, CAZO draws ZO perturbations from N(0, \\tilde{H}_t^{-1}), where \\tilde{H}_t is a diagonal EMA of squared ZO gradients. Only a lightweight adapter is updated, using forward-only finite differences. The paper reports state-of-the-art accuracy on ImageNet-C severity-5 (69.0%) and continual TTA (65.3%) with roughly 70% memory reduction relative to backpropagation-based methods, and it gives an O(1/sqrt(T)) convergence bound under smoothness and bounded-inverse-curvature assumptions.","tokens_in":23543,"tokens_out":10572,"duration_ms":89932,"significance":"If established, the method would be a useful forward-only alternative for memory-constrained TTA, and the empirical Hessian study in Section 4.1 is a valuable contribution. The code release and the breadth of benchmarks are strengths. However, the headline accuracy depends on a feature-alignment term using clean-source statistics; entropy-only CAZO reaches only 56.52% (Supplementary Table 7), close to the 55.5% NoAdapt baseline. The convergence proof also contains a miscalculated moment bound. These issues prevent the claims as stated from being accepted.","major_comments":[{"comment":"The headline result in Table 1 (69.0%) is obtained with the composite loss L_com = L_ent + L_align, where L_align uses feature statistics from clean data [40]. Under the identical protocol with an entropy-only loss, CAZO drops to 56.52%, only 1.0 point above the NoAdapt baseline (55.5%). Since standard TTA is source-free (only the pretrained model and unlabeled test data are available), the claimed state-of-the-art performance is not established for source-free TTA, and the paper does not disclose this dependence as a limitation. The authors should report the entropy-only result as the primary claim or explicitly state the requirement of clean-domain statistics.","section":"Sec. 4.2; Supplementary Table 7"},{"comment":"The bound E[||u||^4] = d(d+2)beta_u is incorrect. For u ~ N(0, \\tilde{H}_t^{-1}) with diagonal entries bounded by beta_u, the correct fourth moment is (tr \\tilde{H}_t^{-1})^2 + 2 tr((\\tilde{H}_t^{-1})^2), which is at most d(d+2)beta_u^2, not d(d+2)beta_u. The proof of Eq. (20) drops the square on beta_u. Because Theorem 1 uses this bound in the learning-rate choice and in the variance term, the stated convergence guarantee and the conditioning condition beta_l^2 > beta_u are not valid as written. The O(1/sqrt(T)) rate may survive, but the constants must be recomputed with beta_u^2, and the proof requires a second-moment version of Assumption 2, which currently bounds only the first moment.","section":"Supplementary Sec. 8, Lemma 1 and Theorem 1"},{"comment":"The per-corruption entries for TENT and CoTTA are inconsistent across tables: TENT reports 2.7% on fog in Table 1 versus 66.5% on fog in Table 2, and CoTTA reports 15.6% on fog in Table 1 versus 52.2% on fog in Table 2. The reported averages (59.8% for TENT, 61.9% for CoTTA) and the claimed gains of +6.3% and +7.1% over these baselines are therefore not credible as reported. The authors should correct the entries and recompute the comparisons.","section":"Table 1 and Table 2"},{"comment":"The motivating observation is a low-rank Hessian whose principal subspace is non-axis-aligned (top eigenvectors in Figures 2 and 3), but the method uses a diagonal covariance diag(D_t / (1 - (1 - nu)^t)). A diagonal matrix cannot represent a rotated low-rank subspace. The paper provides no evidence that the diagonal EMA of squared gradients approximates the top Hessian directions, and Assumption 3 simply assumes boundedness of the diagonal entries rather than any relationship to the observed subspace. The link between the Hessian analysis and the actual sampling mechanism is therefore unverified; the authors should either justify the diagonal approximation or compare against a low-rank non-diagonal preconditioner.","section":"Sec. 4.1, Eq. (6), Assumption 3"}],"minor_comments":[{"comment":"The sentence 'Because E[L(θ_T)] ≤ L(θ*)' has the inequality reversed; the telescoping argument needs E[L(θ_T)] ≥ L(θ*).","section":"Supplementary Sec. 8, Theorem 1 proof"},{"comment":"There is an index mismatch: Eq. (6) defines D_t using \\hat{g}(θ_{t-1}), while Algorithm line 9 says 'Update Hessian diagonal: \\tilde{H}_t^{-1} ← (D_{t-1}, \\hat{g}(θ_t))'. Please clarify whether the covariance used for sampling at step t is based on the gradient from the previous iteration.","section":"Algorithm 1, lines 8-10 and Eq. (6)"},{"comment":"The final bound writes O(μ^2) where μ is undefined; this should be O(ε^2).","section":"Supplementary Eq. (29)"},{"comment":"There are typos: 'pertrubation' should be 'perturbation', and 'competetive' should be 'competitive'.","section":"Section 6.4 heading and Table 3 caption"},{"comment":"The text says the comparison includes the zeroth-order baseline ZO (RGE), but Table 1 does not report ZO; it appears only in Table 4. Please either add it to Table 1 or adjust the text.","section":"Section 6.1 and Table 1"},{"comment":"The quantized experiments do not include ZOA even though related work presents ZOA as a concurrent ZO TTA method for quantized models; a comparison would strengthen the claims.","section":"Section 6.3, Table 5"}],"recommendation":"major_revision","confidential_remarks":"The core idea is promising and the empirical Hessian analysis is interesting, but the reliance on clean-source feature statistics, the incorrect moment bound in the convergence proof, and the inconsistent TENT/CoTTA entries in Table 1 need to be resolved before the manuscript can be accepted. I recommend major revision rather than rejection because these issues appear fixable within the scope of the paper."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things you should know. First, CAZO does beat vanilla ZO under the same loss and adapter (62.9 vs 69.0 on ImageNet-C severity-5), so the curvature-aware sampling idea has empirical legs. Second, the paper's headline 69.0% is not a source-free result: it uses a composite loss with an MSE alignment term that requires feature statistics from clean source data. Supplementary Table 7 shows that with entropy-only loss, CAZO drops to 56.52%, almost the NoAdapt baseline. The paper mentions the loss composition but never flags this dependence as a limitation, and Section 6.1 presents 69.0% as an unqualified SOTA.\n\nWhat's genuinely new: the Hessian observation (low-rank, slow-varying subspace during TTA) and the CAZO algorithm that samples perturbations from a diagonal EMA of squared gradients. The controlled comparison against vanilla ZO under identical conditions is a real contribution, and the memory numbers (1,695 MB vs 6,404 for TENT) are solid. The paper also acknowledges the concurrent ZOA work and includes it as a baseline, which is honest.\n\nSoft spots, in order of severity:\n1. The loss dependence above is load-bearing. If you strip the clean-statistics alignment term, the method's advantage mostly evaporates. That needs to be stated upfront, not buried in the supplement.\n2. The TENT fog entry in Table 1 is 2.7%, while Table 2 shows 66.5% for the same method and corruption. That's an error, and it inflates the claimed margin over TENT by several points.\n3. Lemma 1's moment bound is wrong: with diagonal entries of H^{-1} bounded by beta_u, E||u||^4 is bounded by d(d+2) beta_u^2, not d(d+2) beta_u. The proof has the same mistake. The O(1/sqrt(T)) rate survives, but the constants in Theorem 1 are not as stated.\n4. The motivation is a low-rank, non-axis-aligned Hessian subspace, but the implementation uses a diagonal covariance, which can't represent a rotated subspace. The empirical improvement over vanilla ZO may come from per-dimension scaling rather than the claimed subspace tracking. The paper doesn't test this distinction.\n5. Runtime is slow: 3,127 seconds vs 398 for ZOA and 210 for TENT. Memory is low, but the paper's 'favorable trade-off' language glosses over a ~15x wall-clock cost.\n\nThe hyperparameters (EMA coefficient, perturbation scale, k, adapter position, downsampling ratio) are all tuned on the benchmark; that's standard for TTA but worth remembering when reading the numbers.\n\nBottom line: this paper deserves a serious referee. The method is plausible, the Hessian observation is worth reporting, and the controlled ZO comparison is useful. But the SOTA claim needs to be re-scoped to the composite-loss setting and the technical errors fixed. If you're working on forward-only TTA, read it; just don't quote the 69.0% without the caveat.","headline":"CAZO is a real improvement over vanilla ZO for forward-only TTA, but its headline accuracy depends on clean-source statistics and the paper has technical errors that need fixing.","tokens_in":24073,"tokens_out":4686,"would_cite":false,"duration_ms":39043,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a curvature-aware zeroth-order method (CAZO) adapts a pretrained ViT at test time using only forward passes, reaching 69.0% average accuracy on ImageNet-C severity-5 while using more than 70% less memory than…","keywords":["test-time adaptation","zeroth-order optimization","memory-efficient adaptation","curvature-aware sampling","Hessian low-rank structure","forward-only adaptation","adapter tuning","ImageNet-C"],"falsifier":"A decisive check: recompute the top principal subspace of the true Hessian during adaptation and compare it with the dominant directions of the diagonal EMA covariance used by CAZO; if the overlap is near zero, the curvature link is not what drives the gains. A second decisive check: run CAZO on ImageNet-C with only the entropy term, without clean-data feature alignment; the supplement reports 56.52%, only about one point above no adaptation, which would show the headline result depends on clean-domain statistics rather than on curvature-aware sampling.","tokens_in":1914,"feed_emoji":"📐","tokens_out":2541,"duration_ms":66528,"temperature":0.7,"pith_summary":"This paper tries to establish that test-time adaptation can be made memory-efficient without giving up accuracy: by replacing backpropagation with zeroth-order gradient estimates, and by shaping the random perturbations with a cheap, sliding estimate of the loss curvature, a small adapter can adapt a frozen pretrained model using only forward passes. The central claim is that the Hessian of the adaptation loss stays low-rank and slowly varying during adaptation, so a diagonal exponential-moving-average of squared gradient estimates is enough to build an anisotropic sampling covariance. On ImageNet-C severity-5 the method reports 69.0% average accuracy, above both BP-free and BP-based baselines, and in continual adaptation 65.3%, with memory around 1.7 GB versus 6.4 GB for TENT. The paper's point is that curvature-informed sampling is the key to making zeroth-order optimization practical for on-device test-time adaptation.","feed_headline":"Curvature-aware forward-only TTA hits 69% on ImageNet-C","feed_subtitle":"A sliding diagonal Hessian estimate steers zeroth-order sampling, beating backprop adaptation with 70% less memory.","key_machinery":"The load-bearing object is the curvature proxy $\\tilde{H}_t^{-1} = \\operatorname{diag}\\left(D_t / (1-(1-\\nu)^t)\\right)$, with $D_t = (1-\\nu)D_{t-1} + \\nu \\hat{g}^2(\\theta_{t-1})$, where $\\hat{g}^2$ is the element-wise square of the CAZO gradient estimate. This diagonal matrix approximates the inverse Hessian diagonal on the fly and is used as the covariance of the Gaussian perturbation in the zeroth-order estimator $\\hat{g}(\\theta_t) = \\frac{1}{k}\\sum_{i=1}^k \\frac{L(\\theta_t+\\epsilon u_i)-L(\\theta_t-\\epsilon u_i)}{2\\epsilon} u_i$, with $u_i \\sim \\mathcal{N}(0,\\tilde{H}_t^{-1})$, making the sampling anisotropic: less perturbation along high-curvature directions and more along flat ones. The other machinery is a lightweight adapter inserted at layer 3 of the ViT and a composite loss combining an entropy term with an MSE feature-alignment term that uses clean-domain feature statistics.","core_discovery":"The central discovery the paper claims is a geometric fact about adaptation dynamics: during test-time adaptation of a ViT-B/16 with an early-layer adapter, the Hessian of the loss with respect to adapter parameters is persistently low-rank (top 20 eigenvalues capture more than 96% of the variance, with effective rank about 0.22% of the parameter dimensionality) and its dominant subspace rotates slowly between steps (projection ratio around 0.9). Building on that, CAZO estimates gradients by symmetric finite differences but draws perturbations from a Gaussian with covariance $\\tilde{H}_t^{-1}$, where $\\tilde{H}_t^{-1}$ is a diagonal matrix updated by an exponential moving average of squared gradient estimates. The paper claims this reduces the variance of zeroth-order gradient estimates enough that a forward-only adapter reaches 69.0% average accuracy on ImageNet-C severity-5, 65.3% in continual adaptation, and does so with about 1,695 MB of runtime memory versus 6,404 MB for TENT.","pith_inferences":["Extension: if the diagonal EMA is really tracking a slowly varying dominant subspace, the same curvature proxy could apply to other short-horizon black-box adaptation problems, such as online prompt tuning or continual learning with limited update budgets.","Extension: the paper's entropy-only supplement (56.52% versus 55.5% no adaptation) implies the state-of-the-art claim is not yet established for true source-free test-time adaptation; a natural next test is to estimate the alignment statistics online from the test stream and check whether the accuracy gap persists.","Extension: a rank-restricted Hessian sketch (for example, the top-20 directions) instead of a diagonal proxy would directly test whether the observed low-rank subspace, rather than axis-aligned curvature, is what carries the benefit."],"forward_implications":["On the model-reset ImageNet-C severity-5 protocol, CAZO reports 69.0% average accuracy, higher than the BP-free baselines FOA (65.8%) and ZOA (67.5%) and higher than the BP-based TENT (59.8%), SAR (62.7%), and CoTTA (61.9%).","In continual test-time adaptation without reset, CAZO reports 65.3%, above LCoTTA (62.3%), ETA (61.7%), and SAR (61.6%).","CAZO's gradient estimation needs only forward passes, and its measured runtime CUDA memory is 1,695 MB versus 6,404 MB for TENT, a reduction of more than 70%; memory stays near-constant as the number of perturbations $k$ grows from 2 to 20.","Under 8-bit and 6-bit quantization, CAZO retains 67.8% and 61.2% on ImageNet-C, suggesting the approach transfers to low-bit on-device settings.","The paper proves an $O(1/\\sqrt{T})$ nonconvex convergence rate for CAZO, with constants depending on the curvature bounds $\\beta_l$ and $\\beta_u$."],"supporting_citations":[{"why":"Supplies the random gradient estimation (RGE) estimator and the $O(d/k)$ variance scaling that motivates variance reduction.","marker":"[12]"},{"why":"Gives the classical zeroth-order convergence analysis that CAZO builds on for its $O(1/\\sqrt{T})$ rate.","marker":"[36]"},{"why":"Provides the forward-only FOA baseline and the clean-domain feature statistics used in CAZO's composite loss.","marker":"[40]"},{"why":"Concurrent zeroth-order TTA baseline (ZOA) that CAZO compares against and outperforms.","marker":"[7]"},{"why":"MeZO evidence that zeroth-order updates can fine-tune models with forward passes, motivating the BP-free direction.","marker":"[33]"},{"why":"TENT is the standard BP-based entropy-minimization baseline, used for accuracy and memory comparisons.","marker":"[51]"},{"why":"ImageNet-C is the main benchmark whose severity-5 protocol defines the reported accuracy numbers.","marker":"[19]"},{"why":"AdaptFormer supplies the adapter architecture and scaling factor used in CAZO's parameterization.","marker":"[3]"}],"fun_headline_variants":["Curvature-aware ZO TTA slashes memory 70%","Forward-only TTA matches BP with less memory","CAZO: low-rank Hessian steers zeroth-order TTA","Memory-light TTA: curvature-aware ZO hits 69% on ImageNet-C","Zeroth-order TTA adapts with 70% less memory"],"cache_read_input_tokens":26112,"weakest_assumption_plain":"The load-bearing premise is that the diagonal EMA of squared gradient estimates faithfully tracks the Hessian's dominant curvature subspace, and that the clean-domain feature statistics used by the alignment term are available at test time; if either gives way, the reported accuracy advantage over backpropagation methods collapses.","fun_headline_variants_meta":{"raw":{"variants":["Curvature-aware ZO TTA slashes memory 70%","Forward-only TTA matches BP with less memory","CAZO: low-rank Hessian steers zeroth-order TTA","Memory-light TTA: curvature-aware ZO hits 69% on ImageNet-C","Zeroth-order TTA adapts with 70% less memory"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000415,"raw_usage":{"total_tokens":2177,"prompt_tokens":1015,"completion_tokens":1162,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":631,"completion_tokens_details":{"reasoning_tokens":1068}},"tokens_in":631,"tokens_out":1162,"duration_ms":7685,"temperature":1.0,"reasoning_tokens":1068,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T00:10:42.679046+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A decisive check: recompute the top principal subspace of the true Hessian during adaptation and compare it with the dominant directions of the diagonal EMA covariance used by CAZO; if the overlap is near zero, the curvature link is not what drives the gains. A second decisive check: run CAZO on ImageNet-C with only the entropy term, without clean-data feature alignment; the supplement reports 56.52%, only about one point above no adaptation, which would show the headline result depends on clean-domain statistics rather than on curvature-aware sampling.","supporting_citations":[{"cited_title":"Test-time model adaptation with only forward passes","cited_arxiv_id":null,"evidence_quote":"Provides the forward-only FOA baseline and the clean-domain feature statistics used in CAZO's composite loss."},{"cited_title":"Optimal rates for zero-order con- vex optimization: The power of two function evaluations","cited_arxiv_id":null,"evidence_quote":"Supplies the random gradient estimation (RGE) estimator and the $O(d/k)$ variance scaling that motivates variance reduction."},{"cited_title":"Random gradient- free minimization of convex functions.Foundations of Com- putational Mathematics, 17(2):527–566, 2017","cited_arxiv_id":null,"evidence_quote":"Gives the classical zeroth-order convergence analysis that CAZO builds on for its $O(1/\\sqrt{T})$ rate."},{"cited_title":"Test-time model adaptation for quantized neu- ral networks","cited_arxiv_id":null,"evidence_quote":"Concurrent zeroth-order TTA baseline (ZOA) that CAZO compares against and outperforms."},{"cited_title":"Fine- tuning language models with just forward passes.Advances in Neural Information Processing Systems, 36:53038–53075,","cited_arxiv_id":null,"evidence_quote":"MeZO evidence that zeroth-order updates can fine-tune models with forward passes, motivating the BP-free direction."},{"cited_title":"Tent: Fully test-time adaptation by entropy minimization","cited_arxiv_id":null,"evidence_quote":"TENT is the standard BP-based entropy-minimization baseline, used for accuracy and memory comparisons."},{"cited_title":"Benchmarking neu- ral network robustness to common corruptions and perturba- tions","cited_arxiv_id":null,"evidence_quote":"ImageNet-C is the main benchmark whose severity-5 protocol defines the reported accuracy numbers."},{"cited_title":"Adaptformer: Adapting vision transformers for scalable visual recogni- tion.Advances in Neural Information Processing Systems, 35:16664–16678, 2022","cited_arxiv_id":null,"evidence_quote":"AdaptFormer supplies the adapter architecture and scaling factor used in CAZO's parameterization."}],"review_version":1}