{"id":"1538c685-b3f6-4aa0-8b83-c74dc24ef128","arxiv_id":"2508.02180","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A two-forward-pass zeroth-order adaptation framework enables test-time adaptation of quantized neural networks under domain shift.","lead":"This paper proposes a test-time adaptation method for quantized neural networks that updates the model with just two forward passes instead of gradient backpropagation. It also adds a memory scheme to store and reuse domain knowledge, reporting a 5.0% accuracy gain on ImageNet-C for a quantized ViT-B model.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Zeroth-order estimate noise in a high-dimensional quantized network is the load-bearing risk; the abstract gives no variance-reduction evidence, so the reported 5.0% gain may be setup-specific.","rationale":"The reader's weakest assumption identifies exactly the load-bearing concern: the feasibility of zeroth-order adaptation from two forward passes in a high-dimensional quantized network. Since the review is abstract-only, no internal derivation or experimental protocol is available to check variance-reduction mechanisms, seed sensitivity, or fairness of the FOA comparison. The strongest claim depends on this estimate being informative enough to guide continual adaptation, and the abstract supplies no evidence for that. I therefore keep the reader's UNVERDICTED verdict, while sharpening the required check into a concrete reproducibility and variance-analysis experiment. No ad hominem or theatrical language is needed; the risk is a standard one in zeroth-order optimization and is empirically testable.","tokens_in":770,"tokens_out":1961,"duration_ms":25746,"concrete_test":"Using the released code, run ZOA and FOA on the quantized W6A6 ViT-B model on ImageNet-C across three seeds and three smoothing-parameter values, and on a subset of layers compute the cosine similarity between the two-point zeroth-order direction and a reference gradient direction from a full-precision proxy or a larger-sample zeroth-order estimate. If the cosine similarity is near zero, or if the reported 5.0% improvement over FOA fluctuates by more than its own size across seeds, then the two-forward-pass estimate is not the actual source of the gain.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that two forward passes per iteration yield a usable gradient estimate for adapting a W6A6 ViT-B, a model with roughly 86M parameters. In standard zeroth-order optimization, a two-point estimate along one random direction has variance that grows with parameter dimension and shrinks only with the smoothing parameter; without a variance-reduction or low-dimensional subspace mechanism, this estimate can be nearly orthogonal to the true gradient. The abstract does not describe such a mechanism, nor does it report variance across seeds, domains, or hyperparameters. Therefore the reported 5.0% improvement over FOA on ImageNet-C may come from a favorable smoothing parameter, a low effective dimensionality of the quantized adaptation, or a confounded comparison, rather than from a generally reliable two-forward-pass gradient. This is not an internal inconsistency, but it is a concrete correctness risk that only the full text and experiments can resolve.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This abstract-only manuscript proposes ZOA, a test-time adaptation framework for quantized neural networks. ZOA uses two forward passes to obtain a zeroth-order gradient estimate, avoiding backpropagation through quantized models. It also introduces a domain knowledge management scheme for reusing domain-specific information across adaptation steps. The abstract claims a 5.0% improvement over FOA on ImageNet-C with a W6A6 ViT-B and results on three architectures. No derivation, full experimental protocol, or statistical analysis is included in this version.","tokens_in":964,"tokens_out":4492,"duration_ms":48775,"significance":"If the claimed two-forward-pass adaptation is reliable and the 5.0% ImageNet-C gain is reproducible, the work would address a real deployment gap: quantized models cannot easily be fine-tuned at test time, yet they suffer disproportionately from domain shift. The low compute and memory overhead of ZOA would be an important contribution. However, given the abstract-only evidence, the significance is conditional; the central risk is whether zeroth-order estimates in high-dimensional spaces are accurate enough for adaptation.","major_comments":[{"comment":"The central mechanism is a zeroth-order gradient estimate from two forward passes (Abstract). For a model such as W6A6 ViT-B with roughly 86M parameters, a standard two-point estimator along one random direction has variance that grows with dimension and yields a descent direction that is nearly orthogonal to the true gradient unless variance reduction or a low-dimensional subspace is used. The abstract does not describe any such mechanism, nor does it report the smoothing parameter, the number of adaptation steps, or seed variability. Without this information, the reported 5.0% improvement could be setup-specific rather than a property of the method. This is the load-bearing point that needs to be addressed in the full manuscript.","section":"Abstract"},{"comment":"The abstract claims 'theoretical and empirical illustrations' of the vulnerability of quantized models, but no derivations or experimental details are given. It is impossible to judge whether the reported 5.0% improvement over FOA is statistically meaningful from a single number. The full text must provide the exact experimental protocol: datasets, corruption types, architectures, baselines, and error bars, along with the theoretical statement and its proof.","section":"Abstract"},{"comment":"The 'domain knowledge management scheme' is described only qualitatively. The abstract does not define how domain knowledge is represented, how it is stored with negligible memory, or how it avoids using target-domain statistics in a way that would leak information into the adaptation. Since the method is evaluated on held-out corruptions, any storage of test-domain information during adaptation must be carefully separated from the evaluation protocol; the abstract does not clarify this.","section":"Abstract"},{"comment":"The phrase 'eliminating the computational burden of existing methods' is not supported by the abstract. Zeroth-order optimization typically requires many iterations to reach a good solution, and each iteration costs two forward passes. Without reporting the total compute (number of iterations, forward pass cost) relative to FOA and other baselines, the computational advantage is not established. Please provide a wall-clock or FLOPs comparison.","section":"Abstract"}],"minor_comments":[{"comment":"Please expand the acronyms FOA and ZOA at first use, and cite the FOA method.","section":"Abstract"},{"comment":"The abstract says 'three classical architectures' but does not name them; please list them for reproducibility.","section":"Abstract"},{"comment":"The phrase 'continual' in 'continual zeroth-order adaptation' should be defined; it is not a standard term in TTA literature.","section":"Abstract"},{"comment":"The claim that quantized models suffer 'significantly more pronounced' degradation than full-precision models is stated without a reference to the supporting figure or table; please point to the empirical section.","section":"Abstract"}],"recommendation":"uncertain","confidential_remarks":"This is an abstract-only review. The paper may be perfectly sound, but the evidence provided is insufficient for any verdict other than uncertainty. The stress-test concern about zeroth-order variance in high dimension is the key technical risk; I recommend that the editor require the full manuscript and specifically ask for a variance/noise analysis and seed-to-seed variability before sending to a second round."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThis is an abstract-only read, so take my verdict as provisional. What strikes me as genuinely new: ZOA formulates TTA for quantized models using only two forward passes, sidestepping backprop, which is indeed unsupported on quantized networks. That is a real practical gap, and the domain knowledge management scheme—a small memory buffer reused across corruptions—is a sensible addition. If the full paper backs the ImageNet-C W6A6 ViT-B claim with proper comparisons, it will be useful for edge deployment.\n\nThe soft spot is the one you flagged: a two-point zeroth-order estimate in an 86M-parameter space is high-variance unless the effective dimension of the adaptation is low or some variance-reduction is used. The abstract does not mention subspace selection, coordinate sampling, or multiple random directions per step, nor does it report per-domain or per-seed breakdowns. That is a real correctness risk, but not fatal—quantized networks may indeed have low effective Hessian dimensionality, and the authors say they provide theoretical illustrations. The abstract gives no derivations, so I cannot tell whether those illustrations address the variance question. I would not hold the absence of error bars in an abstract against the paper, but the reported 5.0% is a single number and should be treated as such.\n\nAlso worth flagging: the authors claim theoretical and empirical illustrations that quantized models degrade more under shift, but no math appears in the abstract. That is fine for an abstract, but a referee should demand the theory be either formal or clearly scoped.\n\nOverall, this is a plausible and well-motivated idea, and the citation pattern looks standard. If the full text contains variance analysis or a convincing argument for low effective dimensionality, this deserves publication. If not, the central claim is likely setup-specific. I would send it to review and ask the authors to add per-domain results and an analysis of gradient estimate quality. From the abstract alone, I would not cite it yet.","headline":"Plausible, well-motivated idea for quantized TTA, but the two-forward-pass zeroth-order estimate needs variance evidence before the 5% claim can be trusted.","tokens_in":1418,"tokens_out":1733,"would_cite":false,"duration_ms":21571,"reading_group":"maybe","serious_thinker":"unclear","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Quantized models can adapt at test time with only two forward passes, no backpropagation, and the paper reports a 5.0% gain over first-order adaptation on ImageNet-C.","keywords":["test-time adaptation","quantized neural networks","zeroth-order optimization","domain shift","ImageNet-C","continual learning","vision transformers","low-bit quantization"],"falsifier":"Run ZOA on a quantized architecture and corruption benchmark not reported in the paper—for example, a 4-bit MobileNet on ImageNet-C—and compare test accuracy against the same model with no adaptation; if the two-forward-pass updates produce little or no gain, or degrade accuracy, the central claim that ZOA eliminates the computational burden while improving robustness would be falsified.","tokens_in":655,"feed_emoji":"⚡","tokens_out":4883,"duration_ms":54318,"temperature":0.7,"pith_summary":"Quantized neural networks run fast but degrade badly when deployment data shifts away from training data, and the usual remedy—adapting the model on test data with backpropagation—is often unavailable to them because gradients vanish and the memory and latency costs are prohibitive. This paper proposes a continual zeroth-order adaptation (ZOA) framework that updates a quantized model from just two forward passes, with no backpropagation. It adds a domain knowledge management scheme that stores and reuses previously seen domain information at negligible memory cost, so adaptation can accumulate over long deployments. On a quantized W6A6 ViT-B model, the paper reports a 5.0% improvement over the prior first-order adaptation method (FOA) on ImageNet-C. If correct, ZOA gives quantized models a practical way to keep their accuracy in changing environments without sacrificing the speed that quantization is meant to provide.","feed_headline":"Two forward passes adapt quantized models at test time","feed_subtitle":"A gradient-free update needs no backpropagation, so low-bit models can keep learning in changing environments—and gain 5% on ImageNet-C.","key_machinery":"The core mechanism is zeroth-order (gradient-free) adaptation: instead of backpropagating through the quantized network, the method estimates the parameter update from two forward passes, typically by finite differences around the current weights, and uses that estimate to adapt. The companion mechanism is a domain knowledge management scheme that keeps a small memory of domain-specific knowledge during continual adaptation, letting the model reuse what it has learned about one shift when a similar shift reappears and preventing different domains from interfering. The two forward passes are the load-bearing element: they are what remove the backpropagation, memory, and latency costs that make existing test-time adaptation methods impractical for quantized models.","core_discovery":"The paper's central claim is that test-time adaptation for quantized neural networks can be made both possible and cheap by replacing gradient backpropagation with zeroth-order estimates computed from two forward passes. The authors argue, with theoretical and empirical illustrations, that quantized models suffer more from domain shift than full-precision models, and that existing adaptation methods rely on backpropagation, which is unsupported on quantized models because of vanishing gradients and memory or latency constraints. Their ZOA framework continually adapts parameters using only forward passes, and a domain knowledge management scheme stores and reuses domain-specific information with negligible memory overhead to reduce interference between different domains and support long-term knowledge accumulation. The reported result is a 5.0% improvement over FOA on ImageNet-C for the quantized W6A6 ViT-B model, with the method demonstrated on both transformer-based and CNN-based quantized architectures.","pith_inferences":["A consequence the authors do not state is that the per-step adaptation cost becomes essentially independent of network depth and width if it truly relies on two forward passes only, so adapting a much larger quantized model may cost the same as adapting a small one.","The same two-forward-pass recipe could transfer to other gradient-starved settings, such as binary neural networks or fully quantized training, where backpropagation is likewise problematic.","A testable extension would be to pair the zeroth-order update with a different surrogate objective, such as entropy minimization or consistency regularization, to see whether the reported gain comes from the update rule or from the domain knowledge memory.","If the domain knowledge store is keyed by the kind of corruption, the scheme resembles a lightweight continual-learning memory; measuring whether it also prevents catastrophic forgetting on a long sequence of many shifts would be a natural next check."],"forward_implications":["Quantized models deployed in changing environments—such as a self-driving perception stack moving from daylight to rain—could adapt continuously without the memory and latency penalty of backpropagation.","Because the update costs only two forward passes, adaptation becomes feasible on edge and real-time systems where full-precision test-time adaptation is not.","The domain knowledge management scheme suggests that long-running deployments can accumulate adaptation gains rather than forgetting earlier shifts.","Since the method does not require gradients, it applies to both transformer-based and CNN-based quantized models, as demonstrated in the paper.","The reported 5.0% gain over FOA on W6A6 ViT-B indicates that zeroth-order adaptation can be a stronger choice than backpropagation-based adaptation specifically for quantized models."],"supporting_citations":[],"fun_headline_variants":["Two forward passes replace backprop in test-time adaptation","Quantized models adapt at test time without gradients","Zeroth-order update lets quantized models adapt with two passes","No backprop: test-time adaptation for quantized models","Test-time adaptation for quantized nets with zero gradients and two passes"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method depends on the assumption that a gradient direction estimated from just two forward passes is accurate enough, in the high-dimensional parameter space of a quantized network, to actually improve test accuracy under domain shift.","fun_headline_variants_meta":{"raw":{"variants":["Two forward passes replace backprop in test-time adaptation","Quantized models adapt at test time without gradients","Zeroth-order update lets quantized models adapt with two passes","No backprop: test-time adaptation for quantized models","Test-time adaptation for quantized nets with zero gradients and two passes"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000771,"raw_usage":{"total_tokens":3453,"prompt_tokens":1023,"completion_tokens":2430,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":639,"completion_tokens_details":{"reasoning_tokens":2348}},"tokens_in":639,"tokens_out":2430,"duration_ms":20683,"temperature":1.0,"reasoning_tokens":2348,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T05:05:16.250168+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run ZOA on a quantized architecture and corruption benchmark not reported in the paper—for example, a 4-bit MobileNet on ImageNet-C—and compare test accuracy against the same model with no adaptation; if the two-forward-pass updates produce little or no gain, or degrade accuracy, the central claim that ZOA eliminates the computational burden while improving robustness would be falsified.","supporting_citations":[],"review_version":1}