{"id":"f0e1114b-0dcd-4bab-b603-b76710a397ab","arxiv_id":"2607.05727","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"low","formal_verification":"none","parameter_count":3,"one_line_summary":"SAMPLe adds dual gradient constraints (ERM alignment plus full-batch orthogonality) to SAM-style prompt learning and raises harmonic-mean base-to-new accuracy across CoOp, CoCoOp, MaPLe, TCP and CoPrompt.","lead":"SAMPLe is a plug-in optimizer that steers prompt tuning in CLIP-style vision-language models toward flatter loss minima. It improves accuracy on unseen classes and domains while keeping performance on the training classes, outperforming prior sharpness-aware optimizers.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"EMA quality of mt (Eq. 9) is unmeasured, so the claimed dual-constraint steering (Eq. 13–14) may not be what produces the HM gains.","rationale":"The reader correctly isolates the unvalidated EMA as the softest link in the argument that links the dual-objective mathematics to the empirical HM improvements. The tables themselves remain solid evidence of a practical gain, the short convergence proof is formally consistent under its stated assumptions, and no internal contradiction appears in the derivations. Consequently the CONDITIONAL verdict (empirical claim accepted, mechanistic claim and reproducibility caveats noted) does not need revision; the concrete test above would simply convert the remaining uncertainty into a measured quantity.","tokens_in":26104,"tokens_out":532,"duration_ms":18598,"concrete_test":"On a small few-shot dataset (e.g., DTD or EuroSAT) where the true full-batch gradient is cheap, recompute CoOp+SAMPLe once with the exact ∇F L in place of mt and once with the published EMA (λ=0.15). Report (i) the average cosine(mt, ∇F) trajectory and (ii) the resulting Base/New/HM. If the exact-∇F run does not improve (or even degrades) HM relative to EMA, the dual-constraint explanation is unsupported; if it matches or exceeds the EMA gains, the approximation is vindicated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central mechanistic claim is that SAMPLe’s objective (Eq. 10) simultaneously enforces ERM alignment of ∇Lp with the mini-batch gradient and orthogonality of ∇Lp to the full-batch gradient (Sec. 4.2, Eqs. 13–14), thereby balancing exploitation and exploration at every step. Both the orthogonality derivation and the convergence argument treat mt (the EMA in Eq. 9) as a sufficiently faithful proxy for ∇F L. The paper never reports the cosine similarity (or any other error metric) between mt and the true full-batch gradient on the prompt parameters, nor does it ablate exact full-batch versus EMA. Because the learnable prompt space is tiny, even modest EMA lag or bias can turn the “orthogonal exploration” term into unstructured noise; the observed HM lifts over F-SAM/SAGM could therefore arise from a different, unanalyzed effect of the extra α-term rather than from the dual-constraint geometry the authors invoke.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"The paper proposes SAMPLe, a plug-in sharpness-aware optimizer for prompt learning in VLMs. It addresses the performance–generalization trade-off of learnable prompts by a dual-objective loss (Eq. 10) that simultaneously minimizes empirical risk and a perturbed loss while enforcing (i) alignment of the perturbed gradient with the mini-batch gradient (exploitation) and (ii) orthogonality of that gradient to an EMA approximation of the full-batch gradient (exploration). The method is integrated into CoOp, CoCoOp, MaPLe, CoPrompt and TCP and evaluated on base-to-new, cross-dataset and cross-domain protocols over 11 datasets. A convergence rate of O(log T / √T) is proved under standard bounded-gradient and K-Lipschitz assumptions (Thm. 1 / Appendix 8.1). Empirically, SAMPLe raises harmonic-mean accuracy relative to SAM, F-SAM and SAGM for every backbone (Table 1) and improves transfer averages (Tables 2–3).","tokens_in":26327,"tokens_out":1197,"duration_ms":10592,"significance":"If the dual-constraint geometry is the operative mechanism, SAMPLe supplies a clean, model-agnostic optimizer that consistently improves five distinct prompt learners without architectural changes. The breadth of the experimental suite (five methods × eleven datasets × three protocols), the term-wise ablation (Appendix Table 6), the sensitivity plots for ρ and λ (Fig. 3), and the standard-rate convergence proof are genuine strengths that make the work useful to the prompt-learning community even if the precise geometric story requires further verification.","major_comments":[{"comment":"Sec. 3.1 and 4.2 (Eqs. 9, 13–14): the entire dual-objective argument treats the EMA mt as a faithful proxy for the true full-batch gradient ∇F L. No cosine-similarity, norm-error or other fidelity metric between mt and the exact full-batch gradient on the prompt parameters is reported, nor is an exact-full-batch ablation provided. Because the prompt space is tiny, modest EMA lag can turn the claimed orthogonal-exploration term into unstructured noise; the observed HM gains over F-SAM/SAGM could therefore arise from a different effect of the α-term. A short diagnostic (or exact-full-batch control) is needed to substantiate the mechanistic claim.","section":"Sec. 3.1, 4.2 (Eqs. 9, 13–14)"},{"comment":"Table 1 and Fig. 3: while average HM improvements are consistent, several individual base/new cells show non-monotonic or mixed behaviour (e.g., CoOp+SAMPLe base accuracy drops on StanfordCars relative to CoOp+FSAM; MaPLe+SAMPLe is essentially tied with MaPLe+SAGM on Food101). The paper asserts that SAMPLe “does not sacrifice one [base/new] in favour of the other,” yet the per-dataset tables contain counter-examples. A brief discussion of when the dual constraints fail to preserve base accuracy would strengthen the central claim.","section":"Table 1, Fig. 3"}],"minor_comments":[{"comment":"Fig. 1 caption and surrounding text claim that SAMPLe reaches both flatter minima and lower empirical risk; the visualisation is qualitative only. A quantitative sharpness measure (e.g., largest Hessian eigenvalue or average loss in a ρ-ball) would make the landscape claim more rigorous.","section":"Fig. 1"},{"comment":"Algorithm 1 line 10 writes θ t ← \theta t − ηt \nabla L(\theta t;D) after computing the dual objective; it is unclear whether the gradient of the full dual loss or only the ERM term is used for the parameter update. Clarifying the exact gradient that is back-propagated would aid reproducibility.","section":"Algorithm 1"},{"comment":"Notation for the full-batch gradient alternates between \nabla F L, mt and \nabla LF; a single consistent symbol would improve readability.","section":"Sec. 3.1–4.2"},{"comment":"Table 3 contains a typographical error (“71,03” instead of “71.03”) for CoCoOp+SAMPLe on ImageNet.","section":"Table 3"},{"comment":"The staged-training protocol used for MaPLe and CoPrompt (Appendix 8.6) is described only briefly; stating whether the same schedule is applied to the SAM/F-SAM/SAGM baselines would remove a possible confound.","section":"Appendix 8.6"}],"recommendation":"minor_revision","confidential_remarks":"The mechanistic gap around EMA fidelity is real but fixable with a short diagnostic experiment; it does not appear to invalidate the empirical gains. The paper is a solid systems contribution for the prompt-learning community and fits a CV/ML journal after the requested clarifications."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The useful takeaway is that SAMPLe is a drop-in sharpness-aware optimizer that lifts harmonic mean of base/new accuracy for every prompt learner they tried (CoOp, CoCoOp, MaPLe, CoPrompt, TCP) on the usual 11-dataset suite, plus cross-dataset and ImageNet-variant domain shifts. Gains are modest but consistent (roughly +0.3–1.2 HM over the best prior SAM variant), and they keep base accuracy from collapsing the way pure SAM sometimes does.\n\nWhat is actually new is the dual-objective (Eq. 10): minimize loss at the current point and at a SAM-perturbed point while forcing the perturbed gradient to stay aligned with the mini-batch gradient (exploitation) and orthogonal to the full-batch direction (exploration). They derive the cancellation that leaves only the batch-specific component (Eqs. 13–14) and give a short convergence proof under standard bounded-gradient + K-Lipschitz assumptions at the usual O(log T / √T) rate. That is a legitimate technical step past SAM, F-SAM and SAGM, and it removes the two hand-tuned βs that GCSCoOP still needs. The term-wise ablation and ρ/λ sensitivity plots are clean; the tables are extensive.\n\nThe soft spot is real but not fatal. The whole orthogonality argument treats the EMA mt (Eq. 9) as a faithful proxy for the true full-batch gradient. They never report cosine error of mt versus the actual full gradient on the (tiny) prompt parameters, nor do they ablate exact full-batch versus EMA. In a low-dimensional prompt space that lag could turn the “exploration” term into unstructured noise, so the HM lifts might come from the extra α term rather than the dual-constraint geometry they advertise. Missing error bars and no public code are ordinary for the venue; they do not erase the empirical pattern.\n\nThis is for people who already run CoOp-style prompt learning and want a better optimizer without redesigning the architecture. The math is standard first-order SAM analysis, the citation pattern is fair, and the data are broad enough that a serious referee should see it. I would send it to review; release of code and a quick EMA-error plot would make the mechanistic claim much tighter. Worth engaging if you care about PEFT optimizers.","headline":"Solid plug-in SAM optimizer for VLM prompts with real HM gains across five methods; the dual-constraint story is a clean increment, but the EMA that underpins the geometry is never checked.","tokens_in":27010,"tokens_out":592,"would_cite":true,"duration_ms":7700,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"A plug-in optimizer balances flatness and low loss so that learnable prompts keep both seen-class accuracy and unseen-class generalization.","keywords":["prompt learning","sharpness-aware minimization","vision-language models","generalization","CLIP","base-to-new","optimizer"],"falsifier":"Replace the EMA full-gradient estimate with the true full-batch gradient (or measure its cosine error on the prompt parameters) and check whether the reported gains in harmonic mean and domain-shift accuracy disappear or reverse.","tokens_in":26988,"feed_emoji":"⚖️","tokens_out":597,"duration_ms":7244,"temperature":0.7,"pith_summary":"Prompt learning lets a frozen vision-language model adapt to a new task by updating only a few token vectors. Those few parameters easily overfit the training distribution and land in sharp minima that fail on new classes. This paper claims that a carefully designed sharpness-aware optimizer, SAMPLe, can force every update to satisfy two simultaneous constraints: stay aligned with ordinary empirical-risk minimization (so training loss stays low) and stay orthogonal to the full-batch gradient direction (so the optimizer keeps exploring flatter regions). The dual objective is realized by a single first-order update that subtracts a scaled batch-specific component from the usual SAM perturbation. When the same optimizer is dropped into five existing prompt learners, base-to-new harmonic mean, cross-dataset transfer, and domain-shift accuracy all rise without any change to the model architecture.","feed_headline":"One optimizer lifts every prompt learner’s unseen-class score","feed_subtitle":"SAMPLe forces each step to stay low-loss and flat, raising harmonic means across five methods.","key_machinery":"The dual-objective update of Eq. 10–14: SAM-style ascent to a nearby point, followed by a correction that subtracts the projection onto the full-batch gradient so the final step is forced toward the batch-specific (orthogonal) direction while remaining aligned with the current mini-batch gradient.","core_discovery":"SAMPLe improves generalization of prompt learning by solving, at every step, a dual-objective loss that jointly minimizes empirical risk and enforces orthogonality of the perturbed gradient to the full-batch direction; the resulting flatter, still-low-loss minima transfer better to unseen classes and domains than ordinary SAM, F-SAM or SAGM.","pith_inferences":[],"forward_implications":[],"fun_headline_variants":["SAMPLe flattens prompt loss for stronger unseen-class transfer","Sharpness-aware steps improve prompt generalization across five methods","Dual-objective optimizer yields flatter minima that transfer better","Plug-in SAM variant solves prompt performance-generalization dilemma","SAMPLe keeps every prompt step low-loss and flat for unseen data"],"cache_read_input_tokens":16512,"weakest_assumption_plain":"The method assumes that a cheap exponential moving average of mini-batch gradients is a faithful enough stand-in for the true full-batch gradient that the orthogonality constraint actually steers the optimizer into flatter regions rather than merely adding noise.","fun_headline_variants_meta":{"raw":{"variants":["SAMPLe flattens prompt loss for stronger unseen-class transfer","Sharpness-aware steps improve prompt generalization across five methods","Dual-objective optimizer yields flatter minima that transfer better","Plug-in SAM variant solves prompt performance-generalization dilemma","SAMPLe keeps every prompt step low-loss and flat for unseen data"]},"model":"grok-4.5","effort":"low","cost_usd":0.007954,"raw_usage":{"total_tokens":1865,"prompt_tokens":809,"num_sources_used":0,"completion_tokens":90,"cost_in_usd_ticks":79540000,"prompt_tokens_details":{"text_tokens":809,"audio_tokens":0,"image_tokens":0,"cached_tokens":0},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":966,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":809,"tokens_out":90,"duration_ms":8914,"temperature":1.0,"reasoning_tokens":966,"cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-11T02:58:34.621815+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"Replace the EMA full-gradient estimate with the true full-batch gradient (or measure its cosine error on the prompt parameters) and check whether the reported gains in harmonic mean and domain-shift accuracy disappear or reverse.","supporting_citations":[],"review_version":1}