{"id":"a305992f-3238-4f64-a2ed-d8d63ac7edef","arxiv_id":"2608.06467","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":8,"one_line_summary":"EB-CaP generates per-video class prototypes using energy-based sampling from CLIP text embeddings and combines them with positive and negative target caches to improve subject-specific video facial expression recognition at test time.","lead":"This paper presents EB-CaP, a test-time adaptation method that builds a personalized cache of class prototypes for each new person's video, without retraining the model. It reports accuracy gains over existing adaptation methods on three facial expression recognition datasets with modest memory use.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The EB-CaP sampler (Eqs. 1–2) is a linear spherical interpolation toward CLIP text embeddings, so the claimed personalized class prototypes are not demonstrated; the method's core mechanism may reduce to text-guided interpolation.","rationale":"The reader's weakest_assumption identifies the same soft spot: the sampler in Eqs. (1)-(2) is assumed to generate class-conditional prototypes that remain personalized to the target subject, but no analysis supports that assumption. My stress-test makes the mechanism concern more concrete. The energy E_theta(z,c) = -z^T e_c is linear in z, so the SGLD update is simply a step toward the CLIP text embedding e_c, followed by normalization. The stopping rule 'stop when argmax becomes c' means the chain terminates at the first point on the great circle from z_tgt to e_c where CLIP's text-based classifier predicts c. Such a point is a decision-boundary crossing, not a class-conditional prototype, and it cannot recover a subject-specific class distribution beyond the initialization z_tgt. This is not an external disagreement with consensus; it is an internal mechanistic concern about whether the proposed method does what the central claim requires. The paper's own ablation in Table 6 provides partial evidence that target initialization helps, and the available code and subject-wise tables are useful, but they do not isolate whether the SGLD/EBM machinery is needed. A deterministic interpolation ablation would settle the question directly. Because this concern can be addressed with a clear experiment and the reader's verdict is already CONDITIONAL, I recommend no change to the verdict: the paper should be accepted only after the sampler's role is established or the claims are softened.","tokens_in":19997,"tokens_out":6777,"duration_ms":64753,"concrete_test":"Replace the SGLD sampler in Algorithm 1 with a closed-form spherical interpolation: for each class c, define z_c(lambda) = normalize(z_tgt + lambda * e_c) and choose the smallest lambda such that argmax_{c'} z_c(lambda)^T e_{c'} = c, searching over a fine grid of lambda values within the same 20-step budget. Keep the sampled/positive/negative cache fusion, adaptive entropy gate, diversity gate, and all hyperparameters identical, and rerun BioVid and StressID (and BAH if feasible). If the resulting WAR/F1 are within 0.5 points of Tables 1-3, the energy-based sampling and stochasticity are not load-bearing and the central personalization claim is unsupported; if performance drops by more than a few points, the SGLD dynamics are functionally important.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing assumption of EB-CaP is that the energy-based sampler in Eqs. (1)-(2) produces class-conditional prototypes that are personalized to the target subject. This is where the central claim is least secure. Eq. (1) defines E_theta(z,c) = -z^T e_c, with no dependence on the source-trained recognition model theta; the SGLD gradient in Eq. (2) is simply +e_c, so after l2 renormalization each chain is a noisy spherical interpolation from z_tgt toward the fixed CLIP text embedding e_c. Early stopping when argmax_{c'} z_k^T e_{c'} = c places the sample on the CLIP decision boundary of class c, not at a representative point of that class for the subject. Each sampled prototype therefore lies on a one-dimensional great circle determined by z_tgt and e_c (plus injected noise), and the only subject-specific information is the initialization z_tgt; no source-trained class-conditional distribution is used. The ablation in Table 6 shows target initialization beats noise initialization (81.0 vs 78.2), but that comparison does not establish that the SGLD dynamics, rather than simply retaining z_tgt or interpolating a small distance toward e_c, are responsible. Because the base CLIP score s_tgt already uses the same e_c, the sampled cache is not an independent source of evidence. If the gains on BioVid and StressID survive replacing the SGLD sampler with deterministic closed-form interpolation to the same stopping rule, then the claimed personalization mechanism is not supported, and the novelty of the method reduces to caching text-guided interpolated points plus target caches.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes EB-CaP, a test-time adaptation method for fine-grained video facial expression recognition. For each incoming unlabeled video, EB-CaP samples class-conditioned embeddings from the current video representation using an energy-based model whose energy is the negative cosine similarity to CLIP class text embeddings, and stores these in a personalized sampled cache. In parallel, positive and negative target caches accumulate reliable and uncertain embeddings under an adaptive entropy gate and a diversity gate. Final predictions fuse CLIP scores with similarities to the sampled, positive, and negative caches. Experiments on BioVid, StressID, and BAH report improved WAR and F1 over several optimization- and cache-based TTA baselines, with lower memory usage, and the authors provide subject-level results and an ablation study.","tokens_in":20383,"tokens_out":6375,"duration_ms":55778,"significance":"If the reported results and the proposed mechanism held, EB-CaP would be a practically attractive TTA method for video FER: it avoids test-time parameter updates, does not store source prototypes, and the released code and subject-level tables are useful contributions. The efficiency measurements in Table 5 and the per-subject results in Tables 2-4 are concrete and go beyond average-only reporting. However, the central mechanism is not currently established: the energy-based sampler described by Eqs. (1)-(2) is mathematically equivalent to a noisy spherical interpolation toward CLIP text embeddings, so the claim that it produces personalized class-conditional prototypes is unsupported. In addition, the main ablation compares against a zero-shot CLIP baseline rather than the actual frozen source-trained model, and the statistical evidence covers only BioVid WAR. These issues are load-bearing for the paper's central claims.","major_comments":[{"comment":"The proposed energy-based sampler does not depend on the source-trained recognition model and reduces to text-guided interpolation. In Eq. (1), E_theta(z,c) = -z^T e_c, where e_c is the fixed CLIP text embedding, so the SGLD update in Eq. (2) is z_{k+1} = z_k + (alpha/2) e_c + sqrt(alpha sigma) epsilon_k. After l2 normalization each chain is a noisy spherical interpolation from z_tgt toward e_c, and the early-stopping rule `argmax_{c'} z_k^T e_{c'} = c` simply places the sample on a CLIP decision boundary. The only subject-specific information is the initialization z_tgt; no source-trained class-conditional distribution is used. Since the base CLIP score s_tgt also uses the same e_c, the sampled cache is not an independent source of evidence. The paper should compare against a deterministic closed-form interpolation z = normalize(z_tgt + beta e_c) with the same stopping rule, and should either use a source-dependent energy function or explicitly reframe the method as text-guided interpolation. As written, the claimed personalization mechanism is not demonstrated.","section":"Personalized Sampled Cache, Eqs. (1)-(2)"},{"comment":"The ablation baseline is inconsistent with the actual base model. Figure 3 reports a 'CLIP baseline' WAR of 50.0, which is the zero-shot CLIP value in Table 1, whereas EB-CaP uses a source-trained temporal encoder whose frozen model achieves 69.7 WAR in Table 1. The correct control for the cache ablations is the frozen source-trained model (69.7), not zero-shot CLIP. Without this control, the contribution of the sampled cache (77.9) and target cache (78.6) is overstated, and the apparent gain from 50.0 to 81.0 conflates source training with test-time adaptation. Please rerun the ablation against the correct frozen baseline and report the resulting increments.","section":"Figure 3 and Table 1"},{"comment":"The claim that EB-CaP outperforms state-of-the-art TTA methods on all three datasets is not fully supported by the reported statistics. On BAH the improvement over T3AL is 1.0 WAR (68.9 vs 67.9) and 0.3 F1 (41.0 vs 40.7), and on several individual subjects the method is not best. No error bars or confidence intervals are provided, and the only significance test is a Wilcoxon signed-rank test on BioVid WAR. To support the cross-dataset claim, the authors should report significance tests (or error bars) for all three datasets and for both metrics, and should discuss the BAH case explicitly in terms of effect size.","section":"Results, 'Comparison with State-of-the-Art Methods' and Tables 1-4"},{"comment":"The main SGLD hyperparameters (step size alpha, noise scale sigma, and maximum number of iterations) are set by hand, and the provided sensitivity analysis covers only cache capacities (Figure 5) and fusion weights (supplementary Table 6). Because the sampler is the central contribution, the paper should include a sensitivity study for alpha, sigma, and the iteration limit to show that the reported gains are not artifacts of a particular SGLD configuration. This is especially important given that the sampler reduces to interpolation, where alpha and the stopping rule determine the distance traveled toward e_c.","section":"Implementation Details and Ablation Studies"}],"minor_comments":[{"comment":"The labeling of the rows is confusing: the second row appears to denote a fine-tuned CLIP model, but the footnote 'CLIP-ViT-B/32† denotes full CLIP fine-tuning' does not visually match any row in the table. Please clarify which rows are zero-shot, frozen source-trained, and fully fine-tuned.","section":"Table 1"},{"comment":"Equation (3) writes the fused score as s_fuse = s_tgt + s_s + s_p - s_n, but the supplementary sensitivity analysis uses weights lambda_s, lambda_p, lambda_n. The main text should present the weighted equation or explicitly state that all weights are set to 1.","section":"Eq. (3) and supplementary Table 6"},{"comment":"The abstract says the energy relies only on pretrained CLIP, while Eq. (1) attributes the energy to a 'frozen recognition model parameterized by theta.' Since the energy has no dependence on theta, the notation and wording should be corrected to avoid implying source-model involvement.","section":"Abstract and Eq. (1)"},{"comment":"The notation m_p and m_n is used inconsistently: it is first described as the current number of entries per class and later as a fixed capacity. Please distinguish the capacity constant from the current count.","section":"Section 'Target Caches'"},{"comment":"The reference for Sharafi et al. 2026a has a garbled author list ('Belharbi, M. O., S. Zeeshan'); please correct it.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The stress-test concern is substantiated: the equations confirm that the 'energy-based sampling' is a noisy interpolation toward CLIP text embeddings, and the ablation baseline mismatch further weakens the empirical case. I would require the closed-form interpolation baseline and the corrected frozen-model ablation before any acceptance decision. The paper may be salvageable as a cache-based TTA method with a simpler mechanism, but the current framing overclaims personalization."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the paper reports strong, reproducible-looking gains for test-time cache personalization in video FER, but the central mechanism is oversold. The \"energy-based sampler\" in Eqs. (1)-(2) is, after normalization, a noisy spherical interpolation from the target embedding toward the CLIP text embedding of class c, with early stopping at the decision boundary. That means the sampled prototypes are not demonstrated to be class-conditional in any learned sense; they are text-guided interpolates that retain some target-specific initialization. The paper never runs the control that would separate SGLD dynamics from simple interpolation: initialize at z_tgt, move a small step toward e_c, stop when argmax is c. I suspect that control would reproduce most of the gain. This is a real soft spot in a load-bearing claim.\n\nCredit where due: the overall pipeline is clean and practical. The sampled cache gives immediate class coverage, which fixes the cold-start issue; the positive/negative target caches and the adaptive entropy and diversity gates are sensible and clearly described. On BioVid and StressID the gains over T3AL are substantial and distributed across most subjects, and the complexity table shows reasonable cost. Code is available. Those are real strengths.\n\nOther soft spots, in proportion: the ablation in Fig. 3 compares against zero-shot CLIP (50.0) rather than the fine-tuned frozen backbone (69.7), which makes the sampled cache look like a 28-point gain when the honest comparison is about 8-11 points. No error bars are reported; significance is tested on BioVid WAR only. The BAH improvement is 1.0 WAR point. Hyperparameters (step size 0.01, noise 0.1, capacities) are tuned on BioVid and the sensitivity analysis is on BioVid. None of this kills the paper, but it should be tightened.\n\nWho it is for: people working on affective computing, subject-specific FER, and cache-based TTA for CLIP models. Deserves a serious referee; the empirical contribution is solid enough that the missing control experiment and corrected baseline are exactly what review should ask for. I would not desk-reject it.","headline":"Strong empirical results on video FER test-time adaptation, but the energy-based sampler is effectively text-guided interpolation and the paper's central mechanism claim needs a control experiment.","tokens_in":20957,"tokens_out":2244,"would_cite":false,"duration_ms":20367,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Frozen CLIP model personalizes video expression recognition per subject","keywords":["test-time adaptation","cache personalization","energy-based model","facial expression recognition","video understanding","CLIP","stochastic gradient Langevin dynamics","subject-specific domain shift"],"falsifier":"Use a labeled held-out set for the same target subjects. For each video, compute EB-CaP's sampled prototypes and check whether their nearest labeled frames belong to the conditioning class as often as the original video embedding's nearest frames do. If the sampled prototypes match the conditioning class less often, the sampler is drifting toward text directions rather than subject-specific evidence.","tokens_in":19831,"feed_emoji":"🎭","tokens_out":3839,"duration_ms":29504,"temperature":0.7,"pith_summary":"This paper introduces Energy-Based Cache Personalization (EB-CaP), a test-time adaptation method for fine-grained facial expression recognition in videos. EB-CaP generates, for each unlabeled target video, class-specific prototype embeddings by running stochastic gradient Langevin dynamics chains that start from the current video's CLIP embedding and are guided by the energy $E_{\\theta}(z,c)=-z^{\\top}e_c$, where $e_c$ is the CLIP text embedding of expression class $c$. These sampled prototypes populate a personalized cache that is rebuilt for every video, alongside positive and negative caches of confident and uncertain target embeddings that are maintained per subject. A fused score combines the base CLIP score with similarities from the three caches. On BioVid, StressID, and BAH the method reports the best weighted average recall and F1 among compared optimization- and cache-based test-time adaptation methods, while keeping the recognition model frozen.","feed_headline":"Frozen CLIP personalizes video expression recognition per subject","feed_subtitle":"Synthesized class prototypes from each unlabeled video beat optimization-based test-time adaptation on three benchmarks.","key_machinery":"The load-bearing object is the personalized sampled cache, built by a class-conditional energy-based sampler. The energy is the negative cosine-compatibility between an embedding and the class text embedding, $E_{\\theta}(z,c)=-z^{\\top}e_c$, and sampling is done by stochastic gradient Langevin dynamics, $z_{k+1}=z_k-\\frac{\\alpha}{2}\\nabla_z E_\\theta(z_k,c)+\\sqrt{\\alpha}\\sigma\\epsilon_k$, with each chain initialized from the current target video embedding and stopped early when the classifier's argmax is the conditioning class. This produces $m_s=3$ embeddings per class per video, replaces the usual need to accumulate target samples into class prototypes, and is re-run for every temporal window.","core_discovery":"EB-CaP claims that the cold-start problem of cache-based test-time adaptation can be solved by synthesizing class prototypes on demand rather than collecting them from observed target data. Starting each SGLD chain from the current unlabeled video representation and stopping as soon as the classifier's argmax becomes the conditioning class produces personalized, class-conditional embeddings for every class from the first video on. The paper further claims that combining these synthesized prototypes with a positive cache of reliable target embeddings and a negative cache of uncertain ones, both filtered by an adaptive entropy gate and a diversity gate, yields subject-specific adaptation that outperforms prompt-optimization methods like TPT and PromptAlign and cache methods like TDA, DPE, ReTA, and T3AL on the three benchmarks. The central contribution is that no source samples, source prototypes, target labels, or parameter updates are needed at test time.","pith_inferences":["The same energy-guided sampling idea could be applied to other fine-grained video tasks, such as action-unit detection or pain intensity estimation, where class text embeddings exist and subject-specific shifts dominate; the paper does not test this.","One unresolved question the paper leaves open is whether the sampled prototypes are truly subject-specific or merely class-text directions with noise; a direct comparison of sampled embeddings against source-prototype embeddings from the same classes would clarify the mechanism.","The early-stopping criterion (stop when the argmax becomes c) is cheap but asymmetric: classes that are already predicted for the starting video will stop almost immediately, while others run longer, and this asymmetry may bias cache diversity; a testable variant would impose a minimum chain length."],"forward_implications":["If the central claim holds, test-time adaptation for video facial expression recognition can be done without any backpropagation through the recognition model, removing the instability and latency of entropy-minimization prompt tuning.","A personalized cache covering all classes is available from the very first video of a subject, so rarely observed classes do not have to wait for target samples to accumulate.","Because only pretrained CLIP and a fixed temporal encoder are used, the method can be dropped into existing CLIP-based FER pipelines without retraining or storing source prototypes.","The reported per-subject gains on BioVid and StressID suggest that subject-level evaluation is the right lens for comparing personalization methods, since averages can hide large differences at the individual level."],"supporting_citations":[{"why":"Supplies the frozen CLIP visual and text encoders whose embeddings and logit scale define the base scores and the class text directions used in the energy function.","marker":"(Radford et al. 2021)"},{"why":"Justifies treating a classifier's logits as an energy-based model, which licenses the energy form $E(z,c)=-z^{\\top}e_c$.","marker":"(Duvenaud et al. 2020)"},{"why":"Provides the stochastic gradient Langevin dynamics sampler that generates the personalized class-conditioned prototypes.","marker":"(Welling and Teh 2011)"},{"why":"TPT is a leading optimization-based test-time adaptation baseline that EB-CaP compares against and outperforms on all three datasets.","marker":"(Shu et al. 2022)"},{"why":"TDA is the cache-based test-time adaptation baseline whose positive/negative cache design EB-CaP extends; its reported performance is a direct comparison point.","marker":"(Karmanov et al. 2024)"},{"why":"T3AL supplies the strongest competing video-based test-time adaptation baseline, adapted for video classification in the experiments.","marker":"(Liberatori et al. 2024)"},{"why":"Prior prototype-based cache method that stores source-derived class prototypes; EB-CaP is positioned as the source-free alternative and is compared against it in ablations.","marker":"(Sharafi et al. 2026b)"}],"fun_headline_variants":["On-demand prototypes let frozen CLIP adapt to each subject's expressions","Synthesized prototypes beat online tuning for video emotion recognition","Personalized expression adaptation without parameter updates or source data","Energy-based cache personalizes facial expression recognition per video","Test-time adaptation via synthesized prototypes, no tuning needed"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that pushing the target video embedding toward the CLIP class-text embedding, with early stopping at the argmax switch, produces sampled embeddings that remain representative of the target subject's actual expression rather than drifting toward a generic class direction.","fun_headline_variants_meta":{"raw":{"variants":["On-demand prototypes let frozen CLIP adapt to each subject's expressions","Synthesized prototypes beat online tuning for video emotion recognition","Personalized expression adaptation without parameter updates or source data","Energy-based cache personalizes facial expression recognition per video","Test-time adaptation via synthesized prototypes, no tuning needed"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000674,"raw_usage":{"total_tokens":3099,"prompt_tokens":1005,"completion_tokens":2094,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":621,"completion_tokens_details":{"reasoning_tokens":2014}},"tokens_in":621,"tokens_out":2094,"duration_ms":11265,"temperature":1.0,"reasoning_tokens":2014,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T14:32:51.467389+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Use a labeled held-out set for the same target subjects. For each video, compute EB-CaP's sampled prototypes and check whether their nearest labeled frames belong to the conditioning class as often as the original video embedding's nearest frames do. If the sampled prototypes match the conditioning class less often, the sampler is drifting toward text directions rather than subject-specific evidence.","supporting_citations":[],"review_version":1}