{"id":"8ffbb67c-7323-4419-b081-e7c85e22b181","arxiv_id":"2601.07556","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A backpropagation-free test-time adaptation method, BFT, weights predictions from multiple transformed copies of each EEG test sample using a learning-to-rank module, improving cross-subject decoding without model updates.","lead":"This paper introduces BFT, a test-time adaptation method for EEG-based brain-computer interfaces that improves predictions without backpropagation by combining several transformed versions of each test sample with weights learned by a ranking network. It matters because it could let lightweight, calibration-free BCI decoders run on low-power edge devices while handling both classification and regression.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The learned reliability weights from the source-trained ranking module may not transfer to target subjects; if they misorder transformations, BFT's weighted aggregation—its core contribution—could underperform uniform averaging. Only aggregate accuracy is reported, with no per-subject ranker calibra","rationale":"The reader's weakest assumption identified the same load-bearing premise: test-time transformations are label-preserving on the target and the source-trained ranking module continues to assign reliable weights under domain shift. I agree this is the central risk to BFT's claimed contribution. The paper provides aggregate accuracy tables (Tables II–IV) that show BFT-A/D slightly outperform uniform aggregation (Aug-Mean / MC Dropout) and roughly match T-TIME, but these are averaged over subjects and repeated seeds; the margins are small (often 0.5–1.5 percentage points) and standard deviations overlap. The only direct target-domain evaluation of the ranking module is Fig. 8(b) for one subject, which is not sufficient to establish that the learned weights generalize. The theoretical theorem is a valid variance bound in isolation, but its application to BFT-A is questionable because the weights are functions of the stochastic transformations, and Theorem 2's condition involves the unobserved κ and ρmax. The concrete test I propose would settle the concern by comparing BFT's learned weights to uniform and oracle weights on every held-out subject; this isolates whether the learned ranker adds value beyond temperature sharpening and whether it transfers. If the test shows learned weights are no better than uniform, the paper's core novelty—learning-to-rank for test-time transformation weighting—is not supported, and the verdict should remain conditional on adding such validation. I do not see grounds for rejection: the method is clearly described, code is available, and the basic idea of aggregating forward passes is sound; the issue is evidential support for the ranking mechanism.","tokens_in":20059,"tokens_out":8391,"duration_ms":92912,"concrete_test":"In the same leave-one-subject-out protocol, for every held-out subject, compute an oracle reliability score for each of the K transformations using that subject's ground-truth labels (e.g., the pretrained model's task loss on transformed trials). Then compare three aggregation schemes on the test trials: (a) BFT's source-trained learned weights, (b) uniform weights (Aug-Mean with the same temperature τ=0.5 used in BFT), and (c) oracle weights derived from the target subject's labels. If the accuracy of (a) is significantly closer to (b) than to (c), or if the per-subject NDCG between learned and oracle rankings is near 0.5 on subjects where BFT loses to Aug-Mean, the learned ranker does not transfer and the core weighting mechanism is unsupported. Report paired tests across subjects and per-subject deltas.","verdict_should_be":"UNCHANGED","load_bearing_attack":"BFT's central mechanism is to aggregate predictions over label-preserving test-time transformations using weights produced by a ranking module r(·), trained on source-domain task losses (Section III-C, Eq. 6). The theoretical justification (Theorem 2) implicitly assumes these weights reflect true branch reliability and that transformations are label-preserving on the target (Definition 1). In a leave-one-subject-out deployment, neither condition is verified: (i) a transformation that was reliable on the source may be unreliable for a given target subject (e.g., frequency shift could move discriminative mu/beta-band content), and (ii) r(·) has never seen target features, so it may assign high weight to branches that are actually poor on the target. The only target-domain ranking evidence is Fig. 8(b), a single subject (Driving S1) with NDCG@6 = 0.611 vs. 0.549 for the mapping module—modest, not replicated across subjects or datasets. The main tables aggregate all subjects, so a subject where the ranker fails can be masked by others. If the ranker misorders transformations, BFT's weighted aggregation can be worse than simple uniform Aug-Mean, undercutting the claim that the learning-to-rank module 'enhances the weighting' and that BFT is 'much more practical' than simpler backpropagation-free ensembles. This is load-bearing because the proposed novelty over naive transformation ensembles is precisely the learned reliability weighting, and the theoretical variance-reduction guarantee depends on weights that track true branch reliability.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes BFT, a backpropagation-free test-time adaptation method for EEG-based BCIs. At inference, BFT applies multiple transformations to each test trial (input augmentations in BFT-A, or deterministic feature-dropout masks in BFT-D), obtains several forward-pass predictions, and aggregates them using reliability weights produced by a learning-to-rank module. The ranking module is trained on source-domain task losses, with an auxiliary mapping module that converts soft scores into rank-like values. A variance-decomposition argument (Theorems 1 and 2) is given as theoretical justification. Experiments on three motor-imagery classification datasets and two driver-drowsiness regression datasets under leave-one-subject-out evaluation report accuracy/CC/RMSE improvements over backpropagation-free baselines, competitiveness with backpropagation-based TTA, robustness to simulated test-time noise, and compatibility with 8-bit quantization.","tokens_in":20493,"tokens_out":5760,"duration_ms":62494,"significance":"If the empirical claims hold, BFT is a useful practical contribution: it is black-box, forward-only, task-agnostic, and works for both classification and regression, addressing a real deployment gap for lightweight EEG BCIs. The evaluation is credible in important respects: held-out subjects, public benchmark datasets, multiple random seeds, released code, and direct CPU/GPU latency measurements. The variance decomposition is mathematically correct but standard and does not specifically justify the learned weighting; the novelty is mainly empirical. The paper's central claim that BFT is 'much more practical' than current TTA approaches is plausible but needs stronger evidence that the learned ranking, rather than simple averaging, transfers to target subjects.","major_comments":[{"comment":"The theorem bounds Var(ˆf_w) for arbitrary probability weights and shows that any non-degenerate averaging reduces variance relative to single-shot inference under Assumptions 1–2. It does not show that the learned reliability weights from r(·) are better than uniform weights, nor that reducing this variance improves classification/regression risk. The statement in §IV-D that the learning-to-rank module 'suppresses unreliable branches' and 'helps uncertainty reduction' is therefore not a logical consequence of the theorem. Please add a target-domain analysis comparing learned weighted aggregation to uniform Aug-Mean/MC Dropout in terms of per-subject risk and effective Keff; if the difference is not significant, temper the claim.","section":"§IV-D, Theorem 2 (Eqs. 19–21)"},{"comment":"The only target-domain evaluation of the ranking module is one subject (Driving S1) with NDCG@6 = 0.611, and the classification panel in Fig. 8(a) is also a single subject. Since the contribution over naive transformation ensembles is precisely the learned reliability weighting, per-subject calibration of r(·) on the target is load-bearing. Aggregate accuracies in Tables II–IV can mask subjects where the ranker misorders transformations. Please report per-subject NDCG or rank correlation for all subjects/datasets, and a statistical comparison (e.g., paired test across seeds) of BFT against uniform Aug-Mean and MC Dropout. Without this, the claim that the ranking module 'enhances weighting' is not established.","section":"§V-G, Fig. 8(b)"},{"comment":"The theory assumes label-preserving test-time randomization, but this is not verified for the target domain. For BFT-A, transformations such as frequency shift and sliding windows are plausible for motor imagery, yet a subject whose discriminative rhythm lies in the shifted band could receive misleading branches. The variance decomposition does not capture the resulting bias error. Please either verify label preservation quantitatively on held-out target data (e.g., by comparing branch predictions to oracle labels during evaluation), or weaken the theoretical claim to prediction-variance reduction under the stated assumption.","section":"§III-B / §IV-A, Definition 1"}],"minor_comments":[{"comment":"The text says K=12 for BFT-A, but the enumeration (Identity, 3 Scale factors, Noise, Freq, 5 Slide windows) sums to 11. Please reconcile the count or clarify which transformation is missing.","section":"§V-C"},{"comment":"f is defined as a scalar E[y|x; ζ], but classification outputs are softmax vectors. State whether the variance is computed on a scalar summary (e.g., confidence) or per logit/probability, since this affects the interpretation of V0.","section":"§IV-A, Eq. (9)"},{"comment":"Clarify the domain and codomain of the mapping module m(·): the text says it maps softmax-normalized losses in [0,1] to rank-like values in [1,K], but Eq. (6) applies m to the weight vector w_i. Please make the input/output spaces explicit.","section":"§III-C, Eq. (6)"},{"comment":"T-TIME is reported as N/A under quantized CPU. State explicitly why it is not applicable (e.g., backpropagation through quantized weights) and whether this is part of the practicality argument.","section":"§V-H, Table VI"},{"comment":"Some labels in Figures 2 and 3 appear garbled in the PDF. Ensure the vector graphics are readable in the final version.","section":"Figures 2–3"}],"recommendation":"major_revision","confidential_remarks":"The work is reproducible and the empirical protocol is solid; I see no evidence of circularity or data leakage. The main risk is that the learned ranking module may not transfer across subjects, and the paper's own evidence for this is limited to one subject. The requested per-subject and significance analyses are feasible within the manuscript's scope and would substantially strengthen the central claim. The theoretical section should be reframed as a standard variance-reduction justification for averaging, not as a justification for the learned weights."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a solid applied paper. The BFT idea—aggregate over test-time transformations with weights from a source-trained ranking module, no backprop—is genuinely new in the EEG TTA space, and they show it works on five datasets, across classification and regression, and even under quantization. That's a real contribution for anyone deploying BCIs on edge hardware.\n\nWhat I like: the method is clearly described, the ablations isolating the mapping module are honest, and the comparison set is broad, including backprop-based TTA and plain Aug-Mean. The robustness experiments with temporal and spatial noise are a nice touch. The fact that BFT-A and BFT-D beat uniform averaging on most datasets gives some empirical support for the ranking module, which is the core novelty.\n\nSoft spots: the theory in Section IV is just a standard variance decomposition. Theorem 2 shows that averaging reduces variance under assumptions on branch variances and correlations, but it doesn't say anything about the learned weights being better than uniform—it bounds variance for any weights. So the theoretical justification is overclaiming, but it isn't load-bearing; the empirical results carry the paper.\n\nThe bigger gap is the one the stress-test flags: the ranking module is trained on source-domain task losses, and the paper never shows it stays calibrated on the target subject. There's one NDCG figure for one subject, and the rest is aggregate accuracy. If the ranker misorders transformations on a specific subject, weighted aggregation could underperform uniform averaging. The aggregate results suggest this isn't happening on average, but per-subject variance is hidden. I'd want to see per-subject gains over Aug-Mean and ideally a calibration check on target branches. This is a fixable paper-level weakness, not a fatal one.\n\nAlso: no significance tests across the three seeds, and some of the reported differences (e.g., BFT-A 79.03 vs MEMO 78.19) may be within noise. A paired test would help.\n\nOverall: this paper deserves a serious referee. It's an incremental but useful advance, clearly explained, with reproducible code. It should be sent to review, with the main revision being to temper the theory and shore up the target-domain evaluation.","headline":"A practical, backprop-free TTA for EEG that mostly delivers; theory oversold and target-domain ranker calibration under-evidenced.","tokens_in":20912,"tokens_out":2308,"would_cite":true,"duration_ms":25285,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that test-time adaptation of EEG decoding models can be done without backpropagation by aggregating weighted predictions over label-preserving transformations, matching or beating gradient-based adaptation on five datasets","keywords":["test-time adaptation","EEG decoding","brain-computer interface","backpropagation-free","learning-to-rank","variance reduction","motor imagery","driver drowsiness"],"falsifier":"A concrete test: on a held-out target subject, intentionally choose one transformation that is label-destroying (for example, a frequency shift that moves the signal away from the subject's discriminative band). If BFT's accuracy then drops below the unweighted Aug-Mean baseline or below the best single transformation, the ranking module is amplifying an unreliable branch and the variance-reduction guarantee does not hold in that regime. Alternatively, empirically estimate Var(ŵf) and V0 on target trials with known labels and check whether the inequality Var(ŵf) < V0 holds; if it is violated f","tokens_in":20032,"feed_emoji":"🧠","tokens_out":3064,"duration_ms":32597,"temperature":0.7,"pith_summary":"BFT (Backpropagation-Free Transformations) applies several label-preserving transformations to each test trial, runs them through a frozen pretrained EEG decoder, and combines the resulting predictions with reliability weights learned by a ranking module. This provides calibration-free and backpropagation-free test-time adaptation for both motor-imagery classification and driver-drowsiness regression. The paper proves that this weighted aggregation reduces prediction variance, and experiments on five EEG datasets show it matches or exceeds the accuracy of backpropagation-based T-TIME while being robust to test-time noise and 8-bit quantization. The practical payoff is a lightweight, privacy-preserving, task-agnostic adaptation method suitable for plug-and-play BCIs on resource-constrained devices.","feed_headline":"Forward-pass trick adapts EEG models to new users","feed_subtitle":"Weighted averaging over label-preserving transformations matches backprop-based test-time adaptation on five EEG datasets.","key_machinery":"The central mechanism is the combination of (1) label-preserving test-time transformations—noise addition, amplitude scaling, frequency shift, sliding windows, and monotone-dropout feature masks—that produce multiple stochastic branches per trial, and (2) a learning-to-rank module r(·), trained with a mapping module m(·) that projects task losses into a rank-like space, which assigns a reliability weight to each branch. The weighted aggregation (convex combination with temperature sharpening for classification, top-half averaging for regression) is what carries the variance-reduction argument, and the ranking module is what differentiates BFT from naive averaging such as Aug-Mean or MC Dropo","core_discovery":"The paper's central claim is that prediction variance at test time can be suppressed without any model update: generate multiple predictions for a single test sample by applying structured transformations (knowledge-guided augmentations in BFT-A, or deterministic dropout-style feature masks in BFT-D), then combine them with weights from a learned ranking module. The authors prove in Theorem 2 that the aggregated variance Var(ŵf(x)) is smaller than the single-shot variance V0 whenever the effective number of branches satisfies Keff > κ(1−ρmax)/(1−κρmax), where κ bounds the worst-branch variance relative to V0 and ρmax bounds the absolute inter-branch correlation. Empirically, BFT-A reaches 79","pith_inferences":["The variance-reduction theorem is not EEG-specific: any frozen black-box model with label-preserving perturbations and a reasonable reliability ranking could benefit from this aggregation strategy, so the method likely transfers to other biosignal or non-biosignal domains.","The ranking module is trained on source-domain task losses only; on a target domain where transformations are not label-preserving or the ranker misorders branches, weighted aggregation could amplify an unreliable branch—a testable extension is to calibrate the ranker on a small labeled target-sample set.","BFT-D's deterministic feature masks provide a parameter-free alternative that requires no domain-specific augmentation design, suggesting the method can be dropped into any pretrained network that has a dropout layer.","Because the paper evaluates only a single backbone (EEGNet), a natural extension is to test BFT with transformer-based or hybrid EEG decoders, where the label-preservation assumption on transformations may behave differently."],"forward_implications":["Plug-and-play EEG BCIs can adapt to a new user's signal stream without per-use calibration, backpropagation, or access to the model's internal parameters.","The same forward-only aggregation works for regression (driver drowsiness estimation) and classification, covering tasks that most TTA methods exclude.","BFT remains effective after post-training 8-bit quantization, making it compatible with edge-device and CPU-only deployments where backpropagation-based TTA is infeasible.","The theoretical variance bound provides a formal, parameter-free justification for why test-time ensembling of transformations reduces prediction uncertainty, independent of the specific backbone architecture.","The method's robustness to temporal test-time noise, verified on all five datasets, suggests it can tolerate realistic EEG artifacts without special handling."],"fun_headline_variants":["No backprop needed: EEG models adapt on the fly","Weighted predictions adapt EEG without backprop","Test-time EEG adaptation without model updates","EEG model adapts to new users via forward passes","EEG adaptation that skips backprop and calibration"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that the transformations applied at test time preserve the task-relevant label for the target subject, and that the reliability ranking learned from source-subject task losses continues to assign trustworthy weights on the shifted target domain.","fun_headline_variants_meta":{"raw":{"variants":["No backprop needed: EEG models adapt on the fly","Weighted predictions adapt EEG without backprop","Test-time EEG adaptation without model updates","EEG model adapts to new users via forward passes","EEG adaptation that skips backprop and calibration"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000278,"raw_usage":{"total_tokens":1488,"prompt_tokens":741,"completion_tokens":747,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":485,"completion_tokens_details":{"reasoning_tokens":674}},"tokens_in":485,"tokens_out":747,"duration_ms":7250,"temperature":1.0,"reasoning_tokens":674,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T11:02:11.259408+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A concrete test: on a held-out target subject, intentionally choose one transformation that is label-destroying (for example, a frequency shift that moves the signal away from the subject's discriminative band). If BFT's accuracy then drops below the unweighted Aug-Mean baseline or below the best single transformation, the ranking module is amplifying an unreliable branch and the variance-reduction guarantee does not hold in that regime. Alternatively, empirically estimate Var(ŵf) and V0 on target trials with known labels and check whether the inequality Var(ŵf) < V0 holds; if it is violated f","supporting_citations":[],"review_version":1}