{"id":"efc3e067-b7c7-46ea-9893-dccddb86ffad","arxiv_id":"2506.17249","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A null-space projection score, appended as a virtual UNK logit, improves the speed-accuracy trade-off of early exiting on GLUE.","lead":"This paper proposes a new early-exit criterion for BERT-style language models that combines class logits with a measure of how much of the hidden feature lies in the null space of the classifier weight matrix. The resulting CAP score is designed to reduce premature exits on hard examples.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The NSP score may be nearly constant in the 765-dimensional null space, so CAP could reduce to a logit/energy threshold; without NSP distribution statistics or a constant-l0 ablation, the reported speed-up gains are not tied to the null-space mechanism.","rationale":"The reader's weakest assumption is that the null-space norm is a valid inverse certainty proxy. I agree but sharpen it: even granting the semantic premise, the score may be numerically degenerate. With N=768 and C=2/3, the classifier null space has dimension at least 765; for any representation not strongly aligned with the few class directions, NSP is close to 1. Because Eq. (10) uses softmax, a near-constant l0 is equivalent to an additive shift of the original logits: CAP = sigma(const - LSE(l)), which is just an energy-style score. Then the headline 2.19x speed-up and the 28% improvement over ConsistentEE could be an artifact of per-task alpha selection and threshold grid, not the advertised mechanism. The DIS and error-rate plots are helpful, but they compare whole scores and do not isolate the NSP contribution; no variance or significance is reported either. The proposed distribution statistics and constant-l0 ablation would settle this. This does not move the reader's CONDITIONAL verdict: the paper is plausible and the code is released, but the central mechanism needs this verification before the claims are fully trusted.","tokens_in":12560,"tokens_out":8827,"duration_ms":97915,"concrete_test":"Run the released code on SST-2 with the paper's trained multi-exit BERT, and on the dev set compute at each exit layer: (i) mean/std/min/max of NSP and of alpha*NSP; compare the std of alpha*NSP to the std of the log-sum-exp of the original logits; (ii) an ablation in which l0 is replaced by the per-layer mean of alpha*NSP (a constant), re-sweeping threshold tau over the same grid and plotting accuracy vs speed-up. If the constant-ablation curve lies within noise of the full CAP curve (or if the NSP std is negligible relative to logit LSE std), the null-space component is not the source of the reported gains.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.3 defines NSP(x)=||x_W_perp||/||x|| and Section 4.2 inserts l0=alpha*NSP(x) as an extra softmax logit, yielding CAP in Eq. (10). Everything that makes the method novel rests on NSP carrying sample-dependent, class-irrelevant certainty information. But for a BERT hidden state (N=768) and C<=3, the null space of the classifier weight matrix has dimension at least 765. For a generic feature whose energy is not almost entirely confined to the C class directions, ||x_W||^2/||x||^2 ~ C/N, so NSP is close to 1; even if features are anisotropic, no evidence is given that the remaining variation is large enough to affect softmax probabilities relative to logits. If NSP is effectively a per-layer constant, CAP(x)=sigma(alpha*NSP(x)-LSE(l)) becomes a monotone function of the log-sum-exp of the original logits (similar to an energy score), and the reported advantage over E-LANG etc. could come from per-task alpha and tau selection rather than from a null-space certainty signal. The paper reports DIS and error-rate analyses, but no NSP distribution (mean/std/range per layer) and no ablation with a constant UNK logit, so the central mechanism is not isolated.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a new early-exiting signal, the Certainty-Aware Probability (CAP) score, which augments the standard classifier logits with a scaled null-space-projection (NSP) score treated as a logit for a virtual 'UNK' class. The NSP score is defined as the norm of the feature component orthogonal to the classifier weight column space divided by the feature norm. The authors claim that previous logit-based signals overestimate certainty because they ignore class-irrelevant information, and that CAP yields more reliable exiting decisions. On GLUE, with BERT-base and ALBERT-base, the method is reported to achieve an average 2.19x speed-up over the backbone with negligible accuracy loss, outperforming ConsistentEE by 28% in speed, with additional DIS and error-rate analyses.","tokens_in":12865,"tokens_out":5974,"duration_ms":59189,"significance":"If the proposed mechanism is real, this is a useful plug-in heuristic: it adds negligible per-layer cost and does not require training the exiting signal. The paper is commendable for releasing code, reporting computational cost tables, and including DIS and premature/delayed exit-rate analyses. However, the central novelty is the claim that the null-space component carries sample-dependent certainty information; this claim is not yet empirically isolated. Because the method's advantage could alternatively stem from per-task hyperparameter tuning or from a monotone transform of existing logit-energy scores, the current evidence is suggestive rather than conclusive.","major_comments":[{"comment":"The load-bearing assumption that the NSP score varies meaningfully across samples is not tested. For BERT hidden states with N=768 and C≤3, the classifier weight matrix has a null space of dimension at least 765; if feature energy is not strongly concentrated on the C class directions, NSP(x) is close to 1 and nearly constant. In that case CAP(x) becomes a monotone function of the log-sum-exp of the original logits (roughly a shifted energy score), and the reported gains could come from per-task α and τ tuning rather than from a null-space certainty signal. The paper reports no NSP distribution statistics (mean, standard deviation, range per layer) and no ablation that replaces the sample-dependent NSP with a constant logit. Please add these analyses to tie the observed speed-up to the proposed mechanism.","section":"§3.3, Eq. (3); §4.2, Eq. (10)"},{"comment":"The comparison against baselines is not controlled. Baseline numbers mix results taken from other papers (†, ∗) and from the authors' own implementation (‡), and the per-task values of α and the exiting threshold τ are not reported. The claim of a 28% average speed-up over ConsistentEE at 'negligible performance degradation' requires matched operating points and a common implementation environment; otherwise the advantage may reflect implementation or tuning differences. Please report α and τ for each task and rerun all compared baselines in the same codebase, or at least show that the reported points lie on the same speed-accuracy trade-off curve.","section":"§5.3-5.4, Table 2"},{"comment":"The statement that x_W⊥ is 'closely related to prediction certainty' and that a larger component 'interferes with classification and reduces prediction certainty' is an assertion, not a consequence of the orthogonal decomposition. The orthogonality W^T x_W⊥ = 0 only shows that the classifier is blind to that component; it does not establish that the component is detrimental or that its relative norm is an inverse certainty measure. The paper should validate this premise directly, for example by showing that NSP adds predictive information for the exit decision beyond the log-sum-exp of the original logits (e.g., through a partial-correlation or conditional-DIS analysis).","section":"§3.3"}],"minor_comments":[{"comment":"The protocol for setting thresholds so that all methods operate at the same 4.00× speed-up ratio is not described; without this detail, the premature/delayed error-rate comparison is hard to interpret.","section":"Figure 3"},{"comment":"The DIS analysis is reported only on SST-2 and QNLI; the text claims that CAP 'consistently' outperforms baselines across different layers, which would be better supported by reporting DIS on all six GLUE tasks.","section":"Table 3"},{"comment":"The choice to exclude FastBERT and other strong baselines is not justified in the paper; a sentence explaining the selection criteria would help the reader assess the completeness of the comparison.","section":"§5.2"},{"comment":"The speed-up ratio is computed from the number of saved layers, not from measured wall-clock time; Table 5 provides an overhead check, but a statement that the two measures are consistent on the full GLUE suite would strengthen the efficiency claim.","section":"§5.3, Eq. (11)"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope for the journal, and the code release plus multi-faceted evaluation are positives. The key issue is that the central mechanism—sample-dependent variation in the null-space projection—is not demonstrated; the stress-test concern about NSP being nearly constant is well-founded and should be addressed by distribution statistics and a constant-l0 ablation. If those analyses are provided and show meaningful variation, the paper could be a solid contribution. I would not reject on the current evidence, but the central claim needs the proposed empirical support."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a modest but genuine contribution to early exiting, and the central idea is new enough to merit a referee's time. The CAP score, built from an NSP score and a virtual UNK logit, is a clean construction, and the paper does the right diagnostic work: DIS analysis and premature/delayed exit rates both favor CAP, and the computational overhead is tiny. I'd believe the reported gains are real, but not yet that the null-space mechanism is what drives them.\n\nThe paper's novelty is the specific use of the null-space projection of the hidden feature relative to the classifier weight matrix as a certainty signal. That is not present in the cited prior work. The geometric motivation is coherent, and the authors are careful to define the NSP score and the softmax extension. The empirical analyses in Section 6 are the paper's strongest asset: they show CAP tracks sample difficulty better than logit-based signals and reduces both premature and delayed exits. Code is available.\n\nSoft spots, in order. First, the headline comparison mixes results from different papers and implementations; some baselines are from original papers, some from GPFEE, some from their own implementation. That is common in this literature but it does weaken the 28%-over-ConsistentEE claim. Second, α and the exit threshold τ are tuned per task, and the chosen values are not reported; no variance or significance is given, so the speed-up numbers are point estimates. Third, and most important for the mechanism: the paper never shows the distribution of NSP scores, nor does it ablate with a constant UNK logit. If NSP were near-constant, CAP would collapse to a monotone function of log-sum-exp, i.e. an energy score with a shifted threshold. The DIS results suggest NSP is not constant, because CAP beats E-LANG on DIS, but a constant-l0 ablation would settle it. This is a missing ablation, not a demonstrated flaw.\n\nThe circularity burden is low. The method is not fit to the target; only α and τ are tuned, which are standard. Self-citations are to the authors' own prior early-exiting papers and are not load-bearing.\n\nWho this is for: anyone working on early exiting or confidence estimation for transformer inference. It deserves a serious referee, with a request for the constant-l0 ablation and per-task hyperparameter reporting.","headline":"A genuinely new exiting signal based on null-space projection; the mechanism needs a constant-l0 ablation to rule out a re-parameterized energy score.","tokens_in":13364,"tokens_out":2300,"would_cite":true,"duration_ms":23847,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a null-space-derived 'unknown class' logit yields more reliable early-exit decisions, reaching a 2.19x average speed-up on GLUE with negligible accuracy loss.","keywords":["early exiting","null space projection","prediction certainty","BERT","GLUE","inference acceleration","softmax confidence","class-irrelevant information"],"falsifier":"On a fixed multi-exit BERT, compute the area under the ROC curve of the NSP score, the softmax score, and the CAP score for the binary task of predicting whether an early-layer prediction is correct; if NSP and CAP do not beat the softmax score on most GLUE tasks, the claim that class-irrelevant information improves certainty estimation is falsified.","tokens_in":12354,"feed_emoji":"⚡","tokens_out":8216,"duration_ms":73339,"temperature":0.7,"pith_summary":"Existing early-exiting systems for pre-trained language models decide when to stop inference based on logit-derived confidence, which the authors argue overestimates certainty because it ignores class-irrelevant information in the hidden features. The paper defines an NSP score---the fraction of a feature's norm lying in the null space of the classifier's weight matrix---and appends a scaled version of it to the logits as a virtual 'unknown' class. The resulting Certainty-Aware Probability (CAP) score, the softmax probability of that unknown class, is used as the exiting signal. On GLUE the method achieves an average 2.19x speed-up with negligible performance degradation, beating the previous state of the art by 28%.","feed_headline":"Null-space trick cuts premature exits, speeds BERT 2.19x on GLUE","feed_subtitle":"A null-space 'unknown class' logit makes early-exit confidence more reliable, beating the previous best by 28%.","key_machinery":"The central object is the orthogonal decomposition of the hidden feature with respect to the classifier's weight matrix: $x = x_W + x_{W^\\perp}$, where $W$ is the column space and $W^\\perp$ the null space of $W$. The NSP score, $\\mathrm{NSP}(x) = \\|x_{W^\\perp}\\| / \\|x\\|$, measures the proportion of class-irrelevant information; geometrically it is the cosine of the angle between the feature and its null-space projection. Scaled by $\\alpha$, this scalar becomes a logit for a constructed UNK class, and the softmax probability of that class, $p_0$, is the CAP score used as the exiting signal. The mechanism turns the feature component the classifier cannot see into a measured 'unknownness' that tempers overconfident logit-based exits.","core_discovery":"The central claim is that prediction certainty is better estimated by combining class-relevant logits with the proportion of class-irrelevant information in the feature, measured by the NSP score. Formally, with classifier weight matrix $W$, the feature $x$ decomposes as $x = x_W + x_{W^\\perp}$, and the NSP score is $\\mathrm{NSP}(x) = \\|x_{W^\\perp}\\| / \\|x\\|$. The CAP score is $p_0 = e^{\\alpha\\, \\mathrm{NSP}(x)} / \\bigl(\\sum_{i=1}^{C} e^{l_i} + e^{\\alpha\\, \\mathrm{NSP}(x)}\\bigr)$, the softmax probability of a virtual UNK class. The paper claims this score yields more reliable exiting decisions, reducing premature exits while keeping delayed exits low, and that this reliability, not just higher confidence, explains the improved speed-accuracy trade-off.","pith_inferences":["Because the NSP score depends only on the trained classifier's weight matrix, it could be computed post hoc on any multi-exit network; the paper's experiments use jointly fine-tuned internal classifiers, so a fully post-hoc application is untested.","The per-task tuning of $\\alpha$ and $\\tau$ leaves open whether the 28% gain over ConsistentEE would survive a fixed-hyperparameter comparison across tasks.","The virtual UNK class gives CAP a natural interpretation as an open-set or out-of-distribution score, a test the paper does not run.","The orthogonal decomposition is defined for any linear classifier, so the CAP construction could transfer to vision or other architectures beyond BERT-style PLMs."],"forward_implications":["At roughly 2.00x speed-up, the method holds GLUE accuracy close to BERT-base while averaging 2.19x acceleration across all six tasks.","Compared with the previous SOTA ConsistentEE, the method achieves a 28% higher speed-up at comparable accuracy.","Experiments show lower Premature Exiting Rate and comparable Delayed Exiting Rate, indicating the gain comes from avoiding exits on wrong early predictions.","The overhead is negligible: less than 1.21M FLOPs per layer and less than 0.03% additional parameters, and the method transfers to ALBERT.","The CAP score has higher Difficulty Inversion Scores than logit-based signals at most internal layers, indicating better certainty estimation."],"supporting_citations":[{"why":"Defines ConsistentEE, the learning-based SOTA baseline whose 1.71x average speed-up the paper's 2.19x surpasses by 28%.","marker":"[Zenget al., 2024]"},{"why":"Introduces DeeBERT and the entropy-based exiting signal, the representative logit-based method the paper argues overestimates certainty.","marker":"[Xinet al., 2020]"},{"why":"Proposes Right-Tool with the softmax-score exiting signal, the direct probability-from-logits baseline that CAP extends.","marker":"[Schwartzet al., 2020]"},{"why":"Presents PABEE, the patience-based baseline that is a main comparison and source of the cross-layer consistency idea.","marker":"[Zhouet al., 2020]"},{"why":"Gives PCEE-BERT, the hybrid entropy-patience baseline used for comparison and in the DIS analysis.","marker":"[Zhanget al., 2022]"},{"why":"Provides BERT-base, the backbone model on which all main GLUE experiments are run.","marker":"[Devlinet al., 2019]"},{"why":"Defines the GLUE benchmark used for all task evaluations.","marker":"[Wanget al., 2019]"},{"why":"Presents DisentangledEE, a baseline that also tackles representation decoupling and supplies some comparison numbers.","marker":"[Jiet al., 2023]"}],"fun_headline_variants":["Null-space score boosts early exit certainty, cutting wrong exits","Certainty via null space: early exiting gets 2.19x speedup","Measure class-irrelevant info to make early exits reliable","Null-space projection improves early exit confidence, BERT 2.19x","How to know when to exit: null space adds a virtual class"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that the null-space component of a hidden feature is class-irrelevant and that its relative size is a valid inverse proxy for prediction certainty, a claim the paper asserts without proof.","fun_headline_variants_meta":{"raw":{"variants":["Null-space score boosts early exit certainty, cutting wrong exits","Certainty via null space: early exiting gets 2.19x speedup","Measure class-irrelevant info to make early exits reliable","Null-space projection improves early exit confidence, BERT 2.19x","How to know when to exit: null space adds a virtual class"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000472,"raw_usage":{"total_tokens":2361,"prompt_tokens":975,"completion_tokens":1386,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":591,"completion_tokens_details":{"reasoning_tokens":1295}},"tokens_in":591,"tokens_out":1386,"duration_ms":12274,"temperature":1.0,"reasoning_tokens":1295,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T05:43:59.315869+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a fixed multi-exit BERT, compute the area under the ROC curve of the NSP score, the softmax score, and the CAP score for the binary task of predicting whether an early-layer prediction is correct; if NSP and CAP do not beat the softmax score on most GLUE tasks, the claim that class-irrelevant information improves certainty estimation is falsified.","supporting_citations":[],"review_version":1}