{"id":"5c1a7d74-9a32-4d90-9fc7-1afb15d56e43","arxiv_id":"2504.19605","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"NoPE (no positional encoding) yields better length extrapolation than APE, KERPLE, or RoPE in TF-domain dual-path source separation models with convolution, while explicit PEs help for short inputs.","lead":"This paper compares four positional encodings (absolute, relative, rotary, none) in a time-frequency dual-path transformer for source separation, and finds that removing positional encoding improves generalization to longer inputs and unseen sampling rates when the model contains convolution layers. Explicit encodings still help when inputs are shorter than training segments.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'particularly pronounced with convolutional layers' claim rests on a K=1 ablation that removes model capacity as well as convolution; Table II itself shows NoPE's extrapolation advantage is larger without conv, so the conv-specific mechanism is unverified.","rationale":"After reading the paper, the most defensible headline claim is that NoPE matches or beats explicit PEs on extrapolation in several settings; that part is consistently observed. The soft spot I consider most load-bearing is the conjunctive part of the abstract: that the NoPE extrapolation trend is 'particularly pronounced when the model contains convolutional layers.' The only ablation for this is K=1, which changes capacity and local modeling, and the table's own numbers point the other way. The reader's weakest assumption (single runs, no significance tests) is also legitimate and should be fixed by repeated seeds, but it is a strength-of-evidence issue rather than an internal inconsistency. The K=1 confound is internal: the evidence cannot support the stated mechanism as reported. I would keep the CONDITIONAL verdict: the practical recommendation to consider NoPE for long inputs remains supported by the conv-model results in Table I and Figures 1-2, but the paper must either remove or qualify the 'particularly pronounced' claim or provide a matched-capacity non-convolutional ablation. I also agree with the reader that repeated-seed error bars are needed before the claim is accepted.","tokens_in":8482,"tokens_out":13975,"duration_ms":149378,"concrete_test":"Retrain the K=1 and K=8 WHAMR! RoPE/NoPE models with matched capacity: replace ConvSwiGLU with a linear MLP whose width is increased to match the K=8 model's parameter count, or vary K=1,2,4,8 while rebalancing width. Recompute the Table I/II metrics on 8k/16k min/max under the same training schedule. If the NoPE-vs-RoPE 16k extrapolation gap remains ~4 dB in the capacity-matched no-conv model, the 'particularly pronounced with convolution' claim fails; if the gap shrinks to near zero and RoPE regains the matched-length lead, the original mechanism is supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central finding (ii) — that NoPE extrapolates best and that this trend is 'particularly pronounced when the model contains convolutional layers' — is supported for the conv case by Table I, but the only evidence for the conv-specific part is the K=1 ablation in Table II (Section IV-D). Setting K=1 is not a clean position-information ablation: it removes the local receptive field of every ConvSwiGLU block and cuts model capacity, which likely explains the uniform ~4 dB drop on 8k min relative to Table I. The matched-length reversal (D3 RoPE 13.9 vs D4 NoPE 13.5) and the extrapolation gaps could therefore be optimization/underfitting artifacts rather than loss of implicit positional encoding. Moreover, Table II contradicts the 'particularly pronounced' wording: without conv, NoPE beats RoPE on 16k min/max by 4.6/5.3 dB (11.2/12.0 vs 6.6/6.7), a much larger extrapolation advantage than in the conv case (B4 vs B3: 17.3/18.1 vs 16.9/17.6). Thus the claim that convolution-encoded position is what makes NoPE preferable for long sequences is not established by the reported experiments.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper compares four positional encoding strategies (APE, KERPLE, RoPE, and NoPE) in the TF-Locoformer TF-domain dual-path source separation architecture. Experiments on WHAMR! and MUSDB18-HQ evaluate both matched-length performance and extrapolation to longer input segments and to higher sampling rates. The authors report that explicit PEs perform better on inputs at or shorter than the training length, whereas NoPE yields superior length extrapolation, and they attribute this to convolutional layers implicitly encoding position. A K=1 ablation is presented to support the convolutional mechanism, and additional results show NoPE scaling to a medium model and faster training.","tokens_in":8766,"tokens_out":1855,"duration_ms":20069,"significance":"If the findings are robust, the paper provides practical design guidance for TF-domain dual-path separators: use explicit PEs for in-distribution lengths and NoPE when inputs may exceed training length or when sampling-rate generalization is needed. The study is the first systematic PE comparison for this architecture class, and the consistent NoPE advantage on MUSDB extrapolation (Figures 1–2) and on WHAMR! 16 kHz (Table I, B4 vs B3) is a useful empirical result. However, the evidence is weakened by single training runs without variance reporting and by a confounded K=1 ablation, so the central mechanism claim is not yet established.","major_comments":[{"comment":"The K=1 ablation does not isolate implicit positional encoding from model capacity: setting the ConvSwiGLU kernel size to 1 removes the local receptive field of every block, and the uniform ~4 dB drop on 8k min relative to Table I is consistent with underfitting rather than loss of position information. Therefore the claim that the NoPE extrapolation advantage is 'particularly pronounced when the model contains convolutional layers' (abstract and conclusion) is not supported by this experiment. In fact, Table II shows the opposite trend: without convolution, NoPE beats RoPE on 16k min/max by 4.6/5.3 dB (11.2/12.0 vs 6.6/6.7), a much larger extrapolation advantage than in the conv case (B4 vs B3: 17.3/18.1 vs 16.9/17.6). A clean test would require an architecture that removes or randomizes positional cues without reducing receptive-field capacity, or a matched-capacity control.","section":"Section IV-D, Table II"},{"comment":"All conclusions are based on single training runs, and differences as small as 0.1 dB are treated as decisive. For instance, the claim that explicit PEs are better on shorter inputs rests on C1 vs C2 at 8k min (17.4 vs 17.3 dB), which is within typical run-to-run variability, and the matched-length advantage of RoPE over NoPE is similarly small (B3 vs B4: 17.8 vs 17.8 on 8k min). Without multiple seeds, confidence intervals, or significance tests, findings (i) and parts of finding (ii) are not statistically grounded. I request at least three seeds for the key comparisons (A3/A4, B3/B4, C1/C2, and the MUSDB RoPE/NoPE pairs) or a stated variance across runs.","section":"Section IV-D, Table I and Section IV-E, Figures 1–2"},{"comment":"The RoPE base frequency (theta) is not reported, although this hyperparameter is known to strongly affect length extrapolation. Since TF-Locoformer's original RoPE configuration may not be optimized for the longer-sequence and higher-sampling-rate conditions tested here, the comparison could understate RoPE's extrapolation capability. Please report the theta value used, and ideally include a sensitivity check (e.g., the default and one larger or smaller theta).","section":"Section III-C and Section IV-C"}],"minor_comments":[{"comment":"The phrase 'particularly pronounced when the model contains convolutional layers' is repeatedly stated, but, as noted in Major Comment 1, the reported evidence in Table II points the other way; please soften or revise this claim.","section":"Abstract and Conclusion"},{"comment":"The caption contains a typo: '‘ MIN ‘ VERSION IS USED FOR TRAINING' should read 'The min version is used for training'; similar spacing issues appear in other table captions.","section":"Section IV-D, Table I caption"},{"comment":"It is unclear whether TF-Locoformer-NoPE (M) was trained with the same budget and early-stopping criterion as the reproduced TF-Locoformer (S), and whether the medium-size NoPE result is from a single run; a brief note would improve comparability.","section":"Section IV-D, Table III"},{"comment":"The figures would benefit from error bars or at least point markers for each condition; currently the reader cannot assess whether the differences between curves (e.g., RoPE vs NoPE at T'=6 in Figure 2) are meaningful.","section":"Section IV-E, Figures 1–2"}],"recommendation":"major_revision","confidential_remarks":"The central NoPE extrapolation advantage appears consistent across datasets and is a useful contribution, but the paper's distinguishing mechanistic claim (convolution-driven) is undermined by the K=1 confound, and the lack of any variance reporting makes several 0.1–0.4 dB comparisons non-decisive. A revision with multi-seed results and a cleaner ablation could resolve both issues within the manuscript's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a competent, genuinely useful empirical comparison of positional encodings in TF-domain dual-path separation. The headline result — NoPE matches or beats explicit PEs on matched lengths and generalizes better to longer inputs — is real and consistently observed across WHAMR! and MUSDB. The paper is worth reading and deserving of a serious referee, but the abstract slightly overstates the convolution-specific mechanism.\n\nThe new piece is the systematic comparison: previous work only looked at APE vs RPE in enhancement, and NoPE was mostly a side observation. Here they control the architecture and vary PE, including KERPLE and RoPE, and show the extrapolation benefit holds across two datasets and several training lengths. The 16% training speedup for NoPE over RoPE is a useful practical detail. The MUSDB curves make the saturation-contrast point well: RoPE saturates around 8s while NoPE keeps improving.\n\nWhere it gets soft: no variance or significance testing anywhere. Differences of 0.1–0.2 dB are treated as meaningful, and with a single run per configuration that's not defensible. Also, their short-sequence claim (explicit PEs better when evaluating shorter than training) is contradicted by their own matched-length results: on WHAMR!, NoPE is comparable or better on 8k min for all matched sets (A4 vs A1-A3, B4 vs B1-B3). The C1 vs C2 comparison is a single difference of 0.1–0.3 dB. They push this conclusion harder than the data support.\n\nThe bigger issue: the (ii) claim — 'particularly pronounced when the model contains convolutional layers' — rests on the K=1 ablation (Table II). That ablation removes local receptive field and capacity, not just implicit position encoding. The uniform ~4 dB drop confirms capacity loss. And Table II actually shows NoPE's extrapolation advantage over RoPE on 16k is bigger without conv (B4 vs B3 in Table I: ~0.4 dB for min, 0.5 for max; D4 vs D3 in Table II: 4.6 and 5.3 dB), which is the opposite of 'particularly pronounced.' So the conv-specific mechanism is unverified; what is verified is that NoPE generalizes well in this architecture.\n\nAll that said, the main practical takeaway — use NoPE in TF-domain dual-path with convolution when you expect longer inputs or unseen sampling rates — is reasonably supported. The paper would benefit from repeated runs with error bars, toning down the short-sequence and conv-specific language, and a fairer ablation (e.g., keep K=1 but increase capacity, or compare with a model that has conv but no padding). No code is provided, which limits reproducibility but does not falsify the result.\n\nRecommendation: send to peer review. It's a useful empirical study with a clear question, and the flaws are fixable in revision. I would bring it to reading group to discuss extrapolation methodology.","headline":"NoPE length-extrapolation advantage is real, but the conv-specific claim is overreached and variance is missing.","tokens_in":872,"tokens_out":959,"would_cite":true,"duration_ms":26117,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"In dual-path Transformer separation models, removing positional encoding gives better length extrapolation than explicit PEs, especially with convolutional layers, while explicit PEs win on matched or shorter inputs.","keywords":["positional encoding","length extrapolation","dual-path transformer","time-frequency domain","source separation","convolutional implicit position","sampling-rate generalization","TF-Locoformer"],"falsifier":"Retrain the key comparisons (e.g., RoPE vs NoPE on WHAMR! at 4-second training, and the MUSDB 3-second models) several times with different random seeds and report the distribution of separation metrics; if the NoPE extrapolation advantage over RoPE falls within the seed-to-seed spread, or if NoPE's matched-length parity disappears with repeated runs, the claimed PE trade-off is not established.","tokens_in":8234,"feed_emoji":"🎧","tokens_out":9481,"duration_ms":85115,"temperature":0.7,"pith_summary":"The paper asks how positional encoding (PE) should be chosen in Transformer-based time-frequency dual-path source separation models, using TF-Locoformer as the test bed. It compares absolute encoding (APE), a relative encoding (KERPLE), rotary encoding (RoPE), and no encoding (NoPE) on speech separation and music source separation. The central claim is a trade-off: explicit PEs improve performance when inputs are no longer than the training segments, but NoPE generalizes better to longer inputs and to unseen sampling rates, and this extrapolation advantage is strongest when the model contains convolutional layers. The authors argue that convolutional layers already carry implicit position information, so explicit PE can be omitted without a matched-length penalty. This matters because length extrapolation is what lets a model trained at one sampling rate handle signals at another, a common practical need.","feed_headline":"No positional encoding is best for long audio inputs","feed_subtitle":"Skipping position tags improves dual-path separators on long audio; explicit tags still win on short inputs.","key_machinery":"The carrying mechanism is the TF-Locoformer block, which alternates frequency-axis and time-axis modeling, each consisting of multi-head self-attention surrounded by ConvSwiGLU feed-forward layers. The treatment variable is how position enters those attention layers: additive sinusoidal embeddings (APE), KERPLE's learnable kernelized relative distances added to attention logits, rotary rotations of queries and keys (RoPE), or no explicit position signal (NoPE). The explanatory load is carried by the convolutional layers: because zero-padding lets convolutions encode position locally, the model with NoPE retains positional information when the kernel size K is greater than one, and this implicit encoding generalizes to unseen lengths better than any explicit scheme tested.","core_discovery":"The paper's central discovery is that in TF-domain dual-path Transformers, the best positional encoding depends on whether the input length matches training. With segments at or below training length, explicit PEs—particularly RoPE on matched data—deliver better separation scores; beyond training length, NoPE outperforms explicit PEs, and the gap widens as inference segments grow, for example in music source separation where NoPE keeps improving past the 8-second point at which RoPE saturates. The authors attribute this to position information encoded implicitly by zero-padded convolutions in the model: with kernel size greater than one, NoPE matches explicit PEs at the training length, while with kernel size one (K=1) the explicit PEs are necessary. The same trend transfers from the 8 kHz to the 16 kHz test condition, indicating that NoPE's extrapolation advantage extends to the frequency dimension and thus to sampling-rate generalization.","pith_inferences":["A direct extension is to test NoPE at sampling rates far beyond training (e.g., 8 kHz to 32 or 44.1 kHz) to see whether the frequency-dimension extrapolation holds beyond the 16 kHz case reported.","The K=1 experiments imply an architecture-level rule: the more convolutional the Transformer, the less need for explicit PE; a systematic sweep of kernel size versus PE type could quantify when each becomes redundant.","The length-dependent trade-off suggests a hybrid design—explicit PE for short-range in-distribution accuracy plus convolution-provided position for long-range extrapolation—could beat either extreme.","Because only single-seed results are reported, the precise ordering among PE methods on matched-length inputs should be treated as provisional until replicated."],"forward_implications":["For deployment where inputs may exceed training length or sampling rate may change, NoPE is the safer choice in convolution-containing dual-path Transformers.","For matched-length or shorter inputs, explicit PEs such as RoPE remain preferable, so the choice is a length-dependent design decision.","The implicit positional encoding from convolutions is sufficient to replace explicit PEs in this architecture class, allowing the 16% training-time saving NoPE showed in the WHAMR! setup.","NoPE's length extrapolation transfers from speech separation to music source separation, suggesting the result is not dataset-specific.","Sampling-rate generalization (e.g., 8 kHz training to 16 kHz test) is improved by NoPE relative to RoPE, supporting low-rate training for high-rate deployment."],"supporting_citations":[{"why":"Supplies the base TF-Locoformer architecture and the state-of-the-art separation system that all PE variants are inserted into.","marker":"[12]"},{"why":"Defines RoPE, the rotary positional encoding used as one of the four compared PE methods and as TF-Locoformer's original PE.","marker":"[13]"},{"why":"Defines KERPLE, the relative positional encoding chosen for its reported length-generalization strength, and gives its kernel formula.","marker":"[15]"},{"why":"Provides evidence that convolutional layers encode positional information through zero-padding, the basis for the NoPE condition.","marker":"[16]"},{"why":"Supports the expectation that removing explicit PE can improve generalization to longer sequences.","marker":"[17]"},{"why":"Prior speech-enhancement study reporting KERPLE's good length generalization, used as the comparison anchor for extrapolation behavior.","marker":"[22]"},{"why":"Reports that RoPE does not reliably generalize beyond training length, motivating the search for alternatives in the length-extrapolation comparison.","marker":"[24]"},{"why":"Establishes sampling-frequency independence of TF-domain dual-path models, making frequency-dimension length extrapolation the mechanism behind sampling-rate generalization.","marker":"[10]"},{"why":"Supplies the band-split encoder and band-wise decoding used for the MUSDB18-HQ experiments with the BS-Locoformer variant.","marker":"[29]"},{"why":"Provides the band-split configuration and a band-split RoPE Transformer baseline connected to the MUSDB experimental setup.","marker":"[30]"}],"fun_headline_variants":["NoPE beats PE on long audio","Skip positional encoding for long audio","No position tags win on unseen lengths","Long audio: NoPE outperforms explicit PE","For long audio, drop position tags"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The conclusions rest on single training runs per configuration, so small performance differences—some around 0.1 dB—are treated as real signals rather than run-to-run noise.","fun_headline_variants_meta":{"raw":{"variants":["NoPE beats PE on long audio","Skip positional encoding for long audio","No position tags win on unseen lengths","Long audio: NoPE outperforms explicit PE","For long audio, drop position tags"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000636,"raw_usage":{"total_tokens":2914,"prompt_tokens":911,"completion_tokens":2003,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":527,"completion_tokens_details":{"reasoning_tokens":1951}},"tokens_in":527,"tokens_out":2003,"duration_ms":18174,"temperature":1.0,"reasoning_tokens":1951,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T05:48:18.661921+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain the key comparisons (e.g., RoPE vs NoPE on WHAMR! at 4-second training, and the MUSDB 3-second models) several times with different random seeds and report the distribution of separation metrics; if the NoPE extrapolation advantage over RoPE falls within the seed-to-seed spread, or if NoPE's matched-length parity disappears with repeated runs, the claimed PE trade-off is not established.","supporting_citations":[{"cited_title":"Tf-locoformer: Transformer with local modeling by convolution for speech separation and enhance- ment,","cited_arxiv_id":null,"evidence_quote":"Supplies the base TF-Locoformer architecture and the state-of-the-art separation system that all PE variants are inserted into."},{"cited_title":"Kerple: Kernelized relative positional embedding for length extrapolation,","cited_arxiv_id":null,"evidence_quote":"Defines KERPLE, the relative positional encoding chosen for its reported length-generalization strength, and gives its kernel formula."},{"cited_title":"An exploration of length general- ization in transformer-based speech enhancement,","cited_arxiv_id":null,"evidence_quote":"Prior speech-enhancement study reporting KERPLE's good length generalization, used as the comparison anchor for extrapolation behavior."},{"cited_title":"The impact of positional encoding on length generalization in transformers,","cited_arxiv_id":null,"evidence_quote":"Reports that RoPE does not reliably generalize beyond training length, motivating the search for alternatives in the length-extrapolation comparison."},{"cited_title":"Toward universal speech enhancement for diverse input conditions,","cited_arxiv_id":null,"evidence_quote":"Establishes sampling-frequency independence of TF-domain dual-path models, making frequency-dimension length extrapolation the mechanism behind sampling-rate generalization."},{"cited_title":"Music source separation with band-split rope transformer,","cited_arxiv_id":null,"evidence_quote":"Provides the band-split configuration and a band-split RoPE Transformer baseline connected to the MUSDB experimental setup."}],"review_version":1}