Pith. sign in

REVIEW 3 major objections 4 minor 24 references

Additive decomposition of one-dimensional signals using Transformers

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that a Transformer encoder trained on synthetic data can decompose a noisy one-dimensional signal into its piecewise-constant, smooth, oscillatory, and noise components more accurately and far faster than a tuned…

desk verdict Clean empirical study of a Transformer for 1D signal decomposition, but the headline win over VSD rests on 13 un-replicated signals and needs statistical teeth. read the letter →

arxiv 2506.05942 v1 pith:H2WMEPIV submitted 2025-06-06 cs.LG

Samuele Salti , Andrea Pinto , Alessandro Lanza , Serena Morigi This is my paper
classification cs.LG
keywords Transformersignaldecompositionadditivevariationaldeeplearningtimeseriespiecewiseconstantself-attention
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that a Transformer, a deep learning architecture built on self-attention, can solve the additive signal decomposition problem: given a noisy one-dimensional signal $f = c+s+o+n$, recover the piecewise-constant 'cartoon' part $c$, the smooth low-frequency part $s$, the high-frequency oscillatory part $o$, and the noise $n$. The authors train their model, called TSD, on synthetic signals and report that it matches or beats a state-of-the-art variational method (VSD) on every component, with average root-mean-square error $2.153\times10^{-3}$ versus $4.780\times10^{-3}$ on a 13-signal test set, while running orders of magnitude faster and needing no per-signal hyperparameter tuning. If the result holds beyond the training distribution, it would make decomposition a fast, automatic preprocessing step for downstream tasks such as forecasting, biomedical monitoring, and fault diagnosis. The paper's own experiments are limited to in-distribution synthetic data; that limitation is part of what a sympathetic reader should keep in mind.

What carries the argument

The central object is the Transformer encoder with multi-head self-attention (softmax over query-key dot products) plus sinusoidal positional encodings; it transforms the input token sequence into an output sequence of the same length, and a final 1D convolution plus a shared linear layer maps tokens to the four component signals. Self-attention is doing the work: it gives each output location a global view of the input, which the authors argue helps estimate each component, since long-range dependencies matter for the piecewise-constant and oscillatory parts. Chunking variants control the sequence length, trading memory against accuracy, and the experiments compare the no-chunk, sum, cat, and conv tokenization strategies.

What would settle it

Train or evaluate the same architecture on signals whose components fall outside the synthetic ranges (e.g., smooth frequencies below 2 or above 7, oscillatory frequencies outside 70–80, jumps closer than 40 samples, non-Gaussian or colored noise, or real recordings with known ground truth); if the per-component RMSE no longer stays well below that of the tuned variational baseline, the paper's claim of general data-driven effectiveness would be disproved.

Watch

Extended reading notes

Core claim

The central claim is that the signal decomposition problem (1) can be solved by an encoder-only Transformer that maps a sequence of scalar samples to four sequences of equal length, one per component. The network is trained by minimizing the sum of the mean squared errors of the four components, and on the benchmark it outperforms the convex predictor–nonconvex corrector variational method VSD, both in accuracy (all components, on both the 13-signal and 4000-signal test sets) and in practical terms: no test-time tuning and automatic detection of absent components.

Load-bearing premise

Everything rests on the assumption that the synthetic generative process used for training and testing—the frequency bands, jump distances, amplitude ranges, blending factors, and SNR=20—produces signals representative of the real-world signals the method is supposed to decompose.

Editorial extensions

If this is right

  • TSD removes the need for per-signal hyperparameter tuning: once trained, the same network decomposes any in-distribution signal in one forward pass, about 144 ms per batch of 128 signals on GPU in the chunked variant.
  • The model automatically outputs near-zero values for components absent from the input, whereas variational methods require the user to specify which components to estimate.
  • Because an ensemble of the chunked and no-chunk variants combines the best oscillatory estimate with the best other components, users can trade memory for accuracy depending on their resources.
  • A public synthetic dataset of 18,000 signals with ground-truth components (to be released) gives other researchers a common benchmark for data-driven signal decomposition.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural next test would be out-of-distribution evaluation: the reported superiority over VSD is measured on data drawn from the same generator that produced the training set, so the method's practical value hinges on how much of that structure transfers to real signals.
  • The approach could be plugged into time-series forecasting pipelines as a learned alternative to classical trend-seasonal decomposition; the paper's authors mention such applications only as motivation, so this is an extension, not a claim.
  • The chunking results suggest a scalable design: reducing token count with a convolutional projection keeps most of the accuracy while cutting memory by a factor of roughly 2.5, which hints that attention can operate efficiently on compressed tokens for long signals.
  • A direct extension would be to test the same architecture on two-dimensional images, where cartoon-texture-noise decomposition has a long tradition in variational image processing; the paper names this as future work.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes TSD, a Transformer-encoder architecture that maps a one-dimensional signal f in R^M to four additive components (piecewise constant, smooth, oscillatory, and noise) under the model f = c + s + o + n. The input is either the full signal ("no chunks") or downsampled by one of three chunking strategies (sum, cat, conv), and the training loss is the sum of MSEs over the four components. A synthetic data generator is defined with specific frequency bands, jump parameters, blending factors, and SNR=20. Experiments include an ablation study (Table 1) and a comparison with the variational VSD method (Table 2). The authors report lower RMSE for TSD than VSD on a 13-signal set and additionally evaluate TSD on 4000 held-out synthetic signals, concluding that data-driven methods, and in particular TSD, are effective for the considered signal-decomposition problem.

Significance. If the comparative result were statistically grounded, TSD would offer a practical alternative to variational signal decomposition: no per-signal hyperparameter tuning, fast inference, and automatic detection of absent components. The architecture is simple, the ablation study is thoughtful, and the synthetic-data pipeline is described in enough detail to be reproduced. However, all evidence is synthetic and in-distribution, and the only head-to-head comparison rests on 13 unreplicated signals. The contribution is therefore a plausible proof-of-concept rather than an established improvement over variational methods. The paper would be substantially strengthened by error bars with repeated draws, a clearer description of the VSD evaluation protocol, and at least one out-of-distribution or real-data check.

major comments (3)
  1. [§5.2, Table 2] The central comparison against VSD is not statistically supported. It uses exactly 13 test signals, one random draw per blending factor, with no repeated sampling, no error bars, and no significance test. Because VSD requires about 30 minutes of per-signal hyperparameter tuning, these 13 signals constitute a hand-tuned evaluation set rather than an independent sample; the reported margin (TSD no chunks 2.153e-3 versus VSD 4.780e-3) could shrink or disappear on another draw. The 4000-signal rows evaluate only TSD and therefore do not validate the comparative claim. Please provide multiple random draws with paired confidence intervals or a significance test, or explicitly present the comparison as illustrative rather than as evidence of superiority.
  2. [§5.2] The VSD evaluation protocol is underspecified. The manuscript does not state whether VSD was given the true component support (which of c, s, o are nonzero) for each of the 13 signals, whether the exhaustive hyperparameter tuning was performed on the test signal itself, or what criterion selected the final configuration. Since the paper notes that variational methods require users to manually specify the components to be estimated, this information is necessary to rule out an uneven comparison. Please report the exact protocol, including any validation split, the number of configurations tried, and how the presence or absence of each component was communicated to VSD.
  3. [§4 and §5] All experimental results are in-distribution with respect to the synthetic generator of Section 4: frequency bands [2,7] and [70,80], jump distances [40,50], amplitudes [0.5,1], SNR=20, and the listed blending factors. The introduction and conclusion motivate signal decomposition for real-world applications (environmental, biomedical, financial), but no experiment tests signals outside this generator, so the practical generalization claims are unsupported. Please add an out-of-distribution or real-data experiment, or explicitly scope the paper's claims to the synthetic distribution and temper the introduction and conclusion accordingly.
minor comments (4)
  1. [Multiple sections] There are several typos: "Trasformer-based" in Section 3, "positional encondings" in Section 3, "reminder" for "remainder" in Section 5.1, and "more accurate then" for "than" in Section 5.2.
  2. [Fig. 3 caption] The caption refers to the baseline as "(Girometti et al., 2024)" rather than "VSD", which is inconsistent with the rest of the paper.
  3. [Footnote 1] The dataset availability footnote says "to be published upon acceptance"; an anonymous repository link or a description of how to obtain the data should be provided for review and reproducibility.
  4. [Tables 1 and 2] It would be helpful to state the number of random seeds used for the reported RMSE values and whether the values are averaged over test signals only or also over network initializations.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: TSD is trained and evaluated on held-out synthetic data, and the VSD comparison is an empirical benchmark, not a derivation from the paper's own assumptions.

full rationale

The paper's central claim is empirical: a Transformer trained with a per-component MSE loss on a synthetic training set is evaluated on held-out synthetic test signals. The reported test-set RMSE values in Table 2 are not fitted parameters, and no component of the loss, architecture, or experimental protocol is defined in terms of the reported accuracy metric. The only self-citation concern is the use of VSD (Girometti et al., 2024) as the variational baseline, a paper co-authored by two of the current authors. However, the comparison is a direct numerical experiment rerun on the same 13 signals, not an appeal to the citation's authority. The VSD method is published independently of the Transformer model, and its results are recomputed here with per-signal hyperparameter tuning. There is no uniqueness theorem, no ansatz imported via citation, and no fitted parameter relabeled as a prediction. The evaluation is in-distribution, which limits external generalizability, but that is a scoping limitation, not circularity.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim rests on a synthetic data distribution with hand-chosen frequencies, jump statistics, blending factors, and SNR, plus standard Transformer hyperparameters. There are no invented physical entities. The key assumptions are the additive mixture model, the representativeness of the synthetic generator, and the identifiability of the decomposition.

free parameters (5)
  • Smooth/oscillatory frequency bands = k in [2,7] for smooth, k in [70,80] for oscillatory
    Hand-chosen in Section 5 to define the synthetic data distribution; all results are conditional on this choice.
  • Jump generation parameters = distance d in [40,50], amplitude |a| in [0.5,1]
    Hand-chosen in Section 5 to control the piecewise-constant component; determines task difficulty.
  • Blending factor list = 13 triplets listed in Section 5
    Defines the mixture distribution for training and the 13-signal test set; hand-selected to cover a range of mixtures.
  • Signal-to-noise ratio = 20 dB
    Noise level setting for all generated observations, chosen by hand in Section 5.
  • Network and training hyperparameters = D=512, h=8, N=4, dropout=0.1, lr=1e-4, batch=64, epochs=15000
    Standard Transformer settings and training recipe; not tuned per test signal but affect the reported accuracy.
assumptions (5)
  • domain assumption Signals of interest are additively decomposable as f = c + s + o + n with AWGN noise (Eq. 1).
    Stated in Section 1 as the problem hypothesis; if real signals do not fit this model, the method's target is mis-specified.
  • domain assumption The synthetic generation process in Section 4 produces signals representative of the target application domain.
    Underlies all results; no validation on real-world signals is provided.
  • domain assumption The ground-truth decomposition used for training is the desired output, and the decomposition is identifiable within the generative model.
    The loss in Eq. 5 penalizes distance to particular synthetic components; if the decomposition is non-unique, the target is arbitrary.
  • domain assumption Training and test signals are i.i.d. samples from the same distribution, and in-distribution accuracy transfers to practical use.
    Standard supervised learning assumption; only the in-distribution part is tested.
  • domain assumption ADAM optimization with MSE loss finds parameters that approximate the optimal decomposition mapping.
    Standard deep learning practice; not proven for this non-convex problem.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Additive decomposition of one-dimensional signals using Transformers." pith.science (2026). https://pith.science/paper/H2WMEPIV

@misc{pith2026250605942,
  author       = {Pith},
  title        = {Pith review of: Additive decomposition of one-dimensional signals using Transformers},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/H2WMEPIV}},
  note         = {Machine review of arXiv:2506.05942}
}
read the original abstract

One-dimensional signal decomposition is a well-established and widely used technique across various scientific fields. It serves as a highly valuable pre-processing step for data analysis. While traditional decomposition techniques often rely on mathematical models, recent research suggests that applying the latest deep learning models to this problem presents an exciting, unexplored area with promising potential. This work presents a novel method for the additive decomposition of one-dimensional signals. We leverage the Transformer architecture to decompose signals into their constituent components: piece-wise constant, smooth (low-frequency oscillatory), textured (high-frequency oscillatory), and a noise component. Our model, trained on synthetic data, achieves excellent accuracy in modeling and decomposing input signals from the same distribution, as demonstrated by the experimental results.

Figures

Figures reproduced from arXiv: 2506.05942 by the authors.

Figure 1
Figure 1. Neural architecture of the proposed TSD approach [PITH_FULL_IMAGE:figures/full_fig_p011_1.png] view at source ↗
Figure 2
Figure 2. Qualitative results on a signal from the test set: ground truth signals (black), signals estimated by VSD method (blue) and the proposed TSD [PITH_FULL_IMAGE:figures/full_fig_p012_2.png] view at source ↗
Figure 3
Figure 3. Qualitative results on a signal from the test set: ground truth signals (black), signals estimated by VSD method (blue) and the proposed TSD data [PITH_FULL_IMAGE:figures/full_fig_p012_3.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

24 extracted references · 13 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize "" * " " * ...

    work page

  2. [2]

    author Ba, J. L. , author Kiros, J. R. , & author Hinton, G. E. ( year 2016 ). title Layer normalization . journal arXiv preprint arXiv:1607.06450 \/ ,

    arXiv 2016

  3. [3]

    , author Selesnick, I

    author Cai, G. , author Selesnick, I. W. , author Wang, S. , author Dai, W. , & author Zhu, Z. ( year 2018 ). title Sparsity-enhanced signal decomposition via generalized minimax-concave penalty for gearbox fault diagnosis . journal Journal of Sound and Vibration \/ , volume 432 \/ , pages 213--234 . :https://doi.org/10.1016/j.jsv.2018.06.037

    work page doi:10.1016/j.jsv.2018.06.037 2018

  4. [4]

    , author Zhang, H

    author Chen, Y. , author Zhang, H. , author You, Y. , author Zhang, J. , & author Tang, L. ( year 2025 ). title A hybrid deep learning model based on signal decomposition and dynamic feature selection for forecasting the influent parameters of wastewater treatment plants . journal Environmental Research \/ , volume 266 \/ , pages 120615 . :https://doi.org...

    work page arXiv 2025

  5. [5]

    , author Huska, M

    author Cicone, A. , author Huska, M. , author Kang, S.-H. , & author Morigi, S. ( year 2022 ). title JOT : A variational signal decomposition into jump, oscillation and trend . journal IEEE Transactions on Signal Processing \/ , volume 70 \/ , pages 772--784 . :10.1109/TSP.2022.3145665

    arXiv 2022

  6. [6]

    , author Lu, J

    author Daubechies, I. , author Lu, J. , & author Wu, H.-T. ( year 2011 ). title Synchrosqueezed wavelet transforms: An empirical mode decomposition-like tool . journal Applied and Computational Harmonic Analysis \/ , volume 30 \/ , pages 243--261 . :https://doi.org/10.1016/j.acha.2010.08.002

    doi:10.1016/j.acha.2010.08.002 2011

  7. [7]

    , author Beyer, L

    author Dosovitskiy, A. , author Beyer, L. , author Kolesnikov, A. , author Weissenborn, D. , author Zhai, X. , author Unterthiner, T. , author Dehghani, M. , author Minderer, M. , author Heigold, G. , author Gelly, S. , author Uszkoreit, J. , & author Houlsby, N. ( year 2020 ). title An image is worth 16x16 words: Transformers for image recognition at sca...

    work page 2020

  8. [8]

    , & author Zosso, D

    author Dragomiretskiy, K. , & author Zosso, D. ( year 2014 ). title Variational mode decomposition . journal IEEE Transactions on Signal Processing \/ , volume 62 \/ , pages 531--544 . :10.1109/TSP.2013.2288675

    arXiv 2014

Show all 24 references
  1. [9]

    , & author Rehman, N

    author Eriksen, T. , & author Rehman, N. ( year 2023 ). title Data-driven nonstationary signal decomposition approaches: a comparative analysis. journal Sci Rep \/ , volume 1798 \/

    2023

  2. [10]

    , author Huska, M

    author Girometti, L. , author Huska, M. , author Lanza, A. , & author Morigi, S. ( year 2024 ). title Convex predictor–nonconvex corrector optimization strategy with application to signal decomposition . journal Journal of Optimization Theory and Applications \/ , (pp. pages 1...

    2024 doi

  3. [11]

    author Huang, N. E. , author Shen, Z. , author Long, S. R. , author Wu, M. C. , author Shih, H. H. , author Zheng, Q. , author Yen, N.-C. , author Tung, C. C. , & author Liu, H. H. ( year 1998 ). title The empirical mode decomposition and the hilbert spectrum for nonlinear and...

    1998

  4. [12]

    , & author Ba, J

    author Kingma, D. , & author Ba, J. ( year 2015 ). title Adam: A method for stochastic optimization . In booktitle International Conference on Learning Representations (ICLR) \/ . address San Diega, CA, USA

    2015

  5. [13]

    , author Jiao, S.-B

    author Li, Y.-X. , author Jiao, S.-B. , & author Gao, X. ( year 2021 ). title A novel signal feature extraction technology based on empirical wavelet transform and reverse dispersion entropy . journal Defence Technology \/ , volume 17 \/ , pages 1625--1635 . :https://doi.org/1...

    2021 doi

  6. [14]

    , author Tan, Y

    author Lin, Y.-D. , author Tan, Y. K. , & author Tian, B. ( year 2022 ). title A novel approach for decomposition of biomedical signals in different applications based on data-adaptive gaussian average filtering . journal Biomedical Signal Processing and Control \/ , volume 71...

    2022

  7. [15]

    author Meyers, B. E. , & author Boyd, S. P. ( year 2023 ). title Signal Decomposition Using Masked Proximal Operators \/ . Foundations and trends in signal processing. publisher Now Publishers

    2023

  8. [16]

    , author Joshi, S

    author Singh, P. , author Joshi, S. D. , author Patney, R. K. , & author Saha, K. ( year 2017 ). title The Fourier decomposition method for nonlinear and non-stationary time series analysis . journal Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sc...

    2017

  9. [17]

    author Smith, L. N. , & author Topin, N. ( year 2017 ). title Super-convergence: Very fast training of residual networks using large learning rates . journal arXiv preprint arXiv:1708.07120 \/ ,

    2017 arXiv

  10. [18]

    , author Hinton, G

    author Srivastava, N. , author Hinton, G. , author Krizhevsky, A. , author Sutskever, I. , & author Salakhutdinov, R. ( year 2014 ). title Dropout: A simple way to prevent neural networks from overfitting . journal Journal of Machine Learning Research \/ , volume 15 \/ , pages...

    2014

  11. [19]

    , author Shazeer, N

    author Vaswani, A. , author Shazeer, N. , author Parmar, N. , author Uszkoreit, J. , author Jones, L. , author Gomez, A. N. , author Kaiser, L. , & author Polosukhin, I. ( year 2017 ). title Attention is all you need . In booktitle Proceedings of the 31st International Confere...

    2017

  12. [20]

    , author Zhou, T

    author Wen, Q. , author Zhou, T. , author Zhang, C. , author Chen, W. , author Ma, Z. , author Yan, J. , & author Sun, L. ( year 2023 ). title Transformers in time series: A survey . In editor E. Elkind (Ed.), booktitle Proceedings of the Thirty-Second International Joint Conf...

    2023 doi

  13. [21]

    , author Xu, J

    author Wu, H. , author Xu, J. , author Wang, J. , & author Long, M. ( year 2021 ). title Autoformer: decomposition transformers with auto-correlation for long-term series forecasting . In booktitle Proceedings of the 35th International Conference on Neural Information Processi...

    2021

  14. [22]

    , author Cicone, A

    author Zhou, F. , author Cicone, A. , & author Zhou, H. ( year 2024 ). title IRCNN ^ + : An enhanced iterative residual convolutional neural network for non-stationary signal decomposition . journal Pattern Recognition \/ , volume 155 \/ , pages 110670 . :https://doi.org/10.10...

    2024

  15. [23]

    , author Ma, Z

    author Zhou, T. , author Ma, Z. , author Wen, Q. , author Wang, X. , author Sun, L. , & author Jin, R. ( year 2022 a ). title FEDformer : Frequency enhanced decomposed transformer for long-term series forecasting . journal arXiv preprint arXiv:2201.12740 \/ ,

    2022 arXiv

  16. [24]

    , author Feng, Z

    author Zhou, W. , author Feng, Z. , author Xu, Y. , author Wang, X. , & author Lv, H. ( year 2022 b ). title Empirical Fourier decomposition: An accurate signal decomposition method for nonlinear and non-stationary time series analysis . journal Mechanical Systems and Signal P...

    2022

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.