Pith. sign in

REVIEW 4 major objections 5 minor 45 references

Content-aware Balanced Spectrum Encoding in Masked Modeling for Time Series Classification

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Masked time-series transformers can avoid feature homogenization and low-frequency bias by training a content-aware Fourier-domain decoder alongside the temporal one.

desk verdict Plausible frequency-domain decoder for masked time-series modeling with a genuinely novel Bernstein energy-rebalancing unit, but the reported gains are not trustworthy until the test-set hyperparameter tuning is fixed and code is released. read the letter →

arxiv 2412.13232 v1 pith:DE7DRZRB submitted 2024-12-17 cs.LG

Yudong Han , Haocong Wang , Yupeng Hu , Yongshun Gong , Xuemeng Song , Weili Guan This is my paper · ORCID
classification cs.LG
keywords maskedtime-seriesmodelingself-supervisedrepresentationlearningclassificationFourierdomaindecodingfeaturehomogenizationrankcollapsespectrumenergyrebalancecontent-awareinteractionmodulation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that transformer-based masked time-series modeling—a self-supervised pretraining scheme in which part of a series is hidden and then reconstructed—suffers from two linked defects: attention's long-range averaging drives encoded features toward rank collapse and homogenization, and the network over-fits low-frequency components at the expense of mid- and high-frequency detail. To fix both, it proposes an auxiliary content-aware balanced decoder that operates in the Fourier domain alongside the usual temporal decoder. The decoder's two units—content-aware interaction modulation and spectrum energy rebalance—progressively refine the masked representation, and a dual-constraint loss ties the temporal and spectral reconstructions together. The claimed effect is better classification accuracy under both linear probing and fine-tuning on ten time-series datasets, with the decoder also improving several existing masked-modeling methods when added to them.

What carries the argument

The central object is the content-aware balanced decoder (CBD), a Fourier-domain decoder added in parallel to the temporal decoder. Its first unit, content-aware interaction modulation (CIM), transforms the intermediate representation into the frequency domain and multiplies it by a learned complex modulation signal; the Fourier convolution theorem makes this equivalent to a dynamic convolution in the time domain, which the paper argues restricts unnecessary long-range interaction and adapts the receptive field to local variation. Its second unit, spectrum energy rebalance (SER), models the ideal energy-response function as a Bernstein polynomial evaluated at the softmax-normalized energy of each frequency component, with coefficients regressed from the global energy vector, and multiplies frequency components by this learned scaling. A dual-constraint loss uses Parseval's theorem to enforce consistency between temporal and spectral reconstructions. Together these pieces are meant to progressively refine the masked representation block by block.

What would settle it

Train the same masked-time-series model on a synthetic dataset where informative high-frequency structure and uninformative low-frequency noise have equal spectral energy; if the spectrum energy rebalance unit cannot separate them because it only sees per-component magnitudes, the claim that energy rebalancing drives the gains would be refuted.

Watch

Extended reading notes

Core claim

On its own terms, the central claim is that masked time-series transformers can be made to learn richer, more balanced representations by supervising reconstruction in the frequency domain. The paper reports that vanilla masked time-series models produce interaction matrices whose rank collapses with depth and reconstructed signals whose spectrum energy clusters at low frequencies, and it attributes both symptoms to the same encoder behavior. The content-aware balanced decoder is designed to counteract this: content-aware interaction modulation multiplies the Fourier-transformed features by a learned complex signal, which by the convolution theorem acts as a content-dependent convolution kernel that limits and adapts the interaction scope; spectrum energy rebalance then applies a Bernstein-polynomial approximation of an ideal energy-response function, with coefficients produced by a gating network, to scale each frequency component's energy toward the distribution of the original series. With this decoder and the dual-constraint loss, the paper reports accuracy gains on ten classification datasets, higher-rank interaction matrices, and visible shifts of energy into mid- and high-frequency bands.

Load-bearing premise

The core assumption is that the energy rebalancing needed for a frequency component can be decided from that component's own magnitude alone, so a component's gain does not depend on which frequency it is; if the harmful imbalance couples two different frequencies that happen to have similar energy, the spectrum energy rebalance unit cannot represent the needed correction.

Editorial extensions

If this is right

  • Linear-probing accuracy improves over contrastive and masked-modeling baselines on nearly every one of the ten datasets, with the largest gains on HAR and MotorImagery.
  • Fine-tuning accuracy also improves on most datasets, and the encoder features show higher-rank interaction matrices than the vanilla masked model.
  • The frequency decoder alone outperforms the temporal decoder, and using both decoders together with the dual-constraint loss gives the best results.
  • Adding the content-aware balanced decoder to PatchTST, CRT, and SimMTM improves their average linear-probing and fine-tuning accuracy, most strongly for CRT.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because SER's rebalancing depends on each frequency's energy magnitude rather than its identity, a simpler learned per-frequency scaling or a fixed high-frequency-boosting filter might reproduce much of the gain; the Bernstein machinery is not tested against that cheaper alternative.
  • The dual-constraint formulation suggests a general recipe: any auxiliary reconstruction branch in an alternative transform domain, such as a wavelet or short-time Fourier domain, could provide the same homogenization- and imbalance-reducing pressure, which is testable by swapping the transform.
  • The large gains reported when CBD is added to CRT, a method with a weaker temporal decoder, suggest the benefit is largest when the base model's representation is already homogenized; measuring interaction-matrix rank before adding CBD could predict which models gain most.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper addresses two claimed deficiencies of transformer-based masked time-series modeling: feature homogenization due to rank collapse, and spectrum energy imbalance with over-concentration on low frequencies. It proposes an auxiliary content-aware balanced decoder (CBD) containing a Content-aware Interaction Modulation (CIM) unit and a Spectrum Energy Rebalance (SER) unit, plus a dual-constraint loss coupling temporal and frequency reconstruction. The method is evaluated on ten time-series classification datasets under linear probing and fine-tuning, with ablations, interaction-matrix visualizations, and energy-distribution analyses. The central empirical claim is that CBD improves masked time-series representation learning and classification accuracy when added to several existing masked modeling methods.

Significance. If the empirical claims hold, the CBD would be a useful plug-in decoder for masked time-series transformers, and the spectrum-based diagnosis of rank collapse and energy imbalance is of interest to the self-supervised time-series community. The paper uses standard tools (convolution theorem and Parseval's theorem) correctly in its derivations, and it ships a comparatively large evaluation with per-seed averages, ablations, and qualitative analyses. The main weakness is the evaluation protocol: hyperparameters are selected using test-set accuracy, one plug-in result that declined is excluded from the averaged table, and a large per-dataset negative result is left unanalyzed. These issues bear directly on the claimed 'nearly surpasses' superiority and must be resolved before the central claim is credible.

major comments (4)
  1. [B.2 and Table 3] The evaluation selects hyperparameters using test-set accuracy. Section B.2 states that γ is set to the value 'corresponding to the best performance of each data', and Table 3 says K=12 is chosen because it gives the best HAR linear-probing accuracy. No validation split or fixed selection protocol is described. If γ and K were chosen on the same test labels that then produce the reported accuracies, every reported number is the best of a small test-set search, which optimistically biases the comparisons with baselines and the attribution of gains to SER and CIM. Please define an explicit validation procedure, select γ and K on validation data (or fix them a priori), and report final test-set results under that protocol.
  2. [CBD Generality (Table 4 and surrounding text)] TimeMAE is excluded from the generality experiment after observing a decline. The text explains 'since TimeMAE does not use the origin data for reconstruction, the CBD may receive inaccurate supervisory signals, leading to performance decline', but the averaged gains in Table 4 are computed without TimeMAE. This is selective reporting: the claim that 'CBD, as an additional decoder, improves the classification performance' is contradicted by the TimeMAE result. Please include TimeMAE in Table 4 with full per-dataset numbers, and either provide supporting analysis for the failure mechanism or qualify the plug-in claim accordingly.
  3. [Table 1, dataset SRSCP1] On SelfRegulationSCP1, the proposed method underperforms SimMTM by 4.06% in linear probing (86.72 vs. 90.78) and by 6.47% in fine-tuning (86.25 vs. 92.72), and is also below several contrastive baselines in linear probing. This is the largest per-dataset negative result in the paper, yet the manuscript states that the method 'achieves the highest accuracy across most of datasets' and 'nearly surpasses a bunch of baselines' without analyzing this failure. Please either analyze the failure (e.g., spectral properties or dataset characteristics) or restate the empirical claim to reflect the actual win/loss pattern across the ten datasets.
  4. [Spectrum Energy Rebalance Unit (Eq. 10-12)] The SER rebalance function p_K is evaluated only on the normalized energy Ã(λ_s), so the gain applied to a frequency component depends on that component's magnitude alone, not on its frequency index or phase. Consequently, SER cannot distinguish two components with similar energy but different semantic roles (e.g., narrow-band noise and signal at the same energy level). The claimed ability to 'recalibrate the energy distribution across different frequency components' therefore requires an additional representational assumption, namely that the ideal rebalancing is a function of per-component energy magnitude. Please state this limitation explicitly, or modify the formulation to incorporate frequency-index information, and discuss whether the empirical gains persist under that modification.
minor comments (5)
  1. [B.2] The text refers to 'CLS (Liang et al. 2023)' but the baseline is named CSL in the Baselines section and in Table 1; please correct the abbreviation.
  2. [Optimization Strategy, Eq. (14)] The notation L^(re)_∼ is used for reconstruction losses but the symbol ∼ is not defined; please define it or replace it with explicit subscripts.
  3. [A.1, Eq. (16)-(19)] The proof of the convolution theorem does not track the change of summation variables consistently: after substituting x = v - τ, the outer sum is still written over v in Eq. (19), which obscures the argument. Please rewrite the finite-case proof with consistent indices.
  4. [Figure 4 caption] The caption mentions PhonemeSpectra, but the figure legend and the surrounding text use different shading for datasets; please clarify which dataset each row corresponds to and define the line styles consistently.
  5. [Table 5 and Table 1] The FordB dataset is abbreviated FB in Table 1 but listed as FordB in Table 5; please use a single consistent name throughout.

Circularity Check

2 steps flagged · score 6.0 of 10

Two load-bearing steps reduce to their own inputs: the 'dual-constraint' loss is just the MSE loss rescaled by the paper's own Parseval theorem, and the reported gains are selected by tuning gamma and K on test labels.

  1. renaming known result [Optimization Strategy, Eq. (14); Theorem 2, Eq. (8)]
    "L = L(re)T (˜ZT , Z(gt)T ) + L(dual)F (F (˜ZT ), Z(gt)F ) + γ h L(re)F (˜Z(U )F , Z(gt)F ) + L(dual)T (F −1(˜Z(U )F ), Z(gt)T ) i , (14) ... Specially, the dual-constraint loss ensures that the two decoding branches can enhance the information consistency between temporal domain and frequency domain in the training process."

    Under the paper's own Theorem 2 (Eq. 8), temporal energy equals frequency energy for any signal. Let Δ = ˜ZT − Z(gt)T. Then L(dual)_F = ||F(˜ZT) − F(Z(gt)T)||^2 = ||F(Δ)||^2 = ||Δ||^2 = L(re)_T, and similarly L(dual)_T = ||F^{-1}(˜ZF) − Z(gt)T||^2 = ||˜ZF − F(Z(gt)T)||^2 = L(re)_F. Substituting into Eq. (14) gives L = 2L(re)_T + 2γL(re)_F. The 'dual-constraint' terms are therefore not new constraints; they are the paper's own MSE losses expressed in Fourier coordinates. Any benefit attributed to them in Table 2 is not evidence of an independent 'information consistency' mechanism, because the asserted consistency is Parseval's identity already used to define the loss.

  2. fitted input called prediction [Appendix B.2 (Implementation Details) and 'Performance Variation with Different Order K' (Table 3 discussion)]
    "Due to the distribution difference of frequency information in different datasets, the value of γ corresponding to the best performance of each data is slightly different. ... Thus, we select K = 12 as the optimal value."

    γ is selected per dataset to give the 'best performance of each data' and K is selected as the value giving the best HAR test accuracy (Table 3). These selections use the same test labels that produce the reported 'Ours' entries in Table 1 and the ablation in Table 2. The reported accuracies are therefore maxima over a small hyperparameter search on the test set, not independent predictions. The claim that CBD 'nearly surpasses a bunch of baselines' is statistically forced by the selection procedure rather than confirmed by held-out evaluation.

full rationale

The core CIM and SER units are not circular: they are implemented as explicit complex-valued modulation and Bernstein-polynomial gated rescaling, and the invoked convolution theorem and Parseval theorem are standard external results. The self-citations to Han et al. (2023a, 2024) are motivational and are backed by independent rank-collapse references (Dong et al. 2021; Park and Kim 2022), so they are not load-bearing. However, two steps do reduce by construction. First, the 'dual-constraint loss' in Eq. (14) is, by the paper's own Theorem 2, exactly equal to the existing MSE reconstruction losses up to a fixed rescaling; it is the same loss renamed in Fourier coordinates, so the claimed new constraint is redundant. Second, the empirical superiority claim is compromised because γ is chosen per dataset for 'best performance of each data' and K is chosen from HAR test accuracy, with no validation split described; the reported test numbers are therefore fitted to the test set rather than predicted. Together these make the paper partially circular: one loss component is equivalent to its input by construction, and the headline empirical results are selected on the evaluation data. The CIM/SER mechanism itself retains independent content, so the score is moderate rather than maximal.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

No new physical or conceptual entities are introduced beyond the CIM and SER modules, which are computational components of the proposed architecture rather than postulated entities. Free parameters are limited to the two hyperparameters chosen on test performance; the neural network weights are ordinary trained parameters.

free parameters (2)
  • γ (dual-constraint loss weight) = per-dataset, values not reported
    Section B.2: 'the value of γ corresponding to the best performance of each data is slightly different', meaning γ is tuned toward best test accuracy per dataset.
  • K (Bernstein polynomial order) = 12
    Table 3 selects K by peak HAR linear-probing accuracy (K=12 gives 93.78%); the paper states 'we select K = 12 as the optimal value', and higher K is dismissed as 'unnecessary noise'.
assumptions (4)
  • standard math Frequency-domain convolution theorem (F[K⊗Z] = F[K]⊙F[Z])
    Used in CIM (Theorem 1) to claim that element-wise multiplication in the Fourier domain equals dynamic convolution, which is the basis for restricting interaction scope.
  • standard math Parseval's theorem: temporal energy equals spectral energy
    Used in SER (Theorem 2) to justify that adjusting frequency energy affects temporal encoding; proof in Appendix A.2.
  • domain assumption Transformer-based masked time-series encoders exhibit rank collapse and low-frequency spectral bias
    Motivating observation from Figure 1 and citations (Dong et al. 2021; Park and Kim 2022); the authors do not prove this holds for all MTM architectures and datasets.
  • ad hoc to paper The ideal energy-rebalance response G(·) is well-approximated by a Bernstein polynomial with coefficients linearly generated from the global energy vector
    Eq. (10)-(11) define pK with f(k/K) = (Wc A† + bc)_k; no proof that this functional form can realize the required per-frequency gains, only the Bernstein convergence argument for arbitrary continuous f on [0,1].

how reviews work

0 comments
Cite this review

Pith. "Pith review of Content-aware Balanced Spectrum Encoding in Masked Modeling for Time Series Classification." pith.science (2026). https://pith.science/paper/DE7DRZRB

@misc{pith2026241213232,
  author       = {Pith},
  title        = {Pith review of: Content-aware Balanced Spectrum Encoding in Masked Modeling for Time Series Classification},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DE7DRZRB}},
  note         = {Machine review of arXiv:2412.13232}
}
read the original abstract

Due to the superior ability of global dependency, transformer and its variants have become the primary choice in Masked Time-series Modeling (MTM) towards time-series classification task. In this paper, we experimentally analyze that existing transformer-based MTM methods encounter with two under-explored issues when dealing with time series data: (1) they encode features by performing long-dependency ensemble averaging, which easily results in rank collapse and feature homogenization as the layer goes deeper; (2) they exhibit distinct priorities in fitting different frequency components contained in the time-series, inevitably leading to spectrum energy imbalance of encoded feature. To tackle these issues, we propose an auxiliary content-aware balanced decoder (CBD) to optimize the encoding quality in the spectrum space within masked modeling scheme. Specifically, the CBD iterates on a series of fundamental blocks, and thanks to two tailored units, each block could progressively refine the masked representation via adjusting the interaction pattern based on local content variations of time-series and learning to recalibrate the energy distribution across different frequency components. Moreover, a dual-constraint loss is devised to enhance the mutual optimization of vanilla decoder and our CBD. Extensive experimental results on ten time-series classification datasets show that our method nearly surpasses a bunch of baselines. Meanwhile, a series of explanatory results are showcased to sufficiently demystify the behaviors of our method.

Figures

Figures reproduced from arXiv: 2412.13232 by the authors.

Figure 1
Figure 1. (a) Comparison of rank of the interaction ma [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. (a) Schematic illustration of our two-pronged framework. TED denotes temporal encoder, and TD and CBD represents [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 4
Figure 4. Energy rebalance of SER. The top row depicts the [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figures from the paper (4 more)
Figure 5
Figure 5. Figure 5: Learned Interaction Matrix on the PhonemeSpec [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]
Figure 6
Figure 6. Figure 6: Learned Interaction Matrix on the FingerMove [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]
Figure 7
Figure 7. Figure 7: Accuracy scores of HAR and PhonemeSpectra of [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]
Figure 8
Figure 8. Figure 8: T-SNE visualization of feature vectors on the HAR [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

45 extracted references · 26 canonical work pages

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Anguita, D.; Ghio, A.; Oneto, L.; Parra, X.; and Reyes-Ortiz, J. L. 2012. Human activity recognition on smartphones using a multiclass hardware-friendly support vector machine. In Ambient Assisted Living and Home Care: 4th International Workshop, IWAAL 2012, Vitoria-Gasteiz, Spain, December 3-5, 2012. Proceedings 4, 216--223. Springer

    work page 2012

  4. [4]

    A.; Lines, J.; Flynn, M.; Large, J.; Bostrom, A.; Southam, P.; and Keogh, E

    Bagnall, A.; Dau, H. A.; Lines, J.; Flynn, M.; Large, J.; Bostrom, A.; Southam, P.; and Keogh, E. 2018. The UEA multivariate time series classification archive, 2018. arXiv preprint arXiv:1811.00075

    arXiv 2018

  5. [5]

    M.; Grattarola, D.; Livi, L.; and Alippi, C

    Bianchi, F. M.; Grattarola, D.; Livi, L.; and Alippi, C. 2022. Graph Neural Networks With Convolutional ARMA Filters. IEEE Trans. Pattern Anal. Mach. Intell. , 44(7): 3496--3507

    work page 2022

  6. [6]

    Cheng, M.; Liu, Q.; Liu, Z.; Zhang, H.; Zhang, R.; and Chen, E. 2023. TimeMAE: Self-Supervised Representations of Time Series with Decoupled Masked Autoencoders. arXiv preprint arXiv:2303.00320

    arXiv 2023

  7. [7]

    Cortes, C.; and Vapnik, V. 1995. Support-vector networks. Machine learning, 20: 273--297

    work page 1995

  8. [8]

    A.; Bagnall, A.; Kamgar, K.; Yeh, C.-C

    Dau, H. A.; Bagnall, A.; Kamgar, K.; Yeh, C.-C. M.; Zhu, Y.; Gharghabi, S.; Ratanamahatana, C. A.; and Keogh, E. 2019. The UCR time series archive. IEEE/CAA Journal of Automatica Sinica, 6(6): 1293--1305

    work page 2019

Show all 45 references
  1. [9]

    Devlin, J.; Chang, M.-W.; Lee, K.; and Toutanova, K. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805

    2018 arXiv

  2. [10]

    Dong, J.; Wu, H.; Zhang, H.; Zhang, L.; Wang, J.; and Long, M. 2023. SimMTM: A Simple Pre-Training Framework for Masked Time-Series Modeling. arXiv preprint arXiv:2302.00861

    2023 arXiv

  3. [11]

    Dong, Y.; Cordonnier, J.; and Loukas, A. 2021. Attention is not all you need: pure attention loses rank doubly exponentially with depth. In Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event , volume 139 of Proceedin...

    2021

  4. [12]

    K.; Li, X.; and Guan, C

    Eldele, E.; Ragab, M.; Chen, Z.; Wu, M.; Kwoh, C. K.; Li, X.; and Guan, C. 2021. Time-series representation learning via temporal and contextual contrasting. arXiv preprint arXiv:2106.14112

    2021 arXiv

  5. [13]

    Han, D.; Pan, X.; Han, Y.; Song, S.; and Huang, G. 2023 a . FLatten Transformer: Vision Transformer using Focused Linear Attention. In IEEE/CVF International Conference on Computer Vision, ICCV 2023, Paris, France, October 1-6, 2023 , 5938--5948. IEEE

    2023

  6. [14]

    Han, Y.; Guo, Y.; Yin, J.; Liu, M.; Hu, Y.; and Nie, L. 2021. Focal and Composed Vision-semantic Modeling for Visual Question Answering. In Shen, H. T.; Zhuang, Y.; Smith, J. R.; Yang, Y.; C \' e sar, P.; Metze, F.; and Prabhakaran, B., eds., MM '21: ACM Multimedia Conference,...

    2021

  7. [15]

    Han, Y.; Hu, Y.; Song, X.; Tang, H.; Xu, M.; and Nie, L. 2024. Exploiting the Social-Like Prior in Transformer for Visual Reasoning. In AAAI, 2058--2066. AAAI Press

    2024

  8. [16]

    Han, Y.; Yin, J.; Wu, J.; Wei, Y.; and Nie, L. 2023 b . Semantic-Aware Modular Capsule Routing for Visual Question Answering. TIP, 32: 5537--5549

    2023

  9. [17]

    He, K.; Chen, X.; Xie, S.; Li, Y.; Doll \'a r, P.; and Girshick, R. 2022. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 16000--16009

    2022

  10. [18]

    Hu, Y.; Liu, M.; Su, X.; Gao, Z.; and Nie, L. 2021 a . Video Moment Localization via Deep Cross-Modal Hashing. IEEE Trans. Image Process. , 30: 4667--4677

    2021

  11. [19]

    Hu, Y.; Nie, L.; Liu, M.; Wang, K.; Wang, Y.; and Hua, X. 2021 b . Coarse-to-Fine Semantic Alignment for Cross-Modal Moment Localization. IEEE Trans. Image Process. , 30: 5933--5943

    2021

  12. [20]

    Hu, Y.; Wang, K.; Liu, M.; Tang, H.; and Nie, L. 2024. Semantic Collaborative Learning for Cross-Modal Moment Localization. ACM Trans. Inf. Syst. , 42(2): 50:1--50:26

    2024

  13. [21]

    Huang, Z.; Zhang, Z.; Lan, C.; Zha, Z.; Lu, Y.; and Guo, B. 2023. Adaptive Frequency Filters As Efficient Global Token Mixers. In IEEE/CVF International Conference on Computer Vision, ICCV 2023, Paris, France, October 1-6, 2023 , 6026--6036. IEEE

    2023

  14. [22]

    Liang, Z.; Zhang, J.; Liang, C.; Wang, H.; Liang, Z.; and Pan, L. 2023. Contrastive Shapelet Learning for Unsupervised Multivariate Time Series Representation Learning. CoRR, abs/2305.18888

    2023 arXiv

  15. [23]

    Liu, J.; and Chen, S. 2024. TimesURL: Self-Supervised Contrastive Learning for Universal Time Series Representation Learning. In Wooldridge, M. J.; Dy, J. G.; and Natarajan, S., eds., Thirty-Eighth AAAI Conference on Artificial Intelligence, AAAI 2024, Thirty-Sixth Conference ...

    2024

  16. [24]

    H.; Sinthong, P.; and Kalagnanam, J

    Nie, Y.; Nguyen, N. H.; Sinthong, P.; and Kalagnanam, J. 2022. A time series is worth 64 words: Long-term forecasting with transformers. arXiv preprint arXiv:2211.14730

    2022 arXiv

  17. [25]

    H.; Sinthong, P.; and Kalagnanam, J

    Nie, Y.; Nguyen, N. H.; Sinthong, P.; and Kalagnanam, J. 2023. A Time Series is Worth 64 Words: Long-term Forecasting with Transformers. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023 . OpenReview.net

    2023

  18. [26]

    Ozyurt, Y.; Feuerriegel, S.; and Zhang, C. 2022. Contrastive learning for unsupervised domain adaptation of time series. arXiv preprint arXiv:2206.06243

    2022 arXiv

  19. [27]

    Park, N.; and Kim, S. 2022. How Do Vision Transformers Work? In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022 . OpenReview.net

    2022

  20. [28]

    Z.; DeVito, Z.; Raison, M.; Tejani, A.; Chilamkurthy, S.; Steiner, B.; Fang, L.; Bai, J.; and Chintala, S

    Paszke, A.; Gross, S.; Massa, F.; Lerer, A.; Bradbury, J.; Chanan, G.; Killeen, T.; Lin, Z.; Gimelshein, N.; Antiga, L.; Desmaison, A.; K \" o pf, A.; Yang, E. Z.; DeVito, Z.; Raison, M.; Tejani, A.; Chilamkurthy, S.; Steiner, B.; Fang, L.; Bai, J.; and Chintala, S. 2019. PyTo...

    2019

  21. [29]

    Qin, Z.; Zhang, P.; Wu, F.; and Li, X. 2021. FcaNet: Frequency Channel Attention Networks. In 2021 IEEE/CVF International Conference on Computer Vision, ICCV 2021, Montreal, QC, Canada, October 10-17, 2021 , 763--772. IEEE

    2021

  22. [30]

    A.; Bengio, Y.; and Courville, A

    Rahaman, N.; Baratin, A.; Arpit, D.; Draxler, F.; Lin, M.; Hamprecht, F. A.; Bengio, Y.; and Courville, A. C. 2019. On the Spectral Bias of Neural Networks. In Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, Californ...

    2019

  23. [31]

    Tonekaboni, S.; Eytan, D.; and Goldenberg, A. 2021. Unsupervised representation learning for time series with temporal neighborhood coding. arXiv preprint arXiv:2106.00750

    2021 arXiv

  24. [32]

    F.; Mehri, S.; Rostamzadeh, N.; Bengio, Y.; and Pal, C

    Trabelsi, C.; Bilaniuk, O.; Serdyuk, D.; Subramanian, S.; Santos, J. F.; Mehri, S.; Rostamzadeh, N.; Bengio, Y.; and Pal, C. J. 2017. Deep Complex Networks. CoRR, abs/1705.09792

    2017 arXiv

  25. [33]

    Van der Maaten, L.; and Hinton, G. 2008. Visualizing data using t-SNE. Journal of machine learning research, 9(11)

    2008

  26. [34]

    N.; Kaiser, .; and Polosukhin, I

    Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser, .; and Polosukhin, I. 2017. Attention is all you need. Advances in neural information processing systems, 30

    2017

  27. [35]

    Wang, Y.; Xu, Y.; Yang, J.; Wu, M.; Li, X.; Xie, L.; and Chen, Z. 2024. Graph-Aware Contrasting for Multivariate Time-Series Classification. arXiv:2309.05202

    2024 arXiv

  28. [36]

    Xu, Q.; Zhang, R.; Zhang, Y.; Wang, Y.; and Tian, Q. 2021. A fourier-based framework for domain generalization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 14383--14392

    2021

  29. [37]

    J.; Zhang, Y.; and Xiao, Y

    Xu, Z. J.; Zhang, Y.; and Xiao, Y. 2019. Training Behavior of Deep Neural Network in Frequency Domain. In Gedeon, T.; Wong, K. W.; and Lee, M., eds., Neural Information Processing - 26th International Conference, ICONIP 2019, Sydney, NSW, Australia, December 12-15, 2019, Proce...

    2019

  30. [38]

    J.; and Zhou, H

    Xu, Z. J.; and Zhou, H. 2021. Deep Frequency Principle Towards Understanding Why Deeper Learning Is Faster. In Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Elev...

    2021

  31. [39]

    Xu, Z.-Q. J. 2020. Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networks. Communications in Computational Physics, 28(5): 1746–1767

    2020

  32. [40]

    Yue, Z.; Wang, Y.; Duan, J.; Yang, T.; Huang, C.; Tong, Y.; and Xu, B. 2022. Ts2vec: Towards universal representation of time series. In Proceedings of the AAAI Conference on Artificial Intelligence, 8980--8987

    2022

  33. [41]

    Zerveas, G.; Jayaraman, S.; Patel, D.; Bhamidipaty, A.; and Eickhoff, C. 2021. A transformer-based framework for multivariate time series representation learning. In Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining, 2114--2124

    2021

  34. [42]

    Zhang, W.; Yang, L.; Geng, S.; and Hong, S. 2023 a . Self-Supervised Time Series Representation Learning via Cross Reconstruction Transformer. IEEE Transactions on Neural Networks and Learning Systems

    2023

  35. [43]

    Zhang, X.; Zhao, Z.; Tsiligkaridis, T.; and Zitnik, M. 2022. Self-supervised contrastive pre-training for time series via time-frequency consistency. Advances in Neural Information Processing Systems, 35: 3988--4003

    2022

  36. [44]

    Zhang, Y.; Zhu, H.; Song, Z.; Koniusz, P.; and King, I. 2023 b . Spectral feature augmentation for graph contrastive learning and beyond. In Proceedings of the AAAI Conference on Artificial Intelligence, 11289--11297

    2023

  37. [45]

    Zhou, T.; Ma, Z.; Wen, Q.; Wang, X.; Sun, L.; and Jin, R. 2022. FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting. In International Conference on Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA , volume 162 of Proceedi...

    2022

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.