REVIEW 4 major objections 4 minor 44 references
Combining Discrete Wavelet and Cosine Transforms for Efficient Sentence Embedding
T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read The paper proposes a parameter-free combination of discrete wavelet and cosine transforms that encodes sentences into fixed-size vectors of the word-embedding dimension, matching or exceeding larger baselines on several NLP tasks.
desk verdict A plausible, non-parameterized DWT-DCT sentence embedding that needs a fixed configuration and variance reporting before its headline gains over DCT-only can be trusted. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the conjoint DWT-DCT transform, which first applies a discrete wavelet transform row-wise to the sentence matrix (each row is a word vector), splitting every word into approximation and detail coefficient vectors at one or more levels, and then applies a discrete cosine transform column-wise across the words of the sentence; the final sentence vector is formed by concatenating one or two DCT coefficients from each coefficient matrix, with a skipping trick that keeps the total dimension equal to the word-embedding dimension $d$ instead of $Kd$. This two-stage pipeline is what lets the method compress both the intra-word structure (via DWT) and the inter-word order (via DCT) into a fixed-size vector.
What would settle it
Permute the coordinate order of every word embedding with one shared random permutation and rerun the reported experiments. If downstream accuracy stays close to the unpermuted case, the wavelet stage is not exploiting the local ordering of dimensions; if accuracy collapses to random-pooling levels, the local-correlation assumption is the load-bearing premise.
Extended reading notes
Core claim
The central discovery is that word embeddings behave enough like signals with locally correlated dimensions that a fixed wavelet low-pass/high-pass split can separate salient semantic content from detail, and the resulting coefficients can be summarized across a sentence with DCT without needing learned parameters. The authors show that Level-1 approximation (cA) and detail (cD) coefficients, at half the dimension, match or exceed original embeddings on word-similarity tasks, and that combining them with DCT produces sentence vectors of dimension d (the word-embedding size) that outperform DCT-only vectors of dimension Kd and even match or beat averaging on most tasks. They also report qualitative evidence that low-pass coefficients capture category-level commonalities such as animals or emotions, while high-pass coefficients preserve contrasts such as gender or age.
Load-bearing premise
The method assumes that the dimensions of a word embedding are locally correlated with their neighbors in the vector, so that fixed wavelet low-pass filters that average nearby coordinates preserve the semantics needed for downstream tasks; the paper states it does not prove this correlation.
Editorial extensions
If this is right
- Sentence embedding size stays at the word-embedding dimension $d$ instead of growing as $Kd$, where $K$ is the number of DCT coefficients, so downstream memory and computation drop sharply for the same information.
- Because the transform is parameter-free, it applies directly to any pretrained word embeddings without training, making it a drop-in compression for existing pipelines.
- DWT alone compresses word embeddings to 50–75% of their original size while preserving word-similarity and categorization performance, which is useful when storage or retrieval speed is the bottleneck.
- On several tasks (CR, SST2, MRPC, STS12), the DWT-DCT model outperforms both averaging and DCT-only baselines, indicating that the two transforms capture complementary information rather than redundant signal.
Reading between the lines
- The paper does not compare against learned pooling operators of the same size; a natural next test would be to replace the fixed wavelet filters with a small learned 1-D convolution, which would show whether the fixed basis is near-optimal or leaves performance on the table.
- Because the method fixes the sentence vector to the word dimension, it could be used as a cheap input projection for long-document or sequence models, potentially reducing the cost of self-attention over many tokens.
- The qualitative split between approximation coefficients (which capture category-level similarity) and detail coefficients (which preserve contrasts) suggests that a task-specific weighting of the two channels could improve results further, an option the paper does not explore.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes using the Discrete Wavelet Transform (DWT) to compress word embeddings along their dimension axis, and combining DWT with the Discrete Cosine Transform (DCT) to produce fixed-size sentence embeddings. The method is deterministic and non-parameterized: it applies a mother wavelet row-wise to the sentence embedding matrix, then applies DCT column-wise over the resulting coefficient matrices, keeping one or two DCT coefficients. The authors evaluate the word-level embeddings on similarity and categorization tasks, and the sentence-level embeddings on SentEval classification and semantic textual similarity tasks, comparing against averaging, DCT, EigenSent, p-means, and VLAME baselines. They report that DWT-DCT matches or improves on DCT-only sentence embeddings at substantially lower dimensionality.
Significance. The core idea is simple, novel in combination, and, if validated, practically useful: a fixed-size sentence embedding that is cheap, deterministic, and does not require training. The paper's breadth of evaluation (word similarity, categorization, and nine downstream tasks) and the qualitative analyses are assets. However, the manuscript currently does not supply a reproducible configuration (the mother wavelet and scale are selected per task without a validation protocol), reports no variance or significance measures, and relies on external baseline numbers for the central comparison. These issues must be resolved before the central claim of 'comparable and even superior' performance can be accepted.
major comments (4)
- [§6.1, Tables 6 and 7] The central comparison in Table 7 is not based on a fixed configuration. Section 6.1 states that 'the specific details of MW and its application in each task' are omitted, and Table 6's caption indicates that the mother wavelet used per condition per task is shown in italics. Because the method is presented as non-parameterized (Section 3), the per-task choice of wavelet family, scale, number of DWT levels L, and the K=2 skip trick are effectively free choices made after seeing the results. Without a pre-registered or nested validation protocol that fixes these choices on development data only, the reported gains (e.g., CR 81.14 vs 79.81 in Table 7) may reflect test-set selection. Please report the exact wavelet configuration per task and a validation-based selection procedure.
- [§4.3 and §5.1, Tables 6 and 7] No error bars, confidence intervals, or significance tests are reported for any of the tables. The differences that support the headline claims are small: in Table 7, DWT1-DCT[0] beats DCT c[0] by 1.33 points on CR and 1.15 on SST2, while being 0.07 points worse on MR. Since SentEval classifiers are stochastic, these gaps are likely within run-to-run noise. In addition, the DCT baseline rows in Table 7 are taken from [2] ('as reported in [2]') rather than re-run in the same codebase, so differences in tokenization, classifier hyperparameters, or evaluation versions are uncontrolled. Please re-run all baselines in the same pipeline and report variance or bootstrap confidence intervals.
- [Table 6] Table 6 contains corrupted entries that prevent interpretation. The row A VG(cD) contains the value '76.61coif15' in the MR column, and the A VG(Random Pooling) row contains '8' for TREC and '33' for STS12, which are implausible accuracy and Pearson correlation values. These entries must be corrected or explained, and the table caption should be checked for the repeated model name 'A VG(cD+cAD+cAAD)'.
- [§6.2] The paper admits that 'we did not explicitly prove that the dimensions in the same embedding are correlated.' This correlation is the premise for applying a wavelet filter along the index order of embedding dimensions. The argument that DWT achieves comparable results therefore presupposes the conclusion. Please add a direct test, for example comparing DWT compression to the same transform applied to randomly permuted embedding dimensions and to a random projection of the same output dimension. Without such a control, the proposed mechanism for why DWT helps is not established.
minor comments (4)
- [§3] The description of the DWT coefficient matrices is confusing: for L=2, the text lists cDD, cDA, cAD, and cAA, but Section 4.2 uses 'cD+cAD' and 'cA+cDA' without defining how these matrices are concatenated. Please clarify the naming and the concatenation order.
- [§6] The conclusion refers to 'DCT[0:2]' but Table 7 only shows DCT c[0:1]. Please align the notation between the text and the table.
- [§4.3] The text states that averaging (AVG) with FastText embeddings is the baseline, but Table 6 also lists 'A VG(Random Pooling)' and 'A VG(PCA+PPA)'. Please clarify whether all baselines use FastText and how the random pooling condition was constructed.
- [Tables 1, 6, 7] The captions say that the best results are shown in red, but color is not visible in a printed or grayscale copy. Please use bold or another typographic distinction that survives monochrome printing.
Circularity Check
No derivation-level circularity; the only self-citation concern is the DCT baseline from [2], which is not load-bearing.
-
other
[Section 3 (K-coefficient choice) and Table 7 caption (DCT baseline source)]
"We finally encode a sentence vector of size d by concatenating either 1 or 2 DCT coefficients, which proved to be sufficient empirically and also consistent with observations in [2]. ... Results as opposed to DCT embedding, DCT c*, as reported in [2]."
The DCT baseline numbers and the K=1/2 coefficient convention are taken from [2], which shares an author (Mona Diab) with this paper. This is a self-citation signal, but it is not load-bearing: the DWT-DCT sentence vector is a fixed deterministic composition of DWT and DCT applied to the input word vectors, with no learned parameters and no target-label information entering the transform. The cited DCT values are an external benchmark, and the coefficient choice is a reused hyperparameter convention rather than a result forced by the transform definition. The reported DWT-DCT accuracies therefore do not reduce to this citation by construction; the concern is minor and primarily about benchmark provenance and reproducibility.
full rationale
The paper's central claim is empirical: applying a fixed Discrete Wavelet Transform row-wise to word vectors and then a Discrete Cosine Transform column-wise yields sentence embeddings that are competitive with or better than averaging and DCT baselines on SentEval tasks. The transform itself is a deterministic, non-parameterized function of the input embeddings, so the reported results are not equivalent to an input by construction and no fitted parameter is renamed as a prediction. The only same-group citation is [2], which supplies the DCT baseline values in Table 7 and the K=1/2 coefficient convention; this is benchmark reuse and a hyperparameter convention, not a load-bearing derivation. Section 6.2 explicitly concedes that dimension correlation is not proven, and Section 6.1 admits that per-task mother-wavelet details are omitted; these are reproducibility and validation limitations, not circular reasoning. Thus the derivation chain is self-contained against external benchmarks, with a minor non-load-bearing self-citation.
Assumptions & free parameters
free parameters (4)
- Mother wavelet family and scale =
Varies per task, e.g., 'coif15' in Table 6
- Number of DWT levels L =
1, 2, or 3 depending on task
- Number of DCT coefficients K =
1 or 2
- DWT coefficient skip factor for K=2 =
Skip every other coefficient
assumptions (3)
- domain assumption Word embedding dimensions are sufficiently correlated along their index ordering for a fixed wavelet low-pass filter to preserve semantic information.
- domain assumption The selected mother wavelet families (Coiflets, Daubechies, Symlets) are appropriate for the structure of embedding data.
- domain assumption DCT coefficients over word positions capture sentence-level semantic patterns useful for downstream tasks.
Cite this review
Pith. "Pith review of Combining Discrete Wavelet and Cosine Transforms for Efficient Sentence Embedding." pith.science (2026). https://pith.science/paper/ZEHBHLMY
@misc{pith2026250800420,
author = {Pith},
title = {Pith review of: Combining Discrete Wavelet and Cosine Transforms for Efficient Sentence Embedding},
year = {2026},
howpublished = {\url{https://pith.science/paper/ZEHBHLMY}},
note = {Machine review of arXiv:2508.00420}
}
read the original abstract
Wavelets have emerged as a cutting edge technology in a number of fields. Concrete results of their application in Image and Signal processing suggest that wavelets can be effectively applied to Natural Language Processing (NLP) tasks that capture a variety of linguistic properties. In this paper, we leverage the power of applying Discrete Wavelet Transforms (DWT) to word and sentence embeddings. We first evaluate, intrinsically and extrinsically, how wavelets can effectively be used to consolidate important information in a word vector while reducing its dimensionality. We further combine DWT with Discrete Cosine Transform (DCT) to propose a non-parameterized model that compresses a sentence with a dense amount of information in a fixed size vector based on locally varying word features. We show the efficacy of the proposed paradigm on downstream applications models yielding comparable and even superior (in some tasks) results to original embeddings.
Figures
Reference graph
Works this paper leans on
-
[2]
Efficient sentence embedding using dis- crete cosine transform
Nada Almarwani, Hanan Aldarmaki, and Mona Diab. Efficient sentence embedding using dis- crete cosine transform. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Process- ing (EMNLP-IJCNLP), pages 3672–3678, Hong Kong, China, November 2019. Association f...
work page 2019
-
[1]
SemEval-2012 task 6: A pilot on semantic textual similarity
Eneko Agirre, Daniel Cer, Mona Diab, and Aitor Gonzalez-Agirre. SemEval-2012 task 6: A pilot on semantic textual similarity. In *SEM 2012: The First Joint Conference on Lexical and Computational Semantics – Volume 1: Proceedings of the main conference and the shared task, and Volume 2: Proceedings of the Sixth International Workshop on Semantic Evaluation...
work page 2012
-
[3]
Attributes in lexical acquisition
Abdulrahman Almuhareb. Attributes in lexical acquisition. 2006
work page 2006
-
[4]
A simple but tough-to-beat baseline for sen- tence embeddings
Sanjeev Arora, Yingyu Liang, and Tengyu Ma. A simple but tough-to-beat baseline for sen- tence embeddings. January 2019. 5th International Conference on Learning Representations, ICLR 2017 ; Conference date: 24-04-2017 Through 26-04-2017
work page 2019
-
[5]
Don’t count, predict! a systematic comparison of context-counting vs
Marco Baroni, Georgiana Dinu, and Germ´ an Kruszewski. Don’t count, predict! a systematic comparison of context-counting vs. context-predicting semantic vectors. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 238–247, Baltimore, Maryland, June 2014. Association for Computational Li...
work page 2014
-
[6]
How we blessed distributional semantic evaluation
Marco Baroni and Alessandro Lenci. How we blessed distributional semantic evaluation. pages 1–10, 07 2011
work page 2011
-
[7]
The Wavelet Transform for Image Processing Appli- cations
Toufik Bouden and Mokhtar Nibouche. The Wavelet Transform for Image Processing Appli- cations. 04 2012
work page 2012
-
[8]
Multimodal distributional semantics
Elia Bruni, Nam Khanh Tran, and Marco Baroni. Multimodal distributional semantics. J. Artif. Int. Res., 49(1):1–47, January 2014
work page 2014
Show all 44 references
-
[9]
Brunton and J
Steven L. Brunton and J. Nathan Kutz. Data-Driven Science and Engineering: Machine Learning, Dynamical Systems, and Control. Cambridge University Press, 2019
2019
-
[10]
Signal processing and compression with wavelet packets
Ronald Coifman, Yves Meyer, Steven Quake, and Mladen Wickerhauser. Signal processing and compression with wavelet packets. Tech rep, 01 1991
1991
-
[11]
Senteval: An evaluation toolkit for universal sentence representations
Alexis Conneau and Douwe Kiela. Senteval: An evaluation toolkit for universal sentence representations. CoRR, abs/1803.05449, 2018
2018 arXiv
-
[12]
A wavelet-packets based algorithm for eeg signal compression
Juli´ an L C´ ardenas-Barrera, Juan V Lorenzo-Ginori, and Ernesto Rodr ´ ıguez-Valdivia. A wavelet-packets based algorithm for eeg signal compression. Medical Informatics and the Internet in Medicine, 29(1):15–27, 2004
2004
-
[13]
Ten Lectures on Wavelets
Ingrid Daubechies. Ten Lectures on Wavelets. Society for Industrial and Applied Mathematics, USA, 1992
1992
-
[14]
C ¸ . P. Dautov and M. S. ¨Ozerdem. Wavelet transform and signal denoising using wavelet method. In 2018 26th Signal Processing and Communications Applications Conference (SIU), pages 1–4, 2018
2018
-
[15]
Unsupervised construction of large paraphrase corpora: Exploiting massively parallel news sources
Bill Dolan, Chris Quirk, and Chris Brockett. Unsupervised construction of large paraphrase corpora: Exploiting massively parallel news sources. In COLING 2004: Proceedings of the 20th International Conference on Computational Linguistics, pages 350–356, Geneva, Switzerland, au...
2004
-
[16]
Placing search in context: The concept revisited
Lev Finkelstein, Evgeniy Gabrilovich, Yossi Matias, Ehud Rivlin, Zach Solan, Gadi Wolfman, and Eytan Ruppin. Placing search in context: The concept revisited. volume 20, pages 406– 414, 01 2001
2001
-
[17]
A. Graps. An introduction to wavelets. 1995
1995
-
[18]
Grgic, M
S. Grgic, M. Grgic, and B. Zovko-Cihlar. Performance analysis of image compression using wavelets. IEEE Transactions on Industrial Electronics, 48(3):682–695, 2001
2001
-
[19]
Recent advances in convolutional neural networks
Jiuxiang Gu, Zhenhua Wang, Jason Kuen, Lianyang Ma, Amir Shahroudy, Bing Shuai, Ting Liu, Xingxing Wang, and Gang Wang. Recent advances in convolutional neural networks. CoRR, abs/1512.07108, 2015
2015 arXiv
-
[20]
Simlex-999: Evaluating semantic models with (genuine) similarity estimation
Felix Hill, Roi Reichart, and Anna Korhonen. Simlex-999: Evaluating semantic models with (genuine) similarity estimation. CoRR, abs/1408.3456, 2014
2014 arXiv
-
[21]
Mining and summarizing customer reviews
Minqing Hu and Bing Liu. Mining and summarizing customer reviews. In KDD ’04, 2004
2004
-
[22]
Radu Tudor Ionescu and Andrei M. Butnaru. Vector of locally-aggregated word embeddings (vlawe): A novel document-level representation, 2019
2019
-
[23]
EigenSent: Spectral sentence embeddings using higher-order dynamic mode decomposition
Subhradeep Kayal and George Tsatsaronis. EigenSent: Spectral sentence embeddings using higher-order dynamic mode decomposition. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4536–4546, Florence, Italy, July 2019. Association ...
2019
-
[24]
A. S. Lewis and G. Knowles. Image compression using the 2-d wavelet transform. IEEE Transactions on Image Processing, 1(2):244–250, 1992
1992
-
[25]
A survey on wavelet applications in data mining
Tao Li, Qi Li, Shenghuo Zhu, and Mitsunori Ogihara. A survey on wavelet applications in data mining. SIGKDD Explorations, 4:49–68, 12 2002
2002
-
[26]
Word Embedding for Understanding Natural Language: A Survey, volume 26
Yang Li and Tao Yang. Word Embedding for Understanding Natural Language: A Survey, volume 26. 05 2017
2017
-
[27]
Madhavan
G. Madhavan. The illustrated wavelet transform handbook - introductory theory and ap- plications in science, engineering, medicine and finance [book review]. IEEE Engineering in Medicine and Biology Magazine, 22(1):92–93, 2003. 17
2003
-
[28]
Advances in pre-training distributed word representations
Tomas Mikolov, Edouard Grave, Piotr Bojanowski, Christian Puhrsch, and Armand Joulin. Advances in pre-training distributed word representations. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan, May 2018....
2018
-
[29]
Linguistic regularities in continuous space word representations
Tomas Mikolov, Wen-tau Yih, and Geoffrey Zweig. Linguistic regularities in continuous space word representations. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 746–751, A...
2013
-
[30]
All-but-the-top: Simple and effective postpro- cessing for word representations
Jiaqi Mu, Suma Bhat, and Pramod Viswanath. All-but-the-top: Simple and effective postpro- cessing for word representations. CoRR, abs/1702.01417, 2017
2017 arXiv
-
[31]
Selecting corpus-semantic models for neurolinguistic decoding
Brian Murphy, Partha Talukdar, and Tom Mitchell. Selecting corpus-semantic models for neurolinguistic decoding. In Proceedings of the First Joint Conference on Lexical and Compu- tational Semantics - Volume 1: Proceedings of the Main Conference and the Shared Task, and Volume ...
2012
-
[33]
A sentimental education: Sentiment analysis using subjectivity summarization based on minimum cuts
Bo Pang and Lillian Lee. A sentimental education: Sentiment analysis using subjectivity summarization based on minimum cuts. In Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics (ACL-04), pages 271–278, Barcelona, Spain, July 2004
2004
-
[34]
GloVe: Global vectors for word representation
Jeffrey Pennington, Richard Socher, and Christopher Manning. GloVe: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1532–1543, Doha, Qatar, October 2014. Association for Computatio...
2014
-
[35]
Effective dimensionality reduction for word embeddings
Vikas Raunak, Vivek Gupta, and Florian Metze. Effective dimensionality reduction for word embeddings. In Proceedings of the 4th Workshop on Representation Learning for NLP (RepL4NLP-2019), pages 235–243, Florence, Italy, August 2019. Association for Computa- tional Linguistics
2019
-
[36]
Rioul and M
O. Rioul and M. Vetterli. Wavelets and signal processing. IEEE Signal Processing Magazine, 8(4):14–38, 1991
1991
-
[37]
Concatenated p-mean word embeddings as universal cross-lingual sentence representations
Andreas R¨ uckl´ e, Steffen Eger, Maxime Peyrard, and Iryna Gurevych. Concatenated p-mean word embeddings as universal cross-lingual sentence representations. CoRR, abs/1803.01400, 2018
2018 arXiv
-
[38]
Chapter 10 - compression
Mallat St´ ephane. Chapter 10 - compression. In Mallat St´ ephane, editor, A Wavelet Tour of Signal Processing (Third Edition), pages 481 – 533. Academic Press, Boston, third edition edition, 2009
2009
-
[39]
Speech recognition by wavelet analysis
Nitin Trivedi, Vikesh Kumar, Saurabh Singh, Sachin Ahuja, and Raman Chadha. Speech recognition by wavelet analysis. International Journal of Computer Applications, 15:27–32, 2011
2011
-
[40]
Voorhees and Dawn M
Ellen M. Voorhees and Dawn M. Tice. Building a question answering test collection. In Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and De- velopment in Information Retrieval, SIGIR ’00, page 200–207, New York, NY, USA, 2000. Association for Com...
2000
-
[41]
Bin Wang, Angela Wang, Fenxiao Chen, Yuncheng Wang, and C.-C. Jay Kuo. Evaluating word embedding models: methods and experimental results. APSIPA Transactions on Signal and Information Processing, 8:e19, 2019
2019
-
[42]
On the dimensionality of sentence embed- dings
Hongwei Wang, Hongming Zhang, and Dong Yu. On the dimensionality of sentence embed- dings. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Findings of the Association 18 for Computational Linguistics: EMNLP 2023, pages 10344–10354, Singapore, December 2023. Association ...
2023
-
[43]
Z. Xizhi. The application of wavelet transform in digital image processing. In 2008 Interna- tional Conference on MultiMedia and Information Technology, pages 326–329, 2008
2008
-
[44]
An edge detection approach based on directional wavelet transform
Zhen Zhang, Siliang Ma, Hui Liu, and Yuexin Gong. An edge detection approach based on directional wavelet transform. Computers and Mathematics with Applications, 57(8):1265 – 1271, 2009
2009
-
[45]
Sentence analogies: Exploring linguistic relationships and regularities in sentence embeddings, 2020
Xunjie Zhu and Gerard de Melo. Sentence analogies: Exploring linguistic relationships and regularities in sentence embeddings, 2020. 19
2020
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.