Pith. sign in

REVIEW 4 major objections 6 minor 60 references

Semantic Compression for Word and Sentence Embeddings using Discrete Wavelet Transform

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Discrete wavelet transforms compress word and sentence embeddings by 50–93% with little or no loss in semantic quality.

desk verdict Genuinely new DWT-on-feature-axis compression with plausible 50% results, but the untested dimension-order premise and best-of-N test-set selection mean the headline claims outrun the evidence. read the letter →

arxiv 2508.00220 v1 pith:MEWMPXCU submitted 2025-07-31 cs.CL

Rana Aref Salama , Abdou Youssef , Mona Diab This is my paper
classification cs.CL
keywords discretewavelettransformembeddingcompressionsemanticsimilaritysentenceembeddingsworddimensionalityreductionspectralmethodslargelanguagemodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that the discrete wavelet transform (DWT), applied to the feature vector of a word or sentence embedding as if it were a one-dimensional signal, can cut the vector's size by 50 to 93 percent while keeping its semantic content essentially intact. Across word-similarity and sentence-similarity benchmarks, the compressed embeddings match the full-size originals within a few tenths of a point, and on most downstream classification tasks they are more accurate. This would make DWT a training-free, model-agnostic compression layer that works for static embeddings and large pre-trained language models alike. The authors interpret the result as showing that embedding dimensions are highly redundant and that wavelet approximations condense the core meaning while filtering out noise.

What carries the argument

The central object is the discrete wavelet transform applied along the feature axis of an embedding vector. The vector is treated as a one-dimensional signal, convolved with a low-pass and a high-pass filter, and down-sampled by two; recursively repeating this on the approximation coefficients produces level-2, -3, and -4 representations of one-quarter, one-eighth, and one-sixteenth the original size. The paper evaluates which coefficient set (level-1 approximation cA, level-1 detail cD, or deeper and combined coefficients) best preserves semantics for each task, using designed mother wavelets from the Haar, Daubechies, Symlets, and Coiflets families rather than learned filters.

What would settle it

Take a trained embedding model, randomly permute the feature dimensions identically across all words or sentences, and run the DWT compression pipeline on the permuted vectors; if the compressed representations lose most of their semantic and downstream performance while the original full vectors do not, the reported gains depend on accidental dimension ordering rather than on genuine redundancy. Measuring the performance drop across many random permutations would settle whether the central claim holds.

Watch

Extended reading notes

Core claim

The paper's central claim is that treating an embedding vector as a one-dimensional signal and applying a discrete wavelet transform yields compressed representations that preserve the semantics of the original. Keeping only the down-sampled approximation (cA) or detail (cD) coefficients at level 1 halves the dimension, and recursing to levels 2 through 4 shrinks it to a quarter, an eighth, or a sixteenth. On word-similarity benchmarks (SimLex-999, WS353, MEN) and sentence-similarity benchmarks (STS 2012–2016, STSB, SICK-R), the DWT embeddings score within a fraction of a point of the full-size baselines, and on most downstream classification tasks they score higher. The authors conclude that dimensions of modern embeddings are highly redundant and that approximation coefficients act as a denoised summary, while detail coefficients carry finer semantic distinctions.

Load-bearing premise

The method assumes that the ordering of dimensions inside an embedding is meaningful, so that nearby features are locally correlated and a low-pass filter can summarize them; the paper states this correlation has not been proven and is inferred only from the results.

Editorial extensions

If this is right

  • Embedding dimension can be halved by default with no retraining, cutting memory and inference cost by half for any existing model.
  • The same DWT layer applies to static and contextual embeddings alike, offering a universal preprocessing step that requires no fine-tuning.
  • Approximation coefficients double as a built-in denoising step, and on tasks such as STSB, MRPC, and SST5 the compressed vector outperforms the original.
  • Because larger pre-trained models compress better, the amount of redundant information in embeddings grows with model size, telling practitioners how much dimension is actually needed.
  • Choosing between cA, cD, or deeper levels gives a task-specific size-accuracy trade-off, with 75% compression staying within 2% of the baseline in most settings.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the local-order assumption is validated, DWT could be combined with a learned or random-but-fixed permutation of dimensions before transformation to unlock compression for models whose native feature ordering is not locally smooth.
  • The paper's qualitative nearest-neighbor examples suggest cA and cD capture different semantic facets; a testable extension is a fixed-length representation that concatenates both at half size and may beat either alone across tasks.
  • Because DWT is parameter-free and linear in dimension, it could serve as a cheap preprocessing step for embedding databases and retrieval systems, shrinking index memory while approximately preserving cosine ranking, which this paper does not directly evaluate.
  • The paper's claim that approximation coefficients suppress noise is testable by measuring out-of-distribution robustness of downstream classifiers trained on cA versus original vectors, an experiment the authors do not report.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The manuscript proposes applying the discrete wavelet transform (DWT) along the feature dimension of word and sentence embeddings and using selected approximation/detail coefficients at one or more decomposition levels as compressed embeddings. The method is evaluated on word similarity (SimLex-999, WS353, MEN), sentence similarity (STS 2012-2016, STSB, SICK-R), concept categorization, and SentEval downstream tasks, using GloVe, FastText, BERT, GPT, SBERT, RoBERTa, and InferSent embeddings, with PCA and DCT as baselines. The central reported claim is that DWT reduces embedding dimensionality by 50-93% with little or no loss on similarity tasks and with improved accuracy on most downstream tasks.

Significance. If the central claim survives scrutiny, this is a useful training-free, model-agnostic embedding-compression method with potential memory and speed benefits. The paper's positive aspects include public code, the use of multiple embedding families and benchmarks, a concrete comparison against DCT, and a candid acknowledgment in Section 7 that the local-correlation premise is not proven. However, the current evidence is weakened by post hoc selection of wavelets and coefficient sets on the test benchmarks and by the absence of a control that isolates the effect of local dimension order. The contribution is therefore real but conditional: it is an interesting empirical study whose headline claims need to be re-established under a fixed, non-test-fitting protocol.

major comments (4)
  1. [Section 5 and Section 8] The reported numbers are best-of-N summaries over mother wavelets and coefficient sets selected on the test benchmarks. Section 5 states that 'the best performance per task is recorded,' and Section 8 repeats that the authors 'employed Coiflets, Symlets, Haar, and Daubechies as MWs, with the recorded best overall results.' Thus Tables 1, 2, 5, 6, and 9 do not report the performance of a fixed DWT compression method; they report the maximum over a set of configurations evaluated on the same data used for the headline. The abstract's 'superior accuracy in most downstream tasks' is consequently not a falsifiable prediction of a concrete pipeline. The authors should either fix one wavelet and one coefficient-selection rule a priori, or use a held-out validation split to choose them, and then report the chosen configuration for each embedding and task.
  2. [Section 5 and Section 7] The core claim that DWT exploits local correlations among neighboring embedding dimensions is not tested. The paper applies the transform to the raw vector as provided by each model, and Section 7 concedes 'it has not been proven that features within the same embedding are correlated.' Because DWT's approximation coefficients are local low-pass averages over an arbitrary coordinate ordering, a necessary control is to apply the same DWT pipeline to randomly permuted feature dimensions (or an adversarially reordered vector) and show that compression quality degrades. Without such a control, the observed parity at 50% compression could be explained by the downstream tasks being insensitive to discarding half the coordinates, rather than by DWT-specific exploitation of structure. This is the decisive missing experiment.
  3. [Section 6.2, Table 6] The text claims that DWT 'effectively compresses the base embeddings, outperforming the baselines in all tasks' and that 'DWT embedding surpassed the baselines with all coefficients for all tasks except for CR and TREC where the Level-2 coefficients is comparable.' This is contradicted by Table 6. For RoBERTa Base, DWT cA is below baseline on MR (85.28 vs 85.32), SST2 (91.82 vs 92.31), SICK-E (79.16 vs 80.01), SUBJ (94.21 vs 94.71), and TREC (90.60 vs 92.40); DWT cD is below baseline on SICK-E (79.58 vs 80.01), SUBJ (94.38 vs 94.71), and TREC (90.41 vs 92.40). The abstract's 'superior accuracy in most downstream tasks' is at best true only for specific coefficient choices and mostly for RoBERTa Large. The text should be corrected to match the table and the claim should be qualified.
  4. [Tables 1, 2, 5, 6, and 9] The evaluation reports single runs without variance or significance testing. Many differences are within a few tenths of a point; for example, Table 2 shows BERT base DWTcD vs baseline on SimLex as 60.31 vs 60.75, and Table 5 shows SBERT base DWTcA vs baseline on STS12 as 74.26 vs 74.09. Without repeated runs, standard deviations, or paired significance tests, the 'almost no change' claim is not established. At minimum, the authors should report standard errors across multiple random seeds or bootstrap confidence intervals for the similarity correlations.
minor comments (6)
  1. [Table 1] The GloVe100 row appears twice, and the GloVe+DWTcA entry for SimLex (20.79 vs 12.22 for GloVe100) is a large unexplained jump; please verify whether this is a typo or a selected best configuration, and add a note explaining it.
  2. [Section 6.1.1] The sentence 'we use the combined coefficients; cA+cDA and cA+cDA' should read 'cA+cDA and cD+cAD'.
  3. [Table 4] Concept categorization results are reported as single numbers without specifying whether they are accuracy or F1, and without describing the classifier configuration; this should be stated in the text or caption.
  4. [Figure 2] The axes and units are not labeled, and the highlighted regions are hard to interpret; adding a legend and explicitly naming the word/token would improve reproducibility.
  5. [Throughout] There are several typos and inconsistencies: 'approxmation', 'V oorhees', 'publically', 'ad SICKR', and 'GloVe-Twitter27B' with an unspaced name. Also, Section 5 says the authors 'primarily use Symlets, Daubechies, and Coiflets,' while Section 8 adds Haar; please reconcile the list.
  6. [Equation (1)] The continuous wavelet definition is fine, but the paper never explains how it maps to the discrete filter-bank implementation used in Section 4; a short description of the low-pass/high-pass filter pair and downsampling step would help readers unfamiliar with DWT.

Circularity Check

1 steps flagged · score 4.0 of 10

Headline compression claim is partly post-selected: the wavelet family and coefficient set are chosen per task on the test benchmarks ('best performance per task is recorded'), so the reported 50-93% no-loss compression is a selected best-case, not a fixed-protocol prediction; the transform itself is external and not definitionally circular.

  1. fitted input called prediction [Section 4, footnote 1; Section 5 (Method)]
    "As a proof of concept in this paper, and in the interest of space, we will only be using and reporting on a subset of the MWs that yield the best results in our experiments. ... For the selection of the MW used in the transformation, we primarily use Symlets, Daubechies, and Coiflets wavelets across all experiments and the best performance per task is recorded."

    The abstract claims 'DWT can reduce the dimensionality of embeddings by 50-93% with almost no change in performance,' but this number is not produced by a single pre-specified compression protocol. The paper searches over a family of mother wavelets (Symlets, Daubechies, Coiflets) and multiple coefficient selections (cA, cD, cAA, cDA, cA+cDA, cD+cAD, up to four levels) and then records the best test-set result per task. The configuration is therefore fitted to the same benchmarks that are later reported as evidence for DWT's general efficacy, so the headline performance is a maximum over a configuration search rather than an independent prediction.

full rationale

DWT is a fixed, parameter-free linear transform and the evaluation uses external benchmarks with independent baselines (base embeddings, PCA, DCT), so the core empirical claim is not definitionally circular. The main circularity is the configuration search: Section 4 footnote 1 says the paper reports only wavelets 'that yield the best results in our experiments,' and Section 5 says 'the best performance per task is recorded.' Trying several wavelet families and coefficient sets and then reporting the best test-set numbers turns the headline 'DWT reduces dimensionality by 50-93% with almost no change' into a selected maximum rather than a prediction from a fixed protocol. This is a fitted-input-called-prediction pattern, though it stops short of reducing by construction because the transform and the benchmarks are independent of one another. The paper's Conclusion and Limitations also concede that the local-correlation premise is unproven ('it has not been proven that features within the same embedding are correlated') and infer it from the same experiments that motivate it; that is an explanatory caveat rather than a derivation-level circular step, so it does not raise the score further. The self-citation (Salama et al., 2024) is background and is not load-bearing for any central claim. Overall score 4 reflects partial circularity from post-hoc test-set selection while acknowledging the independent empirical content of the transform and the external evaluations.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The method introduces no learned parameters and no new physical or mathematical entities. The free parameters are experimental choices: wavelet family, coefficient set, and number of levels, all tuned per task on the test benchmarks. The load-bearing domain assumption, local ordering of embedding dimensions, is unproven by the paper's own admission in the Conclusion. This ledger shows that the paper adds an empirical recipe, not a new formalism.

free parameters (3)
  • Mother wavelet family and scale = not reported per experiment; best performance per task recorded
    Choice of mother wavelet controls the filter pair applied to the embedding. Sections 5 and 8 state that Coiflets, Symlets, Haar, and Daubechies were tried, with scale 4 to 17 mentioned, and the best results were kept without reporting the per-task configuration.
  • Coefficient set selection (cA, cD, cAA, cDA, combined) = not reported per task; chosen after seeing results
    The paper reports whichever coefficient set performed best per task, for example cA wins SimLex while cD wins WS353 in Table 1. This is a test-set-driven choice and is a free parameter of the evaluation protocol.
  • Number of DWT levels L = 1 to 4 levels, chosen per experiment
    The compression ratio, from 50% to 93%, is set by the number of recursive DWT levels and is treated as a task hyperparameter in Section 6.1.2 and Figure 4.
assumptions (4)
  • standard math DWT is an orthogonal linear transform whose approximation coefficients concentrate most of the signal energy
    Standard wavelet theory, invoked in Section 4 via (Daubechies, 1992; Madhavan, 2003), underlies the expectation that approximation coefficients retain the main structure of the vector.
  • domain assumption Embedding dimensions are locally ordered and correlated, so adjacent features form a meaningful 1-D signal
    The entire method rests on treating the embedding vector as a signal along its feature axis. Section 5 and Figure 2 motivate this visually, but the Conclusion admits that correlation between dimensions is not proven.
  • domain assumption Approximation coefficients retain the semantic content relevant to similarity and classification
    The paper assumes that low-frequency coefficients are the semantic part and that they 'encapsulate an abstracted representation' (Section 5), rather than deriving this property or ablating it against the detail coefficients.
  • ad hoc to paper Best-per-task selection of wavelet and coefficient set generalizes to unseen data
    The reported numbers are maxima over configurations evaluated on the test sets. The paper provides no validation procedure showing that this selection procedure does not overfit the benchmarks.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Semantic Compression for Word and Sentence Embeddings using Discrete Wavelet Transform." pith.science (2026). https://pith.science/paper/MEWMPXCU

@misc{pith2026250800220,
  author       = {Pith},
  title        = {Pith review of: Semantic Compression for Word and Sentence Embeddings using Discrete Wavelet Transform},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MEWMPXCU}},
  note         = {Machine review of arXiv:2508.00220}
}
read the original abstract

Wavelet transforms, a powerful mathematical tool, have been widely used in different domains, including Signal and Image processing, to unravel intricate patterns, enhance data representation, and extract meaningful features from data. Tangible results from their application suggest that Wavelet transforms can be applied to NLP capturing a variety of linguistic and semantic properties. In this paper, we empirically leverage the application of Discrete Wavelet Transforms (DWT) to word and sentence embeddings. We aim to showcase the capabilities of DWT in analyzing embedding representations at different levels of resolution and compressing them while maintaining their overall quality. We assess the effectiveness of DWT embeddings on semantic similarity tasks to show how DWT can be used to consolidate important semantic information in an embedding vector. We show the efficacy of the proposed paradigm using different embedding models, including large language models, on downstream tasks. Our results show that DWT can reduce the dimensionality of embeddings by 50-93% with almost no change in performance for semantic similarity tasks, while achieving superior accuracy in most downstream tasks. Our findings pave the way for applying DWT to improve NLP applications.

Figures

Figures reproduced from arXiv: 2508.00220 by the authors.

Figure 1
Figure 1. Applying DWT to an image. Level-1 DWT trans￾forms the image into approximation coefficients (that resemble the original image) and vertical, horizontal and diagonal detail coefficients. Level-2 DWT can be obtained recursively from Level-1 coefficients(multi-scale analysis). settings a challenge. Accordingly, in this paper, we investigate the adaptation of DWT from Signal and Image processing to the field of NLP for … view at source ↗
Figure 2
Figure 2. Graph of FastText embedding for the word ’work’(dim=300). Some features are similar in value (high￾lighted in red) and others exhibit significant spikes (high￾lighted in green). ity to perform multi-resolution analysis. DWT ef￾fectively addresses these limitations by examining data change over time or position, i.e. frequency and time localization, and consequently captures varied information across different scales… view at source ↗
Figure 3
Figure 3. Transforming the embedding vector in [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Accuracy results for 4-Levels of DWT for approx￾imation coefficients for downstream tasks using RoBERTa Large embeddings 0.09-1.5% except for SICK-E, SUBJ, and TREC, which experience degradation by 1.3%, 1%, and 7.4%, respectively, although still outperforming the base…
Figure 5
Figure 5. Figure 5: Similarity Matrix between 2 sentences using [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]
Figure 6
Figure 6. Figure 6: Similarity Matrix between 2 sentences using BERT embeddings of dimension 768, and Level-1 cA and [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

60 extracted references · 41 canonical work pages

  1. [1]

    Anish Acharya, Rahul Goel, Angeliki Metallinou, and Inderjit S. Dhillon. 2018. http://arxiv.org/abs/1811.00641 Online embedding compression for text classification using low rank matrix factorization . CoRR, abs/1811.00641

    work page Pith review arXiv 2018

  2. [2]

    Eneko Agirre, Carmen Banea, Claire Cardie, Daniel Cer, Mona Diab, Aitor Gonzalez-Agirre, Weiwei Guo, I \ n igo Lopez-Gazpio, Montse Maritxalar, Rada Mihalcea, German Rigau, Larraitz Uria, and Janyce Wiebe. 2015. https://doi.org/10.18653/v1/S15-2045 S em E val-2015 task 2: Semantic textual similarity, E nglish, S panish and pilot on interpretability . In P...

    doi:10.18653/v1/s15-2045 2015

  3. [3]

    Eneko Agirre, Carmen Banea, Claire Cardie, Daniel Cer, Mona Diab, Aitor Gonzalez-Agirre, Weiwei Guo, Rada Mihalcea, German Rigau, and Janyce Wiebe. 2014. https://doi.org/10.3115/v1/S14-2010 S em E val-2014 task 10: Multilingual semantic textual similarity . In Proceedings of the 8th International Workshop on Semantic Evaluation ( S em E val 2014) , pages ...

    doi:10.3115/v1/s14-2010 2014

  4. [4]

    Eneko Agirre, Carmen Banea, Daniel Cer, Mona Diab, Aitor Gonzalez-Agirre, Rada Mihalcea, German Rigau, and Janyce Wiebe. 2016. https://doi.org/10.18653/v1/S16-1081 Semeval-2016 task 1: Semantic textual similarity, monolingual and cross-lingual evaluation . In SemEval-2016, pages 497--511

    doi:10.18653/v1/s16-1081 2016

  5. [5]

    Eneko Agirre, Daniel Cer, Mona Diab, and Aitor Gonzalez-Agirre. 2012. https://aclanthology.org/S12-1051 S em E val-2012 task 6: A pilot on semantic textual similarity . In * SEM 2012: The First Joint Conference on Lexical and Computational Semantics -- Volume 1: Proceedings of the main conference and the shared task, and Volume 2: Proceedings of the Sixth...

    2012

  6. [6]

    Eneko Agirre, Daniel Cer, Mona Diab, Aitor Gonzalez-Agirre, and Weiwei Guo. 2013. https://aclanthology.org/S13-1004 * SEM 2013 shared task: Semantic textual similarity . In Second Joint Conference on Lexical and Computational Semantics (* SEM ), Volume 1: Proceedings of the Main Conference and the Shared Task: Semantic Textual Similarity , pages 32--43, A...

    work page 2013

  7. [7]

    Nada Almarwani, Hanan Aldarmaki, and Mona Diab. 2019. https://doi.org/10.18653/v1/D19-1380 Efficient sentence embedding using discrete cosine transform . In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3672--3678, Hong ...

    work page doi:10.18653/v1/d19-1380 2019

  8. [8]

    Abdulrahman Almuhareb. 2006. https://api.semanticscholar.org/CorpusID:40718894 Attributes in lexical acquisition . In Semantic Scholar

    work page 2006

Show all 60 references
  1. [9]

    Klaudia Ba azy, Mohammadreza Banaei, R \' e mi Lebret, Jacek Tabor, and Karl Aberer. 2021. https://doi.org/10.18653/v1/2021.repl4nlp-1.32 Direction is what you need: Improving word embedding compression in large language models . In Proceedings of the 6th Workshop on Represent...

    2021 doi

  2. [10]

    Marco Baroni, Georgiana Dinu, and Germ \'a n Kruszewski. 2014. https://doi.org/10.3115/v1/P14-1023 Don ' t count, predict! a systematic comparison of context-counting vs. context-predicting semantic vectors . In Proceedings of the 52nd Annual Meeting of the Association for Com...

    2014 doi

  3. [11]

    Marco Baroni and Alessandro Lenci. 2011. https://aclanthology.org/W11-2501 How we BLESS ed distributional semantic evaluation . In Proceedings of the GEMS 2011 Workshop on GE ometrical Models of Natural Language Semantics , pages 1--10, Edinburgh, UK. Association for Computati...

    2011

  4. [12]

    Rijeka : InTech, cop. 2012

    Toufik Bouden and Mokhtar Nibouche. 2012. https://doi.org/10.5772/35982 The Wavelet Transform for Image Processing Applications , chapter 1. "Rijeka : InTech, cop. 2012"

    2012 doi

  5. [13]

    Elia Bruni, Nam Khanh Tran, and Marco Baroni. 2014. Multimodal distributional semantics. J. Artif. Int. Res., 49(1):1–47

    2014

  6. [14]

    Brunton and J

    Steven L. Brunton and J. Nathan Kutz. 2019. https://doi.org/10.1017/9781108380690 Data-Driven Science and Engineering: Machine Learning, Dynamical Systems, and Control . Cambridge University Press

    2019 doi

  7. [15]

    Daniel Cer, Mona Diab, Eneko Agirre, I \ n igo Lopez-Gazpio, and Lucia Specia. 2017. https://doi.org/10.18653/v1/S17-2001 S em E val-2017 task 1: Semantic textual similarity multilingual and crosslingual focused evaluation . In Proceedings of the 11th International Workshop on...

    2017 doi

  8. [16]

    Alexis Conneau and Douwe Kiela. 2018. http://arxiv.org/abs/1803.05449 Senteval: An evaluation toolkit for universal sentence representations . CoRR, abs/1803.05449

    2018 arXiv

  9. [17]

    Alexis Conneau, Douwe Kiela, Holger Schwenk, Lo\" i c Barrault, and Antoine Bordes. 2017. https://www.aclweb.org/anthology/D17-1070 Supervised learning of universal sentence representations from natural language inference data . In Proceedings of the 2017 Conference on Empiric...

    2017

  10. [18]

    Ingrid Daubechies. 1992. Ten Lectures on Wavelets. Society for Industrial and Applied Mathematics, USA

    1992

  11. [19]

    Ç. P. Dautov and M. S. Özerdem . 2018. https://doi.org/10.1109/SIU.2018.8404418 Wavelet transform and signal denoising using wavelet method . In 2018 26th Signal Processing and Communications Applications Conference (SIU), pages 1--4

    2018

  12. [20]

    Jacob Devlin, Ming - Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. http://arxiv.org/abs/1810.04805 BERT: pre-training of deep bidirectional transformers for language understanding . CoRR, abs/1810.04805

    2018 arXiv

  13. [21]

    Bill Dolan, Chris Quirk, and Chris Brockett. 2004. https://www.aclweb.org/anthology/C04-1051 Unsupervised construction of large paraphrase corpora: Exploiting massively parallel news sources . In COLING 2004: Proceedings of the 20th International Conference on Computational Li...

    2004

  14. [22]

    Lev Finkelstein, Evgeniy Gabrilovich, Yossi Matias, Ehud Rivlin, Zach Solan, Gadi Wolfman, and Eytan Ruppin. 2001. https://doi.org/10.1145/503104.503110 Placing search in context: The concept revisited . In ACM Transactions on Information Systems - TOIS, volume 20, pages 406--414

    2001

  15. [23]

    Chaochen Gao, Xing Wu, Peng Wang, Jue Wang, Liangjun Zang, Zhongyuan Wang, and Songlin Hu. 2023. http://arxiv.org/abs/2112.05638 Distilcse: Effective knowledge distillation for contrastive sentence embeddings

    2023 arXiv

  16. [24]

    Grgic , M

    S. Grgic , M. Grgic , and B. Zovko-Cihlar . 2001. https://doi.org/10.1109/41.925596 Performance analysis of image compression using wavelets . IEEE Transactions on Industrial Electronics, 48(3):682--695

    2001 doi

  17. [25]

    Jiuxiang Gu, Zhenhua Wang, Jason Kuen, Lianyang Ma, Amir Shahroudy, Bing Shuai, Ting Liu, Xingxing Wang, and Gang Wang. 2015. http://arxiv.org/abs/1512.07108 Recent advances in convolutional neural networks . CoRR, abs/1512.07108

    2015 arXiv

  18. [26]

    Amit Kumar Garg

    Maneesha Gupta and Dr. Amit Kumar Garg. 2012. Analysis of image compression algorithm using dct. In https://api.semanticscholar.org/CorpusID:14997921

    2012

  19. [27]

    Felix Hill, Roi Reichart, and Anna Korhonen. 2014. http://arxiv.org/abs/1408.3456 Simlex-999: Evaluating semantic models with (genuine) similarity estimation . CoRR, abs/1408.3456

    2014 arXiv

  20. [28]

    Minqing Hu and Bing Liu. 2004. Mining and summarizing customer reviews. In KDD '04

    2004

  21. [29]

    Subhradeep Kayal and George Tsatsaronis. 2019. https://doi.org/10.18653/v1/P19-1445 E igen S ent: Spectral sentence embeddings using higher-order dynamic mode decomposition . In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4536...

    2019 doi

  22. [30]

    Yeachan Kim, Kang-Min Kim, and SangKeun Lee. 2020. Adaptive compression of word embeddings. In Annual Meeting of the Association for Computational Linguistics

    2020

  23. [31]

    A. S. Lewis and G. Knowles . 1992. https://doi.org/10.1109/83.136601 Image compression using the 2-d wavelet transform . IEEE Transactions on Image Processing, 1(2):244--250

    1992 doi

  24. [32]

    Hao Li, Asim Kadav, Igor Durdanovic, Hanan Samet, and Hans Peter Graf. 2017. http://arxiv.org/abs/1608.08710 Pruning filters for efficient convnets

    2017 arXiv

  25. [33]

    Shaoshi Ling, Yangqiu Song, and Dan Roth. 2016. https://doi.org/10.18653/v1/P16-2063 Word embeddings with limited memory . In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 387--392, Berlin, Germany. Asso...

    2016 doi

  26. [34]

    Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. http://arxiv.org/abs/1907.11692 Roberta: A robustly optimized BERT pretraining approach . CoRR, abs/1907.11692

    2019 arXiv

  27. [35]

    Madhavan

    G. Madhavan . 2003. The illustrated wavelet transform handbook - introductory theory and applications in science, engineering, medicine and finance [book review]. IEEE Engineering in Medicine and Biology Magazine, 22(1):92--93

    2003

  28. [36]

    Yihuan Mao, Yujing Wang, Chufan Wu, Chen Zhang, Yang Wang, Quanlu Zhang, Yaming Yang, Yunhai Tong, and Jing Bai. 2020. https://doi.org/10.18653/v1/2020.coling-main.287 L ada BERT : Lightweight adaptation of BERT through hybrid model compression . In Proceedings of the 28th Int...

    2020 doi

  29. [37]

    Marco Marelli, Stefano Menini, Marco Baroni, Luisa Bentivogli, Raffaella Bernardi, and Roberto Zamparelli. 2014. http://www.lrec-conf.org/proceedings/lrec2014/pdf/363_Paper.pdf A SICK cure for the evaluation of compositional distributional semantic models . In Proceedings of t...

    2014

  30. [38]

    Tomas Mikolov, Edouard Grave, Piotr Bojanowski, Christian Puhrsch, and Armand Joulin. 2018. https://www.aclweb.org/anthology/L18-1008 Advances in pre-training distributed word representations . In Proceedings of the Eleventh International Conference on Language Resources and E...

    2018

  31. [39]

    Brian Murphy, Partha Talukdar, and Tom Mitchell. 2012. Selecting corpus-semantic models for neurolinguistic decoding. In Proceedings of the First Joint Conference on Lexical and Computational Semantics - Volume 1: Proceedings of the Main Conference and the Shared Task, and Vol...

    2012

  32. [41]

    Bo Pang and Lillian Lee. 2004 b . https://doi.org/10.3115/1218955.1218990 A sentimental education: Sentiment analysis using subjectivity summarization based on minimum cuts . In Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics ( ACL -04) ...

    2004

  33. [42]

    Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. https://doi.org/10.3115/v1/D14-1162 G lo V e: Global vectors for word representation . In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing ( EMNLP ) , pages 1532--1543, Doh...

    2014 doi

  34. [43]

    Kamatchi Priya. 2014. A review on linear and non-linear dimensionality reduction techniques. Machine Learning and Applications: An International Journal

    2014

  35. [44]

    Computer Science, Linguistics

    Alec Radford and Karthik Narasimhan. 2018. Improving language understanding by generative pre-training. In "Computer Science, Linguistics"

    2018

  36. [45]

    Vikas Raunak, Vivek Gupta, and Florian Metze. 2019. https://doi.org/10.18653/v1/W19-4328 Effective dimensionality reduction for word embeddings . In Proceedings of the 4th Workshop on Representation Learning for NLP (RepL4NLP-2019), pages 235--243, Florence, Italy. Association...

    2019 doi

  37. [46]

    Rana Salama, Abdou Youssef, and Mona Diab. 2024. Combining discrete wavelet and cosine transforms for efficient sentence embedding. In 5th International Conference on Advanced Natural Language Processing (AdNLP 2024), Vancouver, Canada

    2024

  38. [47]

    Kaiyu Shi and Kai Yu. 2018. https://doi.org/10.21437/Interspeech.2018-1057 Structured Word Embedding for Low Memory Neural Network Language Model . In Proc. Interspeech 2018, pages 1254--1258

    2018 doi

  39. [48]

    Jonathon Shlens. 2014. A tutorial on principal component analysis. Educational, 51

    2014

  40. [49]

    Raphael Shu and Hideki Nakayama. 2017. http://arxiv.org/abs/1711.01068 Compressing word embeddings via deep compositional code learning . CoRR, abs/1711.01068

    2017 arXiv

  41. [50]

    Chaofan Tao, Lu Hou, Wei Zhang, Lifeng Shang, Xin Jiang, Qun Liu, Ping Luo, and Ngai Wong. 2022. http://arxiv.org/abs/2203.10705 Compression of generative pre-trained language models via quantization

    2022 arXiv

  42. [51]

    Martin Vetterli and Jelena Kovacevic. 1996. Wavelets and subband coding. Journal of Electronic Imaging

    1996

  43. [52]

    Voorhees and Dawn M

    Ellen M. Voorhees and Dawn M. Tice. 2000. https://doi.org/10.1145/345508.345577 Building a question answering test collection . In Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’00, page 200–207, N...

    2000

  44. [53]

    Bin Wang and C. C. Jay Kuo. 2020. http://arxiv.org/abs/2002.06652 Sbert-wk: A sentence embedding method by dissecting bert-based word models

    2020 arXiv

  45. [54]

    Bin Wang, C. C. Jay Kuo, and Haizhou Li. 2022. http://arxiv.org/abs/2203.02679 Just rank: Rethinking evaluation with word and sentence similarities

    2022 arXiv

  46. [55]

    Hongwei Wang, Hongming Zhang, and Dong Yu. 2023. https://doi.org/10.18653/v1/2023.findings-emnlp.694 On the dimensionality of sentence embeddings . In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 10344--10354, Singapore. Association for Computat...

    2023 doi

  47. [56]

    Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, and Furu Wei. 2024. http://arxiv.org/abs/2401.00368 Improving text embeddings with large language models

    2024 arXiv

  48. [57]

    Z. Xizhi . 2008. The application of wavelet transform in digital image processing. In 2008 International Conference on MultiMedia and Information Technology, pages 326--329

    2008

  49. [58]

    Zhen Zhang, Siliang Ma, Hui Liu, and Yuexin Gong. 2009. https://doi.org/https://doi.org/10.1016/j.camwa.2008.11.013 An edge detection approach based on directional wavelet transform . Computers and Mathematics with Applications, 57(8):1265 -- 1271

    2009 doi

  50. [59]

    Xunjie Zhu and Gerard de Melo. 2020. http://arxiv.org/abs/2003.04036 Sentence analogies: Exploring linguistic relationships and regularities in sentence embeddings

    2020 arXiv

  51. [60]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before...

  52. [61]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.