S2Sent: Nested Selectivity Aware Sentence Representation Learning

Hui Liu; Jianxiang Zang; Meiling Ning; Nijia Mo; Yonda Wei

REVIEW 4 major objections 5 minor 46 references

S2Sent: Nested Selectivity Aware Sentence Representation Learning

T0 review · 4 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A small learned selector that fuses several Transformer blocks via DCT-based frequency selection improves sentence representations on semantic similarity benchmarks, the paper reports.

desk verdict A plausible lightweight cross-block fusion module for sentence embeddings, with a real but weakly quantified 1–2 point STS gain; the evaluation needs error bars and a less aggressive prose. read the letter →

arxiv 2508.18164 v1 pith:G3DWQORI submitted 2025-08-25 cs.CL

Jianxiang Zang , Nijia Mo , Yonda Wei , Meiling Ning , Hui Liu This is my paper

classification cs.CL

keywords sentencerepresentationlearningcontrastivecross-blockfusionspatialselectionfrequencydiscretecosinetransformsemantictextualsimilarityself-gating

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

S2Sent argues that sentence embeddings do not have to come from the last Transformer block alone. The paper inserts a small learned selector after the encoder that takes hidden states from several blocks, compresses them with low-frequency discrete-cosine-transform features, and fuses them with per-block adaptive weights. On seven semantic similarity tasks, this raises the average Spearman correlation over last-block baselines, for example from 76.21 to 78.50 for BERT with SimCSE, using under 0.5 percent additional parameters and roughly 0.2 percent extra inference latency. The claims imply that the same encoder can be made better by deciding which layers contribute to a sentence embedding, rather than by always using the final layer.

What carries the argument

The load-bearing identity is that global average pooling is exactly the lowest-frequency, or DC, coefficient of the two-dimensional discrete cosine transform. The FS module splits the stacked block-token feature tensor along feature channels and compresses each part with one of the m lowest DCT basis functions, replacing GAP's single frequency with several low frequencies. The SS module then feeds the squeezed feature vector through bottleneck fully connected layers and a softmax to produce per-block, per-feature adaptive weights, which are used to combine the hidden states. This nested selection is the mechanism that the paper credits with reducing both information redundancy and semantic l

What would settle it

Build a similarity test set whose pairs differ chiefly by high-frequency linguistic contrasts, such as negation, antonymy, or word order, and compare S2Sent against its GAP-only variant; if the low-frequency DCT selection smooths away those contrasts and the reported STS gain shrinks or reverses, the load-bearing assumption is wrong.

Watch

Extended reading notes

Core claim

The central claim is that rational cross-block fusion makes better use of a Transformer encoder's semantic information than taking the last layer. S2Sent models the sequence of block hidden states as a three-dimensional tensor and learns, per feature dimension, how much each block should contribute. The weights come from a squeeze-and-excitation self-gating path: the tensor is spatially squeezed by selected low-frequency 2D DCT basis functions, passed through a bottleneck, and normalized with softmax, then used to mix the original block states. Because the DCT is an extension of global average pooling, the frequency-selection step preserves more of each channel's variation during the squeeze

Load-bearing premise

The method assumes that meaning relevant to sentence similarity lives mostly in the low-frequency DCT components of the stacked block features, so the high-frequency bands can be discarded without losing information that matters.

Editorial extensions

If this is right

Appending the selector to an existing encoder and training the same contrastive objective should improve sentence-similarity scores without a new pretraining stage or a change to the upstream architecture.
The GAP-as-DC identity is general: the same spatial-squeeze-then-DCT recipe could replace GAP in channel-attention modules in other layered encoders.
The reported insensitivity to block count and frequency count means practical deployments can choose generous ranges without per-task hyperparameter search.
The paper explicitly leaves open whether generative LLM sentence embeddings could support this kind of dynamic cross-block fusion.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The fixed low-frequency DCT bands are chosen before seeing a sentence, so a natural untested extension is to make the selected bands adaptive per sentence or normalized by sentence length.
The learned per-block weights could serve as an interpretability probe: inspecting which blocks receive high weight for different inputs would show where similarity-relevant semantics actually sits in the encoder.
The appendix's collapse under [CLS] pooling suggests the fusion only keeps its meaning when the pooled representation is a linear function of token embeddings, a constraint worth testing against other pooling schemes.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Zang et al. propose S2Sent, a downstream module that fuses hidden states from multiple Transformer blocks for sentence embeddings, using a squeeze-excitation-style spatial gate with a DCT-based multi-spectral squeeze inside. The core idea is new as a combination: using several low-frequency DCT basis functions rather than GAP for the squeeze, and nesting this inside a cross-block selector. The DCT-as-GAP identity is mathematically correct, and the abstraction that GAP throws away higher-frequency information is fair. If the reported gains are real, the module is useful: it adds less than 0.5% parameters, negligible latency, and improves average STS by roughly 1–2 Spearman points over strong baselines like SimCSE and PromptCSE. That is a modest but legitimate contribution to the sentence representation subfield.

The weaknesses are mostly in the empirical reporting. Table 2 reports 7-seed averages and claims t-test significance, but gives no standard deviations, p-values, or confidence intervals. Some cells actually drop relative to the base (e.g., BERT Last1 SICK-R 71.98 vs 72.19), so the abstract's 'significant improvements across all experimental sets' is not supported by the paper's own numbers. More importantly, Figure 6 explores n and m across multiple values and the headline configuration appears to be selected from that search on the same STS test sets. That makes the +2.29 gain on BERT-SimCSE an upper bound, not an unbiased estimate. The authors should report per-seed numbers, apply some multiple-comparison awareness, and ideally evaluate on a held-out set or with a pre-registered configuration.

Appendix C is a weak spot: the gradient derivation is informal and seems to contain non-sequiturs. It should either be made rigorous or removed. No code is released, which matters because the implementation details of the nested squeeze are nontrivial to reproduce exactly.

The paper's central design idea is not obviously wrong. The DCT-based squeeze is a reasonable extension of SE-style attention, and the low-frequency bias assumption is plausible, though only tested on STS. The citation pattern is mostly fine, including the authors' own prior work; self-citation is not a problem when the cited modules are relevant. The relation to FcaNet is underdiscussed and should be acknowledged.

This paper is for people working on sentence embedding improvements, not a broad audience. It deserves a serious referee, but as a conditional accept: require error bars, code, a fix for the overclaims, and cleanup of Appendix C.

Referee Report

4 major / 5 minor

Summary. The paper proposes S2Sent, a lightweight downstream module that fuses hidden states from multiple Transformer blocks for sentence embedding learning. It combines a Spatial Selection (SS) module based on a squeeze-and-excitation self-gating mechanism with a nested Frequency Selection (FS) module that replaces global average pooling with selected low-frequency DCT basis functions. The authors evaluate S2Sent on seven STS tasks using BERT and RoBERTa backbones with SimCSE and PromptCSE training, reporting average Spearman gains over a Last1 baseline with negligible additional parameters and inference latency, along with transfer-task results and ablations over block count n, frequency basis count m, and reduction ratio r.

Significance. If the empirical results are robust, S2Sent would be a useful and easily integrable post-hoc fusion module for sentence representation pipelines. The paper's strengths are its clear modular design, the correct mathematical observation that GAP is a special case of DCT, systematic ablations over the main hyperparameters, and evaluation across four backbone/training combinations. The additional parameter and latency overheads are small. However, the central claim of 'significant improvements' currently rests on point estimates without error bars or significance statistics, and the headline configuration appears to be selected on the same STS test sets used for reporting. The low-frequency DCT assumption is also not independently validated. These issues need to be addressed before the main result can be considered established.

major comments (4)

[Section 5.1.1, Table 2] The paper states that results are averaged over 7 random seeds and that bold values are significant based on t-tests, but no standard deviations, t-statistics, p-values, or confidence intervals are reported. A claim of 'significant improvements across all experimental sets' is also contradicted by Table 2 itself: e.g., BERT/SimCSE Last1 S2Sent drops on SICK-R (71.98 vs 72.19), BERT/SimCSE Last6 Avg.+S2Sent drops on SICK-R (71.75 vs 72.21), and BERT/PromptCSE Last6 Avg.+S2Sent drops on SICK-R (69.82 vs 69.92). Please report the full distributional statistics and clarify which comparisons are significant, accounting for the 15+ configurations compared per backbone on the same test sets.
[Section 5.2.1, Figures 5 and 6] The main empirical result is vulnerable to test-set hyperparameter selection. Figure 5 compares GAP, Low-4, and Low-8 on the STS test sets, and Figure 6 explores n ∈ {1,3,6,9,12} and m ∈ {1,2,4,8,16} using the same test sets. The headline configuration (e.g., Last3/2D SS with a favorable m) is selected from these results, so the reported +2.29 Spearman gain over Last1 is a selection-adjusted upper bound, not an unbiased estimate. To support the strong conclusion in the abstract, the authors should choose n, m, and r on a validation split (e.g., STS-B dev) and report test results only for the final configuration, or use nested cross-validation / a proper multiple-comparison correction.
[Section 3.2.2, Eq. (10)] The FS module selects only the lowest m DCT basis functions (m ∈ {2,4,8,16}) based on the claim that neural networks are biased toward low-frequency information, citing Hu et al. (2018) and Xu et al. (2020), which are primarily computer-vision results. No textual or embedding-level analysis is provided to show that the discarded high-frequency components are semantically unimportant for sentence similarity. The only supporting evidence is the STS evaluation itself, which is affected by the hyperparameter-selection concern above. Please add a direct analysis (e.g., reconstruction error, ablations retaining high-frequency components, or a spectral analysis of sentence embeddings) to justify this design choice.
[Appendix C, Eq. (13)-(14)] The main text states that the motivation for differentiated gradient flow will be demonstrated, but the demonstration appears only in Appendix C and is not referenced in the main text where promised. The derivation is also difficult to follow: Eq. (13) is circular as written because ∂v/∂U appears on both sides, and the factor '2' in Eq. (14) is unexplained. Since this gradient-flow argument is used to motivate the SS design, please either move it to the main text with a clean, non-circular derivation or clearly label it as a heuristic observation.

minor comments (5)

[Section 3.2.2, Eq. (9)-(10) and Figure 4] The symbol M is undefined when the input is split into m parts (Eq. (9) says U[m] ∈ R^{N×L×D/M}). Additionally, Figure 4 does not specify the exact two-dimensional DCT index sets for m = 2,4,8,16; the phrase 'lowest m frequency basis functions' is ambiguous. Please provide the explicit index sets (e.g., zigzag order) for reproducibility.
[Section 4.1] The training description says 'For BERT, we used 384 groups, while for RoBERTa, we used 256 groups.' The term 'groups' is unclear; please specify what this number denotes (e.g., gradient accumulation steps, number of training samples, or something else).
[Section 5.2.2, Figure 7] The x-axis and y-axis of Figure 7 are not labeled in the text description; it is difficult to verify the claim that 'values of r ranging from 4 to 24 are all considered feasible.' Please add axis labels and mark the tested values of r.
[Section 7, Limitations] The limitations section mentions only the inapplicability to generative LLMs. Given the concerns about test-set hyperparameter selection and the low-frequency assumption, the limitations should also acknowledge these dependencies of the reported results, or the evidence should be strengthened in response.
[General] No code or data release is mentioned. Given the reproducibility-sensitive details of the FS module (DCT basis selection and hyperparameter choices), releasing code would substantially strengthen the paper.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the DCT-GAP identity and the learned fusion weights are independent of the reported STS improvements.

full rationale

S2Sent's derivation chain is self-contained. SS (Eqs. 1-4) defines a learned weighted fusion of block hidden states; the weights come from a trainable bottleneck MLP, not from the evaluation targets. FS (Eqs. 5-11) relies on the exact identity that GAP equals the zero-frequency 2D-DCT coefficient, which is a standard mathematical fact and is not fitted to STS results. The low-frequency selection is presented as a design choice motivated by external works (Hu et al. 2018; Xu et al. 2020) and is tested via ablation; it is not inferred from the target benchmarks. None of the authors' self-citations (e.g., Zang & Liu 2023a,b, 2024a,b; Mo et al. 2025) supplies a load-bearing uniqueness theorem or restricts alternative designs. The paper's empirical claims are comparisons against baselines; possible concerns about missing confidence intervals or test-set hyperparameter search are statistical robustness issues, not circularity. The Limitations section openly restricts scope to encoder models, further indicating the contribution is a concrete module, not a tautology.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The central claim does not introduce new physical or mathematical entities. It depends on standard math (DCT identity) and domain assumptions about layer semantics and low-frequency bias. The three hyperparameters n, m, and r are free choices that the paper partially tunes, weakening the 'no hyperparameter dependence' claim.

free parameters (3)

number of Transformer blocks n = Last 3 (default in main results; 1, 6, 9, 12 also tested)
A hand-chosen hyperparameter. The paper's own Figure 6 shows that n=1 performs worst, so the claim of insensitivity is only valid for n>1.
number of low-frequency DCT basis functions m = 4 or 8 (best in Figure 5; 1, 2, 16 tested)
The FS module needs a fixed m. The paper selects the value from experimental results, and m=1 (GAP) is worse, so the choice matters.
bottleneck reduction ratio r = Not specified; paper says range 4-24 is feasible
r controls the width of the excitation bottleneck. Figure 7 shows performance varies with r, so it is a free design choice that affects the module's capacity.

assumptions (5)

domain assumption Different Transformer blocks encode different semantic granularity, with lower layers capturing syntax and upper layers capturing semantics.
Invoked in Sections 1 and 2.1 via Tenney (2019) and Dai et al. (2022). If all layers encoded the same information, cross-block fusion would not help.
domain assumption Neural networks are biased toward low-frequency information, so selecting low-frequency DCT basis functions is sufficient for spatial squeeze.
State in Section 3.2.2 and cited to Hu et al. (2018) and Xu et al. (2020). The FS module's design relies on this; no task-specific evidence is given that high-frequency DCT components carry irrelevant information for STS.
standard math DCT basis functions form an extension of global average pooling, and the normalization constant can be omitted without changing the learned behavior.
Section 3.2.1, Equations 5-7. This is a standard signal-processing fact, but the omission of normalization means the exact transform used is a fixed-frequency linear projection.
domain assumption Squeeze-and-excitation with a Tanh bottleneck and Sigmoid excitation produces meaningful feature weights for sentence-level fusion.
Section 3.1.2. This borrows from vision models without a proof that the same mechanism transfers to sentence representations.
ad hoc to paper Average pooling of token embeddings is the correct pooling strategy for S2Sent; CLS pooling loses linearity under the weighted fusion.
Appendix A, Table 3 shows CLS pooling sharply degrades with S2Sent (e.g., BERT SimCSE avg 76.21 to 66.88). This limits the method to average pooling, a constraint not mentioned in the main limitations.

how reviews work

0 comments

Cite this review

Pith. "Pith review of S2Sent: Nested Selectivity Aware Sentence Representation Learning." pith.science (2026). https://pith.science/paper/G3DWQORI

@misc{pith2026250818164,
  author       = {Pith},
  title        = {Pith review of: S2Sent: Nested Selectivity Aware Sentence Representation Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/G3DWQORI}},
  note         = {Machine review of arXiv:2508.18164}
}

read the original abstract

The combination of Transformer-based encoders with contrastive learning represents the current mainstream paradigm for sentence representation learning. This paradigm is typically based on the hidden states of the last Transformer block of the encoder. However, within Transformer-based encoders, different blocks exhibit varying degrees of semantic perception ability. From the perspective of interpretability, the semantic perception potential of knowledge neurons is modulated by stimuli, thus rational cross-block representation fusion is a direction worth optimizing. To balance the semantic redundancy and loss across block fusion, we propose a sentence representation selection mechanism S\textsuperscript{2}Sent, which integrates a parameterized nested selector downstream of the Transformer-based encoder. This selector performs spatial selection (SS) and nested frequency selection (FS) from a modular perspective. The SS innovatively employs a spatial squeeze based self-gating mechanism to obtain adaptive weights, which not only achieves fusion with low information redundancy but also captures the dependencies between embedding features. The nested FS replaces GAP with different DCT basis functions to achieve spatial squeeze with low semantic loss. Extensive experiments have demonstrated that S\textsuperscript{2}Sent achieves significant improvements over baseline methods with negligible additional parameters and inference latency, while highlighting high integrability and scalability.

Figures

Figures reproduced from arXiv: 2508.18164 by the authors.

**Figure 1.** Overview of S2Sent. S2Sent builds a parameterized nested selector downstream of Transformer-based encoder, which performs a Spatial Selection (SS) nested with Frequency Selection (FS). and isotropic space (Li et al., 2020; Su et al., 2021; Zhuo et al., 2023). Contrastive learning methods, on the other hand, aim to tightly combine semantically similar examples while further separating different examples (Gao et al.,… view at source ↗

**Figure 2.** Traditional self-gate mechanism (consis [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 4.** Selection for m low frequency basis functions lated in Equation 10, where am and bm are the twodimensional indices of the frequency basis functions corresponding to U[m] . Previous studies (Hu et al., 2018; Xu et al., 2020) have shown that neural networks tend to be biased towards low-frequency information. Therefore, in the FS module, we select the lowest m frequency basis functions. In the experimental section,… view at source ↗

Figures from the paper (2 more)

**Figure 5.** Figure 5: Study for FS, evaluation results on STS tasks [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

**Figure 6.** Figure 6: The impact of n (left) and m (right) [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

46 extracted references · 31 canonical work pages

[1]

URL: " 'urlintro :=

ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...
[2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
[3]

Eneko Agirre, Carmen Banea, Claire Cardie, Daniel Cer, Mona Diab, Aitor Gonzalez-Agirre, Weiwei Guo, Inigo Lopez-Gazpio, Montse Maritxalar, Rada Mihalcea, et al. 2015. Semeval-2015 task 2: Semantic textual similarity, english, spanish and pilot on interpretability. In Proceedings of the 9th international workshop on semantic evaluation (SemEval 2015), pag...

work page 2015
[4]

Eneko Agirre, Carmen Banea, Claire Cardie, Daniel Cer, Mona Diab, Aitor Gonzalez-Agirre, Weiwei Guo, Rada Mihalcea, German Rigau, and Janyce Wiebe. 2014. Semeval-2014 task 10: Multilingual semantic textual similarity. In Proceedings of the 8th international workshop on semantic evaluation (SemEval 2014), pages 81--91

work page 2014
[5]

Eneko Agirre, Carmen Banea, Daniel Cer, Mona Diab, Aitor Gonzalez Agirre, Rada Mihalcea, German Rigau Claramunt, and Janyce Wiebe. 2016. Semeval-2016 task 1: Semantic textual similarity, monolingual and cross-lingual evaluation. In SemEval-2016. 10th International Workshop on Semantic Evaluation; 2016 Jun 16-17; San Diego, CA. Stroudsburg (PA): ACL; 2016....

2016
[6]

Eneko Agirre, Daniel Cer, Mona Diab, and Aitor Gonzalez-Agirre. 2012. Semeval-2012 task 6: A pilot on semantic textual similarity.* sem 2012: The first joint conference on lexical and computational semantics—. In Proceedings of the Sixth International Workshop on Semantic Evaluation (SemEval 2012), Montr \'e al, QC, Canada , pages 7--8

work page 2012
[7]

Eneko Agirre, Daniel Cer, Mona Diab, Aitor Gonzalez-Agirre, and Weiwei Guo. 2013. * sem 2013 shared task: Semantic textual similarity. In Second joint conference on lexical and computational semantics (* SEM), volume 1: proceedings of the Main conference and the shared task: semantic textual similarity, pages 32--43

work page 2013
[8]

Daniel Cer, Mona Diab, Eneko Agirre, I \ n igo Lopez-Gazpio, and Lucia Specia. 2017. Semeval-2017 task 1: Semantic textual similarity multilingual and crosslingual focused evaluation. In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), pages 1--14

work page 2017

Show all 46 references

[9]

Alexis Conneau and Douwe Kiela. 2018. Senteval: An evaluation toolkit for universal sentence representations. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018)

2018
[10]

Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Baobao Chang, and Furu Wei. 2022. Knowledge neurons in pretrained transformers. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8493--8502

2022
[11]

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human...

2019
[12]

Kawin Ethayarajh. 2019. How contextual are contextualized word representations? comparing the geometry of bert, elmo, and gpt-2 embeddings. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on N...

2019
[13]

Tianyu Gao, Xingcheng Yao, and Danqi Chen. 2021. Simcse: Simple contrastive learning of sentence embeddings. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6894--6910

2021
[14]

Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. 2020. Deberta: Decoding-enhanced bert with disentangled attention. arXiv preprint arXiv:2006.03654

2020 arXiv
[15]

Qibin Hou, Li Zhang, Ming-Ming Cheng, and Jiashi Feng. 2020. Strip pooling: Rethinking spatial pooling for scene parsing. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4003--4012

2020
[16]

Jie Hu, Li Shen, and Gang Sun. 2018. Squeeze-and-excitation networks. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pages 7132--7141

2018
[17]

Ting Jiang, Jian Jiao, Shaohan Huang, Zihan Zhang, Deqing Wang, Fuzhen Zhuang, Furu Wei, Haizhen Huang, Denvy Deng, and Qi Zhang. 2022. Promptbert: Improving bert sentence embeddings with prompts. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language P...

2022
[18]

Bohan Li, Hao Zhou, Junxian He, Mingxuan Wang, Yiming Yang, and Lei Li. 2020. On the sentence embeddings from pre-trained language models. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 9119--9130

2020
[19]

Xiang Li, Wenhai Wang, Xiaolin Hu, and Jian Yang. 2019. Selective kernel networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 510--519

2019
[20]

Yuxuan Li, Qibin Hou, Zhaohui Zheng, Ming-Ming Cheng, Jian Yang, and Xiang Li. 2023. Large selective kernel network for remote sensing object detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 16794--16805

2023
[21]

Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692

2019 arXiv
[22]

Marco Marelli, Luisa Bentivogli, Marco Baroni, Raffaella Bernardi, Stefano Menini, and Roberto Zamparelli. 2014. Semeval-2014 task 1: Evaluation of compositional distributional semantic models on full sentences through semantic relatedness and textual entailment. In Proceeding...

2014
[23]

Nijia Mo, Jianxiang Zang, Zhan Wang, and Hui Liu. 2025. Ddualse: Decoupled dual-head squeeze and excitation attention for sequential recommendation. In Proceedings of the Eighteenth ACM International Conference on Web Search and Data Mining, pages 300--308

2025
[24]

JI Nelson and BJ Frost. 1978. Orientation-selective inhibition from beyond the classic visual receptive field. Brain research, 139(2):359--365

1978
[25]

Marek Rei and Anders S gaard. 2019. Jointly learning to label sentences and tokens. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 6916--6923

2019
[26]

Michael P Sceniak, Dario L Ringach, Michael J Hawken, and Robert Shapley. 1999. Contrast's effect on spatial summation by macaque v1 neurons. Nature neuroscience, 2(8):733--739

1999
[27]

Joe Stacey, Yonatan Belinkov, and Marek Rei. 2022. Supervising model attention with human explanations for robust natural language inference. In Proceedings of the AAAI conference on artificial intelligence, volume 36, pages 11349--11357

2022
[28]

Jianlin Su, Jiarun Cao, Weijie Liu, and Yangyiwen Ou. 2021. Whitening sentence representations for better semantics and faster retrieval. arXiv preprint arXiv:2103.15316

2021 arXiv
[29]

I Tenney. 2019. Bert rediscovers the classical nlp pipeline. arXiv preprint arXiv:1905.05950

2019 arXiv
[30]

Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, and Furu Wei. 2024. Improving text embeddings with large language models. arXiv preprint arXiv:2401.00368

2024 arXiv
[31]

Benjamin Warner, Antoine Chaffin, Benjamin Clavi \'e , Orion Weller, Oskar Hallstr \"o m, Said Taghadouini, Alexis Gallagher, Raja Biswas, Faisal Ladhak, Tom Aarsen, et al. 2024. Smarter, better, faster, longer: A modern bidirectional encoder for fast, memory efficient, and lo...

2024 arXiv
[32]

Sanghyun Woo, Jongchan Park, Joon-Young Lee, and In So Kweon. 2018. Cbam: Convolutional block attention module. In Proceedings of the European Conference on Computer Vision (ECCV), pages 3--19

2018
[33]

Kai Xu, Minghai Qin, Fei Sun, Yuhao Wang, Yen-Kuang Chen, and Fengbo Ren. 2020. Learning in the frequency domain. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1740--1749

2020
[34]

Jianxiang Zang and Hui Liu. 2023 a . How to extract and interact? nested siamese text matching with interaction and extraction. In International Conference on Artificial Neural Networks, pages 523--535. Springer

2023
[35]

Jianxiang Zang and Hui Liu. 2023 b . Improving text semantic similarity modeling through a 3d siamese network. In ECAI 2023, pages 2970--2977. IOS Press

2023
[36]

Jianxiang Zang and Hui Liu. 2024 a . Explanation based bias decoupling regularization for natural language inference. In 2024 International Joint Conference on Neural Networks (IJCNN), pages 1--8. IEEE

2024
[37]

Jianxiang Zang and Hui Liu. 2024 b . Modeling selective feature attention for representation-based siamese text matching. arXiv preprint arXiv:2404.16776

2024 arXiv
[38]

Jianxiang Zang, Meiling Ning, Shihan Dou, Jiazheng Zhang, Tao Gui, Qi Zhang, and Xuanjing Huang. 2025 a . Mitigating attention hacking in preference-based reward modeling via interaction distillation. arXiv preprint arXiv:2508.02618

2025
[39]

Jianxiang Zang, Meiling Ning, Yongda Wei, Shihan Dou, Jiazheng Zhang, Nijia Mo, Binhong Li, Tao Gui, Qi Zhang, and Xuanjing Huang. 2025 b . Compression hacking: A supplementary perspective on informatics metric of language models from geometric distortion. arXiv preprint arXiv...

2025
[40]

Yan Zhang, Ruidan He, Zuozhu Liu, Kwan Hui Lim, and Lidong Bing. 2020. An unsupervised sentence embedding method by mutual information maximization. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1601--1610

2020
[41]

Yanzhao Zhang, Richong Zhang, Samuel Mensah, Xudong Liu, and Yongyi Mao. 2022. Unsupervised sentence representation via contrastive learning with mixing negatives. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 11730--11738

2022
[42]

Penghao Zhao, Hailin Zhang, Qinhan Yu, Zhengren Wang, Yunteng Geng, Fangcheng Fu, Ling Yang, Wentao Zhang, Jie Jiang, and Bin Cui. 2024. Retrieval-augmented generation for ai-generated content: A survey. arXiv preprint arXiv:2402.19473

2024 arXiv
[43]

Zhi Zheng, Wenshuo Chao, Zhaopeng Qiu, Hengshu Zhu, and Hui Xiong. 2024. Harnessing large language models for text-rich sequential recommendation. In Proceedings of the ACM Web Conference 2024, pages 3207--3216

2024
[44]

Dawei Zhu, Liang Wang, Nan Yang, Yifan Song, Wenhao Wu, Furu Wei, and Sujian Li. 2024. Longembed: Extending embedding models for long context retrieval. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 802--816

2024
[45]

Yutao Zhu, Huaying Yuan, Shuting Wang, Jiongnan Liu, Wenhan Liu, Chenlong Deng, Haonan Chen, Zheng Liu, Zhicheng Dou, and Ji-Rong Wen. 2023. Large language models for information retrieval: A survey. arXiv preprint arXiv:2308.07107

2023
[46]

Wenjie Zhuo, Yifan Sun, Xiaohan Wang, Linchao Zhu, and Yi Yang. 2023. Whitenedcse: Whitening-based contrastive learning of sentence embeddings. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 12135--12148

2023

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.

[1] [1]

URL: " 'urlintro :=

ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...

[2] [2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

[3] [3]

Eneko Agirre, Carmen Banea, Claire Cardie, Daniel Cer, Mona Diab, Aitor Gonzalez-Agirre, Weiwei Guo, Inigo Lopez-Gazpio, Montse Maritxalar, Rada Mihalcea, et al. 2015. Semeval-2015 task 2: Semantic textual similarity, english, spanish and pilot on interpretability. In Proceedings of the 9th international workshop on semantic evaluation (SemEval 2015), pag...

work page 2015

[4] [4]

Eneko Agirre, Carmen Banea, Claire Cardie, Daniel Cer, Mona Diab, Aitor Gonzalez-Agirre, Weiwei Guo, Rada Mihalcea, German Rigau, and Janyce Wiebe. 2014. Semeval-2014 task 10: Multilingual semantic textual similarity. In Proceedings of the 8th international workshop on semantic evaluation (SemEval 2014), pages 81--91

work page 2014

[5] [5]

Eneko Agirre, Carmen Banea, Daniel Cer, Mona Diab, Aitor Gonzalez Agirre, Rada Mihalcea, German Rigau Claramunt, and Janyce Wiebe. 2016. Semeval-2016 task 1: Semantic textual similarity, monolingual and cross-lingual evaluation. In SemEval-2016. 10th International Workshop on Semantic Evaluation; 2016 Jun 16-17; San Diego, CA. Stroudsburg (PA): ACL; 2016....

2016

[6] [6]

Eneko Agirre, Daniel Cer, Mona Diab, and Aitor Gonzalez-Agirre. 2012. Semeval-2012 task 6: A pilot on semantic textual similarity.* sem 2012: The first joint conference on lexical and computational semantics—. In Proceedings of the Sixth International Workshop on Semantic Evaluation (SemEval 2012), Montr \'e al, QC, Canada , pages 7--8

work page 2012

[7] [7]

Eneko Agirre, Daniel Cer, Mona Diab, Aitor Gonzalez-Agirre, and Weiwei Guo. 2013. * sem 2013 shared task: Semantic textual similarity. In Second joint conference on lexical and computational semantics (* SEM), volume 1: proceedings of the Main conference and the shared task: semantic textual similarity, pages 32--43

work page 2013

[8] [8]

Daniel Cer, Mona Diab, Eneko Agirre, I \ n igo Lopez-Gazpio, and Lucia Specia. 2017. Semeval-2017 task 1: Semantic textual similarity multilingual and crosslingual focused evaluation. In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), pages 1--14

work page 2017