Pith. sign in

REVIEW 20 cited by

AnglE-optimized Text Embeddings

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2309.12871 v9 pith:4RFRWGDT submitted 2023-09-22 cs.CL cs.AIcs.LG

Xianming Li , Jing Li This is my paper
classification cs.CLcs.AIcs.LG
keywords angletextoptimizationcosineembeddingsaturationtasksangle-optimized
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

High-quality text embedding is pivotal in improving semantic textual similarity (STS) tasks, which are crucial components in Large Language Model (LLM) applications. However, a common challenge existing text embedding models face is the problem of vanishing gradients, primarily due to their reliance on the cosine function in the optimization objective, which has saturation zones. To address this issue, this paper proposes a novel angle-optimized text embedding model called AnglE. The core idea of AnglE is to introduce angle optimization in a complex space. This novel approach effectively mitigates the adverse effects of the saturation zone in the cosine function, which can impede gradient and hinder optimization processes. To set up a comprehensive STS evaluation, we experimented on existing short-text STS datasets and a newly collected long-text STS dataset from GitHub Issues. Furthermore, we examine domain-specific STS scenarios with limited labeled data and explore how AnglE works with LLM-annotated data. Extensive experiments were conducted on various tasks including short-text STS, long-text STS, and domain-specific STS tasks. The results show that AnglE outperforms the state-of-the-art (SOTA) STS models that ignore the cosine saturation zone. These findings demonstrate the ability of AnglE to generate high-quality text embeddings and the usefulness of angle optimization in STS.

Discussion (0). Sign in to comment.

Forward citations

Cited by 20 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. BEAVER: An Enterprise Benchmark for Text-to-SQL

    cs.CL 2024-09 unverdicted novelty 8.0 of 10

    BEAVER is the first text-to-SQL benchmark from private enterprise data warehouses, revealing SOTA agentic frameworks achieve only 10.8% accuracy on complex real-world queries.

  2. Will It Go Viral? Grounding Micro-Video Popularity Prediction on the Open Web

    cs.MM 2026-05 unverdicted novelty 7.0 of 10

    WEBSHORTS dataset and SHORTS-CAST framework ground micro-video popularity prediction in structured open-web context collected at upload time and enable selective online adaptation using delayed labels.

  3. External Data Extraction Attacks against Retrieval-Augmented Large Language Models

    cs.CR 2025-10 conditional novelty 7.0 of 10

    SECRET, an adaptive RAG extraction attack that fuses LLM-optimized jailbreak prompts with cluster-guided queries, extracts large portions of private databases from commercial and open LLMs, including ~35% of a sampled...

  4. Beyond Satisfaction: Learning Associations Between Content, Reviews, and Well-Being

    cs.SI 2026-06 conditional novelty 6.0 of 10

    Ratings and sentiment loosely track PERMA well-being facets in Goodreads reviews and align more with immediate hedonic than enduring eudaimonic expressions, while book themes like religion and drives correlate with hi...

  5. ReverseEOL: Improving Training-free Text Embeddings via Text Reversal in Decoder-only LLMs

    cs.CL 2026-06 unverdicted novelty 6.0 of 10

    ReverseEOL improves training-free text embeddings by combining forward and reversed-text representations from frozen decoder-only LLMs.

  6. On the Robustness of Multilingual Text Embedding Rankings Across Learning Tasks, Languages, and Benchmark Datasets

    cs.CL 2026-05 unverdicted novelty 6.0 of 10

    Meta-study of MTEB rankings introduces dataset-composition and ranking-scheme robustness indicators and finds only a small subset of models stay consistently strong across tasks, languages, and evaluation variations.

  7. Kernel Affine Hull Machines as Compute-Efficient Encoders for Frozen Semantic Spaces

    cs.LG 2026-05 unverdicted novelty 6.0 of 10

    Kernel Affine Hull Machines map lexical features to semantic embeddings via RKHS and least-mean-squares, outperforming adapters in reconstruction and retrieval metrics while reducing latency 8.5-fold on a legal benchmark.

  8. Kernel Affine Hull Machines as Compute-Efficient Encoders for Frozen Semantic Spaces

    cs.LG 2026-05 unverdicted novelty 6.0 of 10

    KAHM yields a compute-efficient query encoder that outperforms matched learned adapters in reconstructing a frozen Mixedbread embedding space on an Austrian-law retrieval task while delivering an 8.53x CPU speedup.

  9. SPENCE: A Syntactic Probe for Detecting Contamination in NL2SQL Benchmarks

    cs.CL 2026-04 unverdicted novelty 6.0 of 10

    SPENCE shows older NL2SQL benchmarks like Spider have high performance sensitivity to syntactic changes, indicating likely training contamination, while newer ones like BIRD show little sensitivity and appear largely clean.

  10. Beyond Cosine Similarity: Zero-Initialized Residual Complex Projection for Aspect-Based Sentiment Analysis

    cs.CL 2026-03 unverdicted novelty 6.0 of 10

    Zero-Initialized Residual Complex Projection with anti-collision masked angle loss reaches 0.8923 Macro-F1 on the ASAP dataset for aspect-based sentiment analysis.

  11. A Computational Method for Measuring "Open Codes" in Qualitative Analysis

    cs.CL 2024-11 unverdicted novelty 6.0 of 10

    A method merges codebooks via LLM and evaluates human and AI inductive coding with four new metrics on an online conversation dataset.

  12. NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Models

    cs.CL 2024-05 accept novelty 6.0 of 10

    NV-Embed achieves first place on the MTEB leaderboard across 56 tasks by combining a latent attention layer, causal-mask removal, two-stage contrastive training, and data curation for LLM-based embedding models.

  13. SHIFT: Self-reconstruction Harnesses Implicit Fine-grained Thinking for Retrieval

    cs.IR 2026-07 conditional novelty 5.0 of 10

    SHIFT trains LLM retrievers to reason through latent continuous tokens and reconstruct explicit reasoning traces, improving reasoning-intensive retrieval.

  14. ExploreGS: Explorable 3D Scene Reconstruction with Virtual Camera Samplings and Diffusion Priors

    cs.CV 2025-08 unverdicted novelty 5.0 of 10

    Adding information-gain-selected virtual views refined by video diffusion priors to 3D Gaussian Splatting improves arbitrary-view rendering quality.

  15. Towards Applying Large Language Models to Complement Single-Cell Foundation Models

    cs.LG 2025-07 conditional novelty 5.0 of 10

    A fusion model called scMPT, combining scGPT with an LLM text encoder, improves single-cell cell type classification on most tested datasets, and the paper shows the LLM relies on marker genes and simple expression patterns.

  16. Evaluating Document-Tuned Transformer Representations for Person-level Mental Health Assessment

    cs.CL 2026-06 unverdicted novelty 4.0 of 10

    Document-tuned transformers outperform base transformers by 13.4% Pearson r on person-level mental health prediction across two datasets and remain more accurate under text perturbations.

  17. Decoupling Semantics and Logic: A Training-Free Coarse-to-Fine Pipeline for Video Retrieval-Augmented Generation

    cs.CV 2026-06 unverdicted novelty 4.0 of 10

    A cascaded training-free Video RAG pipeline decouples high-recall semantic prefetching from LLM-driven logical reranking to improve precision on cross-lingual long-video tasks with persona constraints.

  18. A Framework for Deductive Semantic Content Analysis at Scale in Science Education Using Text Embeddings

    cs.CL 2025-08 conditional novelty 4.0 of 10

    A few-shot text embedding classification framework achieves high agreement with human coders (Cohen's Kappa 0.74-0.83) on a simulated exhaustive coding task over 2,899 physics education survey responses.

  19. Query pipeline optimization for cancer patient question answering systems

    cs.CL 2024-12 unverdicted novelty 4.0 of 10

    Three-aspect RAG query pipeline optimization for cancer patient QA introduces HSRDR and SEOS and reports 5.24% accuracy gain on Claude-3-haiku versus chain-of-thought on a custom dataset.

  20. Retrieval-Augmented Generation for Large Language Models: A Survey

    cs.CL 2023-12 unverdicted novelty 3.0 of 10

    A survey of RAG paradigms, components, benchmarks, and challenges for improving LLMs on knowledge-intensive tasks.

Pith tools