BITEMBED converts LLM backbones to ternary BitNet-style encoders, adapts them with contrastive pre-training and teacher distillation, and produces text embeddings at multiple precisions that perform comparably to full-precision baselines on MMTEB.
Bitnet distillation.arXiv preprint arXiv:2510.13998, 2025
2 Pith papers cite this work. Polarity classification is still indexing.
2
Pith papers citing it
years
2026 2verdicts
UNVERDICTED 2representative citing papers
Weight-quantized LLMs retain universal approximation up to 1.58 bits with expressive collapse below it and polynomial degradation in capacity as bit count falls.
citing papers explorer
-
BitNet Text Embeddings
BITEMBED converts LLM backbones to ternary BitNet-style encoders, adapts them with contrastive pre-training and teacher distillation, and produces text embeddings at multiple precisions that perform comparably to full-precision baselines on MMTEB.
-
On the Expressive Power of Weight Quantization in Large Language Models
Weight-quantized LLMs retain universal approximation up to 1.58 bits with expressive collapse below it and polynomial degradation in capacity as bit count falls.