{"id":"dcea35fb-b4d6-43f2-bfbb-141df244dafb","arxiv_id":"2505.11388","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"CompresSAE, a sparse autoencoder with cosine reconstruction loss, compresses dense embeddings 12x with a small retrieval quality drop and beats same-size Matryoshka embeddings online.","lead":"A team from Recombee shows that compressing dense embeddings into sparse high-dimensional vectors with a sparse autoencoder can cut memory use by 12x while keeping most retrieval quality. On a large online A/B test, their method outperformed Matryoshka compression of the same size by about 1.5% in click-through rate.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"A/B test confounds retrieval quality with downstream ranking: missing protocol details (candidate-set size, ranker retraining, offline recall definition) leave the +1.52% CTR claim unsupported as a retrieval-quality effect.","rationale":"The reader's weakest assumption correctly identifies the online A/B as measuring end-to-end CTR rather than retrieval quality. My stress-test confirms this is the most load-bearing concern: both quantitative claims (1.35% sacrifice and +1.52% superiority) are derived from that A/B test, and the offline recall results cannot substitute because the protocol is unspecified and error bars are absent. I add that the lack of candidate-set size and ranker-retraining details makes the ranker-confounding concern concrete: a fixed ranker trained on historical retrieval data can systematically favor one variant's candidate sets. A public offline recall benchmark with confidence intervals would settle the retrieval-quality question; candidate-set recall from the A/B logs would directly test the ranker-interaction hypothesis. This does not change the verdict: the claims are plausible and the online test is substantial evidence, but the missing details justify conditional acceptance. The released code is helpful, but the proprietary dataset prevents independent reproduction. Overall, the concern is valid but already captured by the conditional verdict.","tokens_in":6470,"tokens_out":10990,"duration_ms":113639,"concrete_test":"Release the evaluation protocol and re-run Figure 3 (center) on a public embedding corpus (e.g., Nomic embeddings of a public text dataset) with 5 random seeds, reporting mean and 95% CI recall@100 for CompresSAE k=32 and Matryoshka d=64. Additionally, from the production A/B logs, compute retrieval-only candidate-set recall (fraction of clicked items present in the candidate set) for each variant. If CompresSAE does not significantly exceed Matryoshka on both offline recall and candidate-set recall, the +1.52% CTR claim should be attributed to ranker interaction rather than retrieval quality.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central quantitative claims in Section 4 — that 12x-compressed CompresSAE loses only 1.35% CTR and significantly outperforms same-size Matryoshka by +1.52% — rest entirely on an online A/B test measuring downstream CTR after the full recommender pipeline. The paper does not report how users were randomized, how CTR was aggregated, whether the candidate-set size was equal across variants, or whether the downstream ranker was retrained for each variant. If the ranker is fixed and was trained on data generated by a different retrieval distribution, the observed CTR differences can reflect ranker miscalibration rather than retrieval quality. The offline experiment in Figure 3 (center) could have resolved this, but it reports Recall@100 on a proprietary dataset with no protocol description (ground truth, candidate pool, splits) and no error bars. Consequently, the claim that CompresSAE preserves retrieval quality and outperforms Matryoshka is not independently verifiable; the only evidence is an underspecified A/B test whose outcome can be confounded by the downstream ranking stage.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces CompresSAE, a sparse autoencoder that compresses dense embeddings into high-dimensional k-sparse vectors for candidate retrieval in large-scale recommender systems. The encoder maps a normalized dense embedding to a k-sparse latent vector, and a linear, row-normalized, bias-free decoder reconstructs the input; training minimizes cosine reconstruction loss, optionally with a multi-sparsity auxiliary term. The authors claim a 12x memory reduction over a 768-dimensional Nomic embedding with only a 1.35% CTR sacrifice, and report an online A/B test in which CompresSAE outperforms a same-size 64-dimensional Matryoshka variant by +1.52% CTR. They also describe a kernel-trick retrieval mode that computes similarities in the reconstructed space using the decoder Gram matrix with O(k^2) cost, and they release code. The main evidence is a proprietary offline experiment (Figure 3) and a large online A/B test (Figures 1 and 3, right).","tokens_in":6691,"tokens_out":4169,"duration_ms":44932,"significance":"If the claims hold, CompresSAE is a practically useful post-hoc compression method: it requires no retraining of the backbone encoder, is applicable to any pretrained embedding model, and reduces memory by an order of magnitude with a small measured CTR penalty. The paper's strengths include a released implementation, an online experiment with roughly 8.5 million users per variant, and a clean algebraic observation that reconstructed-space similarity can be computed from sparse codes via a precomputed kernel matrix. The method is also clearly positioned against Matryoshka-style truncation. However, the significance of the central quantitative claims depends on experimental details that are currently missing: the offline recall protocol is underspecified and lacks error bars, and the online A/B test does not report enough information to rule out downstream-ranker confounding. The paper is therefore potentially valuable, but the evidence as written is not yet sufficient to support the strongest statements.","major_comments":[{"comment":"The online A/B test is the load-bearing evidence for the +1.52% CTR claim and the 1.35% CTR sacrifice claim, but the protocol is not described. The paper does not state how users were randomized or whether the user was the randomization unit, how CTR was aggregated and over what time window, whether candidate-set size was equal across variants, or whether the downstream ranker was retrained or held fixed for each retrieval variant. If the ranker was trained on candidates from one retrieval distribution and then evaluated on candidates from another, the observed CTR differences could reflect ranker miscalibration rather than retrieval quality. Please report the full protocol, including confidence intervals for the treatment effects, and either control the candidate-set size or explicitly analyze how candidate-set composition interacts with the ranker.","section":"Section 4, Online Experiment (Figure 1, Figure 3 right)"},{"comment":"There is a mismatch between the training objective and the retrieval mode used in the online A/B test. The loss in Eq. (3) aligns the input x with the reconstructed vector x_hat = W_dec s, i.e., it optimizes cosine similarity in the reconstructed space. However, Section 4 states that the online experiment computes similarity 'directly in the sparse compressed space', i.e., with dot products between sparse codes s, not between reconstructions. These two similarities are equal only if the decoder Gram matrix K = W_dec^T W_dec is the identity (or if the sparse codes are suitably orthogonalized), which is not established. The paper itself shows in Figure 3 (center) that the reconstructed-space retrieval is the better mode, which suggests the direct sparse-space mode is not theoretically aligned with the training objective. The authors should either provide a theoretical justification relating the sparse-code dot product to the cosine loss, or report the online and offline results separately for direct sparse retrieval and reconstructed-space retrieval.","section":"Section 3, Eq. (3) vs. Section 4"},{"comment":"The central offline claim that 'CompresSAE can match the performance of Matryoshka models up to four times larger' is not verifiable in its current form. The figure reports Recall@100 on a proprietary dataset, but the paper does not describe the ground-truth definition, the candidate pool size, the query set, the split, or the number of evaluation items, and no error bars or repeated-run statistics are provided. The reader cannot tell whether the apparent advantage at high compression ratios is within noise. Please specify the evaluation protocol, report variability across runs or seeds, and state how the proprietary dataset is constructed.","section":"Section 4, Offline Experiments (Figure 3 center)"},{"comment":"The claim of statistical significance for the +1.52% effect is mentioned only in the text, and the confidence intervals in Figure 1 are said to be 'anytime-valid 99% confidence intervals' but are not defined or tabulated. The paper should state the exact interval for the comparison against Matryoshka of the same size and clarify the multiple-testing or sequential-testing procedure used, since anytime-valid intervals can depend on stopping rules.","section":"Section 4, Online Experiment (Figure 1)"}],"minor_comments":[{"comment":"The notation in Eq. (3) is slightly malformed; the loss should be written as a function of x and f(x; theta, k) with proper parentheses, e.g., L_cosine(x, f(x; theta, k)).","section":"Section 3.1"},{"comment":"There is a typo in 'PostgresSQL'; the correct project name is 'PostgreSQL'.","section":"Section 3.2"},{"comment":"The center panel is difficult to read: the x-axis 'Compression factor' is not labeled as logarithmic, and the mapping between the k values and the 'd=' annotations is unclear. Please add axis labels, a legend, and error bars or shaded bands.","section":"Figure 3 (center)"},{"comment":"The paper does not discuss how the hyperparameters h, k, and the loss-combination weights were chosen. A short sensitivity study or a statement of the default values and their selection procedure would improve reproducibility.","section":"Section 3"},{"comment":"The phrase 'statistically significantly outperforms Matryoshka of the same size by +1.52%' should be accompanied by the corresponding confidence interval in the same sentence, rather than only in a figure.","section":"Section 4"},{"comment":"The claim that SBERT is '8x larger' than the Nomic variant is based on dimensions (512 vs 64) but the comparison is between different embedding models; a brief clarification that this is a dimensionality comparison and not a model-parameter comparison would avoid confusion.","section":"Section 1"}],"recommendation":"major_revision","confidential_remarks":"The paper is best understood as an industry experience report, and the online A/B result is genuinely valuable evidence. The main risk is that the current manuscript overclaims retrieval-quality superiority from a downstream CTR metric without reporting the full experimental protocol. I believe the central idea is sound and the issues are fixable within the scope of a revision, so I recommend major revision rather than rejection. The authors should be encouraged to add the missing protocol details, error bars, and a direct comparison between sparse-space and reconstructed-space retrieval."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a solid industry paper with a real method and a genuinely impressive online experiment. CompresSAE is a sparse autoencoder trained with cosine reconstruction loss, top-k by absolute value, and a bias-free decoder; the kernel trick for similarity in reconstructed space is a neat algebraic identity that works. The design choices are well motivated for retrieval, and the paper is honest that it builds on prior SAE work [5, 21] while adapting it to a new setting. That is a legitimate contribution, and the code release helps.\n\nThe strongest evidence is the online A/B test: 8.5M users per variant, anytime-valid 99% CIs, and a +1.52% CTR gain over same-size Matryoshka with a 12x compression. That is real, hard-to-get evidence, and it deserves credit. The memory arithmetic is straightforward and the CSR storage claim checks out.\n\nNow the soft spots. The offline evaluation is the weakest link: Figure 3 center reports Recall@100 with no error bars, no description of the ground-truth construction, candidate pool, or splits, and the dataset is proprietary. That makes the offline comparison hard to verify, and the claim that CompresSAE matches Matryoshka models four times larger rests on a single unlabeled curve. The online A/B measures downstream CTR, not retrieval quality in isolation. The stress-test note is right that candidate-set changes can interact with the downstream ranker, and the paper does not report user randomization, metric aggregation, or whether the ranker was retrained. I would not call this a fatal flaw—the direction of the effect and the magnitude are plausible, and the authors do not overclaim causal isolation—but the paper should state the limitation explicitly and provide whatever protocol details they can. If they cannot release the data, they can at least release the evaluation code and a synthetic benchmark.\n\nWho is this for? Practitioners building large-scale retrieval systems, especially with embedding tables in the hundreds of GB range. Researchers in embedding compression will find the design choices and the A/B methodology useful. This paper deserves a serious referee: the method is new, the evidence is substantial, and the soft spots are fixable with better reporting rather than fundamental errors.\n\nRecommendation: peer review, with a request for offline evaluation details and a clear statement of A/B limitations.","headline":"CompresSAE is a sound, practically motivated compression method with strong online evidence, but the paper undersells its own offline protocol and overextends the A/B claim as a pure retrieval-quality result.","tokens_in":7221,"tokens_out":600,"would_cite":true,"duration_ms":7455,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that CompresSAE, a sparse autoencoder trained with cosine reconstruction loss, compresses dense embeddings 12x while preserving retrieval quality and beating same-size Matryoshka compression by +1.52% CTR in an online A/B…","keywords":["embedding compression","sparse autoencoder","candidate retrieval","recommender systems","cosine similarity","Matryoshka representation learning","click-through rate","memory efficiency"],"falsifier":"Run the online comparison again with the downstream ranker frozen and identical candidate-set sizes, logging CTR per user: if the +1.52% advantage over Matryoshka disappears or reverses, the central claim fails; offline, a public embedding dataset showing CompresSAE Recall@100 below a same-memory Matryoshka d=64 would also undermine it.","tokens_in":6318,"feed_emoji":"🗜️","tokens_out":8411,"duration_ms":77306,"temperature":0.7,"pith_summary":"Sparse autoencoders can compress large embedding tables in recommender systems far more aggressively than simple dimension truncation, without retraining the original embedding model. The paper's method, CompresSAE, maps a dense vector into a high-dimensional sparse vector with a fixed number of nonzero entries, for example 4096 dimensions with 32 nonzeros, and is trained to preserve cosine similarity instead of Euclidean distance. Stored in a sparse row format, a 768-dimensional embedding shrinks to one twelfth of its memory footprint. In an online test with about 8.5 million users per arm, the 12x compressed model lost only 1.35% of click-through rate relative to the uncompressed 768-dimensional embeddings, and outperformed a same-size Matryoshka-compressed embedding by +1.52% CTR. If these results hold, recommender systems facing strict memory budgets can use much richer frozen embeddings at a fraction of the storage cost.","feed_headline":"Sparse embedding compression beats Matryoshka in online CTR test","feed_subtitle":"A 4096-dim sparse vector with 32 nonzeros replaces a 768-dim dense vector at 12x lower memory cost.","key_machinery":"The load-bearing object is the CompresSAE sparse autoencoder, whose two design choices carry the argument. The first is the sparsification function $\\phi(\\cdot, k)$, which keeps the $k$ entries with largest absolute value rather than applying a ReLU and then a top-$k$, so negative directions survive; the second is training against cosine distance between input and reconstruction. The kernel trick for retrieval in the reconstructed space is the identity $K = W_{\\text{dec}}^{\\top}W_{\\text{dec}}$, which turns cosine similarity in the reconstructed dense space into $s_x^{\\top}K s_y / (\\sqrt{s_x^{\\top}K s_x}\\sqrt{s_y^{\\top}K s_y})$, computable in $O(k^2)$ using only the sparse codes. Because the decoder is bias-free, the same matrix $K$ works for every pair, so the method offers two inference modes: a fast $O(k)$ dot product directly in the sparse space, and a slower but more accurate $O(k^2)$ kernel-based mode aligned with the original dense space.","core_discovery":"On the paper's own terms, the central discovery is that a sparse autoencoder trained with a cosine reconstruction objective can almost perfectly distill a dense embedding's direction into a k-sparse high-dimensional code, and that this code retains enough information for candidate retrieval. Formally, the encoder $f_{\\text{enc}}(x) = \\phi(W_{\\text{enc}}x + b_{\\text{enc}}, k)$ keeps the $k$ entries with largest absolute values, and the decoder $f_{\\text{dec}}(s) = W_{\\text{dec}}s$ is linear and bias-free; the loss is $\\mathcal{L} = 1 - x^{\\top}\\hat{x}/(\\|x\\|_2\\|\\hat{x}\\|_2)$, combined for $k$ and $4k$ to avoid dead neurons. The paper reports that in production retrieval, a 4096-dimensional CompresSAE code with $k=32$ achieves a +3.44% CTR lift over the SBERT baseline at 25.6 GB per 100 million embeddings, while the uncompressed 768-dimensional Nomic model gives +4.86% at 307.2 GB and a same-size 64-dimensional Matryoshka gives +1.89%. That is the 12x memory reduction with a 1.35% CTR sacrifice against the uncompressed model, and a statistically significant +1.52% CTR advantage over Matryoshka of the same size.","pith_inferences":["A testable consequence not run in the paper: if cosine-loss, absolute-top-$k$ sparsification is what drives the gains, then ablating to ReLU-and-top-$k$ or to $\\ell_2$ reconstruction at constant sparsity should lower Recall@100 on a public embedding benchmark; without such an ablation, the attribution to direction preservation is an interpretation rather than a demonstrated mechanism.","Because the kernel mode uses $K = W_{\\text{dec}}^{\\top}W_{\\text{dec}}$ and only sparse vectors, one could build an approximate search index over the sparse codes that computes $K$-weighted similarities, potentially recovering dense-space ranking without ever materializing dense vectors.","CompresSAE is orthogonal to quantization, so combining it with 4-bit storage could compound the memory reduction well beyond 12x, but this combination is not tested in the paper.","The A/B comparison measures downstream CTR through the full recommender pipeline; if the downstream ranker re-ranks differently for different candidate sets, the +1.52% figure may not isolate retrieval quality. A cleaner test would fix the ranker and candidate-set size while varying only the embedding compressor."],"forward_implications":["A 768-dimensional dense embedding can be replaced by a 4096-dimensional 32-nonzero sparse vector, cutting memory from 307.2 GB to 25.6 GB per 100 million embeddings while keeping most of the retrieval quality.","CompresSAE can be applied on top of any frozen pretrained embedding model, because it trains only the autoencoder and does not require retraining the backbone encoder as Matryoshka-style compression does.","Retrieval cost in the sparse space depends on $k$, not on the embedding dimension, so high-dimensional sparse codes do not slow down dot-product search.","The reconstructed-space retrieval path gives a better accuracy-compression trade-off than direct sparse retrieval in the paper's offline experiments, at a modest $O(k^2)$ cost.","At equal memory, CompresSAE outperforms Matryoshka by +1.52% CTR in an online A/B test with about 8.5 million users per variant."],"supporting_citations":[{"why":"Supplies the SBERT baseline embeddings that the production retrieval replaced and against which CTR lifts are measured.","marker":"[18]"},{"why":"Supplies the Nomic embedding model that CompresSAE compresses in both offline and online experiments.","marker":"[14]"},{"why":"Defines Matryoshka Representation Learning, the same-size compression baseline that CompresSAE is claimed to outperform by +1.52% CTR.","marker":"[8]"},{"why":"Establishes the prior sparse-coding and sparse-autoencoder compression approach the paper builds on and contrasts with its cosine-loss design.","marker":"[21]"},{"why":"Supplies the sparse-autoencoder scaling and architecture conventions, including the multi-$k$ auxiliary loss, that CompresSAE adapts.","marker":"[5]"},{"why":"Defines the Yale sparse matrix format (CSR) used to store and compute dot products on the compressed embeddings.","marker":"[4]"},{"why":"Provides the HNSW approximate nearest-neighbor search used for exact or approximate top-$n$ retrieval in vector databases.","marker":"[13]"}],"fun_headline_variants":["Sparse autoencoder cuts memory 12x, beats Matryoshka CTR","4096-dim sparse code with 32 nonzeros lifts CTR 3.44% over SBERT","CompresSAE: 12x memory reduction, +1.52% CTR over Matryoshka","12x memory savings, sparse embeddings outperform Matryoshka"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The online A/B test assumes that the click-through-rate difference between the two compression methods comes from the compressed embeddings themselves, even though the full recommender pipeline can change how the downstream ranker responds to each candidate set.","fun_headline_variants_meta":{"raw":{"variants":["Sparse autoencoder cuts memory 12x, beats Matryoshka CTR","4096-dim sparse code with 32 nonzeros lifts CTR 3.44% over SBERT","CompresSAE: 12x memory reduction, +1.52% CTR over Matryoshka","12x memory savings, sparse embeddings outperform Matryoshka"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000861,"raw_usage":{"total_tokens":3743,"prompt_tokens":963,"completion_tokens":2780,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":579,"completion_tokens_details":{"reasoning_tokens":2687}},"tokens_in":579,"tokens_out":2780,"duration_ms":18942,"temperature":1.0,"reasoning_tokens":2687,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:53:27.837632+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the online comparison again with the downstream ranker frozen and identical candidate-set sizes, logging CTR per user: if the +1.52% advantage over Matryoshka disappears or reverses, the central claim fails; offline, a public embedding dataset showing CompresSAE Recall@100 below a same-memory Matryoshka d=64 would also undermine it.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the Yale sparse matrix format (CSR) used to store and compute dot products on the compressed embeddings."}],"review_version":1}