{"id":"7e2bc10b-bd0a-4e3a-816f-c5b98fd09f2a","arxiv_id":"2506.18150","paper_version":4,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Client-side digit decomposition and block-diagonal packing enable server-side private embedding lookups in FHE, cutting embedding-lookup latency by up to 56x versus CodedHeLUT and enabling end-to-end encrypted DLRM inference in 24-489 seconds on a single CPU.","lead":"This paper introduces HE-LRM, a system that runs recommendation-model inference on encrypted data by having the client encode sparse features as digit-decomposed one-hot vectors, so the server performs all embedding lookups under fully homomorphic encryption without seeing the model's embedding tables.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central missing evidence is compressed-model accuracy: no test AUC or uncompressed baseline is reported, so the 56x speedup and end-to-end latencies may not translate to a usable DLRM.","rationale":"The reader's verdict is CONDITIONAL, with the weakest assumption being that compressed models retain acceptable accuracy. My stress-test finds this to be the most load-bearing concern. The FHE machinery—client-side digit decomposition, block-diagonal packing, and BSGS matrix-vector multiplication—is technically coherent, and the latency comparisons are plausible as far as they go. However, the end-to-end claim is only meaningful if the compressed DLRM actually makes good predictions. The paper reports no test AUC values, no uncompressed baseline, and no accuracy-quality threshold. Because the compression ratios are enormous (up to 31180x), the accuracy issue is not a minor detail; it determines whether the reported latencies apply to a deployable model. I therefore agree with the reader's identification of this gap, and the proposed test—recomputing AUC for all compression levels against an uncompressed baseline—would directly settle it. Since the reader already made the verdict CONDITIONAL for this reason, my stress-test does not change the verdict.","tokens_in":22139,"tokens_out":6998,"duration_ms":81475,"concrete_test":"Run the released CAFE training code for the uncompressed Criteo DLRM and for each threshold in Table 2 (500, 5000, 50000, 500000, 5000000), under both ReLU and SiLU, and report validation AUC on day 7 with identical preprocessing and averaging over 3 seeds. If any compressed model's AUC falls more than 1 point (or a pre-specified tolerance) below the uncompressed baseline, the practical-value claim of HE-LRM is not established.","verdict_should_be":"UNCHANGED","load_bearing_attack":"HE-LRM's practical claim rests on the compressed DLRM being accurate enough to deploy. Section 5.1 introduces client-side digit decomposition as the core compression, and Table 2 shows compression ratios up to 31180x. Yet Section 6.2 only states that 'more compressed models exhibit slightly higher loss' and 'SiLU models consistently show lower test AUC', with no AUC numbers, no uncompressed baseline, and no CI. Figure 9 shows training curves but not test AUC. Section 6.3 reports FHE latencies without tying them to a quality target. If the AUC drop at 31180x compression is large—for example, more than a point or two on Criteo—then the claimed 56x speedup and 228-489s latencies are achieved on a model that is not practically useful. This is not an internal inconsistency but a load-bearing omission: the paper's title and abstract promise an 'encrypted deep learning recommendation model', and the model's predictive value is part of that promise. The reader's weakest assumption identifies exactly this gap.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents HE-LRM, a fully homomorphic encryption (FHE) based inference pipeline for Deep Learning Recommendation Models (DLRMs) that supports both dense and sparse features entirely under encryption. The main technical contributions are a client-side digit-decomposition scheme that compresses large embedding tables into smaller base-p tables, and a block-diagonal multi-embedding packing strategy that allows multiple embedding lookups to be performed in a single SIMD matrix-vector product. The approach is implemented in the open-source Orion FHE framework and evaluated on the UCI Heart Disease and Criteo datasets, reporting 24.22 seconds and roughly 228-489 seconds end-to-end CPU latencies. The paper also discusses extending the embedding-lookup primitives to LLM prefill and generation, arguing that all prior CKKS-based transformer inference works leak embedding tables by performing client-side lookups. The claimed headline result is a 56x speedup over the prior CodedHeLUT approach, enabled by reducing the embedding lookup to a single multiplicative level and avoiding encrypted indicator functions and bootstraps.","tokens_in":22321,"tokens_out":4461,"duration_ms":51711,"significance":"If the results hold, HE-LRM is a meaningful step toward practical private DLRM inference: it is, to my knowledge, the first end-to-end FHE DLRM implementation that keeps embedding tables server-side and performs the entire lookup under encryption, and the reported latencies on standard CPU hardware are substantially better than prior encrypted-lookup baselines. The paper offers a clean, reproducible system contribution: the code is open-sourced, the benchmark numbers are averaged over three runs, and the comparison against CodedHeLUT includes a detailed per-operation breakdown (Table 1) that helps the reader understand where the speedup comes from. The block-diagonal packing observation is also useful beyond DLRMs, and the discussion of LLM embedding costs identifies a real gap in prior FHE-transformer works. However, the significance is tempered by two omissions: no test AUC or plaintext accuracy baseline is reported for the compressed models, and the FHE parameter set is not specified. Both omissions are directly relevant to whether the reported latencies describe a usable deployment-quality model.","major_comments":[{"comment":"The central practical claim that aggressively compressed DLRMs remain accurate is not supported by any reported test AUC or plaintext baseline. Section 6.2 states only that \"more compressed models exhibit slightly higher loss\" and that \"SiLU models consistently show lower test AUC,\" and Figure 9 shows training curves, not test AUC values. Table 2 reports compression ratios up to 31180x, but no accuracy is attached to any row. Please report test AUC (and ideally a confidence interval) for each compression threshold in Table 2, for both the ReLU and SiLU variants, together with the AUC of an uncompressed baseline DLRM. Without these numbers, the 56x speedup and the end-to-end latencies in Figure 10 may be achieved on models whose accuracy is below the threshold required for practical deployment.","section":"§6.2 and Table 2"},{"comment":"The FHE parameter set is not specified beyond the statement \"we choose an FHE parameter set that enables bootstrapping while maintaining 128-bit security.\" The latency, level budget, and bootstrap count reported in Section 6.3 and Table 1 all depend on the ring degree N, the chain of coefficient moduli, the scaling factor, and the number of levels reserved for bootstrapping. Please report the concrete parameters (e.g., N, modulus bit lengths, scaling factor, and the estimated security level) used for each reported latency. Without this, the results are not reproducible and the comparison in Table 1 cannot be checked.","section":"§6.1 and §6.3"},{"comment":"The 56x speedup is attributed in the abstract to \"client-side digit decomposition,\" but the comparison in Figure 6 and Table 1 bundles at least two distinct optimizations: eliminating the encrypted indicator function through client-side one-hot encoding, and replacing CodedHeLUT's TableMult with a double-hoisted BSGS linear transformation. The text acknowledges the BSGS substitution, but the headline claim would be clearer if the speedup were decomposed, e.g., by reporting CodedHeLUT with and without the BSGS substitution. This matters because a reader may otherwise credit the entire 56x to the digit-decomposition compression scheme alone.","section":"Abstract, §5.1, and Figure 6"},{"comment":"The GPU and ASIC latency projections are presented in the abstract and conclusion as concrete outcomes (\"can reduce end-to-end latencies to seconds and even sub-seconds\"), but they are obtained by feeding Orion's static trace into Cheddar's primitive-level GPU microbenchmarks and Osiris's ASIC cost model, not by running the end-to-end HE-LRM pipeline on those backends. Please state explicitly in Section 6.4 that these are model-based projections, not measured end-to-end results, and report the key assumptions of the Osiris cost model (e.g., array dimensions, clock frequency, memory bandwidth) that drive the sub-second ASIC estimate.","section":"§6.4 and Figure 11"}],"minor_comments":[{"comment":"The abstract reports Criteo latencies of \"228 to 489 seconds,\" and Section 6.3 discusses models around 230 seconds for the highly compressed configurations, while the conclusion says \"489 seconds.\" Please clarify which configuration produces the 489-second number and which produces the 228-second number, so the range is unambiguous.","section":"Abstract and §6.3"},{"comment":"The caption of Figure 9 reads \"Uncompressed (33M rows)\" repeated three times; this looks like a placeholder and should be replaced with the actual compression-ratio labels for each curve.","section":"Figure 9"},{"comment":"The UCI model is said to use the \"x2 activation function rather than ReLU,\" but x2 is not defined or motivated in the text; please explain what this activation is and how it was chosen.","section":"§6.3"},{"comment":"The text says the digit-decomposition strategy is implemented with base p=4 for Criteo, but it does not say how p or the embedding-table threshold are chosen per configuration, nor whether the UCI experiments use the same p. Please specify the exact p and threshold for each row of Table 2.","section":"§6.2"},{"comment":"The claim that the client \"knows the size of the compressed embedding tables, which is encompassed by our threat model\" deserves a sentence clarifying why revealing table dimensions and the base p does not leak information about the learned embeddings themselves; this would preempt a natural security question from readers.","section":"§5.1"},{"comment":"The comparison in Table 1 is helpful, but it should state whether the CodedHeLUT timings are taken from the original paper or reimplemented in the authors' framework, and, if reimplemented, how the reimplementation was validated against the original results.","section":"Table 1"}],"recommendation":"major_revision","confidential_remarks":"The paper builds on Orion and Osiris from the same group, which is appropriate for a systems contribution, but the CodedHeLUT baseline is evidently reimplemented inside the authors' framework. I would ask the editor to have the authors clarify the baseline validation, since the 56x headline depends on the fairness of that comparison. The missing accuracy numbers and unspecified FHE parameters are the main technical blockers; both are straightforward to add and would substantially strengthen the paper's reproducibility."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Read HE-LRM. The core idea is sound and it's a real step forward for FHE private inference on sparse models. The client-side digit decomposition is a simple, elegant trick that turns embedding lookup into a single-level linear transform, avoiding the expensive encrypted indicator function in CodedHeLUT. The 56x speedup is measured, not hand-waved, and the block-diagonal packing for multi-table lookups is a nice addition that maximizes slot usage.\n\nThe paper also ships: implementation in Orion, code released, and a clear breakdown in Table 1 showing where the speedup comes from. That is solid systems work.\n\nThe soft spots are real but fixable. The biggest is that accuracy is under-reported. Section 6.2 says compressed models have 'slightly higher loss' and SiLU has 'lower test AUC', but no AUC numbers, no uncompressed baseline, no confidence intervals. At 31180x compression, the reader cannot judge whether this is a usable DLRM or a toy. The paper's practical claim rests on this, so it's load-bearing. A revision should report test AUC for all compression thresholds with plaintext baselines and the same architecture uncompressed.\n\nSecond, the FHE parameter set is not specified (just '128-bit security'). That makes latencies hard to reproduce or compare. This belongs in the paper or appendix.\n\nThird, the LLM section is more of a position piece: they show lookup costs and correctly point out that prior work leaks embeddings, but they don't train compressed LLM embeddings and the repacking step is left to future work. The abstract's 'apply to LLMs' overstates what is demonstrated and should be softened.\n\nThe GPU/ASIC projections are clearly labeled as projections using external cost models, so those are fine as long as they're not read as measured results.\n\nThe comparison against CodedHeLUT is fair: they reimplement it in the same framework and measure both.\n\nOverall: worthwhile paper that needs revision to add accuracy numbers and FHE parameters. It deserves a serious referee.","headline":"Digit-decomposition embedding lookup is a real advance; missing test AUC and FHE parameter details are load-bearing but fixable gaps.","tokens_in":22890,"tokens_out":2072,"would_cite":true,"duration_ms":23145,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"HE-LRM is the first end-to-end FHE DLRM pipeline that keeps embedding tables server-side and runs Criteo-scale recommendation inference in under eight minutes on a CPU.","keywords":["Fully Homomorphic Encryption","CKKS","DLRM","private inference","embedding table lookup","digit decomposition","block-diagonal packing","baby-step giant-step"],"falsifier":"Compute test AUC on the held-out seventh day of Criteo for each compression threshold and compare against the uncompressed baseline; if the AUC drop at the advertised 59x-31180x compression ratios is not small, the central practical claim collapses. A target check: instrument the FHE trace to verify the embedding lookup uses exactly one multiplicative level and no bootstrap.","tokens_in":21907,"feed_emoji":"🔐","tokens_out":8869,"duration_ms":93652,"temperature":0.7,"pith_summary":"This paper establishes that deep learning recommendation models (DLRMs), which route categorical inputs through huge embedding tables, can be executed end-to-end under fully homomorphic encryption without exposing the tables. The central move is a client-side digit decomposition of each categorical index into base-$p$ digits, so the expensive one-hot expansion and indicator-function circuit move before encryption and the lookup becomes one encrypted matrix-vector product. A block-diagonal packing places many embedding tables in one ciphertext and works with the baby-step giant-step linear transformation, so the lookup consumes a single multiplicative level and no bootstrap, giving a 56-fold speedup over the prior compressed lookup protocol. On a single-threaded CPU the complete private inference takes about 24 seconds on UCI Heart Disease and 228-489 seconds on the industry-scale Criteo click-prediction task, with hardware projections bringing those latencies to seconds and sub-seconds.","feed_headline":"Fully encrypted recommender inference runs in 24 seconds on CPU","feed_subtitle":"Client-side digit decomposition cuts encrypted embedding lookup cost 56x while keeping tables on the server.","key_machinery":"The load-bearing object is the client-side digit decomposition of an index $i \\in \\mathbb{Z}_k$ into $\\ell$ tokens in $\\mathbb{Z}_p$, a deterministic generalization of the Quotient-Remainder trick. Because the mapping depends only on the dimensions of the compressed tables, the client can one-hot encode each decomposed digit and encrypt a single sparse vector without learning anything about table values. The server multiplies that vector by a block-diagonal matrix in which all embedding tables are placed along the diagonal, so one ciphertext can carry simultaneous lookups across many tables. The baby-step giant-step matrix-vector product performs the lookup in one multiplicative level and places the extracted embeddings into contiguous slots, making the output directly compatible with the next encrypted layers.","core_discovery":"HE-LRM is the first end-to-end FHE DLRM pipeline in which dense features, sparse features, and the embedding lookup itself all remain encrypted while the embedding tables stay on the server. The paper's discovery is that a deterministic base-$p$ digit decomposition of a categorical token, computable on the client from the public compressed-table dimensions, removes the need for homomorphic one-hot construction and for the encrypted indicator function earlier methods required. The server receives one-hot encodings of the decomposed digits, multiplies them by a block-diagonal matrix of compressed tables using a baby-step giant-step linear transformation, and produces the embedding vectors in contiguous slots that feed directly into the downstream MLP. This design consumes only one multiplicative level for the lookup, requires no bootstrap, and outperforms the prior compressed-table protocol by up to 56x on the reported configurations.","pith_inferences":["If compressed embeddings hold up on accuracy, the same client-side digit decomposition should transfer to other sparse-index models, such as user-item collaborative filters or graph node embeddings, where table transfer is the bottleneck.","The paper's transformer discussion implies a layout mismatch: its row-major contiguous output must be repacked into column-packed ciphertexts for existing attention implementations; closing that gap could make non-interactive encrypted generation practical, which the paper leaves open.","Fixing the decomposition base at $p=4$ in the Criteo experiments leaves room for tuning: making $p$ and $\\ell$ per-table choices, or learning them during training, could shift the accuracy-versus-compression frontier that the paper only sketches.","The cost picture suggests a simple design rule for FHE-friendly recommenders: cap each table's vocabulary near a power of the decomposition base so the one-hot input fits in one or a few ciphertexts; DLRMs that respect this rule are the ones that reach the ~230-second plateau."],"forward_implications":["A single-threaded CPU can run an entire encrypted DLRM inference in about 24 seconds on a small health-prediction model and in 228-489 seconds on a Criteo-scale click-prediction model, with bootstrapping the dominant cost in the small model.","Embedding tables never leave the server and coded-token mappings do not have to be sent to the client, so the protocol avoids the model-weight leakage inherent in client-side embedding lookups.","The 56x lookup speedup over the prior protocol comes from replacing the encrypted indicator function and its bootstraps with a client-side one-hot encoding plus a single-level matrix-vector product.","Projected GPU and ASIC execution of the same FHE traces reduces end-to-end latencies to seconds and sub-seconds, respectively, which is the regime needed for production recommender traffic.","The same compression and packing primitives bring a GPT-2-scale transformer embedding lookup from hours to minutes and cut the per-generated-token communication rounds down to a small ciphertext upload."],"supporting_citations":[{"why":"prior compressed-table lookup method whose encrypted indicator function costs and baselines the 56x speedup claim rests on","marker":"[68]"},{"why":"Quotient-Remainder compositional embeddings that the digit decomposition generalizes","marker":"[58]"},{"why":"open-source FHE deep-learning framework HE-LRM is integrated into for end-to-end evaluation and level management","marker":"[22]"},{"why":"defines the DLRM architecture and the sparse/dense feature structure being encrypted","marker":"[53]"},{"why":"recent transformer private-inference protocol used to quantify client-side lookup leakage and packing incompatibility for LLMs","marker":"[71]"},{"why":"GPU FHE primitive library whose measured micro-kernels drive the projected GPU latency estimates","marker":"[14]"},{"why":"ASIC cost model used for the projected sub-second latencies","marker":"[23]"},{"why":"UCI Heart Disease dataset used for the small end-to-end evaluation","marker":"[34]"},{"why":"Criteo click-prediction dataset whose 33.8-million-row tables motivate compression and define the large-scale evaluation","marker":"[16]"}],"fun_headline_variants":["HE-LRM: 56x faster encrypted embedding lookup","Encrypted recommender hits 24s on CPU with HE-LRM","Client-side trick speeds FHE recommender 56x","HE-LRM keeps recommender data encrypted, runs 56x faster","Full encryption, 24s inference: HE-LRM does it"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that digit-decomposed tables keep the trained model accurate enough; the paper reports loss curves but does not state test AUC numbers or plaintext baselines, so the practical value of HE-LRM stands or falls on an unstated accuracy retention.","fun_headline_variants_meta":{"raw":{"variants":["HE-LRM: 56x faster encrypted embedding lookup","Encrypted recommender hits 24s on CPU with HE-LRM","Client-side trick speeds FHE recommender 56x","HE-LRM keeps recommender data encrypted, runs 56x faster","Full encryption, 24s inference: HE-LRM does it"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000721,"raw_usage":{"total_tokens":3266,"prompt_tokens":1005,"completion_tokens":2261,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":621,"completion_tokens_details":{"reasoning_tokens":2172}},"tokens_in":621,"tokens_out":2261,"duration_ms":17424,"temperature":1.0,"reasoning_tokens":2172,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T23:23:11.024096+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute test AUC on the held-out seventh day of Criteo for each compression threshold and compare against the uncompressed baseline; if the AUC drop at the advertised 59x-31180x compression ratios is not small, the central practical claim collapses. A target check: instrument the FHE trace to verify the embedding lookup uses exactly one multiplicative level and no bootstrap.","supporting_citations":[{"cited_title":"Privacy-preserving embedding via look-up ta- ble evaluation with fully homomorphic encryption","cited_arxiv_id":null,"evidence_quote":"prior compressed-table lookup method whose encrypted indicator function costs and baselines the 56x speedup claim rests on"},{"cited_title":"Compositional embeddings using complementary partitions for memory-efficient recom- mendation systems","cited_arxiv_id":null,"evidence_quote":"Quotient-Remainder compositional embeddings that the digit decomposition generalizes"},{"cited_title":"MOAI: Module-optimizing architec- ture for non-interactive secure transformer inference","cited_arxiv_id":null,"evidence_quote":"recent transformer private-inference protocol used to quantify client-side lookup leakage and packing incompatibility for LLMs"},{"cited_title":"Ched- dar: A swift fully homomorphic encryption library de- signed for gpu architectures","cited_arxiv_id":null,"evidence_quote":"GPU FHE primitive library whose measured micro-kernels drive the projected GPU latency estimates"},{"cited_title":"Osiris: A systolic approach to accelerating fully homomorphic encryption","cited_arxiv_id":null,"evidence_quote":"ASIC cost model used for the projected sub-second latencies"},{"cited_title":"Criteo display advertising chal- lenge dataset","cited_arxiv_id":null,"evidence_quote":"Criteo click-prediction dataset whose 33.8-million-row tables motivate compression and define the large-scale evaluation"}],"review_version":1}