{"id":"e7eaeeca-1cab-4bd9-a5ba-e148c12e1860","arxiv_id":"2507.01676","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Automatic asymmetric assignment of embedding-table chunks to per-core on-chip buffers yields measured 1.5x to 6.5x P99-latency improvements over the Ascend compiler baseline, with about 20x on a synthetic fixed-index distribution.","lead":"This paper designs four data-flow strategies and an automatic mapping framework that cut measured 99th-percentile embedding-lookup latency on Huawei Ascend accelerators by 1.5x to 6.5x versus the vendor compiler on real workloads, and more on skewed query distributions. Recommender inference dominates datacenter AI traffic, so faster embedding lookups translate directly into cheaper serving or bigger models under the same latency budget.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The auto-mapper relies on Eq. (2)'s unvalidated linear cost model; the unexplained 4x-5x Criteo-1TB fixed-distribution regression shows it can select harmful strategies.","rationale":"The reader's weakest_assumption already identifies Eq. (2); I agree. The Criteo-1TB fixed-distribution row is the sharpest evidence: it is a measured outcome, not a hypothetical, and it contradicts the headline 'much more independent of the query distribution'. On every other workload the baseline degrades far more than the proposed approach under the fixed distribution, but Criteo is inverted: the proposed strategies are 4x-5x worse than baseline. This means the automatic mapper selected a configuration that is severely harmful on a real workload; either the cost model's prediction was wrong or the performance model omitted a crucial interaction. Since the optimizer is the paper's main contribution, this is load-bearing. The authors do provide real hardware measurements and a clear experimental setup, which is independent support, but no validation of the model. A revision that validates Eq. (2) with cross-validation and explains the Criteo row would satisfy the concern; until then the conditional verdict is appropriate. No change to the reader's verdict is needed.","tokens_in":6749,"tokens_out":3116,"duration_ms":33841,"concrete_test":"Cross-validate Eq. (2): fit the beta parameters on five of the six workloads, then predict the held-out workload's per-table P99 for all four strategies and compare against measured latencies. If the model's strategy ranking for the held-out workload differs from the measured ranking (especially for Criteo-1TB fixed), the greedy assignment can choose a harmful strategy. Additionally, rerun Criteo-1TB fixed-distribution at batch 8192 with repeated trials for baseline, symmetric, and asymmetric; if the 538 us baseline repeats and the 2148 us asymmetric P99 persists, the paper must either add a guard (e.g., fall back to baseline when the model predicts a regression) or restrict the distribution-independence claim.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim (abstract: 1.5x-6.5x speedups, 'much more independent of query distribution') depends on the automatic mapping in Sec. III-B choosing safe strategies. That mapping is driven entirely by the per-configuration linear model in Eq. (2), fit with ordinary least squares on hardware measurements and assuming that 'the execution of each embedding layer is independent of the other' (Sec. III-A). The model is never validated on held-out workloads or distributions. Table I contains a concrete failure: for Criteo-1TB under the fixed distribution, baseline P99 is 538 us while symmetric and asymmetric measure 2632 us and 2148 us - 4x-5x regressions. This row is unexplained and directly contradicts the distribution-independence claim. Either Eq. (2) mispredicted the cost of L1/L1-UB (or GM/GM-UB) for this table set, or an interaction outside the additive model (bank conflicts, L1 thrashing, atomic accumulation overhead) dominates. Without a validation of Eq. (2) and an explanation of the Criteo row, the automatic assignment is not established as safe for unseen workloads.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper addresses DLRM inference bottlenecked by embedding lookups on multi-core AI accelerators. It proposes four single-core data-flow strategies (GM, L1, GM-UB, L1-UB) and an automatic asymmetric table-to-core mapping algorithm guided by a linear performance model, with experiments on Ascend 910 hardware. Compared with the vendor's default compiler, the reported head-to-head measurements show 1.5x-6.5x P99 latency improvements on real workload distributions and up to about 20x on a fixed-distribution TenRec case, plus a high-level theoretical comparison with the A100.","tokens_in":6925,"tokens_out":3953,"duration_ms":43570,"significance":"If the results hold, the paper makes a practical contribution: a drop-in data-flow optimization for embedding lookups that improves inference latency and throughput on Ascend-class hardware, with plausible extension to GPUs. The strengths are the use of real hardware profiling, multiple public workloads, three query distributions, and a direct comparison against the vendor compiler. The central weakness is that the automatic mapper relies on a linear performance model (Eq. 2) that is never validated, and Table I contains an unexplained 4x-5x P99 regression for Criteo-1TB under the fixed distribution that directly contradicts the abstract's distribution-independence claim. These issues must be resolved before the automatic mapping and the distribution-independence claim can be accepted.","major_comments":[{"comment":"The proposed methods severely regress on Criteo-1TB with the fixed distribution: baseline P99 is 538 us, while symmetric and asymmetric measure 2632 us and 2148 us, respectively. This is a 4x-5x regression, and it contradicts the abstract's claim that the method is 'much more independent of the query distribution than the baseline': the baseline actually improves from 817 us (uniform) to 538 us (fixed), whereas the proposed symmetric method degrades from 530 us to 2632 us. The paper does not explain this failure. Because the auto-mapper selected the strategies that produced these measurements, this row is evidence that the automatic assignment can choose harmful strategies. The authors must explain the cause (e.g., bank conflicts, L1 thrashing, atomic accumulation overhead) and discuss how it affects the distribution-independence claim.","section":"Section IV, Table I (Criteo-1TB fixed-distribution row)"},{"comment":"The linear additive latency model is never validated against held-out measurements. The model assumes that 'the execution of each embedding layer is independent of the other' and that P99 latency scales linearly with B*si/K and mi, with configuration-specific betas fit by OLS. No predicted-versus-measured comparison is provided, neither per table nor per workload. The Criteo-1TB fixed-distribution regression suggests that the model, or its independence assumption, can be badly wrong under contention. The authors should add a validation of Eq. (2), e.g., reporting the model's prediction error on held-out workloads and distributions, or at least per-table predicted and measured latencies for the configurations actually used. Without such validation, the automatic mapper is not established as safe for unseen workloads.","section":"Section III-A, Eq. (2)"},{"comment":"The greedy algorithm's strategy choices are entirely driven by the fitted betas of Eq. (2), so any model error can flip the selected strategy and produce the kind of regression seen in Table I. The paper should include a robustness or sensitivity analysis: for instance, compare the current model-guided allocation against a simple heuristic (e.g., always L1 for tables that fit, GM otherwise), or perturb the fitted betas and measure how often the selected policies change and by how much the final P99 varies. Additionally, the 'Load Imbalance Factor (LIF) threshold' and the maximum chunk size are heuristic parameters that are mentioned but never quantified; their values and a sensitivity analysis should be reported to make the automatic mapping reproducible and to show that the results are not tuned to the specific workloads.","section":"Section III-B (greedy asymmetric allocation)"}],"minor_comments":[{"comment":"The header reads 'P99 LATENCY [S/BATCH]' but the values are in microseconds (e.g., 22872µ, 817µ). Use 'µs/batch' consistently.","section":"Table I header"},{"comment":"There is a typo: 'We propose innovativedata flow strategies' should read 'We propose innovative data flow strategies'.","section":"Section I (Introduction)"},{"comment":"Several references are malformed or incomplete, e.g., [9] lists 'j. Jean-Baptiste Tien' and [5] has a truncated subtitle. Please fix the reference formatting and complete all entries.","section":"References"},{"comment":"The high-level A100 comparison in Figure 3 relies on unspecified hardware assumptions (e.g., peak bandwidth, conflict-free access). Briefly state the assumed parameters so the estimation is reproducible.","section":"Section IV-B"}],"recommendation":"major_revision","confidential_remarks":"The unexplained Criteo-1TB fixed-distribution regression is the key issue. If the authors cannot explain it or validate Eq. (2), the abstract's distribution-independence claim should be substantially weakened, and the 'automatic' aspect of the method becomes questionable. I recommend the editor require the additional validation and sensitivity analysis described in the major comments before further consideration."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThis paper deserves a serious look. The genuinely new piece is the asymmetric table-to-core partitioning: splitting tables, pinning different chunks in different cores' L1, replicating chunks, clipping indices, and using atomic accumulation. That is absent from the cited prior work (AutoShard does host-level sharding with RL, EVStore is a cache policy, FleetRec/MicroRec are FPGA systems). The four per-core data-flow strategies are also concrete.\n\nWhat the paper does well: the evidence is real hardware measurements on Ascend 910 against the vendor compiler. Table I shows 1.5x-6.5x P99 gains on real distributions across six workloads, and the TenRec fixed-distribution case supports the 'more than 20x' claim (375/18 = 20.8). So the abstract is not overstating that.\n\nThe soft spots are real, though. First, there are no run-to-run statistics or error bars on P99, which for a latency-tail paper matters. Second, Eq. (2) is a per-configuration linear fit that is never validated on held-out workloads, and the greedy auto-mapper depends on it. The paper states the independence assumption, but it doesn't test it. Third, and most important, the Criteo-1TB fixed-distribution row in Table I is a 4x-5x P99 regression against baseline (538 us baseline vs 2632 symmetric and 2148 asymmetric). The paper never explains this, and it contradicts the 'much more independent of query distribution' claim. The stress-test note is right to flag it. One correction: the reader's claim that the 6.5x figure overstates the real-distribution row is wrong; TenRec's real distribution is 108/17 = 6.35x.\n\nI think the paper is worth peer review. The central measured speedups are real evidence, and the asymmetric idea is novel enough to cite. But the automatic mapping contribution is not established until the Criteo row is explained and Eq. (2) is validated (or the auto-mapper is presented as a heuristic without claiming safety). A revision can do that. I'd bring it to reading group.","headline":"Real measured speedups on Ascend 910 and a genuinely new asymmetric table-to-core partitioning idea, but the auto-mapper's unvalidated cost model and the unexplained Criteo fixed-distribution regression are load-bearing soft spots.","tokens_in":7559,"tokens_out":2571,"would_cite":true,"duration_ms":26802,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Asymmetric core assignment cuts embedding-lookup latency by 1.5x-6.5x.","keywords":["deep recommender models","DLRM inference","embedding lookup","asymmetric data flow","table sharding","P99 latency","Ascend accelerator"],"falsifier":"Run the automatic mapping on the six workloads and the three query distributions from the paper, then measure the actual P99 of the chosen strategies on a held-out subset of queries. If the mapping chosen by the linear model is not the best of the four strategies for a workload, or if a measured held-out latency disagrees with the model's prediction by more than the gap between strategies, the independence and linearity assumptions fail. The Criteo-1TB fixed-distribution row in Table I is a concrete place to look: there the automatic strategies measure 2632 microseconds and 2148 microseconds against a 538 microsecond baseline, which is the kind of divergence a validation step would need to catch.","tokens_in":6457,"feed_emoji":"⚡","tokens_out":6975,"duration_ms":69373,"temperature":0.7,"pith_summary":"This paper claims that the random-memory embedding lookups that dominate deep recommender inference can be accelerated by choosing, for each embedding table, one of four data-flow strategies and by assigning the tables asymmetrically across the cores of a multi-core accelerator so that different cores preload different table chunks in on-chip memory. On an Ascend 910 accelerator, the paper reports 1.5x to 6.5x lower P99 latency than the vendor compiler on six real workloads, and more than 20x lower P99 on an adversarial fixed-index query distribution, while making performance much less dependent on the query distribution. If correct, this would let recommender systems meet 99th-percentile latency targets with smaller batches or higher throughput, and it suggests the same design transfers to GPU-like architectures. The automatic mapping is driven by a simple linear performance model of per-table latency, fitted with ordinary least squares.","feed_headline":"Embedding lookups get 1.5x-6.5x faster with asymmetric core mapping","feed_subtitle":"Spreading embedding tables across on-chip cores trims P99 latency while keeping throughput high.","key_machinery":"The load-bearing object is the per-table data-flow strategy set, named GM, GM-UB, L1, and L1-UB. GM reads one embedding row at a time from off-chip global memory with double buffering; GM-UB first moves the table in chunks into the shared unified buffer and performs vectorized lookups; L1 does the single-row read from a persistent L1 buffer; L1-UB combines persistent L1 storage with vectorized chunked lookups. The second object is the greedy asymmetric sharding heuristic, which splits oversized tables into chunks, sorts tables by descending sequence length and ascending size, and allocates each chunk to the core with the lowest predicted P99 under the linear model $J_i = \\beta_0 + \\beta_1 \\, (B\\, s_i / K) + \\beta_2 \\, m_i$ for the strategies that use the unified buffer, with $\\beta$ coefficients fitted by ordinary least squares from hardware measurements. These pieces work together to turn an irregular random-access pattern into conflict-free on-chip accesses.","core_discovery":"On the paper's own terms, the discovery is that DLRM embedding lookup can be split into four data flows — reading one row at a time from global memory or from a persistent on-chip buffer, or vectorized lookups that bring table chunks into the unified buffer before reading — and that these strategies can be combined with an automatic asymmetric table-to-core assignment. The asymmetric step splits each embedding table into chunks, optionally replicated, and places different chunks in the L1 buffers of different cores, effectively multiplying the usable on-chip table capacity by the number of cores. A greedy algorithm driven by the linear cost model in Eq. (2) chooses the strategy for every table, and the paper reports that the resulting schedules outperform the default Ascend compiler on all workloads for real query distributions, with the largest gains when the baseline suffers from L2 cache conflicts. The paper also presents a high-level bandwidth-based estimate suggesting the same approach would be competitive on an Nvidia A100.","pith_inferences":["The fixed-distribution Criteo row in Table I suggests that the linear model can be wrong under worst-case bank conflicts; a natural extension is to add a fallback validation step that keeps the baseline schedule whenever a mapping's predicted P99 exceeds the measured baseline by a safety margin.","If distribution independence holds across more workloads, recommendation platforms could replace per-query-distribution tuning with a single offline mapping computed once per model version, reducing operational overhead.","The same asymmetric chunking idea should extend to embedding tables larger than the global memory by splitting them across multiple devices or by sending only the requested rows from the host, which the paper mentions as future work in passing."],"forward_implications":["With P99 as the SLA metric, an operator can run the same batch size at the same P99 budget while lowering per-query cost, because asymmetric mapping reduces the tail latency caused by slow tables.","Because the mapping is automatic and parameter-light, a new model version or new query skew does not require hand-tuning the data flow.","The method's insensitivity to query distribution means a single compiled schedule can serve both uniform and skewed traffic without recompilation.","If the high-level A100 estimate holds, the same four-strategy design could be offered as a compiler pass for GPU inference stacks."],"supporting_citations":[{"why":"Supplies the workload-statistic premise that DLRM inference dominates AI workloads and is bottlenecked by embedding layers.","marker":"[2]"},{"why":"Frames the table sharding problem as NP-hard and motivates the paper's greedy cost-model heuristic.","marker":"[6]"},{"why":"Provides the Nvidia A100 architecture specifications used in the high-level comparison.","marker":"[7]"},{"why":"Describes the Ascend architecture and software stack that the four data-flow strategies target.","marker":"[8]"},{"why":"Supplies the Criteo-1TB table statistics used in the evaluation.","marker":"[9]"},{"why":"Supplies the Avazu-CTR table statistics used in the evaluation.","marker":"[10]"},{"why":"Supplies the Taobao table statistics used in the evaluation.","marker":"[11]"},{"why":"Supplies the TenRec table statistics used in the evaluation.","marker":"[12]"},{"why":"Supplies the KuaiRec table statistics used in the evaluation.","marker":"[13]"}],"fun_headline_variants":["Automatic asymmetric data flow speeds DLRM embedding lookups 1.5x-6.5x","Asymmetric core mapping speeds embedding lookups by up to 6.5x","Auto asymmetric data flow yields 1.5x-6.5x faster DLRM lookups","Greedy asymmetric mapping of embedding tables speeds DLRM inference"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The automated mapping stands on the assumption that each embedding layer executes independently of the others and that a single linear formula with fitted coefficients predicts each table's P99 latency well enough that the greedy assignment picks a strategy that is at least as good as the baseline.","fun_headline_variants_meta":{"raw":{"variants":["Automatic asymmetric data flow speeds DLRM embedding lookups 1.5x-6.5x","Asymmetric core mapping speeds embedding lookups by up to 6.5x","Auto asymmetric data flow yields 1.5x-6.5x faster DLRM lookups","Greedy asymmetric mapping of embedding tables speeds DLRM inference"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000878,"raw_usage":{"total_tokens":3780,"prompt_tokens":914,"completion_tokens":2866,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":530,"completion_tokens_details":{"reasoning_tokens":2771}},"tokens_in":530,"tokens_out":2866,"duration_ms":22683,"temperature":1.0,"reasoning_tokens":2771,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T20:47:27.835218+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the automatic mapping on the six workloads and the three query distributions from the paper, then measure the actual P99 of the chosen strategies on a held-out subset of queries. If the mapping chosen by the linear model is not the best of the four strategies for a workload, or if a measured held-out latency disagrees with the model's prediction by more than the gap between strategies, the independence and linearity assumptions fail. The Criteo-1TB fixed-distribution row in Table I is a concrete place to look: there the automatic strategies measure 2632 microseconds and 2148 microseconds against a 538 microsecond baseline, which is the kind of divergence a validation step would need to catch.","supporting_citations":[{"cited_title":"The Architectural Implications of Facebook’s DNN- Based Personalized Recommendation,","cited_arxiv_id":null,"evidence_quote":"Supplies the workload-statistic premise that DLRM inference dominates AI workloads and is bottlenecked by embedding layers."},{"cited_title":"AutoShard: Automated Embedding Table Sharding for Recommender Systems,","cited_arxiv_id":null,"evidence_quote":"Frames the table sharding problem as NP-hard and motivates the paper's greedy cost-model heuristic."},{"cited_title":"3.2 The A100 Datacenter GPU and Ampere Architecture,","cited_arxiv_id":null,"evidence_quote":"Provides the Nvidia A100 architecture specifications used in the high-level comparison."},{"cited_title":"Ascend: a scalable and unified architecture for ubiqui- tous","cited_arxiv_id":null,"evidence_quote":"Describes the Ascend architecture and software stack that the four data-flow strategies target."},{"cited_title":"Display advertising challenge,","cited_arxiv_id":null,"evidence_quote":"Supplies the Criteo-1TB table statistics used in the evaluation."},{"cited_title":"Click-through rate prediction,","cited_arxiv_id":null,"evidence_quote":"Supplies the Avazu-CTR table statistics used in the evaluation."},{"cited_title":"How serendipity improves user satisfaction with recom- mendations?","cited_arxiv_id":null,"evidence_quote":"Supplies the Taobao table statistics used in the evaluation."},{"cited_title":"Tenrec: A Large-scale Multipurpose Benchmark Dataset for Recommender Systems,","cited_arxiv_id":null,"evidence_quote":"Supplies the TenRec table statistics used in the evaluation."},{"cited_title":"KuaiRec: A Fully-observed Dataset and Insights for Evaluating Recommender Systems,","cited_arxiv_id":null,"evidence_quote":"Supplies the KuaiRec table statistics used in the evaluation."}],"review_version":1}