{"id":"e91ddafe-3f7b-4393-8b4c-26d5271bdcc5","arxiv_id":"2608.04502","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"AFD-Ledger analytically provisions AFD and collocated deployments, skips 68.8 to 83.5 percent of candidate hardware evaluations, and finds homogeneous AFD wins only 7 of 36 studied settings.","lead":"This paper presents AFD-Ledger, an offline provisioning tool that compares attention-FFN disaggregated and collocated deployments for MoE language models under fixed budgets and latency targets. It finds that disaggregation usually needs carefully chosen heterogeneous hardware to beat collocated serving.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The deployment insights rest on an analytical model physically validated on only three Ascend workloads; its 6.6-9.6% error is close to the point where the paper's own overhead robustness test flips decisions.","rationale":"The reader's conditional verdict is driven by the same weakest assumption I identify: the analytical execution model's ordering accuracy. I agree with that assessment, and the concern does not change the verdict. The strongest reason to worry is the mismatch between where the model is validated and where the headline findings are produced. Section 5.2.1 validates the model on three LongCat 2.0 workloads running on Ascend 910C with 6.6-9.6% throughput error, while the 7-of-36 homogeneous result and the heterogeneous complementarity findings are generated for Qwen3/DeepSeek on an NVIDIA catalog (H200, A100, L40S, etc.) with no per-device validation. The model intentionally omits kernel deficiency, scheduler latency, and network contention (Section 6). Section 5.4.2's own robustness experiment applies only a uniform AFD overhead and shows the two reference decisions flip to collocated around 20% overhead, with degradation beginning near 10%; the validation error is 6.6-9.6%, so the margin between measured error and the flip point is thin. Because the robustness test does not inject per-stage or per-device error, it cannot bound the true risk for the NVIDIA-based conclusions. The search-quality claim (68.8-83.5% reduction with optimum recovery) is separately supported on the enumerated spaces and is less concerning; the model-fidelity issue is the load-bearing one. A calibrated re-run of the 36 E-grid decisions would settle whether the 7-of-36 finding and the complementarity conclusions survive realistic per-device model error. I therefore recommend no change to the CONDITIONAL verdict.","tokens_in":19357,"tokens_out":7688,"duration_ms":65464,"concrete_test":"Re-run all 36 E-grid provisioning decisions with the analytical model calibrated by per-device kernel-efficiency measurements on the actual NVIDIA cards (e.g., measured HBM utilization, kernel efficiency, and dispatch/combine overhead for H200, A100, and L40S), instead of peak roofline values. Then compare the AFD/collocated choice and the selected heterogeneous pair against the paper's 7-of-36 result. As a minimal version, apply the LongCat calibration residuals as per-stage error multipliers (e.g., +10% to AFD attention and FFN stages, +0-5% to collocated stages) and re-run the 36 cases; if the set of homogeneous-AFD wins or the best heterogeneous pair changes, the headline finding is not robust to the model uncertainty the paper itself reports.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The headline deployment insights--homogeneous AFD wins in only 7 of 36 E-grid settings and heterogeneous AFD requires deployment-level complementarity--are produced entirely by the analytical execution model of Sections 2.1 and 2.4 (Eq. 5). The only physical check of that model is Section 5.2.1: three LongCat 2.0 workloads on Ascend 910C, with AFD-to-collocated throughput ratios within 6.6-9.6% of measurement. That validation is on a different model family and a different hardware/software stack than the Qwen3/DeepSeek plus NVIDIA catalog used for the 7-of-36 result. The model explicitly omits kernel deficiency, scheduler latency, and network contention (Section 6). The error band matters because Section 5.4.2's own robustness test (Figure 14) shows the D/E and Q/E decisions degrade near 10% additional AFD overhead and flip to collocated around 20%; the measured validation error is 6.6-9.6%, so the margin between measured error and the flip point is thin. That robustness test also injects a single uniform overhead, not per-stage or per-device errors, so it does not bracket the true model uncertainty for Qwen3/DeepSeek on H200, A100, and L40S. If the real model error on those stacks favors collocated by roughly 10-20%, which is plausible given the omitted runtime overheads, some of the seven homogeneous-AFD wins and part of the heterogeneous complementarity ranking could change. The paper honestly flags model fidelity as a limitation in Section 6, but the central claim depends exactly on that unvalidated-for-NVIDIA ordering.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents AFD-Ledger, an offline provisioning system that decides, under a fixed model, workload, TPOT SLO, hardware budget, hardware catalog, and runtime assumptions, whether the best Attention–FFN Disaggregation (AFD) deployment or the best collocated deployment achieves higher SLO-feasible decode throughput. The system uses an analytical execution model in which each stage's latency is the maximum of compute time and HBM transfer time, and it searches over hardware assignments with a bounded number of full deployment evaluations. The authors reconstruct two existing AFD results (FastAFD and MegaScale-Infer), show that bounded search recovers the exhaustive analytical optimum on the enumerated cases, validate the analytical model against three physical LongCat 2.0 workloads on Ascend 910C, and derive three deployment insights: homogeneous AFD wins in only 7 of 36 studied settings, heterogeneous AFD requires deployment-level hardware complementarity, and role-specific hardware improvements matter mainly when they enable new deployment organizations.","tokens_in":19680,"tokens_out":4459,"duration_ms":43017,"significance":"If the analytical model is sufficiently faithful, AFD-Ledger is a useful contribution to MoE serving: it upgrades the AFD discussion from mechanism-level evaluations to a full-deployment provisioning comparison, and it gives operators a concrete way to decide when disaggregation is worth the request-bearing-capacity tax. The paper is careful to distinguish reported deployments from independently reprovisioned ones, and it honestly labels retrospective reconstructions that rely on appointed parameters. The strengths are real: the FastAFD reconstruction matches the reported result within 0.7%, the MegaScale proxy within 7.1%, the physical LongCat validation preserves the architecture decision on all three workloads, and bounded search recovers the exhaustive optimum in the eighteen enumerated E-grid cases and all five reference cases. The main limitation is that the headline deployment conclusions are generated by an analytical model that is directly validated on only three workloads from a different model family and hardware stack; this is the load-bearing assumption of the paper and needs to be addressed before the 7-of-36 finding can be taken as a general result.","major_comments":[{"comment":"The analytical execution model of Sections 2.1 and 2.4 is validated only on three LongCat 2.0 workloads on Ascend 910C, while the central deployment findings use Qwen3-235B and DeepSeek-V3.2 on an NVIDIA catalog (H200, H100, A100, L40S, L4). The measured AFD-to-collocated throughput error is 6.6–9.6%, and Figure 14 shows that decisions degrade at roughly 10% additional AFD overhead and flip to collocated around 20%. The margin between the measured error and the flip point is therefore thin, and the robustness test injects a single uniform overhead rather than per-stage or per-device errors from the omitted effects (kernel deficiency, scheduler latency, network contention). The paper explicitly flags model fidelity as a limitation in Section 6, but that limitation is not a peripheral caveat: it is the premise on which the 7-of-36 result and the heterogeneous complementarity conclusions rest. I would like to see either direct validation on at least one Qwen or DeepSeek workload using the relevant NVIDIA devices, or a structured sensitivity analysis that applies per-stage/per-device error perturbations large enough to bracket the 6.6–9.6% validation error plus the omitted runtime overheads. Without such evidence, the deployment insights should be reported as conditional on the analytical model's fidelity rather than as robust findings.","section":"Sections 5.2.1, 5.4.2, and 6"},{"comment":"The bounded hardware search is validated by showing that it recovers the best deployment found by exhaustive enumeration of hardware assignments. This is a valid test of search efficiency, but it is a search test, not an accuracy test: the 'global optimum' being recovered is the optimum of the same analytical model used to evaluate all deployments. The abstract and Section 5.2.2 should state this more carefully, because a reader can easily misread 'globally optimal deployment' as a claim about the physical optimum. The actual physical accuracy is established only by the three LongCat 2.0 runs, so the search-quality claims and the model-fidelity claims should be kept separate.","section":"Sections 5.2.2 and 5.3"}],"minor_comments":[{"comment":"Equation (5) and the surrounding text define Θ*_C and Θ*_A but the sentence 'where D_C(Ω) and D_A(Ω) are the legal collocated and AFD plans' is missing a closing period and should be completed.","section":"Section 2.4"},{"comment":"The near-tie threshold is described as the error 'observed in physical validation,' but no precise value or rule is given. Please state the actual numerical threshold used to classify a result as a near tie.","section":"Section 2.4 and Figure 4"},{"comment":"The text repeats the sentence 'We next evaluate whether bounded hardware exploration recovers exhaustive provisioning...' immediately before Figure 8; one of the two occurrences should be removed.","section":"Section 5.2.2"},{"comment":"The first sentence of Section 3 starts with a lowercase 'it' after the preceding period: 'it first uses inexpensive role-specific estimates...' should be capitalized.","section":"Section 3"},{"comment":"The MegaScale-Infer reconstruction relies on appointed parameters (the 64-L20-equivalent budget, KV configuration, and SLO) that are not reported in the source; the 7.1% agreement is therefore a proxy check rather than a pure validation. The paper labels this correctly in the text, but the abstract's phrase 'reproduces reported gains' should not be read as stronger than the proxy methodology supports.","section":"Section 2.2"}],"recommendation":"major_revision","confidential_remarks":"The retrospective analysis leans heavily on unreviewed technical reports and preprints from 2025–2026, especially [7] and [36], and the physical validation uses an internal Meituan LongCat stack. I do not think this is a reason to reject, but the editor may want to verify that the cited artifacts, including the claimed reported numbers, are publicly accessible. The central issue for me is the thin validation margin between the measured 6.6–9.6% model error and the roughly 10–20% overhead range at which Figure 14 shows decisions starting to flip; that gap is what separates a defensible conditional study from a fully load-bearing empirical one."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"This paper does something useful: it turns the AFD-versus-collocated question into a bounded provisioning search and actually checks its work. The bounded hardware search (20 of 64 or 121 assignments, recovering the exhaustive optimum in all enumerated cases) is a real algorithmic contribution, and the reconstructions of FastAFD (within 0.7%) and MegaScale (within 7.1%) give me confidence that the analytical model captures the dominant terms. The deployment-level findings - homogeneous AFD winning only 7 of 36 settings, heterogeneous AFD needing deployment-level complementarity rather than heuristic device-picking, and capability-price boundaries unlocking most of the gain - are genuinely interesting and directly useful to operators and hardware co-design folks. The paper is also admirably honest: it states its limitations in Section 6, distinguishes reconstruction from reprovisioning, and reports near ties rather than overclaiming. The main soft spot is exactly what the stress-test note flags. The headline findings for Qwen3/DeepSeek on NVIDIA GPUs are produced entirely by an analytical model that omits kernel deficiency, scheduler latency, and network contention, and whose only physical check is three LongCat 2.0 workloads on Ascend 910C. The 6.6-9.6% error band is uncomfortably close to the roughly 20% uniform overhead where the paper's own Figure 14 flips decisions. The robustness test also injects a single uniform overhead, not per-stage or per-device errors, so it does not bracket the real uncertainty for the NVIDIA stacks. This does not invalidate the paper, but it does mean the 7-of-36 number and the complementarity ranking are conditional on an unvalidated model ordering. The paper's own honesty mitigates this, but the conclusions are stated more firmly than the evidence supports. Two minor gripes: no artifacts are released (the 5,700-line system and 8,000-line runtime are described but not available), and there is no quantitative comparison against the cited prior provisioning simulators - only a qualitative statement that they assume fixed hardware pools. Both are addressable. Who should read this? Systems researchers comparing disaggregation architectures, anyone doing analytical LLM provisioning, and hardware designers setting specs. It deserves a serious referee. Recommendation: send it out; ask the authors to either validate on NVIDIA-class hardware, add per-stage error robustness, or soften the deployment-insight claims to 'model-based' rather than general. A revision with any one of those changes would be a strong paper.","headline":"A solid, honest provisioning study whose headline deployment insights rest on a thinner and differently-stacked physical validation than the claims suggest; worth serious refereeing, but the authors should either broaden validation or temper conclusions.","tokens_in":705,"tokens_out":885,"would_cite":true,"duration_ms":20656,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"AFD-Ledger claims that when both Attention–FFN disaggregated and collocated deployments are provisioned optimally under the same fixed budget, disaggregation raises throughput in only a minority of studied settings, and a 20-assignment…","keywords":["Attention-FFN disaggregation","MoE inference","deployment provisioning","hardware search","analytical performance model","TPOT SLO","heterogeneous hardware","fixed-budget accounting"],"falsifier":"Measure stage-level TPOT and end-to-end throughput for Qwen3-235B and DeepSeek-V3.2 on the H200/A100 PCIe/L40S catalog at the 1K and 4K contexts used in the paper, and compare architecture decisions with AFD-Ledger's predictions; if any setting where the model predicts AFD wins by more than roughly 10% actually shows collocated serving winning (or vice versa), the ordering-preservation assumption that carries the 7-of-36 and complementarity conclusions fails.","tokens_in":1894,"feed_emoji":"🧮","tokens_out":1926,"duration_ms":69846,"temperature":0.7,"pith_summary":"The paper asks a question that mechanism-level AFD studies leave open: under the same model, workload, TPOT SLO, budget, and hardware catalog, does Attention–FFN disaggregation (AFD) actually serve more tokens than the best collocated deployment? To answer it, AFD-Ledger independently provisions both deployment families with an analytical execution model and a bounded hardware search that evaluates only 20 of 64 or 121 hardware assignments yet recovers the exhaustive optimum wherever exhaustive search is feasible. The answer, across the studied catalogs and 36 homogeneous settings, is that homogeneous AFD wins in only 7 cases, heterogeneous AFD needs the right hardware complementarity rather than intuitive device pairing, and role-specific hardware improvements matter mainly when they enable better deployment organizations. This matters because it turns AFD evaluation from a mechanism question into a full-budget accounting question, where a local gain counts only if it repays the request-bearing capacity given up to create a separate FFN pool.","feed_headline":"AFD wins only 7 of 36 fully provisioned deployments","feed_subtitle":"Same-budget reprovisioning shows disaggregation pays off only with the right hardware pair and workload.","key_machinery":"The load-bearing object is the fixed-budget performance ledger: steady-state decode throughput is $\\Theta = N_r b / T$, where $N_r$ is the number of request-serving GPUs, $b$ the resident requests per GPU, and $T$ the TPOT; an AFD-versus-collocated comparison is the product of the ratios of these three factors under a common specification $\\Omega$. On top of this, AFD-Ledger uses an analytical execution model (stage latency is the slower of peak-compute time and HBM-transfer time, with pipeline microbatches for AFD) to score deployments, and a two-level search: role-specific hardware estimates rank candidate attention-FFN pairs, a bounded refinement loop fully provisions only up to $Q=20$ assignments, and each assignment gets a complete deployment optimization over EP width, replication, batching, worker ratio, and pipeline depth. The search machinery is what lets the paper claim architecture decisions with 68.8% to 83.5% fewer complete evaluations while still recovering exhaustive optima.","core_discovery":"The central discovery is that AFD's reported gains do not automatically translate into deployment-level throughput. The paper reproduces two published AFD results analytically (a 1.404x reconstructed gain versus 1.414x measured for one Qwen3 deployment, and 1.463x versus 1.574x for a DBRX heterogeneous deployment), then shows that after both AFD and collocated families are reprovisioned from scratch under the same specification, the advantage shrinks to near ties (0.990x and 0.947x respectively). Across 36 homogeneous E-grid settings, AFD-Ledger selects AFD in only 7 cases; heterogeneous AFD can reach 1.815x to 1.823x over collocation but only for specific pairs, and the same H200+A100 PCIe pair ranges from 0.483x to 1.815x depending on budget and SLO. The paper's conclusion is that AFD's benefits are deployment-level: they materialize only when expert-memory removal, larger batches, and lower TPOT repay the request-bearing-capacity tax, and future role-specific hardware matters primarily when it unlocks new replica layouts or worker ratios rather than improving one component in isolation.","pith_inferences":["If the ordering-preserving property of the analytical model extends beyond the three validated LongCat 2.0 workloads, the same provisioning framework could be applied to other disaggregation architectures (prefill-decode separation, KV-centric pools) that have not yet been compared under full re-provisioning.","The 7-of-36 win rate is a property of the studied catalogs and SLOs, not a universal bound; the paper's own overhead sweep shows decisions flip to collocated around 20% additional AFD runtime overhead, so tighter SLOs or slower interconnects could shrink AFD's wins further.","A testable co-design consequence: future AFD-specialized accelerators should be evaluated by the deployments they unlock (replicas, worker ratios, batch sizes) rather than by raw FLOP/s or bandwidth, which suggests memory-capacity-per-dollar and price targets deserve more weight than peak compute.","A practical extension would be to run the same pair-provisioning comparison on real heterogeneous clusters with more workloads to measure whether the 6.6–9.6% model error window holds outside LongCat; if it does, exhaustive search can be replaced by AFD-Ledger's 20-assignment search with confidence."],"forward_implications":["Operators comparing AFD with collocated serving must reprovision both architectures under the same specification; mechanism-level gains alone do not determine which deployment serves more tokens.","Homogeneous AFD is the exception, not the rule: in the studied space it wins 7 of 36 settings, and only where batch and TPOT gains repay the request-bearing-capacity tax.","Heterogeneous AFD can deliver large gains (up to roughly 1.8x) but the winning hardware pair is workload- and budget-dependent; the same H200+A100 PCIe pair ranges from 0.483x to 1.815x.","Role-specific hardware improvements (cost, memory capacity) are most valuable when they change the deployment organization, such as enabling more replicas; in one DeepSeek case crossing a capability-price boundary raised throughput 19.59x.","Bounded hardware search with 20 assignments recovers the exhaustive optimum on both tested catalogs, cutting complete deployment evaluations by 68.8% to 83.5%."],"supporting_citations":[{"why":"supplies the Qwen3 AFD deployment result (1.414x measured) that the paper reconstructs analytically and then reprovisions.","marker":"[7]"},{"why":"supplies the DBRX heterogeneous AFD result (1.574x measured) reconstructed in the proxy analysis.","marker":"[36]"},{"why":"the open MoE model used in the three physical validation deployments that set the 6.6–9.6% error window.","marker":"[19]"},{"why":"vendors' device specifications (HBM capacity, bandwidth, compute) that define the hardware catalog for all provisioning runs.","marker":"[20]"},{"why":"hourly GPU prices that set the fixed-budget accounting and capability-price boundaries in the study.","marker":"[1]"},{"why":"prior analytical provisioning work that assumes a fixed hardware pool, which AFD-Ledger extends to hardware-catalog provisioning.","marker":"[25]"}],"fun_headline_variants":["AFD wins only 7 of 36 after full reprovisioning","Full reprovisioning shrinks AFD gains to near ties","AFD's benefits hinge on hardware complementarity","Same-budget reprovisioning exposes AFD's narrow edge"],"cache_read_input_tokens":22272,"weakest_assumption_plain":"The analytical execution model, which estimates each stage's latency as the slower of peak compute time and HBM transfer time and omits kernel deficiencies, scheduler latency, and network contention, is accurate enough to preserve which architecture wins across all studied models and catalogs, even though it was validated on only three LongCat 2.0 workloads with 6.6–9.6% throughput error.","fun_headline_variants_meta":{"raw":{"variants":["AFD wins only 7 of 36 after full reprovisioning","Full reprovisioning shrinks AFD gains to near ties","AFD's benefits hinge on hardware complementarity","Same-budget reprovisioning exposes AFD's narrow edge"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000328,"raw_usage":{"total_tokens":1895,"prompt_tokens":1068,"completion_tokens":827,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":684,"completion_tokens_details":{"reasoning_tokens":757}},"tokens_in":684,"tokens_out":827,"duration_ms":7692,"temperature":1.0,"reasoning_tokens":757,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T14:38:26.377724+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure stage-level TPOT and end-to-end throughput for Qwen3-235B and DeepSeek-V3.2 on the H200/A100 PCIe/L40S catalog at the 1K and 4K contexts used in the paper, and compare architecture decisions with AFD-Ledger's predictions; if any setting where the model predicts AFD wins by more than roughly 10% actually shows collocated serving winning (or vice versa), the ordering-preservation assumption that carries the 7-of-36 and complementarity conclusions fails.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"supplies the Qwen3 AFD deployment result (1.414x measured) that the paper reconstructs analytically and then reprovisions."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"the open MoE model used in the three physical validation deployments that set the 6.6–9.6% error window."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"vendors' device specifications (HBM capacity, bandwidth, compute) that define the hardware catalog for all provisioning runs."},{"cited_title":"RunPod GPU Cloud Pricing.https://www.runpod.io/pricing","cited_arxiv_id":null,"evidence_quote":"hourly GPU prices that set the fixed-budget accounting and capability-price boundaries in the study."}],"review_version":1}