{"record_type":"pith_number_record","schema_url":"https://pith.science/schemas/pith-number/v1.json","pith_number":"pith:2024:BUWFBN5L5ASUOO32ZZS7O2NHFC","short_pith_number":"pith:BUWFBN5L","schema_version":"1.0","canonical_sha256":"0d2c50b7abe825473b7ace65f769a728a7922f820c98b63f1d5a1f858509fd60","source":{"kind":"arxiv","id":"2401.09670","version":3},"attestation_state":"computed","paper":{"title":"DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving","license":"http://creativecommons.org/licenses/by-sa/4.0/","headline":"","cross_cats":[],"primary_cat":"cs.DC","authors_text":"Hao Zhang, Jianbo Hu, Junda Chen, Shengyu Liu, Xin Jin, Xuanzhe Liu, Yibo Zhu, Yinmin Zhong","submitted_at":"2024-01-18T01:03:38Z","abstract_excerpt":"DistServe improves the performance of large language models (LLMs) serving by disaggregating the prefill and decoding computation. Existing LLM serving systems colocate the two phases and batch the computation of prefill and decoding across all users and requests. We find that this strategy not only leads to strong prefill-decoding interferences but also couples the resource allocation and parallelism plans for both phases. LLM applications often emphasize individual latency for each phase: time to first token (TTFT) for the prefill phase and time per output token (TPOT) of each request for th"},"verification_status":{"content_addressed":true,"pith_receipt":true,"author_attested":false,"weak_author_claims":0,"strong_author_claims":0,"externally_anchored":false,"storage_verified":false,"citation_signatures":0,"replication_records":0,"graph_snapshot":true,"references_resolved":false,"formal_links_present":false},"canonical_record":{"source":{"id":"2401.09670","kind":"arxiv","version":3},"metadata":{"license":"http://creativecommons.org/licenses/by-sa/4.0/","primary_cat":"cs.DC","submitted_at":"2024-01-18T01:03:38Z","cross_cats_sorted":[],"title_canon_sha256":"0d33d0c55679c57ec5838e6edba91ad995720d6acad5786bdc84bd4d928b38b8","abstract_canon_sha256":"f569adbd2cd432a8e7b23b038e29ffbf6dc13700229eaca419a06e38455dbcb6"},"schema_version":"1.0"},"receipt":{"kind":"pith_receipt","key_id":"pith-v1-2026-05","algorithm":"ed25519","signed_at":"2026-07-05T08:28:05.047370Z","signature_b64":"9XIVw3eryMElPsg2UgLgrZG5eobzyr+9i0lksBCb/NfBcMlq2T31uK9FkLVlwEM35aqcjlK59Z9pP4OqeTX/Dw==","signed_message":"canonical_sha256_bytes","builder_version":"pith-number-builder-2026-05-17-v1","receipt_version":"0.3","canonical_sha256":"0d2c50b7abe825473b7ace65f769a728a7922f820c98b63f1d5a1f858509fd60","last_reissued_at":"2026-07-05T08:28:05.046908Z","signature_status":"signed_v1","first_computed_at":"2026-07-05T08:28:05.046908Z","public_key_fingerprint":"8d4b5ee74e4693bcd1df2446408b0d54"},"graph_snapshot":{"paper":{"title":"DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving","license":"http://creativecommons.org/licenses/by-sa/4.0/","headline":"","cross_cats":[],"primary_cat":"cs.DC","authors_text":"Hao Zhang, Jianbo Hu, Junda Chen, Shengyu Liu, Xin Jin, Xuanzhe Liu, Yibo Zhu, Yinmin Zhong","submitted_at":"2024-01-18T01:03:38Z","abstract_excerpt":"DistServe improves the performance of large language models (LLMs) serving by disaggregating the prefill and decoding computation. Existing LLM serving systems colocate the two phases and batch the computation of prefill and decoding across all users and requests. We find that this strategy not only leads to strong prefill-decoding interferences but also couples the resource allocation and parallelism plans for both phases. LLM applications often emphasize individual latency for each phase: time to first token (TTFT) for the prefill phase and time per output token (TPOT) of each request for th"},"claims":{"count":0,"items":[],"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"source":{"id":"2401.09670","kind":"arxiv","version":3},"verdict":{"id":null,"model_set":{},"created_at":null,"strongest_claim":"","one_line_summary":"","pipeline_version":null,"weakest_assumption":"","pith_extraction_headline":""},"integrity":{"clean":true,"summary":{"advisory":0,"critical":0,"by_detector":{},"informational":0},"endpoint":"/pith/2401.09670/integrity.json","findings":[],"available":true,"detectors_run":[],"snapshot_sha256":"c28c3603d3b5d939e8dc4c7e95fa8dfce3d595e45f758748cecf8e644a296938"},"references":{"count":0,"sample":[],"resolved_work":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57","internal_anchors":0},"formal_canon":{"evidence_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"},"aliases":[{"alias_kind":"arxiv","alias_value":"2401.09670","created_at":"2026-07-05T08:28:05.046963+00:00"},{"alias_kind":"arxiv_version","alias_value":"2401.09670v3","created_at":"2026-07-05T08:28:05.046963+00:00"},{"alias_kind":"doi","alias_value":"10.48550/arxiv.2401.09670","created_at":"2026-07-05T08:28:05.046963+00:00"},{"alias_kind":"pith_short_12","alias_value":"BUWFBN5L5ASU","created_at":"2026-07-05T08:28:05.046963+00:00"},{"alias_kind":"pith_short_16","alias_value":"BUWFBN5L5ASUOO32","created_at":"2026-07-05T08:28:05.046963+00:00"},{"alias_kind":"pith_short_8","alias_value":"BUWFBN5L","created_at":"2026-07-05T08:28:05.046963+00:00"}],"events":[],"event_summary":{},"paper_claims":[],"inbound_citations":{"count":32,"internal_anchor_count":1,"sample":[{"citing_arxiv_id":"2607.05876","citing_title":"Think Before You Grid-Search: Floor-First Triage for LLM Serving","ref_index":46,"is_internal_anchor":true},{"citing_arxiv_id":"2606.20295","citing_title":"Token-Operations-Oriented Inference Optimization Techniques for Large Models","ref_index":184,"is_internal_anchor":false},{"citing_arxiv_id":"2606.28565","citing_title":"KernelSight-LM: A Kernel-Level LLM Inference Simulator","ref_index":57,"is_internal_anchor":false},{"citing_arxiv_id":"2606.18431","citing_title":"Beyond Prediction: Tail-Aware Scheduling for LLM Inference","ref_index":34,"is_internal_anchor":false},{"citing_arxiv_id":"2606.17081","citing_title":"The Price of Anarchy in Disaggregated Inference","ref_index":40,"is_internal_anchor":false},{"citing_arxiv_id":"2606.12556","citing_title":"ITME: Inference Tiered Memory Expansion with Disaggregated CXL-Hybrid Memories","ref_index":47,"is_internal_anchor":false},{"citing_arxiv_id":"2607.01579","citing_title":"OmniPilot: An Uncertainty-Aware LLM Inference Advisor for Heterogeneous GPU Clusters","ref_index":21,"is_internal_anchor":false},{"citing_arxiv_id":"2606.06453","citing_title":"Vortex: Efficient and Programmable Sparse Attention Serving for AI Agents","ref_index":44,"is_internal_anchor":false},{"citing_arxiv_id":"2606.00735","citing_title":"ViBE: Co-Optimizing Workload Skew and Hardware Variability for MoE Serving","ref_index":50,"is_internal_anchor":false},{"citing_arxiv_id":"2606.31093","citing_title":"Omni-Flow: A Unified Workflow Orchestration and Distributed KV Cache Sharing Framework for Multimodal Inference","ref_index":16,"is_internal_anchor":false},{"citing_arxiv_id":"2606.29708","citing_title":"Demystifying the Design Space and Best Practices for Heterogeneous LLM Inference and Serving","ref_index":1,"is_internal_anchor":false},{"citing_arxiv_id":"2606.29708","citing_title":"Demystifying the Design Space and Best Practices for Heterogeneous LLM Inference and Serving","ref_index":1,"is_internal_anchor":false},{"citing_arxiv_id":"2606.28565","citing_title":"KernelSight-LM: A Kernel-Level LLM Inference Simulator","ref_index":57,"is_internal_anchor":false},{"citing_arxiv_id":"2605.23389","citing_title":"AlignedServe: Orchestrating Prefix-aware Batching to Build a High-throughput and Computing-efficient LLM Serving System","ref_index":43,"is_internal_anchor":false},{"citing_arxiv_id":"2505.09999","citing_title":"ServeGen: Workload Characterization and Generation of Large Language Model Serving in Production","ref_index":56,"is_internal_anchor":false},{"citing_arxiv_id":"2605.07985","citing_title":"Dooly: Configuration-Agnostic, Redundancy-Aware Profiling for LLM Inference Simulation","ref_index":42,"is_internal_anchor":false},{"citing_arxiv_id":"2605.11333","citing_title":"MLCommons Chakra: Advancing Performance Benchmarking and Co-design using Standardized Execution Traces","ref_index":105,"is_internal_anchor":false},{"citing_arxiv_id":"2605.17410","citing_title":"Computational Challenges in Token Economics: Bridging Economic Theory and AI System Design","ref_index":19,"is_internal_anchor":false},{"citing_arxiv_id":"2512.09427","citing_title":"ODMA: On-Demand Memory Allocation Strategy for LLM Serving on LPDDR-Class Accelerators","ref_index":32,"is_internal_anchor":false},{"citing_arxiv_id":"2602.01785","citing_title":"CodeOCR: On the Effectiveness of Vision Language Models in Code Understanding","ref_index":113,"is_internal_anchor":false},{"citing_arxiv_id":"2605.14249","citing_title":"EnergyLens: Predictive Energy-Aware Exploration for Multi-GPU LLM Inference Optimization","ref_index":17,"is_internal_anchor":false},{"citing_arxiv_id":"2404.14294","citing_title":"A Survey on Efficient Inference for Large Language Models","ref_index":274,"is_internal_anchor":false},{"citing_arxiv_id":"2605.11999","citing_title":"The Illusion of Power Capping in LLM Decode: A Phase-Aware Energy Characterisation Across Attention Architectures","ref_index":25,"is_internal_anchor":false},{"citing_arxiv_id":"2605.11333","citing_title":"MLCommons Chakra: Advancing Performance Benchmarking and Co-design using Standardized Execution Traces","ref_index":105,"is_internal_anchor":false},{"citing_arxiv_id":"2605.02821","citing_title":"When Is the Same Model Not the Same Service? A Measurement Study of Hosted Open-Weight LLM APIs","ref_index":15,"is_internal_anchor":false}]},"formal_canon":{"evidence_count":0,"sample":[],"anchors":[]},"links":{"html":"https://pith.science/pith/BUWFBN5L5ASUOO32ZZS7O2NHFC","json":"https://pith.science/pith/BUWFBN5L5ASUOO32ZZS7O2NHFC.json","graph_json":"https://pith.science/api/pith-number/BUWFBN5L5ASUOO32ZZS7O2NHFC/graph.json","events_json":"https://pith.science/api/pith-number/BUWFBN5L5ASUOO32ZZS7O2NHFC/events.json","paper":"https://pith.science/paper/BUWFBN5L"},"agent_actions":{"view_html":"https://pith.science/pith/BUWFBN5L5ASUOO32ZZS7O2NHFC","download_json":"https://pith.science/pith/BUWFBN5L5ASUOO32ZZS7O2NHFC.json","view_paper":"https://pith.science/paper/BUWFBN5L","resolve_alias":"https://pith.science/api/pith-number/resolve?arxiv=2401.09670&json=true","fetch_graph":"https://pith.science/api/pith-number/BUWFBN5L5ASUOO32ZZS7O2NHFC/graph.json","fetch_events":"https://pith.science/api/pith-number/BUWFBN5L5ASUOO32ZZS7O2NHFC/events.json","actions":{"anchor_timestamp":"https://pith.science/pith/BUWFBN5L5ASUOO32ZZS7O2NHFC/action/timestamp_anchor","attest_storage":"https://pith.science/pith/BUWFBN5L5ASUOO32ZZS7O2NHFC/action/storage_attestation","attest_author":"https://pith.science/pith/BUWFBN5L5ASUOO32ZZS7O2NHFC/action/author_attestation","sign_citation":"https://pith.science/pith/BUWFBN5L5ASUOO32ZZS7O2NHFC/action/citation_signature","submit_replication":"https://pith.science/pith/BUWFBN5L5ASUOO32ZZS7O2NHFC/action/replication_record"}},"created_at":"2026-07-05T08:28:05.046963+00:00","updated_at":"2026-07-05T08:28:05.046963+00:00"}