{"id":"45480884-3d10-45dd-99c7-0cdb899754ed","arxiv_id":"2508.06133","paper_version":4,"verdict":"REJECT","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":2,"one_line_summary":"The abstract claims a constant-factor approximation algorithm for LLM serving scheduling, but the full text is an unrelated safety-benchmark paper, leaving the claimed result without any derivation or experiments.","lead":"This preprint's abstract claims a provable scheduling algorithm for faster LLM serving under memory limits, but the full text is an unrelated paper about safety testing of multimodal AI models. As submitted, the central result has no derivation, algorithm definition, or experiments, so nothing in the abstract can be verified.","discovery_kind":"unclear","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The abstract's central claim—NP-hardness plus a constant-factor guarantee for Sorted-F—is unsupported by the submitted full text, which is an unrelated paper; the artifact contains no model, F-metric definition, NP-hardness reduction, or approximation proof.","rationale":"The reader's weakest_assumption—that the manuscript body contains the model, F-metric, NP-hardness reduction, and proof—is exactly the load-bearing premise, and it is false for this artifact. The review rule that treats the full text as in-scope evidence makes the mismatch decisive: the body is a different paper on multimodal safety evaluation, with a footer pointing to another arXiv identifier. Since no derivation is present, there is no mathematical argument to stress-test; the correct finding is structural. We therefore agree with the reader's REJECT and recommend no change to the verdict. If the genuine LLM-serving paper exists elsewhere, it should be submitted with a matching abstract and reviewed on its own; the current artifact cannot support the central claim.","tokens_in":22813,"tokens_out":3415,"duration_ms":32833,"concrete_test":"Fetch the official arXiv record/PDF for arXiv:2508.06133 (not the supplied SDEval text) and search for the terms 'F-metric', 'Sorted-F', 'NP-hard', and 'constant-factor'. Then verify that the body contains (i) a formal model of prefill/decode batches with KV-cache budget, (ii) the F-metric definition, (iii) an NP-hardness reduction with proof, and (iv) a proof of the constant-factor guarantee. If any item is missing, the abstract's central claim is unsupported and the rejection stands. If the official record itself contains the SDEval paper, the mismatch is dispositive.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim requires four things in the manuscript body: (1) a formal statement of the offline/backlogged scheduling model with KV-cache budget and dynamic memory constraints; (2) a definition of the F-metric used by Sorted-F; (3) an NP-hardness reduction; and (4) a theorem and proof bounding Sorted-F's latency by a constant factor of the optimal offline schedule. The submitted full text under arXiv:2508.06133 is 'SDEval: Safety Dynamic Evaluation for Multimodal Large Language Models' by different authors, with footer arXiv:2508.06142v2. It contains none of these items: no scheduling model, no Sorted-F, no F-metric, no NP-hardness argument, no approximation proof, and no serving experiments. Treating the full text as in-scope evidence, the artifact is internally incoherent—the abstract asserts theorems while the body is a different paper. The claim is therefore a claim-without-derivation. This is not an assertion that the underlying mathematics is wrong; it is a structural fact about the artifact as submitted. A secondary modeling concern about the offline/backlogged assumption (known lengths, no preemption/rejection) is real but cannot even be assessed without the missing body.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The submission (arXiv:2508.06133) presents an abstract that claims to study offline scheduling for LLM serving under a fixed KV-cache memory budget, with NP-hardness, unbounded approximation ratios for standard policies, a Sorted-F algorithm using an F-metric, a constant-factor approximation guarantee, exact and heuristic implementations, and serving experiments. The full text supplied, however, is a completely different paper titled \"SDEval: Safety Dynamic Evaluation for Multimodal Large Language Models\" by different authors, with footer arXiv:2508.06142v2. The body contains no scheduling model, no Sorted-F algorithm, no F-metric definition, no theorem statements or proofs, and no LLM serving experiments. The claims in the abstract are therefore unsupported by any content in the manuscript as submitted.","tokens_in":22998,"tokens_out":2737,"duration_ms":31315,"significance":"If the abstract's results were established, they would constitute a useful theoretical and practical contribution: heterogeneous prefill/decode lengths with dynamic KV-cache memory constraints are central to LLM serving, and a provable constant-factor offline approximation would be valuable. The manuscript also promises reproducible implementations and public-workload experiments. However, none of these elements appear in the body. The artifact is internally incoherent: the abstract asserts theorems while the body is an unrelated safety-benchmark paper. Consequently, the significance cannot be assessed, and the submission as it stands is not a reviewable paper.","major_comments":[{"comment":"The body of the submission is not the paper described in the abstract. The full text is SDEval, an MLLM safety dynamic evaluation paper by different authors. There is no definition of the offline/backlogged scheduling model, no formal notion of request prefill/decode lengths, no KV-cache budget constraint, no batch-feasibility condition, no Sorted-F algorithm, no F-metric, and no approximation theorem. The abstract's central claim—\"We prove that Sorted-F achieves a constant-factor approximation guarantee\"—is therefore a claim without derivation. This is a load-bearing structural defect that cannot be repaired by local revision.","section":"Full text (title, Sections 1–7, Appendix)"},{"comment":"The abstract asserts NP-hardness for the scheduling problem and unbounded approximation ratios for FCFS, shortest-output-first, and total-size-based policies. These statements presuppose a formal optimization model. The body contains no such model, no decision problem statement, and no reduction. Without a model, the claims cannot be checked, and the reader cannot even identify the objective function or the feasibility constraints. This is a second load-bearing gap.","section":"Abstract, NP-hardness and unbounded-ratio claims"},{"comment":"The abstract states that \"Experiments on public workloads that combine short conversations and long-document summarization\" show latency reductions and closeness to an LP lower bound. The manuscript body contains no serving experiments, no workload description, no latency measurements, and no LP formulation. The only experiments in the body are on MLLM safety benchmarks (Sections 4–5 and the tables in the supplementary material), which are unrelated to this claimed evaluation.","section":"Abstract, empirical claims"},{"comment":"Even taken on its own, the abstract's offline/backlogged assumption—all requests and their exact prefill and decode lengths are known in advance, with a fixed KV-cache budget—is a substantial idealization relative to real serving systems, where generation lengths are unknown until decode completes and scheduling decisions are made online. The paper provides no discussion of how the constant-factor guarantee would transfer to online settings or to settings with preemption/rejection. Since the body is absent, this concern cannot be assessed beyond the abstract; it is flagged here so that any future resubmission addresses it explicitly.","section":"Abstract, offline/backlogged modeling assumption"}],"minor_comments":[{"comment":"The submission's footer shows arXiv:2508.06142v2 and a different title/authors, inconsistent with the submission ID 2508.06133. The authors should verify that the correct manuscript was uploaded.","section":"Metadata/footer"},{"comment":"The manuscript contains no equation numbers, theorem environments, or algorithm environments. If the correct manuscript is supplied, it must define all terms used in the abstract (e.g., F-metric, batch feasibility, total end-to-end latency) and provide formal theorem statements.","section":"Notation and structure"},{"comment":"In the body's supplementary tables, there are apparent formatting and typographical errors (e.g., Table 29 contains '141.6' in a PAR column, Table 36 contains '268.2', and Table 32 has a mismatched title 'Augmentation Style Transfer'). These are in the unrelated paper, but they further indicate that the submitted artifact is not in a polished state.","section":"Supplementary tables"}],"recommendation":"reject","confidential_remarks":"This appears to be a fundamental submission-level mismatch: the uploaded PDF is a different paper (SDEval, arXiv:2508.06142v2) than the abstract describes. I recommend that the editor verify the submission metadata and determine whether this is an upload error. Regardless of intent, the current artifact cannot be accepted or meaningfully revised; the correct paper, if it exists, would need to be submitted afresh and undergo normal review. The stress-test concern in the reader's report is fully confirmed by the manuscript text itself."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know one thing before spending time on this: the submission is internally incoherent. The abstract describes an LLM serving scheduling paper with theorem claims, and the full text is an entirely different paper about multimodal safety benchmarks. I don't see any version of the serving paper here, so there is no model, no F-metric, no NP-hardness reduction, no constant-factor proof, and no serving experiments to evaluate.\n\nThe abstract itself is worth taking seriously. If the claimed results are real—NP-hardness for variable prefill and decode lengths under a KV-cache budget, unbounded approximation ratios for FCFS/shortest-output/total-size policies, and a constant-factor guarantee for Sorted-F—that would be a genuinely new contribution. The combination is not in the named baselines or prior work I can see. The proposed method, a metric balancing batch size against downstream decode cost, is a reasonable idea, and the planned evaluation (exact DP, local search, LP lower bound) is sensible.\n\nBut none of that is present. Treating every part of the manuscript as in-scope evidence, the abstract asserts a theorem and the body nowhere states or proves it. This is worse than a proof gap: it is a claim-without-derivation. There is also a secondary modeling concern you would want to press on the abstract's offline/backlogged assumption—known prefill and decode lengths, no preemption or rejection—but I can't assess that without the missing model. It is a real concern for transfer to production serving, not a fatal flaw in the abstract's own terms.\n\nMy verdict: desk reject this artifact. The right fix is for the authors to submit again with the matching full text. If they do, the paper deserves a serious referee: the claimed result is important enough to warrant checking a full proof, and the novelty claim is credibly stated. As submitted, the reader's REJECT is right, and I don't see a way to peer-review a document that contains none of the claimed work.","headline":"The abstract promises a significant LLM serving scheduling result, but the submitted full text is an unrelated safety-benchmark paper, so none of the claims can be reviewed; desk reject this artifact, though a resubmission with the correct body would deserve a serious referee.","tokens_in":23572,"tokens_out":2432,"would_cite":false,"duration_ms":27853,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["90B35","68W25"],"pacs":[],"model":"deepseek-v4-flash","headline":"The abstract claims that scheduling LLM requests with mixed prefill and decode lengths is NP-hard, that standard priority policies can be arbitrarily bad, and that the Sorted-F algorithm achieves a constant-factor approximation in the offli","keywords":["LLM serving","offline scheduling","KV-cache budget","prefill length","decode length","constant-factor approximation","NP-hardness","F-metric"],"falsifier":"Run Sorted-F on a random backlog with known prefill and decode lengths and compare its total latency with the exact dynamic program's optimum: a single instance whose ratio exceeds the claimed constant refutes the approximation guarantee. A prior, cheaper check is to open the published artifact and verify that the model, the F-metric, and the proof are actually present — in the material supplied here they are not, so no calculation can currently be checked. Reproducing the reported latency experiments on public workloads would test the empirical side of the claim.","tokens_in":22582,"feed_emoji":"⏱️","tokens_out":14867,"duration_ms":129501,"temperature":0.7,"pith_summary":"The paper's abstract tries to establish a theorem about offline scheduling for LLM inference under a fixed KV-cache budget: when requests have heterogeneous prompt (prefill) and response (decode) lengths, it claims the batching problem is NP-hard, that standard policies such as first-come-first-served, shortest-output-first, and total-size-based prioritization can be worse than optimal by an unbounded factor, and that its Sorted-F algorithm — which repeatedly forms feasible batches using an F-metric that balances batch size against downstream decode cost — achieves a constant-factor approximation. A sympathetic reader would care because a constant-factor offline schedule would give serving systems a provably near-optimal way to dispatch an arriving backlog of requests with known lengths, replacing heuristics that can degrade without bound. The abstract must be read on its own here: the supplied full text is an unrelated paper on dynamic safety evaluation for multimodal large language models, so the model, the F-metric, the NP-hardness reduction, and the claimed proof are not present in the submitted material, and the claims cannot be checked against it.","feed_headline":"Sorted-F batches LLM requests within a constant factor of optimal","feed_subtitle":"Why it matters: common priority rules can be arbitrarily bad once prompt and output lengths differ.","key_machinery":"The central objects are the F-metric and the Sorted-F algorithm. Sorted-F repeatedly forms feasible batches, scoring each candidate batch by an F-metric that balances batch size against the downstream decode cost those requests will incur as they generate tokens. The metric is designed to track the dynamic memory constraint — prompt tokens fix initial KV-cache usage, and each generated token grows that usage — so a batch is feasible only if it fits the fixed budget over the whole of autoregressive generation. The claimed constant-factor guarantee is carried by this repeated, sorted batch-forming rule, while NP-hardness delimits what any policy can hope for. The abstract also names supporting","core_discovery":"Central claim: heterogeneous prompt (prefill) and output (decode) lengths change LLM-serving scheduling. With a backlog of requests of known lengths, a fixed KV-cache budget, and memory growing as tokens generate, the abstract asserts the offline batching problem is NP-hard; first-come-first-served, shortest-output-first, and total-size policies have unbounded approximation ratios; and Sorted-F — repeatedly forming feasible batches scored by an F-metric balancing batch size against downstream decode cost — achieves a constant-factor approximation, with an exact dynamic program, heuristics, and latency experiments. None of these definitions or proofs appears in the supplied full text, an unre","pith_inferences":["My inference: the constant-factor statement, as the abstract frames it, applies only to the offline/backlogged model with known lengths; real serving with unknown generation lengths, preemption, or rejection is a different problem, and the guarantee would not automatically transfer.","My inference: the immediate check for any reader — locating the F-metric definition and the proof in the body — fails for this artifact, because the supplied full text is a different paper; nothing in the submitted material establishes the abstract's theorems.","My inference: if the proof later appears, the natural next test is to run Sorted-F against an exact-optimal baseline on randomly generated backlogs to see whether the worst-case constant is tight on realistic length distributions."],"forward_implications":["If the guarantee holds, an operator with a known backlog can schedule to within a fixed factor of the minimum total latency — a worst-case bound that none of the priority heuristics offers.","The claimed unbounded ratios for common rules mean that batching choices based on intuition can degrade without limit as request mixes become more heterogeneous.","An exact dynamic program, as promised, would solve small instances optimally and give the heuristics a benchmark baseline.","The reported experiments, if reproducible, would make F-metric scheduling a drop-in latency reducer for workloads mixing short conversations with long summarization tasks."],"supporting_citations":[],"fun_headline_variants":["Sorted-F: constant-factor LLM batching under memory limits","Heterogeneous lengths break LLM scheduling heuristics","LLM serving: smart batching beats common rules","Constant-factor scheduling for variable-length LLM requests","Why shortest-output-first fails for LLM serving"],"cache_read_input_tokens":2816,"weakest_assumption_plain":"The guarantee rests on the offline/backlogged model — every request and its prefill and decode lengths are known in advance under a fixed KV-cache budget — and, for this artifact, on the manuscript body actually containing the F-metric, the NP-hardness reduction, and the constant-factor proof, which the supplied full text does not.","fun_headline_variants_meta":{"raw":{"variants":["Sorted-F: constant-factor LLM batching under memory limits","Heterogeneous lengths break LLM scheduling heuristics","LLM serving: smart batching beats common rules","Constant-factor scheduling for variable-length LLM requests","Why shortest-output-first fails for LLM serving"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000263,"raw_usage":{"total_tokens":1444,"prompt_tokens":760,"completion_tokens":684,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":504,"completion_tokens_details":{"reasoning_tokens":618}},"tokens_in":504,"tokens_out":684,"duration_ms":6844,"temperature":1.0,"reasoning_tokens":618,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T22:55:16.398876+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run Sorted-F on a random backlog with known prefill and decode lengths and compare its total latency with the exact dynamic program's optimum: a single instance whose ratio exceeds the claimed constant refutes the approximation guarantee. A prior, cheaper check is to open the published artifact and verify that the model, the F-metric, and the proof are actually present — in the material supplied here they are not, so no calculation can currently be checked. Reproducing the reported latency experiments on public workloads would test the empirical side of the claim.","supporting_citations":[],"review_version":1}