{"id":"cf8a1798-5c32-4955-a1db-005392af0e2b","arxiv_id":"2508.06948","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":0,"one_line_summary":"The abstract claims Kairos, a workflow-aware orchestrator, priority scheduler, and memory-aware dispatcher for shared LLMs, cuts multi-agent end-to-end latency by 17.8-28.4%, but the supplied body is a different paper.","lead":"A proposed AI agent orchestration system claims to cut end-to-end latency by 17.8 to 28.4 percent for workflows that share large language models under heavy load, by scheduling requests by delay sensitivity and dispatching by memory need. The full text provided is an unrelated paper about 6G metasurface antennas, so only the abstract could be reviewed.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Full text is an unrelated metasurface paper; the abstract's empirical claim has no supporting evidence in the submitted manuscript.","rationale":"The reader's verdict is UNVERDICTED with LOW confidence, based on the document mismatch: the supplied full text is a different paper. I agree that the work should be unverdictable from this material. However, the reader's stated weakest_assumption focuses on the technical premise that the orchestrator can accurately infer per-agent latency and memory demands. While that is a valid secondary concern, the load-bearing issue is more immediate: the submission's body is entirely unrelated to the abstract, so there is no evidence for the central empirical claim whatsoever. The reader's rationale already notes the mismatch, but their weakest_assumption does not identify it as the primary attack surface. Therefore agreement is partial. A concrete check is to fetch the actual arXiv PDF and verify its content; this settles whether the concern about missing support lands. If the actual PDF is indeed the Kairos paper, then the mismatch is an artifact of the review pipeline and the technical assumption becomes the main concern; if it is the metasurface paper, the abstract's claim is unsupported. I recommend keeping the reader's UNVERDICTED verdict, as the stress-test does not change it.","tokens_in":2029,"tokens_out":4031,"duration_ms":38208,"concrete_test":"Query the arXiv API for id 2508.06948 (e.g., http://export.arxiv.org/api/query?id_list=2508.06948) and download the actual PDF; verify whether its title and abstract match the Kairos claim and whether the body includes an evaluation section with baselines and latency measurements. If the PDF is the metasurface article, the submitted full text is a mismatch and the abstract's claim cannot be checked.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The provided full text under arXiv:2508.06948 is actually arXiv:2508.06952, 'Extremely Large-Scale Dynamic Metasurface Antennas for 6G Near-Field Networks.' It contains no reference to Kairos, multi-agent serving, LLM scheduling, or the claimed 17.8–28.4% latency reduction. Treating the entire text as in-scope evidence per protocol, the central claim is unsupported: there are no experimental results, baseline definitions, implementation details, or even a system description of Kairos in the body. The reader's weakest assumption about the orchestrator's ability to predict per-agent latency/memory is a meaningful technical premise, but it cannot be evaluated until the document identity is resolved. The most load-bearing concern is not a design flaw; it is that the submitted manuscript does not contain the paper that the abstract describes, so the central claim is unfalsifiable from the supplied material.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The abstract describes Kairos, a multi-agent orchestration system for LLM serving in the public cloud. Kairos comprises a workflow orchestrator that collects agent-specific information, a workflow-aware priority scheduler, and a memory-aware dispatcher, and is claimed to reduce end-to-end latency by 17.8% to 28.4% compared to state-of-the-art systems. The full text supplied under this arXiv identifier, however, is an unrelated article entitled \"Extremely Large-Scale Dynamic Metasurface Antennas for 6G Near-Field Networks: Opportunities and Challenges\" (arXiv:2508.06952). The body contains no mention of Kairos, multi-agent serving, LLM scheduling, or any experimental evaluation, so the paper as submitted does not contain the claimed system or evidence.","tokens_in":2194,"tokens_out":1461,"duration_ms":15058,"significance":"If the claimed architecture and quantitative results were properly presented, the work could be relevant to low-latency multi-agent LLM serving in shared-cloud settings. Workflow-aware scheduling and memory-aware dispatch are plausible mechanisms for reducing queuing and avoiding GPU overloading. However, because the submitted manuscript body is entirely a different paper, there is no verifiable content to substantiate the claimed 17.8%–28.4% latency reduction, no system description, no comparison to baselines, no workload characterization, and no evaluation methodology. The significance of the contribution cannot be assessed from the submitted material.","major_comments":[{"comment":"The full text of the manuscript is arXiv:2508.06952, a paper on XL-DMAs for 6G near-field networks. It never mentions Kairos, LLM serving, multi-agent workflows, or scheduling. The central claim in the abstract — that Kairos reduces end-to-end latency by 17.8% to 28.4% — is therefore entirely unsupported by the submitted body. The paper as submitted does not contain the research it describes.","section":"Full text (all sections)"},{"comment":"The abstract reports \"Experimental results show that Kairos reduces end-to-end latency by 17.8% to 28.4%,\" but the body provides no experimental setup, no workload description, no baseline definitions, no evaluation metrics, and no error bars. Even if the body were the correct paper, this single sentence would be insufficient to support the central quantitative claim; as it stands, there is no evidence at all.","section":"Abstract (experimental claim)"},{"comment":"The design relies on the orchestrator collecting \"agent-specific information for online workflow analysis\" and the scheduler using latency characteristics to set priorities. The submitted text provides no account of how such information is obtained, what assumptions it makes about the predictability of agent-level latency and memory demands, or what overhead the analysis introduces. Because these mechanisms are central to the claimed improvement, their absence is a load-bearing gap.","section":"Abstract (orchestrator premise)"}],"minor_comments":[{"comment":"The title and abstract describe a system called Kairos, but the rendered PDF title is about XL-DMAs for 6G. This is a serious bibliographic mismatch that must be corrected before any substantive review can begin.","section":"Title/Abstract"},{"comment":"The reference list of the supplied text corresponds to the metasurface article and contains no citations related to LLM serving, multi-agent systems, scheduling, or cloud inference. The submitted manuscript therefore lacks any related-work context for the claimed contribution.","section":"References"}],"recommendation":"reject","confidential_remarks":"This appears to be a submission error or a mismatch between the advertised paper and the uploaded full text. The body is a completely different arXiv paper. The editors should verify the correct manuscript and possibly request a resubmission, but as submitted the work cannot be reviewed for scientific content. Recommend rejection as a placeholder decision until the manuscript identity is resolved."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this submission is unreadable as-is. The abstract describes Kairos, a multi-agent orchestration system for shared-LLM serving with a 17.8–28.4% end-to-end latency reduction. The full text is a completely different paper, 'Extremely Large-Scale Dynamic Metasurface Antennas for 6G Near-Field Networks,' which never mentions Kairos, LLM serving, or any of the abstract's claims. So we have an abstract-only snapshot, not a paper.\n\nWhat the abstract offers is a sensible combination: a workflow orchestrator that collects agent-specific info, a priority scheduler that uses latency characteristics, and a memory-aware dispatcher. Those are standard components in the serving literature, and the claimed contribution is their workflow-aware combination for multi-agent workloads. If the experiments are real, a 17.8–28.4% latency reduction over state-of-the-art baselines would be a useful engineering result in a busy area. I can't give more credit than that from the abstract alone.\n\nThe soft spot is not a subtle design flaw—it's that the body doesn't match the abstract. The claimed result has no methodology, no workload description, no baselines, no error bars, and no artifacts in the submitted text. The reader's identified premise—that the orchestrator can predict per-agent latency and memory demands accurately enough at request time—is genuinely load-bearing, but it is also unevaluable until we see the actual Kairos paper. I'd add that the abstract names no prior systems, so it's impossible to check novelty against the literature from this material alone. These are all consequences of the document mismatch, not necessarily defects in the underlying work.\n\nIf I take the abstract at face value, the work is plausibly a serious systems paper in the LLM-inference-serving subfield. But I cannot in good conscience ask a referee to review a manuscript whose body is a different arXiv paper. This should be returned to the authors to fix the submission, then sent to peer review if the actual Kairos text supports the abstract's claims. Right now, there is no version of the paper in front of me that can be evaluated.\n\nRecommendation: desk-return with a request to resubmit the correct full text. If the corrected paper matches the abstract, it deserves a serious referee.","headline":"Supplied manuscript is the wrong paper: abstract describes the Kairos LLM-serving system, body is an unrelated 6G metasurface article, so the central latency claim has no supporting evidence in this submission.","tokens_in":2706,"tokens_out":2340,"would_cite":false,"duration_ms":22130,"reading_group":"no","serious_thinker":"unclear","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Kairos reduces end-to-end latency for multi-agent LLM serving by 17.8% to 28.4% under excessive public-cloud load.","keywords":["multi-agent serving","LLM","end-to-end latency","workflow-aware scheduling","memory-aware dispatch","public cloud","excessive load","orchestration"],"falsifier":"Run a controlled experiment with two agent classes of known, different delay sensitivities and memory footprints on a shared LLM cluster under excessive load, comparing Kairos against a baseline scheduler that ignores agent-level information; if end-to-end latency does not fall by at least the abstract's claimed 17.8% margin, or if the orchestrator's analysis overhead consumes the gain, the central claim is falsified.","tokens_in":1901,"feed_emoji":"⚡","tokens_out":3266,"duration_ms":30074,"temperature":0.7,"pith_summary":"Kairos is a multi-agent orchestration system that aims to lower end-to-end latency for LLM-based multi-agent applications when many agents share the same LLM and the cluster is overloaded. The paper argues that existing serving systems ignore differences in inter-agent latency and resource needs, and that this oversight degrades performance. Kairos combines a workflow orchestrator, a workflow-aware priority scheduler, and a memory-aware dispatcher to correct this. If the experimental results hold, Kairos achieves a 17.8% to 28.4% latency reduction over state-of-the-art baselines.","feed_headline":"Kairos cuts multi-agent LLM latency 17.8–28.4%","feed_subtitle":"Combines workflow analysis, priority scheduling, and memory-aware dispatch to share LLMs under excessive load.","key_machinery":"The central mechanism is the coordinated combination of three components: the workflow orchestrator, which gathers agent-specific information and performs online workflow analysis; the workflow-aware priority scheduler, which ranks request serving order by latency characteristics to cut queuing delay; and the memory-aware dispatcher, which assigns requests to LLM instances based on memory footprints to prevent GPU overload. The key design principle is that request scheduling must account for inter-agent latency and resource differences, not just treat all requests uniformly.","core_discovery":"Kairos is a system designed to optimize end-to-end latency for multi-agent applications that share LLMs in the public cloud. It consists of three components: a workflow orchestrator that collects agent-specific information for online workflow analysis; a workflow-aware priority scheduler that sets serving priority based on latency characteristics to reduce overall queuing; and a memory-aware dispatcher that routes requests to LLM instances according to memory demands to avoid GPU overloading. The central claim is that this combination reduces end-to-end latency by 17.8% to 28.4% compared to state-of-the-art works.","pith_inferences":["The gains depend on how accurately the orchestrator can predict per-agent latency characteristics and memory demands online; if these profiles are noisy or shift over time, the scheduler could mis-rank requests and the improvement could shrink or reverse.","Because the orchestrator itself adds analysis overhead, the net benefit is workload-dependent; the approach is most valuable when queueing and GPU overload dominate the latency budget, and less so in lightly loaded settings.","The same three-component pattern could extend to other shared-model serving contexts, such as multi-tenant GPU clusters hosting multiple fine-tuned models, where memory-aware dispatch could reduce swapping, preemption, and interference.","A direct comparison against an oracle scheduler with perfect agent information would reveal how much of the claimed gain comes from prediction accuracy versus from the scheduling policy itself."],"forward_implications":["If the claimed latency reductions hold, multi-agent serving systems can substantially improve user-perceived performance by incorporating workflow structure into scheduling decisions.","Memory-aware dispatch can prevent GPU overloading under excessive load, potentially increasing throughput and stability of shared LLM clusters.","The orchestration pattern could generalize beyond multi-agent applications to any scenario where multiple request types share a model and have heterogeneous latency or memory demands.","The reported 17.8% to 28.4% improvement suggests that ignoring agent-level information leaves significant performance on the table in overloaded conditions."],"supporting_citations":[],"fun_headline_variants":["Kairos cuts multi-agent LLM latency up to 28.4%","Kairos orchestrates agents to cut LLM latency 17.8–28.4%","Shared LLM overload? Kairos trims multi-agent latency 17.8–28.4%","Kairos: memory-aware dispatch cuts multi-agent LLM latency by 17.8–28.4%"],"cache_read_input_tokens":2816,"weakest_assumption_plain":"The design rests on the premise that an orchestrator can collect agent-specific information online and from it infer each request's latency characteristics and memory demands accurately enough to prioritize and dispatch better than existing schedulers.","fun_headline_variants_meta":{"raw":{"variants":["Kairos cuts multi-agent LLM latency up to 28.4%","Kairos orchestrates agents to cut LLM latency 17.8–28.4%","Shared LLM overload? Kairos trims multi-agent latency 17.8–28.4%","Kairos: memory-aware dispatch cuts multi-agent LLM latency by 17.8–28.4%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00059,"raw_usage":{"total_tokens":2588,"prompt_tokens":712,"completion_tokens":1876,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":456,"completion_tokens_details":{"reasoning_tokens":1773}},"tokens_in":456,"tokens_out":1876,"duration_ms":13646,"temperature":1.0,"reasoning_tokens":1773,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T22:25:14.204588+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a controlled experiment with two agent classes of known, different delay sensitivities and memory footprints on a shared LLM cluster under excessive load, comparing Kairos against a baseline scheduler that ignores agent-level information; if end-to-end latency does not fall by at least the abstract's claimed 17.8% margin, or if the orchestrator's analysis overhead consumes the gain, the central claim is falsified.","supporting_citations":[],"review_version":1}