{"id":"bac36568-0449-4ea6-a6f4-bacbb039c6f7","arxiv_id":"2412.21200","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Distributed LLM agents can gossip prompts and keep queues bounded when α((k+1)M+1)λ < 1, a condition the paper derives and tests in a small pilot.","lead":"The paper studies a network of phones and small computers, each running its own language model, that pass prompts to each other to get better answers without a central server. It gives a simple rule for when those devices' waiting queues stay small, and tests answer quality with a small experiment.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 1's proof applies an isolated single-queue rate condition to a feedback and synchronization network; without a network-level stability proof, Eq. (1) is not established.","rationale":"I agree with the reader's weakest_assumption: the core problem is in Section III, Eq. (5), where R_in < R_out is borrowed from single-queue theory without the assumptions that make it sufficient. The failure is not merely a missing detail; the system is a multiclass network with routes that revisit the origin queue and with fork-join synchronization before aggregation, so rate balance does not imply positive recurrence. A concrete two-class counterexample can be probed by simulation. The experiments do not rescue the theorem: they report queue sizes for a few configurations and never sweep lambda or alpha around the threshold, and the footnote limits evaluation to 10 samples. The text also says k=4 gives the highest accuracy while Tables I and II stop at k=3, an internal inconsistency. For these reasons the manuscript is not ready for acceptance as is; a conditional verdict requiring a formal stability proof and stronger evaluation is appropriate, matching the reader's verdict.","tokens_in":6897,"tokens_out":10364,"duration_ms":110192,"concrete_test":"Simulate Algorithm 1 for n=4, k=3, M=2 with Poisson prompt arrivals and two service classes at each queue: proposer jobs with mean alpha_p and aggregation/concatenation jobs with mean alpha_a greater than alpha_p. Choose lambda so that alpha_p((k+1)M+1)lambda is less than 1 but lambda[(M+1)alpha_a + M k alpha_p] is greater than 1. If any queue length grows without bound over a long horizon (e.g., 10^8 arrivals), Theorem 1's condition is not a valid stability criterion. An analytical alternative is to derive the fluid limit of the network and check whether condition (1) guarantees stability of the fluid limit.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section III's proof of Theorem 1 reduces each device queue to an isolated FCFS queue and invokes 'for a stable FCFS queue, the stability condition is known to be R_in < R_out.' That single-queue criterion requires the arrival process to be exogenous, or at least independent of service, and the service times to have a well-defined mean. Neither holds here: an edge queue receives its own user's prompts at multiple stages (initial inference, per-layer concatenation, final aggregation), proposer jobs that are departures of other queues selected by gossip, and aggregation jobs that are released only after k neighbor responses have returned. The resulting input process is endogenous, multiclass, and synchronized. Per-queue traffic intensity less than 1 is necessary for rate balance but is not generally sufficient for stability in such networks; a rigorous argument (e.g., fluid limit or Foster-Lyapunov) with explicit interarrival and service distributions is required. In addition, the proof uses a single mean inference time alpha for all jobs, although concatenation and aggregation prompts are longer than initial prompts; if these classes have different mean service times, the correct single-queue utilization is sum over classes of lambda_c times E[S_c], not alpha times the total arrival rate. Thus Eq. (1) is not actually derived by the argument given.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a distributed mixture-of-agents (MoA) architecture for edge inference, where each user's device runs its own LLM and devices collaborate by gossiping prompts and responses. The authors model each device's workload as a queue and derive Theorem 1, which states that the device queues remain stable under the condition α((k+1)M+1)λ < 1, where λ is the per-user prompt arrival rate, k is the number of proposers per layer, M is the number of layers, and α is the average LLM inference time. The proof is based on counting the total arrival rate at a device queue and applying a standard single-queue stability condition. The paper also reports experiments on a 10-sample subset of AlpacaEval 2.0 using open-source LLMs, comparing accuracy, latency, and average queue size across different MoA configurations with homogeneous and heterogeneous LLMs.","tokens_in":7241,"tokens_out":4320,"duration_ms":43326,"significance":"If Theorem 1 were rigorously established under explicit and reasonable stochastic assumptions, the result would provide a simple design rule for bounding queue sizes in distributed MoA systems, which is relevant to memory-constrained edge inference. The problem formulation is timely given the growing interest in decentralized LLM collaboration. The paper includes an implementation and releases code, which is a strength. However, as written, the central theoretical claim is not proven: the proof applies a per-queue rate condition to a network with endogenous, multiclass, and synchronized arrivals, without specifying the distributional assumptions needed for that condition to imply stability. The experimental validation is also too limited (10 samples, no error bars) to support the quantitative comparisons. The core idea is salvageable, but the manuscript requires substantial revision to meet the standard of a refereed journal.","major_comments":[{"comment":"The proof of Theorem 1 reduces each device queue to an isolated FCFS queue fed by an exogenous arrival stream and applies the rate condition R_in < R_out. This step is not justified because the arrival process at a device is endogenous and multiclass: it comprises the user's own initial prompts, prompts forwarded from other devices after being generated by those users, and concatenation and aggregation jobs that are released only after synchronization with k neighbors' responses. Under such feedback and synchronization, per-queue traffic intensity less than one is not generally sufficient for stability. The authors need to state explicit stochastic assumptions (e.g., Poisson arrivals, independent and identically distributed service times, and a precise description of the neighbor-selection and layering process) and provide a network-level stability proof, for instance via a Foster-Lyapunov criterion or fluid limit, before Eq. (1) can be accepted.","section":"III (Theorem 1 proof)"},{"comment":"The derivation of Eq. (1) uses a single mean inference time α for all tasks. However, the jobs processed by a device queue are heterogeneous: initial prompts, prompts generated from concatenated responses (original prompt plus up to k responses plus the system prompt), and final aggregation prompts have systematically different lengths and hence different mean service times. For a multiclass FCFS queue the correct utilization is Σ_c λ_c E[S_c], not α × (total arrival rate). Unless α is redefined as the class-weighted average service time and the class rates are computed accordingly, the inequality in Eq. (1) does not follow from the argument presented.","section":"III, Eq. (1)"},{"comment":"The accuracy results are based on only 10 samples from AlpacaEval 2.0, with no error bars, confidence intervals, or statistical tests. Consequently, claims such as 'The highest accuracy and latency are observed with a configuration of M = 2 layers and k = 4 proposers' are not supported by the data: Table I contains no k = 4 row, and the observed differences between configurations (e.g., 39.63% vs. 39.89% for M = 1, k = 2 and M = 2, k = 2) are within the noise one would expect from a 10-sample evaluation. The experimental section should either be substantially expanded or explicitly framed as a demonstration of the system's operation rather than a quantitative comparison.","section":"IV, Tables I and II"}],"minor_comments":[{"comment":"The benchmark name 'Flusk 2.0' should be corrected to the intended name (e.g., FLASK) for consistency with the literature.","section":"Abstract and Section I"},{"comment":"The neighbor selection 'uniformly at random, k out of other n−1 neighboring devices' is not precise about whether selection is with replacement and whether it is independent across layers; these details affect the arrival-rate calculation in Eq. (2).","section":"Section II"},{"comment":"The variable j is used both as the layer index in the outer loop and as an index in the prompt-arrival procedure, which makes the pseudocode confusing. In addition, line 13 ('j ← j + 1') appears to increment the layer inside the per-user procedure, which would be a bug if taken literally.","section":"Algorithm 1"},{"comment":"The phrase 'M = 2layers' should be 'M = 2 layers', and the sentence referencing k = 4 proposers should be aligned with the configurations actually listed in the tables.","section":"Section IV"},{"comment":"The limitation to 10 samples is stated only in a footnote; the abstract and conclusion should also qualify the experimental claims accordingly.","section":"Footnote 1"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is at the intersection of queueing theory and LLM systems. The main theoretical result is a simple rate condition, but the proof is currently a back-of-the-envelope calculation that does not constitute a rigorous stability proof. If the authors can add the missing assumptions and a proper proof, the paper would be of interest to the information theory community. The experimental section is too preliminary for the claims made. I recommend major revision rather than rejection because the core idea is timely and the faults are fixable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nThis paper is a reasonable first cut at a real question: how to keep queues bounded when edge devices run a distributed mixture-of-agents protocol. The specific result is the condition α((k+1)M+1)λ < 1 for the per-device queue, and the rate counting behind it is transparent and internally consistent. The authors also build an actual distributed MoA system and report AlpacaEval numbers, which is more than most theory papers do. Credit where due: the model is clearly described, the system prompt is given, and the footnote admitting only 10 samples is honest.\n\nThe soft spots are real, though. The proof of Theorem 1 is a mean-rate balance applied as though each device were a single isolated FCFS queue. It isn't: the arrival stream at a device is a mix of its own prompts, proposer jobs from other devices, and aggregation jobs that are only released after k responses come back. That last synchronization step generates dependencies, and the paper never shows the input processes are stationary and ergodic enough for the standard rate condition to imply stability. There's also a class-of-service problem: if aggregation or concatenation prompts take longer than the initial inference, the utilization is not α times the total arrival rate but a sum over job classes. So Eq. (1) is plausible, but the proof as written doesn't establish it. This is the main thing a referee should push on.\n\nThe experiments are suggestive at best: 10 samples, no error bars, and the claim in the text about M=2, k=4 doesn't match a table that stops at k=3. Minor, but it adds to the impression that the empirical part was rushed. The abstract also says the stability condition is \"validated experimentally,\" but the experiments only report queue sizes at one λ; they never vary λ to find a threshold. That's an overclaim.\n\nNovelty is modest—this is a direct rate calculation for a known architecture, not a new analytical framework—but the distributed setting is a legitimate extension, and the design rule is easy to use.\n\nI'd send this to peer review, because a good referee can force a proper proof or a properly hedged claim, and the system model is worth having on record. It should not be accepted in its current form; it could become a short, useful paper after revisions. For a reading group, maybe; it's an easy read and a good example of where queueing theory meets LLM inference, but there's not enough rigor to make it a cornerstone cite.","headline":"A plausible rate-balance stability condition for distributed MoA, but the proof is too quick and the experiments are only suggestive.","tokens_in":7620,"tokens_out":7930,"would_cite":false,"duration_ms":88416,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A simple rate inequality keeps device queues bounded in distributed Mixture-of-Agents inference.","keywords":["distributed mixture-of-agents","edge inference","large language models","gossip algorithms","queueing stability","semantic communication","decentralized inference","AlpacaEval"],"falsifier":"Run the distributed MoA with Poisson prompt arrivals and exponential inference times at rates satisfying $\\alpha((k+1)M+1)\\lambda < 1$, and track the longest queue over a long horizon; if any queue grows without bound instead of reaching a stationary distribution, the condition is not sufficient.","tokens_in":6710,"feed_emoji":"🤖","tokens_out":7391,"duration_ms":65205,"temperature":0.7,"pith_summary":"This paper proposes running Mixture-of-Agents (MoA) collaboratively on edge devices without a central server: each user's LLM gossips prompts and responses to $k$ randomly chosen neighbors across $M$ layers, then aggregates the results. Its central theoretical claim is a stability condition for the device queues, $\\alpha((k+1)M+1)\\lambda < 1$, where $\\lambda$ is the prompt generation rate and $\\alpha$ is the mean inference time. The paper validates this condition with experiments on open-source LLMs, observing that more layers and more proposers improve response accuracy at the cost of larger queues and latency. If the condition holds, memory-limited edge devices can participate in MoA without unbounded queue growth.","feed_headline":"Distributed edge LLM queues stay bounded under a simple rate rule","feed_subtitle":"More layers and proposers improve answers, but each one multiplies queue traffic; the paper pins down the exact cutoff.","key_machinery":"The load-bearing object is the per-device input-throughput identity $\\mathrm{R}_{\\mathrm{in}} = ((k+1)M+1)\\lambda$, built from the uniform random neighbor selection: any given device receives its own traffic at rate $\\lambda$ plus each other user's traffic with probability $k/(n-1)$, so one layer contributes $(k+1)\\lambda$, and the total scales by $M$ layers plus one aggregation step. The argument then applies the single-server queue stability rule that a FCFS queue is stable when the mean arrival rate is below the mean service rate, $\\mathrm{R}_{\\mathrm{in}} < 1/\\alpha$.","core_discovery":"The authors claim that in a symmetric network of $n$ edge devices, each running its own LLM, the aggregate arrival rate into any one device queue is $((k+1)M+1)\\lambda$: the device's own prompts arrive at rate $\\lambda$, each of the other $n-1$ users forwards its prompts to this device with probability $k/(n-1)$ per layer, and the process repeats over $M$ layers plus one final aggregation pass. Because an LLM with mean inference time $\\alpha$ completes work at rate $1/\\alpha$, a standard first-come-first-served queue remains stable whenever $((k+1)M+1)\\lambda < 1/\\alpha$, equivalently $\\alpha((k+1)M+1)\\lambda < 1$. For heterogeneous devices the condition is governed by the slowest LLM, i.e., by the largest mean inference time $\\alpha_{\\max}$. Experiments on a four-device testbed confirm the qualitative relationship between the configuration parameters and average queue size, and show that diverse LLMs in the MoA improve response quality.","pith_inferences":["The queueing analysis assumes arrivals at each queue behave like a stationary renewal process with the stated mean rate; if real prompt traffic is bursty or heavy-tailed, the mean-rate inequality alone may not prevent long queue excursions, and a buffer-sizing rule based on the queue tail would be needed.","Because layer $j$ prompts are generated only after layer $j-1$ responses return, the arrivals across layers are not independent; a refined model with feedback could tighten or weaken the stability threshold in regimes where aggregation stalls.","The experimental accuracy comparison uses only ten AlpacaEval samples, so the ranking of MoA configurations should be treated as suggestive; repeating on the full benchmark could change which $(M,k)$ pair is preferred.","The same rate-counting argument could be applied to other multi-hop collaborative inference schemes, such as sparse agent selection with early stopping, yielding analogous stability conditions for those protocols."],"forward_implications":["For any edge deployment, the inequality $\\alpha((k+1)M+1)\\lambda < 1$ gives a direct design rule for choosing the number of layers $M$ and proposers $k$ before queues grow without bound.","When devices run different LLMs, stability is limited by the slowest model, so the fastest devices do not help if one device's mean inference time is too large.","Raising $M$ or $k$ improves response accuracy but increases average queue size and latency, so accuracy targets must be balanced against memory and delay budgets.","The decentralized gossip design removes the central server as a single point of failure, at the cost of each device carrying extra traffic from neighbors.","Using diverse open-source LLMs rather than one repeated model yields higher AlpacaEval accuracy for the same layer and proposer counts."],"supporting_citations":[{"why":"Supplies the MoA proposer-aggregator architecture and the system prompt used for response synthesis.","marker":"[6]"},{"why":"Introduces sparsity and early stopping in multi-agent LLMs, the efficiency direction the distributed design extends.","marker":"[7]"},{"why":"Provides the epidemic gossip algorithms that justify decentralized device-to-device prompt exchange without a central server.","marker":"[15]"},{"why":"Supplies the gossip-algorithms framework whose latency and timeliness properties the paper contrasts with semantic content.","marker":"[16]"},{"why":"Models age of gossip in networks, giving the timeliness perspective the queueing analysis builds on.","marker":"[17]"}],"fun_headline_variants":["Edge MoA: queues stay bounded if αλ((k+1)M+1) < 1","More layers and proposers boost MoA quality, but watch the queue cutoff","Simple inequality gates stable queues in distributed edge MoA","αλ((k+1)M+1) < 1: the edge MoA queue rule"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The proof assumes the combined stream of local and gossiped prompts at each device behaves like a stationary random process with a well-defined mean rate, so the classic rule 'mean arrival rate below mean service rate keeps a queue stable' applies.","fun_headline_variants_meta":{"raw":{"variants":["Edge MoA: queues stay bounded if αλ((k+1)M+1) < 1","More layers and proposers boost MoA quality, but watch the queue cutoff","Simple inequality gates stable queues in distributed edge MoA","αλ((k+1)M+1) < 1: the edge MoA queue rule"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001387,"raw_usage":{"total_tokens":5666,"prompt_tokens":1046,"completion_tokens":4620,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":662,"completion_tokens_details":{"reasoning_tokens":4529}},"tokens_in":662,"tokens_out":4620,"duration_ms":35076,"temperature":1.0,"reasoning_tokens":4529,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T22:59:45.620886+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the distributed MoA with Poisson prompt arrivals and exponential inference times at rates satisfying $\\alpha((k+1)M+1)\\lambda < 1$, and track the longest queue over a long horizon; if any queue grows without bound instead of reaching a stationary distribution, the condition is not sufficient.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the epidemic gossip algorithms that justify decentralized device-to-device prompt exchange without a central server."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the gossip-algorithms framework whose latency and timeliness properties the paper contrasts with semantic content."}],"review_version":1}