{"id":"f632ba12-9d5b-4d26-8944-8177a9b7d54d","arxiv_id":"2509.04979","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"AgentRank-UC ranks agents by mixing usage frequency and measured competence through two PageRank-style equations, and DOVIS defines the telemetry protocol that feeds it.","lead":"This paper proposes a five-layer protocol called DOVIS and a ranking algorithm called AgentRank-UC that lets AI agents rate each other by how often they are used and how well they perform, so future agent-powered web services can discover trustworthy agents. It proves mathematical properties of the ranking and tests them in simulated agent marketplaces.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"AgentRank-UC's Sybil-resistance and outcome-monotonicity guarantees presuppose honest competence telemetry; DOVIS verification is unmodeled and success inflation is never simulated, so the adversarial claim is unsupported.","rationale":"The reader's weakest_assumption—caller integrity—is the single most load-bearing concern. The mathematical core (Theorem 4.1 contraction, Theorem 4.10 strict positivity, Theorem 4.13 perturbation bounds) is largely standard and likely correct. The monotonicity proof gap in Theorem 4.7 is real but fixable: using the Green's-function identity G_{j l} = P_l(hit j) G_{j j}, one can show y_j is nondecreasing in C_{i j}, so the theorem probably holds despite the inadequate proof. By contrast, the strategic-misreporting concern is not fixable by a proof patch: it questions whether the rank vector r tracks true competence at all when callers can profitably lie. The paper acknowledges the threat but offers no formal incentive-compatibility analysis, and its simulations omit the attack. Thus the central claim of Sybil resistance is conditional on an unverified and unmodeled assumption. This supports the reader's CONDITIONAL verdict: the paper is promising but not yet acceptable as an adversarial ranking substrate. The lack of code/data/seeds compounds the issue but is secondary.","tokens_in":25345,"tokens_out":15268,"duration_ms":151628,"concrete_test":"Re-run Experiment 5 (Section 5.9) with a strategic success-inflation attack: Sybil callers inflate n_success for clique callees by δ ∈ {5%, 10%, 20%} of calls, coordinate callee acknowledgments to match call counts, and keep inflated records within the protocol's audit tolerance; compare SybilMass and Quality@10 with the honest-reporting baseline and usage-only. If SybilMass under AgentRank-UC approaches usage-only levels or Quality@10 degrades, the Sybil-resistance claim fails under the paper's own threat model. Also compute, for each δ and audit probability 5%, the expected ranking gain versus expected penalty to check whether honest reporting is an equilibrium.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 2.3 calls strategic misreporting the 'most serious threat,' yet the formal guarantees assume the competence kernel Q is built from truthful caller aggregates. Theorem 4.17's Sybil bound explicitly requires 'collusion does not improve competence (i.e., Q is unchanged on S)'; if Sybils inflate n_success or suppress failures, y_S can approach 1 and bound (2) provides no protection. The proposed defenses—signatures, callee acknowledgments, 1–5% audits—are described but never modeled: no theorem bounds the distortion of Q under rational lying, and no experiment in §5.9 simulates success inflation (the Sybils there honestly report mediocre competence and only pump usage). Consequently, the advertised 'monotone in outcomes,' 'cold-start fair,' and 'Sybil-resistant' properties are guarantees about reported outcomes, not real competence. This is an unverified environmental assumption, and it is load-bearing because the entire DOVIS + AgentRank-UC value proposition is competence-aware discovery in an open, adversarial ecosystem.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DOVIS, a five-layer protocol (Discovery, Orchestration, Verification, Incentives, Semantics) for collecting minimal, signed aggregate telemetry in an open ecosystem of AI agents, and AgentRank-UC, a ranking algorithm that combines a usage PageRank fixed point x = αPᵀx + (1−α)v and a competence fixed point y = βQᵀy + (1−β)w into a fused score r = normalize(xᵖ ⊙ y^{1−p}). The manuscript presents theorems asserting existence, uniqueness, linear convergence, continuity, monotonicity, cold-start fairness, perturbation stability, and Sybil non-amplification, together with simulations on synthetic worlds with archetypal agents, shocks, and adversarial cliques. The central claim is that AgentRank-UC is a well-posed, competence-aware ranking substrate for the Agentic Web when built on DOVIS telemetry.","tokens_in":25635,"tokens_out":17818,"duration_ms":175606,"significance":"If the theoretical guarantees held in full, the paper would provide a useful concrete starting point for competence-aware agent discovery, combining a minimal telemetry schema with a clean two-kernel PageRank-style algorithm. The contraction fixed-point results (Theorems 4.1 and 4.13) are standard but correctly presented, and the explicit Sybil mass bound in Theorem 4.17 is a nice illustration of how teleportation limits usage-only amplification. The DOVIS protocol is described at a level that could inform prototype implementations. However, two load-bearing issues limit the significance: the monotonicity proof in Theorem 4.7 has a genuine gap, and the Sybil-resistance and outcome-monotonicity guarantees are conditional on competence telemetry being truthful, an assumption the paper itself identifies as the most serious threat but never models formally. The simulations are a useful sanity check but do not close those gaps.","major_comments":[{"comment":"The proof asserts without argument that \"Row-normalization preserves monotonicity in the target column j: increasing C_ij raises the relative share of mass directed to j,\" and then concludes that the fixed point y* is monotone in each entry of Q. This is not a standard fact and is generally false: increasing C_ij increases Q_ij but decreases Q_ik for k≠j, so the operator T_Q does not increase entrywise in Q. A perturbation calculation for y = βQᵀy + (1−β)w shows the sign of dy_j depends on differences of Green's function entries (I − βQᵀ)^{-1}; there is no reason it is sign-definite. The monotonicity theorem is therefore not established as stated. Since \"monotone in outcomes\" is a headline guarantee, this gap is load-bearing.","section":"§4.4, Theorem 4.7 (proof Step 2)"},{"comment":"Theorem 4.17's Sybil bound (2) and the Sybil-resistance conclusions in §5.9 assume \"collusion does not improve competence (i.e., Q is unchanged on S)\". But §2.3 explicitly calls success inflation and failure suppression the \"most serious threat,\" and the Exp-5 simulation only pumps usage while Sybils honestly report mediocre competence. No theorem bounds the distortion of Q under strategic misreporting, and no experiment simulates success inflation. Thus the advertised Sybil resistance and the related monotonicity guarantee are guarantees about reported outcomes, not about real competence in the open adversarial setting that motivates DOVIS. The verification/incentive layer needs a formal model, or the claims must be restated as conditional on honest or audit-bounded telemetry.","section":"§2.3, §5.9, Theorem 4.17"},{"comment":"Even within the honest-telemetry setting, the Sybil experiment is too narrow to support the general robustness claim: it considers a single configuration of clique size, link density, and utility penalties, reports only the final epoch (plus a trajectory plot), and gives no error bars or multiple seeds. More importantly, the experiment does not vary the audit rate, penalty strength, or prior weighting that DOVIS's verification layer would introduce, so it does not test the protocol-level defenses that the paper argues are essential. This is a secondary issue relative to the unmodeled success-inflation threat, but it should be addressed in a revision.","section":"§5.9, Exp-5"}],"minor_comments":[{"comment":"The title reads \"it's Algorithm\"; should be \"its Algorithm.\"","section":"Title"},{"comment":"The experiments repeatedly refer to \"archetypes in Section 4.2,\" but the archetypes are defined in Section 5.2 (World Model); Section 4 is the theory section. Please fix the cross-reference.","section":"§5.2 and Exp-1/2/3/4"},{"comment":"Figure 1 (radar chart of archetypes) is referenced but no image appears in the provided text. Ensure the figure is actually included.","section":"Figure 1"},{"comment":"The utility weights θ1...θ5 are introduced without guidance on calibration or default values. If they are free parameters, the paper should say so explicitly and discuss sensitivity; if they are part of the protocol, their semantics should be specified.","section":"§3.3, Step 3"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is a proposal for telemetry-based agent ranking, not a breakthrough. What is genuinely new: the OAT-Lite telemetry schema and the DOVIS five-layer protocol, which are concrete enough to build against. The AgentRank-UC algorithm is a clean assembly of known pieces—PageRank fixed points, Beta-Bernoulli smoothing, softplus utility aggregation—and the contraction/existence result (Thm 4.1) is correct. The perturbation bounds are standard and fine. The cold-start positivity result is trivial but nice to have stated.\n\nWhere it gets soft. The monotonicity theorem (Thm 4.7) has a real gap. The proof asserts that row-normalization preserves monotonicity and that the fixed point is monotone in Q entries as a consequence of operator monotonicity on the cone. Neither implication is justified. Increasing one edge in a row interacts with all the other entries through normalization; one needs a parameter-monotonicity argument, and it is not there. This is fixable, but as written the claim is unsupported.\n\nThe Sybil resistance is also narrower than the abstract implies. Thm 4.17 explicitly assumes the clique's competence is unchanged by collusion. That is exactly the case the simulations test: Sybils honestly report mediocre competence and only pump usage. The paper names success inflation as the most serious threat in Section 2.3, but never models it, and no theorem bounds the distortion of Q under rational lying. The stress-test has this right.\n\nThe empirical section is not reproducible: single runs, no code, no seeds, and the utility weights and routing coefficients are not fully specified. The 'near-oracle' result is also less informative than it looks, since the oracle is just the empirical success rate—the same signal the algorithm is built from.\n\nNone of this is fatal. The paper is clearly written, the architecture is sensible, and the limitations are acknowledged in prose. It is aimed at people designing agent marketplaces or reputation systems, and it is a legitimate contribution to that subfield. I would send it to peer review, but I would expect the referee to require a real monotonicity proof (or a scoped claim), an honest statement of what the Sybil bound covers, and at least one fully specified reproducible experiment. I would cite it for the OAT-Lite/DOVIS proposal, not for the guarantees.","headline":"Useful architecture proposal and a clean contraction result, but the monotonicity proof has a real gap and the Sybil-resistance claim is much narrower than the abstract suggests.","tokens_in":26140,"tokens_out":3276,"would_cite":true,"duration_ms":34079,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"To make an internet of AI agents discoverable, the paper proposes a telemetry protocol (DOVIS) and a ranking algorithm (AgentRank-UC) that fuses usage and competence through two coupled PageRank fixed points, with formal guarantees of uniqu","keywords":["Agentic Web","agent ranking","usage-competence fusion","PageRank fixed point","telemetry protocol","Sybil resistance","cold-start fairness","competence-aware discovery"],"falsifier":"Run a live or simulated marketplace with DOVIS telemetry for dozens of epochs, and independently measure each agent's true competence with held-out test calls; if a colluding clique that passes the audit checks still captures top ranks through inflated n_success or suppressed n_failure, the competence kernel is corrupted and the monotonicity and Sybil guarantees fail in practice.","tokens_in":25175,"feed_emoji":"🤖","tokens_out":3272,"duration_ms":35898,"temperature":0.7,"pith_summary":"The paper argues that the emerging Web of Agents—where autonomous AI agents, not humans, are the primary users of the internet—will be unusable unless there is a principled way to choose which agent to call for a task. It claims that such a ranking can be built from minimal, privacy-preserving telemetry: callers publish only aggregate counts of calls, successes, quality, latency, cost, and risk, and an indexer combines these into two page-rank-like scores, one for usage and one for competence. The central result is that this fused ranking, AgentRank-UC, is unique, converges quickly, never punishes improved performance, gives newcomers a nonzero baseline visibility, and cannot be overwhelmed by Sybil usage-pumping. If correct, the paper supplies the missing discovery substrate for an agent economy: a way to rank agents by proven recent behavior rather than declared capability or raw popularity. The authors support the claim with formal theorems and simulations using archetypal agents, shocks, and colluding cliques.","feed_headline":"Two fixed-point equations rank AI agents by usage and competence","feed_subtitle":"A telemetry protocol plus a usage-competence score promise discovery by proven performance, not just popularity.","key_machinery":"Two coupled PageRank-style operators over the agent graph: x = αP^T x + (1−α)v (usage rank) and y = βQ^T y + (1−β)w (competence rank), with teleportation priors v and w, fused by the geometric mean r = normalize(x^p ⊙ y^{1−p}). The usage and competence kernels P and Q are built from exponentially decayed aggregates (calls, successes, quality, latency, cost, risk), and the teleportation terms are what make the ranking well-posed: they are contraction mappings with unique fixed points, and they enforce a strict positivity floor for every agent, which is the mechanism behind both cold-start fairness and the Sybil non-amplification bound.","core_discovery":"The paper proposes that the Agentic Web—a network where AI agents delegate tasks to one another—needs a global, competence-aware ranking, and claims that this ranking is achievable with a five-layer protocol (DOVIS: Discovery, Orchestration, Verification, Incentives, Semantics) plus an algorithm called AgentRank-UC. AgentRank-UC builds two row-stochastic kernels from decayed telemetry: a usage kernel P capturing who calls whom, and a competence kernel Q capturing how well those calls turn out. The usage rank x and competence rank y are the fixed points of x = αP^T x + (1−α)v and y = βQ^T y + (1−β)w, combined into r = normalize(x^p ⊙ y^{1−p}). The paper proves that these fixed points exist, a","pith_inferences":["The paper's formal guarantees treat the kernels P and Q as fixed inputs, but in deployment the ranking feeds back into caller selection, so the usage kernel co-evolves with the rank itself; a natural extension is to analyze the coupled dynamical system where selection probabilities depend on r.","The Sybil bound leans on competence share y_S remaining below 1; if colluding agents can also inflate competence on easy self-tests, the resistance weakens, suggesting a testable boundary where competence manipulation, not usage pumping, becomes the limiting attack.","The balance parameter p could be made task- or sparsity-dependent rather than a global constant; the paper lists this as future work, and it is a concrete way to let discovery favor competence in sparse niches and usage in mature ones.","The DOVIS incentive layer is described qualitatively; a quantitative mechanism-design question—what reporting cost, audit probability, and penalty make honest reporting a Nash equilibrium—is left open and is directly testable from the paper's parameters."],"forward_implications":["If AgentRank-UC is correct, an open ecosystem of agents can be ranked for discovery using only minimal aggregate telemetry, without exposing raw prompts, responses, or user data.","The fixed-point formulation gives a computationally concrete path: linear-time construction of sufficient statistics and O(log 1/ε) power iterations per ranking, each dominated by a sparse matrix-vector product.","Monotonicity ensures that improving success rate, quality, or latency never lowers an agent's rank, removing the perverse incentive to hide improvements.","Cold-start fairness follows from strictly positive priors: newcomers always retain a minimum visibility share, with a tunable floor via the teleportation parameters.","The Sybil non-amplification bound shows that a clique of colluding agents can inflate usage only up to a hard ceiling, and cannot capture the fused rank unless it also improves genuine competence.","Simulations indicate the ranking can be tuned by the balance parameter p and the half-life H, giving operators an interpretable trade-off between popularity, competence, responsiveness, and stability."],"supporting_citations":[{"why":"Provides the PageRank link-analysis foundation that AgentRank-UC extends to usage and competence graphs.","marker":"[10]"},{"why":"Supplies the affine-contraction and Neumann-series background used in the existence, uniqueness, and convergence proof.","marker":"[14]"},{"why":"Provides the broader PageRank survey and graph-ranking baselines used in the simulation evaluation.","marker":"[15]"},{"why":"Underpins the Markov-chain perturbation and mixing-time arguments in the stability theorem.","marker":"[30]"},{"why":"Motivates the Sybil non-amplification perspective and graph-based defenses the paper extends to agent rankings.","marker":"[33]"},{"why":"Provides the near-optimal Sybil-defense background against which the paper contrasts its usage-pumping bound.","marker":"[34]"},{"why":"Defines the agent interoperability protocols (MCP, ACP, A2A, ANP) that OAT-Lite aims to be compatible with.","marker":"[1]"},{"why":"Supplies the secure-aggregation technique referenced as a privacy-enhancing extension for DOVIS telemetry.","marker":"[22]"}],"fun_headline_variants":["AgentRank-UC ranks AI agents by usage and competence","DOVIS protocol enables privacy-preserving agent ranking","Two fixed-point equations rank AI agents by performance","Usage and competence scores rank agents in the Agentic Web","New algorithm ranks AI agents by proven performance"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The ranking's guarantees rest on callers reporting honest aggregates: if callers can inflate successes or suppress failures without being caught by signatures, callee acknowledgments, and the 1–5% audit rate, the competence kernel Q is corrupted and the theorems no longer describe real agent behavior.","fun_headline_variants_meta":{"raw":{"variants":["AgentRank-UC ranks AI agents by usage and competence","DOVIS protocol enables privacy-preserving agent ranking","Two fixed-point equations rank AI agents by performance","Usage and competence scores rank agents in the Agentic Web","New algorithm ranks AI agents by proven performance"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000162,"raw_usage":{"total_tokens":1107,"prompt_tokens":805,"completion_tokens":302,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":549,"completion_tokens_details":{"reasoning_tokens":237}},"tokens_in":549,"tokens_out":302,"duration_ms":3207,"temperature":1.0,"reasoning_tokens":237,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T05:44:06.895416+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a live or simulated marketplace with DOVIS telemetry for dozens of epochs, and independently measure each agent's true competence with held-out test calls; if a colluding clique that passes the audit checks still captures top ranks through inflated n_success or suppressed n_failure, the competence kernel is corrupted and the monotonicity and Sybil guarantees fail in practice.","supporting_citations":[{"cited_title":"The anatomy of a large-scale hypertextual web search engine","cited_arxiv_id":null,"evidence_quote":"Provides the PageRank link-analysis foundation that AgentRank-UC extends to usage and competence graphs."},{"cited_title":"Langville and Carl D","cited_arxiv_id":null,"evidence_quote":"Supplies the affine-contraction and Neumann-series background used in the existence, uniqueness, and convergence proof."},{"cited_title":"Gibbons, Michael Kaminsky, and Abraham D","cited_arxiv_id":null,"evidence_quote":"Motivates the Sybil non-amplification perspective and graph-based defenses the paper extends to agent rankings."},{"cited_title":"Gibbons, Michael Kaminsky, and Feng Xiao","cited_arxiv_id":null,"evidence_quote":"Provides the near-optimal Sybil-defense background against which the paper contrasts its usage-pumping bound."},{"cited_title":"Practical secure aggregation for privacy-preserving machine learning","cited_arxiv_id":null,"evidence_quote":"Supplies the secure-aggregation technique referenced as a privacy-enhancing extension for DOVIS telemetry."}],"review_version":1}