{"id":"972eef76-1ce5-4a8e-aca4-40c6ea7655ed","arxiv_id":"2608.07949","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"Guixu presents a three-phase valuation pipeline and blockchain-backed payment system for task-aware, budget-constrained data discovery by autonomous AI agents.","lead":"Guixu is a system that helps AI agents find, price, and buy datasets based on the task at hand, not just keywords, with payments and trust signals recorded on a blockchain. It combines task-aware data scoring, budget-optimized selection, and on-chain attestation into one workflow.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The zero-budget accuracy gain is asserted in Section 3 but never measured; without a defined baseline and repeated trials, the central claim is unverifiable.","rationale":"The reader's CONDITIONAL verdict is appropriate: the paper gives a clear system description but no evaluation. My stress-test identifies the same overall weakness but locates the most load-bearing gap slightly differently. The reader focuses on proxy-label propagation fidelity as the key assumption; that is indeed one important unverified component, because the cost-saving claim in Section 2.2 depends on propagated scores matching oracle scores. However, the strongest stated claim is the end-to-end 'measurable accuracy gain even at zero budget.' Even if proxy-label propagation were perfect, the paper still provides no evidence that the valuation pipeline as a whole selects datasets that improve downstream accuracy relative to keyword retrieval. Thus the load-bearing concern is broader: no baseline, no accuracy measurements, and no repeat protocol are supplied for the headline empirical claim. The proposed check directly tests that claim. Because the reader's verdict already conditions acceptance on supplying evaluation results, my read does not change the verdict: it remains CONDITIONAL. I mark agreement as 'partial' because the reader's weakest assumption is a legitimate sub-concern but not the single most load-bearing one; the missing end-to-end accuracy evaluation is more fundamental to the paper's central claim.","tokens_in":5943,"tokens_out":3869,"duration_ms":41460,"concrete_test":"Run Scenario 1 (zero-budget cat-monitor task) with a fixed seed: enumerate all free datasets from Kaggle/Hugging Face, have Guixu select a portfolio, and train the same classifier on (a) the Guixu-selected portfolio, (b) the top-k datasets returned by keyword search (e.g., 'cat' + 'monitor'), and (c) a random portfolio of the same total size and cost. Report test accuracy over at least 5 repeated trials with standard deviations. If (a) does not exceed (b) and (c) by a margin larger than the trial noise, the 'measurable accuracy gain' claim fails. As a secondary check, compute agreement between Phase 2 propagated scores and LLM oracle scores on a held-out sample to confirm propagation fidelity.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central empirical assertion is that 'valuation-driven selection yields a measurable accuracy gain even at zero budget' (Section 3, Scenario 1). Nothing in the manuscript reports an accuracy number, a baseline, a test set, or a repeat protocol. The claim is presented as something attendees can 'observe' in the live demo, but a demonstration is not a measurement: any perceived gain could reflect dataset ordering, the coarse-ranking weights (derived via Shapley from 'historical dataset performance' that is never described), or the LLM oracle's seed scoring rather than the proxy-label propagation and knapsack layers that are the paper's contribution. The proxy-label propagation in Section 2.2 Phase 2 is a second, narrower unverified assumption: it asserts 'substantial cost savings without sacrificing scoring fidelity,' but the only evidence would be agreement between propagated scores and oracle scores, which is not reported. Both gaps are load-bearing because the value proposition is task-aware selection that outperforms retrieval-first baselines; if the end-to-end accuracy gain cannot be reproduced against a baseline, the system is a plausible architecture but not a demonstrated data discovery advance. The strongest claim therefore rests on a missing evaluation, not on an internal inconsistency.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper presents Guixu, a demonstration system for valuation-driven dataset discovery for autonomous AI agents. The system combines a model-context-protocol interface, a three-phase valuation pipeline (multi-signal coarse ranking, proxy-label propagation, and multi-round knapsack optimization), and an on-chain escrow/attestation layer. The authors argue that this architecture moves beyond keyword-based dataset retrieval toward task-aware, budget-constrained, and trust-aware data procurement. The paper describes the architecture, the valuation formulas, the knapsack optimization, and two interactive demonstration scenarios, and it claims that valuation-driven selection yields a measurable accuracy gain even at zero budget.","tokens_in":6148,"tokens_out":2118,"duration_ms":24172,"significance":"If the claims are substantiated, Guixu addresses a genuine gap: existing data discovery platforms are retrieval-oriented and do not support task-aware utility estimation, budget-constrained selection, or trust-aware procurement. The proposed pipeline is a plausible integration of known techniques (embedding similarity, proxy labels, DP-knapsack, Shapley valuation, and blockchain escrow) into an agent-usable workflow. The paper also ships an artifact link and describes concrete system components (MCP server, x402/MPP payment routing, Shamir-secret-sharing key distribution). However, the core novel claims—that proxy-label propagation preserves scoring fidelity and that the overall selection yields measurable accuracy gains—are not backed by experimental evidence in the manuscript. As a demonstration paper, the system architecture is interesting, but the quantitative value proposition remains unverified.","major_comments":[{"comment":"The central claim that \"the valuation-driven selection yields a measurable accuracy gain even at zero budget\" is asserted but never measured. No baseline, test dataset, accuracy metric, or repeated trials are reported anywhere in the paper. Since this claim is the main motivation for the entire valuation pipeline, the demonstration section must either include a concrete experimental protocol with results (e.g., comparing against a keyword-retrieval baseline on a standard classification task) or explicitly relabel this as an anecdotal observation rather than a measurable gain.","section":"Section 3, Scenario 1"},{"comment":"The proxy-label propagation stage claims \"substantial cost savings without sacrificing scoring fidelity,\" but no evidence is given that propagated scores agree with oracle scores. There is no measurement of propagation accuracy, no calibration of the threshold θ, and no validation that the textual-similarity gap Δ correlates with task utility. Because propagated scores directly determine the per-dataset quality value q̂_i that feeds the knapsack optimization, this omission is load-bearing. The paper should report agreement statistics (e.g., correlation or error between propagated and oracle scores) and describe how θ is chosen.","section":"Section 2.2, Phase 2"},{"comment":"The coarse-ranking weights are computed using the Shapley value algorithm \"based on analysis of historical dataset performance,\" but the historical data, the exact Shapley procedure, and the resulting weights are not described. This leaves a key free parameter that directly affects the ranking and all downstream selections. At minimum, the paper should disclose how the historical performance is defined, what features are used in the Shapley computation, and what the resulting weights are.","section":"Section 2.2, Phase 1"}],"minor_comments":[{"comment":"Several mathematical symbols and letters render as invalid characters (e.g., \"푖\", \"푆푖\", \"�\") in the figure, making the formal definitions of scores and constraints unreadable. The figure should be regenerated with a proper math font or the notation should be defined in the caption or surrounding text.","section":"Figure 2"},{"comment":"The phrase \"such as OpenClaw and OpenAI Codex\" includes a name that may be misspelled or refer to a nonstandard tool; please verify the intended agent names and cite the corresponding systems.","section":"Section 1, Introduction"},{"comment":"The knapsack formulation is described as a \"two-dimensional knapsack problem\" with budget and size constraints. It would be helpful to explicitly state that the dynamic programming runs over discretized price units and to give the time complexity, as this is a core algorithmic component.","section":"Section 2.2, Phase 3"},{"comment":"Some references lack access dates for web sources, and the artifact URL should be verified to be publicly accessible before the camera-ready deadline.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"This is a demonstration paper, so the evaluation bar is typically lower than for a research paper. However, the stated claim of a \"measurable accuracy gain\" is a quantitative assertion that should be supported by at least a small experiment or a reproducible reproduction protocol. The current text risks overstating the system's validation. The authors should either add a brief evaluation or soften the claim to match what the demo can actually show."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is a system demonstration, and that is the right lens to read it through. The genuinely new thing is the integration: a three-phase valuation pipeline (coarse ranking with Shapley-weighted signals, proxy-label propagation for cheap scoring, multi-round knapsack for budget-constrained selection) wrapped in an MCP server with agentic payments and on-chain escrow, all feeding trust signals back into the valuation. Each component is known, but nobody has put them together in one agent-facing data discovery workflow. The writing is clear, the architecture is concrete, and the authors are honest about the prior work they build on. The GitHub artifact link is a plus, though the paper does not tell us what is actually in it. The soft spots are real but proportionate to a demo paper. The big one is Section 3, where the zero-budget scenario claims attendees can observe that valuation-driven selection yields a measurable accuracy gain. That is an empirical claim with no number, no baseline, no test set, and no repeat protocol. A live demo is not a measurement; any perceived gain could come from dataset ordering, the Shapley weights, or the LLM oracle's seed scoring rather than the propagation and knapsack layers that are the contribution. The same goes for Phase 2: the paper asserts that proxy-label propagation yields substantial cost savings without sacrificing scoring fidelity, but never measures agreement between propagated and oracle scores, and never describes how the threshold theta is calibrated. The Shapley weights are derived from 'historical dataset performance,' which is also left unspecified. These are load-bearing for the paper's value proposition, so they matter, but for a demo track they are fixable: either report a small-scale evaluation with baselines, or tone down the claims to what the demo actually shows. The paper is not internally inconsistent, and the math that is described (two-dimensional knapsack via DP) is standard and correct. My main disagreement with a strict reading would be to note that in the demo-paper genre, the absence of experiments is not a fatal flaw unless the authors pretend otherwise. Here they do pretend, in one sentence, so the fix is easy. Who is this for? Readers working on agentic data management, data valuation, or blockchain-based data markets. It is a plausible reference point for future systems, and it deserves a serious referee. My recommendation: send it to peer review, and ask the authors to either supply a minimal empirical check (even a handful of datasets with baseline comparisons) or rewrite the zero-budget claim as a demonstration of workflow rather than a measured accuracy gain.","headline":"Cleanly written demo integration of known valuation and procurement pieces; the accuracy claims are asserted, not measured, so the paper needs either data or softer claims before it can be taken at face value.","tokens_in":719,"tokens_out":1609,"would_cite":false,"duration_ms":29460,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Guixu claims that data discovery for autonomous agents should be a valuation problem: score each dataset for task utility, choose a portfolio under budget, and verify the purchase on-chain.","keywords":["data valuation","autonomous agents","dataset discovery","budget-constrained optimization","knapsack","proxy-label propagation","on-chain attestation","agentic payment"],"falsifier":"Run the LLM oracle on every sample of a candidate dataset and compare those scores with Guixu's propagated scores: if mean absolute error or ranking disagreement is large, the proxy-label phase is not preserving fidelity and the measured accuracy gain cannot be attributed to valuation. A second check: repeat the zero-budget scenario with random dataset selection and with keyword search; if accuracy is equal, the valuation pipeline itself adds nothing beyond retrieval.","tokens_in":5747,"feed_emoji":"🤖","tokens_out":3601,"duration_ms":36584,"temperature":0.7,"pith_summary":"Guixu argues that dataset discovery for autonomous agents should be reframed from a retrieval problem into a task-aware valuation and procurement problem. The paper presents a working system that takes a natural-language task, searches heterogeneous data sources, scores each candidate dataset for task-specific utility, and selects a budget-constrained portfolio via knapsack optimization. It claims this valuation-driven selection yields measurable accuracy gains over keyword-based retrieval even at zero budget, and that the full purchase loop can run autonomously with on-chain attestation for trust. If the system works as described, agents could independently acquire verified training data for niche tasks end to end.","feed_headline":"AI agents pick datasets by value, not by keyword","feed_subtitle":"Guixu scores data for the task, spends a budget optimally, and records every purchase on-chain.","key_machinery":"The three-phase valuation pipeline is the load-bearing mechanism. Phase 1 scores candidates by weighted signals (relevance, schema fit, data scale, label quality, metadata completeness, on-chain reputation) with Shapley-valued weights. Phase 2 propagates LLM-oracle scores from high-score and low-score anchors to similar unscored samples when the similarity gap exceeds a threshold, reserving oracle calls for ambiguous samples. Phase 3 solves a two-dimensional knapsack over price and size via dynamic programming, advancing to the next coarse-ranked batch if no feasible portfolio exists. On-chain attestation supplies the reputation signal that feeds back into Phase 1.","core_discovery":"The central claim is that valuation-driven data discovery outperforms keyword-based retrieval because it optimizes what actually matters for the downstream task: per-sample utility, budget, size constraints, and trust. The paper's demonstrator takes a natural-language task such as 'train a safety-helmet classifier with a $2.00 budget,' parses it into task and data schemas, searches sources like Kaggle and Hugging Face, scores each candidate through three phases (coarse ranking on six signals, proxy-label propagation with an LLM oracle for ambiguous samples, and multi-round two-dimensional knapsack optimization), and then executes the purchase through escrow smart contracts, Shamir key sharing, and on-chain attestation that feeds back into future valuations. It claims this yields a measurable accuracy gain even at zero budget and enables agents to move beyond keyword-based retrieval toward task- and budget-aware, trustworthy procurement.","pith_inferences":["The valuation pipeline is not tied to image classification; the same schema-based scoring and knapsack selection could be applied to tabular or text datasets, though the paper only demonstrates vision tasks.","If on-chain reputation feeds valuation, sellers with good attestations may command price premiums, a market dynamic the paper does not explore.","A direct measurement of propagated versus oracle scores would let operators tune the threshold as an explicit cost-accuracy knob, a calibration study the paper leaves implicit.","The zero-budget accuracy claim would be stronger if compared against a human-curated baseline or a simple budget-agnostic ranker; the paper reports the result without that ablation."],"forward_implications":["An agent can complete a niche task end to end, from a natural-language request to a trained model, without human selection of data.","Buyers can make purchase decisions that jointly optimize task utility, price, and trust, instead of picking datasets by keyword hits.","Proxy-label propagation makes per-sample valuation affordable enough for realistic datasets, because only ambiguous samples trigger expensive oracle calls.","The multi-round knapsack handles infeasible budgets by expanding the candidate pool until a feasible portfolio is found.","On-chain attestation creates a feedback loop where past purchases improve future valuation decisions."],"supporting_citations":[{"why":"Supplies the keyword-based dataset search baseline that Guixu contrasts with valuation-driven discovery.","marker":"[1]"},{"why":"Provides the x402 agentic payment protocol that Guixu's payment router uses for micropayments.","marker":"[2]"},{"why":"Defines data discovery and surveys the field, framing the problem that Guixu extends beyond retrieval.","marker":"[3]"},{"why":"Underlies the decentralized key management network used for Shamir secret sharing of dataset keys.","marker":"[4]"},{"why":"Provides the Machine Payment Protocol as an alternative settlement mechanism in Guixu's multi-protocol payment framework.","marker":"[5]"}],"fun_headline_variants":["Agents value data per task, not just find it","Task-aware data buying with on-chain receipts","Value-driven data discovery: budget and trust built in","KG search? Guixu prices data for the job, then attests"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The cost-saving argument rests on propagated proxy scores being close to what the LLM oracle would assign, yet the paper provides no measurement of propagation accuracy or threshold calibration.","fun_headline_variants_meta":{"raw":{"variants":["Agents value data per task, not just find it","Task-aware data buying with on-chain receipts","Value-driven data discovery: budget and trust built in","KG search? Guixu prices data for the job, then attests"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000217,"raw_usage":{"total_tokens":1410,"prompt_tokens":895,"completion_tokens":515,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":511,"completion_tokens_details":{"reasoning_tokens":448}},"tokens_in":511,"tokens_out":515,"duration_ms":6453,"temperature":1.0,"reasoning_tokens":448,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T00:37:00.150298+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the LLM oracle on every sample of a candidate dataset and compare those scores with Guixu's propagated scores: if mean absolute error or ranking disagreement is large, the proxy-label phase is not preserving fidelity and the measured accuracy gain cannot be attributed to valuation. A second check: repeat the zero-budget scenario with random dataset selection and with keyword search; if accuracy is equal, the valuation pipeline itself adds nothing beyond retrieval.","supporting_citations":[{"cited_title":"Companion of the 2024 International Conference on Management of Data , pages =","cited_arxiv_id":null,"evidence_quote":"Supplies the keyword-based dataset search baseline that Guixu contrasts with valuation-driven discovery."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the x402 agentic payment protocol that Guixu's payment router uses for micropayments."},{"cited_title":"SIGMOD Rec","cited_arxiv_id":null,"evidence_quote":"Defines data discovery and surveys the field, framing the problem that Guixu extends beyond retrieval."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Underlies the decentralized key management network used for Shamir secret sharing of dataset keys."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the Machine Payment Protocol as an alternative settlement mechanism in Guixu's multi-protocol payment framework."}],"review_version":1}