SARC-DQ: Runtime Data-Quality Gating for Agentic AI: Silent Evidence Defects, the Incompetence Shield, and Downstream-Only Remediation

Gaston Besanson

REVIEW 4 major objections 4 minor 24 references

SARC-DQ: Runtime Data-Quality Gating for Agentic AI: Silent Evidence Defects, the Incompetence Shield, and Downstream-Only Remediation

T0 review · 4 major / 4 minor · reviewed 2026-08-01 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Metadata-borne data defects are structurally invisible to payload-only agents, and model scale does not buy skepticism; a metadata-aware pre-action gate, not bigger models, is the remedy.

desk verdict Honest, narrow, and worth a serious referee: the flat ladder proves only that agents can't doubt what they never see, while the benchmark, the gate, and the reporting standards are the real contributions. read the letter →

arxiv 2607.26313 v1 pith:Q4FICZFB submitted 2026-07-28 cs.SE cs.AIcs.CY

Gaston Besanson This is my paper

classification cs.SEcs.AIcs.CY

keywords metadata-bornedefectsagenticAIdata-qualitygatingpre-actiongatesilentevidencenewsvendorreplenishmentdownstream-onlyremediationmodelcapabilityscaling

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper's central claim is that a class of enterprise data defects — metadata-borne defects like stale prices or superseded records, which look well-formed in the payload and are betrayed only by freshness, lineage, or provenance — are structurally invisible to an agent that reads only the payload, so no amount of model capability can make an agent doubt them. On a priced replenishment benchmark, a competent agent silently converts an injected metadata-borne defect into a costly action about 60% of the time, with chance-level behavioral doubt markers and zero explicit data-quality flags, and the rate stays flat (60–62%) across four model tiers spanning roughly 15× in inference price. The paper argues the fix is architectural: a cheap metadata-aware gate placed at the point of action detects the defect where the signal lives and substitutes a governed value downstream-only, never writing to source. The gate fully recovers the loss on the channels its predicates cover (e.g., freshness detection goes from 0% for a payload-only critic to 100% for the gate), while portfolio recovery fails because one uncovered defect class (silent unit change) dominates the recoverable pool. A model-free oracle derived from the task's decision geometry predicts the measured conversion with mean absolute error 0.015, giving the flat ladder an analytical form.

What carries the argument

The central object is the payload/metadata channel split plus a Pre-Action Gate with downstream-only remediation. An evidence record has two channels: the payload (fields a naive consumer sees) and metadata (freshness, lineage, provenance, version); a corruption class is tagged as payload-visible or metadata-borne, and this tag determines whether a payload-only reader can ever see the defect. The gate is a small set of deterministic predicates (freshness, lineage present, golden-record uniqueness, cross-source consistency, schema conformance, completeness) evaluated at the point of action, with a governed buffer for quarantine-and-substitute that never writes to source stores (Proposition 1

What would settle it

Run the same priced-replenishment task with a decider that sees freshness/lineage fields appended to the payload: if the metadata-borne ADR drops materially below ~60% and behavioral doubt markers rise above chance (AUC > 0.50), the structural-invisibility claim is falsified. Alternatively, re-run the four-tier ladder on a different model family under identical conditions; a non-flat ladder would show the flatness is family-specific rather than a general property of payload-only deciders.

Watch

Extended reading notes

Core claim

Metadata-borne defects are structurally invisible to a payload-only decider: the payload looks valid and the tell lives in unseen metadata. On a priced replenishment task, a competent agent converts injected metadata-borne defects into costly actions ~60% of the time, with doubt markers at chance (AUC ≤ 0.50) and zero flags, flat across four tiers spanning ≈15× inference price. A metadata-aware Pre-Action Gate detects what a payload-only critic cannot (stale master data 0→100%; schema drift 0→100%) and, via quarantine-and-substitute with no source writes, fully recovers the freshness-channel loss (134 → −0.5), though portfolio recovery is −0.04 because one large uncovered class (silent unit

Load-bearing premise

The load-bearing premise is that the experimental construction matches production GIGO: corruption is injected so the discriminating tell lives only in metadata the decider never sees, and the gate's remedy presumes that metadata is present and correct — if real deployments expose freshness or lineage in context, or lack trustworthy metadata, the flat 60% ladder and the gate's recovery are harness artifacts.

Editorial extensions

If this is right

Organizations cannot rely on frontier-model upgrades to catch metadata-borne evidence defects; expected loss is set by corruption rate and decision geometry, so investment should go to enforcement placement, not just capability.
Runtime gating with downstream-only remediation can fully recover losses on the channels it covers, converting silent losses into clean actions without writing to source stores.
The gate's value is coverage-limited: uncovered defect classes (e.g., unit-consistency, plausible outliers) can dominate portfolios, so a gate must be paired with published coverage gaps and periodic predicate extension.
The model-free oracle provides a planning tool: given corruption rate and task cost geometry, one can predict loss-conversion rate and the maximum benefit of a gate before deployment.
The incompetence shield implies a trade-off between task competence and data-quality resistance: a competent agent is more vulnerable unless enforcement is architectural.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The flat-ladder result is shown within one model family; if the mechanism is correct, similar flatness should appear across families and architectures when the tell is excluded from context, and a cross-family re-run would be a cheap falsification.
Presenting freshness/lineage metadata in the payload (e.g., appending price_age_days or superseded_by_id fields) should collapse the silence — ADR should drop and doubt markers should rise if agents use these fields; the paper's own superseded_golden_record reversal suggests exactly this lever.
The gate assumes metadata exists and is correct; a natural extension is to model the gate's own input-quality dependence (the paper's stress test shows detection falls when metadata degrades), so adding a metadata-trustworthiness predicate would close the loop on badly governed sources.
The min(·) framing implies governance obligations: audits could use the versioned evidence-set lineage (Proposition 1) as evidence artifacts for data-quality duties, treating runtime evidence governance as a first-class control rather than a secondary check.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Two things to know before reading SARC-DQ. First, the reporting is unusually clean: every number comes from a committed results file, pre-registrations are frozen, invalid runs are retained and disclosed, and H3/H4 fail as written without being re-fit to pass. That part is exemplary. Second, the headline flatness — 60% ADR across four model tiers with silence signals at chance — is real but narrower than the abstract implies. The experiment hides the defective signal (freshness, lineage) from the agent's context by construction. No model can doubt data it never sees. So the flat ladder supports the claim "if the tell is absent from context, scale doesn't help," which is close to tautological. The paper acknowledges this in its threats, but the abstract's "capability does not buy skepticism" overstates the finding. What the ladder does not show is whether a frontier model would exploit timestamps or version fields when they are present — and production agents often do see those fields. That boundary is acknowledged, not resolved.

What is genuinely new: the priced same-seed ADR metric (currency delta against the same demand draw), the isolated metadata-borne channel, the "incompetence shield" contrast (a competent agent tracks a bad price; an incompetent one is accidentally safe), and the gate's downstream-only remediation with lineage. The gate beats a payload-only critic on the signals it covers — 62% vs 31% detection, zero false blocks — and recovers the freshness loss fully. The leave-one-class-out decomposition showing silent_unit_change drives the portfolio failure is a nice piece of accounting. The model-free oracle has MAE 0.015, but it shares the materiality definition and cost function with the measured ADR, so it functions more as a consistency check than an independent prediction. That is a minor concern.

Soft spots, in proportion. External validity is thin: one decision domain, four tiers of a single vendor family, injected corruption, and metadata assumed present. The paper admits all of this, and the metadata-degradation stress (A.6) is a good start. The prompt pinning after the pilot is disclosed but still a post-hoc choice; the 60% number depends on prompt construction. My main complaint is framing: the key claim should say "in contexts where the tell is absent," not make a general statement about capability.

This paper deserves a serious referee. The claims are falsifiable, narrowly scoped, and backed by strong internal validity — the measured results are consistent with what is reported. I would send it to review with a request to temper the headline and add one experiment that exposes metadata in context. My guess is some of the flatness would break, but the paper would be better for knowing where.

Referee Report

4 major / 4 minor

Summary. The paper investigates a class of enterprise data-quality defects that are invisible in the payload of an evidence record and betray themselves only through metadata (freshness, lineage, provenance). In a priced newsvendor replenishment task, an instructed decider that sees only the payload converts such injected defects into currency losses with no behavioral sign of doubt. Across four model tiers (roughly 15× in inference price), the action-defect rate is reported as flat at about 60%, with silence markers at chance. The paper proposes a metadata-aware Pre-Action Gate that quarantines and substitutes covered defects, and compares it with a realistic payload-only critic. It also derives a model-free analytical oracle for the conversion rate and reports calibration to MAE 0.015. The paper is methodologically self-aware: several pre-registered hypotheses (H2, H3, H4) were not supported as written, and the authors report decomposed failures rather than re-fitting. The manuscript emphasizes reproducibility, with every reported number macro-generated from committed result summaries.

Significance. The problem addressed—silent evidence defects in agentic systems—is timely and practically important. The paper's strengths are its genuine transparency (committed summaries, provenance links, disclosed invalid runs) and its careful honesty in reporting falsified hypotheses and coverage gaps. The falsification analysis in Appendix A.7 and the portability demonstration in Appendix A.8 are valuable. However, the headline claim that capability does not buy skepticism is currently overstated: the experimental construction withholds the discriminating signal from the context, so the flatness is close to tautological. The analytical oracle is also largely a restatement of the instructed policy rather than an independent prediction. If the authors reframe these claims and add a control condition where metadata is visible, the paper would be a solid systems contribution; in its present form the central scientific message needs revision.

major comments (4)

[§7 (H1 ladder)] The flatness of the ADR ladder is a direct consequence of the experimental construction: the decider is prompt-instructed to apply a fixed newsvendor policy to the shown price, and the metadata channel (where the defect signal lives) is never in context. Any model that follows the instruction will produce the same ordering decision and hence the same conversion rate, independent of tier. Thus M2 ('capability does not buy skepticism') is not an empirical finding about capability but a restatement of the information-hiding design. The paper's own §7.1 reframe shows detectability is a function of what is shown; the ladder should be described as a boundary demonstration, not a scaling law. I recommend either adding a condition where the metadata is present in context (e.g., a timestamp) or narrowing the claim in the abstract and conclusion.
[Corollary 1 and Appendix A.1] The analytical oracle is constructed from the same decision rule and loss function used to score the agent: the agent is instructed to order q*(shown price), and the oracle computes the fraction of episodes where the cost difference exceeds τm. The calibration (MAE 0.015) therefore verifies instruction adherence, not an independent model-free law. The 'model independence' is built in by removing the model from the decision rule. This is not an error per se, but the paper's language overstates it. The oracle should be presented as a closed-form expression for the expected conversion rate under the instructed policy, serving as a consistency check, not as a prediction that could fail.
[§7 (H3)] In the H3 comparison, the numerical advantage of the gate D over the payload-only critic C on residual loss is small: 294 [104.1, 498.0] vs. 311 [120.3, 516.3], with heavily overlapping confidence intervals. The text says 'D does dominate C,' but a superiority claim on loss is not supported by these intervals. The detection-rate advantage (62% vs. 31%) is robust and is the more meaningful evidence for the gate's value. Please either soften the loss-based dominance claim or provide a formal test (e.g., bootstrap difference distribution) that justifies it.
[§5 (Phase 0 pilot)] The 'incompetence shield' argument compares a competent agent (0c, elasticity 0.992) with a naive agent (0a, elasticity 0.000) that converts 0% of defects. The naive agent's zero conversion is because it essentially ignores the price and therefore does not perform the task; this is a non-performing agent rather than an incompetent-but-engaged one. The contrast is not a clean test of an 'incompetence shield.' I suggest either removing this framing from the main argument or designing a control where the agent still makes reasonable decisions but is less responsive to the corrupted price.

minor comments (4)

[Abstract and §7] The phrase 'flat-to-rising (60% at haiku, 62% at fable)' appears without the accompanying interval. Appendix A.4 reports a top-minus-bottom endpoint difference of 0.0125 with a 95% Wald interval [-0.0551, 0.0801]; including this interval in the main text would make the flatness claim quantitatively precise.
[Table 3] The column header 'registered channel' combined with the dagger markers may confuse readers because the dagger denotes classes whose empirical detection crosses the assigned label. Consider renaming to 'assigned channel' and adding a note that the daggers indicate empirically observed crossings.
[§3 and §9] The architectural contribution assumes that metadata is present and correct (stated in §9). This assumption is central to the gate's feasibility; it should appear earlier, in the architecture section, so the reader is aware of the boundary condition from the start.
[Title/§3] The acronym SARC-DQ is not defined until §3. Since it appears in the title and abstract, define it at first use to help readers in the SE community.

Circularity Check

1 steps flagged · score 3.0 of 10

Headline flat-ladder measurement is independent, but the analytical oracle that is said to give it 'an analytical form' is circular: it assumes the instructed newsvendor behavior and shares the measured ADR's materiality definition, so its model-tier independence is constructed, not predicted.

self definitional [Section 4, Corollary 1; Appendix A.1 (conversion law calibration)]
"On the newsvendor substrate the conversion factor ρ admits an explicit oracle form: for a corrupted episode the agent that acts on the newsvendor optimum for the shown price ĉ incurs a paired loss ℓ=C(q ⋆(ĉ))−C(q⋆(c)) against the same realised demand, and the episode is material iff ℓ≥τm C(q ⋆(c)). ... the shown price ĉ is fixed by the injected corruption, not by the agent, so model tier cannot enter—metadata availability, not capability, is what is held fixed. Appendix A shows this oracle conversion tracks the measured agent ADR to a mean absolute error of0.015 ..."

The oracle's conversion indicator uses the same materiality threshold τm and cost function C as the measured ADR, and it assumes the agent takes the newsvendor-optimal action for the shown price — exactly the policy_instructed behavior the experiment implements (Phase 0 elasticity 0.992). The measured ADR is the empirical fraction of episodes satisfying the same materiality condition under that instructed policy. Therefore the 'model-free' flatness is not derived from the measurements: model tier is absent from the formula by construction, so the oracle would predict a flat ladder even if larger models doubted the payload. The MAE 0.015 is a consistency check that the decider followed its instructions, not an independent analytical explanation of M2.

full rationale

The core empirical claims (M1/M2/M3) are pre-registered measurements on a frozen harness, with honest reporting of unsupported predictions (H2/H3/H4) and explicit external-validity limits, so most of the paper is not circular. The four-tier flatness itself is a measurement with a CI and OLS trend, not a fitted prediction. No load-bearing self-citation was found: the SARC prior work supplies terminology/enforcement-site placement, but the gate-vs-critic and ladder results are self-contained against the benchmark. The one substantive circularity is the 'analytical oracle' (Corollary 1/Appendix A). It is presented as a model-free derivation that gives the flat ladder 'an analytical form,' but it assumes the instructed newsvendor optimum and shares the measured ADR's materiality definition, so the absence of a model-tier term is built in rather than predicted. That is a partial (supporting-claim) circularity, not a collapse of the headline measurement; hence score 3.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

No new physical or theoretical entities are postulated. The system artifacts (SARC-DQ predicate family, governed buffer, GIGO-Bench) are shipped code, not postulated entities; the 'incompetence shield' is a named empirical pattern describing measured elasticity differences, not a new mechanism. The central claim rests on design constants (τm, injection rates, flagged injector defaults), one decision-geometry assumption about agent behavior, and the domain assumption that deployed agents truly lack metadata context.

free parameters (3)

materiality threshold τm = 0.005 (registered, not fitted)
Defines when a paired loss counts as a material action defect; both the measured ADR and the analytical oracle use τm·C(q*(c)). A.5's sweep moves mean oracle ADR from 0.6225 to 0.3325, so the headline 60% level depends on this hand-set constant, though the flatness result is less sensitive.
injection sweep rates = {2, 5, 10, 20}% per class
Per-class corruption rates in GIGO-Bench; the paper justifies them against published prevalence bands (20% as the low end of real-world corruption), but they are author-selected design constants that shape pooled ADR and the portfolio recovery (-0.04) via per-class weights.
injector defaults for silent_unit_change and plausible_outlier = flagged defaults (values not given in the text)
The paper states no public base rate exists for these classes; their injector parameters are 'declared, flagged defaults' with Tier-2 calibration deferred. These are free choices, and silent_unit_change drives 83% of the recoverable portfolio denominator, so it dominates the H4 outcome.

assumptions (5)

domain assumption Assumption 1: each evidence record has a content-addressed identifier eid(r) = H(payload, metadata) for collision-resistant H; remediation never mutates source records.
Supports Proposition 1's lineage-preservation guarantee (§4). It is a design stipulation of the architecture, not an externally validated property of deployed systems.
domain assumption Independence: defect presence is independent of the demand realization; non-interference: detection/recovery act only on detected defects.
Stated assumptions behind Proposition 2's bound ADR ≤ ερ(1−ds) (§4); the paper notes ρ is an upper bound, not a constant, so the bound is qualitative.
domain assumption Oracle decision rule: the agent acts on the newsvendor optimum for the shown price q*(ĉ).
Corollary 1's conversion law assumes this behavior; it is proxied empirically by elasticity 0.992 for one competent agent (0c) and unmeasured for the other ladder tiers, so the strong oracle-matching result partly inherits this assumption.
domain assumption Inference price is a usable proxy for model capability tier.
The ladder orders models by 15x price spread; the paper itself disclaims that 'inference-price tier is not itself a scientific measure of capability' (§9), so the flatness result is scoped to a price ladder within one vendor family.
domain assumption Deployed metadata is present and correct in the gate's environment.
The gate's detection is only as good as the metadata it reads (§9, A.6); under full metadata degradation, pooled detection falls from 0.8333 to 0.6667. The remedy therefore assumes the very asset that may be missing in badly governed sources.

how reviews work

0 comments

Cite this review

Pith. "Pith review of SARC-DQ: Runtime Data-Quality Gating for Agentic AI: Silent Evidence Defects, the Incompetence Shield, and Downstream-Only Remediation." pith.science (2026). https://pith.science/paper/Q4FICZFB

@misc{pith2026260726313,
  author       = {Pith},
  title        = {Pith review of: SARC-DQ: Runtime Data-Quality Gating for Agentic AI: Silent Evidence Defects, the Incompetence Shield, and Downstream-Only Remediation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Q4FICZFB}},
  note         = {Machine review of arXiv:2607.26313}
}

read the original abstract

Agentic systems act, so a defect in the evidence they retrieve becomes a wrong action with a currency cost. The most dangerous enterprise defects are metadata-borne: a stale price or a superseded record, perfectly well-formed in the payload and betrayed only by freshness, lineage, or provenance. Such a defect never enters the agent's context, and an agent cannot doubt data it cannot see. On a priced replenishment benchmark, a competent agent silently converts an injected metadata-borne defect into a costly action about 60% of the time, with zero data-quality flags and behavioral doubt markers at chance (AUC <= 0.50). Across four model tiers spanning roughly 15x in inference price, the rate stays flat: capability does not buy skepticism. A metadata-aware pre-action gate with downstream-only remediation recovers the loss fully on the signals its predicates cover and not at all on those they miss. A model-free oracle derived from the task's decision geometry tracks the measured rates with MAE 0.015 (Pearson r = 0.876, interval coverage 15/16 cells), giving the flat ladder an analytical form. Evidence integrity is a systems axis distinct from model capability; mitigation depends on enforcement placement and predicate coverage. Code, frozen results, and a deterministic analysis pipeline: https://github.com/besanson/dqSarc

Figures

Figures reproduced from arXiv: 2607.26313 by the authors.

**Figure 1.** The silent conversion of a metadata-borne defect (top): the payload is valid, so a payloadonly agent acts confidently and converts the defect into a currency loss. A metadata-aware PreAction Gate (bottom) intercepts the same record where the signal actually lives—the metadata— and substitutes a governed value, yielding a corrected action. Model scale moves along the top row; enforcement placement adds the bottom o… view at source ↗

**Figure 2.** Predicted oracle conversion vs. measured agent ADR. Proximity to the diagonal is the [PITH_FULL_IMAGE:figures/full_fig_p014_2.png] view at source ↗

**Figure 3.** Weighted denominator (recoverable $) by class; blue = gate-covered, red = uncovered [PITH_FULL_IMAGE:figures/full_fig_p015_3.png] view at source ↗

Figures from the paper (4 more)

**Figure 4.** Figure 4: Top: portfolio recovery with each class removed (dashed = full portfolio). Bottom: oracle [PITH_FULL_IMAGE:figures/full_fig_p016_4.png]

**Figure 5.** Figure 5: Gate detection on covered classes under metadata degradation. Green survives; red [PITH_FULL_IMAGE:figures/full_fig_p017_5.png]

**Figure 6.** Figure 6: Recovered positive-loss fraction at equal intervention budget. The metadata gate (blue) [PITH_FULL_IMAGE:figures/full_fig_p017_6.png]

**Figure 7.** Figure 7: Second domain (B2C promotion eligibility): per-class ungated ADR (red) and gate re [PITH_FULL_IMAGE:figures/full_fig_p018_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

24 extracted references · 6 linked inside Pith

[1]

arXiv:2602.11348 (ICML 2026)

AgentNoiseBench. arXiv:2602.11348 (ICML 2026)

arXiv 2026
[2]

W. Wang, J. Shi, Z. Ling, Y.-K. Chan, C. Wang, C. Lee, Y. Yuan, J. Huang, W. Jiao, and M. R. Lyu. Learning to ask: when LLM agents meet unclear instruction (introduces NoisyToolBench). arXiv:2409.00557

arXiv
[3]

Ruan et al

Y. Ruan et al. Identifying the risks of LM agents with an LM-emulated sandbox (ToolEmu). ICLR2024. arXiv:2309.15817

arXiv
[4]

J. Sun, S. Y. Min, Y. Chang, and Y. Bisk. Tools fail: detecting silent errors in faulty tools. EMNLP2024. arXiv:2406.19228

arXiv
[5]

Yao et al.τ-bench: a benchmark for tool-agent-user interaction

S. Yao et al.τ-bench: a benchmark for tool-agent-user interaction. arXiv:2406.12045

arXiv
[6]

H. Wang, C. M. Poskitt, and J. Sun. AgentSpec: customizable runtime enforcement for safe and reliable LLM agents.ICSE2026. arXiv:2503.18666

arXiv
[7]

C. L. Wang et al. MI9 — Agent Intelligence Protocol: runtime governance for agentic AI systems. arXiv:2508.03858

arXiv
[8]

ISO/IEC 5259 (2024 series), Data quality for analytics and ML

2024

Show all 24 references

[9]

See [23] (Raha/Baran)
[10]

See [24] (Magellan/DeepMatcher)
[11]

Besanson

G. Besanson. SARC: a governance-by-architecture framework for agentic AI systems. arXiv:2605.07728

arXiv
[12]

Besanson

G. Besanson. Green SARC: predictive cost and carbon governance for agentic AI systems. arXiv:2606.15954
[13]

Rahm and H

E. Rahm and H. H. Do. Data cleaning: problems and current approaches.IEEE Data Eng. Bull.23(4), 2000

2000
[14]

Kim, B.-J

W. Kim, B.-J. Choi, E.-K. Hong, S.-K. Kim, and D. Lee. A taxonomy of dirty data.Data Mining and Knowledge Discovery7(1), 2003

2003
[15]

R. Y. Wang and D. M. Strong. Beyond accuracy: what data quality means to data consumers. J. Management Information Systems12(4), 1996

1996
[16]

International Organization for Standardization

ISO 8000,Data quality(multipart series; master-data parts). International Organization for Standardization. Part 1: Overview, ISO 8000-1:2022

2022
[17]

Everyone wants to do the model work, not the data work

N. Sambasivan et al. “Everyone wants to do the model work, not the data work”: data cascades in high-stakes AI.CHI2021. doi:10.1145/3411764.3445518
[18]

Nagle, T

T. Nagle, T. C. Redman, and D. Sammon. Only 3% of companies’ data meets basic quality standards.Harvard Business Review, 2017 (Friday Afternoon Measurement). 12

2017
[19]

Global data management benchmark reports, 2013–2017 (perception survey)

Experian. Global data management benchmark reports, 2013–2017 (perception survey)

2013
[20]

X. Li, X. L. Dong, K. Lyons, W. Meng, and D. Srivastava. Truth finding on the deep web: is the problem solved?PVLDB6(2), 2013

2013
[21]

Curino, H

C. Curino, H. J. Moon, A. Deutsch, and C. Zaniolo. Automating the database schema evolution process.VLDB Journal22(1), 2013 (MediaWiki)

2013
[22]

Federal Reserve Bank of St. Louis. ALFRED: ArchivaL Federal Reserve Economic Data (vin- tage economic data).https://alfred.stlouisfed.org
[23]

Mahdavi et al

M. Mahdavi et al. Raha: a configuration-free error detection system.SIGMOD2019
[24]

flat to slightly rising

P. Konda et al. Magellan: toward building entity matching management systems.PVLDB 9(12), 2016. A Analytical companion (deterministic, $0) This appendix strengthens the empirical claims with analytical explanations and robustness analyses that addnonew experiment. Every value ...

2016

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.

[1] [1]

arXiv:2602.11348 (ICML 2026)

AgentNoiseBench. arXiv:2602.11348 (ICML 2026)

arXiv 2026

[2] [2]

W. Wang, J. Shi, Z. Ling, Y.-K. Chan, C. Wang, C. Lee, Y. Yuan, J. Huang, W. Jiao, and M. R. Lyu. Learning to ask: when LLM agents meet unclear instruction (introduces NoisyToolBench). arXiv:2409.00557

arXiv

[3] [3]

Ruan et al

Y. Ruan et al. Identifying the risks of LM agents with an LM-emulated sandbox (ToolEmu). ICLR2024. arXiv:2309.15817

arXiv

[4] [4]

J. Sun, S. Y. Min, Y. Chang, and Y. Bisk. Tools fail: detecting silent errors in faulty tools. EMNLP2024. arXiv:2406.19228

arXiv

[5] [5]

Yao et al.τ-bench: a benchmark for tool-agent-user interaction

S. Yao et al.τ-bench: a benchmark for tool-agent-user interaction. arXiv:2406.12045

arXiv

[6] [6]

H. Wang, C. M. Poskitt, and J. Sun. AgentSpec: customizable runtime enforcement for safe and reliable LLM agents.ICSE2026. arXiv:2503.18666

arXiv

[7] [7]

C. L. Wang et al. MI9 — Agent Intelligence Protocol: runtime governance for agentic AI systems. arXiv:2508.03858

arXiv

[8] [8]

ISO/IEC 5259 (2024 series), Data quality for analytics and ML

2024