Pith.

Artificial Intelligence · arXiv:2609.00434 v1 · submitted 31 Aug 2026 · 30 references

SAGE: State-Grounded, Abstention-Aware Evaluation of Task-Oriented Dialogue Agents

Rayan Khoury, Shih-Yao Lin, Pratyush Mishra

The paper says

“Evaluating task-oriented dialogue agents requires judging not merely whether a reply reads well but whether each turn advances the underlying workflow state correctly–a distinction conventional holistic LLM judges can miss because they evaluate the available context as a single unit and require one or more full-model calls per turn.”

more of the abstract

We propose SAGE (State-Grounded Abstention-Aware Evaluation), which compiles a workflow specification and per-turn state diff into atomic, schema-grounded criteria and routes each through a cascade of symbolic and encoder/NLI verifiers that abstain rather than guess, aggregating criterion verdicts into a turn-level decision with an evidence trace. Its recommended operating point, SAGE-Core, decides 81–91% of criteria with only the compiler, symbolic rules, and on-device encoders–at zero paid LLM cost–while SAGE-LLM adds an optional focused-LLM fallback for open-class criteria. Across four slices spanning MultiWOZ, Schema-Guided Dialogue, and ABCD, no evaluated LLM-as-a-judge baseline–including a state-aware GPT-4.1 judge and cheaper GPT-4.1-mini variants–significantly exceeds SAGE-Core on any slice, even though the GPT-4.1 G-Eval judge costs 4.7−−8.0per1,000turnstoSAGE−Core′s0. A two-annotator human audit (n=200, κ=0.94) confirms strong label fidelity on the transcript-visible failure classes–where, excluding the weak-salience IUV class, SAGE-Core is statistically tied with the strongest LLM judge–and honestly scopes ignored-user-value as a state-consistency signal with weak broad-human salience. We analyze construct-validity limits from injected failures and partial symbolic circularity.

from the abstract

Pith says

Machine review · deepseek-v4-flash · 9 Sep 2026 · what was read was not recorded · not checked by a person

It can be wrong. Challenge this review or reply to it.

Holds up, with conditions. Solid, well-executed work on a practical problem: zero-cost evaluation of task-oriented dialogue agents via workflow compilation and a heterogeneous cascade with abstention. The central claim is internally robust on the tested benchmark slices, though external validity is limited by injected failures and one weak failure class. This line is wrong

2 major objections · 6 minor

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

Not checked: figures, code, what the 30 references say.

The paper says, at Human Validation Audit (Table 2); Discussion, Symbolic-injection circularity

The review did not quote this passage.

Pith says: Objection 1 of 2 · major

WNF is the softest of the transcript-visible classes (human-injected agreement 0.73, n=15) and, like IUV, one of the three classes whose S1 verifier shares a detection predicate with the injection operator. The paper convincingly shows the headline paired bootstraps are unchanged when all injected IUV turns are dropped, but no analogous check is reported for WNF.

more of objection 1

Because a shared-predicate class with sub-ceiling human agreement is exactly where circularity could inflate SAGE-Core's injected-label F1 relative to holistic judges, please add a WNF-excluded (or WNF-collapsed) rerun of the pre-specified paired bootstraps and of the human-consensus F1 comparison on the transcript-visible subset. The existing IUV-exclusion logic makes this a straightforward addition, and it would close the remaining circularity gap on the classes humans validate only partially.

The paper says, at Results (ABCD-UV caveat); Discussion, Symbolic-injection circularity

The review did not quote this passage.

Pith says: Objection 2 of 2 · major

The Results caveat for the ABCD-UV slice states that 'most of its criteria are decided by S1/S2 predicates that partially mirror the injection operators,' and the Discussion attributes the ABCD win to 'inverting its own injector on the symbolically-decidable subset.'

more of objection 2

This overstates the shared-predicate circularity for that slice: ABCD-UV is UV-only, and the paper's own Discussion states that UV detection runs an off-the-shelf DeBERTa-v3 NLI model that shares no symbolic predicate with the injection operator (the injected values are merely chosen to be NLI-detectable). Please align the two passages so that the ABCD nominal win (+.056, p=.044) is not misread as pure S1 predicate inversion, and specify which ABCD criteria are in fact decided by S1.

6 minor objections

The paper says, at Results, UV-rich paragraph

Pith says: minor objection 1 of 6

The parenthetical '(≈14× the cost and ≈44× the latency relative to SAGE-LLM)' appears in a sentence comparing G-Eval with SAGE-Core; the multipliers are apparently G-Eval relative to SAGE-LLM. Please state the reference configuration explicitly wherever this comparison is made.

The paper says, at Table 2 caption

Pith says: minor objection 2 of 6

The per-class n's (PC 14, UV 39, WNF 15, IUV 30) sum to 98 while 100 injected turns were sampled; the discrepancy (and the clean-side count) is explained by dropping six turns for annotator disagreement, but the caption should state explicitly that the per-class counts are consensus-set counts.

show 4 more

The paper says, at Experiments, Baselines

Pith says: minor objection 3 of 6

The transcript-only FrugalGPT row is disclosed as 'largely reflect[ing] gpt-4.1-mini alone' because its confidence model is state-aware; this means the row is not a faithful transcript-only cascade. One sentence in the main text noting that a transcript-only FrugalGPT routing would require a transcript-only confidence model would prevent the row from being read as a clean state-access ablation.

The paper says, at Experiments, Metrics

Pith says: minor objection 4 of 6

The headline claim is directional ('no baseline significantly exceeds SAGE-Core'), but all reported tests are two-sided. Reporting the one-sided tail probability or the lower confidence bound for the 'baseline − SAGE-Core' difference on each slice would make the directional claim immediate rather than indirect.

The paper says, at Additional analyses; Appendix I

Pith says: minor objection 5 of 6

The main text cites SGD-X robustness only as 'verdict σ=0.002'; the more informative number is the absolute F1 drop from 0.864 to 0.628 on paraphrased schemas, which the appendix states honestly. Please mention the absolute drop in the main text so the robustness claim is not read as paraphrase immunity.

The paper says, at Human Validation Audit, IUV class

Pith says: minor objection 6 of 6

The 1/30 human-injected agreement rate has a wide binomial confidence interval (roughly one-tenth of a percent to seventeen percent at 95% confidence); reporting the interval or the per-annotator counts would give readers a precise scale for the 'weak broad-human salience' conclusion.

What would settle it

Recompute Table 1's paired bootstrap comparisons after excluding all turns whose injected failure class shares its detection predicate with S1 (i.e., WNF, PC, IUV), retaining only UV-injected and clean turns. If SAGE-Core's F1 advantage over the best LLM judge disappears or reverses on any slice, the circularity concern would be confirmed. The paper's existing human-validated analysis (excluding IUV) already suggests it would not.

from the skeptic's reading

How the review went

4 readings by deepseek-v4-flash · 9 Sep 2026

  1. Reader holds up, with conditions confidence high
  2. Skeptic partly agrees 41 s · 21,873 tokens read, 4,295 written
  3. Editor wrote the verdict above 12 s · 22,778 tokens read, 1,212 written
  4. Referee minor revision: 2 major objections, 6 minor 2 min 58 s · 22,427 tokens read, 18,834 written

Not run on this paper: the simulated authors' defence.

Pith did not check this review's objections against the paper.

The review in full

The review in brief, machine-written

Here's my read on the SAGE paper.

What you should know: this is a genuinely useful piece of engineering for anyone running task-oriented dialogue agents at scale. The core idea—compile a workflow spec and per-turn state diff into atomic criteria, then run each through a cheap-to-expensive cascade (symbolic, NLI encoder, optional focused LLM) that abstains rather than guesses—is both novel and practical. SAGE-Core (no LLM calls) matches or beats expensive GPT-4.1 judges on four benchmark slices at zero API cost. That's a real result.

The empirical work is thorough. Paired bootstrap significance tests with within-resample baseline re-selection, a two-annotator human audit (n=200, κ=0.94), per-tier ablations, cross-dataset generalization to SGD and ABCD, and a paraphrase-robustness check. The paper is also unusually transparent about its weaknesses: partial circularity for three of four failure classes (S1 shares predicates with the injector), the IUV class with near-zero human agreement, and the reliance on injected rather than naturally occurring failures.

Soft spots in proportion. The biggest is that all evaluations use injected failures, so we don't know how SAGE behaves on real-world workflow errors. The IUV class has weak construct validity (1/30 human agreement), but the authors show that excluding it doesn't change headline comparisons—and actually strengthens them. The partial circularity is real but bounded by the human audit showing high agreement on PC/UV/WNF, and on the transcript-visible subset excluding IUV, SAGE-Core is statistically tied with the best LLM judge (F1 0.909 vs 0.920, p=0.68). So the internal validity holds.

Who this is for: industry practitioners building evaluation pipelines for task-oriented agents, and researchers working on cost-aware LLM evaluation or state-grounded dialogue metrics. The paper deserves serious peer review—it's well-motivated, methodologically sound, and honest about limitations. I'd bring it to reading group, cite it if I work on TOD evaluation, and accept it with minor caveats about external validity.

Recommendation: accept with the caveat that a natural-failure study is the obvious next step. No methodological flaws that would warrant rejection.

The strongest objection

The most salient concern is that for WNF, PC, and IUV, the S1 symbolic verifier shares a detection predicate with the injection operator, meaning SAGE-Core's S1 stage is a perfect detector for these injected failures by construction. This creates a form of label leakage that could inflate SAGE-Core's F1 relative to LLM judges that must infer failures holistically. However, the paper's human audit (n=200, κ=0.94) substantially bounds this risk: human-injected agreement is 1.00 for PC, 0.95 for UV, and 0.73 for WNF, showing that these injected failures are perceived as genuine by annotators. The IUV class (agreement 0.03) is weak, but the paper shows that excluding IUV does not change (and in fact strengthens) the headline comparisons. Furthermore, on the transcript-visible subset (excluding IUV), SAGE-Core is statistically tied with the best LLM judge (F1 0.909 vs 0.920, p=0.68), so the circularity does not produce an artificial advantage on the classes humans validate. The remaining concern is about generalizability to naturally occurring failures, but the central claim is explicitly scoped to the evaluated benchmark slices. Therefore, while the partial circularity is a real limitation for external validity, it does not undermine the internal validity of the claim that on these specific slices, no evaluated LLM judge significantly exceeds SAGE-Core.

The reader identifies partial circularity and weak IUV construct validity as the weakest assumption. These are genuine limitations for external validity, but the paper already conducts the human audit and shows that the headline conclusions are robust to excluding the most problematic class (IUV). The central claim is about the specific benchmark slices, not generalization to fully natural failures. The concern does not threaten the internal validity of the comparison on those slices. Therefore, while I agree that the caveats are important, they do not warrant a stronger verdict change. The reader's CONDITIONAL verdict is appropriate and my analysis leaves it unchanged.

The referee's report

If the empirical results hold, SAGE-Core is a credible and practically valuable zero-API-cost alternative to holistic LLM judges for workflow-grounded TOD evaluation, with per-criterion evidence traces, structured repair signals, and tunable risk–coverage frontiers. The experimental methodology is a genuine strength: paired-bootstrap tests that re-select the strongest baseline within each resample, row-for-row aligned slices across all methods, a blind two-annotator audit with released protocols and pinned encoder revisions (making SAGE-Core bitwise deterministic on fixed turns), a seven-setting threshold sweep confirming a 0.007 F1 band, per-tier leave-one-out ablations, SGD-X paraphrase checks, and an oracle-routing robustness analysis. The paper is unusually candid: it reports the significant human-consensus deficit of SAGE-Core relative to the best LLM judge on the full audit set (p=0.01), the weak construct alignment of IUV, and the injected-only benchmark construction, and it distinguishes nominal from Holm-surviving wins.

Pith's reading of the paper (a machine's summary, not the authors')

Task-oriented dialogue agents — booking, support, ticketing — are judged on whether each turn advances an underlying workflow state, not just on whether the reply reads fluently. This paper argues that holistic LLM-as-a-judge evaluation is structurally mismatched to that task: it scores the whole transcript as one unit, can conflate fluent phrasing with correct state updates, and pays for a full model call even on turns a deterministic check could settle. SAGE instead compiles the workflow specification and the per-turn state diff $\Delta_t$ into atomic, schema-grounded criteria and verifies each through a cascade — deterministic symbolic rules, an on-device entailment encoder that abstains when its confidence margin is too low, and an optional focused LLM for the residual. The recommended configuration, SAGE-Core, decides 81–91% of criteria with zero paid LLM calls, and across four benchmark slices spanning three datasets no evaluated LLM judge — including a state-aware GPT-4.1 judge costing $4.7–8.0 per 1,000 turns — significantly exceeds its F1. A two-annotator human audit largely confirms the failure labels humans can see, but exposes one class (ignored user values) as weakly aligned with human judgment and motivates the paper's own caveat that the injected-failure benchmarks partially share predicates with the evaluator itself.

What carries the argument. The load-bearing object is the state diff $\Delta_t$ together with the workflow specification $W=(S,F,\prec,O)$ — schema, required fields, legal ordering relation, and outcome predicate — which the compiler turns into atomic criteria, one per failure class and one per written value. Each criterion is routed through a three-stage cascade that treats abstention as a first-class output: S1, deterministic rules over $\Delta_t$ and the schema (deciding 56–65% of criteria); S2, a pretrained encoder/NLI entailment check whose confidence gate abstains when the margin $m = p(\mathrm{ent}) - p(\mathrm{contra})$ satisfies $|m| < \tau$ (deciding a further 25–31%); and S3, an optional focused single-criterion LLM prompt for the residual open-class criteria. The aggregator turns criterion verdicts into a turn-level decision — failure if any criterion fires, acceptable only if all are decided and false, undecided otherwise — and emits a structured repair signal per violation. This machinery changes the unit of judgment from the whole transcript to individual state-grounded claims, which is what lets most verdicts be produced deterministically at near-zero cost, and it is also what makes the evaluation fragile: for the WNF, PC, and IUV classes the injection operator and the S1 predicate are the same check.

Core claim. The paper's central claim is that evaluation of stateful dialogue agents should be criterion-level and state-grounded: a turn is acceptable only if the per-turn state diff $\Delta_t = \mathrm{diff}(s_{t-1}, s_t)$ satisfies the workflow specification, which decomposes into four failure classes — ignored user value (IUV), wrong next field (WNF), premature confirmation (PC), and unsupported value (UV). The compiler maps each turn into atomic criteria, and the cascade verifies each criterion at the cheapest stage that can decide it, with abstention propagating upward; a turn is a failure if any criterion fires, acceptable only if every criterion is decided and false, otherwise undecided. Empirically, SAGE-Core (compiler plus symbolic rules plus encoder/NLI, no paid LLM) is never significantly exceeded by any evaluated LLM-as-a-judge baseline on any of the four slices: it wins on the Schema-Guided Dialogue slice (+.075, p=.011, the one comparison that survives multiple-comparison correction), wins nominally on two others, and statistically ties the expensive judge on the single-class stress test — all at $0 paid API cost versus $4.7–8.0 per 1,000 turns for the GPT-4.1 judge. The authors' own human audit shows the claim does not extend equally to all four classes: humans endorsed the injected labels on only 1/30 ignored-user-value turns, and while SAGE-Core ties the strongest LLM judge against human consensus on the transcript-visible classes (F1 0.909 vs 0.920, p=.68), it trails overall (0.825 vs 0.904, p=.01) because of that contested class.

The premise it rests on. The benchmarks are built from injected failures whose mutation operators share detection predicates with SAGE's own symbolic verifier for three of the four classes, and human annotators endorsed only 1 of 30 injected ignored-user-value failures, so the measured ranking over LLM judges may not survive a test set of naturally occurring workflow errors.

If the paper is right (Pith's inferences, not the authors')

Deployment cost collapses: SAGE-Core decides 81–91% of compiled criteria with the compiler, symbolic rules, and on-device encoders, issues zero paid API calls, and runs at about 0.03 s warm local latency, versus $4.7–8.0 per 1,000 turns and roughly 12 s per turn for the GPT-4.1 judge on the same slices.

The paid focused-LLM tier is not load-bearing: SAGE-Core matches or exceeds the full SAGE-LLM cascade on every slice, so the zero-cost configuration is also the stronger one.

Simply giving holistic judges the same state context does not close the gap: in the authors' prompt template, state-aware GPT-4.1 judges score up to 0.09 lower in F1 than their transcript-only versions, so the advantage is attributed to changing the unit of judgment rather than to seeing more information.

The abstention signal is calibrated for risk–coverage trading: declining the least-confident 10% of turns raises F1 by up to +0.023 on the encoder-heavy slices, giving operators a tunable reliability knob rather than a single operating point.

On paraphrased schemas (SGD-X), SAGE-Core's verdicts are near-invariant (σ=0.002 vs 0.005–0.015 for LLM judges), indicating that the checks key on slot identity rather than surface names.

The shared predicates between the injectors and S1 mean the headline wins are best read as 'the evaluator recovers the failures it was built to detect'; the decisive next test, which the paper leaves to future work, is a naturally occurring failure set on which an LLM judge could in principle reverse the ranking.

Because SAGE-Core is deterministic, runs on a laptop CPU, and costs nothing per call, the architecture invites a serving-time monitoring extension: continuously checking every live agent turn against the declared workflow, where per-turn API judging would be prohibitively expensive.

The IUV audit result (1/30 human endorsement) suggests that not all state-consistency violations are user-meaningful; an evaluator that weighted failure classes by measured human salience would present a different, and possibly higher, agreement profile.

The paper deliberately stops short of a repair loop even though every verdict carries a structured repair signal; feeding that signal back into the agent and re-scoring would test whether SAGE's evidence trace actually helps agents self-correct, which is its stated downstream purpose.

Check this review

Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.

How often reviews like this one are right

Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper

The paper

Abstract

Evaluating task-oriented dialogue agents requires judging not merely whether a reply reads well but whether each turn advances the underlying workflow state correctly–a distinction conventional holistic LLM judges can miss because they evaluate the available context as a single unit and require one or more full-model calls per turn. We propose SAGE (State-Grounded Abstention-Aware Evaluation), which compiles a workflow specification and per-turn state diff into atomic, schema-grounded criteria and routes each through a cascade of symbolic and encoder/NLI verifiers that abstain rather than guess, aggregating criterion verdicts into a turn-level decision with an evidence trace. Its recommended operating point, SAGE-Core, decides 81–91% of criteria with only the compiler, symbolic rules, and on-device encoders–at zero paid LLM cost–while SAGE-LLM adds an optional focused-LLM fallback for open-class criteria. Across four slices spanning MultiWOZ, Schema-Guided Dialogue, and ABCD, no evaluated LLM-as-a-judge baseline–including a state-aware GPT-4.1 judge and cheaper GPT-4.1-mini variants–significantly exceeds SAGE-Core on any slice, even though the GPT-4.1 G-Eval judge costs 4.7−−8.0per1,000turnstoSAGE−Core′s0. A two-annotator human audit (n=200, κ=0.94) confirms strong label fidelity on the transcript-visible failure classes–where, excluding the weak-salience IUV class, SAGE-Core is statistically tied with the strongest LLM judge–and honestly scopes ignored-user-value as a state-consistency signal with weak broad-human salience. We analyze construct-validity limits from injected failures and partial symbolic circularity.

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-09-23

The authors' main claim, as the classifier picked it out 1.00

“Across four slices spanning MultiWOZ, Schema-Guided Dialogue, and ABCD, no evaluated LLM-as-a-judge baseline--including a state-aware GPT-4.1 judge and cheaper GPT-4.1-mini variants--significantly exceeds SAGE-Core on any slice, even though the GPT-4.1 G-Eval judge costs $4.7--8.0 per 1,000 turns to SAGE-Core's $0.”

How the main claim is stated: asserted 1.00

What the abstract itself states, as probabilities: claims first 0.07 · states limitation 0.96 · human participants 0.62 · controlled comparison 0.94 · randomized assignment 0.02 · states quantitative result 0.99 · compares with prior results 0.98 · mentions code or data availability 0.05

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
Evaluating task-oriented dialogue agents requires judging not merely whether a reply reads well but whether each turn advances the underlying workflow state correctly--a distinction conventional holistic LLM judges can miss because they evaluate the available context as a single unit and require one or more full-model calls per turn. background 0.57 0.10 0.07 asserted 0.95
We propose SAGE (State-Grounded Abstention-Aware Evaluation), which compiles a workflow specification and per-turn state diff into atomic, schema-grounded criteria and routes each through a cascade of symbolic and encoder/NLI verifiers that abstain rather than guess, aggregating criterion verdicts into a turn-level decision with an evidence trace. method 0.77 0.92 0.14 asserted 1.00
Its recommended operating point, SAGE-Core, decides 81--91% of criteria with only the compiler, symbolic rules, and on-device encoders--at zero paid LLM cost--while SAGE-LLM adds an optional focused-LLM fallback for open-class criteria. method 0.84 0.93 0.16 asserted 1.00
Across four slices spanning MultiWOZ, Schema-Guided Dialogue, and ABCD, no evaluated LLM-as-a-judge baseline--including a state-aware GPT-4.1 judge and cheaper GPT-4.1-mini variants--significantly exceeds SAGE-Core on any slice, even though the GPT-4.1 G-Eval judge costs $4.7--8.0 per 1,000 turns to SAGE-Core's $0. result 1.00 0.94 0.49 asserted 1.00
A two-annotator human audit (n=200, $\kappa$=0.94) confirms strong label fidelity on the transcript-visible failure classes--where, excluding the weak-salience IUV class, SAGE-Core is statistically tied with the strongest LLM judge--and honestly scopes ignored-user-value as a state-consistency signal with weak broad-human salience. result 0.99 0.96 0.15 asserted 1.00
We analyze construct-validity limits from injected failures and partial symbolic circularity. limitation 0.96 0.89 0.03 asserted 0.64

What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.97

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-09-23: 1 request, 5,482 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

Figures (4)
Figure 1 of the paper
Figure 1. A fluent, on-topic agent turn that is nonetheless workflow-wrong: the user asks to book in
Figure 2 of the paper
Figure 2. The SAGE pipeline with measured routing (Appendix G). The compiler emits atomic schema-grounded criteria tagged
Figure 3 of the paper
Figure 3. Cost-quality Pareto on MultiWOZ 2.4 UV-rich,
Figure 4 of the paper
Figure 4. Precision, recall, and F1 on MultiWOZ 2.4 UV￾rich (n=250); exact values in

Figures are the authors', reproduced from the paper.

References (30)
  1. [1] Chen, Lingjiao and Zaharia, Matei and Zou, James , journal= on Pith
  2. [2] Liu, Yang and Iter, Dan and Xu, Yichong and Wang, Shuohang and Xu, Ruochen and Zhu, Chenguang , booktitle= on Pith
  3. [3] Zheng, Lianmin and Chiang, Wei-Lin and Sheng, Ying and Zhuang, Siyuan and Wu, Zhanghao and Zhuang, Yonghao and Lin, Zi and Li, Zhuohan and Li, Dacheng and Xing, Eric P. and Zhang, Hao and Gonzalez, Joseph E. and Stoica, Ion , booktitle=. Judging on Pith
  4. [4] Budzianowski, Pawe. Proceedings of EMNLP , year= on Pith
  5. [5] Eric, Mihail and Goel, Rahul and Paul, Shachi and Sethi, Abhishek and Agarwal, Sanchit and Gao, Shuyang and Kumar, Adarsh and Goyal, Anuj and Ku, Peter and Hakkani-Tur, Dilek , booktitle= on Pith
  6. [6] Ye, Fanghua and Manotumruksa, Jarana and Yilmaz, Emine , booktitle= on Pith
  7. [7] Towards Scalable Multi-Domain Conversational Agents: The Schema-Guided Dialogue Dataset , author=. Proceedings of the AAAI Conference on Artificial Intelligence , year= on Pith
  8. [8] Lee, Harrison and Gupta, Raghav and Rastogi, Abhinav and Cao, Yuan and Zhang, Bin and Wu, Yonghui , booktitle= on Pith
  9. [9] Laban, Philippe and Schnabel, Tobias and Bennett, Paul N. and Hearst, Marti A. , journal= on Pith
  10. [10] A Large Annotated Corpus for Learning Natural Language Inference , author=. Proceedings of EMNLP , year= on Pith
  11. [11] A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference , author=. Proceedings of NAACL , year= on Pith
  12. [12] He, Pengcheng and Gao, Jianfeng and Chen, Weizhu , booktitle= on Pith
  13. [13] Reimers, Nils and Gurevych, Iryna , booktitle= on Pith
  14. [14] Wang, Wenhui and Wei, Furu and Dong, Li and Bao, Hangbo and Yang, Nan and Zhou, Ming , booktitle= on Pith
  15. [15] Song, Kaitao and Tan, Xu and Qin, Tao and Lu, Jianfeng and Liu, Tie-Yan , booktitle= on Pith
  16. [16] Selective Classification for Deep Neural Networks , author=. Advances in Neural Information Processing Systems , year= on Pith
  17. [17] Chain-of-Thought Prompting Elicits Reasoning in Large Language Models , author=. Advances in Neural Information Processing Systems , year= on Pith
  18. [18] Prometheus: Inducing Fine-Grained Evaluation Capability in Language Models , author=. International Conference on Learning Representations , year= on Pith
  19. [19] Wang, Yidong and Yu, Zhuohao and Yao, Wenjin and Zeng, Zhengran and Yang, Linyi and Wang, Cunxiang and Chen, Hao and Jiang, Chaoya and Xie, Rui and Wang, Jindong and Xie, Xing and Ye, Wei and Zhang, Shikun and Zhang, Yue , booktitle= on Pith
  20. [20] Dubois, Yann and Galambosi, Bal. Length-Controlled. arXiv preprint arXiv:2404.04475 , year= on PithPith's reviewarXiv
show 10 more
  1. [21] Ong, Isaac and Almahairi, Amjad and Wu, Vincent and Chiang, Wei-Lin and Wu, Tianhao and Gonzalez, Joseph E. and Kadous, M. Waleed and Stoica, Ion , booktitle= on Pith
  2. [22] Aggarwal, Pranjal and Madaan, Aman and Anand, Ankit and Potharaju, Srividya Pranavi and Mishra, Swaroop and Zhou, Pei and Gupta, Aditya and Rajagopal, Dheeraj and Kappaganthu, Karthik and Yang, Yiming and Upadhyay, Shyam and Faruqui, Manaal and Mausam , booktitle= on Pith
  3. [23] Jiang, Dongfu and Ren, Xiang and Lin, Bill Yuchen , booktitle= on Pith
  4. [24] Min, Sewon and Krishna, Kalpesh and Lyu, Xinxi and Lewis, Mike and Yih, Wen-tau and Koh, Pang Wei and Iyyer, Mohit and Zettlemoyer, Luke and Hajishirzi, Hannaneh , booktitle= on Pith
  5. [25] Manakul, Potsawee and Liusie, Adian and Gales, Mark J. F. , booktitle= on Pith
  6. [26] Es, Shahul and James, Jithin and Espinosa-Anke, Luis and Schockaert, Steven , booktitle= on Pith
  7. [27] Action-Based Conversations Dataset: A Corpus for Building More In-Depth Task-Oriented Dialogue Systems , author=. Proceedings of NAACL-HLT , year= on Pith
  8. [28] A Simple Sequentially Rejective Multiple Test Procedure , author=. Scandinavian Journal of Statistics , volume= on Pith
  9. [29] An Introduction to the Bootstrap , author= on Pith
  10. [30] A Coefficient of Agreement for Nominal Scales , author=. Educational and Psychological Measurement , volume= on Pith
Cited by 1 Pith paper

Reviewed papers on Pith that cite this one. The full citation record

  1. Veritas: A Verifier Generation Harness for Agent Tasks cs.AI · Nov 2026

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment