DS@GT ARC at Touch\'e: Large Language Models for Retrieval-Augmented Debate

Anthony Miyaguchi; Conor Johnston

REVIEW 4 major objections 6 minor 13 references

DS@GT ARC at Touch\'e: Large Language Models for Retrieval-Augmented Debate

T0 review · 4 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper reports that six frontier LLM judges agree strongly on retrieval-augmented debate responses, but the agreement is largely a shared-training-data artifact and does not reliably track the official evaluation scores, with the…

desk verdict The consensus-vs-F1 finding is likely real, but the headline numbers rest on an under-specified label model; still worth sending out. read the letter →

arxiv 2608.08143 v1 pith:5A5G5YPT submitted 2026-08-08 cs.IR cs.CL

Anthony Miyaguchi , Conor Johnston This is my paper · ORCID

classification cs.IRcs.CL

keywords largelanguagemodelsretrieval-augmenteddebateGriceanmaximsLLM-as-judgeevaluatorconsensusweaksupervisionself-evaluation

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper extends a shared-task working note to ask a methodological question: can the consensus of multiple LLM judges stand in for official evaluation in retrieval-augmented debate? It answers no. Six frontier LLMs from three providers agree strongly as judges, but the agreement is driven by model family and shared training data, and the consensus-based latent accuracy estimates do not reliably match the official Subtask 2 F1 scores. The largest disconnect is on the Quality maxim, where judges agree on surface plausibility without verifying factual support. Even a dependency-aware weak-supervision model closes only part of the gap. If the finding holds, retrieval-augmented debate evaluations cannot simply average a panel of LLM judges and expect the result to track the official target.

What carries the argument

The central object is the multi-LLM judge panel: six frontier LLMs queried through a retrieval-augmented prompting pipeline, each producing scores from 0 to 1 on the four Gricean maxims. The decisive machinery is the pairwise correlation structure among judges, with same-family judges correlating around 0.84 and 0.77 for the two provider pairs, combined with a weak-supervision label model that treats binarized judge scores and surface heuristics as noisy votes and estimates latent accuracies for comparison against official F1. The misfit between latent accuracy and official F1, with the largest gap on Quality, carries the argument.

What would settle it

Re-annotate a random sample of the evaluation turns with expert adjudicators, then recompute both official F1 and consensus-based latent accuracy; the claim weakens if the revised Quality labels bring the two measures into close agreement.

Watch

Extended reading notes

Core claim

The paper shows that when six frontier LLMs from three families serve as judges of retrieval-augmented debate responses, they agree strongly, especially within the same model family, and a weak-supervision label model over those judges recovers essentially the same consensus as the raw average. Yet that consensus barely tracks the official Subtask 2 F1 scores: dependency-aware latent accuracy averages 0.687 for Quality while the official Quality F1 averages 0.244, and Spearman correlations between latent accuracy and official F1 stay below 0.6 for all maxims, with Manner at 0.075. The authors conclude that high multi-LLM agreement is not a reliable proxy for the official evaluation target, with the largest gap on the Quality maxim.

Load-bearing premise

The official binary annotations on the four Gricean maxims are treated as the ground-truth evaluation target; if those annotations are noisy or miss what debate quality really means, then the gap between LLM consensus and official scores shows disagreement with an imperfect label set rather than a failure of the LLM judges.

Editorial extensions

If this is right

A shared-task leaderboard cannot be safely reconstructed by averaging a panel of frontier LLM judges, because the consensus ranking would not match the official binary-annotation ranking.
Evaluators that draw on a single model family inherit a shared bias, so apparent panel agreement overstates confidence in the scores.
Weak-supervision calibration only changes rankings when the voter pool includes non-redundant judges; adding diverse small models shifts the rankings, while adding more frontier models does not.
For the Quality maxim, any evaluation that relies on multi-LLM agreement should be treated as measuring surface plausibility rather than factual, evidence-based quality.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the official annotations themselves are noisy, the largest gap on Quality could reflect annotation problems as much as judge unreliability; adjudicating a sample of the roughly 4,000 evaluation turns would separate the two.
The same consensus-versus-target comparison could be run as a pre-submission sanity check in other shared tasks with binary labels: when same-family judges dominate, report family-specific correlations before trusting consensus.
A testable extension would split Quality into sub-scores such as factuality, evidence support, reasoning validity, and usefulness, and measure whether judge consensus then tracks official scores better, since the paper argues the collapsed single score invites surface-plausibility agreement.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The paper's core observation is worth knowing: six frontier LLM judges agree strongly within model families, but that consensus does not align well with the official Touché 2025 F1, with the largest gap on Quality. The within-family correlations (Claude 0.844, GPT 0.772) and the weak-supervision results are a useful caution for anyone using multi-LLM agreement as a proxy for evaluation quality in shared tasks.

What the paper does well is mostly in the post-hoc analysis. It extends the working note with organizer-released data, explicitly discusses the circularity of using the same models as generators and judges, and ships code and data links. It also makes a fair point that adding lower-correlation LLaMA-8B judges changes rankings, which shows that judge diversity matters beyond raw strength. The discussion of Quality as a construct that collapses factuality, evidence support, and reasoning validity is sensible.

Now the soft spots. The stress-test is partly right: the headline Spearman numbers in Table 5 (e.g., 0.224 for Quality) are correlations between latent judge-accuracy estimates from a FlyingSquid-style label model and official F1, not direct agreement between observed judge scores and official labels. The label model assumes conditional independence, and Figure 7 shows that assumption is violated for same-family judges. The dependency-aware variant is described only as "explicitly linked" with no model specification, so readers cannot reproduce the corrected numbers. That is a real transparency gap. However, the stress-test overstates when it says the paper never checks observed agreement against F1. Figure 6 directly compares averaged judge scores to official binary annotations and shows the DS@GT systems beat baseline under judge averages but not by the official metric, and Figure 1 gives a concrete example where all six frontier judges marked a response satisfactory while official annotators marked all four maxims unsatisfied. So the qualitative finding is not solely an artifact of the label model. The bigger issues are that the paper presents no significance tests on the Spearman/Kendall values (with only eight systems, the correlations are noisy) and the abstract overclaims that the systems are "strong response generators" when only two of six runs beat the baseline on official Subtask 1. It also does not assess annotation quality, but that is minor for a shared-task paper.

Verdict: this deserves a serious referee. It is a diagnostic contribution for LLM-as-judge reliability and shared-task evaluation, and the honest limitations are a plus. The revision should either add a direct correlation between observed mean judge scores and official F1, validate latent accuracies against observed per-judge accuracy on official labels, and specify the dependency-aware model, or soften the claims to match what is actually shown. Do not desk reject.

Referee Report

4 major / 6 minor

Summary. The paper extends the DS@GT ARC working note for the Touché 2025 Retrieval-Augmented Debate task. It summarizes the official leaderboard results for six LLM-based response-generation and evaluation runs and then analyzes post-hoc whether multi-LLM evaluator consensus is a reliable proxy for official Subtask 2 F1. The analysis uses pairwise judge-score correlations, a FlyingSquid-style weak-supervision label model over LLM judges and surface heuristics, and compares latent judge-accuracy estimates with official F1 via Spearman correlation. The paper finds strong within-family judge agreement, weaker agreement with LLaMA-8B judges, and low Spearman correlations between latent accuracy estimates and official F1, with Quality showing a particularly large gap. The conclusion is that frontier LLM consensus does not reliably align with the official evaluation target.

Significance. If the finding is robust, it is a useful caution for shared-task evaluation: multi-LLM agreement is structured by model family and does not guarantee alignment with official human annotations. The paper's strengths include public code, use of an independent official target (not circular), and the explicit demonstration that within-family agreement inflates consensus. However, the central quantitative evidence passes through an unvalidated label model and a small sample, so the significance currently rests on the direct comparison in Figure 6 vs. Table 3 and on correlations that need further support.

major comments (4)

[Section 4.2 / Table 5] The central claim is supported by Spearman correlations between latent judge-accuracy estimates and official Subtask 2 F1, but these latent estimates are outputs of a FlyingSquid-style label model that assumes conditional independence of voters; Figure 7 shows that assumption is violated for same-family frontier LLMs (e.g., Claude Opus/Sonnet mean correlation 0.844). Because the label model is never validated against observed judge accuracy on official labels, the reported low correlations could be artifacts of the model rather than properties of evaluator consensus. Please add a direct test: compute each judge's observed agreement or F1 against the official labels and correlate those observed quantities with official Subtask 2 F1, and also report the correlation between mean judge scores and official F1.
[Section 3.3 / Table 5] The dependency-aware label model is described only as 'explicitly linked' correlated voters, with no specification of how dependencies are modeled or estimated. Since the dependency-aware Spearman values are headline results (e.g., Quality rises from 0.224 to 0.286), the model needs a full formal description or a reference to a concrete implementation; otherwise the numbers are not reproducible.
[Table 5 / Section 4.2] The Spearman correlations are computed over only eight systems (six DS@GT runs, baseline, and SINAI). With n=8, none of the reported correlations is statistically distinguishable from zero at conventional levels; for example, ρ=0.580 for Relation has an approximate p-value around 0.13. Please report confidence intervals or permutation-test p-values, and discuss the power limitation.
[Section 5 / Section 4.2] The claim that the largest gap is on Quality is supported by comparing latent accuracy average 0.687 with official Quality F1 average 0.244. These are different metrics (latent accuracy vs. F1) and no comparable gap is reported for the other maxims. Please report per-maxim mean latent accuracy and official F1 in the same table, or replace the 'largest gap' claim with a measure that is consistent across maxims.

minor comments (6)

[Section 3.3] The sentence 'We look at evaluator consensus through pairwise Pearson correlation computed over response turns separately per maxim and mean averaged' is unclear about what is averaged; specify whether the mean is over maxims or over judge pairs.
[Table 3 / Figure 6] The direct mismatch between averaged judge scores and official binary annotations is a key observation; consider quantifying it (e.g., rank correlation between mean judge score and official maxim-fulfillment proportion).
[Section 4.2] The number of rows (49,600) and the mapping from response turns to annotation rows should be explained, since 4,000 response turns would not obviously produce 49,600 rows with six judges and four maxims.
[Section 3.3] Binarizing each voter at its median discards information; report sensitivity to the binarization threshold (e.g., using the mean or a fixed threshold such as 0.5).
[Section 4.1 / Table 2] Table 2 reports internal evaluations from the working note; clarify whether these numbers are used later, since the analysis in Section 4.2 uses the organizer-released corpus.
[Section 7] The paper states that the question is whether retrieval-augmented debate can be evaluated reliably through multi-LLM agreement; however, the same six LLMs serve as both generators and judges, and this self-evaluation aspect is not explicitly discussed as a limitation. Add a sentence acknowledging this potential bias.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the consensus-versus-official-F1 comparison uses independent official annotations, and the weak-supervision label model is an unsupervised diagnostic rather than a fitted predictor.

full rationale

The central claim compares multi-LLM evaluator consensus (and latent accuracy estimates derived from it) with the official Subtask 2 F1, which is computed from the organizers' binary annotations. The weak-supervision label model (Sections 3.3 and 4.2) is estimated without access to official labels: voters are binarized at their medians and treated as noisy indicators of a latent label, with ten surface heuristics as additional voters; official F1 plays no role in estimating latent accuracies. The reported Spearman correlations are therefore not forced by construction. The dependency-aware variant is an exploratory post-hoc model and, if anything, selecting it to improve alignment would bias toward higher correlation, so the finding that alignment remains low is not an artifact of fitting. The Kendall tau = 1.0 between the frontier-only label model and raw average is a consistency property of the label model, not a claimed external prediction. The self-citation to the authors' own working note [9] is background describing the submitted systems and is not load-bearing for the new analysis. The paper explicitly acknowledges limitations—the Gricean maxims are broad and the same six LLMs act as both generators and judges—but these are validity concerns, not circular reductions. No equation is defined in terms of the target result, and no fitted parameter is renamed as a prediction.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the official annotations being a valid target and on the chosen correlation and weak-supervision frameworks. No new entities are introduced; free parameters are limited to modeling choices in the label model.

free parameters (3)

Binarization threshold (median) for heuristic voters = per-voter median
Each surface heuristic is binarized at its median to serve as a noisy vote in the weak supervision model (Section 3.3); this is a data-dependent modeling choice.
Voter accuracy latent variables in label model = estimated, not reported
The FlyingSquid-style label model infers latent accuracies for each voter; the estimated values are not listed, only aggregate correlations against official F1.
Heuristic feature set = 10 hand-picked heuristics
The set of ten surface heuristics covering length, lexical overlap, retrieval, readability, and repetition is chosen by the authors without a selection criterion.

assumptions (4)

domain assumption Official Touché binary annotations on Gricean maxims are the ground-truth evaluation target
The paper's central comparison treats official Subtask 2 F1 as the correct measure (Section 4.2).
domain assumption Gricean maxims are the appropriate basis for evaluating debate responses
The evaluation framework is inherited from the task organizers and assumed valid (Introduction, Section 5).
standard math Pearson and Spearman correlations capture evaluator consensus
Consensus is measured via pairwise Pearson correlations averaged per maxim, and ranking alignment via Kendall and Spearman coefficients (Section 3.3).
domain assumption Weak supervision label model assumptions hold for voter independence
FlyingSquid-style label modeling assumes voters are conditionally independent noise sources; the paper acknowledges this by testing a dependency-aware variant (Section 3.3).

how reviews work

0 comments

Cite this review

Pith. "Pith review of DS@GT ARC at Touch\'e: Large Language Models for Retrieval-Augmented Debate." pith.science (2026). https://pith.science/paper/5A5G5YPT

@misc{pith2026260808143,
  author       = {Pith},
  title        = {Pith review of: DS@GT ARC at Touch\'e: Large Language Models for Retrieval-Augmented Debate},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5A5G5YPT}},
  note         = {Machine review of arXiv:2608.08143}
}

read the original abstract

We extend the DS@GT ARC working-note submission to the Touch\'e 2025 Retrieval-Augmented Debate task. The task has two subtasks: generating the next utterance in a simulated debate, and evaluating debate responses according to the Gricean maxims of Quantity, Quality, Relation, and Manner. The DS@GT ARC submission consisted of six leading LLMs from three providers through a retrieval-augmented prompting pipeline. We summarize the results from the working paper and explore whether multi-LLM evaluator agreement is a reliable proxy for official evaluation performance. The analysis shows that frontier LLM systems are strong response generators, and as evaluators they agree strongly within model families. However this consensus does not reliably track the official evaluation target, with the largest gap on the Quality maxim. The accompanying source code for this paper is located at https://github.com/dsgt-arc/touche-2025-rad and https://github.com/dsgt-arc/touche-2025-rad-analysis.

Figures

Figures reproduced from arXiv: 2608.08143 by the authors.

**Figure 1.** An example debate exchange from the organizer-released corpus (topic 7, first system turn of our gpt-4.1 run). The topic, simulated user turn, and retrieved arguments are provided by the organizers while the system generates the response. Official annotators marked all four maxims unsatisfied because the system argues the same stance as the user, while all six frontier LLM judges marked all four satisfied [PITH_FUL… view at source ↗

**Figure 2.** The function signature of the responding endpoint the GenIRSim system expects for a valid simulation. The messages are passed in OpenAI-compatible chat completion format, with the roles of "user" and "assistant" [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** The function interface for the evaluation portion of the GenIRSim API. A separate request is made for quantity, quality, manner, and relation. 3.1 Debate Response The debate response is split into a retrieval and generation phase. In the retrieval phase, we obtain the top ten documents from the argument search system as evidence for a response. The search system utilizes organizer-hosted Elasticsearch and Stella emb… view at source ↗

Figures from the paper (4 more)

**Figure 4.** Figure 4: The prompt is used to generate a debate response in the first subtask. Evidence from the claims database is formatted as YAML at the start of the prompt, while the context of the entire argument is serialized at the end of the prompt. ## issue. {{ issue }} ## argument.…

**Figure 5.** Figure 5: The prompt is used to generate all measures of the requested debate. The call to the LLM leverages structured output through the chat completion API provided by OpenRouter [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]

**Figure 6.** Figure 6: Average subtask 1 maxim scores by response-generation system, averaged over released judge scores on the organizer-released corpus; this differs from the official Subtask 1 metric. for Manner, suggesting that dependency-aware modeling discounts some redundancy in agre…

**Figure 7.** Figure 7: Mean pairwise judge-score correlation across Gricean dimensions. Frontier LLM judges show strong agreement, especially within model families, while SINAI LLaMA8B judges are less correlated with the frontier judge pool. that sense, Quality overlaps with the other Grice…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

13 extracted references · 9 canonical work pages

[1]

In: Kamps, J., Goeuriot, L., Crestani, F., Maistro, M., Joho, H., Davis, B., Gurrin, C., Kruschwitz, U., Caputo, A

Fröbe, M., Wiegmann, M., Kolyada, N., Grahm, B., Elstner, T., Loebe, F., Hagen, M., Stein, B., Potthast, M.: Continuous Integration for Reproducible Shared Tasks with TIRA.io. In: Kamps, J., Goeuriot, L., Crestani, F., Maistro, M., Joho, H., Davis, B., Gurrin, C., Kruschwitz, U., Caputo, A. (eds.) Advances in Information Retrieval. 45th European Conferenc...

2023
[2]

In: III, H.D., Singh, A

Fu, D., Chen, M., Sala, F., Hooper, S., Fatahalian, K., Re, C.: Fast and three- rious: Speeding up weak supervision with triplet methods. In: III, H.D., Singh, A. (eds.) Proceedings of the 37th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 119, pp. 3280–3291. PMLR (13–18 Jul 2020), https://proceedings.mlr.pre...

work page 2020
[3]

In: Speech acts, pp

Grice, H.P.: Logic and conversation. In: Speech acts, pp. 41–58. Brill (1975)

work page 1975
[4]

In: Bouamor, H., Pino, J., Bali, K

Hagström, L., Saynova, D., Norlund, T., Johansson, M., Johansson, R.: The Ef- fect of Scaling, Retrieval Augmentation and Form on the Factual Consistency of Language Models. In: Bouamor, H., Pino, J., Bali, K. (eds.) Proceedings of the 2023 Conference on Empirical Methods in Natural Language Process- ing. pp. 5457–5476. Association for Computational Lingu...

work page doi:10.18653/v1/2023.emnlp-main.332 2023
[5]

In: Salakhutdinov, R., Kolter, Z., Heller, K., Weller, A., Oliver, N., Scarlett, J., Berkenkamp, F

Khan, A., Hughes, J., Valentine, D., Ruis, L., Sachan, K., Radhakrishnan, A., Grefenstette, E., Bowman, S.R., Rocktäschel, T., Perez, E.: Debating with More Persuasive LLMs Leads to More Truthful Answers. In: Salakhutdinov, R., Kolter, Z., Heller, K., Weller, A., Oliver, N., Scarlett, J., Berkenkamp, F. (eds.) Pro- ceedings of the 41st International Confe...

arXiv doi:10.48550/arxiv.2402.06782 2024
[6]

In: Carrillo-de-Albornoz, J., Gonzalo, J., Plaza, L., García Seco de Herrera, A., Mothe, J., Piroi, F., Rosso, P., Spina, D., Faggioli, G., Ferro, N

Kiesel, J., Çöltekin, Ç., Gohsen, M., Heineking, S., Heinrich, M., Fröbe, M., Ha- gen, T., Aliannejadi, M., Anand, S., Erjavec, T., Hagen, M., Kopp, M., Ljubešić, N., Meden, K., Mirzakhmedova, N., Morkevičius, V., Scells, H., Wolter, M., Zelch, I., Potthast, M., Stein, B.: Overview of Touché 2025: Argumentation Systems. In: Carrillo-de-Albornoz, J., Gonza...

work page 2025
[7]

In: Goeuriot, L., Mulhem, P., Quénot, G., Schwab, D., Di Nunzio, G.M., Soulier, L., Galuščáková, P., García Seco de Herrera, A., Faggioli, G., Ferro, N

Kiesel, J., Gohsen, M., Mirzakhmedova, N., Hagen, M., Stein, B.: Who Will Evalu- ate the Evaluators? Exploring the Gen-IR User Simulation Space. In: Goeuriot, L., Mulhem, P., Quénot, G., Schwab, D., Di Nunzio, G.M., Soulier, L., Galuščáková, P., García Seco de Herrera, A., Faggioli, G., Ferro, N. (eds.) Experimental IR Meets Multilinguality, Multimodality...

work page 2024
[8]

In: Ku, L.W., Martins, A., Srikumar, V

Liu, X., Liu, P., He, H.: An Empirical Analysis on Large Language Models in De- bate Evaluation. In: Ku, L.W., Martins, A., Srikumar, V. (eds.) Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Vol- ume 2: Short Papers). pp. 470–487. Association for Computational Linguistics, Bangkok, Thailand (Aug 2024). https://doi...

work page doi:10.18653/v1/2024.acl-short.44 2024

Show all 13 references

[9]

In: Working Notes of the Conference and Labs of the Evaluation Forum (CLEF 2025)

Miyaguchi, A., Johnston, C., Potdar, A.: DS@GT at Touché: Large Language Models for Retrieval-Augmented Debate. In: Working Notes of the Conference and Labs of the Evaluation Forum (CLEF 2025). CEUR Workshop Proceedings, vol. 4038, pp. 4641–4649. CEUR-WS.org, Madrid, Spain (20...

2025
[10]

https://doi.org/10.48550/arXiv.2506.18203, https://arxiv.org/abs/2506.18203

Saad-Falcon, J., Buchanan, E.K., Chen, M.F., Huang, T.H., McLaughlin, B., Bhathal, T., Zhu, S., Athiwaratkun, B., Sala, F., Linderman, S., Mirhoseini, A., Ré, C.: Shrinking the generation-verification gap with weak verifiers (Jun 2025). https://doi.org/10.48550/arXiv.2506.1820...

2025 doi arXiv
[11]

In: Merlo, P., Tiedemann, J., Tsarfaty, R

Skitalinskaya, G., Klaff, J., Wachsmuth, H.: Learning From Revisions: Quality Assessment of Claims in Argumentation at Scale. In: Merlo, P., Tiedemann, J., Tsarfaty, R. (eds.) Proceedings of the 16th Conference of the European Chapter of the Association for Computational Lingu...

2021
[12]

In: Working Notes of the Confer- ence and Labs of the Evaluation Forum (CLEF 2025)

Vallecillo-Rodríguez, M.E., Martín-Valdivia, M.T., Montejo-Ráez, A.: SINAI at Touché: From Generation to Evaluation through Multistep and Comparative Prompting for Retrieval-Augmented Debate. In: Working Notes of the Confer- ence and Labs of the Evaluation Forum (CLEF 2025). C...

2025
[13]

https://doi.org/10.48550/arXiv.2412.19048, https: //arxiv.org/abs/2412.19048

Zhang, D., Li, J., Zeng, Z., Wang, F.: Jasper and Stella: Distillation of SOTA embedding models (Dec 2024). https://doi.org/10.48550/arXiv.2412.19048, https: //arxiv.org/abs/2412.19048

2024 doi arXiv

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.

[1] [1]

In: Kamps, J., Goeuriot, L., Crestani, F., Maistro, M., Joho, H., Davis, B., Gurrin, C., Kruschwitz, U., Caputo, A

Fröbe, M., Wiegmann, M., Kolyada, N., Grahm, B., Elstner, T., Loebe, F., Hagen, M., Stein, B., Potthast, M.: Continuous Integration for Reproducible Shared Tasks with TIRA.io. In: Kamps, J., Goeuriot, L., Crestani, F., Maistro, M., Joho, H., Davis, B., Gurrin, C., Kruschwitz, U., Caputo, A. (eds.) Advances in Information Retrieval. 45th European Conferenc...

2023

[2] [2]

In: III, H.D., Singh, A

Fu, D., Chen, M., Sala, F., Hooper, S., Fatahalian, K., Re, C.: Fast and three- rious: Speeding up weak supervision with triplet methods. In: III, H.D., Singh, A. (eds.) Proceedings of the 37th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 119, pp. 3280–3291. PMLR (13–18 Jul 2020), https://proceedings.mlr.pre...

work page 2020

[3] [3]

In: Speech acts, pp

Grice, H.P.: Logic and conversation. In: Speech acts, pp. 41–58. Brill (1975)

work page 1975

[4] [4]

In: Bouamor, H., Pino, J., Bali, K

Hagström, L., Saynova, D., Norlund, T., Johansson, M., Johansson, R.: The Ef- fect of Scaling, Retrieval Augmentation and Form on the Factual Consistency of Language Models. In: Bouamor, H., Pino, J., Bali, K. (eds.) Proceedings of the 2023 Conference on Empirical Methods in Natural Language Process- ing. pp. 5457–5476. Association for Computational Lingu...

work page doi:10.18653/v1/2023.emnlp-main.332 2023

[5] [5]

In: Salakhutdinov, R., Kolter, Z., Heller, K., Weller, A., Oliver, N., Scarlett, J., Berkenkamp, F

Khan, A., Hughes, J., Valentine, D., Ruis, L., Sachan, K., Radhakrishnan, A., Grefenstette, E., Bowman, S.R., Rocktäschel, T., Perez, E.: Debating with More Persuasive LLMs Leads to More Truthful Answers. In: Salakhutdinov, R., Kolter, Z., Heller, K., Weller, A., Oliver, N., Scarlett, J., Berkenkamp, F. (eds.) Pro- ceedings of the 41st International Confe...

arXiv doi:10.48550/arxiv.2402.06782 2024

[6] [6]

In: Carrillo-de-Albornoz, J., Gonzalo, J., Plaza, L., García Seco de Herrera, A., Mothe, J., Piroi, F., Rosso, P., Spina, D., Faggioli, G., Ferro, N

Kiesel, J., Çöltekin, Ç., Gohsen, M., Heineking, S., Heinrich, M., Fröbe, M., Ha- gen, T., Aliannejadi, M., Anand, S., Erjavec, T., Hagen, M., Kopp, M., Ljubešić, N., Meden, K., Mirzakhmedova, N., Morkevičius, V., Scells, H., Wolter, M., Zelch, I., Potthast, M., Stein, B.: Overview of Touché 2025: Argumentation Systems. In: Carrillo-de-Albornoz, J., Gonza...

work page 2025

[7] [7]

In: Goeuriot, L., Mulhem, P., Quénot, G., Schwab, D., Di Nunzio, G.M., Soulier, L., Galuščáková, P., García Seco de Herrera, A., Faggioli, G., Ferro, N

Kiesel, J., Gohsen, M., Mirzakhmedova, N., Hagen, M., Stein, B.: Who Will Evalu- ate the Evaluators? Exploring the Gen-IR User Simulation Space. In: Goeuriot, L., Mulhem, P., Quénot, G., Schwab, D., Di Nunzio, G.M., Soulier, L., Galuščáková, P., García Seco de Herrera, A., Faggioli, G., Ferro, N. (eds.) Experimental IR Meets Multilinguality, Multimodality...

work page 2024

[8] [8]

In: Ku, L.W., Martins, A., Srikumar, V

Liu, X., Liu, P., He, H.: An Empirical Analysis on Large Language Models in De- bate Evaluation. In: Ku, L.W., Martins, A., Srikumar, V. (eds.) Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Vol- ume 2: Short Papers). pp. 470–487. Association for Computational Linguistics, Bangkok, Thailand (Aug 2024). https://doi...

work page doi:10.18653/v1/2024.acl-short.44 2024