DRE: An Effective Dual-Refined Method for Integrating Small and Large Language Models in Open-Domain Dialogue Evaluation

Bohao Yang; Chenghua Lin; Chen Tang; Haoteng Tang; Kun Zhao; Liang Zhan; Siyuan Dai

REVIEW 4 major objections 4 minor 17 references

DRE: An Effective Dual-Refined Method for Integrating Small and Large Language Models in Open-Domain Dialogue Evaluation

T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read DRE claims that a small model's distance–probability signals, injected into an LLM prompt and then multiplied back into its score, align open-domain dialogue evaluation with human judgment better than any compared metric.

desk verdict Useful empirical recipe for dialogue evaluation, but the method as written is not reproducible until the score equation and disentanglement step are fixed. read the letter →

arxiv 2506.04516 v1 pith:WPQCAMLR submitted 2025-06-04 cs.CL

Kun Zhao , Bohao Yang , Chen Tang , Siyuan Dai , Haoteng Tang , Chenghua Lin , Liang Zhan This is my paper

classification cs.CL

keywords open-domaindialogueevaluationsmalllanguagemodelslargedualrefinementDREcontrastivelearningadversarialnegativeresponsesone-to-manyproblem

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that a small, task-specific language model can correct the blind spots of a large language model when judging open-domain dialogue, where a single context admits many valid responses and adversarial negatives reuse context words. It introduces SLIDE, which mixes the small model's score with the large model's score, and DRE, which first feeds the small model's distance and probability signals into the large model's prompt and then recalibrates the large model's numeric score with a coefficient built from those same signals. Across DailyDialog++, PersonaChat, and TopicalChat, DRE reports the strongest Pearson correlations with human ratings among all compared metrics, reaching 0.753, 0.752, and 0.747 respectively, above the best LLM-only baseline. If true, this gives a practical recipe for making automatic dialogue evaluation more reliable without relying on human annotation at evaluation time.

What carries the argument

The load-bearing mechanism is the contrastively trained small model and its robust/non-robust decomposition of response embeddings. A Sentence-Transformer encoder (DistilBERT backbone) is trained with a triplet contrastive loss plus disentanglement and classification losses so that each response embedding splits into a robust part, used for scoring, and a non-robust part, discarded as noise. From the robust part the model computes a normalized cosine distance $s_d$ and a classification probability $s_p$, whose combination $1 - s_d + s_p$ is the SLM's score. DRE deploys this same signal twice: once as auxiliary information inside the LLM prompt (interior refinement) and once as a multiplicative coefficient $c = (1 - s_d + s_p) s_{\mathrm{Inf}}$ applied to the LLM score (exterior refinement). This two-fold use of one small-model signal is what the paper argues carries the performance gain.

What would settle it

Re-train the SLM on DailyDialog++ following the stated losses and check whether the learned robust/non-robust representations actually satisfy the three separation conditions in Eq. 14 and raise classification accuracy above the no-disentanglement model. If the split cannot be realized, or if removing it leaves DRE's Pearson correlations approximately unchanged, then the disentangled signals are not the source of the reported improvement.

Watch

Extended reading notes

Core claim

DRE's central claim is that one small model's judgment, expressed as two numbers, can be used twice to improve an LLM evaluator. After contrastive training, the SLM outputs a normalized cosine distance $s_d$ between the context and the robust part of the response embedding, plus a probability $s_p$ that the response is positive. These are combined into an SLM score $1 - s_d + s_p$. In DRE's interior refinement, those values are placed in the LLM prompt, and the LLM returns an initial score $\mathrm{Score}_{\mathrm{LLM}}$ and an influence score $s_{\mathrm{Inf}}$; in the exterior refinement, the final score is $(1 - s_d + s_p) s_{\mathrm{Inf}} \mathrm{Score}_{\mathrm{LLM}}$. The paper reports that this two-stage refinement outperforms SLIDE, LLM-only evaluators, and traditional metrics on all three benchmarks, with the best Pearson correlations 0.753 on DailyDialog++ (Claude3), 0.752 on PersonaChat (Gemini), and 0.747 on TopicalChat (Claude3).

Load-bearing premise

The method assumes a response can be split into a reliable part and an unreliable part, and that only the reliable part should drive the small model's distance and probability signals; the paper never specifies how this split is trained or computed, so the exact signals that power DRE may not be reproducible.

Editorial extensions

If this is right

If DRE is correct, the best LLM-only evaluator on these benchmarks (LLM-Chiang, best Pearson 0.677–0.678) is not the ceiling: adding a cheap SLM signal lifts Pearson correlation to 0.747–0.753 on the same test sets.
The ablation pattern suggests interior refinement alone can hurt (In-DRE falls below Non-DRE in several settings), while exterior refinement drives most of the gain and the two stages together are best; downstream users should expect prompt injection to matter mainly as a complement to score recalibration.
The method is not tied to one LLM: DRE improves GPT-3.5, Claude3, Gemini, Llama3.3-70B, Qwen2.5-70B, and DeepSeek-R1-70B, and an SLM trained only on DailyDialog++ transfers to PersonaChat and TopicalChat without retraining.
The classification split on DailyDialog++—SLM 91.05% overall vs GPT-4 88.91%, with the SLM stronger on positives (91.83% vs 80.43%) and GPT-4 stronger on negatives (97.40% vs 90.28%)—is the empirical foundation for the complementary-strengths story.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A natural extension the paper does not test: the same interior/exterior refinement could be applied to other open-ended evaluation tasks with one-to-many structure, such as story generation, creative summarization, or assistant response selection, by training the SLM on task-specific positive and adversarial pairs.
Because exterior refinement alone nearly matches the full method in several rows, a simpler variant that skips prompt injection and recalibrates the LLM score with a learned SLM coefficient may capture most of the benefit at lower API cost; that hypothesis is directly testable.
The PersonaChat and TopicalChat evaluation sets were generated by GPT-4 with self-refine, and the paper reports 98% human approval of generated responses; if those adversarial negatives are systematically easier to distinguish than DailyDialog++'s human-written ones, cross-dataset gains could vary, and a per-dataset difficulty analysis would settle whether the improvement is constant across data s

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: DRE reports a real-looking gain in correlation with human judgments (roughly 0.63–0.68 up to 0.70–0.75) and the idea of having an SLM inform an LLM judge through two separate pressure points is worth a look. But the paper as submitted cannot be reproduced from its equations: the exterior-refinement coefficient defined in Eq 22 drops out of the final score in Eq 23, and the robust/non-robust embedding decomposition is never actually specified.

What is new and good: the dual-refinement pipeline is new relative to G-EVAL and LLM-Chiang. The observation that SLMs and LLMs have complementary strengths—SLMs better on positive responses, LLMs on negatives—is supported by Table 3, and the augmented PersonaChat/TopicalChat datasets are a concrete resource. The authors evaluate six LLMs, including open-weight ones, and the DRE rows consistently beat LLM-only baselines. That is a genuine empirical contribution, assuming the numbers survive re-evaluation.

The load-bearing flaws are in the method description. Eq 22 defines c = sc * sInf, and the text says the product of sc and sInf constrains the LLM score, but Eq 23 gives Score = sc * ScoreLLM, silently dropping sInf. If that is literal, the influence score the LLM was asked to produce has no effect on the final score, and DRE collapses into SLIDE-style rescaling. If it is a typo, the algorithm is under-specified. Either way Table 4 is not derivable from the equations as printed. Nearly as serious: the disentanglement of response embeddings into robust and non-robust parts is invoked in Eq 13 and Eq 17, but the paper never says how sep(hr) is implemented or trained. The loss terms d1, d2, d3 presuppose hpr, hpn, har, han but no architecture is given.

The evaluation also has leakage concerns. The 0.5 threshold in Eq 20 is selected on the DailyDialog++ test set (Figure 6), and the integration rule itself is motivated by observations from Table 3 on that same test set. dmin/dmax normalization in Eq 18 appears to be computed over the whole dataset, which can leak test-set statistics. These are not minor nits; they are reproducibility blockers. There are also no confidence intervals or significance tests, which matters for comparing 0.753 vs 0.677.

I don't think this is a rejection on the merits. The central idea—task-specific SLM signals can correct LLM bias in open-ended evaluation—is coherent, and the consistency of the gains across six LLMs and three datasets is encouraging. But the paper needs a major revision before its numbers can be trusted: fix the equations, specify the disentanglement architecture, move threshold and normalization choices to validation or disclose them as test-set-informed, and release code/data. A serious editor should send this to referees; with proper rewriting it could be a useful addition to the NLG evaluation literature.

Referee Report

4 major / 4 minor

Summary. The paper proposes two hybrid SLM-LLM metrics for open-domain dialogue evaluation. SLIDE trains a small encoder (Sentence-Transformer/DistilBERT) with contrastive learning plus a claimed disentanglement of robust/non-robust embeddings, then combines an SLM score (from cosine distance and classification probability) with an LLM score via a hand-written selection rule. DRE extends this with an interior refinement (SLM outputs and interpretation text injected into the LLM prompt, yielding ScoreLLM and an influence score sInf) and an exterior refinement (a coefficient c = sc * sInf that should rescale ScoreLLM). Experiments on DailyDialog++, PersonaChat, and TopicalChat report Pearson/Spearman correlations with human judgments, with DRE rows (e.g., DRE(Claude3) 0.753 on DailyDialog++) generally above LLM-only baselines and SLIDE. The central claim is that DRE achieves state-of-the-art alignment with human judgment in open-domain dialogue evaluation.

Significance. The idea of using a task-specific small model to correct and constrain LLM evaluations is plausible and practically attractive, especially given the cost asymmetry between SLM fine-tuning and LLM API calls. The paper's strengths are its breadth across six LLM backbones, the cross-dataset generalization experiment in Section 5.5, and the claim that datasets will be released. If the reported gains are real, the two-stage refinement is a useful contribution to dialogue evaluation. However, the significance is currently limited by three load-bearing gaps: Eq. (23) is inconsistent with the stated dual-refinement mechanism, the disentanglement operator that produces the core SLM signals is not specified, and key thresholds and normalization statistics appear to be chosen on test data. These issues are fixable in principle, but they must be addressed before the empirical claims can be accepted.

major comments (4)

[Section 3.4.2, Eqs. (22)-(23)] Eq. (22) defines c = sc * sInf, but Eq. (23) computes Score = sc * ScoreLLM, silently dropping sInf. As written, the "Influence Degree" produced by the interior refinement stage does not enter the exterior refinement, so the final score is not the promised product of sc and sInf. If Eq. (23) is a typo for Score = c * ScoreLLM, then the published equations do not let a reader reproduce the DRE rows of Table 4; if it is literal, DRE collapses to a static rescaling. Either way this is a central specification error. Additionally, sc = 1 - sd + sp is not bounded above by 1 (sd can be near 0 and sp near 1), so Eq. (23) can push final scores beyond the 0-5 scale; the paper gives no clipping or renormalization rule, which makes cross-metric and cross-model comparability in Table 4 unclear.
[Section 3.3, Eqs. (13)-(15) and Eq. (17)] The disentanglement mechanism is undefined. The loss functions in Eq. (13) and the evaluation formula in Eq. (17) presuppose vectors hpr, hpn, har, han and a separator sep(hr), but the paper never specifies the architecture, parameterization, or training objective that realizes this decomposition. Since only the "robust" embedding is used for the distance sd and the probability sp in Eq. (18), and since those two quantities drive both SLIDE and DRE, the central claims of the paper cannot be reproduced or verified without this specification. Please provide the network definition, the exact loss terms, and the training procedure, or remove the dependence on the unspecified split.
[Section 5.2, Section 5.3, and Eq. (18)] There is a serious risk of test-set leakage and circular design. Section 5.2 selects the 0.5 threshold in Eq. (20) by running experiments on the DailyDialog++ set (Figure 6), and Section 5.3's Table 3, which reports positive/negative accuracy on the same human-annotated test set, is used to justify the SLM-for-positive/LLM-for-negative split encoded in Eq. (20). Moreover, Eq. (18) defines dmin and dmax as "the smallest distance" and "the biggest distance" without stating whether these extrema are computed on the training, validation, or test portion; if test-set extrema are used, the normalization leaks information about the test distribution. The generalization experiment in Table 5 does not repair this, because the threshold and normalization choices were still fixed using the DailyDialog++ test set. Please move all threshold selection and normalization statistics to training/validation data, or report the full procedure with explicit data splits.
[Table 4 and Section 5.4] The main SOTA claims are based on correlation differences of roughly 0.01-0.05 (e.g., DRE(Claude3) Pearson 0.753 vs SLIDE(Gemini) 0.760 and vs DRE(Gemini) 0.748), but the paper reports no significance tests, confidence intervals, or multiple-comparison correction across the three LLMs and two correlation metrics. With only 600 evaluation samples per dataset, these differences may be within sampling variation. The authors should report bootstrap confidence intervals or a paired test (e.g., Williams or permutation test) for the headline comparisons before claiming state-of-the-art performance.

minor comments (4)

[Section 3.3] There are several typos and notational errors: "objecitive" should be "objective"; the sentence "hi is the hidden state of the hi" is incomplete; and in Eq. (12) the expression ||hc_i - xp_i|| - ||hc_i - xa_i|| should refer to hp_i and ha_i, not xp_i and xa_i.
[Figure 2 and Section 3.4.2] The prompt example shows "Cosine Similarity Distance: 0.569639659208593" and then "SLM_d: 1.569639659208593", while the text defines sd as a normalized cosine distance and ScoreSLM as 1 - sd + sp. The relationship between the unnormalized distance, the normalized sd, and the displayed value should be clarified, and the terms "Cosine Similarity Distance", "SLM_d", and "ScoreSLM" should be used consistently.
[Section 6 and Section 3.4.2] The naming of the second LLM output is inconsistent: Section 3.4.2 calls it "Influence Degree" and writes sInf, while the conclusion calls it "Inference score" and the text also uses "ScoreInf". Please unify the terminology.
[Section 4.1, Table 2] The statistics for TopicalChat and PersonaChat list different numbers of adversarial negative responses than positive responses in the training split (e.g., 8,050 vs 16,678), but the text says each context has five positive and five adversarial negative responses. The table should explain the discrepancy or be corrected.

Circularity Check

1 steps flagged · score 6.0 of 10

The hybrid selection rule and 0.5 threshold are fitted to the evaluation benchmarks whose test correlations are then reported as SOTA; Eq. 23 additionally omits the influence score, leaving DRE underspecified as written.

fitted input called prediction [Section 5.2 (threshold sweep), Section 5.3 (Table 3) -> Eq. (20); results in Section 5.4, Table 4]
"To analyze how different thresholds affect accuracy, we conducted experiments using multiple threshold values and found that 0.5 yielded the most consistent and optimal performance. ... Based on these insights, we propose a hybrid approach that combines the SLM, a task-specific model, with LLMs for comprehensive dialogue evaluation."

The 0.5 threshold used in Eq. (20) is chosen by sweeping classification accuracy on the DailyDialog++ dataset (Section 5.2), and the hybrid selection rule is motivated in Section 5.3 by measuring, on the same benchmark, that the SLM is stronger on positives while LLMs are stronger on negatives. Table 4 then reports the SLIDE/DRE correlations on those same test splits as the central state-of-the-art claim. The key integration rule and threshold are therefore fitted to the evaluation data rather than derived from held-out evidence; the headline gain is partly an artifact of selecting the rule on the test set. This is not an equation-level identity, but it is a fitted-input-called-prediction circularity.

full rationale

The central equations are not definitionally identical to the reported human correlations: ScoreSLM and sc are computed from SLM cosine distance and classifier probability, ScoreLLM comes from an LLM prompt, and no parameter is optimized directly against human Pearson scores. The main circular component is the reuse of the evaluation benchmark for design: Eq. (20)'s threshold is tuned on DailyDialog++ (Section 5.2), and the complementary-strength insight motivating Eq. (20) is measured on that same dataset (Section 5.3), after which Table 4 reports correlations on the same test splits. This partially inflates the claimed SOTA result. Some independence is provided by the cross-dataset generalization experiments in Section 5.5, which keeps the score from reaching 8-10. Self-citation is present (e.g., CMN, Zhao et al. 2023) but is not load-bearing. Separately, and non-circularly, Eq. (22)-(23) are inconsistent: Eq. (22) defines c = sc * sInf and says the product constrains ScoreLLM, but Eq. (23) gives Score = sc * ScoreLLM, silently dropping sInf. If Eq. (23) is literal, DRE collapses to a static reweighting; if it is a typo, the published algorithm is not reproducible without guessing. Eq. (20)'s second branch also appears to test ScoreLLM < 0.5 where the surrounding text says the threshold applies to ScoreSLM, and sc = 1 - sd + sp can exceed 1, so final scores can leave the 0-5 scale with no stated clipping. These are correctness/reproducibility concerns, not circularity, and do not raise the circularity score further.

Assumptions & free parameters 4 free parameters · 4 assumptions · 1 invented entities

The method rests on several empirical assumptions about embedding geometry and model complementarity, plus an underspecified disentanglement step. The free parameters are few but some are tuned on the test sets. No new theoretical entities are introduced beyond the robust/non-robust decomposition, which lacks independent evidence.

free parameters (4)

contrastive margin = 0.5
Set to 0.5 in Eq 12; no sensitivity analysis is reported.
SLIDE threshold = 0.5
Chosen in Eq 20 after experiments on the DailyDialog++ test set (Figure 6); tuning on the test set is post hoc.
dmin/dmax normalization extrema = computed over 'all examples' in Eq 18
If computed on the evaluation set, the normalized distance sd uses test-set statistics, a transductive leak.
disentanglement loss coefficients z1, z2, z3 = 0
Set to 0 in Eq 13 to force divergence; no ablation of these choices.

assumptions (4)

domain assumption Positive responses are closer to their context in embedding space than negative responses, and sp is higher for positives.
Used in Eq 19 to define ScoreSLM = 1 - sd + sp and in DRE's coefficient sc; this is an empirical claim, not proven.
ad hoc to paper SLMs are better at identifying positive responses and LLMs are better at identifying negative responses.
Observed on the test set (Table 3) and used to design the selection rule in Eq 20 and the DRE refinement; this makes the design dependent on the evaluation set.
domain assumption GPT-4-generated positive and adversarial responses for PersonaChat and TopicalChat are valid training labels.
Used to augment datasets in Section 4.1; the validity rests on GPT-4 quality plus self-refine, human-checked on 1,200 samples.
domain assumption The robust/non-robust embedding split exists and is learnable.
The losses in Eq 13 presuppose the availability of hpr, hpn, har, han without specifying the separation architecture.

invented entities (1)

Robust and non-robust embedding decomposition
purpose: Used to compute robust cosine distance and classifier probability for SLM scoring
The paper does not specify how this decomposition is computed; it is an internal postulated representation with no external validation.

how reviews work

0 comments

Cite this review

Pith. "Pith review of DRE: An Effective Dual-Refined Method for Integrating Small and Large Language Models in Open-Domain Dialogue Evaluation." pith.science (2026). https://pith.science/paper/WPQCAMLR

@misc{pith2026250604516,
  author       = {Pith},
  title        = {Pith review of: DRE: An Effective Dual-Refined Method for Integrating Small and Large Language Models in Open-Domain Dialogue Evaluation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WPQCAMLR}},
  note         = {Machine review of arXiv:2506.04516}
}

read the original abstract

Large Language Models (LLMs) excel at many tasks but struggle with ambiguous scenarios where multiple valid responses exist, often yielding unreliable results. Conversely, Small Language Models (SLMs) demonstrate robustness in such scenarios but are susceptible to misleading or adversarial inputs. We observed that LLMs handle negative examples effectively, while SLMs excel with positive examples. To leverage their complementary strengths, we introduce SLIDE (Small and Large Integrated for Dialogue Evaluation), a method integrating SLMs and LLMs via adaptive weighting. Building on SLIDE, we further propose a Dual-Refinement Evaluation (DRE) method to enhance SLM-LLM integration: (1) SLM-generated insights guide the LLM to produce initial evaluations; (2) SLM-derived adjustments refine the LLM's scores for improved accuracy. Experiments demonstrate that DRE outperforms existing methods, showing stronger alignment with human judgment across diverse benchmarks. This work illustrates how combining small and large models can yield more reliable evaluation tools, particularly for open-ended tasks such as dialogue evaluation.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

17 extracted references · 7 canonical work pages

[2]

arXiv preprint arXiv:2309.16609

Qwen technical report. arXiv preprint arXiv:2309.16609. Banerjee, Satanjeev and Alon Lavie

arXiv
[4]

GPTScore: Evaluate as you desire. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 6556–6576, Association for Computational Linguistics, Mexico City, Mexico. Gopalakrishnan, Karthik, Behnam Hedayatnia, Qinlang Chen, Anna Gottardi...

work page 2024
[7]

In Proceedings of the 5th Workshop on NLP for Conversational AI (NLP4ConvAI 2023), pages 47–58, Association for Computational Linguistics, Toronto, Canada

LLM-eval: Unified multi-dimensional automatic evaluation for open-domain conversations with large language models. In Proceedings of the 5th Workshop on NLP for Conversational AI (NLP4ConvAI 2023), pages 47–58, Association for Computational Linguistics, Toronto, Canada. Liu, Yang, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu

work page 2023
[8]

In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 2511–2522, Association for Computational Linguistics, Singapore

G-eval: NLG evaluation using gpt-4 with better human alignment. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 2511–2522, Association for Computational Linguistics, Singapore. Loakman, Tyler, Aaron Maladry, and Chenghua Lin

work page 2023
[9]

In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 6676–6689

The iron (ic) melting pot: Reviewing human evaluation in humour, irony and sarcasm generation. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 6676–6689. Madaan, Aman, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al

work page 2023
[10]

arXiv preprint arXiv:1908.10084

Sentence-bert: Sentence embeddings using siamese bert-networks. arXiv preprint arXiv:1908.10084. Sai, Ananya B., Akash Kumar Mohankumar, Siddharth Arora, and Mitesh M. Khapra

arXiv 1908
[11]

arXiv preprint arXiv:2312.11805

Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805. Touvron, Hugo, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al

arXiv
[12]

arXiv preprint arXiv:2302.13971

Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971. Wang, Jiaan, Yunlong Liang, Fandong Meng, Zengkui Sun, Haoxiang Shi, Zhixu Li, Jinan Xu, Jianfeng Qu, and Jie Zhou

arXiv

Show all 17 references

[14]

In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 10967–10982

Evaluating evaluation metrics: A framework for analyzing nlg evaluation metrics using measurement theory. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 10967–10982. Yang, Bohao, Chen Tang, and Chenghua Lin

2023
[15]

In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 11826–11830, IEEE

Improving medical dialogue generation with abstract meaning representations. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 11826–11830, IEEE. Ye, Kai, Tiejin Chen, Hua Wei, and Liang Zhan

2024
[2016]

CoRR, abs/1511.08198

Towards universal paraphrastic sentence embeddings. CoRR, abs/1511.08198. 25 Computational Linguistics Volume vv, Number nn Xiao, Ziang, Susu Zhang, Vivian Lai, and Q Vera Liao

arXiv
[2019]

ArXiv, abs/2308.11995

Topical-chat: Towards knowledge-grounded open-domain conversations. ArXiv, abs/2308.11995. Gu, Jia-Chen, Tianda Li, Quan Liu, Xiaodan Zhu, Zhenhua Ling, Zhiming Su, and Si Wei

arXiv
[2020]

ArXiv, abs/1904.09675

Bertscore: Evaluating text generation with bert. ArXiv, abs/1904.09675. Zhao, Kun, Bohao Yang, Chenghua Lin, Wenge Rong, Aline Villavicencio, and Xiaohui Cui

1904 arXiv
[2022]

Towards a unified multi-dimensional evaluator for text generation. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 2023–2038, Association for Computational Linguistics, Abu Dhabi, United Arab Emirates. 26

2022
[2023]

arXiv preprint arXiv:2303.08774

Gpt-4 technical report. arXiv preprint arXiv:2303.08774. Bai, Jinze, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al

arXiv
[2024]

arXiv preprint arXiv:2402.03456

Constrained multiview representation for self-supervised contrastive learning. arXiv preprint arXiv:2402.03456. Forgues, Gabriel, Joelle Pineau, Jean-Marie Larchevêque, and Réal Tremblay

arXiv
[2025]

arXiv preprint arXiv:2501.12948

Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948. Gupta, Prakhar, Yulia Tsvetkov, and Jeffrey P . Bigham

arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [2]

arXiv preprint arXiv:2309.16609

Qwen technical report. arXiv preprint arXiv:2309.16609. Banerjee, Satanjeev and Alon Lavie

arXiv

[2] [4]

GPTScore: Evaluate as you desire. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 6556–6576, Association for Computational Linguistics, Mexico City, Mexico. Gopalakrishnan, Karthik, Behnam Hedayatnia, Qinlang Chen, Anna Gottardi...

work page 2024

[3] [7]

In Proceedings of the 5th Workshop on NLP for Conversational AI (NLP4ConvAI 2023), pages 47–58, Association for Computational Linguistics, Toronto, Canada

LLM-eval: Unified multi-dimensional automatic evaluation for open-domain conversations with large language models. In Proceedings of the 5th Workshop on NLP for Conversational AI (NLP4ConvAI 2023), pages 47–58, Association for Computational Linguistics, Toronto, Canada. Liu, Yang, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu

work page 2023

[4] [8]

In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 2511–2522, Association for Computational Linguistics, Singapore

G-eval: NLG evaluation using gpt-4 with better human alignment. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 2511–2522, Association for Computational Linguistics, Singapore. Loakman, Tyler, Aaron Maladry, and Chenghua Lin

work page 2023

[5] [9]

In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 6676–6689

The iron (ic) melting pot: Reviewing human evaluation in humour, irony and sarcasm generation. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 6676–6689. Madaan, Aman, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al

work page 2023

[6] [10]

arXiv preprint arXiv:1908.10084

Sentence-bert: Sentence embeddings using siamese bert-networks. arXiv preprint arXiv:1908.10084. Sai, Ananya B., Akash Kumar Mohankumar, Siddharth Arora, and Mitesh M. Khapra

arXiv 1908

[7] [11]

arXiv preprint arXiv:2312.11805

Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805. Touvron, Hugo, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al

arXiv

[8] [12]

arXiv preprint arXiv:2302.13971

Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971. Wang, Jiaan, Yunlong Liang, Fandong Meng, Zengkui Sun, Haoxiang Shi, Zhixu Li, Jinan Xu, Jianfeng Qu, and Jie Zhou

arXiv