Pith. sign in

REVIEW 4 major objections 5 minor 31 references

Learning from Relevant Subgoals in Successful Dialogs using Iterative Training for Task-oriented Dialog Systems

T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that an iterative training loop which samples dialogs from the model and keeps only subgoals whose replacement flips a success evaluator can train a task-oriented dialog system to new state-of-the-art accuracy on…

desk verdict The paper presents a genuinely new subgoal-selection method that appears to help, but the SOTA claim needs variance bars and a guard against the success evaluator doing double duty. read the letter →

arxiv 2411.16305 v1 pith:WBHTJ5ZF submitted 2024-11-25 cs.CL cs.AIcs.LG

Magdalena Kaiser , Patrick Ernst , György Szarvas This is my paper · ORCID
classification cs.CLcs.AIcs.LG
keywords task-orienteddialoguesubgoalselectioniterativetrainingdistantsupervisiondirectpreferenceoptimizationcounterfactualreplacementMultiWOZ2.2end-to-enddialogsystems
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes SUIT, an iterative training procedure for task-oriented dialog systems that learns from sparse end-of-dialog success signals by singling out the subgoals that actually matter. The central claim is that a counterfactual selection rule—sampling several dialogs per user goal, then keeping only the belief states or action/response pairs whose replacement with a counterpart from an unsuccessful dialog flips the success evaluation—produces a small, high-quality training set. On MultiWOZ 2.2, training on the 2,166 selected subgoal samples outperforms training on all 31,586 successful-dialog samples, and two iterations of the loop set a new combined score of 105.02. This matters because end-to-end dialog systems usually receive feedback only at the end of a conversation, and naive use of successful dialogs cannot distinguish turns that cause success from turns that merely co-occur with it.

What carries the argument

The load-bearing mechanism is one-replacement-at-a-time counterfactual evaluation. A 'subgoal' is a turn-level piece of a dialog: either the predicted belief state or the joint action/response pair. For each successful sampled dialog, the method finds an unsuccessful dialog with the same user goal and swaps in the corresponding subgoal at the corresponding turn; if the standardized evaluation function flips the dialog from successful to unsuccessful, the original subgoal is kept as relevant training data. This gives a parameter-free credit assignment signal, and the outer loop alternates this selection with SFT or DPO training, sampling fresh dialogs from the newly trained model at each iteration.

What would settle it

Run the same iterative loop with the selected subgoals replaced by an equal-sized random sample of subgoals from successful dialogs; if the random set reproduces the 105.02 combined score, relevance selection is not the cause. Alternatively, rerun subgoal selection with a different goal-completion evaluator: if the selected subgoals change and the reported gain disappears, the result is an artifact of the specific evaluation function.

Watch

Extended reading notes

Core claim

The discovery is that dialog-level success can be decomposed into turn-level credit without human labels or learned reward models. For each user goal, SUIT samples multiple dialogs from the current model, evaluates each with the reference MultiWOZ success function, and applies distant supervision: for every successful dialog, it replaces one turn's belief state or action/response pair with the corresponding turn from an unsuccessful dialog sharing the same goal. If the modified dialog is judged unsuccessful, the original subgoal is marked relevant and becomes a positive training sample, and the replacing subgoal becomes a negative sample for DPO. This selection is what lets a much smaller training set outperform a much larger one, and the procedure can be repeated by sampling from the updated model. The paper reports that the best two-iteration model reaches INFORM 90.0, SUCCESS 87.1, and COMBINED 105.02 on MultiWOZ 2.2, improving on the previous state of the art.

Load-bearing premise

The selection procedure assumes the automatic success evaluator is a faithful oracle for whether the user's goal was met, and that each subgoal's contribution to success can be assessed by swapping it in isolation.

Editorial extensions

If this is right

  • Training on selected subgoals rather than all successful dialogs improves INFORM from 87.0 to 89.8 and SUCCESS from 79.4 to 84.0 in the first SFT iteration, despite using thousands fewer samples.
  • A second iteration can still help: SUCCESS rises from 84.0 to 87.1, showing that iterative resampling adds useful signal.
  • Both SFT and DPO benefit from the selected data, and the best configuration combines DPO-selected data with a final SFT pass.
  • Because the method uses the evaluation function and the model's own samples, it can be applied to any off-the-shelf language model without reward models or human preference annotations.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct extension is to use the selected subgoals as counterfactual explanations: the negative replacements found during selection are concrete examples of what would make the same dialog fail, which could support error analysis or user-facing justifications.
  • The one-turn-at-a-time rule likely underestimates joint dependencies; subgoals that only matter in combination would be missed, so a multi-turn replacement variant is a natural test.
  • The method's transfer to other datasets depends on having paired successful and unsuccessful dialogs for the same goal; on data without multiple rollouts, a user simulator or paraphrase-based goal perturbation would be needed to create the contrast.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces SUIT, an iterative training method for end-to-end task-oriented dialog systems. Starting from a Flan-T5-large model trained with supervised fine-tuning, SUIT samples multiple dialog variants per user goal, evaluates them with the Nekvinda and Dušek success function, and identifies 'relevant subgoals' by turn-wise counterfactual replacement: a subgoal from a successful dialog is kept as training data only if replacing it with the corresponding part of an unsuccessful dialog flips the success evaluation. These selected subgoals are then used for SFT or DPO training, and the process is repeated for up to two iterations. The authors report a COMBINED score of 105.02 on MultiWOZ 2.2, which they claim is a new state of the art, surpassing DIACTTOD's 104.4.

Significance. If the reported gains are robust, the work is a useful contribution: it offers a simple, model-agnostic way to extract dense training signals from sparse dialog-level success, and the ablation in Table 1—where training on the filtered subgoal set (2166 samples) outperforms training on all successful subgoals (31586 samples)—supports the core filtering idea. The method uses only an off-the-shelf LLM and the standard MultiWOZ evaluator, which makes it easy to reproduce in principle, and the paper explicitly names its limitations, including reliance on ground-truth goals and turn-wise replacement. However, the significance is currently limited by the coupling between the evaluator used for data selection and the evaluator used for final scoring, and by the absence of any variance or significance reporting for the headline SOTA claim.

major comments (4)
  1. [§2.3, Algorithm 1, Table 1] The distant-supervision rule in Algorithm 1 (step 3) selects a subgoal as relevant only if replacing it with a counterpart from an unsuccessful dialog flips the Nekvinda and Dušek success function from success to failure, and Table 1 then reports INFORM, SUCCESS, and COMBINED scores computed with the same function. This creates a selection-evaluation coupling: the training data are enriched for patterns that this particular rule-based checker treats as necessary, so the reported 0.62-point COMBINED improvement over DIACTTOD may reflect overfitting to the checker rather than improved goal completion. The manuscript should evaluate with a different success evaluator (for example, the MultiWOZ 2.1 or 2.4 evaluator, or a human evaluation on a sample) and should report results across at least three seeds to establish that the advantage is not an artifact of this coupling.
  2. [§3.1 and Appendix A.1] The stopping criterion is described as 'the COMBINED score is not increasing any further' (Section 3.1), and Appendix A.1 states 'As stopping criteria we use the COMBINED score,' but the paper never states whether this decision was made on a held-out development set or on the test set. If the test set was used to choose the number of iterations, the reported 105.02 is an optimistically selected number and the comparison with DIACTTOD, which was not selected in this way, is not apples-to-apples. Please clarify the stopping procedure and, if test-set information was used, rerun the comparison using a development set for model selection.
  3. [Table 1] No error bars, number of seeds, or significance tests are reported for any of the SUIT variants or the baselines. The headline margin over DIACTTOD is 0.62 COMBINED points (105.02 vs. 104.4), and the margin of SUIT2 (DPO-SFT) over SUIT1 (SFT) is only 0.37 points; these gaps are within the range of typical seed variance for MultiWOZ experiments reported elsewhere in the literature. Without a multi-seed comparison and a significance test, the claim 'SUIT reaches new state-of-the-art performance' is not supported. At minimum, please report mean and standard deviation over three to five seeds for the main configurations.
  4. [§2.3 and Table 7] The one-replacement-at-a-time rule assumes that subgoal contributions are independent and additive. Replacing a belief state while keeping the original action/response, or replacing an action/response while keeping the state, can create an internally incoherent turn; for example, the negative state in Table 7 swaps the train departure and destination while the response is unchanged. The evaluator may then flip for mechanical reasons—because the dialog is no longer a coherent whole—rather than because the original subgoal is causally important. The paper should provide evidence that the selected subgoals are coherent and genuinely relevant, for example by human annotation of a sample of selected subgoals or by an ablation that performs joint replacements of state and action/response.
minor comments (5)
  1. [§2.2 and Algorithm 1] The notation in Algorithm 1 is confusing: line (3) uses 'Do' and 'S′ot' without explicitly defining Do as an unsuccessful dialog, and the set-builder notation in line (1) is hard to parse. Please clarify the definitions of Do, S′ot, and Dc.
  2. [Table 3] The column headers of Table 3 are ambiguous: '# Dialogs' is followed by two numeric columns without clear labels, and the meaning of the six columns under '# Successful Dialogs per Goal' is not immediately clear from the caption. Please relabel the columns or explain them in the caption.
  3. [§6 (Limitations)] The limitations section acknowledges reliance on ground-truth goals and the turn-wise replacement design, but it does not mention the coupling between the success evaluator used for subgoal selection and the final evaluation metric. This coupling is a central risk of the approach and should be discussed as a limitation.
  4. [Appendix A.1, footnote 1] The phrase 'We enforce that the samples contain greedy generations' is vague. Please specify how the greedy generation is combined with the k=2 stochastic samples to produce the stated k^2+1 dialogs per user goal.
  5. [§7] In the Ethical Considerations section, 'GPT/LAMA models' should likely read 'GPT/LLaMA models'; as written it is unclear whether the authors mean the LAMA benchmark or something else.

Circularity Check

0 steps flagged · score 2.0 of 10

No formal circularity: SUIT's subgoal selection and final scores share the same external Nekvinda-Dušek evaluator, which mildly couples the training signal to the benchmark, but the SOTA claim still depends on held-out generalization and is not a self-citation or definitional reduction.

full rationale

SUIT is an empirical pipeline rather than a derivation, and its central claim—that training on distantly supervised 'relevant subgoals' improves MultiWOZ 2.2 COMBINED score—is not equivalent to its inputs by construction. Subgoals are selected from sampled training dialogs using the Nekvinda-Dušek success function (Sec. 2.3, Alg. 1), then the model is trained on those subgoals, and the reported INFORM/SUCCESS/COMBINED values are computed on a held-out test set using the same evaluation protocol. Improvement therefore requires real generalization, and the comparison against baselines is run under the same metric. The fact that the Nekvinda-Dušek evaluator is used both to label subgoal relevance and to score final results is a genuine risk of metric overfitting, but it is not a circular reduction: the evaluator is an external benchmark, not a quantity derived from the model or from the paper's own assumptions. No fitted parameter is renamed as a prediction, no uniqueness theorem is imported from the authors' prior work, and the only self-citation (Vlastelica et al., 2023) appears in related work and is not load-bearing. The score of 2 reflects the minor evaluator-coupling concern and the presence of a non-load-bearing self-citation, not a formal circularity.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The method rests on several assumptions about the evaluation oracle and the counterfactual procedure, plus a moderate number of hand-set hyperparameters. No new physical or mathematical entities are introduced. The most consequential input is the external success evaluator, which is reused for both data selection and final scoring.

free parameters (5)
  • k = number of sampled states/actions per turn = 2
    Chosen by hand for sampling dialog variants; affects the size and diversity of candidate dialogs and the resulting subgoal training set.
  • Fraction of user goals sampled per iteration = 4218 of 8437 goals
    An efficiency choice that limits which goals can appear in the subgoal training data and introduces selection effects.
  • Number of training iterations = 2
    Stopping is based on the COMBINED score; the procedure stops after the second iteration because the score does not increase further.
  • DPO beta = 0.1
    Set for direct preference optimization without a reported sensitivity sweep.
  • Learning rates and batch sizes = SFT: lr 5e-5, batch 2; DPO: lr 1e-6, batch 2
    Hyperparameters chosen for the Flan-T5-large model; no ablation is reported.
assumptions (4)
  • domain assumption The Nekvinda and Dusek success function is a faithful oracle for user goal completion.
    This function defines success as the last offered entity satisfying all constraints and all requestable slots being mentioned. It is used both for selecting subgoals and for final evaluation, so any misalignment with real user satisfaction affects both the training signal and the reported score.
  • domain assumption Turn-wise replacement of a successful subgoal with the same-turn subgoal from an unsuccessful dialog isolates the causal contribution of that subgoal.
    The counterfactual test changes only one turn at a time, so it assumes subgoal contributions are independent and additive. Multi-turn interactions and ordering effects are not captured, as the authors partly acknowledge in the Limitations section.
  • domain assumption Ground-truth user goals in MultiWOZ 2.2 are correct and complete.
    Success evaluation is performed against these goals, and the method does not generate new goals or simulate users, which the authors list as a limitation.
  • domain assumption Sampling from the current model produces enough successful and unsuccessful dialogs for the same user goal.
    The subgoal selection procedure requires at least one successful and one unsuccessful dialog per goal. Table 3 shows that many goals have zero or all successful samples, and those goals are excluded from training, which may bias the training distribution.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Learning from Relevant Subgoals in Successful Dialogs using Iterative Training for Task-oriented Dialog Systems." pith.science (2026). https://pith.science/paper/WBHTJ5ZF

@misc{pith2026241116305,
  author       = {Pith},
  title        = {Pith review of: Learning from Relevant Subgoals in Successful Dialogs using Iterative Training for Task-oriented Dialog Systems},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WBHTJ5ZF}},
  note         = {Machine review of arXiv:2411.16305}
}
read the original abstract

Task-oriented Dialog (ToD) systems have to solve multiple subgoals to accomplish user goals, whereas feedback is often obtained only at the end of the dialog. In this work, we propose SUIT (SUbgoal-aware ITerative Training), an iterative training approach for improving ToD systems. We sample dialogs from the model we aim to improve and determine subgoals that contribute to dialog success using distant supervision to obtain high quality training samples. We show how this data improves supervised fine-tuning or, alternatively, preference learning results. SUIT is able to iteratively generate more data instead of relying on fixed static sets. SUIT reaches new state-of-the-art performance on a popular ToD benchmark.

Figures

Figures reproduced from arXiv: 2411.16305 by the authors.

Figure 1
Figure 1. Successful dialog example. of people in [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of training procedure in SUIT. We sample multiple dialogs for one user goal, where each dialog Ds consists of user turns Ust, and system turns, which are split into dialog states Bst, system actions Ast and responses Rst. We evaluate dialog success at the end of each generated dialog. For every successful dialog Ds, we replace parts of system turns (subgoals) with the respective parts coming from wrong dial… view at source ↗
Figure 3
Figure 3. Example for input/ouput representation in S [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

31 extracted references · 15 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Leonard Adolphs, Tianyu Gao, Jing Xu, Kurt Shuster, Sainbayar Sukhbaatar, and Jason Weston. 2023. https://doi.org/10.18653/v1/2023.acl-long.493 The CRINGE loss: Learning what language not to model . In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8854--8874, Toronto, Canada. Associa...

    doi:10.18653/v1/2023.acl-long.493 2023

  4. [4]

    Namo Bang, Jeehyun Lee, and Myoung-Wan Koo. 2023. Task-optimized adapters for an end-to-end task-oriented dialogue system. In Findings of the Association for Computational Linguistics: ACL 2023, pages 7355--7369

    work page 2023

  5. [5]

    Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. 2017. https://proceedings.neurips.cc/paper_files/paper/2017/file/d5e2c0adad503c91f91df240d0cd4e49-Paper.pdf Deep reinforcement learning from human preferences . In Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc

    2017

  6. [6]

    Yihao Feng, Shentao Yang, Shujian Zhang, Jianguo Zhang, Caiming Xiong, Mingyuan Zhou, and Huan Wang. 2023. https://arxiv.org/abs/2302.10342 Fantastic rewards and how to tame them: A case study on reward learning for task-oriented dialogue systems . Preprint, arXiv:2302.10342

    work page Pith review arXiv 2023

  7. [7]

    Mohammad Gheshlaghi Azar, Zhaohan Daniel Guo, Bilal Piot, Remi Munos, Mark Rowland, Michal Valko, and Daniele Calandriello. 2024. https://proceedings.mlr.press/v238/gheshlaghi-azar24a.html A general theoretical paradigm to understand learning from human preferences . In Proceedings of The 27th International Conference on Artificial Intelligence and Statis...

    2024

  8. [8]

    Shangmin Guo, Biao Zhang, Tianlin Liu, Tianqi Liu, Misha Khalman, Felipe Llinares, Alexandre Rame, Thomas Mesnard, Yao Zhao, Bilal Piot, Johan Ferret, and Mathieu Blondel. 2024. https://arxiv.org/abs/2402.04792 Direct language model alignment from online ai feedback . Preprint, arXiv:2402.04792

    arXiv 2024

Show all 31 references
  1. [9]

    Wanwei He, Yinpei Dai, Yinhe Zheng, Yuchuan Wu, Zheng Cao, Dermot Liu, Peng Jiang, Min Yang, Fei Huang, Luo Si, et al. 2022. Galaxy: A generative pre-trained model for task-oriented dialog with semi-supervised learning and explicit policy injection. In Proceedings of the AAAI ...

    2022

  2. [10]

    Ehsan Hosseini-Asl, Bryan McCann, Chien-Sheng Wu, Semih Yavuz, and Richard Socher. 2020. A simple language model for task-oriented dialogue. Advances in Neural Information Processing Systems, 33:20179--20191

    2020

  3. [11]

    Timo Kaufmann, Paul Weng, Viktor Bengs, and Eyke Hüllermeier. 2024. https://arxiv.org/abs/2312.14925 A survey of reinforcement learning from human feedback . Preprint, arXiv:2312.14925

    2024

  4. [12]

    Chia-Hsuan Lee, Hao Cheng, and Mari Ostendorf. 2021. https://doi.org/10.18653/v1/2021.emnlp-main.404 Dialogue state tracking with a language model using schema-driven prompting . In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 4...

    2021 doi

  5. [13]

    Yohan Lee. 2021. https://doi.org/10.18653/v1/2021.findings-emnlp.112 Improving end-to-end task-oriented dialog system with a simple auxiliary task . In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 1296--1303, Punta Cana, Dominican Republic. Asso...

    2021 doi

  6. [14]

    Nurul Lubis, Christian Geishauser, Michael Heck, Hsien-chin Lin, Marco Moresi, Carel van Niekerk, and Milica Gasic. 2020. https://doi.org/10.18653/v1/2020.coling-main.41 LAVA : Latent action spaces via variational auto-encoding for dialogue policy optimization . In Proceedings...

    2020 doi

  7. [15]

    Tom \'a s Nekvinda and Ond r ej Du s ek. 2021. https://doi.org/10.18653/v1/2021.gem-1.4 Shades of BLEU , flavours of success: The case of M ulti WOZ . In Proceedings of the 1st Workshop on Natural Language Generation, Evaluation, and Metrics (GEM 2021), pages 34--46, Online. A...

    2021 doi

  8. [16]

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2024. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36

    2024

  9. [17]

    Govardana Sachithanandam Ramachandran, Kazuma Hashimoto, and Caiming Xiong. 2022. [caspi] causal-aware safe policy improvement for task-oriented dialogue. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 92--102

    2022

  10. [18]

    Charlie Snell, Sherry Yang, Justin Fu, Yi Su, and Sergey Levine. 2022. Context-aware language modeling for goal-oriented dialogue systems. In Findings of the Association for Computational Linguistics: NAACL 2022, pages 2351--2366

    2022

  11. [19]

    Feifan Song, Bowen Yu, Minghao Li, Haiyang Yu, Fei Huang, Yongbin Li, and Houfeng Wang. 2024. https://doi.org/10.1609/aaai.v38i17.29865 Preference ranking optimization for human alignment . Proceedings of the AAAI Conference on Artificial Intelligence, 38(17):18990--18998

    2024 doi

  12. [20]

    Sebastian Steindl, Ulrich Sch \"a fer, and Bernd Ludwig. 2024. Counterfactual dialog mixing as data augmentation for task-oriented dialog systems. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-CO...

    2024

  13. [21]

    Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul Christiano

    Nisan Stiennon, Long Ouyang, Jeff Wu, Daniel M. Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul Christiano. 2020. Learning to summarize from human feedback. In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIP...

    2020

  14. [22]

    Haipeng Sun, Junwei Bao, Youzheng Wu, and Xiaodong He. 2023. https://doi.org/10.18653/v1/2023.findings-acl.708 M ars: Modeling context & state representations with contrastive learning for end-to-end task-oriented dialog . In Findings of the Association for Computational Lingu...

    2023 doi

  15. [23]

    Xin Tian, Liankai Huang, Yingzhan Lin, Siqi Bao, Huang He, Yunyi Yang, Hua Wu, Fan Wang, and Shuqi Sun. 2021. https://doi.org/10.18653/v1/2021.nlp4convai-1.8 Amendable generation for dialogue state tracking . In Proceedings of the 3rd Workshop on Natural Language Processing fo...

    2021 doi

  16. [24]

    Marin Vlastelica, Patrick Ernst, and Gyuri Szarvas. 2023. https://doi.org/10.1609/aaai.v37i11.26602 Taming continuous posteriors for latent variational dialogue policies . Proceedings of the AAAI Conference on Artificial Intelligence, 37(11):13673--13681

    2023 doi

  17. [25]

    Qingyang Wu, James Gung, Raphael Shu, and Yi Zhang. 2023. https://doi.org/10.18653/v1/2023.sigdial-1.24 D iact TOD : Learning generalizable latent dialogue acts for controllable task-oriented dialogue systems . In Proceedings of the 24th Annual Meeting of the Special Interest ...

    2023 doi

  18. [26]

    Jing Xu, Andrew Lee, Sainbayar Sukhbaatar, and Jason Weston. 2023. Some things are more cringe than others: Preference optimization with the pairwise cringe loss. arXiv preprint arXiv:2312.16682

    2023 arXiv

  19. [27]

    Xiao Yu, Qingyang Wu, Kun Qian, and Zhou Yu. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.759 KRLS : Improving end-to-end response generation in task oriented dialog with reinforced keywords learning . In Proceedings of the 2023 Conference on Empirical Methods in Natural ...

    2023 doi

  20. [28]

    Hongyi Yuan, Zheng Yuan, Chuanqi Tan, Wei Wang, Songfang Huang, and Fei Huang. 2023. https://proceedings.neurips.cc/paper_files/paper/2023/file/23e6f78bdec844a9f7b6c957de2aae91-Paper-Conference.pdf Rrhf: Rank responses to align language models with human feedback . In Advances...

    2023

  21. [29]

    Xiaoxue Zang, Abhinav Rastogi, Srinivas Sunkara, Raghav Gupta, Jianguo Zhang, and Jindong Chen. 2020. Multiwoz 2.2: A dialogue dataset with additional annotation corrections and state tracking baselines. In Proceedings of the 2nd Workshop on Natural Language Processing for Con...

    2020

  22. [30]

    Jeffrey Zhao, Raghav Gupta, Yuan Cao, Dian Yu, Mingqiu Wang, Harrison Lee, Abhinav Rastogi, Izhak Shafran, and Yonghui Wu. 2022. https://arxiv.org/abs/2201.08904 Description-driven task-oriented dialog modeling . CoRR, abs/2201.08904

    2022 arXiv

  23. [31]

    Tiancheng Zhao, Kaige Xie, and Maxine Eskenazi. 2019. https://doi.org/10.18653/v1/N19-1123 Rethinking action spaces for reinforcement learning in end-to-end dialog agents with latent variable models . In Proceedings of the 2019 Conference of the North A merican Chapter of the ...

    2019 doi

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.