Pith. sign in

REVIEW 4 major objections 5 minor 34 references

QExplorer: Large Language Model Based Query Extraction for Toxic Content Exploration

T0 review · 4 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read QExplorer, an LLM fine-tuned with instruction SFT followed by DPO on search-engine feedback, extracts queries that find disguised toxic content more effectively than human auditors and strong baselines.

desk verdict Useful search-engine-as-reward DPO recipe for query extraction, but the offline human comparison is inflated by self-retrieval and shared labels; the online deployment still supports the core claim. read the letter →

arxiv 2502.18480 v1 pith:H7LU2CIO submitted 2025-02-06 cs.IR cs.AIcs.CL

Shaola Ren , Li Ke , Longtao Huang , Dehong Gao , Hui Xue This is my paper · ORCID
classification cs.IRcs.AIcs.CL
keywords queryextractiontoxiccontentexplorationlargelanguagemodelsdirectpreferenceoptimizationsupervisedfine-tuningsearchenginefeedbacksafetye-commerceplatformmoderation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a general-purpose chat LLM, adapted in two stages, can replace the most labor-intensive step of toxic content exploration: turning a single reported violation into search queries that uncover similar hidden violations. The authors propose QExplorer, which first trains the LLM to imitate auditor-written queries via instruction supervised fine-tuning, then applies direct preference optimization using the search engine's own feedback as the reward: a query that exposes a higher proportion of toxic items is treated as the preferred one. On a held-out set of reported items on a large Chinese second-hand trading platform, the best QExplorer model reaches a query hit rate of 0.561 versus 0.489 for human-labeled queries, and finds 1321 toxic items among the top 100 retrievals versus 725 for humans. In online deployment over 20 weeks, model-suggested queries add a 59.9% increase in toxic item detection on top of human effort. If these results hold, content safety teams can offload much of the query-writing burden from human auditors to an automated model while increasing detection.

What carries the argument

The load-bearing mechanism is the pair of training stages plus the feedback signal that connects them. Instruction SFT teaches the model the surface task—given a reported toxic item, output risk-exploration keywords—using auditor annotations filtered to phrases with non-zero exposure or length at most 10. The preference stage then constructs (preferred, dispreferred) keyword pairs from grouped similar items, where preference is decided by $p(k|S)$, the proportion of logged exposed items retrieved by keyword $k$ that turn out to be toxic; pairs are separated by the 0.05 threshold in Eq. (4). DPO adjusts the model so preferred keyword sets become more likely than dispreferred ones, with LoRA keeping the update cheap on 7B-class models. The key point is that the second stage optimizes for search outcomes, not for agreement with human annotations.

What would settle it

Draw a random sample of reported toxic items, ask independent auditors to label every item exposed by each candidate query (human and model) without knowing the query source, and recompute query hit rate and hit@100 against this exhaustive label set. If QExplorer's advantage over humans disappears or reverses under exhaustive labeling, the logged-feedback signal was measuring label coverage rather than query effectiveness.

Watch

Extended reading notes

Core claim

The central claim, stated on the paper's own terms, is that search-engine feedback can be converted into a preference signal that makes a lightly fine-tuned 7B LLM a more effective query extractor than the human auditors it is trained to imitate. The model is not asked to annotate what is toxic; it is asked to output the short, distinctive phrases that, when run against the platform's search index, return items that auditors subsequently label toxic. The paper's evidence is the offline comparison in Table 5 and the online results in Table 7: Qwen1.5-7B-Chat (QExplorer, LoRA) produces 296 queries of which 166 are effective (query hit rate 0.561), versus 321 human queries of which 157 are effective (0.489), and its queries accumulate 1321 toxic items in the top 100 results against 725 for human queries. Online, 61.6% of model-suggested queries are accepted by auditors and yield a 59.9% mean increment in detected toxic items.

Load-bearing premise

The load-bearing premise is that a keyword's quality is faithfully measured by $p(k|S)$, the fraction of logged exposed items that auditors later labeled toxic; if those logs or labels are incomplete, or biased by the queries auditors chose to run, the preference signal and the offline evaluation share the same bias.

Editorial extensions

If this is right

  • Query extraction for toxic content exploration can be largely automated with a 7B chat model, leaving auditors to confirm and search rather than invent queries from scratch.
  • Adding DPO on search feedback raises query hit rate by about 10.4 percentage points over instruction SFT alone (0.508 to 0.561 on Qwen1.5-7B-Chat), and the same recipe improves ChatGLM3-6B.
  • In online operation, model-suggested queries add roughly 59.9% more detected toxic items and 39.9% more hit queries, even though overlapping queries are credited to the human side.
  • Large closed LLMs such as GPT-4 Turbo can beat human query hit rate but still trail the domain-aligned 7B model, indicating that task-specific alignment with search feedback is doing real work.
  • The current system handles text only; multimodal toxic items still require human risk understanding, which the paper lists as future work.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The reported gains are measured against the same auditor-produced toxic labels that generated the training signal; a deployment study with independently recruited auditors and log-free labeling would test whether the advantage survives a change in who defines 'toxic'.
  • The 0.05 threshold on $p(k|S)$ is likely prevalence-dependent; platforms with very rare toxic items may need a lower threshold or a different effectiveness signal such as precision at fixed exposure.
  • The two-stage recipe should transfer to other camouflaged-content search tasks—fraudulent listings, prohibited goods, policy-violating ads—wherever a search index and a stream of reported examples exist.
  • Because only about 600 preference pairs were used, generating candidate queries via beam search from different checkpoints and labeling them with the same engine feedback could scale the approach; the paper names this as future work.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. QExplorer proposes a two-stage LLM fine-tuning method (instruction SFT followed by DPO) for extracting search queries that help auditors discover toxic content on the Xianyu second-hand trading platform. The preference signal for DPO is derived from search-engine feedback: a keyword is preferred if a higher proportion of the items it retrieves are later labeled toxic. The paper reports offline experiments on 402 reported items from a single day, claiming that Qwen1.5-7B-Chat fine-tuned with QExplorer achieves a query hit rate of 0.561 versus 0.489 for human-labeled queries and 1321 hit@100 versus 725 for humans (Table 5). It also reports a 20-week online deployment in which model-suggested queries yield a 59.9% average toxic-item detection increment over human queries (Table 7).

Significance. The paper addresses a practical and important problem: scaling toxic-content exploration on a large e-commerce platform. Its strengths include a real deployment over 20 weeks, a clear two-stage training recipe, ablations over base models and preference-data construction, and a comparison against several strong baselines (GPT-4 Turbo, Qwen2.5-72B-Instruct, BART, Bi-LSTM-CRF). If the central claim holds, the method offers a deployable tool that can augment human auditors. However, the offline evidence for 'outperforming humans' is not yet conclusive because of a circularity between the DPO reward and the evaluation metrics, an unmatched human baseline, and the absence of statistical significance testing.

major comments (4)
  1. [§3.2, Eq. (4); §4.2.2, §4.3, Table 5] The DPO reward p(k|S) in Eq. (4) and the offline evaluation metrics (query hit rate and hit@100) are both computed using the same Xianyu search engine and the same auditor-produced toxic labels from the same log period. Since the offline retrieval data is collected before auditors begin exploration (Section 4.2.2), the reported toxic item itself is still in the index. A query that reproduces a distinctive substring of the reported content will therefore count as a 'hit' and contribute to hit@100 even if it retrieves no novel toxic items. DPO explicitly trains the model to prefer such search-feedback-positive terms, so a substantial part of the observed offline improvement over humans in Table 5 may reflect self-retrieval rather than genuine exploration effectiveness. The authors should either exclude the source reported item from the retrieved set when computing the metrics, or validate on an independent label source that was not used to construct the DPO preference data.
  2. [§3.1, §3.3, Table 5] The human baseline is not a matched task. The human queries are production annotations made during normal risk understanding (Section 3.1), not queries elicited under the same instruction used for the model, which explicitly asks for keywords that help detect more similar risks. The paper reports that human queries have a lower hit rate and lower hit@100, but this could be because the production task is not identical to the exploration-yield maximization task. A controlled study in which auditors are given the same instruction and the same reported items as the model would be needed to support the claim that QExplorer extracts more effective queries than humans.
  3. [§4.5, Table 5] All offline conclusions rest on a single test date (10.20.2024, 402 items) with no error bars, confidence intervals, or significance tests. The reported differences are large, and the paper's later online results provide supporting evidence, but the offline claim of human-level performance is not statistically grounded. At minimum, the authors should report per-item variability or bootstrap confidence intervals, and ideally evaluate on multiple test dates to show stability.
  4. [§3.3, §4.7, Table 7] The online evaluation is a valuable real-world signal, but it inherits the same self-retrieval ambiguity as the offline evaluation, and the comparison is confounded by the stated policy that human-annotated queries are searched with higher priority (Section 3.3). The 59.9% toxic-item detection increment is reported as an average over 20 weeks without a definition of the denominator or a per-week breakdown (the time series in Figure 4 is visually helpful but not statistically summarized). The authors should state whether the reported item itself is excluded from the count of 'additional toxic items', and report the distribution of weekly increments.
minor comments (5)
  1. [§4.3] Typo: 'we ave designed' should be 'we have designed'.
  2. [§3.2] Typo: 'controlls' should be 'controls'.
  3. [§3.2, Eq. (6)] The sentence 'L_SFT is as equation (5)' should read 'L_DPO is as equation (5)'; otherwise Eq. (6) is undefined.
  4. [§4.1, Table 4] The test dataset name 'D_reported_20241020' and the number 402 are given, but the number of unique reported toxic items is not stated; clarify whether each reported item contributes one query instance or multiple.
  5. [§4.7, Figure 4] The online metrics would benefit from a precise textual definition of 'toxic item detection increment' and 'hit query increment' in the main text, not just a brief mention in the metrics list of Section 4.3.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the offline evaluation is a temporally held-out test of a model trained on search feedback, not a restatement of the training objective.

full rationale

The paper's claimed derivation is an empirical training pipeline: instruction SFT on auditor-annotated query extraction, followed by DPO preference alignment where the preference signal is p(k|S), the proportion of toxic items among logged exposed items retrieved by a keyword (Eq. 4). The offline evaluation uses query hit rate and number of hit@100, computed on a separate temporal split: training data spans 11.19.2023 to 05.19.2024, while the offline test is built from reported toxic items of 10.20.2024 (Section 4.1). Retrieval data for evaluation is collected before auditors begin exploration (Section 4.2.2), and all methods are evaluated on the same retrieval tasks. Although the DPO reward and the offline metrics both draw on the same Xianyu search engine and auditor toxicity labels, they are not identical objects: the reward is a historical proportion over logged exposed items, while the offline metrics are freshly collected top-100 retrieval results scored against auditor labels on the held-out content. No equation in the paper reduces the reported hit rate or hit@100 to the training objective by construction; the evaluation would still be able to reveal overfitting or failure to generalize. The only self-citation appears as background in Section 2.1 ([32] in the list of 2-step keyphrase extraction methods) and is not load-bearing for the central claim. The mild concern that training and evaluation share the same search-feedback environment is an evaluation-quality issue, not a circular derivation, so no circularity step is flagged.

Assumptions & free parameters 7 free parameters · 5 assumptions · 0 invented entities

Free parameters are empirical thresholds and training hyperparameters; the main non-standard one is the preference threshold p=0.05. Axioms are mostly domain assumptions about label quality and search-engine feedback; no invented entities are introduced.

free parameters (7)
  • preference threshold p = 0.05
    Defines preferred vs dispreferred queries in D_comp (Eq. 4); chosen empirically and tested in ablation (p=0 in D_comp2).
  • SFT keyword length bounds = 2 < len(k) <= 10
    Data filter from Section 3.1; affects which reported items and annotations enter instruction SFT.
  • SFT epochs = 2
    Training setting in Section 4.4; no tuning curve shown.
  • DPO epochs = 5
    Training setting in Section 4.4.
  • learning rate = 3e-5
    Used in both stages (Section 4.4).
  • DPO beta = 0.1
    Hyperparameter in Eq. 5 controlling deviation from reference policy (Section 4.4).
  • SFT loss weight gamma = 1.0
    Weight in combined loss Eq. 6 (Section 4.4).
assumptions (5)
  • domain assumption Auditor annotations in Xianyu system logs are a reliable ground truth for query usefulness and for toxic labels in evaluation.
    Used to construct the SFT dataset D (Section 3.1) and to label retrieved items for offline metrics (Section 4.2.2).
  • domain assumption Search-engine feedback p(k|S), the fraction of toxic items among exposed items, is a faithful reward for query effectiveness in exploration.
    Central to DPO preference data construction in Eq. 4 (Section 3.2); if biased, both training and evaluation inherit the bias.
  • domain assumption The search system is approximately static during the 1-2 hour retrieval-data collection window.
    Stated in Section 4.2.2; required for hit@100 and query hit rate to be comparable across methods before item removal.
  • domain assumption Concatenating clustered similar toxic items preserves enough semantic context for long-context training and preference ordering.
    Used to build D_cat and D_comp in Section 3.1 (Eq. 2) and Section 3.2.
  • standard math The DPO loss and its combination with SFT loss (Eqs. 5-6) are valid as derived in Rafailov et al. [24].
    The paper applies the published DPO objective without re-deriving it; correctness of the central method assumes this derivation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of QExplorer: Large Language Model Based Query Extraction for Toxic Content Exploration." pith.science (2026). https://pith.science/paper/H7LU2CIO

@misc{pith2026250218480,
  author       = {Pith},
  title        = {Pith review of: QExplorer: Large Language Model Based Query Extraction for Toxic Content Exploration},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/H7LU2CIO}},
  note         = {Machine review of arXiv:2502.18480}
}
read the original abstract

Automatically extracting effective queries is challenging in information retrieval, especially in toxic content exploration, as such content is likely to be disguised. With the recent achievements in generative Large Language Model (LLM), we are able to leverage the capabilities of LLMs to extract effective queries for similar content exploration directly. This study proposes QExplorer, an approach of large language model based Query Extraction for toxic content Exploration. The QExplorer approach involves a 2-stage training process: instruction Supervised FineTuning (SFT) and preference alignment using Direct Preference Optimization (DPO), as well as the datasets construction with feedback of search system. To verify the effectiveness of QExplorer, a series of offline and online experiments are conducted on our real-world system. The offline empirical results demonstrate that the performance of our automatic query extraction outperforms that of several LLMs and humans. The online deployment shows a significant increase in the detection of toxic items.

Figures

Figures reproduced from arXiv: 2502.18480 by the authors.

Figure 1
Figure 1. Framework of the toxic content detection system. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. A diagram of QExplorer is presented. It contains [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. The diagram of the online implementation. The [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Online performance over time. proposed method, QExplorer. Results of offline experiments show that the queries suggested by Qwen1.5-7B-Chat (QExplorer) out￾perform the queries crafted by human. Online experiments reveal that Qwen1.5-7B-Chat (QExplorer) can suggest a su…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

34 extracted references · 7 canonical work pages

  1. [23]

    Wenjun Peng, Guiyang Li, Yue Jiang, Zilong Wang, Dan Ou, Xiaoyi Zeng, Derong Xu, Tong Xu, and Enhong Chen. 2024. Large Language Model based Long-tail Query Rewriting in Taobao Search. arXiv:2311.03758 [cs.IR] https://arxiv.org/ abs/2311.03758

    arXiv 2024

  2. [1]

    Lee Giles

    Rabah Alzaidy, Cornelia Caragea, and C. Lee Giles. 2019. Bi-LSTM-CRF Sequence Labeling for Keyphrase Extraction from Scholarly Documents. In The World Wide Web Conference (San Francisco, CA, USA) (WWW ’19) . Association for Computing Machinery, New York, NY, USA, 2551–2557. https://doi.org/10.1145/ 3308558.3313642

    work page arXiv 2019

  3. [2]

    Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, Binyuan Hui, Luo Ji, Mei Li, Junyang Lin, Runji Lin, Dayiheng Liu, Gao Liu, Chengqiang Lu, Keming Lu, Jianxin Ma, Rui Men, Xingzhang Ren, Xuancheng Ren, Chuanqi Tan, Sinan Tan, Jianhong Tu, Peng Wang, Shijie Wang, Wei Wang, Shengguang Wu, Benfeng X...

    arXiv 2023

  4. [3]

    Jason P. C. Chiu and Eric Nichols. 2016. Named Entity Recognition with Bidirec- tional LSTM-CNNs. arXiv:1511.08308 [cs.CL] https://arxiv.org/abs/1511.08308

    work page Pith review arXiv 2016

  5. [4]

    Minseok Choi, Chaeheon Gwak, Seho Kim, Si Hyeong Kim, and Jaegul Choo

    work page

  6. [5]

    Md Faisal Mahbub Chowdhury, Gaetano Rossiello, Michael Glass, Nan- dana Mihindukulasooriya, and Alfio Gliozzo. 2022. Applying a Generic Sequence-to-Sequence Model for Simple and Effective Keyphrase Generation. arXiv:2201.05302 [cs.CL] https://arxiv.org/abs/2201.05302

    work page Pith review arXiv 2022

  7. [6]

    Chi, Jeff Dean, Jacob Devlin, Adam Roberts, Denny Zhou, Quoc V

    Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, Albert Web- son, Shixiang Shane Gu, Zhuyun Dai, Mirac Suzgun, Xinyun Chen, Aakanksha Chowdhery, Alex Castro-Ros, Marie Pellat, Kevin Robinson, Dasha Valter, Sharan Narang, Gaurav Mishra, Adams Yu, Vincent Zhao, Yanping...

    arXiv 2022

  8. [7]

    Zhengxiao Du, Yujie Qian, Xiao Liu, Ming Ding, Jiezhong Qiu, Zhilin Yang, and Jie Tang. 2022. GLM: General Language Model Pretraining with Autoregressive Blank Infilling. arXiv:2103.10360 [cs.CL] https://arxiv.org/abs/2103.10360

    arXiv 2022

Show all 34 references
  1. [8]

    Team GLM, :, Aohan Zeng, Bin Xu, Bowen Wang, Chenhui Zhang, Da Yin, Dan Zhang, Diego Rojas, Guanyu Feng, Hanlin Zhao, Hanyu Lai, Hao Yu, Hongning Wang, Jiadai Sun, Jiajie Zhang, Jiale Cheng, Jiayi Gui, Jie Tang, Jing Zhang, Jingyu Sun, Juanzi Li, Lei Zhao, Lindong Wu, Lucen Zh...

    2024 arXiv

  2. [9]

    Xiaotao Gu, Zihan Wang, Zhenyu Bi, Yu Meng, Liyuan Liu, Jiawei Han, and Jingbo Shang. 2021. UCPhrase: Unsupervised Context-aware Quality Phrase Tagging. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining (KDD ’21) . ACM. https://doi.org/10.11...

    2021

  3. [10]

    Kazi Saidul Hasan and Vincent Ng. 2014. Automatic Keyphrase Extraction: A Survey of the State of the Art. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , Kristina Toutanova and Hua Wu (Eds.). Association for ...

    2014 doi

  4. [11]

    Soufiane Hayou, Nikhil Ghosh, and Bin Yu. 2024. LoRA+: Efficient Low Rank Adaptation of Large Models. arXiv:2402.12354 [cs.LG] https://arxiv.org/abs/ 2402.12354

    2024 arXiv

  5. [12]

    Dan Hendrycks, Steven Basart, Saurav Kadavath, Mantas Mazeika, Akul Arora, Ethan Guo, Collin Burns, Samir Puranik, Horace He, Dawn Song, and Ja- cob Steinhardt. 2021. Measuring Coding Challenge Competence With APPS. arXiv:2105.09938 [cs.SE] https://arxiv.org/abs/2105.09938

    2021 arXiv

  6. [13]

    Jiwoo Hong, Noah Lee, and James Thorne. 2024. ORPO: Monolithic Preference Optimization without Reference Model. arXiv:2403.07691 [cs.CL] https://arxiv. org/abs/2403.07691

    2024 arXiv

  7. [14]

    Zhenyu Hou, Yilin Niu, Zhengxiao Du, Xiaohan Zhang, Xiao Liu, Aohan Zeng, Qinkai Zheng, Minlie Huang, Hongning Wang, Jie Tang, and Yuxiao Dong. 2024. ChatGLM-RLHF: Practices of Aligning Large Language Models with Human Feedback. arXiv:2404.00934 [cs.CL] https://arxiv.org/abs/2...

    2024 arXiv

  8. [15]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. LoRA: Low-Rank Adaptation of Large Language Models. arXiv:2106.09685 [cs.CL] https://arxiv.org/abs/2106.09685

    2021 arXiv

  9. [16]

    Guillaume Lample, Miguel Ballesteros, Sandeep Subramanian, Kazuya Kawakami, and Chris Dyer. 2016. Neural Architectures for Named Entity Recognition. arXiv:1603.01360 [cs.CL] https://arxiv.org/abs/1603.01360

    2016 arXiv

  10. [17]

    Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov, and Luke Zettlemoyer. 2019. BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Gener- ation, Translation, and Comprehension. arXiv:1910.13461 [cs.CL] ht...

    2019 arXiv

  11. [18]

    Rui Liu, Zheng Lin, Peng Fu, and Weiping Wang. 2020. Reinforced Keyphrase Generation with BERT-based Sentence Scorer. In 2020 IEEE Intl Conf on Par- allel & Distributed Processing with Applications, Big Data & Cloud Comput- ing, Sustainable Computing & Communications, Social C...

    2020

  12. [19]

    Yi Luan, Mari Ostendorf, and Hannaneh Hajishirzi. 2017. Scientific Informa- tion Extraction with Semi-supervised Neural Tagging. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing , Martha Palmer, Rebecca Hwa, and Sebastian Riedel (Eds.)....

    2017 doi

  13. [20]

    Zakariae Alami Merrouni, Bouchra Frikh, and Brahim Ouhbi. 2016. Automatic keyphrase extraction: An overview of the state of the art. In 2016 4th IEEE International Colloquium on Information Science and Technology (CiSt) . 306–313. https://doi.org/10.1109/CIST.2016.7805062

    2016

  14. [21]

    Swaroop Mishra, Daniel Khashabi, Chitta Baral, and Hannaneh Hajishirzi. 2022. Cross-Task Generalization via Natural Language Crowdsourcing Instructions. arXiv:2104.08773 [cs.CL] https://arxiv.org/abs/2104.08773

    2022 arXiv

  15. [22]

    Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schul- man, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Pe- ter Welinder, Paul Christiano, Jan Leike,...

    2022 arXiv

  16. [24]

    Manning, and Chelsea Finn

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D. Manning, and Chelsea Finn. 2023. Direct Preference Optimization: Your Language Model is Secretly a Reward Model. arXiv:2305.18290 [cs.LG] https://arxiv.org/ abs/2305.18290

    2023 arXiv

  17. [25]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guil- laume Lample. 2023. LLaMA: Open and Efficient Foundation ...

    2023 arXiv

  18. [26]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yas- mine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhos- ale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucu- rull, David Esiobu, Jude Fernandes, Jeremy...

    2023 arXiv

  19. [27]

    Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M

    Jason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V. Le. 2022. Finetuned Language Models Are Zero-Shot Learners. arXiv:2109.01652 [cs.CL] https://arxiv.org/abs/ 2109.01652

    2022 arXiv

  20. [28]

    Di Wu, Wasi Uddin Ahmad, and Kai-Wei Chang. 2024. Pre-trained Lan- guage Models for Keyphrase Generation: A Thorough Empirical Study. arXiv:2212.10233 [cs.CL] https://arxiv.org/abs/2212.10233

    2024 arXiv

  21. [29]

    Shusheng Xu, Wei Fu, Jiaxuan Gao, Wenjie Ye, Weilin Liu, Zhiyu Mei, Guangju Wang, Chao Yu, and Yi Wu. 2024. Is DPO Superior to PPO for LLM Alignment? A Comprehensive Study. arXiv:2404.10719 [cs.CL] https://arxiv.org/abs/2404.10719

    2024 arXiv

  22. [30]

    question

    An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Cheng- peng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma, Jin Xu, Jingren Zhou, Jinze Bai, Jinzhe...

    2024 arXiv

  23. [31]

    Qi Zhang, Yang Wang, Yeyun Gong, and Xuanjing Huang. 2016. Keyphrase Extraction Using Deep Recurrent Neural Networks on Twitter. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing , Jian Su, Kevin Duh, and Xavier Carreras (Eds.). Associat...

    2016 doi

  24. [32]

    Lin Zhao, Longtao Huang, Liangjun Zang, Jizhong Han, and Songlin Hu. 2019. LMLSTM:Extract Event-Oriented Keyphrase From News Stream. In 2019 Interna- tional Joint Conference on Neural Networks (IJCNN) . 1–7. https://doi.org/10.1109/ IJCNN.2019.8852151

    2019

  25. [33]

    Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhangchi Feng, and Yongqiang Ma. 2024. LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models. arXiv:2403.13372 [cs.CL] https://arxiv.org/abs/2403. 13372 A Data Examples A sample from our preference...

    2024 arXiv

  26. [2023]

    arXiv:2310.08221 [cs.CL] https://arxiv.org/abs/2310.08221

    SimCKP: Simple Contrastive Learning of Keyphrase Representations. arXiv:2310.08221 [cs.CL] https://arxiv.org/abs/2310.08221

    arXiv

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.