Pith. sign in

REVIEW 4 major objections 5 minor 60 references

PersonaFeedback: A Large-scale Human-annotated Benchmark For Personalization

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read PersonaFeedback, an 8,298-case benchmark with explicit personas, shows that even state-of-the-art LLMs are unreliable at choosing more personalized responses, especially on hard items.

desk verdict A genuinely useful personalization benchmark that deserves refereeing, but its hard-tier 'models fall short' claim needs a human ceiling baseline and uncertainty quantification before it can carry the weight the paper puts on it. read the letter →

arxiv 2506.12915 v1 pith:7BG3ECLB submitted 2025-06-15 cs.CL

Meiling Tao , Chenghao Zhu , Dongyi Ding , Tiannan Wang , Yuchen Eleanor Jiang , Wangchunshu Zhou This is my paper
classification cs.CL
keywords LLMpersonalizationpersonabenchmarkbinary-choiceevaluationrewardmodelsretrieval-augmentedgenerationhumanannotationFleisskappa
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces PersonaFeedback, a human-annotated benchmark built to test whether an LLM can actually use a user's persona when answering, rather than inferring who the user is from chat history. It contains 8,298 test cases in which a model sees an explicit persona profile, a query, and two candidate answers, and must pick the more personalized and helpful one. The cases are divided into easy, medium, and hard tiers according to how well nine annotators agree on the better answer. Across many proprietary and open models, the paper finds that the hardest cases are far from solved, that reasoning-specialized models do not outperform ordinary chat models, and that retrieval-augmented prompting does not help unless the persona is explicitly given. If correct, the benchmark isolates personalization as a distinct capability that current training and evaluation practices only partially address.

What carries the argument

The central object is the PersonaFeedback test case: a triplet $(P, x, \{a_1, a_2\})$ where $P$ is an explicit persona profile, $x$ is an open-ended query, and the model must choose between two candidate answers. The ground-truth choice is the majority of nine human annotators, and Fleiss' kappa—a measure of annotator agreement—assigns the case to easy, medium, or hard based on how clearly the better answer stands out. This setup isolates personalization from persona inference by making $P$ explicit, and the forced binary choice turns personalization into a quantifiable classification problem. The construction pipeline, which generates questions from persona memory, produces answers at different personalization levels, and screens pairs by annotator agreement, is what creates the difficulty tiers.

What would settle it

Take a random sample of hard-tier items and re-annotate them with an independent panel of similar size; if the new majority labels disagree with the original labels at a rate close to chance and model rankings change materially, the hard tier is measuring panel-specific agreement rather than stable personalization ability.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central claim is that LLM personalization can be directly measured with a human-annotated binary-choice task, and that measured this way, current models are far weaker at personalization than their general reasoning scores suggest. The authors argue that separating persona inference from personalization is necessary, because existing benchmarks that infer personas from chat history conflate two distinct abilities. The empirical heart is that on hard-tier pairs—where nine annotators only moderately agree—leading proprietary reasoning models, chat models, and reward models all remain well below reliable performance, while no retrieval-augmented setup matches simply giving the model the persona profile. The paper further claims that training a small reward model on persona-conditioned preference pairs substantially improves its scores, suggesting the gap is learnable.

Load-bearing premise

The ground truth for each test case is the majority vote of nine annotators, and hard-tier cases are kept even though those annotators agree only moderately, so the hard-tier label may reflect one particular annotator majority rather than a stable fact about which answer is more personalized.

Editorial extensions

If this is right

  • Reasoning-focused training is not a shortcut to personalization; models that lead on math and general agent tasks do not lead on hard persona choices.
  • Scale helps within a model family: larger open-source models personalize more reliably than smaller ones, so capacity and training-data investments can move the metric.
  • General reward models trained on broad helpfulness data under-serve user-specific questions; a modest amount of persona-conditioned preference data improves them.
  • Retrieval of user memories is not sufficient: without an explicit persona statement, RAG performs no better than ignoring the user, so systems should make learned user profiles explicit.
  • The hard tier remains an open evaluation target that current models cannot reliably pass, giving the field a concrete headroom signal.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: because the benchmark scores choosing rather than writing, a model that scores well could still fail to generate the chosen style of response; a generation-side variant would test whether discrimination transfers to production.
  • Editorial inference: the RAG result may depend on retrieval quality; a controlled variant that feeds the model the exact relevant memory would separate retrieval failure from personalization failure.
  • Editorial inference: hard-tier labels rest on moderate agreement, so item-response or annotator-modeling scoring rather than raw majority vote could yield a more stable benchmark.
  • Editorial inference: the low correlation with existing quality dimensions suggests personalization is a separate axis worth its own multi-aspect rubric.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The manuscript introduces PersonaFeedback, a benchmark of 8,298 human-annotated binary-choice test cases for evaluating LLM personalization. The design decouples persona inference from response personalization by giving explicit persona profiles, and the evaluation asks a model to choose which of two candidate responses is more personalized and helpful. The dataset is split into easy, medium, and hard tiers using Fleiss' kappa thresholds on nine annotator judgments, and the authors evaluate a wide range of proprietary, open-source, reasoning, chat, and reward models. The main empirical claim is that even strong reasoning models fall short on the hard tier, and the paper reports several additional findings on scaling, reward models, RAG, and explicit persona conditioning. The data, protocols, and evaluation pipeline are promised for public release.

Significance. If the benchmark's validity concerns are resolved, PersonaFeedback would be a valuable contribution: it provides a large human-annotated resource in an under-served area, uses explicit kappa-based filtering, includes an honest limitations section, and evaluates many model families under several conditioning settings. The decoupling of persona inference from personalization is a genuinely useful design choice, and the public release of the dataset would enable follow-up work on personalization-aware training and evaluation. However, the central claim that the benchmark 'directly evaluates' response generation is weakened by the binary-choice task format, and the absolute 'fall short' conclusion on hard items is not calibrated against a human-performance ceiling or uncertainty estimates.

major comments (4)
  1. [§3.4, Table 2] The hard tier is defined by 0.4 < Fleiss' κ ≤ 0.6 among nine annotators, which for a binary choice corresponds to individual pairwise agreement as low as roughly 70%. The paper reports no leave-one-out human baseline, no expected agreement between a fresh annotator and the majority label, and no confidence intervals for model accuracies. Consequently, the headline claim that state-of-the-art models 'fall short' on hard items is not calibrated: if the majority label is highly reproducible, 65–71% accuracy is clearly poor; if it is reproducible only near 70–75%, the conclusion changes. Please report a human ceiling (e.g., leave-one-annotator-out agreement with the majority) and per-tier confidence intervals or paired significance tests.
  2. [Abstract, §1, §4.1] The abstract states that PersonaFeedback 'directly evaluates LLMs' ability to provide personalized responses,' but the actual task is binary selection between two candidate answers that were already generated by undisclosed models. Selection ability is not the same as generation ability: a model can recognize a tailored response it could not produce, and a good generator may not be a calibrated judge. The central claim should be reframed as evaluating response selection or recognition, or the paper should add a generation-based evaluation in which models produce responses and humans (or a validated automatic metric) judge personalization.
  3. [§3.4] The four LLMs used to rewrite and construct the answer pool are never named, despite the fact that many evaluated models come from the same general families (OpenAI GPT, Qwen, DeepSeek). If an evaluated model can identify stylistic traces of its own or related outputs, its accuracy could be inflated; if not, the benchmark may be systematically harder for certain families. The paper should disclose the four generators and report whether accuracy varies by which model generated the chosen or rejected answer.
  4. [Table 2, §4.2 Takeaway 1] The finding that reasoning models do not improve personalization relies on very small differences, for example o3-mini (79.9 total average) versus GPT-4.1 (80.0) or GPT-4.5-preview (79.4). With item-level binary data, these differences require paired significance testing (e.g., McNemar's test) and confidence intervals before being presented as evidence of equivalence or ranking. The same issue affects the 'larger is better' and reward-model comparisons.
minor comments (5)
  1. [§2.1] The sentence 'Recently, a few lines of work have investigated in benchmarking personalization of LLMs []' contains an empty citation and a grammatical error; it should be completed and fixed.
  2. [Appendix H] The heading 'Limiations' is a typo and should read 'Limitations.'
  3. [Appendix G] In the answer-degradation prompt, 'You are an export in modify the AI assistant answer' should be 'You are an expert in modifying the AI assistant answer.'
  4. [§4.2] Takeaway 4 and Takeaway 5 appear to have misplaced headings: the text under 'Takeaway 4: RAG Falls Short in Personalization' begins with the heading 'Takeaway 5: Persona Learning Should Be Made Explicit,' and the explanatory paragraph follows. This should be reorganized.
  5. [Table 2] The footnote states that the random baseline is 50, but no per-column standard errors or confidence intervals are provided; given the binary nature of the task, these would help readers assess whether the small differences between closely ranked models are meaningful.

Circularity Check

1 steps flagged · score 2.0 of 10

No load-bearing circularity; one self-definitional gloss in the hard-tier description and minor non-load-bearing self-citations only.

  1. self definitional [Abstract; Section 3.4 Difficulty Levels]
    "even state-of-the-art LLMs that can solve complex real-world reasoning tasks could fall short on the hard tier of PersonaFeedback where even human evaluators may find the distinctions challenging. Hard: Pairs of answers with moderate consistency between the evaluators (0.4 < κ≤ 0.6) are classified as a hard difficulty, indicating that the personalization differences between these answers are small and the task is more difficult."

    The hard tier is defined as the set of pairs on which the nine annotators achieved only moderate agreement (0.4 < κ ≤ 0.6). The abstract's clause 'where even human evaluators may find the distinctions challenging' is therefore a restatement of the tier's inclusion criterion, not an independent empirical demonstration of human difficulty. This does not by itself force the model accuracy numbers: Table 2 scores are measured against the human majority labels and could have come out high or low. The circularity is confined to the rhetorical interpretation of the tier, so it does not invalidate the benchmark's central model-evaluation claim.

full rationale

PersonaFeedback's evaluation signal is external to the models: nine human annotators choose the more personalized and helpful answer, the majority vote defines ground truth, and pairs with low agreement are dropped (Section 3.4). Each model's binary-choice accuracy is then a separate measurement against those labels; no equation or fitting step makes a model's score equal to an input, and there is no fitted parameter renamed as a prediction. The only mild circularity is rhetorical: the hard tier is defined by moderate annotator agreement, so calling it 'where even human evaluators may find the distinctions challenging' restates the selection criterion. The model results (Table 2) are not entailed by that definition, and Appendix H explicitly acknowledges the subjectivity of evaluator judgments. Overlapping-author citations ([35], [40], [52]) occur in data generation and related work; they are not invoked as uniqueness theorems or as independent validation of the benchmark, so they do not carry the load. The central claim of a model shortfall on subtle personalization therefore has independent, human-grounded content; a low score is appropriate.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The benchmark's validity rests on several assumptions about human judgment, the proxy relationship between selection and generation, and the representativeness of synthetic personas. The only hand-chosen parameters that affect the dataset composition are the kappa thresholds and the persona-masking ratio. No invented entities are needed. The strongest circularity concern is that the candidate answers are generated by LLMs, and some of the evaluated models may be from the same families, creating a potential self-recognition confound.

free parameters (2)
  • Fleiss kappa thresholds = 0.4 and 0.6
    Difficulty tiers medium and hard are defined by kappa intervals (0.4, 0.6] and (>0.6); these hand-chosen thresholds determine which pairs enter the benchmark and thus the reported accuracy.
  • Persona masking ratio for A2 = 80%
    Answers of type A2 are generated by randomly masking 80% of the persona profile; this ratio influences the pool of candidate answers and hence the difficulty distribution.
assumptions (5)
  • domain assumption Human annotators can reliably judge which of two responses is more personalized and helpful given a persona profile.
    The entire ground truth of the benchmark depends on majority labels from 9 human evaluators; if this assumption fails, the labels are noise, especially in the hard tier.
  • domain assumption Binary-choice selection accuracy is a valid proxy for the ability to generate personalized responses.
    The abstract claims the benchmark evaluates LLMs' ability to 'provide personalized responses,' but the task is choosing between two existing responses; this assumes selection and generation abilities align.
  • domain assumption Synthetic personas expanded from 20 real profiles are sufficiently diverse and realistic to represent real users.
    The benchmark's external validity rests on the persona construction protocol; the authors acknowledge in Appendix H that personas may contain biases or simplifications.
  • domain assumption The answer pool generated by four undisclosed LLMs plus human editing spans the relevant space of personalized responses.
    Candidate answers are LLM-generated; if the answer pool lacks certain types of personalization, the benchmark cannot measure them.
  • domain assumption The RAG implementation used in Figure 3 is representative of retrieval-augmented personalization systems.
    The paper concludes RAG falls short based on its specific retrieval setup, but retriever details are not specified; this assumption is needed for the takeaway to generalize.

how reviews work

0 comments
Cite this review

Pith. "Pith review of PersonaFeedback: A Large-scale Human-annotated Benchmark For Personalization." pith.science (2026). https://pith.science/paper/7BG3ECLB

@misc{pith2026250612915,
  author       = {Pith},
  title        = {Pith review of: PersonaFeedback: A Large-scale Human-annotated Benchmark For Personalization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7BG3ECLB}},
  note         = {Machine review of arXiv:2506.12915}
}
read the original abstract

With the rapid improvement in the general capabilities of LLMs, LLM personalization, i.e., how to build LLM systems that can generate personalized responses or services that are tailored to distinct user personas, has become an increasingly important research and engineering problem. However, unlike many new challenging benchmarks being released for evaluating the general/reasoning capabilities, the lack of high-quality benchmarks for evaluating LLM personalization greatly hinders progress in this field. To address this, we introduce PersonaFeedback, a new benchmark that directly evaluates LLMs' ability to provide personalized responses given pre-defined user personas and queries. Unlike existing benchmarks that require models to infer implicit user personas from historical interactions, PersonaFeedback decouples persona inference from personalization, focusing on evaluating the model's ability to generate responses tailored to explicit personas. PersonaFeedback consists of 8298 human-annotated test cases, which are categorized into easy, medium, and hard tiers based on the contextual complexity of the user personas and the difficulty in distinguishing subtle differences between two personalized responses. We conduct comprehensive evaluations across a wide range of models. The empirical results reveal that even state-of-the-art LLMs that can solve complex real-world reasoning tasks could fall short on the hard tier of PersonaFeedback where even human evaluators may find the distinctions challenging. Furthermore, we conduct an in-depth analysis of failure modes across various types of systems, demonstrating that the current retrieval-augmented framework should not be seen as a de facto solution for personalization tasks. All benchmark data, annotation protocols, and the evaluation pipeline will be publicly available to facilitate future research on LLM personalization.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

60 extracted references · 25 canonical work pages

  1. [1]

    Llama 3 model card

    AI@Meta. Llama 3 model card. 2024. URLhttps://github.com/meta-llama/llama3/blob/main/MODEL_CARD. md

    work page 2024

  2. [2]

    Introducing Claude, 2023

    Anthropic. Introducing Claude, 2023. URLhttps://www.anthropic.com/index/introducing-claude

    work page 2023

  3. [3]

    Explainable recommendations via attentive multi-persona collaborative filtering

    Oren Barkan, Yonatan Fuchs, Avi Caciularu, and Noam Koenigstein. Explainable recommendations via attentive multi-persona collaborative filtering. In Proceedings of the 14th ACM Conference on Recommender Systems, pages 468–473, 2020

    work page 2020

  4. [4]

    Ralph Allan Bradley and Milton E. Terry. Rank analysis of incomplete block designs: I. the method of paired comparisons. Biometrika, 39:324, 1952. URLhttps://api.semanticscholar.org/CorpusID:125209808

    work page 1952

  5. [5]

    Large language models for user interest journeys.arXiv preprint arXiv:2305.15498, 2023

    Konstantina Christakopoulou, Alberto Lalama, Cj Adams, Iris Qu, Yifat Amir, Samer Chucri, Pierce Vollucci, Fabio Soldo, Dina Bseiso, Sarah Scodel, et al. Large language models for user interest journeys.arXiv preprint arXiv:2305.15498, 2023

    arXiv 2023

  6. [6]

    Training verifiers to solve math word problems, 2021

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems, 2021. URLhttps://arxiv.org/abs/2110.14168

    arXiv 2021

  7. [7]

    Language-based user profiles for recommendation, 2024

    Yijia Dai, Joyce Zhou, and Thorsten Joachims. Language-based user profiles for recommendation, 2024

    work page 2024

  8. [8]

    DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai D...

    arXiv 2025

Show all 60 references
  1. [9]

    Rlhf workflow: From reward modeling to online rlhf.arXiv preprint arXiv:2405.07863, 2024

    Hanze Dong, Wei Xiong, Bo Pang, Haoxiang Wang, Han Zhao, Yingbo Zhou, Nan Jiang, Doyen Sahoo, Caiming Xiong, and Tong Zhang. Rlhf workflow: From reward modeling to online rlhf.arXiv preprint arXiv:2405.07863, 2024

    2024 arXiv

  2. [10]

    Quantile regression for distributional reward models in rlhf.arXiv preprint arXiv:2409.10164, 2024

    Nicolai Dorka. Quantile regression for distributional reward models in rlhf.arXiv preprint arXiv:2409.10164, 2024

    2024 arXiv

  3. [11]

    Statistics (international student edition).Pisani, R

    David Freedman, Robert Pisani, and Roger Purves. Statistics (international student edition).Pisani, R. Purves, 4th edn. WW Norton & Company,New York, 2007

    2007

  4. [12]

    Assistgui: Task-oriented desktop graphical user interface automation.arXiv preprint arXiv:2312.13108, 2023

    Difei Gao, Lei Ji, Zechen Bai, Mingyu Ouyang, Peiran Li, Dongxing Mao, Qinchen Wu, Weichen Zhang, Peiyi Wang, Xiangwu Guo, et al. Assistgui: Task-oriented desktop graphical user interface automation.arXiv preprint arXiv:2312.13108, 2023. 11

    2023 arXiv

  5. [13]

    Maxwell Harper and Joseph A

    F. Maxwell Harper and Joseph A. Konstan. The movielens datasets: History and context.ACM Trans. Interact. Intell. Syst., 5(4), December 2015. ISSN 2160-6455. doi: 10.1145/2827872. URLhttps://doi.org/10.1145/ 2827872

    2015 doi

  6. [14]

    Lora: Low-rank adaptation of large language models.arXiv preprint arXiv:2106.09685, 2021

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models.arXiv preprint arXiv:2106.09685, 2021

    2021 arXiv

  7. [15]

    Mistral 7b.arXiv preprint arXiv:2310.06825, 2023

    Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. Mistral 7b.arXiv preprint arXiv:2310.06825, 2023

    arXiv

  8. [16]

    Know me, respond to me: Benchmarking llms for dynamic user profiling and personalized responses at scale

    Bowen Jiang, Zhuoqun Hao, Young-Min Cho, Bryan Li, Yuan Yuan, Sihao Chen, Lyle Ungar, Camillo J Taylor, and Dan Roth. Know me, respond to me: Benchmarking llms for dynamic user profiling and personalized responses at scale. arXiv preprint arXiv:2504.14225, 2025

    2025

  9. [17]

    Do llms understand user preferences? evaluating llms on user rating prediction, 2023

    Wang-Cheng Kang, Jianmo Ni, Nikhil Mehta, Maheswaran Sathiamoorthy, Lichan Hong, Ed Chi, and Derek Zhiyuan Cheng. Do llms understand user preferences? evaluating llms on user rating prediction, 2023

    2023

  10. [18]

    Smith, and Hannaneh Hajishirzi

    Nathan Lambert, Valentina Pyatkin, Jacob Morrison, LJ Miranda, Bill Yuchen Lin, Khyathi Chandu, Nouha Dziri, Sachin Kumar, Tom Zick, Yejin Choi, Noah A. Smith, and Hannaneh Hajishirzi. Rewardbench: Evaluating reward models for language modeling.https://huggingface.co/spaces/al...

    2024

  11. [19]

    Teach llms to personalize–an approach inspired by writing education.arXiv preprint arXiv:2308.07968, 2023

    Cheng Li, Mingyang Zhang, Qiaozhu Mei, Yaqing Wang, Spurthi Amba Hombaiah, Yi Liang, and Michael Bendersky. Teach llms to personalize–an approach inspired by writing education.arXiv preprint arXiv:2308.07968, 2023

    2023 arXiv

  12. [20]

    Evocodebench: An evolving code generation benchmark aligned with real-world code repositories.arXiv preprint arXiv:2404.00599, 2024

    Jia Li, Ge Li, Xuanming Zhang, Yihong Dong, and Zhi Jin. Evocodebench: An evolving code generation benchmark aligned with real-world code repositories.arXiv preprint arXiv:2404.00599, 2024

    2024 arXiv

  13. [21]

    Personal llm agents: Insights and survey about the capability, efficiency and security, 2024

    Yuanchun Li, Hao Wen, Weijun Wang, Xiangyu Li, Yizhen Yuan, Guohong Liu, Jiacheng Liu, Wenxing Xu, Xiang Wang, Yi Sun, Rui Kong, Yile Wang, Hanfei Geng, Jian Luan, Xuefeng Jin, Zilong Ye, Guanjing Xiong, Fan Zhang, Xiang Li, Mengwei Xu, Zhijun Li, Peng Li, Yang Liu, Ya-Qin Zha...

    2024

  14. [22]

    Deepseek-v3 technical report.arXiv preprint arXiv:2412.19437, 2024

    Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report.arXiv preprint arXiv:2412.19437, 2024

    2024 arXiv

  15. [23]

    Skywork-reward: Bag of tricks for reward modeling in llms.arXiv preprint arXiv:2410.18451, 2024

    Chris Yuhao Liu, Liang Zeng, Jiacai Liu, Rui Yan, Jujie He, Chaojie Wang, Shuicheng Yan, Yang Liu, and Yahui Zhou. Skywork-reward: Bag of tricks for reward modeling in llms.arXiv preprint arXiv:2410.18451, 2024

    2024 arXiv

  16. [24]

    Gaia: a benchmark for general ai assistants

    Grégoire Mialon, Clémentine Fourrier, Thomas Wolf, Yann LeCun, and Thomas Scialom. Gaia: a benchmark for general ai assistants. InThe TwelfthInternational Conference on Learning Representations, 2023

    2023

  17. [25]

    Inf-orm-llama3.1-70b, 2024

    Xiaoyu Tan Minghao Yang, Chao Qu. Inf-orm-llama3.1-70b, 2024. URL [https://huggingface.co/infly/ INF-ORM-Llama3.1-70B](https://huggingface.co/infly/INF-ORM-Llama3.1-70B)

    2024

  18. [26]

    Pearl: Personalizing large language model writing assistants with generation-calibrated retrievers

    Sheshera Mysore, Zhuoran Lu, Mengting Wan, Longqi Yang, Steve Menezes, Tina Baghaee, Emmanuel Bara- jas Gonzalez, Jennifer Neville, and Tara Safavi. Pearl: Personalizing large language model writing assistants with generation-calibrated retrievers. arXiv preprint arXiv:2311.09...

    2023 arXiv

  19. [27]

    Justifying recommendations using distantly-labeled reviews and fine-grained aspects

    Jianmo Ni, Jiacheng Li, and Julian McAuley. Justifying recommendations using distantly-labeled reviews and fine-grained aspects. In Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan, editors,Proceedings of the 2019 Conference on Empirical Methods in Natural Language Proces...

    2019 doi

  20. [28]

    Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and...

    2022

  21. [29]

    Offsetbias: Leveraging debiased data for tuning evaluators, 2024

    Junsoo Park, Seungyeon Jwa, Meiying Ren, Daeyoung Kim, and Sanghyuk Choi. Offsetbias: Leveraging debiased data for tuning evaluators, 2024. 12

    2024

  22. [30]

    Integrating summarization and retrieval for enhanced personalization via large language models

    Chris Richardson, Yao Zhang, Kellen Gillespie, Sudipta Kar, Arshdeep Singh, Zeynab Raeesy, Omar Zia Khan, and Abhinav Sethy. Integrating summarization and retrieval for enhanced personalization via large language models. arXiv preprint arXiv:2310.20081, 2023

    arXiv

  23. [31]

    Optimization methods for personalizing large language models through retrieval augmentation

    Alireza Salemi, Surya Kallumadi, and Hamed Zamani. Optimization methods for personalizing large language models through retrieval augmentation. InProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 752–762, 2024

    2024

  24. [32]

    Lamp: When large language models meet personalization, 2024

    Alireza Salemi, Sheshera Mysore, Michael Bendersky, and Hamed Zamani. Lamp: When large language models meet personalization, 2024

    2024

  25. [33]

    User modeling in the era of large language models: Current research and future directions

    Zhaoxuan Tan and Meng Jiang. User modeling in the era of large language models: Current research and future directions. arXiv preprint arXiv:2312.11518, 2023

    2023 arXiv

  26. [34]

    Democratizing large language models via personalized parameter-efficient fine-tuning.arXiv preprint arXiv:2402.04401, 2024

    Zhaoxuan Tan, Qingkai Zeng, Yijun Tian, Zheyuan Liu, Bing Yin, and Meng Jiang. Democratizing large language models via personalized parameter-efficient fine-tuning.arXiv preprint arXiv:2402.04401, 2024

    2024 arXiv

  27. [35]

    Rolecraft-glm: Advancing personalized role-playing in large language models.arXiv preprint arXiv:2401.09432, 2023

    Meiling Tao, Xuechen Liang, Tianyu Shi, Lei Yu, and Yiting Xie. Rolecraft-glm: Advancing personalized role-playing in large language models.arXiv preprint arXiv:2401.09432, 2023

    2023 arXiv

  28. [36]

    Qwq-32b: Embracing the power of reinforcement learning, March 2025

    Qwen Team. Qwq-32b: Embracing the power of reinforcement learning, March 2025. URLhttps://qwenlm. github.io/blog/qwq-32b/

    2025

  29. [37]

    Llama 2: Open foundation and fine-tuned chat models.CoRR, abs/2307.09288, 2023

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton-Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, W...

    2023 doi arXiv

  30. [38]

    Interpretable preferences via multi- objective reward modeling and mixture-of-experts

    Haoxiang Wang, Wei Xiong, Tengyang Xie, Han Zhao, and Tong Zhang. Interpretable preferences via multi- objective reward modeling and mixture-of-experts. InEMNLP, 2024

    2024

  31. [39]

    Weaver: Foundation models for creative writing.arXiv preprint arXiv:2401.17268, 2024

    Tiannan Wang, Jiamin Chen, Qingrui Jia, Shuai Wang, Ruoyu Fang, Huilin Wang, Zhaowei Gao, Chunzhao Xie, Chuou Xu, Jihong Dai, et al. Weaver: Foundation models for creative writing.arXiv preprint arXiv:2401.17268, 2024

    2024 arXiv

  32. [40]

    Ai persona: Towards life-long personalization of llms.arXiv preprint arXiv:2412.13103, 2024

    Tiannan Wang, Meiling Tao, Ruoyu Fang, Huilin Wang, Shuai Wang, Yuchen Eleanor Jiang, and Wangchunshu Zhou. Ai persona: Towards life-long personalization of llms.arXiv preprint arXiv:2412.13103, 2024

    2024 arXiv

  33. [41]

    Helpsteer: Multi-attribute helpfulness dataset for steerlm

    Zhilin Wang, Yi Dong, Jiaqi Zeng, Virginia Adams, Makesh Narsimhan Sreedhar, Daniel Egert, Olivier Delalleau, Jane Polak Scowcroft, Neel Kant, Aidan Swope, et al. Helpsteer: Multi-attribute helpfulness dataset for steerlm. arXiv preprint arXiv:2311.09528, 2023

    2023 arXiv

  34. [42]

    Helpsteer2: Open-source dataset for training top-performing reward models

    ZhilinWang, YiDong, OlivierDelalleau, JiaqiZeng, GeraldShen, DanielEgert, JimmyJZhang, MakeshNarsimhan Sreedhar, and Oleksii Kuchaiev. Helpsteer2: Open-source dataset for training top-performing reward models. arXiv preprint arXiv:2406.08673, 2024

    2024 arXiv

  35. [43]

    Personalized large language models, 2024

    Stanisław Woźniak, Bartłomiej Koptyra, Arkadiusz Janz, Przemysław Kazienko, and Jan Kocoń. Personalized large language models, 2024

    2024

  36. [44]

    Travelplanner: A benchmark for real-world planning with language agents

    Jian Xie, Kai Zhang, Jiangjie Chen, Tinghui Zhu, Renze Lou, Yuandong Tian, Yanghua Xiao, and Yu Su. Travelplanner: A benchmark for real-world planning with language agents. 2024

    2024

  37. [45]

    Iterative preference learning from human feedback: Bridging theory and practice for rlhf under kl-constraint, 2024

    Wei Xiong, Hanze Dong, Chenlu Ye, Ziqi Wang, Han Zhong, Heng Ji, Nan Jiang, and Tong Zhang. Iterative preference learning from human feedback: Bridging theory and practice for rlhf under kl-constraint, 2024

    2024

  38. [47]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report.arXiv preprint arXiv:2412.15115, 2024

    2024 arXiv

  39. [48]

    Refgpt: Dialogue generation of gpt, by gpt, and for gpt.arXiv preprint arXiv:2305.14994, 2023

    Dongjie Yang, Ruifeng Yuan, Yuantao Fan, Yifei Yang, Zili Wang, Shusen Wang, and Hai Zhao. Refgpt: Dialogue generation of gpt, by gpt, and for gpt.arXiv preprint arXiv:2305.14994, 2023

    2023 arXiv

  40. [49]

    Palr: Personalization aware llms for recommendation, 2023

    Fan Yang, Zheng Chen, Ziyan Jiang, Eunah Cho, Xiaojiang Huang, and Yanbin Lu. Palr: Personalization aware llms for recommendation, 2023

    2023

  41. [50]

    Bookgpt: A general framework for book recommen- dation empowered by large language model.arXiv preprint arXiv:2305.15673, 2023

    Aakas Zhiyuli, Yanfang Chen, Xuan Zhang, and Xun Liang. Bookgpt: A general framework for book recommen- dation empowered by large language model.arXiv preprint arXiv:2305.15673, 2023

    2023 arXiv

  42. [51]

    Instruction-following evaluation for large language models.arXiv preprint arXiv:2311.07911, 2023

    Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou. Instruction-following evaluation for large language models.arXiv preprint arXiv:2311.07911, 2023

    2023 arXiv

  43. [52]

    Learning to predict persona information for dialogue personalization without explicit persona description

    Wangchunshu Zhou, Qifei Li, and Chenle Li. Learning to predict persona information for dialogue personalization without explicit persona description. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki, editors, Findings of the Association for Computational Linguistics: ACL...

    2023 doi

  44. [53]

    +HelpSteer2 Personalized Trained

    Thomas P Zollo, Andrew Wei Tung Siah, Naimeng Ye, Ang Li, and Hongseok Namkoong. Personalllm: Tailoring llms to individual preferences.arXiv preprint arXiv:2409.20296, 2024. 14 Appendix A Persona Selection Protocol Selection ProcessTo ensure that the personas we build are auth...

    2024 arXiv

  45. [54]

    Avoid purely factual queries

    The question must be based on a scene. Avoid purely factual queries. The question should focus on open-ended, exploratory, or subjective judgment types

  46. [55]

    Do not awkwardly mash memory scenes and persona configuration

    The question must sound natural, self-consistent and realistic, as if asked by a real person. Do not awkwardly mash memory scenes and persona configuration. You can reference the persona configuration for inspiration, but the question should arise organically from the scene. B...

  47. [56]

    The response should reflect the tone and style, the professional AI assistant, friendly and helpful

  48. [57]

    Take into account the user’s preferences and characteristics to provide a personalized response

  49. [58]

    Instead, infer what the user might want based on their existing persona configuration, rather than just combining field details

    The answer should meet the user’s needs without overtly showcasing field information. Instead, infer what the user might want based on their existing persona configuration, rather than just combining field details

  50. [59]

    The response should directly solve the user’s question while reflecting an understanding of their preferences. Here is the current persona config: {persona_description} Here is an example of output: {example} Here is the user question: {question} Generate a personalized answer...

  51. [60]

    Reduce personalization: Make the answer more generic. For example, change a personalized element in the original response into something more universal, something that would be acceptable to anyone, or at least not fully aligned with the user’s specific interests and preferences

  52. [61]

    For instance, add overly general or irrelevant information, or provide vague or overly broad advice

    Lower helpfulness: Make the answer vague or less precise, not solving the user’s specific problem well. For instance, add overly general or irrelevant information, or provide vague or overly broad advice. Place the degraded answer in the <degraded_answer >tag. The modified ans...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.