Pith. sign in

REVIEW 3 major objections 5 minor 36 references

Aligning LLMs by Predicting Preferences from User Writing Samples

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read An LLM writing assistant can learn a user's style from a few writing samples by iteratively refining and verifying inferred preferences, improving generation compliance over CIPHER by 33%.

desk verdict Useful incremental method and benchmark, but the 33% gain over CIPHER is measured through a self-referential GPT-4o loop; human validation covers only the combined variant. read the letter →

arxiv 2505.23815 v1 pith:ADFMFPAE submitted 2025-05-27 cs.CL cs.LG

Stéphane Aroca-Ouellette , Natalie Mackraz , Barry-John Theobald , Katherine Metcalf This is my paper
classification cs.CLcs.LG
keywords preferenceinferenceLLMpersonalizationwritingstylealignmentiterativerefinementconsistencyverificationLLM-as-a-Judgeassistivebenchmarkin-contextlearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces PROSE, a way to teach an LLM writing assistant a person's style from a few of their own writing samples: the assistant infers a short description of the user's preferences and then writes while conditioned on that description. The claim is that two extra steps make the inferred description far more accurate—iteratively refining the description by drafting, comparing the draft to what the user actually wrote, and revising, plus checking each preference component against several of the user's past samples and pruning the ones that do not hold up. Evaluated on a new benchmark the authors built (PLUME), PROSE improves the preference-compliance of generated summaries and emails by 33% over CIPHER, the prior state of the art. A 16-participant human study supports the result for the combined PROSE-plus-in-context-learning variant, with win rates of 69.4% over ICL and 91.8% over CIPHER. If the claim holds, writing assistants can be personalized from a handful of samples without fine-tuning, and the learned preferences remain human-readable and editable.

What carries the argument

The load-bearing object is the inferred preference description ($\hat{p}_{desc}$), a concise, imperative-form list of style directives in the LLM's own words that conditions every generated summary or email. PROSE's machinery around it is a three-step loop: iterative refinement alternates between generating a candidate text conditioned on the current description and prompting the LLM to add, remove, or update directives so the next draft better matches the user's demonstration (up to $S=5$ rounds); a breakdown step splits the description into individual preference components; and consistency verification scores each component against relevant prior user demonstrations on a $-2$ to $+2$ scale and prunes components below a threshold $v$. The stored components are re-aggregated into a fresh description at each new user interaction, and this description is what lets the assistant generate in the user's voice while remaining inspectable and editable.

What would settle it

Run the same two-way comparisons with a larger panel of human raters scoring outputs with the paper's own per-preference-component rubric: if the human win rate for PROSE over CIPHER lands nowhere near the 33% PPCM gap (or the 91.8% reported for the PROSE+ICL variant), the advantage is an artifact of the LLM judge agreeing with itself rather than a real gain in respecting user preferences.

Watch

Extended reading notes

Core claim

The paper's core discovery is that how a preference description is produced matters as much as whether one is produced: a single-shot inference pass (the CIPHER approach) yields generic or drifting descriptions, while PROSE's loop of generate, compare, refine homes in on the user's actual voice, and its verification pass removes components that overfit a single sample. The result is measured as a 33% improvement over CIPHER in per-preference-component match (PPCM) of the final generations across two writing tasks, with a mean improvement of 14.8% from iterative refinement and 1.5–1.7% from consistency verification. The paper further finds that preference descriptions and in-context examples are complementary—PROSE handles tone and nuance while ICL captures structural preferences—so combining them gives up to 9% over ICL alone. It also shows that PLUME's metrics (Preference-Similarity for inferred preferences and PPCM for generations) correlate far better with each other than PRELUDE's accuracy and Levenshtein-distance metrics do, which supports trusting the measured differences.

Load-bearing premise

The measured gains rest on trusting GPT-4o to stand in for the human twice: once as the user whose writing defines the ground-truth preferences and once as the judge scoring how well outputs and inferred preferences match those ground truths.

Editorial extensions

If this is right

  • Writing assistants can be personalized from a handful of user writing samples, without fine-tuning or preference rankings, and the learned preferences remain visible and editable by the user.
  • Iterative refinement is the main engine of the gain (14.8% of the total), so any preference-inference method that stops after a single inference pass is leaving most of the available improvement unused.
  • Because PROSE and in-context learning capture complementary preference types—tone and nuance versus explicit structure—combining them is the strongest configuration, reaching up to 9% over ICL alone and 47% over CIPHER.
  • Preference descriptions are roughly one-tenth the prompt tokens of in-context examples at generation time, and consistency verification prunes irrelevant components, so personalized output is also cheaper to produce.
  • PLUME's better-correlated metrics (Preference-Similarity and PPCM both reach $\rho \approx 0.73$ across tasks) give future preference-inference research a testbed where measured gains in inferred preferences actually track gains in generated writing.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The 33% headline is measured by GPT-4o judging GPT-4o, and the paper's own 16-participant human study covers only PROSE+ICL rather than PROSE alone, so the size of PROSE's real advantage over CIPHER for human readers is not yet pinned down by direct evidence.
  • The reported 11% performance drop when preference components were reordered by length suggests the preference description is a fragile carrier of information; a representation less sensitive to ordering might transfer PROSE's gains to smaller models, where PROSE currently trails ICL.
  • The refine-and-verify loop is not specific to prose: the same pattern could infer preferences for code formatting, presentation slides, or data visualization from user examples, since it only requires demonstrations to compare against.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces PROSE, a method for inferring user writing preferences from demonstrations, based on two components: iterative refinement of the inferred preference description and consistency verification of preference components across multiple user writing samples. The authors also introduce PLUME, a new assistive-writing benchmark designed to address limitations they identify in the existing PRELUDE benchmark, and they evaluate PROSE with four LLMs (Qwen2.5-7B, Qwen2.5-72B, GPT-4o-mini, GPT-4o) on summarization and email-writing tasks. The main reported result is that PROSE outperforms the CIPHER baseline by 33% on PLUME, with ablations showing that each component contributes positively, and a small human study suggesting that PROSE combined with ICL is preferred over ICL and CIPHER.

Significance. If the headline claims hold, PROSE would be a meaningful advance in preference inference for assistive writing, with practical benefits in interpretability, token efficiency, and sample efficiency relative to ICL and prior edit-based methods. The paper's strengths include a carefully constructed new benchmark (PLUME) that mitigates known flaws in PRELUDE, a systematic ablation design that isolates the contributions of iterative refinement and consistency verification, consistent results across four LLMs, and a human evaluation that provides some evidence for the combined method. However, the validity of the central quantitative claim rests on a self-referential evaluation loop in which GPT-4o serves as both the synthetic user and the judge of preference similarity and preference compliance, and the human study does not directly test the headline PROSE-versus-CIPHER comparison.

major comments (3)
  1. [Section 4.2, Section B, Section 6] The central claim that PROSE 'more accurately infers nuanced human preferences' and improves over CIPHER by 33% is supported only by metrics that use GPT-4o as both the synthetic human user and the LLM-as-a-Judge. Section 4.2 states that PLUME uses GPT-4o as a proxy human user, and Section B defines both Preference-Similarity and PPCM as GPT-4o judge prompts. The human evaluation in Section 6 compares only PROSE+ICL against ICL and against CIPHER, not PROSE Full against CIPHER, and it uses 16 participants; the authors' own limitation statement in Section 6.1 concedes that a full-scale human trial is missing. Because the abstract and Section 7 make a claim about human preferences, this gap is load-bearing: the current evidence does not establish that the 33% improvement transfers to real human preferences. A concrete remedy would be a human evaluation of PROSE Full versus CIPHER on the same tasks, or a validation study showing that PPCM and Preference-Similarity scores correlate with human ratings on held-out generations.
  2. [Section 5] Section 5 states that 'for all LLMs, S and v are determined via a hyperparameter sweep over v∈{0,0.25,0.5,0.75,1} and S∈{2,3,4,5}' and then lists the selected values per LLM. The paper does not state whether this sweep is performed on a validation split that is separate from the PLUME tasks whose metrics are reported in Table 2. If the same tasks are used for both hyperparameter selection and final evaluation, the reported ablation results and the 33% improvement are susceptible to selection bias. Please clarify the validation protocol; if no held-out split was used, re-run the evaluation with hyperparameters fixed on a held-out task or topic, or report the sensitivity of the main conclusions to the sweep.
  3. [Section 6, Table 2] The 33% improvement over CIPHER conflates the PROSE algorithm with the re-engineered prompt templates. Table 2 shows that PROSECE, which uses PROSE's prompts but single-step inference and no consistency verification, already achieves a mean PPCM of 0.74 versus 0.14 for CIPHER-1; the isolated contribution of the PROSE algorithm over PROSECE is the 12% difference between PROSE Full and PROSECE. The abstract and Section 7 attribute all of the 33% to PROSE, which overstates the algorithmic contribution. The authors should either report the prompt-engineering contribution separately or explicitly state that the 33% reflects the combination of PROSE and the improved prompts.
minor comments (5)
  1. [Algorithm 2, Section 3.2] In Algorithm 2, line 19, the verification call passes \hat{p}^s_{desc} to the verification prompt, but it should pass the preference component \hat{p}_c that is being verified; the same inconsistency appears in the equation for v_score in Section 3.2.
  2. [Introduction, first paragraph] The phrase 'share share with my friends' contains a duplicated word; please correct it.
  3. [Section 6.1] The statement that 'PROSE Full used 5.87x (prompt) and 6.07x (generated) more tokens' is ambiguous; it should read '5.87 times as many prompt tokens and 6.07 times as many generated tokens'.
  4. [Table 2 and RQ1] The text in RQ1 states that each component of iterative refinement improves performance, but in Table 2 the Pref. Sim. value for PROSEu,a,S>1 (1.43) is slightly below that of PROSEu,a (1.47). The claim holds consistently only on PPCM; please clarify that the improvements are primarily on the generation-quality metric.
  5. [Section 6] The reported human win rates (69.4% and 91.8%) would be more informative with confidence intervals or a significance test; please add them or state that the study is descriptive.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the GPT-4o-as-user-and-judge setup is a validity limitation, not a by-construction reduction of PROSE's result to its inputs.

full rationale

The paper's derivation chain is not circular. PROSE infers a preference description from user demonstrations via iterative refinement and consistency verification; the PLUME ground-truth preference sets are handcrafted lists (Section D, Table 9), and the synthetic human is only used to instantiate demonstrations from those lists (Sections 4.2 and 5; Figure 9). The evaluation metrics, Preference-Similarity and PPCM, are LLM-as-a-Judge prompts that compare inferred descriptions and generations against those handcrafted true preferences (Section B, Figures 4-5), not against PROSE's own outputs. Using GPT-4o as both synthetic user and judge is a real external-validity limitation: it risks measuring agreement with GPT-4o's stylistic priors rather than human preference, and the authors concede that 'a full-scale human trial would provide a greater understanding' (Section 6.1). However, this is not a by-construction equivalence: no parameter is fitted to the evaluation metric and then reported as a prediction, no self-citation carries the argument, and the human evaluation with 16 participants provides partial independent support (Section 6). The order-sensitivity remark citing Aroca-Ouellette et al. (2021) is incidental and not load-bearing. Therefore the central algorithmic claim retains independent content.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The central claims rest on assumptions about the validity of a fully synthetic evaluation loop (GPT-4o as user and judge) and on two hyperparameters tuned on the benchmark itself. No new theoretical entities are introduced.

free parameters (2)
  • S (maximum refinement steps) = 5 for all LLMs
    Selected by a hyperparameter sweep over S in {2,3,4,5}; limits how many iterative refinement rounds PROSE performs, directly affecting the inferred preference description.
  • v (consistency verification threshold) = 0.25 for Qwen2.5-7B, 0.5 for GPT-4o-mini and GPT-4o, 0.75 for Qwen2.5-72B
    Selected by a sweep over v in {0, 0.25, 0.5, 0.75, 1}; determines which preference components are pruned during consistency verification.
assumptions (3)
  • domain assumption GPT-4o can act as a faithful synthetic human user whose writing reflects the ground-truth preference sets.
    Section 4.2: PLUME uses GPT-4o as a proxy human; all demonstrations and ground-truth-conditioned outputs in the main experiments come from this proxy.
  • domain assumption GPT-4o-as-a-Judge produces valid Preference-Similarity and PPCM scores that track true human preferences.
    Sections B.1 and B.2 define both metrics as GPT-4o judge prompts; the headline 33% improvement is computed from these scores.
  • domain assumption The handcrafted PLUME preference components are orthogonal, non-default, and equally detectable by the judge model.
    Section 4.2 reworks preferences to be orthogonal and to avoid LLM default behavior; the benchmark's ability to separate methods depends on this design.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Aligning LLMs by Predicting Preferences from User Writing Samples." pith.science (2026). https://pith.science/paper/ADFMFPAE

@misc{pith2026250523815,
  author       = {Pith},
  title        = {Pith review of: Aligning LLMs by Predicting Preferences from User Writing Samples},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ADFMFPAE}},
  note         = {Machine review of arXiv:2505.23815}
}
read the original abstract

Accommodating human preferences is essential for creating aligned LLM agents that deliver personalized and effective interactions. Recent work has shown the potential for LLMs acting as writing agents to infer a description of user preferences. Agent alignment then comes from conditioning on the inferred preference description. However, existing methods often produce generic preference descriptions that fail to capture the unique and individualized nature of human preferences. This paper introduces PROSE, a method designed to enhance the precision of preference descriptions inferred from user writing samples. PROSE incorporates two key elements: (1) iterative refinement of inferred preferences, and (2) verification of inferred preferences across multiple user writing samples. We evaluate PROSE with several LLMs (i.e., Qwen2.5 7B and 72B Instruct, GPT-mini, and GPT-4o) on a summarization and an email writing task. We find that PROSE more accurately infers nuanced human preferences, improving the quality of the writing agent's generations over CIPHER (a state-of-the-art method for inferring preferences) by 33\%. Lastly, we demonstrate that ICL and PROSE are complementary methods, and combining them provides up to a 9\% improvement over ICL alone.

Figures

Figures reproduced from arXiv: 2505.23815 by the authors.

Figure 1
Figure 1. Overview of PROSE. (top) The user provides a task description and demonstration to PROSE, which executes iterative refinement and then a consistency verification step. Iterative refinement updates the inferred preference description by generating a writing sample conditioned on the current preference description, comparing the sample to the user’s demonstration, and updating the preference description to better desc… view at source ↗
Figure 2
Figure 2. Preference compliance performance (PPCM) for CIPHER-1, in-context learning (ICL), PROSE, Oracle, [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Generation quality (PPCM) for PROSE, CIPHER-1, in-context learning (ICL), and PROSE+ICL by [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: LLM-as-a-Judge prompts to assess the similarity between the true and inferred preferences. The system prompt is prepended to the user prompt following the LLM’s chat template. “<...>” indicates that the text is formatted from a variable. inferred preference i one of th…
Figure 5
Figure 5. Figure 5: LLM-as-a-Judge prompts for the per preference-component match metric (PPCM) used in the PLUME environment. The system prompt is prepended to the user prompt following the LLM’s chat template. “<...>” indicates that the text is formatted from a variable. agent completio…
Figure 6
Figure 6. Figure 6: Performance for PROSE, CIPHER-1, in-context learning (ICL), and PROSE+ICL given different numbers of [PITH_FULL_IMAGE:figures/full_fig_p021_6.png]
Figure 7
Figure 7. Figure 7: The prompts used by PROSE to infer the user preferences from user examples are provided in Appendix [PITH_FULL_IMAGE:figures/full_fig_p025_7.png]
Figure 8
Figure 8. Figure 8: LLM prompts for preference inference on PLUME’s summarization and e-mail writing tasks. The system prompt is prepended to each user prompt following the LLM’s chat template. “{...|...}” means that of the two options is selected based on the task and “<...>” indicates t…
Figure 8
Figure 8. Figure 8: LLM prompts for preference inference on the PLUME’s summarization and e-mail writing tasks. The system prompt is prepended to each user prompt following the LLM’s chat template. “{...|...}” means that of the two options is selected based on the task and “<...>” indicat…
Figure 9
Figure 9. Figure 9: LLM prompts for the synthetic human on the PLUME’s summarization and e-mail writing tasks. The system prompt is prepended to the user prompt following the LLM’s chat template. “{...|...}” means that of the two options is selected based on the task and “<...>” indicates…
Figure 10
Figure 10. Figure 10: LLM prompts for the no preference baseline in the PLUME environment. The system prompt is prepended to the user prompt following the LLM’s chat template. “<...>” indicates that the text is formatted from a variable. task content refers to the content of either the art…
Figure 11
Figure 11. Figure 11: LLM prompts for the in-context learning baseline in the PLUME environment. The system prompt is prepended to the user prompt following the LLM’s chat template. “<...>” indicates that the text is formatted from a variable, and completion l refers to an example completi…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

36 extracted references · 12 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    PROST : P hysical reasoning about objects through space and time

    Aroca-Ouellette, S., Paik, C., Roncone, A., and Kann, K. PROST : P hysical reasoning about objects through space and time. In Zong, C., Xia, F., Li, W., and Navigli, R. (eds.), Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pp.\ 4597--4608, Online, August 2021. Association for Computational Linguistics. doi:10.18653/v1/2021.fi...

    doi:10.18653/v1/2021.findings-acl.404 2021

  3. [3]

    Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A.,...

    1901

  4. [4]

    Art or artifice? large language models and the false promise of creativity

    Chakrabarty, T., Laban, P., Agarwal, D., Muresan, S., and Wu, C.-S. Art or artifice? large language models and the false promise of creativity. In Proceedings of the CHI Conference on Human Factors in Computing Systems, pp.\ 1--34, 2024

    work page 2024

  5. [5]

    Aligning llm agents by learning latent preference from user edits

    Gao, G., Taymanov, A., Salinas, E., Mineiro, P., and Misra, D. Aligning llm agents by learning latent preference from user edits. In Thirty-eigth Conference on Neural Information Processing Systems, 2024. URL https://arxiv.org/abs/2404.15269

    arXiv 2024

  6. [6]

    Levenshtein, V. I. Binary codes capable of correcting deletions, insertions and reversals. Soviet Physics Doklady, 10: 0 707, February 1966

    work page 1966

  7. [7]

    Inference-time intervention: Eliciting truthful answers from a language model

    Li, K., Patel, O., Vi\' e gas, F., Pfister, H., and Wattenberg, M. Inference-time intervention: Eliciting truthful answers from a language model. In Oh, A., Naumann, T., Globerson, A., Saenko, K., Hardt, M., and Levine, S. (eds.), Advances in Neural Information Processing Systems, volume 36, pp.\ 41451--41530. Curran Associates, Inc., 2023

    work page 2023

  8. [8]

    Lin, X., Dai, Z., Verma, A., Ng, S.-K., Jaillet, P., and Low, B. K. H. Prompt optimization with human feedback, 2024. URL https://arxiv.org/abs/2405.17346

    arXiv 2024

Show all 36 references
  1. [9]

    Lindsey, J., Gurnee, W., Ameisen, E., Chen, B., Pearce, A., Turner, N. L., Citro, C., Abrahams, D., Carter, S., Hosmer, B., Marcus, J., Sklar, M., Templeton, A., Bricken, T., McDougall, C., Cunningham, H., Henighan, T., Jermyn, A., Jones, A., Persic, A., Qi, Z., Thompson, T. B...

    2025

  2. [10]

    Llm-powered hierarchical language agent for real-time human-ai coordination

    Liu, J., Yu, C., Gao, J., Xie, Y., Liao, Q., Wu, Y., and Wang, Y. Llm-powered hierarchical language agent for real-time human-ai coordination. In Dastani, M., Sichman, J. S., Alechina, N., and Dignum, V. (eds.), Proceedings of the 23rd International Conference on Autonomous Ag...

    2024

  3. [11]

    Gpt-4 technical report

    OpenAI. Gpt-4 technical report. ArXiv, abs/2303.08774, 2023. URL https://arxiv.org/abs/2303.08774

    2023 arXiv

  4. [12]

    Training language models to follow instructions with human feedback

    Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35: 0 27730--27744, 2022

    2022

  5. [13]

    Z., Sumers, T

    Peng, A., Bobu, A., Li, B. Z., Sumers, T. R., Sucholutsky, I., Kumar, N., Griffiths, T. L., and Shah, J. A. Preference-conditioned language-guided abstraction. In Proceedings of the 2024 ACM/IEEE International Conference on Human-Robot Interaction, pp.\ 572--581, 2024

    2024

  6. [14]

    and Hruschka, E

    Pezeshkpour, P. and Hruschka, E. Large language models sensitivity to the order of options in multiple-choice questions. In Duh, K., Gomez, H., and Bethard, S. (eds.), Findings of the Association for Computational Linguistics: NAACL 2024, pp.\ 2006--2017, Mexico City, Mexico, ...

    2024 doi

  7. [15]

    Language models are unsupervised multitask learners

    Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., and Sutskever, I. Language models are unsupervised multitask learners. 2019

    2019

  8. [16]

    D., Ermon, S., and Finn, C

    Rafailov, R., Sharma, A., Mitchell, E., Manning, C. D., Ermon, S., and Finn, C. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36, 2024

    2024

  9. [17]

    Lamp: When large language models meet personalization, 2024

    Salemi, A., Mysore, S., Bendersky, M., and Zamani, H. Lamp: When large language models meet personalization, 2024. URL https://arxiv.org/abs/2304.11406

    2024 arXiv

  10. [18]

    Whose opinions do language models reflect? In International Conference on Machine Learning, pp.\ 29971--30004

    Santurkar, S., Durmus, E., Ladhak, F., Lee, C., Liang, P., and Hashimoto, T. Whose opinions do language models reflect? In International Conference on Machine Learning, pp.\ 29971--30004. PMLR, 2023

    2023

  11. [19]

    Show, don't tell: Aligning language models with demonstrated feedback

    Shaikh, O., Lam, M., Hejna, J., Shao, Y., Bernstein, M., and Yang, D. Show, don't tell: Aligning language models with demonstrated feedback. arXiv preprint arXiv:2406.00888, 2024

    2024 arXiv

  12. [20]

    Shashidhar, S., Chinta, A., Sahai, V., and Tur, D. H. Unsupervised human preference learning. In Al-Onaizan, Y., Bansal, M., and Chen, Y.-N. (eds.), Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp.\ 3412--3445, Miami, Florida, USA, No...

    2024 doi

  13. [21]

    Pmg : Personalized multimodal generation with large language models, 2024

    Shen, X., Zhang, R., Zhao, X., Zhu, J., and Xiao, X. Pmg : Personalized multimodal generation with large language models, 2024. URL https://arxiv.org/abs/2404.08677

    2024 arXiv

  14. [22]

    M., Lowe, R., Voss, C., Radford, A., Amodei, D., and Christiano, P

    Stiennon, N., Ouyang, L., Wu, J., Ziegler, D. M., Lowe, R., Voss, C., Radford, A., Amodei, D., and Christiano, P. Learning to summarize from human feedback. In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS '20, Red Hook, NY, US...

    2020

  15. [23]

    Principle-driven self-alignment of language models from scratch with minimal human supervision

    Sun, Z., Shen, Y., Zhou, Q., Zhang, H., Chen, Z., Cox, D., Yang, Y., and Gan, C. Principle-driven self-alignment of language models from scratch with minimal human supervision. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL https://openreview....

    2023

  16. [24]

    D., Yang, Y., and Gan, C

    Sun, Z., Shen, Y., Zhang, H., Zhou, Q., Chen, Z., Cox, D. D., Yang, Y., and Gan, C. SALMON : Self-alignment with instructable reward models. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=xJbsmB8UMx

    2024

  17. [25]

    Democratizing large language models via personalized parameter-efficient fine-tuning, 2024

    Tan, Z., Zeng, Q., Tian, Y., Liu, Z., Yin, B., and Jiang, M. Democratizing large language models via personalized parameter-efficient fine-tuning, 2024. URL https://arxiv.org/abs/2402.04401

    2024 arXiv

  18. [26]

    M., Thiergart, L., Leech, G., Udell, D., Vazquez, J

    Turner, A. M., Thiergart, L., Leech, G., Udell, D., Vazquez, J. J., Mini, U., and MacDiarmid, M. Steering language models with activation engineering, 2024. URL https://arxiv.org/abs/2308.10248

    2024 arXiv

  19. [27]

    Qwen2.5 technical report

    Yang, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Li, C., Liu, D., Huang, F., Wei, H., Lin, H., Yang, J., Tu, J., Zhang, J., Yang, J., Yang, J., Zhou, J., Lin, J., Dang, K., Lu, K., Bao, K., Yang, K., Yu, L., Li, M., Xue, M., Zhang, P., Zhu, Q., Men, R., Lin, R., Li, ...

    2024 arXiv

  20. [28]

    Y., Hartmann, B., and Yang, Q

    Zamfirescu-Pereira, J., Wong, R. Y., Hartmann, B., and Yang, Q. Why johnny can’t prompt: how non-ai experts try (and fail) to design llm prompts. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, pp.\ 1--21, 2023

    2023

  21. [29]

    Q., and Artzi, Y

    Zhang*, T., Kishore*, V., Wu*, F., Weinberger, K. Q., and Artzi, Y. Bertscore: Evaluating text generation with bert. In International Conference on Learning Representations, 2020. URL https://openreview.net/forum?id=SkeHuCVFDr

    2020

  22. [30]

    E., and Stoica, I

    Zheng, L., Chiang, W.-L., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., Li, D., Xing, E., Zhang, H., Gonzalez, J. E., and Stoica, I. Judging LLM -as-a-judge with MT -bench and chatbot arena. In Thirty-seventh Conference on Neural Information Processing Systems Da...

    2023

  23. [31]

    I., Han, Z., Paster, K., Pitis, S., Chan, H., and Ba, J

    Zhou, Y., Muresanu, A. I., Han, Z., Paster, K., Pitis, S., Chan, H., and Ba, J. Large language models are human-level prompt engineers. arXiv preprint arXiv:2211.01910, 2022

    2022 arXiv

  24. [32]

    Hydra: Model factorization framework for black-box llm personalization, 2024

    Zhuang, Y., Sun, H., Yu, Y., Qiang, R., Wang, Q., Zhang, C., and Dai, B. Hydra: Model factorization framework for black-box llm personalization, 2024. URL https://arxiv.org/abs/2406.02888

    2024 arXiv

  25. [33]

    M., Stiennon, N., Wu, J., Brown, T

    Ziegler, D. M., Stiennon, N., Wu, J., Brown, T. B., Radford, A., Amodei, D., Christiano, P., and Irving, G. Fine-tuning language models from human preferences. arXiv preprint arXiv:1909.08593, 2019

    1909 arXiv

  26. [34]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  27. [35]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  28. [36]

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.