REVIEW 5 major objections 5 minor 26 references
Expectation Alignment of Language Models for Real-World User Expectations
T0 review · 5 major / 5 minor · reviewed 2026-08-02 · deepseek-v4-flash
Pith's one-line read The paper establishes a measurable alignment gap: even the strongest LLM averages 2.72/5 on real user expectations, and expectation coverage tops out near 40 percent.
desk verdict A genuinely new evaluation angle, undercut by an unvalidated extraction step: the paper's strongest numbers likely overstate how badly models miss real user expectations. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
Expectation extraction is the load-bearing mechanism: a prompted LLM converts post-hoc user follow-ups into standalone criteria ('be concise,' 'mention my resume,' 'use subtle humor'), which then serve as rubrics instead of expert checklists or user simulations. ExpectBench is the benchmark built on those rubrics; its ten discovered dimensions (practicality, compliance, empathy, etc.) carry the analysis. LENS is the generative mechanism: a small observer model encodes the query into latent expectation tokens, a lightweight trainable projector maps them into the main LLM's representation space, and the frozen main LLM generates conditioned on that signal, with only the projector trained. Expe
What would settle it
Take a random sample of ExpectBench instances and have the original users (or blind human annotators) judge each extracted expectation: were these 'expectations' things they actually wanted from the first response? If a large share are judged inventions, restatements of the query, or post-hoc rationalizations, then the low scores and LENS gains measure the extractor's construct rather than real user expectations. A cheaper version: measure inter-annotator agreement on expectation extraction from the same follow-up messages; if agreement is low, the rubrics are unstable.
Extended reading notes
Core claim
The central discovery is a systematic gap between benchmark competence and what users actually want. The authors mine 4.8 million real human–AI conversations, extract expectations from follow-up messages (corrections, clarifications, complaints), and turn them into rubrics for ExpectBench (12,000 conversations, 34,876 expectations). Six models score low: averages run from about 2.0 to 2.7 out of 5, with the strongest at 2.72. Because an LLM judge agrees with human raters when given explicit expectations (κ≈0.63, r≈0.96), the low scores are treated as genuine. Predicting expectations from the initial query alone reaches about 40% coverage even with ten predictions, versus an average of 2.91 t
Load-bearing premise
The benchmark's validity rests on the assumption that expectations extracted by an LLM from follow-up messages match what the user actually expected from the original response; no per-item human agreement is reported for extraction itself.
Editorial extensions
If this is right
- Benchmark scores and user satisfaction can diverge sharply: a model can be factually correct and still fail the user's actual criteria.
- Satisfaction and anticipation are separable capabilities; generation improvements alone will not close the gap if the model cannot predict what users value, since coverage at a budget of 10 stays near 40%.
- Expectation-level evaluation is feasible at scale: an LLM judge given explicit expectation rubrics agrees with human raters (κ≈0.63, r≈0.96), enabling large-scale user-centric evaluation.
- Lightweight latent conditioning works: training only a small projector while freezing the main model yields consistent satisfaction gains of about 0.2 on a 5-point scale across two 7–8B model families and most dimensions.
- Dimension-level results identify where to focus: surface criteria like coherence and consistency are comparatively easier, while practical grounding, versatility, and compliance remain the hardest to satisfy and to predict.
Reading between the lines
- If the extraction assumption holds, current post-training recipes that optimize benchmarks or human-preference proxies may not transfer to real-user satisfaction; expectation-aware data collection could become a standard ingredient.
- Expectation coverage could be repurposed as a standalone, query-only capability test: a model that cannot anticipate expectations at near-ceiling coverage is unlikely to satisfy them, so coverage may predict downstream usefulness without needing generation.
- The 40% coverage ceiling despite a budget of 10 (versus 2.91 average true expectations) suggests the bottleneck is not output length but prior knowledge about users; conditioning on user history or demographic context is the natural next experiment.
- One can test LENS's mechanism directly: if the latent tokens truly carry expectation information, ablating or scrambling them should degrade satisfaction on expectation-heavy dimensions more than on surface dimensions; the paper does not report this ablation.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper argues that standard benchmarks overstate LLM competence because they ignore the diverse, often implicit expectations real users bring to open-ended interactions. It introduces ExpectBench, constructed from 4.8M WildChat conversations by extracting user expectations from follow-up messages using an LLM, filtering to 12,000 multi-turn instances with 34,876 expectations. The authors report that six LLMs achieve low expectation satisfaction (GPT-4o scores 2.72/5 on a 1–5 scale) and that even with a prediction budget of 10, the best model covers only about 40% of gold expectations, concluding that current models fundamentally fail to anticipate and satisfy user expectations. They then propose LENS, a two-stage framework in which a frozen observer produces latent expectation tokens, a trained projector feeds them into a frozen main LLM, and claim consistent expectation-satisfaction improvements (e.g., 2.17→2.37 for LLaMA-3.1-8B, 2.39→2.50 for Mistral-7B).
Significance. If the construct validity of the extracted expectations holds, ExpectBench would be a valuable and novel resource: it uses real user follow-up behavior rather than expert rubrics, provides a multilingual, multi-turn corpus, separates the evaluation target from the judge, and reports a human-consistency check for the scoring step (κ=0.626, r=0.963 on 283 expectations). LENS is also a lightweight and reproducible recipe that keeps the base LLM frozen and trains only a projector. However, the significance of all three contributions—the benchmark, the diagnosis of low satisfaction/coverage, and the LENS gains—depends on whether the gold expectation set E(q) faithfully represents expectations the user held before seeing the initial response, an assumption that is not directly validated. The paper is transparent about its LLM-based extraction pipeline and its human check of scoring only, which makes the gap clear but unresolved.
major comments (5)
- [§3.1, Appendix A.2.1, A.2.2] The benchmark's ground truth is LLM-extracted, and no per-item human agreement on extraction is reported. Appendix A.5.2 validates only the scoring of already-given expectations; Appendix A.1 describes a general manual review, not a measurement of extractor precision/recall. The running example in A.2.2 itself includes 'Can you add that I have just returned to America...' as an extracted expectation, which the prompt's own rule ('Do not represent a new or extended request') would seem to exclude. If many extracted expectations are requests first made after the initial response, then the low satisfaction scores (§4.1) and the 40% coverage ceiling (§5.2) measure the model's inability to predict future turns, not its failure to satisfy pre-existing user expectations. Please report per-item extraction agreement on a sample and quantify the fraction of expectations that are expressed or infer
- [Eq. (8), Appendix C.1] The coverage metric and the headline '40% coverage ceiling' are computed with a semantic matcher (BART-MNLI) using a threshold τ=0.8. No sensitivity analysis or human validation of this matching threshold is provided. Because both the gold expectations and model predictions are free-form paraphrases, the coverage value is directly controlled by τ; a different threshold could materially change the reported ceiling. Please report coverage across a range of τ values and/or validate a sample of predicted/gold pairs against human judgment. Without this, the claim that expectation prediction is 'intrinsically challenging' is not robust.
- [§3.5, §6, Table 2] There is a partial circularity in the evaluation loop: GPT-4o is used to extract gold expectations, to score satisfaction, and (via the observer model prompted with the same extraction rubric) to define the latent signal for LENS; the only human validation is the scoring step on 283 expectations. The LENS gains in Table 2 may therefore reflect better alignment with GPT-4o's scoring preferences rather than with real users. The manuscript should include a human evaluation of LENS outputs on a held-out sample, or at least a per-dimension human audit of the extracted expectations used to construct the benchmark.
- [Table 2, §6.2] The LENS improvements are small (overall +0.20 and +0.11 on a 5-point scale) and are reported without confidence intervals or significance tests. Given that the evaluator's Cohen's κ is 0.626, the consistency claim for LENS (RQ3) is not statistically established. Please report bootstrap confidence intervals or per-model significance tests for the overall and dimension-level gains, and clarify whether the improvements are consistent across the 283 human-validated expectations.
- [§5.2, Figure 7] The claim that 'the best-performing model still achieves limited coverage' is interpreted as a fundamental limitation, but no human upper bound or chance baseline is reported. Users on average express 2.91 expectations, and the model is asked to predict 10; a coverage of 40% means matching roughly 1.2 of 2.91 expectations per query. Without a human-prediction baseline on the same task, the conclusion that models are 'intrinsically' poor at anticipation is not yet supported. A small human study (e.g., annotators given only the initial query and asked to list likely expectations) would sharpen the diagnostic claim.
minor comments (5)
- [Appendix A.4] The text says the benchmark is limited to 'the top 10 most frequent languages', but Figure 4(b) and the surrounding text list nine languages. Please reconcile.
- [§4 and Table 3] The model is called 'DeepSeek-R1-7B' in the main text but 'DeepSeek-R1-Distill-Qwen-7B' in Table 3; please use a single consistent name.
- [Figures 5, 11, 12] Several figures contain unreadable character-substitution artifacts (e.g., '/uni000...' sequences), making them impossible to interpret. Please regenerate with proper font embedding.
- [Appendix A.5.2] The heatmap in Figure 10 has empty bins and the color scale is not defined; a normalized agreement matrix or a weighted kappa would be easier to interpret.
- [§3.3] The dimension-discovery process is described as iterative, but the number of subsets (15) and the stop criterion for convergence are not given in the main text; please state them in Appendix A.3.
Circularity Check
Expectation-prediction ceiling is an artifact of response-dependent gold labels.
-
self definitional
[Sections 3.1 and 5.1 (Eq. 1 / Eq. 8)]
"we treat follow-up messages as natural supervision signals that expose gaps between what the user expected and what the response provided. ... given an initial user query q, a model is asked to predict a set of expectations ... These predicted expectations are compared against the ground-truth expectation set E(q), which is extracted from follow-up user feedback."
E(q) is constructed from follow-up messages that react to a specific, unobserved original response. Thus the gold expectations encode deficiencies of that response, not expectations independent of it. The prediction task supplies only q, never the original response, so perfect coverage is impossible in general; the reported ~40% ceiling is partly forced by this target definition rather than by models' inability to anticipate user expectations. The paper's conclusion that 'expectation prediction is intrinsically challenging' is therefore an artifact of the construction. The example in A.2.2 confirms that post-hoc revision requests (e.g., 'Can you add that I have just returned to America...') are converted into gold expectations, making the task 'guess future revision requests' rather than '
full rationale
The paper's central derivation chain is not a formal derivation but an empirical benchmark construction. Expectation extraction is anchored in real WildChat follow-up messages, and the scoring step is validated against human annotators (κ=0.626, r=0.963), so the satisfaction scores (e.g., GPT-4o 2.72/5) have independent empirical content. LENS is a standard supervised pipeline (train projector on gold expectations, evaluate on held-out test) and does not reduce to its inputs by construction. There is no load-bearing self-citation chain or imported uniqueness theorem. The one genuine circularity is in the expectation-prediction diagnostic: the gold set E(q) is defined as gaps exposed by follow-ups to an original response, while the predictor sees only the query. Low coverage is therefore partly an artifact of the target definition, undermining the headline claim that models cannot anticipate what users expect. Because this affects a central claim but not the entire benchmark, the overall circularity score is 6.
Assumptions & free parameters
free parameters (3)
- semantic matching threshold tau =
0.8
- number of latent expectation tokens =
4
- rejected response length threshold =
450 tokens
assumptions (5)
- domain assumption Follow-up messages in WildChat reliably express the user's expectations about the preceding response
- domain assumption LLM extraction with the provided prompt yields faithful, complete, and non-redundant expectations
- domain assumption GPT-4o scores with explicit expectation rubrics approximate human satisfaction judgments
- domain assumption WildChat is representative of real-world LLM use
- domain assumption The ten-dimension taxonomy is stable and complete
invented entities (1)
-
Latent expectation representation z(q)/h(q) and expectation tokens
Cite this review
Pith. "Pith review of Expectation Alignment of Language Models for Real-World User Expectations." pith.science (2026). https://pith.science/paper/4QMW5NXL
@misc{pith2026260720485,
author = {Pith},
title = {Pith review of: Expectation Alignment of Language Models for Real-World User Expectations},
year = {2026},
howpublished = {\url{https://pith.science/paper/4QMW5NXL}},
note = {Machine review of arXiv:2607.20485}
}
read the original abstract
Large language models (LLMs) have demonstrated remarkable performance on standard benchmarks, yet it remains largely unexplored whether they truly meet user expectations. Existing evaluation approaches, relying on model heuristics, expert rubrics, or user simulation, fail to capture the diversity and subtlety of real human expectations, causing models to appear competent while misaligning with what users actually seek. We present the first systematic study of user expectations in real-world LLM interactions, proposing a principled procedure to extract semantically rich expectations and introducing ExpectBench, a benchmark grounded in real user expectations. Analyses reveal that current LLMs struggle to satisfy and anticipate what users hope to obtain, highlighting a fundamental source of misalignment. Building on these observations, we propose LENS, a lightweight latent expectation-aware response generation framework. LENS enables models to internalize user expectations and generate better-aligned responses, consistently improving expectation satisfaction and underscoring the importance of explicitly modeling user expectations for realistic human-AI alignment.
Figures
Figures from the paper (8 more)
Reference graph
Works this paper leans on
-
[1]
Chivalry, Warrior Culture, High, Spiritual
-
[2]
Legalistic, Mubarizun, Longbow Competitions, Scholarship
-
[3]
Defensive Tactics, Royal Army, Mountain Herding, Expert Artisans
-
[4]
Steppe Tolerance, Forest Folk, Druzhina, Forest Wardens
-
[5]
Sacred Hunts, Frostspire, Undead, Research
-
[6]
Quarry, Fervent Temple Builders, Highland Warriors, City Keepers User Expectations
-
[8]
Provide geographic context by specifying neighboring countries and their relative positions
-
[9]
Model Answer: Sure, here is a brief description for each of the countries based on your sketches:
Offer historical or cultural references by suggesting real nations and civilizations that can serve as inspiration for each fictional country. Model Answer: Sure, here is a brief description for each of the countries based on your sketches:
Show all 26 references
-
[10]
- List of Races: Humans, Noble Elves, Halflings
The Kingdom of Athelar - Coat of Arms and Colors: A silver lion rearing on a field of deep blue, symbolizing valor and nobility. - List of Races: Humans, Noble Elves, Halflings. - Description: A realm grounded in a culture of chivalry and honor, Athelar is renowned for its kni...
-
[11]
- List of Races: Humans, Gnomes, Half-Orcs
The Confederation of Farenthia - Coat of Arms and Colors: A golden scales on a field of emerald green, symbolizing justice and balance. - List of Races: Humans, Gnomes, Half-Orcs. - Description: Known for its legalistic society, Farenthia upholds rule of law and scholarly purs...
-
[12]
- List of Races: Dwarves, Mountain Elves, Hill Giants
The Duchy of Mordhima - Coat of Arms and Colors: A white mountain on a field of dark grey, symbolizing sturdiness and elevation. - List of Races: Dwarves, Mountain Elves, Hill Giants. - Description: Nestled among towering peaks, Mordhima is famed for its defensive tactics and ...
-
[13]
- List of Races: Wood Elves, Centaurs, Fauns, and Human Forest Folk
The Realm of Sylvaine - Coat of Arms and Colors: A silver tree on a field of deep forest green, symbolizing nature and endurance. - List of Races: Wood Elves, Centaurs, Fauns, and Human Forest Folk. - Description: Known for its appreciation of nature and steppes, Sylvaine is h...
-
[14]
- List of Races: Frost Giants, Necromancers, Wraiths, Mage Scholars
The Domain of Frostspire - Coat of Arms and Colors: An icy spear topped with a glowing rune on a field of black, symbolizing the unknown and ethereal. - List of Races: Frost Giants, Necromancers, Wraiths, Mage Scholars. - Description: Dominated by sacred hunts and arcane resea...
-
[15]
Between the Lines of Duty
The Republic of Eyrethia - Coat of Arms and Colors: A golden hammer crossed by a silver sword on a field of crimson, symbolizing toil and strength. - List of Races: Humans, Dwarves, Highland Minotaurs. - Description: Noted for its vast quarries and fervent temple building, Eyr...
-
[20]
Between the Lines
The user expects thematic analysis to explore deeper meanings and connections within the narrative and character arcs. Model Answer: Title: Between the Lines of Duty Series: Crossover - Lines of Duty x Between the Lines Genre: Drama, Crime, Thriller Synopsis: In the heart of L...
-
[21]
The user community expects CW4Kids to develop a distinct identity separate from Kids WB, particularly through updated bumpers
-
[22]
There is a clear preference for maintaining popular legacy shows such as Batman Beyond and Static Shock during the transition
-
[23]
Users anticipate and welcome the introduction of new programs, especially TMNT: Fast Forward, as signals of meaningful change
-
[24]
Between the Lines of Duty
The presentation quality and visual branding are viewed as critical to audience engagement and long-term acceptance. Model Answer: Forum: Cartoon Craze Central 30 Expectation Alignment of Language Models for Real-World User Expectations User 1: SaturdayMorningMaster Subject: C...
-
[25]
The user expects the incorporation of aging and possibly more cynical character developments for the original series leads
-
[26]
The user is interested in the narrative dynamics of inter-generational collaboration and conflicts, particularly around media involvement in police investigations
-
[27]
The user expects exploration of thematic elements related to flawed systems and procedural weaknesses impacting investigations
-
[28]
The user anticipates the integration of relevant social and political issues and trends within the storyline to maintain contemporary relevance
-
[29]
Between the Lines of Duty
The user expects thematic analysis to explore deeper meanings and connections within the narrative and character arcs. Model Answer: "Between the Lines of Duty" imagines a thrilling crossover between two iconic British television series, blending the tense intrigue of "Between...
2020
-
[30]
35 Expectation Alignment of Language Models for Real-World User Expectations D.3. Prompt for LENS Illustrative Prompt and Output for LENS Stage 1 Observer Prompt: You are required to generate {num_tokens} informative tokens that summarize the user’s implicit expectations based...
-
[2024]
has_expectation
Association for Computing Machinery. ISBN 9798400706851. DeepSeek-AI. Deepseek-r1: Incentivizing reasoning ca- pability in llms via reinforcement learning, 2025. URL https://arxiv.org/abs/2501.12948. GLM, T., Zeng, A., Xu, B., Wang, B., Zhang, C., Yin, D., Rojas, D., Feng, G.,...
2025 arXiv
Reviewed August 2, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.