{"id":"9be32ca7-d41a-4867-a3db-7df7b467cd9a","arxiv_id":"2505.07886","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"PLHF uses a prompt-optimized LLM evaluator trained on a few human-labeled examples to serve as the scoring metric, and reports improved prompt optimization for generative tasks where no clear metric exists.","lead":"The paper introduces PLHF, a prompt-optimization framework that trains a small language model to grade outputs using a handful of human-labeled examples, then uses that grader to optimize the prompt of the main answer-generating model. It reports better output quality than using GPT-4o as a judge across four datasets, but the evaluation for three of them relies on an unvalidated GPT-4o 'pseudo-human' judge.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Public-dataset results rest on an unvalidated GPT-4o pseudo-human judge, so the central claim that PLHF improves output quality is not established for 3 of 4 datasets.","rationale":"The reader's weakest assumption is that the pseudo-human judge is a valid proxy for real human preferences on the newly generated outputs. My read agrees: this is the most load-bearing concern because the public-dataset portion of the central claim, which spans three of the four datasets and all of the cross-domain generalization evidence, depends entirely on an unvalidated LLM judge. The paper's own motivation establishes that GPT-4o can disagree with humans, so the burden is on the authors to show that the specific prompt-optimized GPT-4o judge behaves like a human rater for these tasks. Without such validation, the observed improvements on public datasets may be an artifact of optimizing against one LLM judge and evaluating against another, with shared biases inflating the apparent gains. The industrial SQL-QA experiment, which uses human expert evaluation, is real evidence and prevents the entire framework from being dismissed. However, it does not rescue the broad claim covering all four datasets. I considered other potential concerns, such as missing error bars and the absence of strong baselines, but those are secondary to the validity of the evaluation metric. The concrete test I propose directly settles the concern by measuring pseudo-human agreement with human raters and re-ranking methods under human judgments. Since the reader already made the same identification and assigned a conditional verdict, my read does not move the verdict.","tokens_in":11319,"tokens_out":5808,"duration_ms":59713,"concrete_test":"Select the final outputs of Base LLM, PO with GPT-4o, and PLHF on a held-out sample of 100 inputs per public dataset. Have independent human raters score these outputs using the original task rubrics, for example the Likert satisfaction scale for SGD and the holistic scoring rubric for AES. Compute (a) Pearson or Spearman correlation between the pseudo-human judge scores and human scores, and (b) the ranking of methods under human scores. If the pseudo-human judge correlates weakly with humans (for example r < 0.5) or PLHF does not rank first under human scores, the central claim fails for the public datasets.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.3 states that for the public datasets (SGD, AES-ASAP, AES-2.0), outputs are graded by a pseudo-human judge: GPT-4o with prompt optimizations via DSPy. The paper provides no validation that this judge agrees with human raters on the newly generated outputs. This is especially problematic because Figure 1 motivates PLHF precisely by showing that GPT-4o's verdicts can disagree with human preferences, and the responder in PLHF is optimized against a GPT-3.5 evaluator trained on only a few human labels. The observed gains on public datasets, for example +6.59% to +8.71% on SGD and +8.45% to +8.68% on AES-ASAP, could reflect reward hacking, where the responder learns to exploit the evaluator's and pseudo-judge's shared LLM biases, rather than genuine improvement in human-perceived output quality. Since Table 2's public-dataset rows are the only evidence for the cross-dataset generalization claim, and the pseudo-human judge is itself a prompt-optimized LLM, the central claim that PLHF outperforms prior output grading strategies in output quality is unsupported for these datasets. The industrial SQL-QA result, which uses real human experts, provides positive evidence but only for one domain.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes PLHF, a prompt-optimization framework with two modules: a responder R that generates task outputs and an evaluator E that grades those outputs. E is a base LLM (GPT-3.5) whose prompt is optimized, via DSPy or TextGrad, on a small set of human-labeled (input, output, score) triplets; R's prompt is then optimized with the same PO framework using E as the metric. The authors claim that this requires only a single round of human feedback and that PLHF outperforms prior output-grading strategies, including using GPT-4o as an evaluator, on three public datasets (SGD, AES-ASAP, AES-2.0) and one industrial SQL-QA dataset. The industrial SQL-QA results are judged by real human experts; the public-dataset results are judged by a 'pseudo-human judge' (prompt-optimized GPT-4o).","tokens_in":11507,"tokens_out":5020,"duration_ms":48387,"significance":"The problem addressed—optimizing prompts for generative tasks without a golden metric and with scarce human feedback—is timely and practically important. The two-module architecture is simple and modular, and it can wrap existing PO frameworks. The industrial deployment with real expert labels provides positive evidence that the framework can work in a product setting. However, the public-dataset evidence does not currently establish the central claim, because the pseudo-human judge is an unvalidated LLM and the paper's own motivation is that LLM judges misalign with humans. If the authors add a human-agreement validation or real human evaluation for the public datasets, the paper could make a solid contribution.","major_comments":[{"comment":"The central claim in Section 3.4 that PLHF achieved the best performance 'for all the four datasets' rests on the public-dataset rows of Table 2, but those rows are scored by a pseudo-human judge that is a prompt-optimized GPT-4o model. Section 3.3 states: 'we use GPT-4o with prompt optimizations via DSPy as the pseudo-human judge.' The paper provides no validation that this judge agrees with the actual human raters on the newly generated outputs, even though the original human raters are unavailable. This is not a minor concern: Figure 1 motivates PLHF precisely by showing that GPT-4o's verdicts can disagree with human preferences, and the responder in PLHF is optimized against a GPT-3.5 evaluator trained on only a few human labels. The observed gains on SGD and AES-ASAP, for example +8.71% and +8.68% with TextGrad, could reflect reward hacking toward shared LLM biases rather than genuine improvement in human-perceived output quality. I request a human-agreement study (e.g., correlation and agreement metrics between the pseudo-human judge and human raters on a held-out sample of generated outputs), or a re-evaluation with real human labels for at least the main comparisons. Without this, the cross-dataset generalization claim is unsupported for three of the four datasets.","section":"Section 3.4, Table 2, Appendix D"},{"comment":"Table 2 reports only point estimates (relative improvements) for the responder task, with no error bars, confidence intervals, or significance tests. The only variance information appears in Appendix D, and it is limited to PLHF on SGD and AES-ASAP; no baseline variance is reported. For AES-2.0, the advantage of PLHF over 'PO with GPT-4o' is +2.62% (DSPy) and +4.30% (TextGrad), which may be within noise. To support the claim that PLHF 'outperforms' the baselines, the authors should report multiple independent runs, standard deviations or confidence intervals, and paired significance tests for all methods and datasets.","section":"Section 3.4, Table 2, Appendix D"},{"comment":"The paper's contribution statement in the abstract claims PLHF 'requires only a single round of human feedback to complete the entire prompt optimization process,' but Algorithm 1's outer loop is 'while there are new training samples added into D,' which suggests the framework can consume additional human feedback over time. Section 2 also says 'human grading no greater than linear (with respect to the number of training samples) times.' These statements are not mutually consistent. Please clarify whether the intended setting is a single, fixed training set D or an iterative data-collection process, and state precisely how many human labels are used in the main experiments for each dataset.","section":"Abstract, Section 2, Algorithm 1"}],"minor_comments":[{"comment":"The abstract contains the typo 'näive' (should be 'naive').","section":"Section 3.1.2"},{"comment":"The sentence in Section 3.1.2, 'The dataset, named as AES-2.0, contains 24,000 student-written argumentative essays. Each essay was scored on a scale of 1 to 6 as the holistic rating 1 judged by human experts,' has an awkward and unclear phrasing ('as the holistic rating 1 judged by human experts').","section":"Section 3.1.2"},{"comment":"The reference list entry for the ASAP dataset is malformed: 'Hamner Ben, Morgan Jaison, lynnvandev, Shermis Mark, and Ark Tom Vander. 2012. The hewlett foundation: Automated essay scoring.' The author names are inconsistently ordered and some appear to be GitHub usernames; please fix the citation.","section":"References"},{"comment":"Figure 2's caption contains a garbled phrase: 'where PO can be PO arbitrary prompt optimization method.' This should read 'where PO can be an arbitrary prompt optimization method.'","section":"Figure 2"},{"comment":"For the public datasets, the paper does not state how many training samples (human-labeled triplets) are used in the main experiments; Appendix D sweeps the number of samples up to 100, but the relationship between those curves and the numbers in Table 2 is unclear. Please state the exact sample sizes and the positive-sample thresholds (e.g., 'Labeled Score ≥ 8' in the toy example) for each dataset.","section":"Section 3.2, Appendix D"}],"recommendation":"major_revision","confidential_remarks":"The central issue is the unvalidated pseudo-human judge. This is fixable with additional experiments, so I am not recommending rejection, but the paper's main cross-dataset claim will not be convincing without human-agreement validation or real human evaluation on the public datasets. If the authors cannot provide such validation, they should substantially weaken the claims and present the industrial SQL-QA result as the primary positive evidence."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nYou should know two things about this paper. First, it tackles a genuinely useful problem: prompt optimization for tasks where no clear metric exists, and it packages that into a straightforward two-module design—train a small evaluator on a few human labels via prompt optimization, then use that evaluator as the metric for the main responder. That combination looks new, and the description is clean. Second, the empirical support for the central claim is much weaker than the abstract suggests, because three of the four datasets are evaluated by a pseudo-human judge that is itself a prompt-optimized GPT-4o. The paper never validates that this judge agrees with real humans on the newly generated outputs, which is a serious circularity given that the paper's own motivation is that LLM judges can be unreliable.\n\nWhat's good: the industrial SQL-QA result, where real human experts score the outputs, is the strongest evidence. PLHF beats the GPT-4o evaluator by a large margin (+18.92% vs +10.81% with DSPy, and +18.92% vs +2.70% with TextGrad). That suggests the approach can work in a real deployment. I also appreciate the careful algorithm description and the toy example in Appendix B.\n\nWhere it's soft: the public dataset results in Table 2 are the only cross-domain evidence, and they rest entirely on the pseudo-human judge. The gains on SGD and AES-ASAP could come from the responder learning to exploit shared biases between the GPT-3.5 evaluator and the GPT-4o judge, rather than from genuine human preference. The main table has no error bars or significance tests; the only variance estimates are in Appendix D for a subset of sample sizes. And the closest prior work on human feedback for prompt optimization (Lin et al. 2024) is cited but never used as a baseline. A supervised reward model trained on the same human labels would also be a natural comparison.\n\nThese are fixable. Validate the pseudo-judge against a few hundred human ratings, add confidence intervals, include Lin et al. or a reward model baseline. If the results hold under those conditions, the paper would be a solid contribution.\n\nFor now, it's a conditional accept in spirit. The problem is real, the method is sensible, and one dataset is genuinely positive, but the headline claim needs stronger evidence to be taken at face value. I'd send it to peer review, but with a clear request for a major revision.\n\nBest,","headline":"Clean two-module idea for prompt optimization without a metric, but public-dataset evidence rests on an unvalidated GPT-4o pseudo-judge; the industrial result is the only solid proof.","tokens_in":12110,"tokens_out":4633,"would_cite":false,"duration_ms":40080,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A few human scores can train an evaluator that beats GPT-4o for prompt optimization.","keywords":["prompt optimization","human feedback","LLM evaluator","few-shot learning","RLHF","output quality grading","DSPy","TextGrad"],"falsifier":"Collect real human ratings on a fresh batch of outputs produced by PLHF-optimized and GPT-4o-optimized responders, then compare which set humans prefer; if humans prefer the GPT-4o-optimized outputs on any public dataset, the paper's central claim for that dataset fails. A cheaper check is to measure human–pseudo-human agreement on a held-out sample of responder outputs before trusting the reported public-dataset scores.","tokens_in":11039,"feed_emoji":"🎯","tokens_out":6824,"duration_ms":54533,"temperature":0.7,"pith_summary":"PLHF is a framework for optimizing prompts when no fixed metric defines output quality, such as joke generation, essay scoring, or dialogue satisfaction. It asks human experts to grade a small set of input–output pairs once, then prompt-optimizes a separate evaluator LLM to reproduce those judgments. That evaluator is then used as the metric for optimizing the responder's prompt. Across four datasets, including an industrial SQL question-answering product, PLHF reports the best output quality compared with using GPT-4o, exact matching, or embedding similarity as the grading function, under both DSPy and TextGrad prompt optimizers. The practical claim is that a small GPT-3.5 evaluator tuned on a few human labels can outperform a state-of-the-art general LLM judge at steering prompt optimization.","feed_headline":"Trained evaluator beats GPT-4o as prompt-optimization metric","feed_subtitle":"PLHF learns a task-specific judge from one round of human scores and wins on all four test sets.","key_machinery":"The central mechanism is the evaluator module, a prompt-optimized LLM that acts as a learned grading function for the responder module. The paper builds the evaluator as an auxiliary supervised task: with a trivial metric such as Accuracy or Mean Absolute Error, an existing prompt optimizer (DSPy or TextGrad) tunes the evaluator's prompt to match a handful of human scores. The same optimizer then maximizes the evaluator's scores while searching for the responder's prompt. This makes the missing reward signal measurable in a closed loop, inspired by RLHF but with the reward model's prompt optimized on few human labels rather than its weights trained at scale.","core_discovery":"The paper's central claim is that PLHF, a duo-module design, lets prompt optimization proceed without a well-defined metric by first learning the metric itself. Given a small labeled set of (input, output, score) triplets scored by human experts, PLHF optimizes the prompt of an evaluator LLM so that its scores match the human labels on a conventional loss such as Accuracy or Mean Absolute Error. Once the evaluator is prompt-optimized, PLHF uses it as the grading function to optimize the responder's prompt. The paper reports that PLHF achieves the best output quality on all four datasets, beating prompt optimization with GPT-4o as the evaluator even though PLHF's evaluator is built on GPT-3.5, and beating exact-matching and embedding-similarity grading.","pith_inferences":["The reported public-dataset gains depend on the pseudo-human judge agreeing with real human raters; replacing GPT-4o with true human labels on newly generated outputs could reorder the results.","PLHF could be combined with active sample selection, because the human labels are the bottleneck and choosing which input–output pairs to grade could further cut the label budget.","A testable extension is to take an evaluator tuned on one dataset and use it unchanged to score outputs of a related task, which the paper does not examine."],"forward_implications":["Prompt optimization can be run end-to-end with a single round of human scoring, removing the repeated expert queries that block iterative frameworks.","A comparatively weak base model with a tuned prompt can serve as a better task-specific judge than a frontier model used off-the-shelf.","The evaluator module turns the human preference signal into a reusable artifact, so subsequent responder updates need no further human calls.","Exact-matching and embedding-similarity grading are consistently worse than learned evaluators, suggesting they are poor reward proxies for open-ended generation."],"supporting_citations":[{"why":"Supplies DSPy, the prompt optimizer used for both the evaluator and responder subtasks and to build the pseudo-human judge.","marker":"Khattab et al., 2024"},{"why":"Supplies TextGrad, the second prompt optimizer used to verify that PLHF's gains hold across PO methods.","marker":"Yuksekgonul et al., 2024"},{"why":"Describes the GPT-4 model family; GPT-4o is the strong LLM-judge baseline and the basis for the pseudo-human judge.","marker":"Achiam et al., 2023"},{"why":"RLHF is the inspiration for shaping an evaluator function from human feedback.","marker":"Ouyang et al., 2022"},{"why":"Few-shot in-context learning is the principle PLHF relies on to learn from a limited number of human-scored samples.","marker":"Brown et al., 2020"},{"why":"Supplies the Schema Guided Dialogue dataset with human satisfaction ratings used in the experiments.","marker":"Sun et al., 2021"},{"why":"Supplies the AES-ASAP automated essay scoring dataset with human essay scores.","marker":"Ben et al., 2012"},{"why":"Supplies the AES-2.0 argumentative essay dataset with holistic human ratings.","marker":"Kaggle, 2024"}],"fun_headline_variants":["One human round yields task-specific prompt judge","Learn the metric from few scores, then optimize prompts","Single human pass trains GPT-3.5 evaluator to beat GPT-4o","PLHF: few-shot human feedback turns prompt tuning into metric learning"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the pseudo-human judge — GPT-4o with a prompt optimized via DSPy — agrees with real human preferences on the newly generated outputs of the public datasets; the paper never checks this agreement, even though its own motivation is that pre-trained LLM judges diverge from humans.","fun_headline_variants_meta":{"raw":{"variants":["One human round yields task-specific prompt judge","Learn the metric from few scores, then optimize prompts","Single human pass trains GPT-3.5 evaluator to beat GPT-4o","PLHF: few-shot human feedback turns prompt tuning into metric learning"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000183,"raw_usage":{"total_tokens":1274,"prompt_tokens":867,"completion_tokens":407,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":483,"completion_tokens_details":{"reasoning_tokens":336}},"tokens_in":483,"tokens_out":407,"duration_ms":4381,"temperature":1.0,"reasoning_tokens":336,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T22:32:01.637525+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Collect real human ratings on a fresh batch of outputs produced by PLHF-optimized and GPT-4o-optimized responders, then compare which set humans prefer; if humans prefer the GPT-4o-optimized outputs on any public dataset, the paper's central claim for that dataset fails. A cheaper check is to measure human–pseudo-human agreement on a held-out sample of responder outputs before trusting the reported public-dataset scores.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the Schema Guided Dialogue dataset with human satisfaction ratings used in the experiments."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the AES-ASAP automated essay scoring dataset with human essay scores."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the AES-2.0 argumentative essay dataset with holistic human ratings."}],"review_version":1}