{"id":"57fc2775-d5e7-47c1-974a-39cc873df35c","arxiv_id":"2412.03343","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Possibility Exploration Fine-Tuning (PEFT) conditions LLMs on a random possibility number and trains with unlikelihood to generate diverse, controllable responses without added latency, as shown on dialogue and story tasks.","lead":"The paper introduces PEFT, a fine-tuning method that adds a random 'possibility number' to training prompts and uses unlikelihood training so a language model can produce several distinct responses to one prompt at no extra latency. It reports higher diversity and lower persona bias than baselines on dialogue and story generation, though the evaluation relies on GPT-4o for both data creation and scoring.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Possibility-number controllability is untested and unlikely to generalize: k is randomly assigned per response (Sec. 4.2), so no consistent semantic meaning exists across prompts, yet the abstract's core claim depends on it.","rationale":"The reader identified the same weakest assumption: controllability is asserted but never validated. I agree and consider it load-bearing because it is the key novelty distinguishing PEFT from simple one-to-many fine-tuning (OTMFT). Without a reliable mapping from k to a semantic mode, PEFT reduces to OTMFT with a random conditioning variable that may still increase diversity, but the advertised 'controllable possibility number' fails. The paper's own results do not test this: diversity is measured across responses generated with randomly sampled k values, which conflates diversity from the one-to-many data with controllability. The random assignment of k is a design choice, and there is no reason to expect a consistent ordering of possibilities across prompts; the unlikelihood loss can only enforce uniqueness within each prompt's training batch. A direct controllability check is therefore necessary before accepting the central claim. Other weaknesses (lack of significance tests, GPT-4o-based evaluation) are secondary because they affect the strength of the diversity improvement, not the core mechanism. Thus the verdict should remain CONDITIONAL, conditioned on either demonstrating controllability or softening the claim.","tokens_in":15952,"tokens_out":7566,"duration_ms":73152,"concrete_test":"Take 100 held-out dialogue prompts. For each prompt, generate 5 responses for k=1 and 5 responses for k=2 using temperature 0.75. Compute mean pairwise SBERT similarity within the k=1 set, within the k=2 set, and between the k=1 and k=2 sets, for each prompt. If the possibility number is a true control, intra-k similarity should be substantially higher than inter-k similarity, and the semantic themes should be consistent across prompts for a fixed k. Report the average intra-k vs inter-k similarity gap. If the gap is small (e.g., <0.05), the possibility number is not acting as a selective control; the abstract's controllability claim should be revised.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that PEFT responses 'correspond with a controllable possibility number' (Abstract) is not supported by the training design. In Section 4.2, each response r_i is paired with a possibility number k_i 'randomly sampled from [1..M]' (M=9, N=4). Thus the mapping k->response is arbitrary and prompt-specific. The unlikelihood objective in Section 4.3 only enforces that within each training batch, each response is uniquely associated with one k. With only 4 examples per prompt, the model can memorize this arbitrary assignment, but for unseen prompts there is no cross-prompt regularity to learn: the same k has no consistent semantic role. At inference, conditioning on k cannot reliably select a particular 'k-th possible response'; k likely functions merely as a random seed that perturbs generation. The evaluation (Section 5) samples L possibility numbers and measures diversity of the resulting set, but never checks whether a fixed k produces semantically similar responses across samples or across prompts, nor whether different k correspond to distinct, predictable modes. Therefore the paper's headline capability—controllable possibility exploration—is unvalidated. The Limitations section does not acknowledge this gap.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes Possibility Exploration Fine-Tuning (PEFT), a fine-tuning framework intended to increase the semantic diversity of LLM responses while preserving coherence and latency. The method constructs a one-to-many training set by using GPT-4o and List Prompting to generate four distinct responses per prompt, then augments each response with a randomly assigned possibility number k drawn from 1..M (with M=9). Training combines maximum likelihood on positive (prompt, k, response) triples with an unlikelihood loss whose negative samples reuse the same response under a different k, thereby penalizing the model for generating that response under the wrong possibility number. Experiments on open-domain dialogue (Mistral 7B and LLAMA 2 7B) and story generation report that PEFT improves lexical and semantic diversity scores, achieves a good diversity-coherence-latency trade-off compared to list prompting and a conditional variational framework, and increases the entropy of persona attributes, which the authors interpret as reduced demographic bias.","tokens_in":16185,"tokens_out":3548,"duration_ms":37021,"significance":"If the results hold, PEFT offers a practical, latency-neutral way to improve output diversity, and the demonstration that fine-tuning on one-to-many data can reduce demographic skew is of interest to the dialogue and fairness communities. The method is transparent and the repository is provided, which supports reproducibility. However, the paper's headline claim of a 'controllable possibility number' is not validated by any direct experiment, and the evaluation pipeline contains multiple circular uses of GPT-4o (training-data generation, coherence judging, persona extraction, and persona seeker). The absence of significance tests, error bars, or full temperature sweeps further weakens the quantitative claims. The core idea is defensible, but the current evidence does not support the strong wording in the abstract.","major_comments":[{"comment":"The claim that PEFT responses correspond to a 'controllable possibility number' is not supported by the training design or the experiments. In §4.2, k_i is 'randomly sampled from [1,..,M]' for each response, so there is no cross-prompt semantic meaning attached to a particular k. The unlikelihood objective in §4.3 only forces a one-to-one mapping within each training batch (N=4 examples). The evaluation in §5 samples possibility numbers and measures set-level diversity, but never tests whether a fixed k produces semantically consistent responses across prompts or whether different k values reliably select distinct, predictable semantic modes. Without such a test, the abstract's central claim remains speculative; the method may simply be OTMFT plus a random seed. Please add a direct controllability experiment or explicitly demote this claim.","section":"Abstract and §4.2–4.3"},{"comment":"The comparison reports only 'optimal temperatures for each method,' with the optimal temperature chosen on the same test set used for evaluation. This selection procedure can overfit the test set and makes the claimed improvements (e.g., PEFT 0.585 vs. OTMFT 0.522 in Table 1) hard to interpret. In addition, the test set contains only 300 prompts, and no significance tests, confidence intervals, or multiple-run variance are reported. The paper uses the word 'significantly' throughout, but there is no statistical support. Please report results for all temperatures, add bootstrap confidence intervals or repeated-seed variance, and, if possible, a significance test.","section":"§5.4, Tables 1–3"},{"comment":"The coherence metric uses GPT-4o and LLAMA 3 with a fixed threshold (rating <6), yet the paper also uses GPT-4o to generate the training data, to serve as the persona seeker, and to extract persona attributes. This creates a circular evaluation pipeline: the fine-tuned model is trained to imitate GPT-4o's diverse responses and is then judged by the same model family. The bias-reduction result (Table 4) is especially vulnerable because the persona attributes are extracted by GPT-4o and the conversations are driven by ChatGPT. Please provide a human evaluation or at least an evaluation using a model that was not involved in data generation, and show the sensitivity of the incoherence measure to the threshold choice.","section":"§5.2.2, §6, Appendix A.4"},{"comment":"The contribution of the unlikelihood term relative to the one-to-many data itself is not isolated. OTMFT already improves diversity over OTOFT, and the tables compare PEFT to OTMFT at different temperatures; the only head-to-head at the same temperature (t=0.75, Table 1: OTMFT 0.565 vs. PEFT 0.585, and t=1.0, Table 2: 0.579 vs. 0.583) shows small differences. An ablation that trains PEFT without the unlikelihood loss (i.e., pure OTMFT with random k) at identical settings would clarify whether the possibility-number conditioning adds value beyond the richer training data. Without this, the improvement over OTMFT may come only from the negative training, which could also be achieved by simpler data augmentation.","section":"§4.3 and §6"},{"comment":"The training data are generated by GPT-4o with List Prompting, so the diversity gains reported may reflect the diversity of the teacher outputs rather than the PEFT framework itself. The paper does not compare against fine-tuning on a larger one-to-many dataset generated by a different procedure, nor does it vary N (number of responses per prompt) or M (max possibility number). A controlled experiment isolating the effect of the data source and the number of responses would strengthen the generalizability claim. As it stands, the reader cannot tell whether PEFT would add diversity beyond simply training on any diverse set of GPT-4o outputs.","section":"§5.1 and §4.1"}],"minor_comments":[{"comment":"The MLE loss sums over t starting at 0, but language-modeling likelihoods typically start at t=1 since r_0 is a start token. Please fix the index or clarify the tokenization convention.","section":"§4.3, Eq. (1)"},{"comment":"The prompt template in A.3.2 uses '$k' in LaTeX-style math; this is likely a typo for the actual integer value. Please make the notation consistent.","section":"§4.2 and Appendix A.3.2"},{"comment":"The acronym PEFT is widely used for Parameter-Efficient Fine-Tuning; the paper's use of the same acronym for Possibility Exploration Fine-Tuning may confuse readers. Consider renaming or explicitly distinguishing it from the established term.","section":"General"},{"comment":"The 'Incoh' column label is not defined in the captions; please spell out 'incoherence rate' in the table captions.","section":"Table 3 and Table 5"}],"recommendation":"major_revision","confidential_remarks":"The paper is methodologically transparent and the central idea is worth exploring, but the headline 'controllable possibility number' is currently unvalidated, and the evaluation lacks statistical rigor. The heavy reliance on GPT-4o for both data creation and assessment creates a circularity that the authors should address with a human study or a non-GPT evaluator. The paper may be better framed as a study of one-to-many fine-tuning for diversity rather than as a controllable generation method. I would like to see the direct controllability test and the ablation separating OTMFT from the unlikelihood component before recommending acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe genuinely useful part of this paper is the one-to-many fine-tuning with an unlikelihood penalty: train on several distinct responses per prompt, batch them together, and you get semantic diversity at inference with no added latency. The experiments support that consistently across Mistral and Llama-2 on dialogue and story generation, and the release of code and data makes it checkable. Even the simpler OTMFT baseline beats the base model on the diversity-coherence trade-off at lower temperature, which is a solid result.\n\nWhat is not supported is the headline 'controllable possibility number.' In Section 4.2, each training response is paired with a k randomly sampled from [1..9], so the mapping from k to a semantic mode is arbitrary and prompt-specific. The unlikelihood loss only forces each response to be uniquely associated with one k within that prompt's batch; it cannot learn a cross-prompt meaning for 'k=3.' At inference, conditioning on k is effectively a random seed that perturbs generation, not a way to select a particular mode. The evaluation never tests whether the same k yields similar responses across prompts or repeated samples, and the Limitations section does not acknowledge this gap. The fix is either to validate controllability directly or to drop the word 'controllable' from the abstract.\n\nThe evaluation has the usual soft spots: no significance tests, best temperature selected per method, 300-example test sets, and GPT-4o in the loop for data creation, coherence judging, persona attribute extraction, and (as ChatGPT) the persona conversations. That weakens the debiasing claim; higher entropy is movement toward uniformity, not necessarily debiasing in a richer sense. Novelty is moderate because the possibility number is close to a control code (CTRL-style), which the paper does not cite.\n\nStill, this deserves a serious referee. The core recipe is simple, transparent, and reproducible, and the diversity claim is likely true in broad strokes. Send it to review with a clear request: substantiate the controllability claim or remove it, and report some estimate of variance.","headline":"Useful diversity fine-tuning recipe, but the 'controllable possibility number' claim is untested and likely overreaches.","tokens_in":16732,"tokens_out":2934,"would_cite":true,"duration_ms":26787,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Fine-tuning with possibility numbers makes one prompt yield many diverse replies.","keywords":["linguistic diversity","possibility exploration fine-tuning","one-to-many data","unlikelihood training","semantic diversity","demographic bias","dialogue generation","story generation"],"falsifier":"Generate responses for several different prompts with the same possibility number $k$ and different $k$ values, then measure the SBERT similarity of responses sharing the same $k$ across prompts. If same-$k$ responses are no more similar to each other than responses from different $k$'s, the controllable possibility-number mapping has not been learned.","tokens_in":15726,"feed_emoji":"💬","tokens_out":5565,"duration_ms":46721,"temperature":0.7,"pith_summary":"The paper claims that the low linguistic diversity of LLM outputs can be fixed without added latency or compute by fine-tuning on one-to-many data where each training response is tagged with a randomly assigned 'possibility number.' The central idea is that asking the model to produce the response for possibility k during training, and using unlikelihood loss to push the same response away from other k values, teaches the model to associate different k's with semantically different response modes. On open-domain dialogue and story generation, the authors report that their method, PEFT, beats the base model and existing diversity techniques on semantic diversity while keeping coherence and inference speed at base-model levels. They also report that the increased semantic diversity carries over to the personas a chatbot invents, raising the demographic entropy of generated personas and thereby reducing representational bias.","feed_headline":"Possibility numbers let one prompt yield many diverse replies","feed_subtitle":"A fine-tuning trick that adds no latency also cuts demographic bias in chatbot personas.","key_machinery":"The PE (Possibility Exploration) dataset paired with a likelihood-unlikelihood training objective. Each training prompt is replicated $N$ times with $N$ semantically distinct responses, each assigned a possibility number $k$ sampled from $[1, M]$; the unlikelihood term then treats each response as a negative example for every other $k$ in the batch. This creates a one-to-one association between a possibility number and a response meaning, which is what lets the model generate a controllable set of diverse outputs at inference.","core_discovery":"PEFT (Possibility Exploration Fine-Tuning) is a fine-tuning framework that treats open-ended generation as a one-to-many problem: for each prompt, the training set contains several distinct valid responses, each paired with a possibility number $k$ randomly drawn from $[1, M]$ with $M > N$. The model is trained with a maximum-likelihood term that teaches it to produce response $r_i$ when prompted with possibility number $k_i$, and an unlikelihood term that penalizes producing the same $r_i$ when a different possibility number is given, with penalties applied at the first token of each word. During inference, the user samples $L$ possibility numbers and generates the corresponding responses independently and simultaneously, so the number of distinct responses is controlled by the choice of $k$ and the cost is the same as generating $L$ responses with the base model. The paper's experiments on Mistral 7B and LLAMA 2 show that this increases semantic diversity (measured by $1 -$ SBERT pairwise similarity) while holding the percentage of incoherent responses low, and that the diversity gain transfers to persona generation, where PEFT raises the Shannon entropy of gender, age, location, education, and occupation attributes in chatbot-generated personas.","pith_inferences":["If the controllability mapping from $k$ to semantics is reliable, PEFT could serve as a cheap way to sample diverse candidates for self-consistency or tree-of-thought reasoning, where candidate diversity is the bottleneck; the paper does not test this.","A direct extension would be to apply PEFT during the instruction-tuning phase rather than post-hoc, as the authors note in their limitations, which would make the diversity benefit available zero-shot for arbitrary tasks.","The same possibility-number conditioning could also act as a controllable diversity knob at inference time, but only if the mapping generalizes across prompts; measuring same-$k$ semantic consistency across different prompts would settle that."],"forward_implications":["Fine-tuned with PEFT, a dialogue model can offer several semantically different replies per turn at base-model latency, which suits real-time chatbots.","PEFT combines with existing decoding methods such as temperature sampling and diverse beam search to push diversity further without architectural changes.","Because PEFT emphasizes semantic over lexical diversity, it also flattens demographic skew in persona generation, providing a fine-tuning-based route to debiasing.","The framework is task-agnostic: applying it to story generation reproduces the diversity and coherence gains seen in dialogue."],"supporting_citations":[{"why":"Supplies the unlikelihood training objective that PEFT uses to push responses apart.","marker":"(Welleck et al., 2019)"},{"why":"The conditional variational framework that PEFT is compared against as a strong fine-tuning baseline.","marker":"(Bao et al., 2021)"},{"why":"Introduces List Prompting, used both as a baseline and to generate the one-to-many training data with GPT-4o.","marker":"(Hayati et al., 2023)"},{"why":"One of the dialogue datasets used to build the training and test contexts.","marker":"(Smith et al., 2020)"},{"why":"SBERT, the sentence-embedding model used to compute semantic diversity as 1 minus pairwise similarity.","marker":"(Reimers and Gurevych, 2019)"},{"why":"QLoRA, the quantization-aware fine-tuning method used to train the models efficiently.","marker":"(Dettmers et al., 2024)"},{"why":"Provides temperature/nucleus sampling decoding, which PEFT is combined with in the experiments.","marker":"(Holtzman et al., 2019)"}],"fun_headline_variants":["Possibility numbers unlock diverse LLM replies at zero latency cost","Fine-tuning trick yields many diverse answers from one prompt","PEFT boosts linguistic diversity without slowing down chatbots","One prompt, many responses: PEFT cuts bias and adds diversity"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The model must learn, from just four training examples per prompt, a reliable mapping from a randomly assigned possibility number to a distinct response meaning, and the paper never verifies that this mapping holds for prompts outside the training set.","fun_headline_variants_meta":{"raw":{"variants":["Possibility numbers unlock diverse LLM replies at zero latency cost","Fine-tuning trick yields many diverse answers from one prompt","PEFT boosts linguistic diversity without slowing down chatbots","One prompt, many responses: PEFT cuts bias and adds diversity"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000178,"raw_usage":{"total_tokens":1329,"prompt_tokens":1008,"completion_tokens":321,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":624,"completion_tokens_details":{"reasoning_tokens":254}},"tokens_in":624,"tokens_out":321,"duration_ms":3888,"temperature":1.0,"reasoning_tokens":254,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T22:30:00.084775+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Generate responses for several different prompts with the same possibility number $k$ and different $k$ values, then measure the SBERT similarity of responses sharing the same $k$ across prompts. If same-$k$ responses are no more similar to each other than responses from different $k$'s, the controllable possibility-number mapping has not been learned.","supporting_citations":[{"cited_title":"Can You Put it All Together: Evaluating Conversational Agents' Ability to Blend Skills","cited_arxiv_id":"2004.08449","evidence_quote":"One of the dialogue datasets used to build the training and test contexts."}],"review_version":1}