{"id":"1d9ac89c-86b7-4e89-976e-6b7da1979170","arxiv_id":"2607.19243","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Persona prompting beats CAA steering and DPO adapters at making an English-prompted LLM match its own German, Spanish, and Bulgarian answer distributions, and transfers better to cultural scenarios.","lead":"This paper tests four ways to push an English-prompted LLM into answering as if it were asked in German, Spanish, or Bulgarian, from a simple persona prompt to activation steering and preference-trained adapters. Persona prompting worked best overall, suggesting that much of the target-language knowledge is already in the model and only needs the right context to be selected.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Benchmark target is the model's own target-language output distribution; without external validation, JSD improvements may measure self-mimicry rather than cultural alignment, weakening the 'selection problem' claim.","rationale":"The reader's weakest_assumption correctly identifies the potential circularity of using the model's own target-language outputs as the target for alignment. I agree this is the most load-bearing concern for interpreting the benchmark results, because the 'selection problem' conclusion leans heavily on the quantitative JSD improvements. However, I do not think it overturns the paper's main qualitative finding: the generalization dataset provides an independent, human-curated test on which prompting achieves 86% target-centric choices versus a 32.7% baseline, which is strong evidence for latent cultural knowledge independent of the circular benchmark. Thus the concern lands as a need for external validation of the benchmark anchor, not as a fatal flaw. The paper's own limitations section already acknowledges potential noise in the LLM-based extraction and the narrow supervision for CAA/DPO, so the appropriate verdict remains CONDITIONAL, with no change needed. A concrete human-anchored re-evaluation would settle whether the benchmark improvements reflect genuine cultural alignment or merely self-consistency.","tokens_in":19159,"tokens_out":9413,"duration_ms":109899,"concrete_test":"Take a random sample of 50 subjects per target language from the factual benchmark. For each, collect answers from 3–5 native speakers (or use a validated culturally specific QA dataset such as NormAd or an expert-annotated subset) to form a human target distribution. Recompute the intervention improvements from Table 1 using JSD-to-human-target and TPM against the human distribution, instead of the model's own target-language distribution. If prompting/CAA/DPO no longer show consistent significant improvements, the original benchmark results are an artifact of self-mimicry; if they do, the internal anchor is validated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that cross-lingual inconsistency is partly a selection problem rests on the assumption that the model's target-language answer distribution is a meaningful target for alignment. In §5.2, contrast pairs for CAA and benchmark-derived DPO are built from baseline extraction outputs: the 'target-preferred' candidate is simply the answer the model gives when prompted in the target language. The benchmark JSD-to-target metric then measures how closely an English-prompted intervention can reproduce the model's own language-dependent behavior. If those target-language outputs are noisy, translationally contaminated, or themselves English-influenced (e.g., because the model relies on the semantic hub's English anchor even under a Bulgarian prompt), then all reported JSD improvements are improvements in self-mimicry, not in genuine cultural alignment. The generalization dataset is a partial external check, but it is only used as a transfer probe for prompting; the benchmark scores for CAA and DPO are never anchored to an independent human or culturally validated target. The paper's own limitations admit that extraction relies on an LLM judge and that target-language answers could be normalized incorrectly. Therefore, the benchmark evidence does not by itself establish that the model 'already possesses' a target-language perspective; it only shows that the model can be coaxed into mimicking its own alternative output distribution.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper investigates whether cross-lingual factual inconsistency in LLMs can be mitigated at inference time by steering an English-prompted model to answer as if it were queried in a target language. It introduces a multilingual factual benchmark and a generalization benchmark of culturally rooted second-person scenarios, and compares zero-shot persona prompting, CAA, and DPO (benchmark-derived and generalization-derived) on Gemma 3 12B Instruct for German, Spanish, and Bulgarian. The main empirical claim is that persona prompting is the strongest overall intervention: it improves benchmark JSD to target, remains safe on ordinary facts, and transfers strongly to the generalization dataset, whereas CAA and DPO produce narrower, less transferable gains. The authors interpret this as evidence that cross-lingual inconsistency is at least partly a selection problem rather than a pure knowledge deficit.","tokens_in":19464,"tokens_out":5517,"duration_ms":54400,"significance":"If the findings hold, the paper provides a useful comparative framework for thinking about intervention depth versus transfer, and a concrete result that simple contextual prompting can outperform mechanistic interventions for cross-lingual consistency. It also releases a new benchmark and code. The distributional evaluation with repeated sampling is more robust than single-answer accuracy. However, the force of the conclusion depends on the validity of treating the model's own target-language answer distribution as the alignment target, and on clear train/test separation for the trained interventions.","major_comments":[{"comment":"The positive completions for CAA vectors and benchmark-derived DPO are the model's own target-language answers from baseline extraction; the metric JSD-to-target therefore measures agreement with the model's internal target-language distribution, not with an externally validated cultural target. The generalization dataset is an external check, but it is only used as a transfer probe; CAA/DPO benchmark scores are never anchored to an independent target. This weakens the claim that CAA/DPO 'successfully move the model toward the target language' as cultural alignment, and it partially undermines the 'selection problem' interpretation if the target-language samples are themselves English-influenced. Please provide external validation (e.g., human annotation of the target-language distributions on a subset, or comparison against a culturally curated gold standard) or explicitly reframe these","section":"§5.2, Eq. (1), Table 1"},{"comment":"The paper reports benchmark-derived DPO improves JSD in all 9 cells, but never specifies whether the contrast pairs used for DPO training are disjoint from the subjects in the reported test split. Since the contrast pairs are built from the baseline extraction outputs on the benchmark, if the same subjects appear in training and evaluation, the in-domain gains are expected and do not demonstrate generalization. Please state explicitly how subjects were split for DPO training versus evaluation, and report results for a held-out subject split. The same clarification is needed for CAA vector construction relative to the test split.","section":"§6.3, Table 1, §F.3"},{"comment":"The strict-disjoint v0 split averages about 8 subjects per language. Paired Wilcoxon tests and CLES on n=8 have very low power and high sampling variability; the p<0.05 results in v0 (e.g., Table 1 bg v0 steered) should be interpreted with caution. Either pool across languages (with appropriate mixed-effects modeling), report bootstrap confidence intervals, or drop significance claims for v0.","section":"§F.2.3, Table 1"},{"comment":"The claim that 'all four interventions successfully move the model' is contradicted by several cells in Table 1 (e.g., es v2 prompting JSD 0.468 > baseline 0.423, TPM 0.737 < 0.778; bg v2 steered TPM 0.750 < baseline 0.779). The later text acknowledges '8 out of 9', so the broader statement should be corrected to avoid overgeneralization.","section":"§6.3, Table 1"}],"minor_comments":[{"comment":"Target Probability Mass (TPM) is mentioned but never formally defined; please give the formula.","section":"§C.2"},{"comment":"The figure is referenced but appears not to be included; ensure all figures are present.","section":"§5.2, Figure 1"},{"comment":"The phrase '|∆|formats' appears to be a typo; it should read '|∆TPM| and |∆JSD|'.","section":"Table 3 caption"},{"comment":"The notation 'CLES [JSD-T] ≈0.503' is unclear; please define what JSD-T denotes.","section":"§7.1"},{"comment":"Including identical pairs means positive and negative can be identical; the discussion acknowledges this, but a brief rationale for why this variant was tested would help.","section":"§5.2, v2 variant"},{"comment":"The statement that 'some answers may have been normalized incorrectly' is appreciated; please also state whether any manual audit of the extraction was performed on a random sample.","section":"Limitations"}],"recommendation":"major_revision","confidential_remarks":"The core concern is the circularity of the target distribution for CAA/DPO, and the lack of explicit train/test separation for DPO. The prompting result is more robust because it is not trained on the benchmark, but the paper's comparative claims about CAA/DPO need reframing or external validation. The small v0 sample sizes also make some significance claims fragile."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the strongest result here is the transfer asymmetry — persona prompting moves the model to ~86% target-centric choices on a held-out cultural generalization set, while CAA and DPO stay at baseline. That is a clean, new empirical contrast and it holds up despite the measurement caveats. The benchmark JSD numbers are weaker evidence than the paper suggests, because the target distribution is the model's own target-language outputs; a JSD improvement there partly measures self-mimicry. But the stress-test note overstates the damage: the generalization dataset is manually curated (Gemini-generated, then human-reviewed), not derived from the same baseline, so the 'selection problem' claim does not collapse. It is mainly the CAA/DPO benchmark claims that need external anchoring, and the paper is appropriately cautious about those methods' limited transfer.\n\nWhat's new: first application of CAA to cross-lingual preference realignment, a new multilingual factual benchmark (7 relations, 6 languages), a 50-item-per-language generalization probe, and a three-way comparison with safety and transfer checks. The method descriptions are detailed enough to reproduce, and the hyperparameter sweeps for steering are thorough. The paper is honest about limitations: one base model, three target languages, LLM extractor/judge noise, and the DPO/steering supervision being narrow. That is good practice.\n\nSoft spots, in proportion: (1) The central benchmark metric—JSD to target-language distribution—is not an external ground truth; it measures agreement with the model's own alternative output distribution. For CAA and DPO this is genuinely circular-ish, and the reader's 4/10 circularity burden feels right. (2) The v0 contrast split averages ~8 subjects per language; that is very small for Wilcoxon/CLES claims, though the paper does report effect sizes and significance honestly. (3) DPO train/test separation isn't clearly described — the paper says held-out test data, but it doesn't specify whether the contrast pairs for training vs evaluation are disjoint by subject. That matters for the JSD numbers. (4) Code and datasets are only linked, not verifiably shipped; the GitHub link may be live, but the review pipeline can't confirm it. (5) Single model, Gemma 3 12B, limits generality.\n\nNone of these are fatal. The paper's main claim—simple prompting gives broad, safe, transferable alignment—is well-supported, especially by the generalization experiment, which is difficult to explain away as self-mimicry. The comparison is fair and the caveats are explicit.\n\nBottom line: worth a serious referee. It is not a theory paper; it is an empirical data point, and a useful one. I'd suggest the authors re-anchor the benchmark evaluation with human judgments or at least an external multilingual QA benchmark, and clarify DPO data splitting. For a reading group, it's a good discussion piece on evaluation validity in multilingual alignment.","headline":"Prompting beats steering and DPO for transferable cross-lingual alignment, and the generalization set supports it; the benchmark JSD numbers are partially self-referential and need an external anchor.","tokens_in":19899,"tokens_out":2750,"would_cite":true,"duration_ms":30919,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A simple persona prompt shifts an English-prompted language model's answers toward German, Spanish, or Bulgarian preferences, outperforming activation steering and fine-tuning.","keywords":["cross-lingual factual consistency","persona prompting","inference-time steering","contrastive activation addition","direct preference optimization","multilingual evaluation","cultural alignment","Jensen-Shannon distance"],"falsifier":"Independently collect human-verified target-typical answers for the same subjects from native German, Spanish, and Bulgarian speakers, rebuild contrast pairs from those instead of from the model's own target-language outputs, and re-run the persona-prompting comparison. If prompting no longer reduces Jensen-Shannon distance to the human targets, the paper's central claim that target preferences are already latent and context-retrievable is not supported. A direct blind comparison of persona-prompted English answers against the model's own target-language answers would also settle whether the t","tokens_in":19055,"feed_emoji":"🌐","tokens_out":5712,"duration_ms":69829,"temperature":0.7,"pith_summary":"The paper asks whether cross-lingual factual inconsistency—a language model changing its answer distribution purely because the prompt language changed—can be fixed at inference time, without retraining. It compares four interventions on a 12-billion-parameter multilingual model: zero-shot persona prompting, activation steering by contrastive activation addition, and two preference-optimized adapters. The central finding is that the simplest intervention, telling the model to adopt the perspective of a typical German, Spanish, or Bulgarian person, is the strongest overall: it improves targeted benchmark cases, stays safe on ordinary facts, and transfers to broader cultural choices, while the deeper interventions are sharper on benchmark metrics but narrower and more fragile. The authors take this as evidence that much of the target-language perspective is already latent in the model's weights, making cross-lingual inconsistency at least partly a selection problem rather than a knowledge deficit. A sympathetic reader would care because it suggests cheap, reversible, low-risk steering can outperform invasive retraining for multilingual alignment.","feed_headline":"Simple persona prompt beats deeper fixes for cross-lingual bias","feed_subtitle":"Telling a language model to think like a local German, Spanish, or Bulgarian shifts its answers and transfers beyond facts.","key_machinery":"The carrying mechanism is the contrast between the model's own English-preferred and target-language-preferred answer distributions. The paper operationalises this by repeated sampling of open-ended generations, extracting canonical answers, and measuring pairwise Jensen-Shannon distance and target probability mass. Persona prompting works by prepending a fixed instruction to adopt a typical local perspective; the same contrast pairs also feed contrastive activation addition steering vectors and direct preference optimization training pairs, allowing the three methods to be compared on identical preference signals. The v0/v1/v2 contrast-construction variants are the pivot: strict disjoint pa","core_discovery":"On its own terms, the paper claims that an English-prompted language model can be made to answer as if queried in a target language, and that simple contextual prompting does this better than more invasive methods. Built on a curated multilingual factual benchmark and a new generalization dataset of 50 culturally rooted scenarios per language, the experiments show persona prompting reducing Jensen-Shannon distance to the target-language distribution in eight of nine evaluated settings, while also being the only intervention that moved both target probability mass up and distributional distance down on ordinary held-out facts. Contrastive activation steering produced the sharpest causal shift","pith_inferences":["The paper's target anchor is the model's own target-language output; an independent, human-validated target distribution would be a stronger test of whether 'alignment' is genuine rather than self-consistency.","Because the generalization dataset was used to train one adapter and test transfer, a cleaner test would keep a held-out cultural dataset unseen by all interventions, including any prompt used for tuning.","The success of persona prompting suggests a testable extension: measuring how much of the effect survives when the persona is described without country names (e.g., by cuisine, norms, or institutions), which would separate genuine cultural knowledge from label-triggered stereotyping.","Extending to a typologically distant, non-Latin-script language such as Korean would test whether the latent-perspective claim holds when tokenization distances are large."],"forward_implications":["Cross-lingual factual inconsistency can be reduced at inference time without retraining: persona prompting improved alignment in 8 of 9 benchmark cells on the tested model.","Simple contextual cues transfer beyond benchmark facts, raising target-centric choice rates on the generalization dataset from about 33% to 86%, whereas steering and adapters stayed at baseline levels.","Activation steering can causally manipulate cross-lingual preferences, but its gains are configuration-dependent and risk collateral degradation of ordinary factual recall.","Lightweight adaptation via preference optimization yields stable, benchmark-local gains, but only when the contrastive candidate set is neither too sparse nor self-contradictory.","The selection-problem interpretation implies that better prompting and context design may be a more cost-effective path to multilingual alignment than weight modification."],"fun_headline_variants":["Persona prompt beats DPO and activation edits for cross-lingual bias","Telling LLM to 'think like a German' fixes factual inconsistency","Simple context shift outperforms invasive methods for LLM bias","Zero-shot persona prompting best for cross-lingual factual truth","Persona prompting wins over CAA and DPO for language bias"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The target-language answer distribution produced by the model itself is treated as the target to align to; if those samples are noisy, biased, or English-influenced, the improvements only show agreement with an unvalidated internal anchor.","fun_headline_variants_meta":{"raw":{"variants":["Persona prompt beats DPO and activation edits for cross-lingual bias","Telling LLM to 'think like a German' fixes factual inconsistency","Simple context shift outperforms invasive methods for LLM bias","Zero-shot persona prompting best for cross-lingual factual truth","Persona prompting wins over CAA and DPO for language bias"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000209,"raw_usage":{"total_tokens":1248,"prompt_tokens":752,"completion_tokens":496,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":496,"completion_tokens_details":{"reasoning_tokens":405}},"tokens_in":496,"tokens_out":496,"duration_ms":5525,"temperature":1.0,"reasoning_tokens":405,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T12:58:07.452546+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Independently collect human-verified target-typical answers for the same subjects from native German, Spanish, and Bulgarian speakers, rebuild contrast pairs from those instead of from the model's own target-language outputs, and re-run the persona-prompting comparison. If prompting no longer reduces Jensen-Shannon distance to the human targets, the paper's central claim that target preferences are already latent and context-retrievable is not supported. A direct blind comparison of persona-prompted English answers against the model's own target-language answers would also settle whether the t","supporting_citations":[],"review_version":1}