PICACO: Pluralistic In-Context Value Alignment of LLMs via Total Correlation Optimization

Dongyao Zhu; Han Jiang; Xiaoyuan Yi; Xing Xie; Zhihua Wei; Ziang Xiao

REVIEW 4 major objections 4 minor 31 references

PICACO: Pluralistic In-Context Value Alignment of LLMs via Total Correlation Optimization

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read PICACO claims that pluralistic in-context value alignment can be automated without fine-tuning: a meta-instruction optimized to maximize the conditional total correlation between intended values and LLM responses reliably balances up to…

desk verdict Solid empirical recipe with a broken information-theoretic justification—worth refereeing after the theory is reframed or fixed. read the letter →

arxiv 2507.16679 v3 pith:5PVC6Z2X submitted 2025-07-22 cs.CL cs.AIcs.CY

Han Jiang , Dongyao Zhu , Zhihua Wei , Xiaoyuan Yi , Ziang Xiao , Xing Xie This is my paper

classification cs.CLcs.AIcs.CY

keywords valuealignmentin-contexttotalcorrelationpluralisticpromptoptimizationinstructionbottleneckLLM-as-judgeSchwartzvalues

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

PICACO claims that the "instruction bottleneck" of in-context alignment — an LLM's failure to honor several intended values within one prompt — can be broken without any fine-tuning, by automatically optimizing the wording of a meta-instruction. The paper frames pluralistic alignment as maximizing the conditional total correlation between the intended value set and the model's responses, then solves this black-box optimization with an EM-like loop that alternates between collecting well-aligned responses and refining the instruction. If the claim holds, practitioners could generate instructions that balance multiple or even conflicting values — helpful and harmless, or four Confucian virtues — using only a few dozen task prompts and an LLM judge, and could lift cheap models to the alignment level of more expensive ones. The paper reports consistent superiority over most of nine ICA baselines across five value compositions and three target LLMs, with human-judge agreement supporting the evaluation.

What carries the argument

The load-bearing object is the conditional total correlation $\mathrm{TC}_e(V,y|x) = \sum_k I_e(v_k;y|x) - I_e(V;y|x)$ (Eq. 1), an information-theoretic quantity that is maximized exactly when the response encodes the correlation across all $K$ values while avoiding shared redundancy. Since it is intractable, the paper derives a variational lower bound (Eq. 2) via a Barber–Agakov bound for the per-value conformity terms and a conditional CLUB upper bound for the redundancy term; this yields the computable objective in which $q_\omega$ (implemented as GPT-4o-mini) scores each response's conformity to each value, $\beta$ controls the conformity-redundancy trade-off, and $q_\varphi$ (embedding cosine similarity) penalizes repeating surface details of the value exemplars $s$. The EM-like Variational Information Maximization loop then does the actual work: a response-enhancement step samples candidates and keeps the top $M_1$ by $q_\omega - q_\varphi$ score, and an instruction-refinement step generates new meta-instructions and keeps the one maximizing the empirical objective.

What would settle it

Run PICACO twice with deliberately biased value evaluators — one instructed to reward only literal mention of the value names, one instructed to reward only generic helpful style — and then have independent human raters score the resulting responses on genuine value conformity; if the two optimized instructions produce indistinguishable human ratings, the method's reported gains come from the judge rather than from real value alignment. A simpler fixed-target check: measure the correlation between the evaluator's per-response scores and human value-conformity ratings on a held-out response set; if that correlation is low, the instruction's score gains may not transfer to human judgments.

Watch

Extended reading notes

Core claim

On its own terms, this paper's central claim is that a meta-instruction optimized to maximize $\mathrm{TC}_e(V,y|x) = \sum_k I_e(v_k;y|x) - I_e(V;y|x)$ makes a frozen LLM conform to all $K$ intended values at once, because the objective rewards responses that carry information about each value individually while penalizing redundancy among the value set. The computable surrogate (Eq. 2) trades off a pluralistic conformity term, scored by an LLM judge $q_\omega$, against a redundancy term $q_\varphi$ that detects superficial copying of the demonstration text; the EM-like Variational Information Maximization loop alternately keeps the most aligned sampled responses and proposes new meta-instructions that make those responses more probable. Across Confucianism, Modern Liberalism, Helpful-Harmless balance, Helpfulness, and Harmlessness compositions, PICACO attains the best or second-best conformity score in nearly every cell of the main tables, and supplementary results show the gains survive a change of judge model, a weaker value evaluator, jailbreak templates, and out-of-distribution tasks.

Load-bearing premise

The whole method trusts that its LLM judge, GPT-4o-mini, scores genuine human value conformity; if the judge rewards surface style or value name-dropping instead, the optimized instruction inherits exactly that bias.

Editorial extensions

If this is right

A practitioner can obtain a value-specific meta-instruction from about 50 task prompts and 8 demonstrations, with no GPU training and no hand-authored instruction template.
A single instruction can hold up to eight distinct values in balance; as the value set grows, conformity degrades more gracefully and with lower cross-value variance than for the alternatives tested.
The optimized instructions transfer to task types unseen during optimization, such as creative writing and thread replies, so re-optimization may not be needed for new domains.
Under a jailbreak template, PICACO-aligned models keep toxicity low while still answering helpfully — the instruction learns safe-refusal patterns rather than surface avoidance.
Cheaper target models steered by PICACO match the alignment scores of a more expensive bare model, offering a cost-performance trade-off at inference time.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The objective's separation into a per-value reward and a redundancy penalty invites replacing the mutual-information terms with other dependence measures (e.g., distance correlation or HSIC); because the EM loop is agnostic to the inner score, such variants are directly testable without redesigning the method.
If the LLM judge $q_\omega$ carries a systematic bias — for instance rewarding value name-dropping or a generic advisory style — the optimized instruction inherits it, so a natural audit is to run the selection loop against a panel of diverse judges and compare the resulting instructions; the paper's own fake-alignment examples suggest this risk is real for abstract value sets.
The observed GPT-3.5-Turbo+PICACO versus bare O4-Mini comparison suggests a broader hypothesis the paper leaves implicit: instruction optimization is a general cheap-model-scaling lever, and the correlation objective may be reusable beyond value alignment, for any multi-constraint instruction-following task.
Making the relevance filter part of the optimization objective itself — rather than a post-hoc evaluation weight — is a testable extension directly motivated by the paper's finding that fake alignment inflates Schwartz-value scores.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

This paper is worth a look: it does something new and the experiments are unusually thorough, but the information-theoretic argument at the center does not actually work. PICACO's empirical recipe—an EM-style loop that samples responses, keeps the ones that score high on an LLM judge minus a similarity-based penalty, then rewrites the meta-instruction to favor those responses—is a sensible, practical extension of OPRO-style prompt optimization with a real novelty: the combination of a response pool and a redundancy penalty for pluralistic value alignment. The evaluation is solid: five value compositions, three target LLMs, nine ICA baselines, ablations, an OOD test, a jailbreak test, a human study with three annotators, and cross-checks with alternative judges and a different qω. Code and data are linked. That is credit where it is due.

The soft spot is the theory. The paper claims Eq. 2 is a lower bound on total correlation, derived from a Barber-Agakov bound and a conditional CLUB bound. For that to work, qϕ(s|x,y) must be a valid conditional distribution. Instead, it is defined in App. D.1.2 as 1 + Sim(s,y) - Sim(x,y). That is not normalized over s, and it can be negative when Sim(x,y) > 1 + Sim(s,y). When it goes negative, log qϕ is undefined, so the bound isn't even well-defined. The paper never addresses this. As a result, the redundancy-reduction term is an unnormalized heuristic penalty with a TC-flavored name, not a proven component of a total-correlation objective. The ablation shows it helps empirically, but that does not rescue the formal claim. This needs to be fixed, either by deriving a proper lower bound with a normalized qϕ or by explicitly repositioning PICACO as heuristic instruction optimization and dropping the TC-maximization framing.

The other issues are minor. beta and the Schwartz relevance weighting are hand-set without sensitivity analysis. The optimizer-judge overlap (qω is GPT-4o-mini and the main judge is GPT-4o) is real but the paper does cross-check with another judge and an alternative qω, so I don't see it as fatal. No commit hash is provided, though the code link is a start.

Overall: the empirical method is plausible and the results are presented honestly. The load-bearing flaw is the theoretical justification, not the experiments. This paper deserves peer review—a serious referee can ask the authors to reframe the contribution without dismissing the empirical work.

Referee Report

4 major / 4 minor

Summary. The paper proposes PICACO, a training-free in-context alignment method that iteratively refines a meta-instruction by maximizing a conditional total-correlation objective between a set of target values and LLM responses. The objective is approximated with a value evaluator qω (implemented as GPT-4o-mini) for pluralistic conformity and a cosine-similarity-based redundancy evaluator qφ, and optimized through an EM-like procedure over sampled response pools. Experiments cover five value compositions, three target LLMs, nine baselines, ablations, OOD generalization, jailbreak robustness, and a small human study, reporting consistent but small improvements (typically 0.02--0.17 on a 1--5 scale).

Significance. If the theoretical formulation were valid, the paper would contribute a principled, training-free method for pluralistic ICA and a concrete way to escape the instruction bottleneck. The empirical study is unusually thorough for this setting: multiple value compositions, three target models, two judge models, a three-annotator human validation, ablations, OOD generalization, adversarial robustness, and cost estimates. The main weakness is that the central information-theoretic derivation is not established as written, because the redundancy evaluator qφ is not a valid conditional distribution and the probability estimator p(y|x) is an unadorned cosine-similarity heuristic. The empirical gains may still stand as an effective prompt-optimization heuristic, but the total-correlation interpretation needs repair or explicit downgrading.

major comments (4)

[§3.2 (Eq. 2), App. C.3, App. D.1.2 (Eq. 19)] The derivation of the TC lower bound is invalid as stated. qφ(s|x,y)=1+Sim(s,y)-Sim(x,y) is not a probability distribution over s: it is not normalized, has no specified support, and can be negative (for example when Sim(x,y) exceeds 1+Sim(s,y)), making log qφ undefined in Eqs. (2)--(4). The conditional CLUB bound in Eq. (15) requires a conditional distribution q(s|x,y) whose induced joint q(y,s|x) captures the y--s dependence; no such distribution is supplied. Consequently Eq. (2) is not a proven lower bound on TCe, and the ablation in Table 10 showing that removing qφ hurts performance does not by itself validate the total-correlation interpretation. The method may work as an unnormalized heuristic, but the paper's central theoretical claim needs to be either proved with a properly normalized qφ over a defined support or explicitly downgraded.
[App. D.1.2 (Eq. 20) and Eqs. (3)--(4)] The estimator p(y|x)= (1/n) Σ_i Sim(y,y_i) is used as a generation probability in the optimization objective, but cosine similarity is not a probability and the estimator has no stated justification. Since the instruction refinement step weights responses by p_e(y_i,j|x_i), this choice directly changes the quantity being optimized. Provide a justification or a validity check (e.g., comparison against white-box token log-probabilities for LLaMA-3.1-8B-Instruct) before relying on it as a probability.
[Eqs. (2)--(4)] The optimization objective is not internally consistent across equations. Eq. (2) has β multiplying the conformity sum; Eq. (3) drops β from the first term; Eq. (4) replaces log qφ(s|x,y) with (1/K) log qφ(s|x,y) inside the product over k. The text does not explain these changes, so it is unclear what quantity PICACO actually optimizes and whether the reported gains come from the total-correlation objective or from these ad hoc reweightings.
[§4.2 and App. F.2] The optimization signal qω is GPT-4o-mini and the primary evaluation judge is GPT-4o, both from the same model family; the DeepSeek-V3.1 and Moonshot-V1-8k robustness checks and the three-annotator human study reduce but do not remove the risk that PICACO is fitted to LLM-judge preferences such as value name-dropping. This limitation should be stated explicitly in the main paper, and the authors should report whether the optimized instructions remain stable when qω is replaced by a non-LLM or human-derived evaluator.

minor comments (4)

[App. C.4.1] There is a typo in the definition of the conflicting components: the second component should be v_u_2 = 'helpless but harmless', not v_u_1 twice.
[Tables 16--18] The column headers for Helpfulness and Harmlessness list all eight HH values even though only four are target values in each composition; this makes the tables harder to read than necessary.
[App. D.1.3 and Alg. 1] The symbol M2 is overloaded: it denotes the noisy-response pool size in Eq. (3) and Algorithm 1, but also the number of newly generated meta-instructions in the instruction refinement step. Use distinct notation.
[§4.2 and App. F.1] The sentence 'This advantage is statistically significant across all test items' is stronger than what Table 7 supports, since the significance tests are reported only for GPT-3.5-Turbo with GPT-4o as judge.

Circularity Check

2 steps flagged · score 4.0 of 10

PICACO's headline conformity gains are measured with the same rubric (Fig. 9) and the same model family (GPT-4o-mini for optimization, GPT-4o for evaluation) as the objective being optimized, so the main empirical claim is partially fitted to the judge; cross-family judges and a human study supply real but secondary grounding.

fitted input called prediction [Sec. 4.1 (Data, Implementation, and Evaluation); Sec. 3.2 Eqs. (3)–(4); Apps. D.1.2 and D.3]
"PICACO's qω is implemented using GPT-4o-mini, and qϕ is based on cosine similarity among s, x, and y. ... For evaluation, we employ GPT-4o-2024-08-06 and DeepSeek-V3.1 as judge models and report overall conformity on a 1-5 scale. ... The model is prompted with the evaluation template shown in Fig. 9, and the resulting scores are converted into probability distributions. ... The evaluation template for the two judge models is shown in Fig. 9."

By construction, the optimization objective (Eq. 3) is dominated by the log q_ω conformity term, where q_ω is GPT-4o-mini scored with the Fig. 9 template, and Eq. 4 selects the meta-instruction by that same score. The headline claim ('PICACO consistently outperforms most baselines', Table 1) is the same conformity construct, same Fig. 9 template, same 1-5 scale, scored by GPT-4o, a same-family judge. The reported 'prediction' is therefore largely the fitted objective transferred to a sibling model. It is not statistically forced: the 800 test prompts are disjoint from the 50 optimization prompts, and DeepSeek-V3.1, Moonshot-V1-8k q_ω, and a three-annotator human study (Krippendorff's α up to 0.93) provide independent grounding.
other [App. D.1.2 Eq. (19); App. C.2–C.3 Eqs. (7)–(8); Sec. 3.2 Eq. (2)]
"qϕ(s|x,y) = 1 + Sim(s,y) − Sim(x,y). (19) ... qϕ(s|x,y) is also a variational distribution to tell whether the generated responses y contain too many unnecessary details in the exemplars s. ... By Combining Eq. 6 and Eq. 7, we can approximately have: TCe(V,y|x) ≥ E p̂(x){ Σ_k E_pe(y|x)[log qω(vk|x,y)] − E_pe(y|x)[log qϕ(s|x,y)] + E_p(y|x)[log qϕ(s|x,y)] }. (8)"

The redundancy terms in Eqs. (2)/(8) are derived from a conditional CLUB upper bound (App. C.3), whose proof requires q_φ(s|x,y) to be a valid conditional distribution over s (the condition is KL[p(y|x)p(s|x)||q(y,s|x)] ≥ KL[p(y,s|x)||q(y,s|x)]). But Eq. (19) defines q_φ as an unnormalized cosine score over a single fixed text s: cosine similarities lie in [−1,1], so q_φ ∈ [−1,3] and can be negative whenever Sim(x,y) > 1 + Sim(s,y), making log q_φ undefined in Eqs. (2)–(4); no support set or normalization constant is given, and a normalized q over a one-point support would be identically 1 (log = 0, vacuous). The paper imposes the probability requirement explicitly on q_ω ('q_ω(v_k|·)>0 ... Σ q_ω(v_k|·)=1') but never on q_φ.

full rationale

Verdict: partial circularity, score 4. The load-bearing empirical claim—that PICACO's meta-instruction improves pluralistic value conformity—rests on a judge rubric that is the same construct the method optimizes. Eq. (3) retains responses and Eq. (4) selects the instruction by Σ log q_ω, where q_ω is GPT-4o-mini graded with the Fig. 9 template; the headline Tables 1 and 13 report 'overall conformity' on that same Fig. 9 template and 1–5 scale judged by GPT-4o, a same-family model. This is a genuine but partial fitted-input-called-prediction pattern: the quantities are not identical (disjoint test prompts, sibling judge), and the paper's App. F.2 robustness suite—DeepSeek-V3.1 judge, Moonshot-V1-8k q_ω, and a three-annotator human study with Krippendorff's α up to 0.93—provides independent grounding that prevents the central claim from fully reducing to the fit. A separate, non-circularity issue is flagged because it undercuts the theoretical wrapper: the CLUB bound in App. C.3 requires q_φ to be a conditional distribution over s, but App. D.1.2 defines q_φ = 1 + Sim(s,y) − Sim(x,y), an unnormalized, possibly negative score for a single fixed s; hence Eqs. (2)/(8) are not a proven lower bound on total correlation, and 'total correlation optimization' is, as written, a conformity-score maximization plus a cosine-similarity penalty. Self-citations (Yao et al. 2024 FULCRA; Biedma et al. 2024 VALUELEXSTEMS; Duan et al. 2025 AdAEM, sharing Yi/Xie as authors) are used only as data or benchmark sources, never as the justifying premise, so no self-citation chain is load-bearing. Ablations (Table 10) show the components matter empirically, so the method has independent content beyond its TC label. Net: the empirical ranking is credible but anchored to the optimizing judge's rubric, and the information-theoretic derivation is formally unsupported; this warrants a 4, not a 6+.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The central claim rests on several unverified approximations: the value evaluator proxy, the CLUB upper-bound condition, the V=f(s) mapping, instruction-prompt independence, and a cosine-similarity estimate of generation probability. Beta is hand-set per value family, and the Schwartz evaluation was reweighted by Relevance after observing fake alignment. No new entities are introduced.

free parameters (3)

beta (β) = 0.5 (Schwartz) / 5 (HH)
Controls the trade-off between pluralistic conformity and redundancy in Eq. (2); set differently by value type with no sensitivity analysis. It directly shapes the optimized meta-instruction and could change rankings.
Relevance weighting for Schwartz evaluation = µ' = µconf × Relevance / 5
Introduced after observing fake alignment and applied only to the Confucianism and Modern Liberalism scores in Table 1. It alters the headline results and is validated only by a small human study.
Sampling hyperparameters N, M1, M2, T = N=50, M1=10, M2=15, T=10
Chosen by hand for all experiments; no ablation on these values is reported, and API cost scales with them.

assumptions (5)

domain assumption qω(vk|x,y), implemented by GPT-4o-mini, is a valid proxy for true value conformity of a response.
Sec. 3.2 and D.1.2: the meta-instruction is optimized to maximize log qω; if qω rewards superficial value mentions, PICACO overfits. Supported only by indirect human-study evidence.
domain assumption The conditional CLUB bound condition holds: KL[p(y|x)p(s|x) || q(y,s|x)] ≥ KL[p(y,s|x) || q(y,s|x)], so the variational upper bound on I(y;s|x) is valid.
App. C.3, Eq. 14: the paper calls this a mild condition but never verifies it for the cosine-similarity qϕ.
domain assumption The latent values V are a deterministic function of textual observations s, allowing the data processing inequality to replace I(V;y|x) with I(s;y|x).
App. C.3: 'V=f(s) where f is a deterministic mapping'; this is needed for the redundancy-reduction step.
standard math The meta-instruction e is independent of the task prompt x, and pe(vk) is a delta at the specified value, so E_pe(vk)E_pe(y|x,vk) collapses to E_pe(y|x).
App. C.3, Eq. 11-12: a modeling assumption used to derive the empirical objective; plausible for a fixed instruction but not tested.
ad hoc to paper The generation probability p(y|x) can be approximated by the mean cosine similarity among n=10 sampled responses.
App. D.1.2, Eq. 20: used to weight responses in Eq. (4); no justification is given beyond computational convenience.

how reviews work

0 comments

Cite this review

Pith. "Pith review of PICACO: Pluralistic In-Context Value Alignment of LLMs via Total Correlation Optimization." pith.science (2026). https://pith.science/paper/5PVC6Z2X

@misc{pith2026250716679,
  author       = {Pith},
  title        = {Pith review of: PICACO: Pluralistic In-Context Value Alignment of LLMs via Total Correlation Optimization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5PVC6Z2X}},
  note         = {Machine review of arXiv:2507.16679}
}

read the original abstract

In-Context Learning has shown great potential for aligning Large Language Models (LLMs) with human values, helping reduce harmful outputs and accommodate diverse preferences without costly post-training, known as In-Context Alignment (ICA). However, LLMs' comprehension of input prompts remains agnostic, limiting ICA's ability to address value tensions--human values are inherently pluralistic, often imposing conflicting demands, e.g., stimulation vs. tradition. Current ICA methods therefore face the Instruction Bottleneck challenge, where LLMs struggle to reconcile multiple intended values within a single prompt, leading to incomplete or biased alignment. To address this, we propose PICACO, a novel pluralistic ICA method. Without fine-tuning, PICACO optimizes a meta-instruction that navigates multiple values to better elicit LLMs' understanding of them and improve their alignment. This is achieved by maximizing the total correlation between specified values and LLM responses, theoretically reinforcing value correlation while reducing distractive noise, resulting in effective value instructions. Extensive experiments on five value sets show that PICACO works well with both black-box and open-source LLMs, outperforms several recent strong baselines, and achieves a better balance across up to 8 distinct values.

Figures

Figures reproduced from arXiv: 2507.16679 by the authors.

**Figure 1.** GPT-4o’s responses when instructed to follow multiple helpful and harmless requirements (top) and Schwartz values (bottom). In both cases, some of the specified values are disregarded. nAI, 2024; 2025; Meta, 2025; Gemini et al., 2024), but also introduced social concerns such as generating hate speech and reinforcing biases (Bommasani et al., 2021; Shevlane et al., 2023). Alignment techniques (Ouyang et al., 2022; … view at source ↗

**Figure 2.** An illustration of PICACO. PICACO alternates between the following two steps: 1) in the Response Enhancement step, PICACO samples responses with current best meta-instruction e t−1 and updates both the aligned response pool and the regularization response pool according to qω and qϕ; 2) in the Instruction Refinement step, it searches for the meta-instruction that maximizes TCe given the current response pools, {y t … view at source ↗

**Figure 3.** Conformity score statistics of Q+IF, MODULAR PLURALISM, and PICACO across four numbers of HH values. choices discussed in Sec. 3.2 indeed play a significant role in PICACO’s superiority. d) Smaller models appear more sensitive to the choice of ICA methods. For GPT-3.5-Turbo and Gemini-1.5-Flash, most baselines achieve better alignment than the vanilla query (Q), particularly on HH values. In contrast, LLaMA-3.1-8B-… view at source ↗

Figures from the paper (10 more)

**Figure 4.** Figure 4: (a) Negligible changes in overall conformity brought by using GPT-4o for meta-instruction sampling. (b) Overall conformity of Q, Q+IF, OPRO, and PICACO with the two LLMs in [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

**Figure 5.** Figure 5: GPT-3.5-Turbo’s continuation of ”An unpopular opinion...” when aligned with two Schwartz values, Tradition and Hedonism, using PICACO, Q+IF, and MODULAR PLURALISM [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]

**Figure 6.** Figure 6: Example demonstrations used in this paper. Notably, as a variational distribution, qω is not required to accurately approximate the true posterior (the genuine value conformity). The closeness of qω to the posterior only affects how tight the bound is, so qω only needs…

**Figure 7.** Figure 7: The template for response sampling in the Response Enhancement step. For example, to compute p(y|x), we sample n = 10 responses {y1, y2, ..., yn} from p in a single call with input x, and p(y|x) = 1 n Xn i=1 Sim(y, yi). (20) D.1.3. VIM SAMPLING The following provides a…

**Figure 8.** Figure 8: The template for meta-instruction optimization in the Instruction Refinement step. D.2. Baselines Q+IF (Ganguli et al., 2023) is a na¨ıve ICA method that uses a meta-instruction as the system prompt to align the model with the intended values. In this paper, Q+IF emplo…

**Figure 9.** Figure 9: The template for value conformity evaluation. D.3. Evaluation protocol To evaluate the meta-instructions produced by PICACO, we collect responses for each test set following the same procedure used in the Response Enhancement step ( [PITH_FULL_IMAGE:figures/full_fig_p…

**Figure 10.** Figure 10: The template used for evaluating Relevance in Schwartz value alignment. GPU The optimization process of PICACO doesn’t require any GPU, and the GPU RAM usage only depends on the implementation of qω or qϕ. In this paper, qω is implemented using the GPT-4o-mini API, an…

**Figure 11.** Figure 11: GPT-3.5-Turbo’s continuation of ”When I am perseverant, I...” when aligned with Self-direction and Conformity using PICACO, Q+IF, and MODULAR PLURALISM. The detailed results from the main paper are presented below. The main results of Gemini-1.5-Flash are shown in [P…

**Figure 12.** Figure 12: Two optimized meta-instructions for the Confucianism composition. G. LLM usage In accordance with guidelines regarding the use of LLMs, we clarify that ChatGPT was used only to correct minor grammatical errors and to polish the phrasing of certain sentences in this pa…

**Figure 13.** Figure 13: Two responses from GPT-3.5-Turbo for the Confucianism composition; the one guided by MP+SYSTEM 1 is severely superficial. 40 [PITH_FULL_IMAGE:figures/full_fig_p040_13.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

31 extracted references · 17 canonical work pages

[1]

• Ask a clarifying question or restate their concern to show genuine empathy

Hear (Benevolence) • Begin by validating the user’s feelings or situation. • Ask a clarifying question or restate their concern to show genuine empathy

work page
[2]

• Briefly explain why this standard matters and how it guides your advice

Ethics (Conformity) • Cite at least one relevant law, ethical principle, or community norm. • Briefly explain why this standard matters and how it guides your advice

work page
[3]

• Note its origin and show respectfully how it informs modern solutions

Legacy (Tradition) • Introduce cultural references (proverb, custom, story) tied to the user’s context. • Note its origin and show respectfully how it informs modern solutions

work page
[4]

• Propose clear, low-barrier safeguards and a fallback option

Protections (Security) • Identify two potential risks (emotional, physical, informational, or social). • Propose clear, low-barrier safeguards and a fallback option

work page
[5]

HELPS Check

Self-Review • Conclude with a three-bullet “HELPS Check” labeling how you addressed each pillar. • Confirm your tone is inclusive, non-stereotypical, and balanced. Before sending, ask yourself: • Is my empathy specific and sincere? • Are norms accurate and locally relevant? • Is cultural content respectful and purposeful? • Are all risks paired with concr...

work page arXiv
[7]

doi: 10.18653/v1/2024.emnlp-main.85

Association for Computational Linguistics. doi: 10.18653/v1/2024.emnlp-main.85. URL https:// aclanthology.org/2024.emnlp-main.85/. Gupta, R., Sullivan, R., Li, Y ., Phatale, S., and Rastogi, A. Robust multi-objective preference alignment with online dpo. InProceedings of the Thirty-Ninth AAAI Conference on Artificial Intelligence and Thirty-Seventh Confer...

doi:10.18653/v1/2024.emnlp-main.85 2024
[15]

QIU, Y ., Zhao, Z., Ziser, Y ., Korhonen, A., Ponti, E., and Cohen, S

URL https://openreview.net/forum? id=6Mxhg9PtDE. QIU, Y ., Zhao, Z., Ziser, Y ., Korhonen, A., Ponti, E., and Cohen, S. B. Spectral editing of activations for large language model alignment. InThe Thirty-eighth Annual Conference on Neural Information Processing Systems,

work page
[16]

Rafailov, R., Sharma, A., Mitchell, E., Manning, C

URL https://openreview.net/forum? id=pqYceEa87j. Rafailov, R., Sharma, A., Mitchell, E., Manning, C. D., Ermon, S., and Finn, C. Direct preference optimization: Your language model is secretly a reward model. InThirty- seventh Conference on Neural Information Processing Systems, 2023. URL https://openreview.net/ forum?id=HPuSIXJaa9. Rame, A., Couairon, G....

work page 2023

Show all 31 references

[17]

Ramesh, S

URL https://openreview.net/forum? id=lSbbC2VyCu. Ramesh, S. S., Hu, Y ., Chaimalas, I., Mehta, V ., Sessa, P. G., Ammar, H. B., and Bogunovic, I. Group robust preference optimization in reward-free RLHF. InThe Thirty-eighth Annual Conference on Neural Information Processing Sy...

2024
[19]

Sanz-Guerrero, M

URL https://proceedings.mlr.press/ v202/santurkar23a.html. Sanz-Guerrero, M. and V on Der Wense, K. Corrective in- context learning: Evaluating self-correction in large lan- guage models. In Drozd, A., Sedoc, J., Tafreshi, S., Akula, A., and Shu, R. (eds.),The Sixth Workshop o...

2025 arXiv
[25]

power”and v2=“anti-power

URL https://openreview.net/forum? id=EdIGMCHk4l. Zhang, J., Elgohary, A., Magooda, A., Khashabi, D., and Durme, B. V . Controllable safety alignment: Inference- time adaptation to diverse safety requirements. InThe Thirteenth International Conference on Learning Rep- resentati...

2022
[26]

When I am perseverant, I

and HH-RLHF (Bai et al., 2022a) for Schwartz and HH value alignment, respectively. From FULCRA, we collect all samples embodying at least three values in theConfucianismcomposition (Benevolence, Conformity, Tradition, Security), yielding only 1.5k samples; qualified samples fo...
[41]

findings-acl.41/

URL https://aclanthology.org/2025. findings-acl.41/. Chen, X., Liao, B., Qi, J., Eustratiadis, P., Monz, C., Bisazza, A., and Rijke, M. The sifo benchmark: Investigating the sequential instruction following ability of large language models. InFindings of the Association for Co...

2025
[185]

naacl-long.185/

URL https://aclanthology.org/2024. naacl-long.185/. Wei, T., Wen, W., Qiao, R., Sun, X., and Ma, J. Rocketeval: Efficient automated LLM evaluation via grading checklist. InThe Thirteenth International Conference on Learning Representations, 2025. URL https://openreview. net/fo...

2024
[210]

emnlp-main.210/

URL https://aclanthology.org/2024. emnlp-main.210/. Wang, Q., Tang, Z., and He, B. Can LLM simulations truly reflect humanity? a deep dive. InThe Fourth Blogpost Track at ICLR 2025, 2025. URL https: //openreview.net/forum?id=dMrhmQdrdW. Wang, Y ., Li, H., Han, X., Nakov, P., a...

2024
[242]

findings-emnlp.242/

URL https://aclanthology.org/2024. findings-emnlp.242/. Lyu, X., Min, S., Beltagy, I., Zettlemoyer, L., and Hajishirzi, H. Z-ICL: Zero-shot in-context learning with pseudo- demonstrations. In Rogers, A., Boyd-Graber, J., and Okazaki, N. (eds.),Proceedings of the 61st Annual Me...

2024 doi
[301]

findings-emnlp.301/

URL https://aclanthology.org/2020. findings-emnlp.301/. Gemini, Anil, R., Borgeaud, S., Alayrac, J.-B., Yu, J., Sori- cut, R., Schalkwyk, J., Dai, A. M., Hauth, A., Millican, K., Silver, D., Johnson, M., Antonoglou, I., Schrittwieser, J., Glaese, A., Chen, J., Pitler, E., Lill...

2020 arXiv
[326]

findings-naacl.326/

URL https://aclanthology.org/2025. findings-naacl.326/. Choi, H. K. and Li, Y . PICLe: Eliciting diverse behaviors from large language models with persona in-context learn- ing. In Salakhutdinov, R., Kolter, Z., Heller, K., Weller, A., Oliver, N., Scarlett, J., and Berkenkamp,...

2025
[486]

naacl-long.486/

URL https://aclanthology.org/2024. naacl-long.486/. Yao, X.An Introduction to Confucianism. Introduction to Religion. Cambridge University Press, 2000. ISBN 9780521644303. URL https://books.google. co.in/books?id=tAE2OJ9bPG0C. Yao, X.The Encyclopedia of Confucianism. Routledge,

2024
[585]

acl-long.585/

URL https://aclanthology.org/2025. acl-long.585/. Ye, J., Wang, Y ., Huang, Y ., Chen, D., Zhang, Q., Moniz, N., Gao, T., Geyer, W., Huang, C., Chen, P.-Y ., Chawla, N. V ., and Zhang, X. Justice or prejudice? quantifying biases in LLM-as-a-judge. InThe Thirteenth Internationa...

2025
[637]

findings-emnlp.637/

URL https://aclanthology.org/2024. findings-emnlp.637/. Holm, S. A simple sequentially rejective multiple test procedure.Scandinavian Journal of Statistics, 6(2):65– 70, 1979. ISSN 03036898, 14679469. URL http: //www.jstor.org/stable/4615733. Hopfe, L. M. and Woodward, M. R.Re...

2024
[721]

emnlp-main.721/

URL https://aclanthology.org/2023. emnlp-main.721/. Dillion, D., Mondal, D., Tandon, N., and Gray, K. AI language model rivals expert ethicist in perceived moral expertise.Scientific Reports, 15(1):4084, 2025. doi: 10.1038/s41598-025-86510-0. Dong, H., Xiong, W., Goyal, D., Zh...

2023
[838]

acl-long.838/

URL https://aclanthology.org/2025. acl-long.838/. Han, S., Shenfeld, I., Srivastava, A., Kim, Y ., and Agrawal, P. Value augmented sampling for language model alignment and personalization. InICLR 2024 Workshop on Reliable and Responsible Foundation Models, 2024. URLhttps: //a...

2025 arXiv
[972]

findings-emnlp.972/

URL https://aclanthology.org/2024. findings-emnlp.972/. Pitis, S., Xiao, Z., Roux, N. L., and Sordoni, A. Improving context-aware preference modeling for language mod- els. InThe Thirty-eighth Annual Conference on Neural 15 PICACO: Pluralistic In-Context Value Alignment via To...

2024
[1937]

URL http://www

ISSN 01621459, 1537274X. URL http://www. jstor.org/stable/2279372. Ganguli, D., Askell, A., Schiefer, N., Liao, T. I., Lukoˇsi¯ut˙e, K., Chen, A., Goldie, A., Mirhoseini, A., Olsson, C., Hernandez, D., et al. The capacity for moral self- correction in large language models, 20...

2023
[1969]

URL https: //www.goodreads.com/book/show/ 11900202-the-pursuit-of-happiness

ISBN 9780451037343. URL https: //www.goodreads.com/book/show/ 11900202-the-pursuit-of-happiness. Roosevelt, F. D.Public Papers of the Presidents of the United States: F .D. Roosevelt, 1938, Volume 7. U.S. Government Printing Office, Washington, D.C., 1938. Accessed: 2025-02-08...

1938 arXiv
[2013]

Ye, H., Zhang, T., Xie, Y ., Zhang, L., Ren, Y ., Zhang, X., and Song, G

ISBN 9780415515221. Ye, H., Zhang, T., Xie, Y ., Zhang, L., Ren, Y ., Zhang, X., and Song, G. Generative psycho-lexical approach for constructing value systems in large language models. In Che, W., Nabende, J., Shutova, E., and Pilehvar, M. T. (eds.),Proceedings of the 63rd An...

doi
[2022]

cc/paper_files/paper/2022/file/ b1efde53be364a73914f58805a001731-Paper-Conference

URL https://proceedings.neurips. cc/paper_files/paper/2022/file/ b1efde53be364a73914f58805a001731-Paper-Conference. pdf. Pal, S., Das, S., and Srihari, R. K. Beyond discrete personas: Personality modeling through journal intensive conver- sations. In Rambow, O., Wanner, L., Ap...

2022
[2023]

Jiang, A

URL https://openreview.net/forum? id=g0QovXbFw3. Jiang, A. Q., Sablayrolles, A., Roux, A., Mensch, A., Savary, B., Bamford, C., Chaplot, D. S., de las Casas, D., Hanna, E. B., Bressand, F., Lengyel, G., Bour, G., Lample, G., 13 PICACO: Pluralistic In-Context Value Alignment vi...

2024 arXiv
[2024]

Cohen, J

URL https://proceedings.mlr.press/ v235/choi24e.html. Cohen, J. Statistical power analysis.Current Di- rections in Psychological Science, 1(3):98–101, 1992. ISSN 09637214. URL http://www.jstor.org/ stable/20182143. Confucius.The Analects: Xue Er (Learning). Penguin Books, 1998...

1992
[2025]

ISBN 979-8-89176-251-0

Association for Computational Linguistics. ISBN 979-8-89176-251-0. doi: 10.18653/v1/2025.acl-long

2025 doi

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

• Ask a clarifying question or restate their concern to show genuine empathy

Hear (Benevolence) • Begin by validating the user’s feelings or situation. • Ask a clarifying question or restate their concern to show genuine empathy

work page

[2] [2]

• Briefly explain why this standard matters and how it guides your advice

Ethics (Conformity) • Cite at least one relevant law, ethical principle, or community norm. • Briefly explain why this standard matters and how it guides your advice

work page

[3] [3]

• Note its origin and show respectfully how it informs modern solutions

Legacy (Tradition) • Introduce cultural references (proverb, custom, story) tied to the user’s context. • Note its origin and show respectfully how it informs modern solutions

work page

[4] [4]

• Propose clear, low-barrier safeguards and a fallback option

Protections (Security) • Identify two potential risks (emotional, physical, informational, or social). • Propose clear, low-barrier safeguards and a fallback option

work page

[5] [5]

HELPS Check

Self-Review • Conclude with a three-bullet “HELPS Check” labeling how you addressed each pillar. • Confirm your tone is inclusive, non-stereotypical, and balanced. Before sending, ask yourself: • Is my empathy specific and sincere? • Are norms accurate and locally relevant? • Is cultural content respectful and purposeful? • Are all risks paired with concr...

work page arXiv

[6] [7]

doi: 10.18653/v1/2024.emnlp-main.85

Association for Computational Linguistics. doi: 10.18653/v1/2024.emnlp-main.85. URL https:// aclanthology.org/2024.emnlp-main.85/. Gupta, R., Sullivan, R., Li, Y ., Phatale, S., and Rastogi, A. Robust multi-objective preference alignment with online dpo. InProceedings of the Thirty-Ninth AAAI Conference on Artificial Intelligence and Thirty-Seventh Confer...

doi:10.18653/v1/2024.emnlp-main.85 2024

[7] [15]

QIU, Y ., Zhao, Z., Ziser, Y ., Korhonen, A., Ponti, E., and Cohen, S

URL https://openreview.net/forum? id=6Mxhg9PtDE. QIU, Y ., Zhao, Z., Ziser, Y ., Korhonen, A., Ponti, E., and Cohen, S. B. Spectral editing of activations for large language model alignment. InThe Thirty-eighth Annual Conference on Neural Information Processing Systems,

work page

[8] [16]

Rafailov, R., Sharma, A., Mitchell, E., Manning, C

URL https://openreview.net/forum? id=pqYceEa87j. Rafailov, R., Sharma, A., Mitchell, E., Manning, C. D., Ermon, S., and Finn, C. Direct preference optimization: Your language model is secretly a reward model. InThirty- seventh Conference on Neural Information Processing Systems, 2023. URL https://openreview.net/ forum?id=HPuSIXJaa9. Rame, A., Couairon, G....

work page 2023