{"paper":{"title":"Aligning Deep Implicit Preferences by Learning to Reason Defensively","license":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/","headline":"CDRA reframes LLM alignment as a critique-driven reasoning process to infer unstated goals, contexts, and risk tolerances.","cross_cats":[],"primary_cat":"cs.AI","authors_text":"Peiming Li, Shiyu Li, Xi Chen, Yang Tang, Zhiyuan Hu","submitted_at":"2025-10-13T09:26:47Z","abstract_excerpt":"Personalized alignment is crucial for enabling Large Language Models (LLMs) to engage effectively in user-centric interactions. However, current methods face a dual challenge: they fail to infer users' deep implicit preferences (including unstated goals, semantic context and risk tolerances), and they lack the defensive reasoning required to navigate real-world ambiguity. This cognitive gap leads to responses that are superficial, brittle and short-sighted. To address this, we propose Critique-Driven Reasoning Alignment (CDRA), which reframes alignment from a scalar reward-matching task into a"},"claims":{"count":4,"items":[{"kind":"strongest_claim","text":"Experiments demonstrate that CDRA excels at discovering and aligning with users' true preferences while executing robust reasoning.","source":"verdict.strongest_claim","status":"machine_extracted","claim_id":"C1","attestation":"unclaimed"},{"kind":"weakest_assumption","text":"The assumption that a simulated multi-faceted cognitive council can reliably produce critique-annotated reasoning chains that accurately deconstruct query semantics and reveal latent risks for the DeepPref benchmark curation.","source":"verdict.weakest_assumption","status":"machine_extracted","claim_id":"C2","attestation":"unclaimed"},{"kind":"one_line_summary","text":"CDRA reframes LLM alignment as a critique-driven reasoning process using the DeepPref benchmark and Pers-GenPRM to infer implicit preferences and guide policy via process-level RL.","source":"verdict.one_line_summary","status":"machine_extracted","claim_id":"C3","attestation":"unclaimed"},{"kind":"headline","text":"CDRA reframes LLM alignment as a critique-driven reasoning process to infer unstated goals, contexts, and risk tolerances.","source":"verdict.pith_extraction.headline","status":"machine_extracted","claim_id":"C4","attestation":"unclaimed"}],"snapshot_sha256":"e91c24ba52e64970a228d3d5a66e350a0fb76c1094886baa42c91f765cb16aec"},"source":{"id":"2510.11194","kind":"arxiv","version":3},"verdict":{"id":"2a53780c-9319-497a-bcb6-acc29a7cc340","model_set":{"reader":"grok-4.3"},"created_at":"2026-05-18T08:07:56.171496Z","strongest_claim":"Experiments demonstrate that CDRA excels at discovering and aligning with users' true preferences while executing robust reasoning.","one_line_summary":"CDRA reframes LLM alignment as a critique-driven reasoning process using the DeepPref benchmark and Pers-GenPRM to infer implicit preferences and guide policy via process-level RL.","pipeline_version":"pith-pipeline@v0.9.0","weakest_assumption":"The assumption that a simulated multi-faceted cognitive council can reliably produce critique-annotated reasoning chains that accurately deconstruct query semantics and reveal latent risks for the DeepPref benchmark curation.","pith_extraction_headline":"CDRA reframes LLM alignment as a critique-driven reasoning process to infer unstated goals, contexts, and risk tolerances."},"integrity":{"clean":true,"summary":{"advisory":0,"critical":0,"by_detector":{},"informational":0},"endpoint":"/pith/2510.11194/integrity.json","findings":[],"available":true,"detectors_run":[],"snapshot_sha256":"c28c3603d3b5d939e8dc4c7e95fa8dfce3d595e45f758748cecf8e644a296938"},"references":{"count":0,"sample":[],"resolved_work":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57","internal_anchors":0},"formal_canon":{"evidence_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"}