Pith. sign in

REVIEW 5 major objections 6 minor 80 references

Multi-Domain Explainability of Preferences

T0 review · 5 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Pith's one-line read LLM-discovered concepts reveal what drives preference judgments.

desk verdict A genuinely useful concept-based explainability pipeline for preference mechanisms, with a real but addressable gap: the concept dictionary is built from human preferences and untested for the other eleven mechanisms. read the letter →

arxiv 2505.20088 v2 pith:RMYE6REX submitted 2025-05-26 cs.CL

classification cs.CL
keywords concept-basedexplainabilityLLM-as-a-Judgerewardmodelspreferencelearningmulti-domainwhite-boxmodelhierarchicalregressionLLMalignment
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Human preferences, LLM-as-a-Judge, and reward models all select between responses, but what actually drives their choices is largely opaque. This paper tries to establish that a fully automated pipeline can make those drivers visible: an LLM discovers human-interpretable concepts that separate chosen from rejected responses, represents each query-response pair as a concept vector, and trains a white-box hierarchical regression that attributes preference decisions to shared and domain-specific concept weights. The authors claim these explanations are usable: when LLMs generate responses guided by the top concepts of a judge, that judge prefers them over unguided or randomly guided responses, and re-prompting judges with concepts important to humans resolves tie cases in line with human preferences. If true, this gives alignment and evaluation work a scalable way to inspect and steer the mechanisms that decide what counts as a good response.

What carries the argument

The load-bearing object is the Hierarchical Multi-Domain Regression (HMDR) model, a white-box logistic regression whose per-domain weight vector is the sum $\beta^{(d)} = b + s^{(d)}$ of a shared vector $b$ and a domain-specific deviation $s^{(d)}$. Its objective adds the standard domain-specific log-loss and an extra shared loss that forces $b$ alone to be predictive, with $\ell^1$ penalties on both components for sparsity. This decomposition is what lets a single model explain twelve mechanisms across eight domains while still generalizing to unseen domains using only the shared weights; explanation strength is the lift $0.5(b_j+s^{(d)}_j)$ for concept $j$ in domain $d$.

What would settle it

Train the same HMDR pipeline with concepts discovered separately from each explained mechanism's own preference labels and compare accuracy and top concepts against the human-derived 624-concept dictionary; if the mechanism-specific dictionary materially improves prediction or surfaces top concepts absent from the shared set, then the paper's claim that the explanations capture that mechanism's drivers is falsified.

Watch

Extended reading notes

Core claim

The central discovery is that concept-based explanations of preferences can be produced end-to-end without manual concept curation, and that those explanations capture the concepts that actually influence each mechanism's choices. The pipeline discovers concepts from preference triplets by batching queries and asking an LLM to propose concepts that distinguish chosen from rejected responses, deduplicates them into a 624-concept dictionary shared across eight domains, and encodes each triplet as a vector (comparison-based Comp-rep or score-difference Score-rep). A white-box Hierarchical Multi-Domain Regression model then predicts each mechanism's choices, decomposing concept weights into a shared component and domain-specific deviations; explanation importance is read off as the expected probability lift, about $50(b_j+s^{(d)}_j)$. The paper reports that this explainable model matches the strongest few-shot LLM ensemble at 66.0% accuracy on human preferences, and that in application-driven tests explanations guide generation to win over judges and improve tie resolution with humans by up to 10.8 points.

Load-bearing premise

The concept vocabulary is discovered from human-preference triplets and then assumed to cover the concepts that LLM judges and reward models actually use, with no test of that coverage.

Editorial extensions

If this is right

  • If a judge's top concepts are fed into a generator, the judge prefers the resulting responses over vanilla ones, with win-rate gains up to +20.8 points over randomly chosen concepts for Gemini-Pro.
  • Re-prompting a judge on tie cases with concepts humans care about raises agreement with human preferences by up to 10.8 points, with the human-minus-judge weight difference working best.
  • The white-box model matches a strong eight-prompt few-shot Gemini-Flash ensemble at 66.0% accuracy on human preferences while remaining interpretable.
  • In leave-one-domain-out testing, the shared weights alone transfer to a held-out domain, so the explanations are not overfit to the eight training domains.
  • Automatically discovered concepts recover known manual findings (humans weigh clarity, authority, and confidence; LLMs weigh factuality and helpfulness) and add domain-specific concepts such as concealment in safety.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the shared-versus-specific weight decomposition gives a compact signature of each mechanism, so one could compare a new reward model to human preferences by the distance between their weight vectors without collecting new explanations.
  • Beyond the paper: since the dictionary is built from human-labeled triplets, a cheap coverage test—ask each judge to name its own concepts and check overlap with the 624-concept set—would determine whether judge-specific drivers are missing; the paper does not run this test.
  • Beyond the paper: the Tie Break result suggests concept-guided re-prompting could serve as a general position-bias correction in LLM evaluation, applied not only to exact ties but to near-ties or unstable predictions.
  • Beyond the paper: because Score-rep gives finer-grained concept values, HMDR weights trained on it could be used as features to predict which mechanisms will agree with human raters on a new domain, a testable extension of the authors' analysis.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. This paper proposes a four-stage pipeline for concept-based explainability of preference mechanisms (human preferences, LLM-as-a-Judge models, and reward models) across eight domains. An LLM (Gemini-1.5-Pro) discovers candidate concepts from preference triplets, represents each triplet as a concept vector (Comp-rep or Score-rep), and a white-box Hierarchical Multi-Domain Regression (HMDR) model learns shared and domain-specific concept weights. The authors evaluate the pipeline by predicting preferences for twelve mechanisms, validating LLM concept annotations with a human study, and running two application-driven tests: Judge Hack (generating responses guided by top-ranked concepts improves the explained judge's preference) and Tie Break (re-prompting judges with human-explanation concepts improves agreement with human preferences). The paper claims the method is fully automated and that the explanations capture what actually drives each mechanism, with prediction performance comparable to the strongest few-shot LLM ensemble baselines.

Significance. If the identified gaps are addressed, this would be a substantial contribution to NLP interpretability: it scales concept-based explainability to multiple domains and mechanisms using LLMs, introduces a white-box hierarchical regression model that decomposes effects into shared and domain-specific components, and proposes application-driven protocols for evaluating explanation quality. The scale of the study (twelve mechanisms, eight domains, 624 concepts) and the release of code are strong assets. The demonstration that explanation-guided generation is preferred by the explained judges, and that concept-guided re-prompting can improve judge-human agreement, are valuable and credible directions. However, the central claim that explanations capture what drives each mechanism is currently under-supported by the concept-coverage evidence and by the lack of statistical rigor in several head-to-head comparisons. These issues are fixable within the manuscript's scope, but they materially affect the strength of the claims as written.

major comments (5)
  1. [Section 3.1 and Appendix C.1.2] The abstract and Section 1 claim the method is 'fully automated,' but Appendix C.1.2 states that ten concepts were 'manually extracted' and designated as fixed concepts, and that these fixed concepts are explicitly injected into the discovery prompt in 50% of batches. More importantly, concept discovery is performed only on triplets labeled by human preferences in the main setup, and the planned ablation using Gemini-1.5-Flash labels 'was not completed due to budget.' Because the same 624-concept vocabulary is then reused to explain all twelve mechanisms, there is no evidence that the vocabulary covers concepts actually used by the nine LLM judges and two reward models. If a mechanism relies on a concept outside this dictionary, the HMDR regression can only attribute its decisions to correlated proxies, which would make the explanation incomplete or actively misleading. The authors should provide a coverage test (e.g., discover concepts from a held-out mechanism's labels on a subset and compare, or at least analyze whether prediction errors concentrate on examples where the relevant concepts are out of vocabulary) and should either remove the 'fully automated' claim or qualify it in light of the manual fixed concepts.
  2. [Figure 3 and Table 1] The headline accuracy comparisons are reported without error bars, confidence intervals, or significance tests. For human preferences, Ours (Comp-rep) achieves 66.0% and the strongest LLM-as-a-Judge baseline (the Gemini-Flash 10-shot ensemble) also achieves 66.0%, and several rows in Table 1 differ by less than one percentage point. Without variance estimates over the 25 in-domain splits and 5 out-of-domain seeds, the claim that the method is 'comparable to or better than' black-box baselines is not statistically supported. Please report standard deviations and run paired significance tests for the primary head-to-head comparisons, at least for the human-preference row and the mean rows in Tables 1 and 4.
  3. [Section 5.2, Table 2] The Judge Hack results are tested only against the 50% chance level (with Bonferroni correction), not against the random-concept control. For the QRM judge with Gemini-Flash generator, the explanation-guided win rate is 50.1%, barely above chance, while the random-concept control is 36.0%. A paired statistical comparison between explanation-guided and random-guided responses across the 4,800 comparisons (or per judge-generator pair) is needed to support the claim that explanations outperform random concepts 'by a much larger margin.' Adding confidence intervals for the win rates would also help the reader assess the reliability of the effect.
  4. [Section 5.2, Table 3] The text claims a 'consistent and meaningful trend' with the accuracy ranking 'random ≤ LaaJ ≤ human ≤ differences' across judges, but Table 3 contains violations: for Gemini-1.5-Flash zero-shot, Random Concepts (+1.8) outperforms the LaaJ's own explanation (+0.0), and for GPT-4o-mini with 10-shot, Random Concepts (+5.2) outperforms the GPT-4o-m explanation (+2.1). These differences are also reported without significance testing. The monotonic-ranking claim is therefore stronger than the data support. Please recompute with paired significance tests and temper the conclusion accordingly, or explain why these violations are consistent with the proposed narrative.
  5. [Appendix C.3] The lift decomposition derives E[lift_j] ≈ 0.5 (b_j + s_j) by assuming E[σ(z)] = 0.5 from the symmetric augmentation and by asserting that the second-order Taylor remainder is negligible because |Δz_j| < 1/4. The bound on |Δz_j| is not derived or empirically verified for the trained models, and the expectation requires that the test distribution be symmetric in the same way as the augmented training distribution. Please provide empirical verification of the bound and of the symmetry assumption on the actual test sets, or present the result as an approximation with these caveats clearly stated.
minor comments (6)
  1. [Abstract and Figure 3] The abstract states that the method 'outperforms baselines,' while Section 5.1 reports accuracy 'comparable to the strongest baseline' (66.0% vs 66.0%). Please revise the abstract to say 'comparable to or better than' to avoid overclaiming.
  2. [Section 5.1] The sentence 'we find that CoT prompting degrades LaaJ performance, whereas few-shot prompting improves it, consistent with Calderon et al. (2025)' appears twice consecutively in the same paragraph. Please remove the duplicate.
  3. [Section 1] The contribution list contains the phrase 'an fully automated method'; this should be 'a fully automated method.'
  4. [Table 2] The caption and text report that there are 4,800 total comparisons, but the per-cell number of comparisons used for each judge-generator pair is not broken out. Please add this information so that the win rates can be interpreted with appropriate confidence.
  5. [Section 4.2] The claim that 'our method is the first complete pipeline for preferences explainability' is strong given prior work on concept-based regression and LLM-assisted concept discovery; please soften the claim or situate it more carefully against the baselines discussed in Section 2.
  6. [Table 3] The baseline row labeled '0.0' is not explicitly defined in the table; the text indicates it is the accuracy without resolving ties, but this should be stated in the caption for clarity.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the derivation chain is self-contained and validated against external benchmarks.

full rationale

The central pipeline (concept discovery, concept representations, HMDR regression, and explanation extraction) contains no step in which an output is defined in terms of its own prediction or in which a fitted parameter is renamed as a prediction. Concept discovery (Section 3.1) uses a held-out portion of the human-preference data to generate a vocabulary; the concept vectors (Section 3.2) are produced by Gemini-1.5-Pro from query and response content without access to the mechanism labels being explained; the HMDR (Section 3.3) is a standard logistic regression fit to each mechanism's labels, and prediction accuracy is measured on held-out test splits. The two application-driven evaluations are not forced: in Judge Hack, explanation-guided responses are generated by models other than the judge, and the comparison with random concepts provides a control; in Tie Break, the judge is re-prompted with human-derived concepts and agreement with human preference is an external criterion. The only same-model condition (Gemini-Pro generating both concept features and labels) is a potential confounding factor, but the paper's cross-model results (e.g., GPT-4o judge, QRM reward model) and random-concept baselines show the findings are not an artifact of self-confirmation. Self-citations (Calderon et al., 2025) are used for a statistical validation procedure and a side observation about CoT prompting, not as a load-bearing uniqueness or derivation step, and the cited test has independent statistical content. The paper explicitly flags its own limitations (Section 7; Appendix C.1.2) regarding linearity, causality, computational cost, and the incomplete Flash-preference discovery ablation; these are validity and coverage concerns, not circular reductions. No quoted equation or fitted value is reused as its own evidence, and the central explanatory claims are tested against held-out data and external baselines rather than being derived from the fitted values themselves.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim rests on several unproven premises: the validity of the alt-test, the sufficiency of the concept vocabulary for all mechanisms, the symmetry of the test distribution in the lift calculation, and the reliability of Reddit upvotes as human preference labels. The free parameters are mostly standard regularization hyperparameters, but the ten manually fixed concepts are a human-chosen addition that weakens the automation claim.

free parameters (5)
  • alpha_shared_loss_weight = 1/|D| = 0.125
    Hyperparameter balancing the shared loss against domain-specific losses. Chosen by hand and fixed across all experiments.
  • lambda_b_shared_L1 = selected from {2/|D|^2, 1/(2|D|), 1/|D|}
    L1 regularization strength on the shared weight vector, tuned via 5-fold cross-validation.
  • lambda_s_specific_L1 = selected from {1/|D|^2, 2/|D|^2, 1/(2|D|), 1/|D|}
    L1 regularization strength on domain-specific deviations, tuned via 5-fold cross-validation.
  • shared_concept_threshold = 4 out of 8 domains
    A concept is called 'shared' if discovered in at least half the domains. Hand-set threshold that determines the shared/specific split.
  • ten_fixed_concepts = Specificity, Clarity, Relevance, Helpfulness, Empathy, Accuracy, Informativeness, Creativity, Safety, Engagement
    Manually extracted common concepts injected into the discovery prompt in 50% of batches. This is a human intervention that contradicts the 'fully automated' claim.
assumptions (5)
  • domain assumption The Alternative Annotator Test (Calderon et al., 2025) is a valid statistical procedure for replacing human annotators with LLMs.
    The paper uses this test to validate Gemini-Pro concept annotations in Section 5.1 and Appendix B.1, and the test comes from the same author group.
  • domain assumption LLM concept annotations are reliable enough to serve as input features.
    Passes the alt-test on a 400-annotation human study, but the annotator agreement is low (kappa = 0.27), indicating the task is subjective and the ground truth is soft.
  • domain assumption The concept vocabulary discovered from human-preference triplets is sufficient to explain all twelve mechanisms.
    Section 3.1 and Appendix C.1.2: the main concept discovery uses human preferences; the ablation using flash preferences was not completed due to budget, so coverage for other mechanisms is unproven.
  • ad hoc to paper The test distribution is symmetric in the same way as the training augmentation, so E[sigma(z)] = 0.5 in the lift derivation.
    Appendix C.3 uses this to simplify the global lift to 50*(b_j + s_j). The training set is augmented with reversed examples, but the test set need not be symmetric.
  • domain assumption Reddit upvotes with the stated filtering criteria are a reliable operationalization of human preference.
    Section 4.3 defines the human preference labels; the paper does not validate this choice against other human preference sources.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Multi-Domain Explainability of Preferences." pith.science (2026). https://pith.science/paper/RMYE6REX

@misc{pith2026250520088,
  author       = {Pith},
  title        = {Pith review of: Multi-Domain Explainability of Preferences},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RMYE6REX}},
  note         = {Machine review of arXiv:2505.20088}
}
read the original abstract

Preference mechanisms, such as human preference, LLM-as-a-Judge (LaaJ), and reward models, are central to aligning and evaluating large language models (LLMs). Yet, the underlying concepts that drive these preferences remain poorly understood. In this work, we propose a fully automated method for generating local and global concept-based explanations of preferences across multiple domains. Our method utilizes an LLM to identify concepts that distinguish between chosen and rejected responses, and to represent them with concept-based vectors. To model the relationships between concepts and preferences, we propose a white-box Hierarchical Multi-Domain Regression model that captures both domain-general and domain-specific effects. To evaluate our method, we curate a dataset spanning eight challenging and diverse domains and explain twelve mechanisms. Our method achieves strong preference prediction performance, outperforming baselines while also being explainable. Additionally, we assess explanations in two application-driven settings. First, guiding LLM outputs with concepts from LaaJ explanations yields responses that those judges consistently prefer. Second, prompting LaaJs with concepts explaining humans improves their preference predictions. Together, our work establishes a new paradigm for explainability in the era of LLMs.

Figures

Figures reproduced from arXiv: 2505.20088 by the authors.

Figure 1
Figure 1. Top – Preference Mechanisms: Given a triplet of a user query q and two responses, r1 and r2, each mechanism selects a preferred response. Our work automatically discovers concepts for global ex￾planations within a multi-domain learning framework, providing a structured view of how concepts influence preferences generally (shared) and within each domain (specific). Bottom – An Explanation: Explaining the QRM reward m… view at source ↗
Figure 2
Figure 2. Method Illustration: Given a dataset of triplets (q, r1, r2), our four-stage method generates both local and global explanations of preference mechanisms. interpretable concepts and finding their relative im￾pact on the decision of the mechanism. The method consists of four stages as illustrated in [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Human Preference Results: Average accu￾racy across eight domains. Unlike other baselines, our method is explainable while also achieving performance comparable to the strongest LLM-as-a-Judge: an ensem￾ble of Gemini-Flash using eight 10-shot prompts. replace human annotators. The LLM achieves an advantage probability of ρ = 0.85, indicating its annotations were as good as or better (closer to ma￾jority vote) than th… view at source ↗
Figures from the paper (16 more)
Figure 4
Figure 4. Figure 4: Explanations Analysis: Lifts of four mechanisms: human preferences (yellow H), GPT-4o (green O), Gemini-1.5-Flash (red G), and QRM (blue R). Right: shared contributions of 24 concepts, selected from the top ten shared weights of the four mechanisms. Left: For each doma…
Figure 5
Figure 5. Figure 5: Model Agreements: Columns represent the gold labels (with tie cases removed). Each row shows the accuracy of a model against these labels. Our method (top two rows) achieves higher accuracy than other models, excluding those from the same family (highlighted in red). T…
Figure 6
Figure 6. Figure 6: Subdomains and Tasks: For each domain, we show the percentage of user queries annotated by the LLM with each subdomain (top two rows) and task (bottom two rows). Subdomains and tasks were discovered by the LLM and used to batch examples for concept discovery. We retain…
Figure 7
Figure 7. Figure 7: Hyperparameters Analysis: Light-colored bars represent the number of non-zero shared weights, while dark-colored bars indicate the average number of non-zero specific weights. The solid line with star markers represents model accuracy, and the dashed line with X marker…
Figure 8
Figure 8. Figure 8: Explanations of Human Preferences. 0 2 4 6 (%) Expertise Depth of Knowledge Helpfulness Justification Informativeness Relevance Usefulness Solution Orientation Tone Professionalism Shared 0.0 2.5 5.0 7.5 (%) Professionalism Insightfulness Expertise Depth of Knowledge H…
Figure 9
Figure 9. Figure 9: Explanations of Llama-3.1 Preferences. 25 [PITH_FULL_IMAGE:figures/full_fig_p025_9.png]
Figure 10
Figure 10. Figure 10: Explanations of Gemini-1.5-Flash Preferences. 0 2 4 6 (%) Expertise Depth of Knowledge Informativeness Helpfulness Authority Justification Usefulness Confidence User Intent Understanding Relevance Shared 0 2 4 6 (%) Fluency Expertise Conceptual Understanding Depth of …
Figure 11
Figure 11. Figure 11: Explanations of Gemini-1.5-Flash with 10-shots Preferences. 26 [PITH_FULL_IMAGE:figures/full_fig_p026_11.png]
Figure 12
Figure 12. Figure 12: Explanations of Gemini-1.5-Pro Preferences. 0 2 4 (%) Depth of Knowledge Specificity Usefulness Accuracy Professionalism Relevance Helpfulness Expertise Comprehensiveness Informativeness Shared 0 5 (%) Specificity Relevance Depth of Knowledge Usefulness Completeness D…
Figure 13
Figure 13. Figure 13: Explanations of Gemini-1.5-Pro with Chain-of-Thoughts Preferences. 27 [PITH_FULL_IMAGE:figures/full_fig_p027_13.png]
Figure 14
Figure 14. Figure 14: Explanations of GPT-4o Preferences. 0 2 4 (%) Usefulness Expertise Depth of Knowledge Helpfulness Specificity Informativeness Solution Orientation Comprehensiveness Directness Factuality Shared 0 5 10 (%) Specificity Helpfulness Usefulness Expertise Generalizability D…
Figure 15
Figure 15. Figure 15: Explanations of GPT-4o with Chain-of-Thoughts Preferences. 28 [PITH_FULL_IMAGE:figures/full_fig_p028_15.png]
Figure 16
Figure 16. Figure 16: Explanations of GPT-4o-mini Preferences. 0 2 4 (%) Expertise Depth of Knowledge Positive Framing Usefulness Tone Comprehensiveness Helpfulness Solution Orientation Specificity Informativeness Shared 0 2 4 (%) Professionalism User Intent Understanding Expertise Tone De…
Figure 17
Figure 17. Figure 17: Explanations of GPT-4o-mini with Chain-of-Thoughts Preferences. 29 [PITH_FULL_IMAGE:figures/full_fig_p029_17.png]
Figure 18
Figure 18. Figure 18: Explanations of QRM 8b Reward Model Preferences. 0 1 2 3 (%) Neutrality Usefulness Focus Solution Orientation Relevance User Intent Understanding Directness Comprehensiveness Helpfulness Factuality Shared 0 2 4 (%) Non-Judgmental Direct Question Answering User Intent …
Figure 19
Figure 19. Figure 19: Explanations of SkyWork 8b Reward Model Preferences. 30 [PITH_FULL_IMAGE:figures/full_fig_p030_19.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

80 extracted references · 24 canonical work pages

  1. [1]

    Ahmed Alqaraawi, Martin Schuessler, Philipp Wei , Enrico Costanza, and Nadia Berthouze. 2020. https://doi.org/10.1145/3377325.3377519 Evaluating saliency map explanations for convolutional neural networks: a user study . In IUI '20: 25th International Conference on Intelligent User Interfaces, Cagliari, Italy, March 17-20, 2020 , pages 275--285. ACM

  2. [2]

    Mart \' n Arjovsky, L \' e on Bottou, Ishaan Gulrajani, and David Lopez - Paz. 2019. https://arxiv.org/abs/1907.02893 Invariant risk minimization . volume abs/1907.02893

  3. [3]

    Brown, Jack Clark, Sam McCandlish, Chris Olah, Benjamin Mann, and Jared Kaplan

    Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, Nicholas Joseph, Saurav Kadavath, Jackson Kernion, Tom Conerly, Sheer El Showk, Nelson Elhage, Zac Hatfield - Dodds, Danny Hernandez, Tristan Hume, Scott Johnston, Shauna Kravec, Liane Lovitt, Neel Nanda, Catherine Olsson...

  4. [4]

    Peters, and Arman Cohan

    Iz Beltagy, Matthew E. Peters, and Arman Cohan. 2020. https://arxiv.org/abs/2004.05150 Longformer: The long-document transformer . CoRR, abs/2004.05150

  5. [7]

    Shai Ben - David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jennifer Wortman Vaughan. 2010. https://doi.org/10.1007/S10994-009-5152-4 A theory of learning from different domains . volume 79, pages 151--175

  6. [8]

    Steven Bird and Edward Loper. 2004. https://aclanthology.org/P04-3031/ NLTK: the natural language toolkit . In Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics, Barcelona, Spain, July 21-26, 2004 - Poster and Demonstration. ACL

  7. [9]

    McDonald, and Fernando Pereira

    John Blitzer, Ryan T. McDonald, and Fernando Pereira. 2006. https://aclanthology.org/W06-1615/ Domain adaptation with structural correspondence learning . In EMNLP 2006, Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing, 22-23 July 2006, Sydney, Australia , pages 120--128. ACL

  8. [10]

    Nitay Calderon, Eyal Ben - David, Amir Feder, and Roi Reichart. 2022. https://doi.org/10.18653/V1/2022.ACL-LONG.533 Docogen: Domain counterfactual generation for low resource domain adaptation . In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2022, Dublin, Ireland, May 22-27, 2022 , p...

Show all 80 references
  1. [11]

    Nitay Calderon, Naveh Porat, Eyal Ben - David, Alexander Chapanin, Zorik Gekhman, Nadav Oved, Vitaly Shalumov, and Roi Reichart. 2024. https://aclanthology.org/2024.findings-emnlp.7 Measuring the robustness of NLP models to domain shifts . In Findings of the Association for Co...

  2. [12]

    Nitay Calderon and Roi Reichart. 2024. https://doi.org/10.48550/ARXIV.2407.19200 On behalf of the stakeholders: Trends in NLP model interpretability in the era of llms . CoRR, abs/2407.19200

  3. [13]

    Nitay Calderon, Roi Reichart, and Rotem Dror. 2025. https://doi.org/10.48550/ARXIV.2501.10970 The alternative annotator test for llm-as-a-judge: How to statistically justify replacing human annotators with llms . CoRR, abs/2501.10970

  4. [14]

    Vinod Kumar Chauhan, Jiandong Zhou, Ping Lu, Soheila Molaei, and David A. Clifton. 2024. https://doi.org/10.1007/S10462-024-10862-8 A brief review of hypernetworks in deep learning . Artif. Intell. Rev., 57(9):250

  5. [15]

    Shijie Chen, Yu Zhang, and Qiang Yang. 2024. https://doi.org/10.1145/3663363 Multi-task learning in natural language processing: An overview . ACM Comput. Surv. , 56(12):295:1--295:32

  6. [16]

    Jordan, Joseph E

    Wei - Lin Chiang, Lianmin Zheng, Ying Sheng, Anastasios Nikolas Angelopoulos, Tianle Li, Dacheng Li, Banghua Zhu, Hao Zhang, Michael I. Jordan, Joseph E. Gonzalez, and Ion Stoica. 2024. https://openreview.net/forum?id=3MW8GKNyzI Chatbot arena: An open platform for evaluating l...

  7. [17]

    Nadezhda Chirkova, Vassilina Nikoulina, Jean - Luc Meunier, and Alexandre B \' e rard. 2024. https://doi.org/10.48550/ARXIV.2407.01126 Investigating the potential of sparse mixtures-of-experts for multi-domain neural machine translation . CoRR, abs/2407.01126

  8. [18]

    Carlos Cinelli, Andrew Forney, and Judea Pearl. 2024. https://journals.sagepub.com/doi/abs/10.1177/00491241221099552?casa_token=heN_qPknVXUAAAAA:Z5ppXluGJeGfBZo_a_Kl1goJN_KGJaYDyx2tVkIeu0-p7Nxo0vNL3rQzzh-9G79ZKcsnKT1AuqlGcA&casa_token=yEPT-RwJTNsAAAAA:joP_VbMo73OhfTbrcgE_YQ3C_...

  9. [19]

    Ronan Collobert and Jason Weston. 2008. https://doi.org/10.1145/1390156.1390177 A unified architecture for natural language processing: deep neural networks with multitask learning . In Machine Learning, Proceedings of the Twenty-Fifth International Conference (ICML 2008), Hel...

  10. [20]

    Ganqu Cui, Lifan Yuan, Ning Ding, Guanming Yao, Wei Zhu, Yuan Ni, Guotong Xie, Zhiyuan Liu, and Maosong Sun. 2023. https://doi.org/10.48550/ARXIV.2310.01377 Ultrafeedback: Boosting language models with high-quality feedback . CoRR, abs/2310.01377

  11. [21]

    Josef Dai, Xuehai Pan, Ruiyang Sun, Jiaming Ji, Xinbo Xu, Mickel Liu, Yizhou Wang, and Yaodong Yang. 2024. https://openreview.net/forum?id=TyFrPOKYXw Safe RLHF: safe reinforcement learning from human feedback . In The Twelfth International Conference on Learning Representation...

  12. [22]

    Hal Daum \' e III . 2007. https://aclanthology.org/P07-1033/ Frustratingly easy domain adaptation . In ACL 2007, Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics, June 23-30, 2007, Prague, Czech Republic . The Association for Computationa...

  13. [23]

    DeepMind. 2024. https://blog.google/technology/ai/google-gemini-next-generation-model-february-2024/ Our next-generation model: Gemini 1.5

  14. [24]

    Darshan Deshpande, Selvan Sunitha Ravi, Sky CH - Wang, Bartosz Mielczarek, Anand Kannappan, and Rebecca Qian. 2024. https://doi.org/10.48550/ARXIV.2412.14140 GLIDER: grading LLM interactions and decisions using explainable ranking . CoRR, abs/2412.14140

  15. [25]

    Nicolai Dorka. 2024. https://doi.org/10.48550/ARXIV.2409.10164 Quantile regression for distributional reward models in RLHF . CoRR, abs/2409.10164

  16. [26]

    Finale Doshi-Velez and Been Kim. 2017. https://arxiv.org/pdf/1702.08608 Towards a rigorous science of interpretable machine learning . arXiv preprint arXiv:1702.08608

  17. [27]

    Kawin Ethayarajh, Yejin Choi, and Swabha Swayamdipta. 2022. https://proceedings.mlr.press/v162/ethayarajh22a.html Understanding dataset difficulty with V-usable information . In International Conference on Machine Learning, ICML 2022, 17-23 July 2022, Baltimore, Maryland, USA ...

  18. [28]

    Amir Feder, Nadav Oved, Uri Shalit, and Roi Reichart. 2021. https://doi.org/10.1162/COLI\_A\_00404 Causalm: Causal model explanation through counterfactual language models . Comput. Linguistics, 47(2):333--386

  19. [29]

    Lempitsky

    Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, Fran c ois Laviolette, Mario Marchand, and Victor S. Lempitsky. 2017. https://doi.org/10.1007/978-3-319-58347-1\_10 Domain-adversarial training of neural networks . In Domain Adaptation in Compute...

  20. [30]

    Yair Ori Gat, Nitay Calderon, Amir Feder, Alexander Chapanin, Amit Sharma, and Roi Reichart. 2024. https://openreview.net/forum?id=UMfcdRIotC Faithful explanations of black-box NLP models using llm-generated counterfactuals . In The Twelfth International Conference on Learning...

  21. [31]

    Arnav Gudibande, Eric Wallace, Charlie Snell, Xinyang Geng, Hao Liu, Pieter Abbeel, Sergey Levine, and Dawn Song. 2023. https://doi.org/10.48550/ARXIV.2305.15717 The false promise of imitating proprietary llms . CoRR, abs/2305.15717

  22. [32]

    Smith, and Luke Zettlemoyer

    Suchin Gururangan, Mike Lewis, Ari Holtzman, Noah A. Smith, and Luke Zettlemoyer. 2022. https://doi.org/10.18653/V1/2022.NAACL-MAIN.407 Demix layers: Disentangling domains for modular language modeling . In Proceedings of the 2022 Conference of the North American Chapter of th...

  23. [33]

    Tom Hosking, Phil Blunsom, and Max Bartolo. 2024. https://openreview.net/forum?id=7W3GLNImfS Human feedback is not gold standard . In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024 . OpenReview.net

  24. [34]

    Alon Jacovi and Yoav Goldberg. 2020. https://doi.org/10.18653/V1/2020.ACL-MAIN.386 Towards faithfully interpretable NLP systems: How should we define and evaluate faithfulness? In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020...

  25. [35]

    Ali Jalali, Pradeep Ravikumar, Sujay Sanghavi, and Chao Ruan. 2010. https://proceedings.neurips.cc/paper/2010/hash/00e26af6ac3b1c1c49d7c3d79c60d000-Abstract.html A dirty model for multi-task learning . In Advances in Neural Information Processing Systems 23: 24th Annual Confer...

  26. [36]

    Dongfu Jiang, Yishan Li, Ge Zhang, Wenhao Huang, Bill Yuchen Lin, and Wenhu Chen. 2024 a . https://openreview.net/forum?id=EE1CBKC0SZ Tigerscore: Towards building explainable metric for all text generation tasks . Trans. Mach. Learn. Res., 2024

  27. [37]

    Junqi Jiang, Tom Bewley, Saumitra Mishra, Freddy L \' e cu \' e , and Manuela Veloso. 2024 b . https://doi.org/10.48550/ARXIV.2411.16502 Interpreting language reward models via contrastive explanations . CoRR, abs/2411.16502

  28. [38]

    Timo Kaufmann, Paul Weng, Viktor Bengs, and Eyke H \" u llermeier. 2023. https://doi.org/10.48550/ARXIV.2312.14925 A survey of reinforcement learning from human feedback . CoRR, abs/2312.14925

  29. [39]

    Cai, James Wexler, Fernanda B

    Been Kim, Martin Wattenberg, Justin Gilmer, Carrie J. Cai, James Wexler, Fernanda B. Vi \' e gas, and Rory Sayres. 2018. http://proceedings.mlr.press/v80/kim18d.html Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (TCAV) . In P...

  30. [40]

    Seungone Kim, Jamin Shin, Yejin Choi, Joel Jang, Shayne Longpre, Hwaran Lee, Sangdoo Yun, Seongjin Shin, Sungdong Kim, James Thorne, and Minjoon Seo. 2024 a . https://openreview.net/forum?id=8euJaTveKw Prometheus: Inducing fine-grained evaluation capability in language models ...

  31. [41]

    Seungone Kim, Juyoung Suk, Shayne Longpre, Bill Yuchen Lin, Jamin Shin, Sean Welleck, Graham Neubig, Moontae Lee, Kyungjae Lee, and Minjoon Seo. 2024 b . https://aclanthology.org/2024.emnlp-main.248 Prometheus 2: An open source language model specialized in evaluating other la...

  32. [42]

    help me help the ai

    Sunnie S. Y. Kim, Elizabeth Anne Watkins, Olga Russakovsky, Ruth Fong, and Andr \' e s Monroy - Hern \' a ndez. 2023. https://doi.org/10.1145/3544548.3581001 "help me help the ai": Understanding how explainability can support human-ai interaction . In Proceedings of the 2023 C...

  33. [43]

    Pang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Emma Pierson, Been Kim, and Percy Liang. 2020. http://proceedings.mlr.press/v119/koh20a.html Concept bottleneck models . In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July ...

  34. [44]

    Smith, and Hannaneh Hajishirzi

    Nathan Lambert, Valentina Pyatkin, Jacob Morrison, LJ Miranda, Bill Yuchen Lin, Khyathi Raghavi Chandu, Nouha Dziri, Sachin Kumar, Tom Zick, Yejin Choi, Noah A. Smith, and Hannaneh Hajishirzi. 2024. https://doi.org/10.48550/ARXIV.2403.13787 Rewardbench: Evaluating reward model...

  35. [45]

    Harrison Lee, Samrat Phatale, Hassan Mansoor, Thomas Mesnard, Johan Ferret, Kellie Lu, Colton Bishop, Ethan Hall, Victor Carbune, Abhinav Rastogi, and Sushant Prakash. 2024. https://openreview.net/forum?id=uydQ2W41KO RLAIF vs. RLHF: scaling reinforcement learning from human fe...

  36. [46]

    Dawei Li, Bohan Jiang, Liangjie Huang, Alimohammad Beigi, Chengshuai Zhao, Zhen Tan, Amrita Bhattacharjee, Yuxuan Jiang, Canyu Chen, Tianhao Wu, Kai Shu, Lu Cheng, and Huan Liu. 2024 a . https://doi.org/10.48550/ARXIV.2411.16594 From generation to judgment: Opportunities and c...

  37. [47]

    Junlong Li, Fan Zhou, Shichao Sun, Yikai Zhang, Hai Zhao, and Pengfei Liu. 2024 b . https://doi.org/10.18653/V1/2024.ACL-LONG.99 Dissecting human and LLM preferences . In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pa...

  38. [48]

    Yitong Li, Timothy Baldwin, and Trevor Cohn. 2018. https://doi.org/10.18653/V1/N18-2076 What's in a domain? learning domain-robust text representations using adversarial training . In Proceedings of the 2018 Conference of the North American Chapter of the Association for Compu...

  39. [49]

    Zachary C. Lipton. 2018. https://doi.org/10.1145/3233231 The mythos of model interpretability . Commun. ACM , 61(10):36--43

  40. [50]

    Shir Lissak, Nitay Calderon, Geva Shenkman, Yaakov Ophir, Eyal Fruchter, Anat Brunstein Klomek, and Roi Reichart. 2024. https://doi.org/10.18653/V1/2024.NAACL-LONG.113 The colorful future of llms: Evaluating and improving llms as emotional supporters for queer youth . In Proce...

  41. [51]

    Chris Yuhao Liu, Liang Zeng, Jiacai Liu, Rui Yan, Jujie He, Chaojie Wang, Shuicheng Yan, Yang Liu, and Yahui Zhou. 2024. https://doi.org/10.48550/ARXIV.2410.18451 Skywork-reward: Bag of tricks for reward modeling in llms . CoRR, abs/2410.18451

  42. [52]

    Josh Magnus Ludan, Qing Lyu, Yue Yang, Liam Dugan, Mark Yatskar, and Chris Callison - Burch. 2023. https://doi.org/10.48550/ARXIV.2310.19660 Interpretable-by-design text classification with iteratively generated concept bottleneck . CoRR, abs/2310.19660

  43. [53]

    Siwen Luo, Hamish Ivison, Soyeon Caren Han, and Josiah Poon. 2024. https://doi.org/10.1145/3649450 Local interpretations for explainable natural language processing: A survey . ACM Comput. Surv. , 56(9):232:1--232:36

  44. [54]

    Qing Lyu, Marianna Apidianaki, and Chris Callison - Burch. 2022. https://doi.org/10.48550/ARXIV.2209.11326 Towards faithful model explanation in NLP: A survey . CoRR, abs/2209.11326

  45. [55]

    Andreas Madsen, Siva Reddy, and Sarath Chandar. 2023. https://doi.org/10.1145/3546577 Post-hoc interpretability for neural NLP: A survey . ACM Comput. Surv. , 55(8):155:1--155:42

  46. [56]

    Meta. 2024. https://ai.meta.com/blog/meta-llama-3/ Introducing llama 3.1: Our most capable models to date

  47. [57]

    Tim Miller. 2019. https://doi.org/10.1016/J.ARTINT.2018.07.007 Explanation in artificial intelligence: Insights from the social sciences . Artif. Intell., 267:1--38

  48. [58]

    Omer Nahum, Nitay Calderon, Orgad Keller, Idan Szpektor, and Roi Reichart. 2024. https://doi.org/10.48550/ARXIV.2410.18889 Are llms better than reported? detecting label errors and mitigating their effect on model performance . CoRR, abs/2410.18889

  49. [59]

    OpenAI. 2024. https://openai.com/index/hello-gpt-4o/ Hello gpt-4o

  50. [60]

    Maxime Peyrard, Sarvjeet Singh Ghotra, Martin Josifoski, Vidhan Agarwal, Barun Patra, Dean Carignan, Emre Kiciman, Saurabh Tiwary, and Robert West. 2022. https://doi.org/10.18653/V1/2022.EMNLP-MAIN.387 Invariant language modeling . In Proceedings of the 2022 Conference on Empi...

  51. [61]

    Eleonora Poeta, Gabriele Ciravegna, Eliana Pastor, Tania Cerquitelli, and Elena Baralis. 2023. https://doi.org/10.48550/ARXIV.2312.12936 Concept-based explainable artificial intelligence: A survey . CoRR, abs/2312.12936

  52. [62]

    Goldstein, Jake M

    Forough Poursabzi - Sangdeh, Daniel G. Goldstein, Jake M. Hofman, Jennifer Wortman Vaughan, and Hanna M. Wallach. 2021. https://doi.org/10.1145/3411764.3445315 Manipulating and measuring model interpretability . pages 237:1--237:52

  53. [63]

    QwenTeam. 2024. https://qwenlm.github.io/blog/qwen2.5/ Qwen2.5: A party of foundation models

  54. [64]

    Manning, Stefano Ermon, and Chelsea Finn

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning, Stefano Ermon, and Chelsea Finn. 2023. http://papers.nips.cc/paper\_files/paper/2023/hash/a85b405ed65c6477a4fe8302b5e06ce7-Abstract-Conference.html Direct preference optimization: Your language model is sec...

  55. [65]

    Guy Rotman and Roi Reichart. 2022. https://doi.org/10.1162/TACL\_A\_00515 Multi-task active learning for pre-trained transformer-based models . Trans. Assoc. Comput. Linguistics, 10:1209--1228

  56. [66]

    Pierrehumbert

    Paul R \" o ttger, Bertie Vidgen, Dirk Hovy, and Janet B. Pierrehumbert. 2022. https://doi.org/10.18653/V1/2022.NAACL-MAIN.13 Two contrasting data annotation paradigms for subjective NLP tasks . In Proceedings of the 2022 Conference of the North American Chapter of the Associa...

  57. [67]

    Sebastian Ruder. 2017. https://arxiv.org/abs/1706.05098 An overview of multi-task learning in deep neural networks . CoRR, abs/1706.05098

  58. [68]

    Bowman, Esin Durmus, Zac Hatfield - Dodds, Scott R

    Mrinank Sharma, Meg Tong, Tomasz Korbak, David Duvenaud, Amanda Askell, Samuel R. Bowman, Esin Durmus, Zac Hatfield - Dodds, Scott R. Johnston, Shauna Kravec, Timothy Maxwell, Sam McCandlish, Kamal Ndousse, Oliver Rausch, Nicholas Schiefer, Da Yan, Miranda Zhang, and Ethan Per...

  59. [69]

    Prasann Singhal, Tanya Goyal, Jiacheng Xu, and Greg Durrett. 2023. https://doi.org/10.48550/ARXIV.2310.03716 A long way to go: Investigating length correlations in RLHF . CoRR, abs/2310.03716

  60. [70]

    Tom AB Snijders and Roel Bosker. 2011. https://www.torrossa.com/en/resources/an/4912508 Multilevel analysis: An introduction to basic and advanced multilevel modeling

  61. [71]

    Oikarinen, Berk Ustun, and Tsui - Wei Weng

    Chung - En Sun, Tuomas P. Oikarinen, Berk Ustun, and Tsui - Wei Weng. 2024. https://doi.org/10.48550/ARXIV.2412.07992 Concept bottleneck large language models . CoRR, abs/2412.07992

  62. [72]

    Tomer Volk, Eyal Ben - David, Ohad Amosy, Gal Chechik, and Roi Reichart. 2023. https://doi.org/10.18653/V1/2023.FINDINGS-EMNLP.610 Example-based hypernetworks for multi-source adaptation to unseen domains . In Findings of the Association for Computational Linguistics: EMNLP 20...

  63. [73]

    Haoxiang Wang, Wei Xiong, Tengyang Xie, Han Zhao, and Tong Zhang. 2024 a . https://aclanthology.org/2024.findings-emnlp.620 Interpretable preferences via multi-objective reward modeling and mixture-of-experts . pages 10582--10592

  64. [74]

    Zhichao Wang, Bin Bi, Shiva Kumar Pentyala, Kiran Ramnath, Sougata Chaudhuri, Shubham Mehrotra, Zixu James Zhu, Xiang - Bo Mao, Sitaram Asur, and Na Claire Cheng. 2024 b . https://doi.org/10.48550/ARXIV.2407.16216 A comprehensive survey of LLM alignment techniques: Rlhf, rlaif...

  65. [75]

    Benjamin Warner, Antoine Chaffin, Benjamin Clavié, Orion Weller, Oskar Hallström, Said Taghadouini, Alexis Gallagher, Raja Biswas, Faisal Ladhak, Tom Aarsen, Nathan Cooper, Griffin Adams, Jeremy Howard, and Iacopo Poli. 2024. https://arxiv.org/abs/2412.13663 Smarter, better, f...

  66. [76]

    Chawla, and Xiangliang Zhang

    Jiayi Ye, Yanbo Wang, Yue Huang, Dongping Chen, Qihui Zhang, Nuno Moniz, Tian Gao, Werner Geyer, Chao Huang, Pin - Yu Chen, Nitesh V. Chawla, and Xiangliang Zhang. 2025. https://openreview.net/forum?id=3GTtZFiajM Justice or prejudice? quantifying biases in llm-as-a-judge . In ...

  67. [77]

    Manzil Zaheer, Guru Guruganesh, Kumar Avinava Dubey, Joshua Ainslie, Chris Alberti, Santiago Onta \ n \' o n, Philip Pham, Anirudh Ravula, Qifan Wang, Li Yang, and Amr Ahmed. 2020. https://proceedings.neurips.cc/paper/2020/hash/c8512d142a2d849725f31a9a7a361ab9-Abstract.html Bi...

  68. [78]

    Haiyan Zhao, Hanjie Chen, Fan Yang, Ninghao Liu, Huiqi Deng, Hengyi Cai, Shuaiqiang Wang, Dawei Yin, and Mengnan Du. 2024. https://doi.org/10.1145/3639372 Explainability for large language models: A survey . ACM Trans. Intell. Syst. Technol. , 15(2):20:1--20:38

  69. [79]

    Xing, Hao Zhang, Joseph E

    Lianmin Zheng, Wei - Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023. http://papers.nips.cc/paper\_files/paper/2023/hash/91f18a1287b398d378ef22505bf41832-Abstr...

  70. [80]

    Yftah Ziser and Roi Reichart. 2018. https://doi.org/10.18653/V1/N18-1112 Pivot based language modeling for improved neural domain adaptation . In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language T...

  71. [81]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  72. [82]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.