Inside the Unfair Judge: A Mechanistic Interpretability Account of LLM-as-Judge Bias

Huaxing Liu; Shuai Li; Sixian Li; Xiang Wang; Xiuying Chen; Zirui Song; Zixiang Xu

REVIEW 2 major objections 5 minor 211 references

Inside the Unfair Judge: A Mechanistic Interpretability Account of LLM-as-Judge Bias

T0 review · 2 major / 5 minor · reviewed 2026-07-14 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read LLM judge bias lives as a low-dimensional direction in the model’s hidden state, and steering along it both creates and cancels unfair scores.

desk verdict Solid three-part MI account of LLM-as-judge bias (geometry, bidirectional steering, cross-domain prediction) that earns referee time; main residual is the surface-cue/score-shift operationalization of bias, which the authors partially close with human TOST and out-of-substrate tests. read the letter →

arxiv 2607.11871 v1 pith:ULXVDONQ submitted 2026-07-13 cs.LG cs.AIcs.CL

Zixiang Xu , Sixian Li , Huaxing Liu , Xiang Wang , Shuai Li , Zirui Song , Xiuying Chen This is my paper · ORCID

classification cs.LGcs.AIcs.CL

keywords LLM-as-judgescoringbiasmechanisticinterpretabilityactivationsteeringsubspacerepresentationengineeringoutcomeprediction

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

When large language models are used as automatic judges of answers, their scores move with surface cues that have nothing to do with answer quality—prestige labels, peer consensus notes, length, tone, identity claims. Prior work mostly treated that as black-box input-output noise and tried to fix it with better prompts. This paper argues the same bias has a clear internal geometry. Clean judging inputs sit in a tight activation manifold; biased inputs are displaced along a low-dimensional, type-specific subspace that becomes sharper in deeper layers and is recovered by several independent estimators. Steering the hidden state along that subspace bidirectionally controls scores: adding the direction makes a clean answer look biased, subtracting it restores fair scoring on a biased answer, while matched-norm random directions barely move the score. The same direction features also let a simple linear predictor flag judge failures on three benchmarks never seen during training, beating text-only detectors. If the account holds, bias is no longer only a catalog of prompt tricks but a representation object that can be measured, steered, and predicted.

What carries the argument

The bias direction (or low-dimensional bias subspace) recovered from effective bias samples via directional-change and discriminative-boundary estimators; unit-normalized and used both for activation steering (add or subtract at mid-to-late layers) and for linear outcome prediction.

What would settle it

Show that matched-norm random directions or swapped bias-type directions produce score shifts as large as the recovered bias direction under the same validity and rank-preservation constraints, or that a linear projection onto those features fails to beat a strong text baseline on truly held-out domains.

Watch

Extended reading notes

Core claim

LLM-as-judge scoring bias admits a representation-level account: baseline activations form a tight manifold while biased inputs are displaced along a low-dimensional, type-specific subspace that sharpens with depth; that same subspace is an interventional handle that bidirectionally controls scores and supplies features that predict judge degradation on held-out domains.

Load-bearing premise

The paper treats score shifts caused by carefully constructed surface-cue edits as bias, and fits the direction only on the strong-shift subset of those edits; if many of those cues are rationally quality-relevant or the strong-shift tail is unrepresentative, the geometry may not underwrite the broader fairness claim.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

This is a clean empirical MI paper that actually delivers the three things it promises: a typed, depth-sharpening bias subspace in judge residual streams; bidirectional activation steering that beats matched-norm random and type-swap controls; and a linear projection that predicts score degradation on three held-out benchmarks (AUC ~0.82 vs text ~0.63).

What is new is not the toolkit—steering vectors and linear concept directions are established—but the systematic application to the evaluator role across seven bias types, seven judges, and nine benchmarks, with honest differentiation from SteerFair and LAGER. The behavioral asymmetry (large negative penalties, weak/heterogeneous positives) is replicated across models and prompt configs; geometry is recovered by independent estimator families with high within-family agreement; and the causal loop is closed with attack/defense plus random and swap controls. The operational predictor is the part that transfers, and they correctly anchor the claim to the simple linear projection rather than the overfit GBDT.

Soft spots are real but proportionate. Direction fitting is restricted to the effective-bias / biased-core case-control subset defined by score shift (δ_s=2). That is the load-bearing premise: if those surface cues are rationally quality-relevant, or if the threshold selects a non-representative tail, the fairness claim weakens. They partially address this with TOST human equivalence on bit-identical and prose-rewrite types, matched-budget text attacks, CV defense retaining ≥80% of in-sample W1 reduction, and prediction on benchmarks never used for vector estimation. White-box coverage is only three mid-scale open models; artifacts are promised but not yet hashed. Free parameters (thresholds, α*, layer) are standard for this class of work and do not invent circularity.

Math and citation pattern look solid; related work is fair. This is for people who care about judge reliability inside RLHF and eval pipelines, and for MI readers who want a non-generative application of representation engineering. It deserves a serious referee. I would engage with it and expect it to survive review with ordinary revisions (code release, sensitivity tables).

Referee Report

2 major / 5 minor

Summary. The paper argues that LLM-as-judge scoring bias is a representation-level phenomenon: baseline activations form a tight manifold while biased inputs are displaced along a low-dimensional, type-specific subspace that sharpens with depth. Across seven judges, seven bias types, and nine benchmarks, the authors (i) document strong behavioral asymmetry (negative surface cues penalize more than positive ones reward), (ii) recover the bias subspace with directional and discriminative estimators that agree within family and partially across architectures, (iii) show bidirectional activation steering along that subspace (attack on clean inputs, defense on biased ones) far above matched-norm random and bias-type-swap controls, and (iv) train a linear projection onto the same features that predicts score degradation on three entirely held-out benchmarks (AUC ~0.82 vs ~0.63 text baseline). The contribution is a unified geometric–causal–operational account rather than a new mitigation recipe.

Significance. If the three-part claim holds, the paper supplies a mechanistic account of a widely used evaluation primitive that currently sits inside RLHF and benchmark pipelines. The combination of multi-estimator geometry, bidirectional causal control with random and swap ablations, and a simple transferable linear predictor is stronger than typical input–output bias catalogs and is operationally useful for white-box judges. Strengths include the multi-judge behavioral replication, the TOST human equivalence checks on both prose-rewrite and bit-identical-body perturbations, the matched-budget text-attack comparison, 5-fold CV defense retaining ≥80% of in-sample W1 reduction, and cross-domain prediction sensitivity analysis. These make the result a solid contribution to mechanistic interpretability of evaluators and to fairness auditing of LLM judges.

major comments (2)

[§3.1–3.3, App. F, C/C.1] Sections 3.1–3.3 and Appendix F: direction estimation is restricted to the effective-bias / biased-core case-control subset (score shift ≥ δ_s = 2 and Mahalanobis 90th percentile). The paper correctly treats null-shift samples as null observations and evaluates causal/predictive claims outside the fitting substrate, but the fairness claim still depends on the premise that the surface cues are not rationally quality-relevant. Appendix C/C.1 TOST tests and the matched-budget text comparison (J.3) substantially mitigate this; the manuscript should state more explicitly in the main text (not only the appendix) that the geometry is conditioned on the established surface-cue operationalization of the LLM-as-judge literature, and report a brief sensitivity of recovered directions to δ_s ∈ {1,2,3} so readers can see how much the subspace depends on the strong-shift tail.
[§4.4, App. J.1–J.2] Section 4.4 and Appendices J.1–J.2: bidirectional steering is presented as interventional sufficiency, not unique natural pathway—this scoping is appropriate. The random-direction control is strong (order-of-magnitude gap); the bias-type-swap control correctly shows an intermediate effect (shared subspace + type-specific component). For the central causal claim, the paper should either (a) add a short path-patching / mediation sketch on one bias type and one layer, or (b) keep the current scoping but move the “we do not claim unique pathway” language into the main-text causal paragraph rather than only the discussion, so readers do not over-read the attack/defense results as full causal identification.

minor comments (5)

[Table 2, §4.2] Table 2 vs Table 4 / Figure 3: the positive-aggregate convention (five score-inflating types vs all seven) is explained in Appendix I.1 but should be flagged once in the main-text caption of Table 2 to avoid confusion with the full-pool +0.07 figure mentioned in §4.2.
[Fig. 1–2] Figure 1 MDS and Figure 2 Δh MDS: axis scales differ across panels; a shared color legend and a note that MDS is used for visualization only (not for the estimators) would help non-MI readers.
[App. H, I.3] Appendix H limitations: white-box coverage is limited to three mid-scale judges; the cross-architecture cosine band [0.47, 0.62] is useful—consider promoting one sentence of that partial-transfer result into the main-text geometry section.
[App. G] Reproducibility: seed, split, and ~1400 A100-hour budget are stated; ensure the promised code release includes the exact α-search hyperparameters (Algorithm 1) and the nested question-ID split files so the cross-domain AUC can be regenerated.
[§3.2] Minor notation: ℳ_base is introduced as a manifold but used as an empirical cluster; a one-line clarification that it is the empirical support of H_base^(l) would avoid geometric overclaim.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: geometry, bidirectional steering, and cross-domain prediction are independently validated outside the case-control fitting substrate used to estimate directions.

full rationale

The paper's three claims (activation geometry of bias, bidirectional causal control via steering, and operational prediction of score degradation) are empirical findings from controlled experiments, not derivations that reduce to their inputs by construction. Bias directions are estimated on an effective-bias / biased-core case-control subset defined by observed score shifts (δ_s=2), which is an acknowledged selection of the phenomenon rather than a circular definition of the target quantity; causal attack/defense, random-direction and bias-type-swap controls, 5-fold CV defense, and the linear-projection outcome predictor are all evaluated on inputs outside that substrate (held-out folds, three entirely unseen benchmarks). No equation equates a 'prediction' to a fitted constant; no uniqueness theorem or ansatz is imported via self-citation as a load-bearing premise; self-citations (e.g., to SocialMaze or related representation work by overlapping authors) supply benchmarks or background and are not required for the central results. The operationalization of bias via surface cues is a construct-validity premise (already partially closed by human TOST equivalence tests), not circularity. The work is self-contained against its external benchmarks and controls.

Assumptions & free parameters 5 free parameters · 4 assumptions · 2 invented entities

The central claim rests on the linear-representation hypothesis, a surface-cue operationalization of bias taken from prior judge-bias literature, and several numerical thresholds that define the fitting substrate and feasibility set. The 'baseline manifold' and 'bias direction/subspace' are empirical constructs recovered from data rather than postulated particles; their independent handle is the bidirectional steering and cross-domain prediction. Free parameters are mostly thresholds and search bounds that are stated and partially ablated.

free parameters (5)

effective-bias score-shift threshold δ_s = 2
Paired instances enter the direction-fitting substrate only if |Δs| ≥ δ_s integer points in the expected direction; set to 2 throughout.
outcome-prediction degradation threshold δ_o = 1
Positive class for the operational predictor is score drop ≥ δ_o; set more permissively than δ_s.
biased-core Mahalanobis percentile = 90th percentile
𝒟_far retains only the subset of effective negative samples above the 90th percentile of Mahalanobis distance from the baseline cluster.
steering strength α* and feasibility floors = per (layer, vector, bias type); V floor 0.93
α* maximizes Wasserstein shift subject to validity V(α) ≥ 0.93 and Spearman ρ_S(α) ≥ ρ_text_S; localized by two-stage search (~100 forward passes).
intervention layer l = dev-set max-separability layer
Chosen as the layer maximizing biased-vs-baseline separability on the development set (typically mid-to-late, 15–31).

assumptions (4)

domain assumption Linear representation hypothesis: high-level behaviors (here, typed scoring bias) concentrate along low-dimensional directions or subspaces in residual-stream activations.
Invoked throughout Sections 3.2–3.4 and 4.3–4.4; standard in the representation-engineering literature the paper builds on.
domain assumption Surface-cue operationalization of scoring bias: two inputs that differ only in semantics-irrelevant framing (prestige tags, consensus notes, etc.) and receive systematically different scores constitute measurable bias.
Section 3.1 and Appendix A.1; adopted from the existing LLM-as-judge bias literature to keep results comparable.
ad hoc to paper Case-control framing: null-shift samples are null observations rather than negative examples of the bias phenomenon, so direction estimation may restrict to effective-bias samples.
Section 3.3; authors defend it as intrinsic to the phenomenon but it shapes the recovered geometry.
domain assumption Human raters' quality assessments on template markers and prose rewrites are the ground truth for whether score drops reflect surface bias rather than genuine quality change.
Appendix C / C.1 TOST equivalence tests; used to support construct validity of the bias labels.

invented entities (2)

baseline activation manifold ℳ_base independent evidence
purpose: Empirical region occupied by unbiased judging activations; biased inputs are defined as geometrically distant from it.
Recovered from ℋ_base via MDS/PCA; not a new physical entity but a named geometric construct central to the account.
type-specific bias direction / low-dimensional bias subspace independent evidence
purpose: The interventional handle along which biased scoring concentrates; estimated by directional and discriminative families.
Core object of the paper; independent evidence comes from multi-estimator agreement, bidirectional steering, random/swap controls, and cross-domain prediction.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Inside the Unfair Judge: A Mechanistic Interpretability Account of LLM-as-Judge Bias." pith.science (2026). https://pith.science/paper/ULXVDONQ

@misc{pith2026260711871,
  author       = {Pith},
  title        = {Pith review of: Inside the Unfair Judge: A Mechanistic Interpretability Account of LLM-as-Judge Bias},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ULXVDONQ}},
  note         = {Machine review of arXiv:2607.11871}
}

read the original abstract

Existing studies of LLM-as-judge scoring bias work predominantly at the input-output level: they perturb inputs, measure score deltas, and propose prompt-level mitigations. We argue that the same biases admit a representation-level account in the judge's hidden state, complementary to the input-output view and operationally useful in ways it does not afford. We report three findings, across seven judges, seven bias types, and nine benchmarks. Geometry: baseline judging inputs occupy a tight activation manifold while biased inputs are displaced along a low-dimensional, type-specific subspace that sharpens with depth and is recovered consistently by three families of estimators. Causal control: steering hidden states along this subspace drives scoring in both directions, forward shifts reproducing biased scoring on clean inputs and reverse shifts restoring baseline scoring on biased ones, while matched-norm random directions produce shifts an order of magnitude smaller. Operational: a simple linear projection onto the same bias-direction features anticipates judge failures on three entirely unseen benchmarks, substantially outperforming text-based alternatives. Reading bias as activation geometry, rather than as input-output noise, unifies geometric structure, causal control, and operational prediction within a single framework. The project page is available at https://xzx34.github.io/unfair-judge/

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

211 extracted references · 3 canonical work pages

[1]

2025 , eprint=

Qwen3 Technical Report , author=. 2025 , eprint=

2025
[2]

2025 , url=

Qwen2.5 Technical Report , author=. 2025 , url=

2025
[4]

2022 , eprint=

Lin, Stephanie and Hilton, Jacob and Evans, Owain , booktitle=. 2022 , eprint=

2022
[5]

2019 , eprint=

Talmor, Alon and Herzig, Jonathan and Lourie, Nicholas and Berant, Jonathan , booktitle=. 2019 , eprint=

2019
[6]

and Lu, Xinghua , booktitle=

Jin, Qiao and Dhingra, Bhuwan and Liu, Zhengping and Cohen, William W. and Lu, Xinghua , booktitle=. 2019 , eprint=

2019
[7]

, booktitle=

Rein, David and Hou, Betty Li and Stickland, Asa Cooper and Petty, Jackson and Pang, Richard Yuanzhe and Dirani, Julien and Michael, Julian and Bowman, Samuel R. , booktitle=. 2024 , eprint=

2024
[8]

2018 , eprint =

Clark, Peter and Cowhey, Isaac and Etzioni, Oren and Khot, Tushar and Sabharwal, Ashish and Schoenick, Carissa and Tafjord, Oyvind , title =. 2018 , eprint =

2018
[10]

2024 , howpublished =

GPT-4o mini: Advancing Cost-Efficient Intelligence , author =. 2024 , howpublished =

2024

Show all 211 references

[11]

2024 , eprint=

GPT-4 Technical Report , author=. 2024 , eprint=

2024
[12]

2024 , eprint=

GPT-4o System Card , author=. 2024 , eprint=

2024
[13]

2025 , eprint=

DeepSeek-V3 Technical Report , author=. 2025 , eprint=

2025
[14]

Gemma 3 , url=

Gemma Team , year=. Gemma 3 , url=
[15]

2024 , month =

Phi-4 Technical Report , author =. 2024 , month =

2024
[16]

2025 , howpublished =

GPT-4.1 , author =. 2025 , howpublished =

2025
[17]

, booktitle=

Parrish, Alicia and Chen, Angelica and Nangia, Nikita and Padmakumar, Vishakh and Phang, Jason and Thompson, Jana and Htut, Phu Mon and Bowman, Samuel R. , booktitle=. 2022 , eprint=

2022
[18]

Proceedings of the International Conference on Learning Representations (ICLR) , year=

Measuring Massive Multitask Language Understanding , author=. Proceedings of the International Conference on Learning Representations (ICLR) , year=
[19]

and Zhang, Hao and Gonzalez, Joseph E

Zheng, Lianmin and Chiang, Wei-Lin and Sheng, Ying and Zhuang, Siyuan and Wu, Zhanghao and Zhuang, Yonghao and Lin, Zi and Li, Zhuohan and Li, Dacheng and Xing, Eric P. and Zhang, Hao and Gonzalez, Joseph E. and Stoica, Ion , booktitle=. Judging. 2023 , eprint=

2023
[20]

Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL) , year=

Large Language Models are not Fair Evaluators , author=. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL) , year=. 2305.17926 , archivePrefix=

arXiv
[21]

Findings of the Association for Computational Linguistics: ACL 2024 , year=

Benchmarking Cognitive Biases in Large Language Models as Evaluators , author=. Findings of the Association for Computational Linguistics: ACL 2024 , year=. 2309.17012 , archivePrefix=

2024 arXiv
[22]

International Conference on Learning Representations (ICLR) , year=

Evaluating Large Language Models at Evaluating Instruction Following , author=. International Conference on Learning Representations (ICLR) , year=. 2310.07641 , archivePrefix=

arXiv
[23]

and Zhang, Xiangliang , booktitle=

Ye, Jiayi and Wang, Yanbo and Huang, Yue and Chen, Dongping and Zhang, Qihui and Moniz, Nuno and Gao, Tian and Geyer, Werner and Huang, Chao and Chen, Pin-Yu and Chawla, Nitesh V. and Zhang, Xiangliang , booktitle=. Justice or Prejudice? Quantifying Biases in. 2025 , eprint=

2025
[24]

Humans or

Chen, Guiming Hardy and Chen, Shunian and Liu, Ziche and Jiang, Feng and Wang, Benyou , booktitle=. Humans or. 2024 , eprint=

2024
[25]

Split and Merge: Aligning Position Biases in

Li, Zongjie and Wang, Chaozheng and Ma, Pingchuan and Wu, Daoyuan and Wang, Shuai and Gao, Cuiyun and Liu, Yang , booktitle=. Split and Merge: Aligning Position Biases in. 2024 , eprint=

2024
[26]

and Feng, Shi , booktitle=

Panickssery, Arjun and Bowman, Samuel R. and Feng, Shi , booktitle=. 2024 , eprint=

2024
[27]

Judging the Judges: Evaluating Alignment and Vulnerabilities in

Thakur, Aman Singh and Choudhary, Kartik and Ramayapally, Venkat Srinik and Vaidyanathan, Sankaran and Hupkes, Dieuwke , booktitle=. Judging the Judges: Evaluating Alignment and Vulnerabilities in. 2025 , eprint=

2025
[28]

A Survey on

Gu, Jiawei and Jiang, Xuhui and Shi, Zhichao and Tan, Hexiang and Zhai, Xuehao and Xu, Chengjin and Li, Wei and Shen, Yinghan and Ma, Shengjie and Liu, Honghao and Wang, Yuanzhuo and Guo, Jian , year=. A Survey on. 2411.15594 , archivePrefix=

arXiv
[29]

2412.05579 , archivePrefix=

Li, Haitao and Dong, Qian and Chen, Junjie and Su, Huixue and Zhou, Yujia and Ai, Qingyao and Ye, Ziyi and Liu, Yiqun , year=. 2412.05579 , archivePrefix=

arXiv
[30]

2023 , eprint=

Verbosity Bias in Preference Labeling by Large Language Models , author=. 2023 , eprint=

2023
[31]

Length-Controlled

Dubois, Yann and Galambosi, Bal. Length-Controlled. Conference on Language Modeling (COLM) , year=. 2404.04475 , archivePrefix=

arXiv
[32]

Self-Preference Bias in

Wataoka, Koki and Takahashi, Tsubasa and Ri, Ryokan , year=. Self-Preference Bias in. 2410.21819 , archivePrefix=

arXiv
[33]

2024 , eprint=

Large Language Models are Inconsistent and Biased Evaluators , author=. 2024 , eprint=

2024
[34]

Judging the Judges: A Systematic Study of Position Bias in

Shi, Lin and Ma, Chiyu and Liang, Weicheng and Ma, Xinyi and Shi, Weijian and Vosoughi, Soroush , booktitle=. Judging the Judges: A Systematic Study of Position Bias in. 2025 , eprint=

2025
[35]

2023 , eprint=

Liu, Yang and Iter, Dan and Xu, Yichong and Wang, Shuohang and Xu, Ruochen and Zhu, Chenguang , booktitle=. 2023 , eprint=

2023
[36]

2024 , eprint=

Wang, Yidong and Yu, Zhuohao and Zeng, Zhengran and Yang, Linyi and Wang, Cunxiang and Chen, Hao and Jiang, Chaoya and Xie, Rui and Wang, Jindong and Xie, Xing and Ye, Wei and Zhang, Shikun and Zhang, Yue , booktitle=. 2024 , eprint=

2024
[37]

International Conference on Learning Representations (ICLR) , year=

Prometheus: Inducing Fine-grained Evaluation Capability in Language Models , author=. International Conference on Learning Representations (ICLR) , year=. 2310.08491 , archivePrefix=

arXiv
[38]

2025 , eprint=

Zhu, Lianghui and Wang, Xinggang and Wang, Xinlong , booktitle=. 2025 , eprint=

2025
[39]

and Cuadron, Alejandro and Wang, Chenguang and Popa, Raluca Ada and Stoica, Ion , booktitle=

Tan, Sijun and Zhuang, Siyuan and Montgomery, Kyle and Tang, William Y. and Cuadron, Alejandro and Wang, Chenguang and Popa, Raluca Ada and Stoica, Ion , booktitle=. 2025 , eprint=

2025
[40]

and Hajishirzi, Hannaneh , year=

Lambert, Nathan and Pyatkin, Valentina and Morrison, Jacob and Miranda, LJ and Lin, Bill Yuchen and Chandu, Khyathi and Dziri, Nouha and Kumar, Sachin and Zick, Tom and Choi, Yejin and Smith, Noah A. and Hajishirzi, Hannaneh , year=. 2403.13787 , archivePrefix=

arXiv
[41]

Preference Leakage: A Contamination Problem in

Li, Dawei and Sun, Renliang and Huang, Yue and Zhong, Ming and Jiang, Bohan and Han, Jiawei and Zhang, Xiangliang and Wang, Wei and Liu, Huan , year=. Preference Leakage: A Contamination Problem in. 2502.01534 , archivePrefix=
[42]

Replacing Judges with Juries: Evaluating

Verga, Pat and Hofstatter, Sebastian and Althammer, Sophia and Su, Yixuan and Piktus, Aleksandra and Arkhangorodsky, Arkady and Xu, Minjie and White, Naomi and Lewis, Patrick , year=. Replacing Judges with Juries: Evaluating. 2404.18796 , archivePrefix=

arXiv
[43]

Beyond the Surface: Enhancing

Lai, Peng and Zheng, Jianjie and Cheng, Sijie and Chen, Yun and Li, Peng and Liu, Yang and Chen, Guanhua , year=. Beyond the Surface: Enhancing. 2508.03550 , archivePrefix=
[44]

2023 , eprint=

Activation Addition: Steering Language Models Without Optimization , author=. 2023 , eprint=

2023
[45]

Advances in Neural Information Processing Systems (NeurIPS) , year=

Inference-Time Intervention: Eliciting Truthful Answers from a Language Model , author=. Advances in Neural Information Processing Systems (NeurIPS) , year=. 2306.03341 , archivePrefix=

arXiv
[46]

and Wang, Zifan and Mallen, Alex and Basart, Steven and Koyejo, Sanmi and Song, Dawn and Fredrikson, Matt and Kolter, J

Zou, Andy and Phan, Long and Chen, Sarah and Campbell, James and Guo, Phillip and Ren, Richard and Pan, Alexander and Yin, Xuwang and Mazeika, Mantas and Dombrowski, Ann-Kathrin and Goel, Shashwat and Li, Nathaniel and Byun, Michael J. and Wang, Zifan and Mallen, Alex and Basa...

arXiv
[47]

Steering

Panickssery, Nina and Gabrieli, Nick and Schulz, Julian and Tong, Meg and Hubinger, Evan and Turner, Alexander Matt , booktitle=. Steering. 2024 , eprint=

2024
[48]

Findings of the Association for Computational Linguistics: ACL 2022 , year=

Extracting Latent Steering Vectors from Pretrained Language Models , author=. Findings of the Association for Computational Linguistics: ACL 2022 , year=. 2205.05124 , archivePrefix=

2022 arXiv
[49]

International Conference on Learning Representations (ICLR) , year=

Discovering Latent Knowledge in Language Models Without Supervision , author=. International Conference on Learning Representations (ICLR) , year=. 2212.03827 , archivePrefix=

arXiv
[50]

Conference on Language Modeling (COLM) , year=

The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets , author=. Conference on Language Modeling (COLM) , year=. 2310.06824 , archivePrefix=

arXiv
[51]

International Conference on Machine Learning (ICML) , year=

The Linear Representation Hypothesis and the Geometry of Large Language Models , author=. International Conference on Machine Learning (ICML) , year=. 2311.03658 , archivePrefix=

arXiv
[52]

Advances in Neural Information Processing Systems (NeurIPS) , year=

Refusal in Language Models Is Mediated by a Single Direction , author=. Advances in Neural Information Processing Systems (NeurIPS) , year=. 2406.11717 , archivePrefix=

arXiv
[53]

Locating and Editing Factual Associations in

Meng, Kevin and Bau, David and Andonian, Alex and Belinkov, Yonatan , booktitle=. Locating and Editing Factual Associations in. 2022 , eprint=

2022
[54]

International Conference on Learning Representations (ICLR) , year=

Progress Measures for Grokking via Mechanistic Interpretability , author=. International Conference on Learning Representations (ICLR) , year=. 2301.05217 , archivePrefix=

arXiv
[55]

Proceedings of the 6th BlackboxNLP Workshop , year=

Emergent Linear Representations in World Models of Self-Supervised Sequence Models , author=. Proceedings of the 6th BlackboxNLP Workshop , year=. 2309.00941 , archivePrefix=

arXiv
[56]

The Internal State of an

Azaria, Amos and Mitchell, Tom , booktitle=. The Internal State of an. 2023 , eprint=

2023
[57]

Advances in Neural Information Processing Systems (NeurIPS) , year=

Inspecting and Editing Knowledge Representations in Language Models , author=. Advances in Neural Information Processing Systems (NeurIPS) , year=. 2304.00740 , archivePrefix=

arXiv
[58]

and McDougall, Callum and MacDiarmid, Monte and Tamkin, Alex and Durmus, Esin and Hume, Tristan and Mosconi, Francesco and Freeman, C

Templeton, Adly and Conerly, Tom and Marcus, Jonathan and Lindsey, Jack and Bricken, Trenton and Chen, Brian and Pearce, Adam and Citro, Craig and Ameisen, Emmanuel and Jones, Andy and Cunningham, Hoagy and Turner, Nicholas L. and McDougall, Callum and MacDiarmid, Monte and Ta...
[59]

2023 , howpublished=

Towards Monosemanticity: Decomposing Language Models With Dictionary Learning , author=. 2023 , howpublished=

2023
[60]

and Espinosa-Anke, Luis , year=

Siddique, Zara and Turner, Liam D. and Espinosa-Anke, Luis , year=. Shifting Perspectives: Steering Vector Ensembles for Robust Bias Mitigation in. 2503.05371 , archivePrefix=
[61]

2025 , eprint=

Representation Engineering for Large-Language Models: Survey and Research Challenges , author=. 2025 , eprint=

2025
[62]

Activation Steering for Bias Mitigation: An Interpretable Approach to Safer

Dubey, Shivam , year=. Activation Steering for Bias Mitigation: An Interpretable Approach to Safer. 2508.09019 , archivePrefix=

arXiv
[63]

International Conference on Machine Learning (ICML) , year=

Discovering Bias in Latent Space: An Unsupervised Debiasing Approach , author=. International Conference on Machine Learning (ICML) , year=. 2406.03631 , archivePrefix=

arXiv
[64]

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL) , year=

Null It Out: Guarding Protected Attributes by Iterative Nullspace Projection , author=. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL) , year=. 2004.07667 , archivePrefix=

2004 arXiv
[65]

International Conference on Machine Learning (ICML) , year=

Linear Adversarial Concept Erasure , author=. International Conference on Machine Learning (ICML) , year=. 2201.12091 , archivePrefix=

arXiv
[66]

2023 , eprint=

Belrose, Nora and Schneider-Joseph, David and Ravfogel, Shauli and Cotterell, Ryan and Raff, Edward and Biderman, Stella , booktitle=. 2023 , eprint=

2023
[67]

International Conference on Learning Representations (ICLR) , year=

Eliminating Position Bias of Language Models: A Mechanistic Approach , author=. International Conference on Learning Representations (ICLR) , year=. 2407.01100 , archivePrefix=

arXiv
[68]

International Conference on Learning Representations (ICLR) , year=

Programming Refusal with Conditional Activation Steering , author=. International Conference on Learning Representations (ICLR) , year=. 2409.05907 , archivePrefix=

arXiv
[69]

2025 , eprint=

Li, Haitao and Chen, Junjie and Ai, Qingyao and Chu, Zhumin and Zhou, Yujia and Dong, Qian and Liu, Yiqun , booktitle=. 2025 , eprint=

2025
[70]

Adaptive Distraction: Probing

Wang, Yanbo and Xu, Zixiang and Huang, Yue and Gao, Chujie and Wu, Siyuan and Ye, Jiayi and Chen, Pin-Yu and Chen, Xiuying and Zhang, Xiangliang , booktitle =. Adaptive Distraction: Probing
[71]

Wang, Yanbo and Xu, Zixiang and Huang, Yue and Wang, Xiangqi and Song, Zirui and Gao, Lang and Wang, Chenxi and Tang, Robert and Zhao, Yue and Cohan, Arman and Zhang, Xiangliang and Chen, Xiuying , booktitle =
[74]

Xu, Zixiang and Wang, Yanbo and Huang, Yue and Wang, Chenxi and Gao, Lang and Song, Zirui and Chen, Zhaorun and Zhang, Xiangliang and Chen, Xiuying , year=
[78]

Proceedings of the 43rd International Conference on Machine Learning , volume=

Large-Scale Terminal Agentic Trajectory Generation from Dockerized Environments , author=. Proceedings of the 43rd International Conference on Machine Learning , volume=. 2026 , eprint=

2026
[81]

Advances in Neural Information Processing Systems , pages=

Hu, Yikuan and Zhu, Jifeng and Tang, Lanrui and Huang, Chen , editor=. Advances in Neural Information Processing Systems , pages=. 2025 , url=

2025
[85]

Preprints , doi=

The Evaluation Bottleneck of Vision-Language-Action Models: An Evaluation-Centric Survey , author=. Preprints , doi=
[86]

Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , year =

Shaping the Safety Boundaries: Understanding and Defending Against Jailbreaks in Large Language Models , author =. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , year =
[87]

2025 , eprint=

Evaluate Bias without Manual Test Sets: A Concept Representation Perspective for LLMs , author=. 2025 , eprint=

2025
[88]

Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , year =

When Personalization Tricks Detectors: The Feature-Inversion Trap in Machine-Generated Text Detection , author =. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , year =
[89]

2026 , eprint=

The Cylindrical Representation Hypothesis for Language Model Steering , author=. 2026 , eprint=

2026
[90]

Decoding Echo Chambers:

Wang, Chenxi and Liu, Zongfang and Yang, Dequan and Chen, Xiuying , booktitle =. Decoding Echo Chambers:. 2025 , address =

2025
[91]

Word Form Matters:

Wang, Chenxi and Gu, Tianle and Wei, Zhongyu and Gao, Lang and Song, Zirui and Chen, Xiuying , booktitle =. Word Form Matters:. 2025 , address =

2025
[92]

2025 , eprint=

Under the Shadow of Babel: How Language Shapes Reasoning in LLMs , author=. 2025 , eprint=

2025
[93]

Findings of the Association for Computational Linguistics: ACL 2025 , year =

A Cognitive Writing Perspective for Constrained Long-Form Text Generation , author =. Findings of the Association for Computational Linguistics: ACL 2025 , year =

2025
[94]

2026 , eprint=

A Fano-Style Accuracy Upper Bound for LLM Single-Pass Reasoning in Multi-Hop QA , author=. 2026 , eprint=

2026
[95]

2026 , eprint=

ProbeLLM: Automating Principled Diagnosis of LLM Failures , author=. 2026 , eprint=

2026
[96]

Guardian-as-an-Advisor: Advancing Next-Generation Guardian Models for Trustworthy

Huang, Yue and Zhuang, Haomin and Ye, Jiayi and Bao, Han and Wang, Yanbo and Hua, Hang and Wu, Siyuan and Chen, Pin-Yu and Zhang, Xiangliang , booktitle =. Guardian-as-an-Advisor: Advancing Next-Generation Guardian Models for Trustworthy. 2026 , address =

2026
[97]

2026 , eprint=

NARRA-Gym for Evaluating Interactive Narrative Agents , author=. 2026 , eprint=

2026
[98]

2025 , eprint=

Building a Foundational Guardrail for General Agentic Systems via Synthetic Data , author=. 2025 , eprint=

2025
[99]

2026 , eprint=

Emergent Social Intelligence Risks in Generative Multi-Agent Systems , author=. 2026 , eprint=

2026
[100]

Proceedings of the AAAI Conference on Artificial Intelligence , volume=

SPA: Achieving Consensus in LLM Alignment via Self-Priority Optimization , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=. 2026 , doi=

2026
[101]

2024 , eprint=

Graph Neural Network Explanations are Fragile , author=. 2024 , eprint=

2024
[102]

2025 , eprint=

Provably Robust Explainable Graph Neural Networks against Graph Perturbation Attacks , author=. 2025 , eprint=

2025
[103]

2026 , eprint=

``Someone Hid It'': Query-Agnostic Black-Box Attacks on LLM-Based Retrieval , author=. 2026 , eprint=

2026
[104]

2025 , eprint=

DrugAgent: Automating AI-aided Drug Discovery Programming through LLM Multi-Agent Collaboration , author=. 2025 , eprint=

2025
[106]

2025 , url=

Shawn Li and Huixian Gong and Hao Dong and Tiankai Yang and Zhengzhong Tu and Yue Zhao , booktitle=. 2025 , url=

2025
[107]

2026 , eprint=

Defenses Against Prompt Attacks Learn Surface Heuristics , author=. 2026 , eprint=

2026
[108]

2025 , url=

Tiankai Yang and Yi Nian and Shawn Li and Ruiyao Xu and Yuangang Li and Jiaqi Li and Zhuo Xiao and Xiyang Hu and Ryan Rossi and Kaize Ding and Xia Hu and Yue Zhao , booktitle=. 2025 , url=

2025
[109]

2025 , url=

Tiankai Yang and Junjun Liu and Michael Siu and Jiahang Wang and Zhuangzhuang Qian and Chanjuan Song and Cheng Cheng and Xiyang Hu and Yue Zhao , booktitle=. 2025 , url=

2025
[114]

2025 , eprint=

Graph Synthetic Out-of-Distribution Exposure with Large Language Models , author=. 2025 , eprint=

2025
[115]

Hewett, Mojan Javaheripi, Piero Kauffmann, James R

Marah Abdin, Jyoti Aneja, Harkirat Behl, Sébastien Bubeck, Ronen Eldan, Suriya Gunasekar, Michael Harrison, Russell J. Hewett, Mojan Javaheripi, Piero Kauffmann, James R. Lee, Yin Tat Lee, Yuanzhi Li, Weishung Liu, Caio C. T. Mendes, Anh Nguyen, Eric Price, Gustavo de Rosa, Ol...

2024 arXiv
[116]

Discovering bias in latent space: An unsupervised debiasing approach

Dyah Adila, Shuai Zhang, Boran Han, and Yuyang Wang. Discovering bias in latent space: An unsupervised debiasing approach. In International Conference on Machine Learning (ICML), 2024

2024
[117]

OrigamiBench : An interactive environment to synthesize flat-foldable origamis, 2026

Naaisha Agarwal, Yihan Wu, Yichang Jian, Yikuan Hu, Nishad Mansoor, Mohan Li, Yifei Peng, Wang-Zhou Dai, Yao-Xiang Ding, and Emanuele Sansone. OrigamiBench : An interactive environment to synthesize flat-foldable origamis, 2026. URL https://arxiv.org/abs/2603.13856

2026
[118]

Refusal in language models is mediated by a single direction

Andy Arditi, Oscar Obeso, Aaquib Syed, Daniel Paleka, Nina Panickssery, Wes Gurnee, and Neel Nanda. Refusal in language models is mediated by a single direction. In Advances in Neural Information Processing Systems (NeurIPS), 2024

2024
[119]

The internal state of an LLM knows when it's lying

Amos Azaria and Tom Mitchell. The internal state of an LLM knows when it's lying. In Findings of the Association for Computational Linguistics: EMNLP 2023, 2023

2023
[120]

Representation engineering for large-language models: Survey and research challenges, 2025

Lukasz Bartoszcze, Sarthak Munshi, Bryan Sukidi, Jennifer Yen, Zejia Yang, David Williams-King, Linh Gupta, Kosi Zhu, and Ruhi Jaafar. Representation engineering for large-language models: Survey and research challenges, 2025

2025
[121]

LEACE : Perfect linear concept erasure in closed form

Nora Belrose, David Schneider-Joseph, Shauli Ravfogel, Ryan Cotterell, Edward Raff, and Stella Biderman. LEACE : Perfect linear concept erasure in closed form. In Advances in Neural Information Processing Systems (NeurIPS), 2023

2023
[122]

Burke, Tristan Hume, Shan Carter, Tom Henighan, and Christopher Olah

Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nicholas Turner, Cem Anil, Carson Denison, Amanda Askell, Robert Lasenby, Yifan Wu, Shauna Kravec, Nicholas Schiefer, Tim Maxwell, Nicholas Joseph, Zac Hatfield-Dodds, Alex Tamkin, Karina Ngu...

2023
[123]

Discovering latent knowledge in language models without supervision

Collin Burns, Haotian Ye, Dan Klein, and Jacob Steinhardt. Discovering latent knowledge in language models without supervision. In International Conference on Learning Representations (ICLR), 2023

2023
[124]

Humans or LLMs as the judge? a study on judgement bias

Guiming Hardy Chen, Shunian Chen, Ziche Liu, Feng Jiang, and Benyou Wang. Humans or LLMs as the judge? a study on judgement bias. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2024

2024
[125]

Think you have solved question answering? try ARC , the AI2 reasoning challenge, 2018

Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try ARC , the AI2 reasoning challenge, 2018

2018
[126]

Training verifiers to solve math word problems

Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021

2021 arXiv
[127]

Zhang, Han Bao, Hanwei Xu, Haocheng Wang, Haowei Zhang, Honghui Ding, Huajian Xin, Huazuo Gao, Hui Li, Hui Qu, J

DeepSeek-AI, Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Daya Guo, Dejian Yang, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei L...

2025 arXiv
[128]

Activation steering for bias mitigation: An interpretable approach to safer LLMs , 2025

Shivam Dubey. Activation steering for bias mitigation: An interpretable approach to safer LLMs , 2025

2025
[129]

Hashimoto

Yann Dubois, Bal \'a zs Galambosi, Percy Liang, and Tatsunori B. Hashimoto. Length-controlled AlpacaEval : A simple way to debias automatic evaluators. In Conference on Language Modeling (COLM), 2024

2024
[130]

Shaping the safety boundaries: Understanding and defending against jailbreaks in large language models

Lang Gao, Jiahui Geng, Xiangliang Zhang, Preslav Nakov, and Xiuying Chen. Shaping the safety boundaries: Understanding and defending against jailbreaks in large language models. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume ...

2025
[131]

Evaluate bias without manual test sets: A concept representation perspective for llms, 2025 b

Lang Gao, Kaiyang Wan, Wei Liu, Chenxi Wang, Zirui Song, Zixiang Xu, Yanbo Wang, Veselin Stoyanov, and Xiuying Chen. Evaluate bias without manual test sets: A concept representation perspective for llms, 2025 b . URL https://arxiv.org/abs/2505.15524

2025 arXiv
[132]

When personalization tricks detectors: The feature-inversion trap in machine-generated text detection

Lang Gao, Xuhui Li, Chenxi Wang, Mingzhe Li, Wei Liu, Zirui Song, Jinghui Zhang, Rui Yan, Preslav Nakov, and Xiuying Chen. When personalization tricks detectors: The feature-inversion trap in machine-generated text detection. In Proceedings of the 64th Annual Meeting of the As...

2026
[133]

The cylindrical representation hypothesis for language model steering, 2026 b

Lang Gao, Jinghui Zhang, Wei Liu, Fengxian Ji, Chenxi Wang, Zirui Song, Akash Ghosh, Youssef Mohamed, Preslav Nakov, and Xiuying Chen. The cylindrical representation hypothesis for language model steering, 2026 b . URL https://arxiv.org/abs/2605.01844

2026 arXiv
[134]

A survey on LLM -as-a-judge, 2024

Jiawei Gu, Xuhui Jiang, Zhichao Shi, Hexiang Tan, Xuehao Zhai, Chengjin Xu, Wei Li, Yinghan Shen, Shengjie Ma, Honghao Liu, Yuanzhuo Wang, and Jian Guo. A survey on LLM -as-a-judge, 2024

2024
[135]

Measuring massive multitask language understanding

Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. Proceedings of the International Conference on Learning Representations (ICLR), 2021

2021
[136]

Li, and Jacob Andreas

Evan Hernandez, Belinda Z. Li, and Jacob Andreas. Inspecting and editing knowledge representations in language models. In Advances in Neural Information Processing Systems (NeurIPS), 2023

2023
[137]

ASTRO : Automatic strategy optimization for non-cooperative dialogues

Yikuan Hu, Chen Huang, and Wenqiang Lei. ASTRO : Automatic strategy optimization for non-cooperative dialogues. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar, editors, Findings of the Association for Computational Linguistics: ACL 2025, pages 3...

2025 doi
[138]

ReMindRAG : Low-cost LLM -guided knowledge graph traversal for efficient RAG

Yikuan Hu, Jifeng Zhu, Lanrui Tang, and Chen Huang. ReMindRAG : Low-cost LLM -guided knowledge graph traversal for efficient RAG . In D. Belgrave, C. Zhang, H. Lin, R. Pascanu, P. Koniusz, M. Ghassemi, and N. Chen, editors, Advances in Neural Information Processing Systems, vo...

2025
[139]

Building a foundational guardrail for general agentic systems via synthetic data, 2025

Yue Huang, Hang Hua, Yujun Zhou, Pengcheng Jing, Manish Nagireddy, Inkit Padhi, Greta Dolcetti, Zhangchen Xu, Subhajit Chaudhury, Ambrish Rawat, Liubov Nedoshivina, Pin-Yu Chen, Prasanna Sattigeri, and Xiangliang Zhang. Building a foundational guardrail for general agentic sys...

2025
[140]

Emergent social intelligence risks in generative multi-agent systems, 2026 a

Yue Huang, Yu Jiang, Wenjie Wang, Haomin Zhuang, Xiaonan Luo, Yuchen Ma, Zhangchen Xu, Zichen Chen, Nuno Moniz, Zinan Lin, Pin-Yu Chen, Nitesh V Chawla, Nouha Dziri, Huan Sun, and Xiangliang Zhang. Emergent social intelligence risks in generative multi-agent systems, 2026 a . ...

2026 arXiv
[141]

Probellm: Automating principled diagnosis of llm failures, 2026 b

Yue Huang, Zhengzhe Jiang, Yuchen Ma, Yu Jiang, Xiangqi Wang, Yujun Zhou, Yuexing Hao, Kehan Guo, Pin-Yu Chen, Stefan Feuerriegel, and Xiangliang Zhang. Probellm: Automating principled diagnosis of llm failures, 2026 b . URL https://arxiv.org/abs/2602.12966

2026 arXiv
[142]

Narra-gym for evaluating interactive narrative agents, 2026 c

Yue Huang, Yuchen Ma, Jiayi Ye, Wenjie Wang, Zipeng Ling, Xingjian Hu, Yuexing Hao, Zichen Chen, Zhangchen Xu, Yunhong He, Zhengqing Yuan, Yujun Zhou, Kehan Guo, Chaoran Chen, Toby Jia-Jun Li, Stefan Feuerriegel, and Xiangliang Zhang. Narra-gym for evaluating interactive narra...

2026 arXiv
[143]

Spa: Achieving consensus in llm alignment via self-priority optimization

Yue Huang, Xiangqi Wang, and Xiangliang Zhang. Spa: Achieving consensus in llm alignment via self-priority optimization. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 31220--31228, 2026 d . doi:10.1609/aaai.v40i37.40384. URL https://ojs.aaa...

2026 doi
[144]

Guardian-as-an-advisor: Advancing next-generation guardian models for trustworthy LLM s

Yue Huang, Haomin Zhuang, Jiayi Ye, Han Bao, Yanbo Wang, Hang Hua, Siyuan Wu, Pin-Yu Chen, and Xiangliang Zhang. Guardian-as-an-advisor: Advancing next-generation guardian models for trustworthy LLM s. In Findings of the Association for Computational Linguistics: ACL 2026, pag...

2026
[145]

ContextGuard : Structured self-auditing for context learning in language models, 2026 a

Hongbo Jin, Chi Wang, Haoran Tang, Zhongjing Du, Xu Jiang, Jingqi Tian, Qiaoman Zhang, and Jiayu Ding. ContextGuard : Structured self-auditing for context learning in language models, 2026 a . URL https://arxiv.org/abs/2605.26827

2026 arXiv
[146]

Context-CoT : Enhancing context learning via high-quality reasoning synthesis, 2026 b

Hongbo Jin, Mingnan Zhu, Jingqi Tian, Xu Jiang, Zhongjing Du, Haoran Tang, Siyi Xie, Qiaoman Zhang, and Jiayu Ding. Context-CoT : Enhancing context learning via high-quality reasoning synthesis, 2026 b . URL https://arxiv.org/abs/2605.25354

2026 arXiv
[147]

Cohen, and Xinghua Lu

Qiao Jin, Bhuwan Dhingra, Zhengping Liu, William W. Cohen, and Xinghua Lu. PubMedQA : A dataset for biomedical research question answering. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on N...

2019
[148]

Prometheus: Inducing fine-grained evaluation capability in language models

Seungone Kim, Jamin Shin, Yejin Cho, Joel Jang, Shayne Longpre, Hwaran Lee, Sangdoo Yun, Seongjin Shin, Sungdong Kim, James Thorne, and Minjoon Seo. Prometheus: Inducing fine-grained evaluation capability in language models. In International Conference on Learning Representati...

2024
[149]

Benchmarking cognitive biases in large language models as evaluators

Ryan Koo, Minhwa Lee, Vipul Raheja, Jong Inn Park, Zae Myung Kim, and Dongyeop Kang. Benchmarking cognitive biases in large language models as evaluators. In Findings of the Association for Computational Linguistics: ACL 2024, 2024

2024
[150]

Beyond the surface: Enhancing LLM -as-a-judge alignment with human via internal representations, 2025

Peng Lai, Jianjie Zheng, Sijie Cheng, Yun Chen, Peng Li, Yang Liu, and Guanhua Chen. Beyond the surface: Enhancing LLM -as-a-judge alignment with human via internal representations, 2025

2025
[151]

Smith, and Hannaneh Hajishirzi

Nathan Lambert, Valentina Pyatkin, Jacob Morrison, LJ Miranda, Bill Yuchen Lin, Khyathi Chandu, Nouha Dziri, Sachin Kumar, Tom Zick, Yejin Choi, Noah A. Smith, and Hannaneh Hajishirzi. RewardBench : Evaluating reward models for language modeling, 2024

2024
[152]

Lee, Inkit Padhi, Karthikeyan Natesan Ramamurthy, Erik Miehling, Pierre Dognin, Manish Nagireddy, and Amit Dhurandhar

Bruce W. Lee, Inkit Padhi, Karthikeyan Natesan Ramamurthy, Erik Miehling, Pierre Dognin, Manish Nagireddy, and Amit Dhurandhar. Programming refusal with conditional activation steering. In International Conference on Learning Representations (ICLR), 2025

2025
[153]

Preference leakage: A contamination problem in LLM -as-a-judge, 2025 a

Dawei Li, Renliang Sun, Yue Huang, Ming Zhong, Bohan Jiang, Jiawei Han, Xiangliang Zhang, Wei Wang, and Huan Liu. Preference leakage: A contamination problem in LLM -as-a-judge, 2025 a

2025
[154]

LLMs -as-judges: A comprehensive survey on LLM -based evaluation methods, 2024 a

Haitao Li, Qian Dong, Junjie Chen, Huixue Su, Yujia Zhou, Qingyao Ai, Ziyi Ye, and Yiqun Liu. LLMs -as-judges: A comprehensive survey on LLM -based evaluation methods, 2024 a

2024
[155]

CalibraEval : Calibrating prediction distribution to mitigate selection bias in LLMs -as-judges

Haitao Li, Junjie Chen, Qingyao Ai, Zhumin Chu, Yujia Zhou, Qian Dong, and Yiqun Liu. CalibraEval : Calibrating prediction distribution to mitigate selection bias in LLMs -as-judges. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (AC...

2025
[156]

Graph neural network explanations are fragile, 2024 b

Jiate Li, Meng Pang, Yun Dong, Jinyuan Jia, and Binghui Wang. Graph neural network explanations are fragile, 2024 b . URL https://arxiv.org/abs/2406.03193

2024 arXiv
[157]

Provably robust explainable graph neural networks against graph perturbation attacks, 2025 c

Jiate Li, Meng Pang, Yun Dong, Jinyuan Jia, and Binghui Wang. Provably robust explainable graph neural networks against graph perturbation attacks, 2025 c . URL https://arxiv.org/abs/2502.04224

2025 arXiv
[158]

Rossi, Yan Liu, Xiyang Hu, and Yue Zhao

Jiate Li, Defu Cao, Li Li, Wei Yang, Yuehan Qin, Chenxiao Yu, Tiannuo Yang, Ryan A. Rossi, Yan Liu, Xiyang Hu, and Yue Zhao. ``someone hid it'': Query-agnostic black-box attacks on llm-based retrieval, 2026 a . URL https://arxiv.org/abs/2602.00364

2026 arXiv
[159]

Inference-time intervention: Eliciting truthful answers from a language model

Kenneth Li, Oam Patel, Fernanda Vi \'e gas, Hanspeter Pfister, and Martin Wattenberg. Inference-time intervention: Eliciting truthful answers from a language model. In Advances in Neural Information Processing Systems (NeurIPS), 2023

2023
[160]

DPU : Dynamic prototype updating for multimodal out-of-distribution detection

Shawn Li, Huixian Gong, Hao Dong, Tiankai Yang, Zhengzhong Tu, and Yue Zhao. DPU : Dynamic prototype updating for multimodal out-of-distribution detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025 d . URL https://arxiv.o...

2025 arXiv
[161]

Treble counterfactual VLM s: A causal approach to hallucination, 2025 e

Shawn Li, Jiashu Qu, Yuxiao Zhou, Yuehan Qin, Tiankai Yang, and Yue Zhao. Treble counterfactual VLM s: A causal approach to hallucination, 2025 e . URL https://arxiv.org/abs/2503.06169

2025 arXiv
[162]

Defenses against prompt attacks learn surface heuristics, 2026 b

Shawn Li, Chenxiao Yu, Zhiyu Ni, Hao Li, Charith Peris, Chaowei Xiao, and Yue Zhao. Defenses against prompt attacks learn surface heuristics, 2026 b . URL https://arxiv.org/abs/2601.07185

2026
[163]

Split and merge: Aligning position biases in LLM -based evaluators

Zongjie Li, Chaozheng Wang, Pingchuan Ma, Daoyuan Wu, Shuai Wang, Cuiyun Gao, and Yang Liu. Split and merge: Aligning position biases in LLM -based evaluators. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2024 c

2024
[164]

TruthfulQA : Measuring how models mimic human falsehoods

Stephanie Lin, Jacob Hilton, and Owain Evans. TruthfulQA : Measuring how models mimic human falsehoods. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (ACL), 2022

2022
[165]

Drugagent: Automating ai-aided drug discovery programming through llm multi-agent collaboration, 2025

Sizhe Liu, Yizhou Lu, Siyu Chen, Xiyang Hu, Jieyu Zhao, Yingzhou Lu, and Yue Zhao. Drugagent: Automating ai-aided drug discovery programming through llm multi-agent collaboration, 2025. URL https://arxiv.org/abs/2411.15692

2025 arXiv
[166]

G-Eval : NLG evaluation using GPT-4 with better human alignment

Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu. G-Eval : NLG evaluation using GPT-4 with better human alignment. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2023

2023
[167]

The geometry of truth: Emergent linear structure in large language model representations of true/false datasets

Samuel Marks and Max Tegmark. The geometry of truth: Emergent linear structure in large language model representations of true/false datasets. In Conference on Language Modeling (COLM), 2024

2024
[168]

Locating and editing factual associations in GPT

Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associations in GPT . In Advances in Neural Information Processing Systems (NeurIPS), 2022

2022
[169]

Llama 3.1-8b

Meta. Llama 3.1-8b. https://huggingface.co/meta-llama/Llama-3.1-8B, 2024 a

2024
[170]

Llama 3.3-70b

Meta. Llama 3.3-70b. https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct, 2024 b

2024
[171]

Progress measures for grokking via mechanistic interpretability

Neel Nanda, Lawrence Chan, Tom Lieberum, Jess Smith, and Jacob Steinhardt. Progress measures for grokking via mechanistic interpretability. In International Conference on Learning Representations (ICLR), 2023 a

2023
[172]

Emergent linear representations in world models of self-supervised sequence models

Neel Nanda, Andrew Lee, and Martin Wattenberg. Emergent linear representations in world models of self-supervised sequence models. In Proceedings of the 6th BlackboxNLP Workshop, 2023 b

2023
[173]

Gpt-4o mini: Advancing cost-efficient intelligence

OpenAI. Gpt-4o mini: Advancing cost-efficient intelligence. https://openai.com/index/gpt-4o-mini-advancing-cost-efficient-intelligence/, 2024

2024
[174]

OpenAI. Gpt-4.1. https://openai.com/index/gpt-4-1/, 2025

2025
[175]

Bowman, and Shi Feng

Arjun Panickssery, Samuel R. Bowman, and Shi Feng. LLM evaluators recognize and favor their own generations. In Advances in Neural Information Processing Systems (NeurIPS), 2024 a

2024
[176]

Steering Llama 2 via contrastive activation addition

Nina Panickssery, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, and Alexander Matt Turner. Steering Llama 2 via contrastive activation addition. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL), 2024 b

2024
[177]

The linear representation hypothesis and the geometry of large language models

Kiho Park, Yo Joong Choe, and Victor Veitch. The linear representation hypothesis and the geometry of large language models. In International Conference on Machine Learning (ICML), 2024

2024
[178]

Alicia Parrish, Angelica Chen, Nikita Nangia, Vishakh Padmakumar, Jason Phang, Jana Thompson, Phu Mon Htut, and Samuel R. Bowman. BBQ : A hand-built bias benchmark for question answering. In Findings of the Association for Computational Linguistics: ACL 2022, 2022

2022
[179]

Null it out: Guarding protected attributes by iterative nullspace projection

Shauli Ravfogel, Yanai Elazar, Hila Gonen, Michael Twiton, and Yoav Goldberg. Null it out: Guarding protected attributes by iterative nullspace projection. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL), 2020

2020
[180]

Linear adversarial concept erasure

Shauli Ravfogel, Michael Twiton, Yoav Goldberg, and Ryan Cotterell. Linear adversarial concept erasure. In International Conference on Machine Learning (ICML), 2022

2022
[181]

David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R. Bowman. GPQA : A graduate-level google-proof Q&A benchmark. In Conference on Language Modeling (COLM), 2024

2024
[182]

Verbosity bias in preference labeling by large language models, 2023

Keita Saito, Akifumi Wachi, Koki Wataoka, and Youhei Akimoto. Verbosity bias in preference labeling by large language models, 2023. NeurIPS 2023 Workshop on Instruction Tuning and Instruction Following

2023
[183]

Judging the judges: A systematic study of position bias in LLM -as-a-judge

Lin Shi, Chiyu Ma, Weicheng Liang, Xinyi Ma, Weijian Shi, and Soroush Vosoughi. Judging the judges: A systematic study of position bias in LLM -as-a-judge. In Proceedings of the Joint International Conference on Natural Language Processing and the Asia-Pacific Chapter of the A...

2025
[184]

Turner, and Luis Espinosa-Anke

Zara Siddique, Liam D. Turner, and Luis Espinosa-Anke. Shifting perspectives: Steering vector ensembles for robust bias mitigation in LLMs , 2025

2025
[185]

QUITE : A query rewrite system beyond rules with LLM agents, 2025 a

Yuyang Song, Hanxu Yan, Jiale Lao, Yibo Wang, Yufei Li, Yuanchun Zhou, Jianguo Wang, and Mingjie Tang. QUITE : A query rewrite system beyond rules with LLM agents, 2025 a . URL https://arxiv.org/abs/2506.07675

2025
[186]

Mmac-copilot: Multi-modal agent collaboration operating system copilot

Zirui Song, Yaohang Li, Meng Fang, Zhenhao Chen, Zecheng Shi, Yuan Huang, and Ling Chen. Mmac-copilot: Multi-modal agent collaboration operating system copilot. arXiv preprint arXiv:2404.18074, 2024

2024 arXiv
[187]

Injecting domain-specific knowledge into large language models: A comprehensive survey

Zirui Song, Bin Yan, Yuhan Liu, Miao Fang, Mingzhe Li, Rui Yan, and Xiuying Chen. Injecting domain-specific knowledge into large language models: A comprehensive survey. arXiv preprint arXiv:2502.10708, 2025 b

2025 arXiv
[188]

The evaluation bottleneck of vision-language-action models: An evaluation-centric survey

Zirui Song, Huaxing Liu, Xiang Wang, Shuai Li, Xinye Li, Yuheng Ji, Lang Gao, Jinghui Zhang, Xianhui Meng, Xiaojun Chang, and Xiuying Chen. The evaluation bottleneck of vision-language-action models: An evaluation-centric survey. Preprints, 2026. doi:10.20944/preprints202606.0...

2026 doi
[189]

Large language models are inconsistent and biased evaluators, 2024

Rickard Stureborg, Dimitris Alikaniotis, and Yoshi Suhara. Large language models are inconsistent and biased evaluators, 2024

2024
[190]

Nishant Subramani, Nivedita Suresh, and Matthew E. Peters. Extracting latent steering vectors from pretrained language models. In Findings of the Association for Computational Linguistics: ACL 2022, 2022

2022
[191]

CommonsenseQA : A question answering challenge targeting commonsense knowledge

Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. CommonsenseQA : A question answering challenge targeting commonsense knowledge. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), 2019

2019
[192]

Tang, Alejandro Cuadron, Chenguang Wang, Raluca Ada Popa, and Ion Stoica

Sijun Tan, Siyuan Zhuang, Kyle Montgomery, William Y. Tang, Alejandro Cuadron, Chenguang Wang, Raluca Ada Popa, and Ion Stoica. JudgeBench : A benchmark for evaluating LLM -based judges. In International Conference on Learning Representations (ICLR), 2025

2025
[193]

StealthRank : LLM ranking manipulation via stealthy prompt optimization, 2025

Yiming Tang, Yi Fan, Chenxiao Yu, Tiankai Yang, Yue Zhao, and Xiyang Hu. StealthRank : LLM ranking manipulation via stealthy prompt optimization, 2025. URL https://arxiv.org/abs/2504.05804

2025 arXiv
[194]

Gemma 3, 2025 a

Gemma Team. Gemma 3, 2025 a . URL https://goo.gle/Gemma3Report

2025
[195]

Qwen2.5 technical report, 2025 b

Qwen Team. Qwen2.5 technical report, 2025 b . URL https://huggingface.co/Qwen/Qwen2.5-72B

2025
[196]

Turner, Callum McDougall, Monte MacDiarmid, Alex Tamkin, Esin Durmus, Tristan Hume, Francesco Mosconi, C

Adly Templeton, Tom Conerly, Jonathan Marcus, Jack Lindsey, Trenton Bricken, Brian Chen, Adam Pearce, Craig Citro, Emmanuel Ameisen, Andy Jones, Hoagy Cunningham, Nicholas L. Turner, Callum McDougall, Monte MacDiarmid, Alex Tamkin, Esin Durmus, Tristan Hume, Francesco Mosconi,...

2024
[197]

Judging the judges: Evaluating alignment and vulnerabilities in LLMs -as-judges

Aman Singh Thakur, Kartik Choudhary, Venkat Srinik Ramayapally, Sankaran Vaidyanathan, and Dieuwke Hupkes. Judging the judges: Evaluating alignment and vulnerabilities in LLMs -as-judges. In Proceedings of the Fourth Workshop on Generation, Evaluation and Metrics (GEM), 2025

2025
[198]

Activation addition: Steering language models without optimization, 2023

Alexander Matt Turner, Lisa Thiergart, David Udell, Gavin Leech, Ulisse Mini, and Monte MacDiarmid. Activation addition: Steering language models without optimization, 2023

2023
[199]

Replacing judges with juries: Evaluating LLM generations with a panel of diverse models, 2024

Pat Verga, Sebastian Hofstatter, Sophia Althammer, Yixuan Su, Aleksandra Piktus, Arkady Arkhangorodsky, Minjie Xu, Naomi White, and Patrick Lewis. Replacing judges with juries: Evaluating LLM generations with a panel of diverse models, 2024

2024
[200]

A cognitive writing perspective for constrained long-form text generation

Kaiyang Wan, Honglin Mu, Rui Hao, Haoran Luo, Tianle Gu, and Xiuying Chen. A cognitive writing perspective for constrained long-form text generation. In Findings of the Association for Computational Linguistics: ACL 2025, pages 9832--9844, Vienna, Austria, 2025. Association fo...

2025
[201]

A fano-style accuracy upper bound for llm single-pass reasoning in multi-hop qa, 2026

Kaiyang Wan, Lang Gao, Honglin Mu, Preslav Nakov, Yuxia Wang, and Xiuying Chen. A fano-style accuracy upper bound for llm single-pass reasoning in multi-hop qa, 2026. URL https://arxiv.org/abs/2509.21199

2026 arXiv
[202]

Word form matters: LLM s' semantic reconstruction under typoglycemia

Chenxi Wang, Tianle Gu, Zhongyu Wei, Lang Gao, Zirui Song, and Xiuying Chen. Word form matters: LLM s' semantic reconstruction under typoglycemia. In Findings of the Association for Computational Linguistics: ACL 2025, pages 16870--16885, Vienna, Austria, 2025 a . Association ...

2025
[203]

Decoding echo chambers: LLM -powered simulations revealing polarization in social networks

Chenxi Wang, Zongfang Liu, Dequan Yang, and Xiuying Chen. Decoding echo chambers: LLM -powered simulations revealing polarization in social networks. In Proceedings of the 31st International Conference on Computational Linguistics, pages 3913--3923, Abu Dhabi, UAE, 2025 b . As...

2025
[204]

Under the shadow of babel: How language shapes reasoning in llms, 2025 c

Chenxi Wang, Yixuan Zhang, Lang Gao, Zixiang Xu, Zirui Song, Yanbo Wang, and Xiuying Chen. Under the shadow of babel: How language shapes reasoning in llms, 2025 c . URL https://arxiv.org/abs/2506.16151

2025 arXiv
[205]

Do LLM s ``feel''? emotion circuits discovery and control, 2025 d

Chenxi Wang, Yixuan Zhang, Ruiji Yu, Yufei Zheng, Lang Gao, Zirui Song, Zixiang Xu, Gus Xia, Huishuai Zhang, Dongyan Zhao, and Xiuying Chen. Do LLM s ``feel''? emotion circuits discovery and control, 2025 d . URL https://arxiv.org/abs/2510.11328

2025
[206]

Large language models are not fair evaluators

Peiyi Wang, Lei Li, Liang Chen, Zefan Cai, Dawei Zhu, Binghuai Lin, Yunbo Cao, Qi Liu, Tianyu Liu, and Zhifang Sui. Large language models are not fair evaluators. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL), 2024 a

2024
[207]

Adaptive distraction: Probing LLM contextual robustness with automated tree search

Yanbo Wang, Zixiang Xu, Yue Huang, Chujie Gao, Siyuan Wu, Jiayi Ye, Pin-Yu Chen, Xiuying Chen, and Xiangliang Zhang. Adaptive distraction: Probing LLM contextual robustness with automated tree search. In D. Belgrave, C. Zhang, H. Lin, R. Pascanu, P. Koniusz, M. Ghassemi, and N...

2025
[208]

DyFlow : Dynamic workflow framework for agentic reasoning

Yanbo Wang, Zixiang Xu, Yue Huang, Xiangqi Wang, Zirui Song, Lang Gao, Chenxi Wang, Robert Tang, Yue Zhao, Arman Cohan, Xiangliang Zhang, and Xiuying Chen. DyFlow : Dynamic workflow framework for agentic reasoning. In D. Belgrave, C. Zhang, H. Lin, R. Pascanu, P. Koniusz, M. G...

2025
[209]

PandaLM : An automatic evaluation benchmark for LLM instruction tuning optimization

Yidong Wang, Zhuohao Yu, Zhengran Zeng, Linyi Yang, Cunxiang Wang, Hao Chen, Chaoya Jiang, Rui Xie, Jindong Wang, Xing Xie, Wei Ye, Shikun Zhang, and Yue Zhang. PandaLM : An automatic evaluation benchmark for LLM instruction tuning optimization. In International Conference on ...

2024
[210]

Ponder & press: Advancing visual GUI agent towards general computer control, 2024 c

Yiqin Wang, Haoji Zhang, Jingqi Tian, and Yansong Tang. Ponder & press: Advancing visual GUI agent towards general computer control, 2024 c . URL https://arxiv.org/abs/2412.01268

2024 arXiv
[211]

Kakade, Hao Peng, and Heng Ji

Ziqi Wang, Hanlin Zhang, Xiner Li, Kuan-Hao Huang, Chi Han, Shuiwang Ji, Sham M. Kakade, Hao Peng, and Heng Ji. Eliminating position bias of language models: A mechanistic approach. In International Conference on Learning Representations (ICLR), 2025 g

2025
[212]

Self-preference bias in LLM -as-a-judge, 2024

Koki Wataoka, Tsubasa Takahashi, and Ryokan Ri. Self-preference bias in LLM -as-a-judge, 2024

2024
[213]

Large-scale terminal agentic trajectory generation from dockerized environments

Siwei Wu, Yizhi Li, Yuyang Song, Wei Zhang, Yang Wang, Riza Batista-Navarro, Xian Yang, Mingjie Tang, Bryan Dai, Jian Yang, and Chenghua Lin. Large-scale terminal agentic trajectory generation from dockerized environments. In Proceedings of the 43rd International Conference on...

2026
[214]

Graph synthetic out-of-distribution exposure with large language models, 2025 a

Haoyan Xu, Zhengtao Yao, Ziyi Wang, Zhan Cheng, Xiyang Hu, Mengyuan Li, and Yue Zhao. Graph synthetic out-of-distribution exposure with large language models, 2025 a . URL https://arxiv.org/abs/2504.21198

2025 arXiv
[215]

Yu, Mengyuan Li, and Yue Zhao

Haoyan Xu, Zhengtao Yao, Xuzhi Zhang, Ziyi Wang, Langzhou He, Yushun Dong, Philip S. Yu, Mengyuan Li, and Yue Zhao. GLIP-OOD : Zero-shot graph OOD detection with graph foundation model, 2025 b . URL https://arxiv.org/abs/2504.21186

2025 arXiv
[216]

Cross-lingual pitfalls: Automatic probing cross-lingual weakness of multilingual large language models

Zixiang Xu, Yanbo Wang, Yue Huang, Xiuying Chen, Jieyu Zhao, Meng Jiang, and Xiangliang Zhang. Cross-lingual pitfalls: Automatic probing cross-lingual weakness of multilingual large language models. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar...

2025 doi
[217]

GTA : Graph theory agent and benchmark for algorithmic graph reasoning with LLM s

Zixiang Xu, Yanbo Wang, Yue Huang, Chenxi Wang, Lang Gao, Zirui Song, Zhaorun Chen, Xiangliang Zhang, and Xiuying Chen. GTA : Graph theory agent and benchmark for algorithmic graph reasoning with LLM s. OpenReview, 2025 d . URL https://openreview.net/forum?id=bcGClKY3gQ

2025
[218]

SocialMaze : A benchmark for evaluating social reasoning in large language models, 2025 e

Zixiang Xu, Yanbo Wang, Yue Huang, Jiayi Ye, Haomin Zhuang, Zirui Song, Lang Gao, Chenxi Wang, Zhaorun Chen, Yujun Zhou, Sixian Li, Wang Pan, Yue Zhao, Jieyu Zhao, Xiangliang Zhang, and Xiuying Chen. SocialMaze : A benchmark for evaluating social reasoning in large language mo...

2025 arXiv
[219]

Qwen3 technical report, 2025 a

An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jia...

2025 arXiv
[220]

AD-AGENT : A multi-agent framework for end-to-end anomaly detection

Tiankai Yang, Junjun Liu, Michael Siu, Jiahang Wang, Zhuangzhuang Qian, Chanjuan Song, Cheng Cheng, Xiyang Hu, and Yue Zhao. AD-AGENT : A multi-agent framework for end-to-end anomaly detection. In Findings of the Association for Computational Linguistics: IJCNLP-AACL 2025, 202...

2025 arXiv
[221]

AD-LLM : Benchmarking large language models for anomaly detection

Tiankai Yang, Yi Nian, Shawn Li, Ruiyao Xu, Yuangang Li, Jiaqi Li, Zhuo Xiao, Xiyang Hu, Ryan Rossi, Kaize Ding, Xia Hu, and Yue Zhao. AD-LLM : Benchmarking large language models for anomaly detection. In Findings of the Association for Computational Linguistics: ACL 2025, 202...

2025
[222]

Auditing multi-agent LLM reasoning trees outperforms majority vote and LLM -as-judge, 2026

Wei Yang, Shixuan Li, Heng Ping, Peiyu Zhang, Paul Bogdan, and Jesse Thomason. Auditing multi-agent LLM reasoning trees outperforms majority vote and LLM -as-judge, 2026. URL https://arxiv.org/abs/2602.09341

2026
[223]

Chawla, and Xiangliang Zhang

Jiayi Ye, Yanbo Wang, Yue Huang, Dongping Chen, Qihui Zhang, Nuno Moniz, Tian Gao, Werner Geyer, Chao Huang, Pin-Yu Chen, Nitesh V. Chawla, and Xiangliang Zhang. Justice or prejudice? quantifying biases in LLM -as-a-judge. In International Conference on Learning Representation...

2025
[224]

Towards more accurate US presidential election via multi-step reasoning with large language models, 2024

Chenxiao Yu, Zhaotian Weng, Yuangang Li, Zheng Li, Xiyang Hu, and Yue Zhao. Towards more accurate US presidential election via multi-step reasoning with large language models, 2024. URL https://arxiv.org/abs/2411.03321

2024 arXiv
[225]

Evaluating large language models at evaluating instruction following

Zhiyuan Zeng, Jiatong Yu, Tianyu Gao, Yu Meng, Tanya Goyal, and Danqi Chen. Evaluating large language models at evaluating instruction following. In International Conference on Learning Representations (ICLR), 2024

2024
[226]

Xing, Hao Zhang, Joseph E

Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. Judging LLM -as-a-judge with MT -bench and chatbot arena. In Advances in Neural Information Proce...

2023
[227]

JudgeLM : Fine-tuned large language models are scalable judges

Lianghui Zhu, Xinggang Wang, and Xinlong Wang. JudgeLM : Fine-tuned large language models are scalable judges. In International Conference on Learning Representations (ICLR), 2025

2025
[228]

Byun, Zifan Wang, Alex Mallen, Steven Basart, Sanmi Koyejo, Dawn Song, Matt Fredrikson, J

Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, Shashwat Goel, Nathaniel Li, Michael J. Byun, Zifan Wang, Alex Mallen, Steven Basart, Sanmi Koyejo, Dawn Song, Matt Fredrikson, J. Zico...

2023

Pith tools

Reviewed July 14, 2026 · model on record in the stance chip above.

[1] [1]

2025 , eprint=

Qwen3 Technical Report , author=. 2025 , eprint=

2025

[2] [2]

2025 , url=

Qwen2.5 Technical Report , author=. 2025 , url=

2025

[3] [4]

2022 , eprint=

Lin, Stephanie and Hilton, Jacob and Evans, Owain , booktitle=. 2022 , eprint=

2022

[4] [5]

2019 , eprint=

Talmor, Alon and Herzig, Jonathan and Lourie, Nicholas and Berant, Jonathan , booktitle=. 2019 , eprint=

2019

[5] [6]

and Lu, Xinghua , booktitle=

Jin, Qiao and Dhingra, Bhuwan and Liu, Zhengping and Cohen, William W. and Lu, Xinghua , booktitle=. 2019 , eprint=

2019

[6] [7]

, booktitle=

Rein, David and Hou, Betty Li and Stickland, Asa Cooper and Petty, Jackson and Pang, Richard Yuanzhe and Dirani, Julien and Michael, Julian and Bowman, Samuel R. , booktitle=. 2024 , eprint=

2024

[7] [8]

2018 , eprint =

Clark, Peter and Cowhey, Isaac and Etzioni, Oren and Khot, Tushar and Sabharwal, Ashish and Schoenick, Carissa and Tafjord, Oyvind , title =. 2018 , eprint =

2018

[8] [10]

2024 , howpublished =

GPT-4o mini: Advancing Cost-Efficient Intelligence , author =. 2024 , howpublished =

2024