Pith. sign in

REVIEW 3 major objections 4 minor 29 references

Position Bias Undermines Preference Consistency in Listwise LLM-Based Reranking

T0 review · 3 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read High relevance can hide an LLM reranker's worst order stability.

desk verdict Useful audit framework and a striking counterexample, but the STELLA_LW result may be confounded by unseeded internal shuffles and the paper lacks error bars. read the letter →

arxiv 2608.03091 v1 pith:2PPFY2WC submitted 2026-08-04 cs.IR

Ethan Bito , Yongli Ren , Estrid He This is my paper · ORCID
classification cs.IR
keywords listwisererankingpositionbiaspermutationinvariancepreferenceconsistencypairwiseinstabilityglobalinconsistencyoutputlargelanguagemodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that position bias in listwise LLM-based rerankers should be evaluated as a failure of the induced preference system, not just as changes in final ranked lists or marginal position exposure. It introduces three measures: pairwise preference instability (PPI), global preference inconsistency (GPI), and listwise output consistency (LOC). Across three decoder-only LLMs, two datasets, and list lengths of 15, 25, and 50, these three measures rank methods the same way, but they diverge from top-ranked relevance and from marginal exposure curves. The key demonstration is that STELLA_LW, a calibration method that flattens exposure and attains the highest HR@5, has the worst PPI, GPI, and LOC in every setting. The paper concludes that reducing marginal exposure skew is insufficient to establish ranking-function validity in LLM-based reranking.

What carries the argument

The framework's central object is the induced preference system $Q$, whose entries $q(i,j)$ record the fraction of sampled permutations in which candidate $i$ is ranked above candidate $j$. On top of $Q$, PPI conditions pairwise preference probabilities on coarse input-position buckets and takes the maximum bucket-to-bucket swing; GPI computes the disagreement between $Q$ and the best-fitting global ranking under a weighted Kemeny objective; and LOC averages Kendall's $\tau$ across the sampled rankings. Together they characterize candidate-order sensitivity at the pairwise, global, and output levels, and they are the measurement device that separates consistency from effectiveness and exposure.

What would settle it

Recompute the main table with substantially more permutations per query, or with bootstrap confidence intervals on PPI, GPI, and LOC, and check whether STELLA_LW still ranks worst on consistency and whether the overall method ordering is stable; if the ordering changes, the quantitative conclusions collapse even if the qualitative framework remains.

Watch

Extended reading notes

Core claim

The central claim is that a reranker's validity under equivalent candidate permutations is a property of the preference system each serialized ranking induces, and that this property is not implied by relevance or exposure metrics. Treating rankings from $M$ sampled permutations as observations, the paper defines pairwise preference probabilities $q(i,j)$, then measures pairwise instability, global inconsistency via a weighted Kemeny disagreement with the best-fitting ranking, and output consistency via average Kendall's $\tau$. The experiments show that these three consistency measures give the same method ordering across all settings, while effectiveness and marginal exposure can disagree with them; in particular, STELLA_LW improves top-ranked relevance and flattens exposure yet produces the most unstable and least coherent preferences. The paper therefore establishes that marginal exposure correction alone cannot certify permutation-consistent reranking.

Load-bearing premise

The measurement assumption that 20 sampled permutations per query give reliable estimates of pairwise preference probabilities and consistency scores is load-bearing, and the paper reports no variance, confidence intervals, or significance tests around its method ordering.

Editorial extensions

If this is right

  • Evaluating LLM rerankers with HR or nDCG alone can miss severe permutation instability; PPI, GPI, and LOC provide a complementary axis.
  • Marginal position-exposure flattening should not be treated as evidence of permutation invariance.
  • SGS-style sequential selection can improve all three consistency measures at the cost of multiple sequential inference passes.
  • Because PPI, GPI, and LOC agree across models and datasets, one of them may suffice as a practical consistency probe.
  • Deployment decisions should weigh consistency against inference cost and effectiveness together.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: if the observed alignment of PPI, GPI, and LOC generalizes, a lightweight consistency probe using a handful of permutations on a sample of queries could serve as a guardrail during model selection.
  • Beyond the paper: a control experiment that ties candidate relevance to near-equal quality would isolate pure position bias from content-driven preference, sharpening what PPI and GPI actually measure.
  • Beyond the paper: because STELLA_LW's calibration stage uses stochastic probing, its instability could originate in the calibration procedure rather than the base LLM; separating those sources would show where order sensitivity enters.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper studies candidate-order sensitivity of decoder-only LLM listwise rerankers for recommendation. Treating rankings produced under M=20 random permutations of a candidate set as observations of an induced preference system, it introduces three metrics: Pairwise Preference Instability (PPI), Global Preference Inconsistency (GPI), and Listwise Output Consistency (LOC). Experiments with three LLMs (Llama-3.2-3B-Instruct, Mistral-7B-Instruct-v0.3, Qwen2.5-7B-Instruct), two datasets (MovieLens-32M, Amazon Books), and list lengths K=15/25/50 compare zero-shot reranking against three baselines: bootstrapping, SGS, and STELLA_LW. The main finding is that PPI, GPI, and LOC are closely aligned with one another but can diverge from recommendation effectiveness and from marginal position-exposure bias: STELLA_LW obtains the highest HR@5 while exhibiting the worst PPI, GPI, and LOC, and flatter exposure curves. The authors conclude that reducing marginal exposure skew is insufficient to establish ranking-function validity in LLM-based reranking.

Significance. If the measurements hold, the paper makes a useful contribution to evaluating LLM-based rerankers. It shifts the evaluation criterion from output-level agreement to the structure of the induced preference system, and the observed divergence between effectiveness, exposure, and consistency is a non-obvious empirical result. The three metrics are defined directly from ranked outputs with no fitted parameters, the code is released, and the experimental design covers multiple models, datasets, and list lengths. The central claim is falsifiable and the framework is likely to be reused by the community. However, the strength of the conclusion depends on the reliability of the measurements, which currently lack uncertainty quantification and careful control of stochasticity in the baselines.

major comments (3)
  1. [Section 4, Table 1] The paper reports a single point estimate for every metric and never reports variance, confidence intervals, or significance tests. With M=20 sampled permutations per query and the number of evaluation queries not stated in the paper, the PPI, GPI, and LOC estimates may be substantially affected by sampling noise; in Table 1 several effectiveness differences are small (e.g., nDCG@5 of 0.2301 vs. 0.2324 for Llama-3B on MovieLens-32M). The central counterexample (STELLA_LW's high PPI/GPI and low LOC) and the claimed alignment of the three consistency measures need per-query standard errors, bootstrap confidence intervals, or paired significance tests before the method ordering can be trusted.
  2. [Section 4 (STELLA_LW/SGS)] The statement that "variation across evaluation rankings is induced by candidate permutation rather than decoding randomness or formatting errors" covers the extraction of token log-probabilities, but STELLA_LW and SGS are described as using internal reshuffles: STELLA_LW updates a posterior using "the input position of the model's top-ranked candidate over the original and shuffled candidate orders," and SGS "reshuffles the remaining candidates." If these internal shuffles are not seeded per evaluation permutation, then repeated evaluation of the same permutation could produce different rankings for STELLA_LW and SGS. Within-permutation algorithmic randomness would inflate PPI and GPI and deflate LOC, precisely the pattern observed for STELLA_LW, and would confound the paper's central counterexample. The authors should report seed handling for all internal shuffles and, ideally, the variance across repeated runs with identical evaluation permutations.
  3. [Section 3.3 (GPI)] GPI is computed from an approximate best-fitting ranking sigma_hat obtained by sorting on aggregate win scores followed by local swaps, because exact Kemeny rank aggregation is NP-hard. The reported value D(sigma_hat) is therefore an upper bound on the true minimum disagreement, and the approximation gap may vary across methods and datasets. The paper provides no validation of the approximation, despite GPI being one of the three metrics claimed to be closely aligned with PPI and LOC. Please validate the greedy approximation on a subset of queries with exact enumeration (e.g., K=15) or with random-restart/ILP-based search, and report the approximation gap.
minor comments (4)
  1. [Throughout] The notation is inconsistent: the text uses "STELLALW" while Table 1 and Figure captions use "STELLA_LW"; please unify the spelling.
  2. [Section 3.2] The sentence "Conditioning on every ordered position pair is sparse because there are K(K-1) possible pairs and only M sampled permutations" is confusing: the sparsity issue is in the number of observed bucket pairs per candidate pair, not in the number of ordered position pairs. Please rephrase.
  3. [Section 4] The paper states that position buckets are "coarse buckets B, such as head, middle, and tail," but never specifies the number of buckets or their boundaries in the experimental setup. Since PPI values depend directly on this choice, the bucket definition should be stated explicitly in Section 4.
  4. [Section 4] "Remaining settings are provided in the repository" is insufficient for reproducibility; at minimum, the paper should state the number of evaluation queries, the position bucket boundaries, and the generation/decoding settings used for STELLA_LW's calibration stage.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the consistency metrics are defined directly from sampled rankings, and the central claim is an empirical observation rather than a fitted or self-cited result.

full rationale

The paper's derivation chain is measurement-based and self-contained. PPI, GPI, and LOC are defined directly from the M=20 sampled rankings with no fitted constants or parameters (Sections 3.2-3.4); GPI is computed as a normalized Kemeny disagreement from the same pairwise probabilities, which the paper explicitly says are 'not intended to be statistically independent' (Section 3.4), so no target quantity is used as an input. The central conclusion that reducing marginal exposure skew is insufficient to establish ranking-function validity is supported by the empirical Table 1 pattern in which STELLA_LW flattens exposure yet has the highest PPI/GPI and lowest LOC; this is an observed contrast, not a reduction. Self-citations [2,3] appear only as related-work references and for deterministic token-log-probability extraction; neither carries the load-bearing inference. The possible internal-shuffle confound in STELLA_LW is a measurement-validity concern (mixing algorithmic stochasticity with position sensitivity), not a circular definition of the paper's metrics or claim.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The evaluation framework rests on four premises: the normative unordered-set assumption, the trust that log-probability-based rankings faithfully represent the model, the approximate greedy solution to Kemeny aggregation, and the hand-chosen experimental parameters (M=20, bucket definitions, k=5). These are reasonable for a workshop-length study but are not independently verified.

free parameters (4)
  • Number of sampled permutations M = 20
    All preference probabilities q(i,j) and LOC are estimated from M=20 permutations per query (Section 4); results may depend on this sampling budget.
  • Position buckets B = 3 buckets (head, middle, tail)
    PPI conditions on these coarse buckets (Section 3.2); boundaries and granularity are hand-chosen with no sensitivity analysis.
  • Top-k for exposure metric = 5
    Marginal exposure E_k(p) is reported for k=5 (Figure 1); conclusions about exposure flattening could change with k.
  • GPI greedy approximation = N/A
    The approximate optimum from greedy local swaps (Section 3.3) is used to define GPI; the approximation is a hand-chosen algorithmic choice with unknown bias.
assumptions (4)
  • domain assumption Candidates form an unordered set, so a coherent ranking function should be invariant under permutation of the serialized input.
    Normative premise motivating PPI/GPI/LOC; introduced in Section 3.1.
  • domain assumption Rankings extracted from token log-probabilities over candidate marker identifiers are deterministic and faithfully represent the LLM's preference.
    Section 4 states this extraction makes ranking deterministic for fixed input; the analysis assumes this score-based ranking reflects the reranker's true preference.
  • domain assumption The greedy pairwise-swap optimization in Section 3.3 finds a ranking close enough to the true Kemeny optimum that the resulting GPI is a valid measure of global incoherence.
    GPI is defined using the approximate best-fitting ranking; no bound on the approximation gap is given.
  • standard math The weighted Kemeny disagreement objective D(sigma) is an appropriate formalization of global preference inconsistency.
    Follows Dwork et al. [6]; the modeling choice is reasonable but not the only one.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Position Bias Undermines Preference Consistency in Listwise LLM-Based Reranking." pith.science (2026). https://pith.science/paper/2PPFY2WC

@misc{pith2026260803091,
  author       = {Pith},
  title        = {Pith review of: Position Bias Undermines Preference Consistency in Listwise LLM-Based Reranking},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2PPFY2WC}},
  note         = {Machine review of arXiv:2608.03091}
}
read the original abstract

Large language models (LLMs) have emerged as promising listwise rerankers for recommender systems, but their reliability under equivalent candidate permutations remains unclear. Since recommendation candidates form an unordered set, a reranker should not depend on the arbitrary order used to serialize them. However, decoder-only LLM rerankers can allow input order to affect model scores, pairwise preferences, and rankings. We study how position bias affects the ranking process induced by LLM-based rerankers. Instead of measuring only changes in final ranked lists, we treat rankings produced under equivalent candidate permutations as observations of an induced preference system. We introduce an evaluation framework measuring pairwise preference instability, global preference inconsistency, and listwise output consistency. This framework characterizes candidate-order sensitivity at the pairwise, global, and output levels. Experiments across multiple LLMs, datasets, and list lengths show that these consistency measures are closely aligned, but can diverge from recommendation effectiveness and marginal position-exposure bias. Improving relevance or flattening exposure across positions does not necessarily restore stable pairwise preferences, globally coherent preference structures, or consistent ranked outputs. These results show that reducing marginal exposure skew is insufficient to establish ranking-function validity in LLM-based reranking. Code is available at https://github.com/ejbito/InvariRank .

Figures

Figures reproduced from arXiv: 2608.03091 by the authors.

Figure 1
Figure 1. Top-5 exposure by input position on MovieLens-32M using Llama-3.2-3B-Instruct for list lengths [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. PPI and GPI across candidate-list lengths for Llama-3.2-3B [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

29 extracted references · 2 canonical work pages

  1. [1]

    Keqin Bao, Jizhi Zhang, Yang Zhang, Wenjie Wang, Fuli Feng, and Xiangnan He. 2023. TALLRec: An Effective and Efficient Tuning Framework to Align Large Language Model with Recommendation. InProceedings of the 17th ACM Conference on Recommender Systems (RecSys ’23). ACM, 1007–1014. doi:10.1145/ 3604915.3608857

    arXiv 2023

  2. [2]

    Ethan Bito, Yongli Ren, and Estrid He. 2025. Evaluating Position Bias in Large Language Model Recommendations. arXiv:2508.02020 [cs.IR] https://arxiv.org/ abs/2508.02020

    arXiv 2025

  3. [3]

    Ethan Bito, Yongli Ren, and Estrid He. 2026. One Pass, Any Order: Position- Invariant Listwise Reranking for LLM-Based Recommendation. InProceedings of the 49th International ACM SIGIR Conference on Research and Development in In- formation Retrieval(Australia)(SIGIR ’26). Association for Computing Machinery, New York, NY, USA, 3625–3629. doi:10.1145/3805...

    arXiv 2026

  4. [4]

    Wen-Shuo Chao, Zhi Zheng, Hengshu Zhu, and Hao Liu. 2024. Make Large Language Model a Better Ranker. InFindings of the Association for Computational Linguistics: EMNLP 2024, Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (Eds.). Association for Computational Linguistics, Miami, Florida, USA, 918–929. doi:10.18653/v1/2024.findings-emnlp.51

    work page doi:10.18653/v1/2024.findings-emnlp.51 2024

  5. [5]

    Sunhao Dai, Ninglu Shao, Haiyuan Zhao, Weijie Yu, Zihua Si, Chen Xu, Zhongx- iang Sun, Xiao Zhang, and Jun Xu. 2023. Uncovering ChatGPT’s Capabilities in Recommender Systems. InProceedings of the 17th ACM Conference on Recom- mender Systems (RecSys ’23). ACM, 1126–1132. doi:10.1145/3604915.3610646

    arXiv 2023

  6. [6]

    Sivakumar

    Cynthia Dwork, Ravi Kumar, Moni Naor, and D. Sivakumar. 2001. Rank aggrega- tion methods for the Web. InProceedings of the 10th International Conference on World Wide Web(Hong Kong, Hong Kong)(WWW ’01). Association for Comput- ing Machinery, New York, NY, USA, 613–622. doi:10.1145/371920.372165

    arXiv 2001

  7. [7]

    Luke Friedman, Sameer Ahuja, David Allen, Zhenning Tan, Hakim Sidahmed, Changbo Long, Jun Xie, Gabriel Schubiner, Ajay Patel, Harsh Lara, Brian Chu, Zexi Chen, and Manoj Tiwari. 2023. Leveraging Large Language Models in Conversational Recommender Systems. arXiv:2305.07961 [cs.IR] https://arxiv. org/abs/2305.07961

    arXiv 2023

  8. [8]

    Jingtong Gao, Bo Chen, Weiwen Liu, Xiangyang Li, Yichao Wang, Wanyu Wang, Huifeng Guo, Ruiming Tang, and Xiangyu Zhao. 2025. LLM4Rerank: LLM-based Auto-Reranking Framework for Recommendations. arXiv:2406.12433 [cs.IR] https://arxiv.org/abs/2406.12433

    arXiv 2025

Show all 29 references
  1. [9]

    Shijie Geng, Shuchang Liu, Zuohui Fu, Yingqiang Ge, and Yongfeng Zhang. 2022. Recommendation as language processing (rlp): A unified pretrain, personalized prompt & predict paradigm (p5). InProceedings of the 16th ACM conference on recommender systems. 299–315

    2022

  2. [10]

    Jiayuan He, Jianzhong Qi, and Kotagiri Ramamohanarao. 2019. A joint context- aware embedding for trip recommendations. In2019 IEEE 35th international conference on data engineering (ICDE). IEEE, 292–303

    2019

  3. [11]

    Jiayuan He, Jianzhong Qi, and Kotagiri Ramamohanarao. 2020. Timesan: A time- modulated self-attentive network for next point-of-interest recommendation. In 2020 International joint conference on neural networks (IJCNN). IEEE, 1–8

    2020

  4. [12]

    Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, Yongdong Zhang, and Meng Wang. 2020. LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation. arXiv:2002.02126 [cs.IR] https://arxiv.org/abs/2002.02126

    2020 arXiv

  5. [13]

    Yupeng Hou, Junjie Zhang, Zihan Lin, Hongyu Lu, Ruobing Xie, Julian McAuley, and Wayne Xin Zhao. 2024. Large Language Models are Zero-Shot Rankers for Recommender Systems. arXiv:2305.08845 [cs.IR] https://arxiv.org/abs/2305.08845

    2024 arXiv

  6. [14]

    Le, Abhishek Kumar, James Glass, Alexander Ratner, Chen-Yu Lee, Ranjay Krishna, and Tomas Pfister

    Cheng-Yu Hsieh, Yung-Sung Chuang, Chun-Liang Li, Zifeng Wang, Long T. Le, Abhishek Kumar, James Glass, Alexander Ratner, Chen-Yu Lee, Ranjay Krishna, and Tomas Pfister. 2024. Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization. arXiv:24...

    2024 arXiv

  7. [15]

    Junling Liu, Chao Liu, Peilin Zhou, Renjie Lv, Kang Zhou, and Yan Zhang. 2023. Is ChatGPT a Good Recommender? A Preliminary Study. arXiv:2304.10149 [cs.IR] https://arxiv.org/abs/2304.10149

    2023 arXiv

  8. [16]

    Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang

    Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2023. Lost in the Middle: How Language Models Use Long Contexts. arXiv:2307.03172 [cs.CL] https://arxiv.org/abs/2307.03172

    2023 arXiv

  9. [17]

    Tianhui Ma, Yuan Cheng, Hengshu Zhu, and Hui Xiong. 2023. Large Language Models are Not Stable Recommender Systems. arXiv:2312.15746 [cs.IR] https: //arxiv.org/abs/2312.15746

    2023 arXiv

  10. [18]

    Zhen Qin, Rolf Jagerman, Kai Hui, Honglei Zhuang, Junru Wu, Le Yan, Jiaming Shen, Tianqi Liu, Jialu Liu, Donald Metzler, Xuanhui Wang, and Michael Ben- dersky. 2024. Large Language Models are Effective Text Rankers with Pairwise Ranking Prompting. arXiv:2306.17563 [cs.IR] http...

    2024 arXiv

  11. [19]

    Ruiyang Ren, Yuhao Wang, Kun Zhou, Wayne Xin Zhao, Wenjie Wang, Jing Liu, Ji-Rong Wen, and Tat-Seng Chua. 2024. Self-Calibrated Listwise Reranking with Large Language Models. arXiv:2411.04602 [cs.IR] https://arxiv.org/abs/2411. 04602

    2024 arXiv

  12. [20]

    Scott Sanner, Krisztian Balog, Filip Radlinski, Ben Wedin, and Lucas Dixon. 2023. Large Language Models are Competitive Near Cold-start Recommenders for Language- and Item-based Preferences. arXiv:2307.14225 [cs.IR] https://arxiv. org/abs/2307.14225

    2023 arXiv

  13. [21]

    Weiwei Sun, Lingyong Yan, Xinyu Ma, Shuaiqiang Wang, Pengjie Ren, Zhumin Chen, Dawei Yin, and Zhaochun Ren. 2024. Is ChatGPT Good at Search? Investi- gating Large Language Models as Re-Ranking Agents. arXiv:2304.09542 [cs.CL] https://arxiv.org/abs/2304.09542

    2024 arXiv

  14. [22]

    Raphael Tang, Crystina Zhang, Xueguang Ma, Jimmy Lin, and Ferhan Türe

  15. [23]

    Wei Wei, Xubin Ren, Jiabin Tang, Qinyong Wang, Lixin Su, Suqi Cheng, Jun- feng Wang, Dawei Yin, and Chao Huang. 2024. Llmrec: Large language models with graph augmentation for recommendation. InProceedings of the 17th ACM international conference on web search and data mining. 806–815

    2024

  16. [24]

    Likang Wu, Zhi Zheng, Zhaopeng Qiu, Hao Wang, Hongchao Gu, Tingjia Shen, Chuan Qin, Chen Zhu, Hengshu Zhu, Qi Liu, Hui Xiong, and Enhong Chen. 2024. A Survey on Large Language Models for Recommendation. arXiv:2305.19860 [cs.IR] https://arxiv.org/abs/2305.19860

    2024 arXiv

  17. [25]

    Zhichao Xu, Daniel Cohen, Bei Wang, and Vivek Srikumar. 2024. In-Context Example Ordering Guided by Label Distributions. arXiv:2402.11447 [cs.CL] https://arxiv.org/abs/2402.11447

    2024 arXiv

  18. [26]

    Fan Yang, Zheng Chen, Ziyan Jiang, Eunah Cho, Xiaojiang Huang, and Yan- bin Lu. 2023. PALR: Personalization Aware LLMs for Recommendation. arXiv:2305.07622 [cs.IR] https://arxiv.org/abs/2305.07622

    2023 arXiv

  19. [27]

    Zhenrui Yue, Sara Rabhi, Gabriel de Souza Pereira Moreira, Dong Wang, and Even Oldridge. 2023. LlamaRec: Two-Stage Recommendation using Large Language Models for Ranking. arXiv:2311.02089 [cs.IR] https://arxiv.org/abs/2311.02089

    2023 arXiv

  20. [28]

    Zexuan Zhong and Danqi Chen. 2021. A Frustratingly Easy Approach for Entity and Relation Extraction. InProceedings of the 2021 Conference of the North Ameri- can Chapter of the Association for Computational Linguistics: Human Language Technologies, Kristina Toutanova, Anna Rum...

    2021 doi

  21. [2024]

    InProceedings of the 2024 conference of the North American chapter of the Association for Computational Linguistics: human language technologies (volume 1: long papers)

    Found in the middle: Permutation self-consistency improves listwise ranking in large language models. InProceedings of the 2024 conference of the North American chapter of the Association for Computational Linguistics: human language technologies (volume 1: long papers). 2327–2340

    2024

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.