Pith. sign in

REVIEW 4 major objections 5 minor 63 references

Perspective Transition of Large Language Models for Solving Subjective Tasks

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A prompt-only method, Reasoning through Perspective Transition, lets a large language model rank direct, expert-role, and third-person viewpoints by its own confidence and answer from the top-ranked one, consistently beating…

desk verdict A genuinely useful prompt-only method for subjective tasks, with a real but addressable confound between confidence-based perspective selection and prompt complexity. read the letter →

arxiv 2501.09265 v2 pith:LFTLVW5H submitted 2025-01-16 cs.CL cs.AI

Xiaolong Wang , Yuanchi Zhang , Ziyue Wang , Yuzhuang Xu , Fuwen Luo , Yile Wang , Peng Li , Yang Liu This is my paper · ORCID
classification cs.CLcs.AI
keywords perspectivetransitionsubjectivetasksin-contextlearningconfidence-basedselectionchain-of-thoughtpromptinglargelanguagemodelszero-shotreasoningrole
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes Reasoning through Perspective Transition (RPT), a prompt-only method that lets a large language model choose, for each subjective question, among answering directly, adopting an expert role, or reasoning as a third-person observer. The model first lists plausible perspectives, ranks them by its own stated confidence, and then answers using the top-ranked perspective, all within a single prompt and a single inference pass. Across twelve subjective tasks and four LLMs, RPT consistently beats fixed-perspective baselines such as chain-of-thought and expert prompting, with average gains of roughly 2 to 6 points. The paper argues this works because an LLM's confidence ranking over perspectives is informative: the perspective the model is most confident in is the one most likely to yield a correct answer.

What carries the argument

The central object is the perspective taxonomy together with the confidence-ranking step. RPT defines three answer stances: direct perspective (answer from the model's own understanding), role perspective (answer as an assigned expert persona), and third-person perspective (simulate a dialogue among agents and summarize). The load-bearing mechanism is Step 2: the same model that will answer is asked to rank these perspectives by how confident it is that its answer would be true, and the top-ranked perspective is then used for the final response. This turns the model's internal prior over viewpoints into a zero-training selector, so the method behaves like a 'mixture of perspectives' in which exactly one perspective is activated per query.

What would settle it

Run RPT and a reversed-confidence control with identical prompt structure; if the control matches or beats RPT's accuracy on the same twelve datasets, then confidence-based perspective selection is not the operative mechanism, and one should also check whether per-instance accuracy rises monotonically with the model's stated confidence.

Watch

Extended reading notes

Core claim

RPT is a unified three-step in-context-learning procedure. Given a task description and question, the model enumerates the available perspectives (direct, role, third-person), assigns a confidence score to each, and produces the final answer from the highest-confidence perspective. The three steps are concatenated into one prompt, so inference cost stays close to a single-perspective pass. On zero-shot tests, RPT reports the best average across the board: for GPT-3.5 zero-shot it reaches 77.94 points versus 73.38 for the strongest baseline, for GPT-4 80.81 versus 78.18, and it also improves on open-source models (e.g., Llama-3 zero-shot average 64.12 versus the best baseline 60.85). Few-shot experiments on stance and cultural-norm datasets show the same pattern, and ablations find that removing any one perspective costs 1.32–2.53 points while removing all three costs 7.60 points. The authors interpret the results as evidence that dynamic perspective selection, not a single fixed reasoning pathway, is what lets LLMs handle the ambiguity of subjective tasks.

Load-bearing premise

The central assumption is that a language model's self-reported confidence about which perspective will yield the right answer is actually predictive of correctness; if that ranking is unreliable, the measured gains could come from the extra reasoning steps the prompt demands rather than from genuinely selecting the best perspective.

Editorial extensions

If this is right

  • RPT requires no fine-tuning or extra parameters; it works with the same single-prompt interface used by chain-of-thought prompting.
  • Because the selection and final reasoning are combined in one inference pass, RPT keeps inference cost near that of a single perspective rather than a k-fold ensemble.
  • The gains hold across both closed-source (GPT-3.5, GPT-4) and open-source (Llama-3, Qwen-2) models, with the largest average improvements on the stronger closed-source models.
  • No single fixed perspective dominates: different datasets favor different perspectives, which is why the dynamic switch yields the best average performance.
  • The method is orthogonal to deeper chain-of-thought or reasoning models and can be combined with them.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the same confidence-ranking mechanism could be applied to other prompt-level decisions, such as choosing which few-shot exemplars, which reasoning strategy, or which decoding configuration, potentially turning any fixed prompt choice into a dynamic one.
  • Beyond the paper: because RPT achieves ensemble-like accuracy at single-pass cost, it may serve as a cheap upper-bound estimator for ensemble performance, letting practitioners decide when a full ensemble is worth running.
  • Beyond the paper: a direct stress test would be to reverse the confidence ranking; if accuracy does not drop, the operative cause would be the extra deliberation rather than perspective selection itself, which the paper's own B.1 analysis does not fully rule out.
  • Beyond the paper: the paper's Limitations section notes RPT selects among three coarse perspective types and operates in a single round of dialogue; extending the taxonomy to first-person, second-person, or multi-turn perspective shifts might improve coverage on dialogue-heavy subjective tasks, but that extension is untested here.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes Reasoning through Perspective Transition (RPT), a zero-training in-context method for subjective NLP tasks. RPT prompts an LLM to enumerate direct, role, and third-person perspectives; rank them by self-reported confidence; and answer from the highest-confidence perspective, all in one forward pass (Eq. 4). The authors evaluate RPT on 12 datasets across four LLMs in zero-shot and few-shot settings, reporting average gains over single-perspective and ensemble baselines, along with ablations, cost analyses, and confidence-calibration analyses.

Significance. If the mechanism were fully validated, RPT would be a useful, low-cost prompt recipe: it requires no fine-tuning, adds little inference overhead compared with ensembling, and the perspective taxonomy is simple to apply. The paper's strengths are breadth (four models, 12 tasks, zero- and few-shot), the internal control in Table 6 (forcing second/lowest/random confidence perspectives), and the explicit cost analysis. However, the central mechanism is currently underdetermined because the main comparison does not control for prompt complexity, the self-confidence evidence is based on same-pass generations, and several per-dataset results contradict the 'consistent outperformance' claim.

major comments (4)
  1. [Abstract; §4.2, Table 3] The abstract and §4.2 state that RPT 'consistently outperforms' fixed-perspective baselines, but Table 3 contains multiple counterexamples. For zero-shot Llama-3 on Humor, RPT scores 65.00 versus 66.25 for CoT-SC; for Qwen-2 on SNARKS, RPT scores 69.61 versus 79.56 for CoT-SC; and for GPT-4 on Humor, RPT scores 67.50 versus 73.75 for CoT-SC. The average improvements of 3–5 points therefore conceal systematic losses on individual tasks, and no significance tests or confidence intervals are reported anywhere in the paper. The 'consistent outperformance' claim should be replaced by a qualified claim, and the per-dataset results should be accompanied by bootstrap intervals or repeated-run variance.
  2. [§3, Eq. (4); §5.1; Appendix B.3] The main comparison is confounded by prompt structure. RPT's unified prompt T = T1⊕T2⊕T3 asks the model to enumerate perspectives, rank them by confidence, and then reason from the selected perspective; every fixed-perspective baseline uses a shorter, single-perspective prompt. The observed gains could therefore come from the extra deliberative structure ('consider all three, then commit') rather than from the trustworthiness of the confidence ranking. The ablations in Figure 4 and Table 5 remove perspectives but retain the ranking instruction, so they do not isolate the confidence mechanism; they also shrink the available perspective set rather than holding prompt structure fixed. I recommend adding a matched control: the same full prompt with the selection step replaced by a fixed perspective (e.g., 'always answer from the role perspective after considering all three'), and the same prompt with confidence ranking removed and selection made randomly. Only such controls can support the abstract's 'dynamically select' mechanism.
  3. [§B.1, Fig. 10; Eq. (4)] The confidence-accuracy evidence is computed on a single output sequence: P, C, and R are produced by the same model in the same forward pass (Eq. 4), and Fig. 10 compares the model's stated confidence with the correctness of that same generation. This shows calibration of the selected answer, not that the confidence ranking identifies the perspective whose independent answer would be best. A direct validation would generate separate answers from each perspective using matched prompts, then compare the accuracy of RPT's selected answer with (i) the answer from the highest-confidence perspective, (ii) the answer from a fixed perspective, and (iii) an oracle-best perspective. Without this, §B.1's conclusion that 'LLMs are capable of ranking the confidence of perspective' is underdetermined.
  4. [§4.1; §4.2, Tables 3–4] The paper reports a single deterministic run with temperature 0 for each cell, with no error bars, bootstrap intervals, or significance tests. Given test-set sizes as small as 70–80 examples (Humor, IPA, Entailment in Table 2), differences of 2–6 points are plausibly within sampling noise. For example, the claimed Llama-3 average gain of 3.27 points over the best baseline rests on differences that are below the resolution of several small datasets. The authors should report confidence intervals or repeated sampling with temperature greater than 0, and should state which per-dataset differences are statistically reliable.
minor comments (5)
  1. [Tables 3–4] The tables contain formatting artifacts such as '50 .00' and '33 .33'; the tables should be re-rendered carefully.
  2. [References] References Wang et al. 2024c and Wang et al. 2024d have identical titles and venues; one is a duplicate and should be removed or distinguished.
  3. [Appendix B.1] There is a typo in the sentence 'PRT offers perspective flexibility'—it should read 'RPT'.
  4. [Figure 7] The legend uses 'CoT', but the main text refers to several CoT variants; please clarify which variant is plotted.
  5. [Introduction, §2] The term 'subjective tasks' is used without an operational definition; consider defining it concretely (for example, tasks with high annotator disagreement or no objectively verifiable answer).

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: RPT's confidence-based perspective selection is evaluated against external task labels and baselines, and its internal mechanism is probed with forced low/random-confidence controls rather than assumed by construction.

full rationale

The central claim—that an LLM's self-reported confidence ranking over three perspectives identifies the perspective whose answer is most likely correct—is an empirical hypothesis, not a definitional identity. Equation (2) defines how P and C are generated, and Equation (4) combines the pipeline into one prompt, but neither equation defines the reported accuracy gains in terms of C. The gains are measured against fixed external labels in Tables 3 and 4, and the mechanism is tested in Table 6 by forcing the model to take the second-highest, lowest, or a random perspective; the monotone degradation is an external, falsifiable observation. The confidence–accuracy correlation in Figure 10 is likewise an empirical measurement, not a tautology. The only notable self-citations are the RiC baseline (Wang et al., 2024c) and the supporting statement in Appendix B.6 that CoT is not advantageous on subjective tasks (Wang et al., 2024d); these are used as comparisons or motivation, not as the proof of RPT's effectiveness, and the main result would stand without them. No equation reduces a prediction to a fitted input or imports a uniqueness theorem. Hence no circular step.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on hand-crafted prompts, a fixed three-way perspective taxonomy, and the premise that LLM confidence is meaningful. None of these are derived; the paper provides internal validation only. No new theoretical entities are introduced.

free parameters (3)
  • Number of demonstrations in few-shot main results = 3
    Appendix B.4 selects 3-shot because it 'yields the best performance'; this choice shapes Table 4.
  • Unified prompt template T1+T2+T3 = Exact wording in Section 3 / Table 1
    The method's behavior depends on hand-crafted instructions for perspective enumeration, confidence ranking, and final answer; no robustness to prompt variants is reported.
  • Perspective taxonomy size and members = 3 (direct, role, third-person)
    The categorization is 'based on related works' (Limitations), and the paper does not test other divisions; it is a design choice the central claim depends on.
assumptions (4)
  • domain assumption LLM self-reported confidence is a reliable proxy for answer correctness when selecting a perspective
    Step 2 of Section 3 ranks perspectives by confidence; Section B.1/B.7 tests this internally but it remains an empirical assumption about the same model.
  • domain assumption The three-perspective taxonomy is sufficient to capture the useful ways of solving subjective tasks
    Section 3 defines P = {p1,...,pn} with n usually 3; limitations admit the taxonomy could be refined.
  • domain assumption A single unified prompt can elicit perspective enumeration, confidence ranking, and final answer without mutual interference
    Section 3 'Combine All Steps through Unified Prompting' asserts this; no controlled comparison of staged vs unified prompting is reported.
  • domain assumption Zero temperature yields deterministic outputs representative of model behavior
    Section 4.1 sets temperature 0; no repeated samples or variance estimates are provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Perspective Transition of Large Language Models for Solving Subjective Tasks." pith.science (2026). https://pith.science/paper/LFTLVW5H

@misc{pith2026250109265,
  author       = {Pith},
  title        = {Pith review of: Perspective Transition of Large Language Models for Solving Subjective Tasks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LFTLVW5H}},
  note         = {Machine review of arXiv:2501.09265}
}
read the original abstract

Large language models (LLMs) have revolutionized the field of natural language processing, enabling remarkable progress in various tasks. Different from objective tasks such as commonsense reasoning and arithmetic question-answering, the performance of LLMs on subjective tasks is still limited, where the perspective on the specific problem plays crucial roles for better interpreting the context and giving proper response. For example, in certain scenarios, LLMs may perform better when answering from an expert role perspective, potentially eliciting their relevant domain knowledge. In contrast, in some scenarios, LLMs may provide more accurate responses when answering from a third-person standpoint, enabling a more comprehensive understanding of the problem and potentially mitigating inherent biases. In this paper, we propose Reasoning through Perspective Transition (RPT), a method based on in-context learning that enables LLMs to dynamically select among direct, role, and third-person perspectives for the best way to solve corresponding subjective problem. Through extensive experiments on totally 12 subjective tasks by using both closed-source and open-source LLMs including GPT-4, GPT-3.5, Llama-3, and Qwen-2, our method outperforms widely used single fixed perspective based methods such as chain-of-thought prompting and expert prompting, highlights the intricate ways that LLMs can adapt their perspectives to provide nuanced and contextually appropriate responses for different problems.

Figures

Figures reproduced from arXiv: 2501.09265 by the authors.

Figure 1
Figure 1. Two examples for showing challenges of solv [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. An example of solving dark humor detection task by different perspectives. (a) direct perspective: the [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. An overview of RPT pipeline. For each input question, RPT explores the available perspectives and then [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (12 more)
Figure 4
Figure 4. Figure 4: The impact of different perspectives on the [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: The relationship between the performance and prediction lengths of the 3-shot experiments on GPT-4. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: The inference cost of RPT when using differ [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 8
Figure 8. Figure 8: Case of SemEval task. We use GPT-4 to analyze attitudes toward Donald Trump. Our RPT method [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]
Figure 9
Figure 9. Figure 9: Keyword statistics of different perspectives in [PITH_FULL_IMAGE:figures/full_fig_p008_9.png]
Figure 10
Figure 10. Figure 10: Analysis on the correlation between confi [PITH_FULL_IMAGE:figures/full_fig_p013_10.png]
Figure 11
Figure 11. Figure 11: The averaged confidence level by our RPT [PITH_FULL_IMAGE:figures/full_fig_p013_11.png]
Figure 12
Figure 12. Figure 12: Human evaluation consistency on confi￾dence [PITH_FULL_IMAGE:figures/full_fig_p014_12.png]
Figure 13
Figure 13. Figure 13: The proportion of selected perspectives for [PITH_FULL_IMAGE:figures/full_fig_p014_13.png]
Figure 14
Figure 14. Figure 14: Cases of SocNorm task. We provide results of two widely used models, GPT-4 and Qwen-2. In each of [PITH_FULL_IMAGE:figures/full_fig_p015_14.png]
Figure 15
Figure 15. Figure 15: Cases of SemEval task. We provide detailed responses of three models, GPT-4, Qwen-2, and Llama [PITH_FULL_IMAGE:figures/full_fig_p016_15.png]
Figure 16
Figure 16. Figure 16: The performance of baselines and our RPT method by using different numbers of demonstrations [PITH_FULL_IMAGE:figures/full_fig_p017_16.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

63 extracted references · 17 canonical work pages

  1. [1]

    Aakriti Agrawal, Mucong Ding, Zora Che, Chenghao Deng, Anirudh Satheesh, John Langford, and Furong Huang. 2024. https://arxiv.org/abs/2410.04571 Ensemw2s: Can an ensemble of llms be leveraged to obtain a stronger llm? ArXiv preprint, abs/2410.04571

    arXiv 2024

  2. [2]

    Dor Bank, Daniel Greenfeld, and Gal Hyams. 2019. https://arxiv.org/abs/1710.00209 Improved training for self training by confidence assessments . In Intelligent Computing, pages 163--173, Cham. Springer International Publishing

    work page Pith review arXiv 2019

  3. [4]

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert - Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litw...

    2020

  4. [5]

    Sky CH-Wang, Arkadiy Saakyan, Oliver Li, Zhou Yu, and Smaranda Muresan. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.215 Sociocultural norm similarities and differences via situational alignment and explainable textual entailment . In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 3548--3564, Singapore. ...

    work page doi:10.18653/v1/2023.emnlp-main.215 2023

  5. [6]

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. 2021. https://arxiv.org/abs/2107.03374 Evaluating large language models trained on code . ArXiv preprint, abs/2107.03374

    arXiv 2021

  6. [7]

    Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. 2023. https://arxiv.org/abs/2204.02311 Palm: Scaling language modeling with pathways . Journal of Machine Learning Research, 24(240):1--113

    arXiv 2023

  7. [8]

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. https://arxiv.org/abs/2110.14168 Training verifiers to solve math word problems . ArXiv preprint, abs/2110.14168

    arXiv 2021

  8. [9]

    Sjoerd de Vries and Dirk Thierens. 2024. https://arxiv.org/abs/2409.16071 Learning with confidence: Training better classifiers from soft labels . ArXiv preprint, abs/2409.16071

    work page Pith review arXiv 2024

Show all 63 references
  1. [10]

    DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei F...

    2025 arXiv

  2. [11]

    Tenenbaum, and Igor Mordatch

    Yilun Du, Shuang Li, Antonio Torralba, Joshua B. Tenenbaum, and Igor Mordatch. 2024. https://openreview.net/forum?id=zj7YuTE4t8 Improving factuality and reasoning in language models through multiagent debate . In Forty-first International Conference on Machine Learning

    2024

  3. [12]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. https://arxiv.org/abs/2407.21783 The llama 3 herd of models . ArXiv preprint, abs/2407.21783

    2024 arXiv

  4. [13]

    António Farinhas, Haau-Sing Li, and André F. T. Martins. 2024. https://arxiv.org/abs/2409.07131 Reranking laws for language generation: A communication-theoretic perspective . ArXiv preprint, abs/2409.07131

    2024 arXiv

  5. [14]

    Mor Geva, Daniel Khashabi, Elad Segal, Tushar Khot, Dan Roth, and Jonathan Berant. 2021. https://doi.org/10.1162/tacl_a_00370 Did aristotle use a laptop? a question answering benchmark with implicit reasoning strategies . Transactions of the Association for Computational Lingu...

    2021 doi

  6. [15]

    Dan Hendrycks, Collin Burns, Steven Basart, Andrew Critch, Jerry Li, Dawn Song, and Jacob Steinhardt. 2021. https://openreview.net/forum?id=dNy\_RKzJacY Aligning AI with shared human values . In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event...

    2021

  7. [16]

    Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, et al. 2022. https://arxiv.org/abs/2203.15556 Training compute-optimal large language models . ArXiv prepr...

    2022 arXiv

  8. [17]

    Jing Huang and Diyi Yang. 2023. https://doi.org/10.18653/v1/2023.findings-emnlp.509 Culturally aware natural language inference . In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 7591--7609, Singapore. Association for Computational Linguistics

    2023 doi

  9. [18]

    Sophie Jentzsch and Kristian Kersting. 2023. https://doi.org/10.18653/v1/2023.wassa-1.29 C hat GPT is fun, but it is not funny! humor is still challenging large language models . In Proceedings of the 13th Workshop on Computational Approaches to Subjectivity, Sentiment, & Soci...

    2023 doi

  10. [19]

    Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. 2023. https://arxiv.org/abs/2310.06825 Mistral 7b . ArXiv preprint, abs/2310.06825

    2023 arXiv

  11. [20]

    Kamil Kanclerz, Konrad Karanowski, Julita Bielaniewicz, Marcin Gruza, Piotr Mi kowski, Jan Kocon, and Przemyslaw Kazienko. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.823 PALS : Personalized active learning for subjective tasks in NLP . In Proceedings of the 2023 Confere...

    2023 doi

  12. [21]

    Mikhail Khodak, Nikunj Saunshi, and Kiran Vodrahalli. 2018. https://aclanthology.org/L18-1102 A large self-annotated corpus for sarcasm . In Proceedings of the Eleventh International Conference on Language Resources and Evaluation ( LREC 2018) , Miyazaki, Japan. European Langu...

    2018

  13. [22]

    Tongyoung Kim, Soojin Yoon, Seongku Kang, Jinyoung Yeo, and Dongha Lee. 2024. https://arxiv.org/abs/2408.08686 Sc-rec: Enhancing generative retrieval with self-consistent reranking for sequential recommendation . ArXiv preprint, abs/2408.08686

    2024 arXiv

  14. [23]

    Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. http://papers.nips.cc/paper\_files/paper/2022/hash/8bb0d291acd4acf06ef112099c16f326-Abstract-Conference.html Large language models are zero-shot reasoners . In Advances in Neural Informati...

    2022

  15. [24]

    Aobo Kong, Shiwan Zhao, Hao Chen, Qicheng Li, Yong Qin, Ruiqi Sun, Xin Zhou, Enzhi Wang, and Xiaohang Dong. 2024. https://aclanthology.org/2024.naacl-long.228 Better zero-shot reasoning with role-play prompting . In Proceedings of the 2024 Conference of the North American Chap...

    2024

  16. [25]

    Jia Li, Yuqi Zhu, Yongmin Li, Ge Li, and Zhi Jin. 2024. https://arxiv.org/abs/2410.03234 Showing llm-generated code selectively based on confidence of llms . ArXiv preprint, abs/2410.03234

    2024 arXiv

  17. [26]

    Haipeng Luo, Qingfeng Sun, Can Xu, Pu Zhao, Jianguang Lou, Chongyang Tao, Xiubo Geng, Qingwei Lin, Shifeng Chen, and Dongmei Zhang. 2023. https://arxiv.org/abs/2308.09583 Wizardmath: Empowering mathematical reasoning for large language models via reinforced evol-instruct . ArX...

    2023 arXiv

  18. [27]

    Rui Mao, Guanyi Chen, Xulang Zhang, Frank Guerin, and Erik Cambria. 2024. https://aclanthology.org/2024.lrec-main.693 GPTE val: A survey on assessments of C hat GPT and GPT -4 . In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Re...

    2024

  19. [28]

    J. A. Meaney, Steven Wilson, Luis Chiruzzo, Adam Lopez, and Walid Magdy. 2021. https://doi.org/10.18653/v1/2021.semeval-1.9 S em E val 2021 task 7: H a H ackathon, detecting and rating humor and offense . In Proceedings of the 15th International Workshop on Semantic Evaluation...

    2021 doi

  20. [29]

    Giovanni Messuti, ortensia Amoroso, Ferdinando Napolitano, Mariarosaria Falanga, Paolo Capuano, and Silvia Scarpetta. 2024. https://arxiv.org/abs/2410.06120 Uncertainty estimation via ensembles of deep learning models and dropout layers for seismic traces . ArXiv preprint, abs...

    2024 arXiv

  21. [30]

    Saif Mohammad, Svetlana Kiritchenko, Parinaz Sobhani, Xiaodan Zhu, and Colin Cherry. 2016. https://doi.org/10.18653/v1/S16-1003 S em E val-2016 task 6: Detecting stance in tweets . In Proceedings of the 10th International Workshop on Semantic Evaluation ( S em E val-2016) , pa...

    2016 doi

  22. [31]

    Michael Mohler, Mary Brunson, Bryan Rink, and Marc Tomlinson. 2016. https://aclanthology.org/L16-1668 Introducing the LCC metaphor datasets . In Proceedings of the Tenth International Conference on Language Resources and Evaluation ( LREC '16) , pages 4221--4227, Portoro z , S...

    2016

  23. [32]

    OpenAI. 2022. https://chatgpt.com/ Chat GPT

    2022

  24. [33]

    OpenAI. 2023. https://arxiv.org/abs/2303.08774 GPT -4 technical report . ArXiv preprint, abs/2303.08774

    2023 arXiv

  25. [34]

    OpenAI. 2024. https://openai.com/o1/ Learning to reason with llms

    2024

  26. [35]

    David Premack and Guy Woodruff. 1978. https://doi.org/10.1017/S0140525X00076512 Does the chimpanzee have a theory of mind? Behavioral and Brain Sciences, 1(4):515–526

    1978 doi

  27. [36]

    Ofir Press, Muru Zhang, Sewon Min, Ludwig Schmidt, Noah Smith, and Mike Lewis. 2023. https://doi.org/10.18653/v1/2023.findings-emnlp.378 Measuring and narrowing the compositionality gap in language models . In Findings of the Association for Computational Linguistics: EMNLP 20...

    2023 doi

  28. [37]

    Paul Rottger, Bertie Vidgen, Dirk Hovy, and Janet Pierrehumbert. 2022. https://doi.org/10.18653/v1/2022.naacl-main.13 Two contrasting data annotation paradigms for subjective NLP tasks . In Proceedings of the 2022 Conference of the North American Chapter of the Association for...

    2022 doi

  29. [38]

    Baptiste Roziere, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Tal Remez, J \'e r \'e my Rapin, et al. 2023. https://arxiv.org/abs/2308.12950 Code llama: Open foundation models for code . ArXiv preprint, abs/2308.12950

    2023 arXiv

  30. [39]

    Rachel Rudinger, Jason Naradowsky, Brian Leonard, and Benjamin Van Durme. 2018. https://doi.org/10.18653/v1/N18-2002 Gender bias in coreference resolution . In Proceedings of the 2018 Conference of the North A merican Chapter of the Association for Computational Linguistics: H...

    2018 doi

  31. [40]

    Zayne Sprague, Fangcong Yin, Juan Diego Rodriguez, Dongwei Jiang, Manya Wadhwa, Prasann Singhal, Xinyu Zhao, Xi Ye, Kyle Mahowald, and Greg Durrett. 2024. https://arxiv.org/abs/2409.12183 To cot or not to cot? chain-of-thought helps mainly on math and symbolic reasoning . ArXi...

    2024 arXiv

  32. [41]

    Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R Brown, Adam Santoro, Aditya Gupta, Adri \`a Garriga-Alonso, et al. 2022. https://arxiv.org/abs/2206.04615 Beyond the imitation game: Quantifying and extrapolating the capabil...

    2022 arXiv

  33. [42]

    Huaman Sun, Jiaxin Pei, Minje Choi, and David Jurgens. 2023. https://arxiv.org/abs/2311.09730 Aligning with whom? large language models have gender and racial biases in subjective nlp tasks . ArXiv preprint, abs/2311.09730

    2023 arXiv

  34. [43]

    Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. 2019. https://doi.org/10.18653/v1/N19-1421 C ommonsense QA : A question answering challenge targeting commonsense knowledge . In Proceedings of the 2019 Conference of the North A merican Chapter of the Associa...

    2019 doi

  35. [44]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth \'e e Lacroix, Baptiste Rozi \`e re, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. https://arxiv.org/abs/2302.13971 Llama: Open and efficient foundation language models . ArXiv pr...

    2023 arXiv

  36. [45]

    Lennart Wachowiak and Dagmar Gromann. 2023. https://doi.org/10.18653/v1/2023.acl-long.58 Does GPT -3 grasp metaphors? identifying metaphor mappings with generative language models . In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Vol...

    2023 doi

  37. [46]

    Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, et al. 2024 a . http://dx.doi.org/10.1007/s11704-024-40231-1 A survey on large language model based autonomous agents . Frontiers of Computer Science, 18(6):186345

    2024 doi

  38. [47]

    Xiaolong Wang, Yile Wang, Sijie Cheng, Peng Li, and Yang Liu. 2024 b . https://aclanthology.org/2024.lrec-main.405 DEEM : Dynamic experienced expert modeling for stance detection . In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language...

    2024

  39. [49]

    Xiaolong Wang, Yile Wang, Yuanchi Zhang, Fuwen Luo, Peng Li, Maosong Sun, and Yang Liu. 2024 d . https://doi.org/10.18653/v1/2024.acl-long.844 Reasoning in conversation: Solving subjective tasks through dialogue simulation for large language models . In Proceedings of the 62nd...

    2024 doi

  40. [50]

    Xintao Wang, Yunze Xiao, Jen-tse Huang, Siyu Yuan, Rui Xu, Haoran Guo, Quan Tu, Yaying Fei, Ziang Leng, Wei Wang, Jiangjie Chen, Cheng Li, and Yanghua Xiao. 2024 e . https://doi.org/10.18653/v1/2024.acl-long.102 I n C haracter: Evaluating personality fidelity in role-playing a...

    2024 doi

  41. [51]

    Le, Ed H

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023. https://openreview.net/pdf?id=1PL1NIMMrw Self-consistency improves chain of thought reasoning in language models . In The Eleventh International Conference...

    2023

  42. [52]

    Zhenhailong Wang, Shaoguang Mao, Wenshan Wu, Tao Ge, Furu Wei, and Heng Ji. 2024 f . https://aclanthology.org/2024.naacl-long.15 Unleashing the emergent cognitive synergy in large language models: A task-solving agent through multi-persona self-collaboration . In Proceedings o...

    2024

  43. [53]

    Chi, Quoc V

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. 2022. http://papers.nips.cc/paper\_files/paper/2022/hash/9d5609613524ecf4f15af0f7b31abca4-Abstract-Conference.html Chain-of-thought prompting elicits reasoning...

    2022

  44. [54]

    Wellman, David Cross, and Julanne Watson

    Henry M. Wellman, David Cross, and Julanne Watson. 2001. https://doi.org/https://doi.org/10.1111/1467-8624.00304 Meta-analysis of theory-of-mind development: The truth about false belief . Child Development, 72(3):655--684

    2001

  45. [55]

    Alex Wilf, Sihyun Lee, Paul Pu Liang, and Louis-Philippe Morency. 2024. https://doi.org/10.18653/v1/2024.acl-long.451 Think twice: Perspective-taking improves large language models ' theory-of-mind capabilities . In Proceedings of the 62nd Annual Meeting of the Association for...

    2024 doi

  46. [56]

    Adina Williams, Nikita Nangia, and Samuel Bowman. 2018. https://doi.org/10.18653/v1/N18-1101 A broad-coverage challenge corpus for sentence understanding through inference . In Proceedings of the 2018 Conference of the North A merican Chapter of the Association for Computation...

    2018 doi

  47. [57]

    Zhiheng Xi, Wenxiang Chen, Xin Guo, Wei He, Yiwen Ding, Boyang Hong, Ming Zhang, Junzhe Wang, Senjie Jin, Enyu Zhou, et al. 2023. https://arxiv.org/abs/2309.07864 The rise and potential of large language model based agents: A survey . ArXiv preprint, abs/2309.07864

    2023 arXiv

  48. [58]

    Benfeng Xu, An Yang, Junyang Lin, Quan Wang, Chang Zhou, Yongdong Zhang, and Zhendong Mao. 2023. https://arxiv.org/abs/2305.14688 Expert P rompting: Instructing large language models to be distinguished experts . ArXiv preprint, abs/2305.14688

    2023 arXiv

  49. [59]

    An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, et al. 2024. https://arxiv.org/abs/2407.10671 Qwen2 technical report . ArXiv preprint, abs/2407.10671

    2024 arXiv

  50. [60]

    Zhen Yang, Ming Ding, Qingsong Lv, Zhihuan Jiang, Zehai He, Yuyi Guo, Jinfeng Bai, and Jie Tang. 2023. https://arxiv.org/abs/2309.03241 Gpt can solve mathematical problems without a calculator . ArXiv preprint, abs/2309.03241

    2023 arXiv

  51. [61]

    Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. 2023. http://papers.nips.cc/paper\_files/paper/2023/hash/271db9922b8d1f4dd7aaef84ed5ac703-Abstract-Conference.html Tree of thoughts: Deliberate problem solving with large languag...

    2023

  52. [62]

    Zhangyue Yin, Qiushi Sun, Cheng Chang, Qipeng Guo, Junqi Dai, Xuanjing Huang, and Xipeng Qiu. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.936 Exchange-of-thought: Enhancing large language model capabilities through cross-model communication . In Proceedings of the 2023 C...

    2023 doi

  53. [63]

    Zhuosheng Zhang, Aston Zhang, Mu Li, and Alex Smola. 2023. https://openreview.net/pdf?id=5NTt8GFjUHkr Automatic chain of thought prompting in large language models . In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023 ...

    2023

  54. [64]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before...

  55. [65]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.