Pith. sign in

REVIEW 4 major objections 5 minor 38 references

Concept Navigation and Classification via Open-Source Large Language Model Processing

T0 review · 4 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read An open-source LLM guided by human validation classifies frames and topics at human-coder accuracy.

desk verdict Useful pipeline description, but the validation is circular and the reported F1 scores don't match the precision/recall values, so the performance claims shouldn't be taken at face value. read the letter →

arxiv 2502.04756 v2 pith:MIQIAGQW submitted 2025-02-07 cs.CL cs.AIcs.LG

Maël Kubli This is my paper · ORCID
classification cs.CLcs.AIcs.LG
keywords largelanguagemodelsframeanalysistopicmodelinglatentconstructextractionhuman-in-the-loopvalidationopen-sourceLLMtextclassificationcomputationalsocialscience
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes a hybrid framework in which an open-source large language model (LLaMA 3) detects, summarizes, and generates candidate classes for latent constructs — frames, narratives, and topics — and a human expert then refines those classes before the model classifies the full corpus. The central claim is that this combination of automated summarization with human-in-the-loop validation yields classifications comparable to human coders, while remaining scalable to large text collections. The author applies the framework to European Parliament debates on AI, US newspaper articles on encryption, and the 20 Newsgroups benchmark, reporting accuracy scores close to those of trained human coders, such as 0.79 frame-classification accuracy on parliamentary sentences versus 0.83 human accuracy. If correct, the framework offers a practical route to measuring abstract concepts in political and media text without the cost of fully manual coding. The load-bearing premise is that the human-coded benchmark is an independent standard, though the paper constructs that benchmark from the same LLM-generated class scheme it evaluates.

What carries the argument

The central mechanism is the iterative construct-class generation loop: sentence- or paragraph-level units are passed through a two-step few-shot detection prompt that allows free reasoning before a forced yes-or-no answer, summarized, and then sampled in overlapping batches so the model repeatedly proposes candidate frame or topic classes from the summaries. A human-in-the-loop interface presents each candidate class with example unit IDs and a classification prompt, and the researcher selects, merges, or discards classes before the classification stage. Classification itself is a staged chain-of-thought procedure: the model summarizes each unit, scores its fit to every refined class on a seven-point Likert scale, and then chooses the highest-scoring label, or two labels, in a final decision step. The accumulation and refinement loop is what gives the framework its claimed precision: the human step converts overlapping raw model categories into a stable, conceptually clean codebook, and the Likert-fit plus final-choice design lets the model apply that codebook consistently at scale.

What would settle it

Recruit fresh coders who have never seen the LLM's output, give them a coding scheme built from an external theoretical framework or from a fully independent inductive reading of the texts, and have them label a random sample drawn from the same corpora; if the LLM's agreement with this independent benchmark is substantially below the reported accuracies of 0.84 for frame presence, 0.79 for frame classification, and 0.65 for topic classification, the claim of human-comparable performance would be falsified.

Watch

Extended reading notes

Core claim

The paper's central discovery is that an open-source LLM, when embedded in an iterative generate-then-validate loop, can extract and classify latent constructs at a level comparable to human annotators. In the proposed pipeline, the LLM first produces reasoned binary judgments about whether a frame or topic is present, then summarizes each positive unit, then proposes a small set of candidate classes from overlapping batches of summaries. The researcher prunes, merges, and redefines these classes based on domain knowledge, and the refined class definitions are fed back to the model, which rates each unit's fit to every class on a seven-point scale before a final label-selection step. Across three datasets the model's accuracy is close to the human-coder benchmark, such as 0.84 versus 0.89 for frame presence in parliamentary debates and 0.65 versus 0.68 for topic classification in 20 Newsgroups, and the author reports that class refinement is what most improves the LLM's performance.

Load-bearing premise

The paper's reported accuracy numbers assume that the human-coded validation set is an independent ground truth, but that set was built from the same LLM-generated and author-refined class scheme that the model is later tested against, so agreement may reflect the model matching its own categories rather than an external standard.

Editorial extensions

If this is right

  • If the framework works as claimed, researchers can produce interpretable, domain-specific codebooks for frames and topics from large corpora with far less manual annotation than traditional content analysis.
  • The reported human-comparable accuracy suggests that open-source models, not just proprietary ones, can support reliable text annotation in political communication and media studies.
  • The human-in-the-loop refinement stage becomes a reusable asset: the distilled class definitions and example IDs can be published as a transparent codebook for replication.
  • Because the same pipeline handles sentence-level frames, paragraph-level frames, and full-document topics, a single methodological template could replace several bespoke classification workflows.
  • The framework's dependence on human validation means the practical bottleneck shifts from coding every unit to refining class definitions once, which is what makes large-scale studies feasible.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper, the proposed generate-then-validate loop resembles a general method for concept discovery that could be applied to other latent constructs such as stance, sentiment, or rhetorical strategy, since the same summarization-classification scaffold appears task-agnostic.
  • If the benchmark-independence issue were corrected, the reported accuracy gaps might widen; a fairer test would compare the LLM against an externally derived codebook rather than one it helped generate.
  • A testable extension would be to replace the human refinement step with an active-learning or embedding-based deduplication layer to see how much of the accuracy gain comes specifically from expert judgment versus simple class merging.
  • The paper's own conclusion suggests fine-tuning on small domain-specific sets as the next step; a concrete implementation would freeze the pipeline and vary only the fine-tuning corpus to measure marginal gains.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a hybrid framework for extracting and classifying latent constructs (frames, topics, narratives) from text, combining open-source LLM generation and summarization with human-in-the-loop refinement of the construct classes. The framework is applied to three datasets: EU parliamentary debates on AI, US newspaper articles on encryption, and the 20 Newsgroups corpus. The main empirical claim is that, after human refinement of LLM-generated categories, the LLM classifies frames and topics with accuracy comparable to human coders, with reported F1 scores around 0.68–0.73. The paper includes detailed prompts in an appendix and reports intercoder reliability for human coders.

Significance. If the empirical claims were supported, the framework would be a useful contribution to computational social science: it provides a transparent, open-source pipeline for semi-automated latent construct extraction with human oversight, and the inclusion of full prompts and multiple datasets is a practical strength. The claimed scalability and adaptability to different text types are relevant to ongoing work on LLM-based text annotation. However, the central performance evidence is undermined by a circular validation benchmark and by arithmetic inconsistencies in the reported metrics, so the significance of the contribution as currently evidenced is limited.

major comments (4)
  1. [Section 5.2, Table 2; Section 5.3, Table 3; Section 5.4, Table 4; Section 5.5, Table 5] The reported F1 scores are arithmetically incompatible with the reported precision and recall values. For Table 2, Frame Classification reports F1=0.73 with precision=0.40 and recall=0.74; the harmonic mean is 0.52. For Table 3, Frame Classes reports F1=0.68 with precision=0.61 and recall=0.52; the harmonic mean is 0.56. For Table 4, Topics reports F1=0.68 with precision=0.33 and recall=0.60; the harmonic mean is 0.43. These discrepancies are repeated in Table 5. Since the F1 values are not reproducible from the tabulated precision and recall, the quantitative performance claims cannot be interpreted as standard F1 scores, and all conclusions based on these numbers are unsupported.
  2. [Section 5.1, 'Validation of Classification Approach'; Sections 3.2–3.3] The human benchmark is not independent of the LLM-generated taxonomy being evaluated. The class schemes used by the human coders—the 11 EU frames, 11 encryption frames, and 25 topics—were produced by the LLM and then refined by the author (Sections 3.2–3.3, 5.2, 5.4). The human coders then coded instances using these same classes, and disagreements were discarded via 'filtering for coder agreement' (Section 5.1), leaving 996, 335, and 675 units. Agreement between the LLM and this benchmark therefore largely measures the LLM's consistency with its own proposed categories, not agreement with an external standard. This circularity is load-bearing because every reported accuracy, precision, recall, and F1 value in Sections 5.2–5.5 is computed against this non-independent benchmark.
  3. [Section 5.4, 'Topic Classification'] The 20 Newsgroups evaluation does not validate recovery of the original 20 annotated categories. The paper states that only 11 of the 20 original topics were directly identified, and that additional topics were generated and grouped into 25 LLM-derived topics, including a miscellaneous category. The human coders then coded a sample using this 25-topic scheme. As a result, the reported accuracy and F1 values measure agreement with a human-coded version of the LLM's own topic scheme rather than with the established 20 Newsgroups ground truth, so the claim that the method performs 'comparably to human coders' on the benchmark is not supported.
  4. [Abstract; Section 6] The abstract states that the framework 'guarantees methodological robustness and ensures conceptual precision.' This is an overclaim: the paper presents one demonstration on three datasets, and the validation issues described above mean the framework's robustness and precision are not established. The conclusion more cautiously acknowledges limitations, but the abstract's categorical wording should be revised to reflect the conditional nature of the evidence.
minor comments (5)
  1. [Section 5.1, Table 1] The relationship between 'accuracy' and 'Krippendorff's Alpha' is not defined; for instance, accuracy for frame classification in EU debates is 0.83 but Krippendorff's Alpha is 0.60. Clarify which coding decision (exact match vs. partial match) each metric refers to.
  2. [Section 5.4] The list of 25 topics appears to contain 24 entries plus a miscellaneous class, but the enumeration is not clearly labeled; please verify the count and add a table for readability.
  3. [Section 3.1, Eq. (1)] The attention equation is standard and correct, but it is presented without a citation to the original source in the equation caption; the reference to Vaswani et al. appears in the prose, so consider adding it near the equation.
  4. [Section 3.4, Eq. (3)] The fit evaluation formula uses m as the number of frames evaluated, but the text later allows up to two final labels; clarify whether the formula applies to the per-frame rating step or to the final selection step.
  5. [Appendix S2.2.2] There is a typo in the prompt text: 'Praragraph' should be 'Paragraph'.

Circularity Check

2 steps flagged · score 6.0 of 10

Validation benchmarks are constructed from the LLM's own generated, author-refined class schemes, so the reported agreement measures self-consistency rather than external validity.

  1. fitted input called prediction [Table 4 caption and Section 5.4]
    "Performance metrics (precision, recall, and F1 score) of the LLM in classifying articles into 25 LLM-derived topics. These metrics illustrate the model’s performance relative to human coders in the topic classification task."

    The 20 Newsgroups dataset has an external 20-category gold standard, but the paper replaces it with 25 LLM-derived topics and then reports the LLM's accuracy/F1 'relative to human coders' who coded the sample into those same 25 LLM-derived topics. The target classes are therefore the model's own generated taxonomy; agreement with human coders shows only that the LLM can reproduce its own class scheme, not that it recovers the dataset's independent categories.

  2. self definitional [Sections 5.1-5.2]
    "Initially, LLM identified 83 potential frames related to AI, such as AI Benefits, AI Risks, and AI Ethics. Recognizing overlaps and less relevant frames, a refinement process was conducted through human review... This led to a distilled list of 11 distinct frames: AI Benefits, AI Risks, AI Ethics, AI Regulation, AI Impact, AI Innovation, AI Development, AI Potential, AI Limitations, AI Concerns, and No Frame."

    The human-coded benchmark described in Section 5.1 (996 usable sentences after filtering for coder agreement) is the reference against which the LLM's frame classification is evaluated in Table 2. The class list used for human coding is the author-refined list of frames that the LLM itself generated in Section 5.2. Thus the 'ground truth' for frame classification is defined by the model's own output taxonomy; the reported accuracy and F1 measure consistency with the LLM-generated construct definitions rather than alignment with an independent, externally established frame standard.

full rationale

The central performance claim — that the hybrid LLM/human framework classifies frames and topics comparably to human coders — rests on benchmarks whose category schemes are themselves outputs of the LLM (later author-refined). For the frame tasks, the 11-class scheme is generated by the LLM and refined by the author, then used by human coders, so the LLM is being evaluated against its own taxonomy. For the 20 Newsgroups task, an independent 20-category gold standard was available, but the paper constructs a 25-topic LLM-derived scheme and reports accuracy/F1 relative to human coders coding into that scheme, rather than against the original labels. This does not make the agreement forced (humans could still disagree with the LLM's labels), but it means the reported numbers validate self-consistency, not external construct validity. Separately, the reported F1 values are arithmetically inconsistent with the reported precision and recall (e.g., Table 2 reports F1=0.73 with precision=0.40 and recall=0.74, whose harmonic mean is 0.52), further undermining the quantitative evidence, though that is a numerical reliability issue rather than a circularity step. The paper's comparisons to traditional topic models and its qualitative framework description are not circular; the circularity is localized to the validation design for the quantitative performance claims.

Assumptions & free parameters 6 free parameters · 5 assumptions · 1 invented entities

The central claim rests on several hand-chosen hyperparameters and on the assumption that LLM-generated, author-refined categories are valid. The evaluation benchmark is not independent of the model's own categories, which is a key circularity. No new physical or social entities are introduced beyond the data-derived classification schemes.

free parameters (6)
  • Batch size in class generation = 50 for frames, 100 for topics
    Chosen by the author; affects which summaries are seen together and thus the candidate categories produced.
  • Overlap proportion = 20%
    Chosen by the author to ensure exhaustive coverage; controls redundancy across batches.
  • Maximum candidate classes per call = 9 for frames, 21 for topics
    Arbitrary caps on the number of categories the LLM can output per iteration.
  • Final number of construct classes = 11 (EU frames), 11 (encryption frames), 25 (topics)
    Chosen by the author after human refinement; directly determines the classification task.
  • Likert scale range = 1 to 7
    Chosen for fit scoring; affects score distribution and final label selection.
  • Context window threshold = 4096 tokens
    Imposed by LLaMA 3; led to exclusion of 115 long articles from 20 Newsgroups.
assumptions (5)
  • domain assumption LLM summaries preserve the latent constructs (frames, topics) present in the original text.
    Invoked in Section 3.1 where summarization is used as the basis for all subsequent class generation.
  • domain assumption The 7-point Likert fit score reflects the true conceptual fit of a class to a text segment and is comparable across classes.
    Section 3.4 defines fit as an average of ratings and uses the highest score for labeling.
  • domain assumption Google Translate preserves framing and topic distinctions when translating non-English speeches.
    Section 4.1 states that non-English speeches were translated via the Google Translation API.
  • domain assumption Two human coders agreeing on a label is a valid gold standard for evaluating the LLM.
    Section 5.1 uses filtered coder agreement as the benchmark for all reported accuracies.
  • ad hoc to paper The author's iterative refinement of LLM-generated categories yields theoretically valid, non-overlapping constructs.
    Sections 3.3 and 5.2 describe human-in-the-loop refinement as improving validity, but this is not independently verified.
invented entities (1)
  • LLM-generated and author-refined construct classes (e.g., the 11 EU frames, 11 encryption frames, 25 topics)
    purpose: Classification scheme for labeling text segments
    The categories are produced by the LLM and selected by the author based on the corpus itself; no external theoretical validation or falsifiable prediction is provided. For 20 Newsgroups, some categories match original classes, but the final 25-topic scheme deviates and includes a miscellaneous class.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Concept Navigation and Classification via Open-Source Large Language Model Processing." pith.science (2026). https://pith.science/paper/MIQIAGQW

@misc{pith2026250204756,
  author       = {Pith},
  title        = {Pith review of: Concept Navigation and Classification via Open-Source Large Language Model Processing},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MIQIAGQW}},
  note         = {Machine review of arXiv:2502.04756}
}
read the original abstract

This paper presents a novel methodological framework for detecting and classifying latent constructs, including frames, narratives, and topics, from textual data using Open-Source Large Language Models (LLMs). The proposed hybrid approach combines automated summarization with human-in-the-loop validation to enhance the accuracy and interpretability of construct identification. By employing iterative sampling coupled with expert refinement, the framework guarantees methodological robustness and ensures conceptual precision. Applied to diverse data sets, including AI policy debates, newspaper articles on encryption, and the 20 Newsgroups data set, this approach demonstrates its versatility in systematically analyzing complex political discourses, media framing, and topic classification tasks.

Figures

Figures reproduced from arXiv: 2502.04756 by the authors.

Figure 1
Figure 1. Construct Class Generation Framework The proposed methodological framework leverages open-source LLMs, such as the LLaMA 3 model, to extract and classify latent constructs such as frames, narratives, and topics from large textual corpora. The methodology is designed to address the limitations of traditional topic modeling methods, such as Latent Dirichlet Allocation (LDA) and Structural Topic Models (STM), which oft… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

38 extracted references · 24 canonical work pages

  1. [1]

    \ Zhai, C

    aggarwal2012survey APACrefauthors Aggarwal, C C. \ Zhai, C. APACrefauthors \ 2012 . A survey of text clustering algorithms A survey of text clustering algorithms . Mining text data 77--128

    work page 2012

  2. [2]

    , Kubli, M

    alizadeh2023open APACrefauthors Alizadeh, M. , Kubli, M. , Samei, Z. , Dehghani, S. , Bermeo, J D. , Korobeynikova, M. \ Gilardi, F. APACrefauthors \ 2023 . Open-Source LLMs for Text Annotation: A Practical Guide for Model Setting and Fine-Tuning Open-source llms for text annotation: A practical guide for model setting and fine-tuning . arXiv preprint arX...

    arXiv 2023

  3. [3]

    , Ng, A Y

    blei2003latent APACrefauthors Blei, D M. , Ng, A Y. \ Jordan, M I. APACrefauthors \ 2003 . Latent dirichlet allocation Latent dirichlet allocation . Journal of machine Learning research 3 Jan 993--1022

    2003

  4. [4]

    borgatti2009social APACrefauthors Borgatti, S P. \ Li, X. APACrefauthors \ 2009 . On social network analysis in a supply chain context On social network analysis in a supply chain context . Journal of supply chain management 45 2 5--22

    work page 2009

  5. [5]

    , Mann, B

    brown2020language APACrefauthors Brown, T. , Mann, B. , Ryder, N. , Subbiah, M. , Kaplan, J D. , Dhariwal, P. others APACrefauthors \ 2020 . Language models are few-shot learners Language models are few-shot learners . Advances in neural information processing systems 33 1877--1901

    2020

  6. [6]

    , Bollenbacher, J

    chew2023llm APACrefauthors Chew, R. , Bollenbacher, J. , Wenger, M. , Speer, J. \ Kim, A. APACrefauthors \ 2023 . LLM-assisted content analysis: Using large language models to support deductive coding Llm-assisted content analysis: Using large language models to support deductive coding . arXiv preprint arXiv:2306.14924

    arXiv 2023

  7. [7]

    , Dunn, A

    dagdelen2024structured APACrefauthors Dagdelen, J. , Dunn, A. , Lee, S. , Walker, N. , Rosen, A S. , Ceder, G. Jain, A. APACrefauthors \ 2024 . Structured information extraction from scientific text with large language models Structured information extraction from scientific text with large language models . Nature Communications 15 1 1418

    work page 2024

  8. [8]

    , Chang, M W

    devlin2018bert APACrefauthors Devlin, J. , Chang, M W. , Lee, K. \ Toutanova, K. APACrefauthors \ 2018 . Bert: Pre-training of deep bidirectional transformers for language understanding Bert: Pre-training of deep bidirectional transformers for language understanding . arXiv preprint arXiv:1810.04805

    arXiv 2018

Show all 38 references
  1. [9]

    , Heidenreich, T

    eisele2023capturing APACrefauthors Eisele, O. , Heidenreich, T. , Litvyak, O. \ Boomgaarden, H G. APACrefauthors \ 2023 . Capturing a news frame--comparing machine-learning approaches to frame analysis with different degrees of supervision Capturing a news frame--comparing mac...

    2023

  2. [10]

    APACrefauthors \ 1993

    entman1993framing APACrefauthors Entman, R M. APACrefauthors \ 1993 . Framing: Toward clarification of a fractured paradigm Framing: Toward clarification of a fractured paradigm . Journal of communication 43 4 51--58

    1993

  3. [11]

    APACrefauthors \ 2007

    entman2007framing APACrefauthors Entman, R M. APACrefauthors \ 2007 . Framing bias: Media in the distribution of power Framing bias: Media in the distribution of power . Journal of communication 57 1 163--173

    2007

  4. [12]

    , Alizadeh, M

    gilardi2023chatgpt APACrefauthors Gilardi, F. , Alizadeh, M. \ Kubli, M. APACrefauthors \ 2023 . ChatGPT outperforms crowd workers for text-annotation tasks Chatgpt outperforms crowd workers for text-annotation tasks . Proceedings of the National Academy of Sciences 120 30 e2305016120

    2023

  5. [13]

    APACrefauthors \ 2022

    grootendorst2022bertopic APACrefauthors Grootendorst, M. APACrefauthors \ 2022 . BERTopic: Neural topic modeling with a class-based TF-IDF procedure Bertopic: Neural topic modeling with a class-based tf-idf procedure . arXiv preprint arXiv:2203.05794

    2022 arXiv

  6. [14]

    huang2023advancing APACrefauthors Huang, Y. , Xu, J. , Lai, J. , Jiang, Z. , Chen, T. , Li, Z. others APACrefauthors \ 2023 . Advancing transformer architecture in long-context large language models: A comprehensive survey Advancing transformer architecture in long-context lar...

    2023 arXiv

  7. [15]

    APACrefauthors \ 2010

    jain2010data APACrefauthors Jain, A K. APACrefauthors \ 2010 . Data clustering: 50 years beyond K-means Data clustering: 50 years beyond k-means . Pattern recognition letters 31 8 651--666

    2010

  8. [16]

    , Murty, M N

    jain1999data APACrefauthors Jain, A K. , Murty, M N. \ Flynn, P J. APACrefauthors \ 1999 . Data clustering: a review Data clustering: a review . ACM computing surveys (CSUR) 31 3 264--323

    1999

  9. [17]

    , Schuster, M

    johnson2017google APACrefauthors Johnson, M. , Schuster, M. , Le, Q V. , Krikun, M. , Wu, Y. , Chen, Z. Dean, J. APACrefauthors \ 2017 . Google’s multilingual neural machine translation system: Enabling zero-shot translation Google’s multilingual neural machine translation sys...

    2017

  10. [18]

    , Ramalingam, S

    kartchner2023zero APACrefauthors Kartchner, D. , Ramalingam, S. , Al-Hussaini, I. , Kronick, O. \ Mitchell, C. APACrefauthors \ 2023 . Zero-Shot Information Extraction for Clinical Meta-Analysis using Large Language Models Zero-shot information extraction for clinical meta-ana...

    2023

  11. [19]

    , Gu, S S

    kojima2022large APACrefauthors Kojima, T. , Gu, S S. , Reid, M. , Matsuo, Y. \ Iwasawa, Y. APACrefauthors \ 2022 . Large language models are zero-shot reasoners Large language models are zero-shot reasoners . Advances in neural information processing systems 35 22199--22213

    2022

  12. [20]

    , Teoh, J

    lam2024concept APACrefauthors Lam, M S. , Teoh, J. , Landay, J A. , Heer, J. \ Bernstein, M S. APACrefauthors \ 2024 . Concept Induction: Analyzing Unstructured Text with High-Level Concepts Using LLooM Concept induction: Analyzing unstructured text with high-level concepts us...

    2024

  13. [21]

    , Perez, E

    lewis2020retrieval APACrefauthors Lewis, P. , Perez, E. , Piktus, A. , Petroni, F. , Karpukhin, V. , Goyal, N. Douwe, K. APACrefauthors \ 2020 . Retrieval-augmented generation for knowledge-intensive nlp tasks Retrieval-augmented generation for knowledge-intensive nlp tasks . ...

    2020

  14. [22]

    , Yao, S

    mccoy2024embers APACrefauthors McCoy, R T. , Yao, S. , Friedman, D. , Hardy, M D. \ Griffiths, T L. APACrefauthors \ 2024 . Embers of autoregression show how large language models are shaped by the problem they are trained to solve Embers of autoregression show how large langu...

    2024

  15. [23]

    APACrefauthors \ 2014

    milojevic2014network APACrefauthors Milojevi \'c , S. APACrefauthors \ 2014 . Network analysis and indicators Network analysis and indicators . Measuring scholarly impact: Methods and practice Measuring scholarly impact: Methods and practice \ ( \ 57--82). Springer

    2014

  16. [24]

    , Dong, C

    mu2024large APACrefauthors Mu, Y. , Dong, C. , Bontcheva, K. \ Song, X. APACrefauthors \ 2024 . Large language models offer an alternative to the traditional approach of topic modelling Large language models offer an alternative to the traditional approach of topic modelling ....

    2024 arXiv

  17. [25]

    \ McKeown, K

    nenkova2012survey APACrefauthors Nenkova, A. \ McKeown, K. APACrefauthors \ 2012 . A survey of text summarization techniques A survey of text summarization techniques . Mining text data 43--76

    2012

  18. [26]

    pilault2020extractive APACrefauthors Pilault, J. , Li, R. , Subramanian, S. \ Pal, C. APACrefauthors \ 2020 . On extractive and abstractive neural document summarization with transformer language models On extractive and abstractive neural document summarization with transform...

    2020

  19. [27]

    radford2019language APACrefauthors Radford, A. , Wu, J. , Child, R. , Luan, D. , Amodei, D. \ Sutskever, I. APACrefauthors \ 2019 . Language models are unsupervised multitask learners Language models are unsupervised multitask learners . OpenAI blog 1 8 9

    2019

  20. [28]

    , Stewart, B M

    roberts2014structural APACrefauthors Roberts, M E. , Stewart, B M. , Tingley, D. , Lucas, C. , Leder-Luis, J. , Gadarian, S K. Rand, D G. APACrefauthors \ 2014 . Structural topic models for open-ended survey responses Structural topic models for open-ended survey responses . A...

    2014

  21. [29]

    , Bentley, L R

    tai2024examination APACrefauthors Tai, R H. , Bentley, L R. , Xia, X. , Sitt, J M. , Fankhauser, S C. , Chicas-Mosier, A M. \ Monteith, B G. APACrefauthors \ 2024 . An examination of the use of large language models to aid analysis of textual data An examination of the use of ...

    2024

  22. [30]

    , Wu, C K

    tam2024let APACrefauthors Tam, Z R. , Wu, C K. , Tsai, Y L. , Lin, C Y. , Lee, H y. \ Chen, Y N. APACrefauthors \ 2024 . Let me speak freely? a study on the impact of format restrictions on performance of large language models Let me speak freely? a study on the impact of form...

    2024 arXiv

  23. [31]

    APACrefauthors \ 2023

    tornberg2023chatgpt APACrefauthors T \"o rnberg, P. APACrefauthors \ 2023 . Chatgpt-4 outperforms experts and crowd workers in annotating political twitter messages with zero-shot learning Chatgpt-4 outperforms experts and crowd workers in annotating political twitter messages...

    2023 arXiv

  24. [32]

    , Shazeer, N

    vaswani2017attention APACrefauthors Vaswani, A. , Shazeer, N. , Parmar, N. , Uszkoreit, J. , Jones, L. , Gomez, A N. Polosukhin, I. APACrefauthors \ 2017 . Attention is all you need Attention is all you need . Advances in neural information processing systems 30

    2017

  25. [33]

    \ Dubey, H

    vatsal2024survey APACrefauthors Vatsal, S. \ Dubey, H. APACrefauthors \ 2024 . A survey of prompt engineering methods in large language models for different nlp tasks A survey of prompt engineering methods in large language models for different nlp tasks . arXiv preprint arXiv...

    2024 arXiv

  26. [34]

    , Chen, W

    xu2024large APACrefauthors Xu, D. , Chen, W. , Peng, W. , Zhang, C. , Xu, T. , Zhao, X. Chen, E. APACrefauthors \ 2024 . Large language models for generative information extraction: A survey Large language models for generative information extraction: A survey . Frontiers of C...

    2024

  27. [35]

    , Jin, H

    yang2024harnessing APACrefauthors Yang, J. , Jin, H. , Tang, R. , Han, X. , Feng, Q. , Jiang, H. Hu, X. APACrefauthors \ 2024 . Harnessing the power of llms in practice: A survey on chatgpt and beyond Harnessing the power of llms in practice: A survey on chatgpt and beyond . A...

    2024

  28. [36]

    , Guruganesh, G

    zaheer2020big APACrefauthors Zaheer, M. , Guruganesh, G. , Dubey, K A. , Ainslie, J. , Alberti, C. , Ontanon, S. Ahmed, A. APACrefauthors \ 2020 . Big bird: Transformers for longer sequences Big bird: Transformers for longer sequences . Advances in neural information processin...

    2020

  29. [37]

    , Ladhak, F

    zhang2024benchmarking APACrefauthors Zhang, T. , Ladhak, F. , Durmus, E. , Liang, P. , McKeown, K. \ Hashimoto, T B. APACrefauthors \ 2024 . Benchmarking large language models for news summarization Benchmarking large language models for news summarization . Transactions of th...

    2024

  30. [38]

    , Yuan, H

    zhu2023large APACrefauthors Zhu, Y. , Yuan, H. , Wang, S. , Liu, J. , Liu, W. , Deng, C. Wen, J R. APACrefauthors \ 2023 . Large language models for information retrieval: A survey Large language models for information retrieval: A survey . arXiv preprint arXiv:2308.07107

    2023

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.