TACOMORE: Exploring a replicable prompting protocol for LLM-assisted corpus analysis

Bingru Li; Han Wang; Nicholas Groom

REVIEW 3 major objections 4 minor 144 references

TACOMORE: Exploring a replicable prompting protocol for LLM-assisted corpus analysis

T0 review · 3 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A five-element prompting protocol called TACOMORE moves LLM corpus analysis from baseline failure to human-rated accuracy, the authors claim.

desk verdict A useful domain-specific prompting protocol, but the evaluation has a post-hoc exclusion problem and overclaims. read the letter →

arxiv 2412.10139 v2 pith:ZCPRGNHR submitted 2024-12-13 cs.CL

Bingru Li , Han Wang , Nicholas Groom This is my paper · ORCID

classification cs.CL

keywords promptengineeringlargelanguagemodelscorpuslinguisticsdiscourseanalysiskeywordcollocationconcordancereplicability

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

LLMs are good at counting words but not at interpreting them, and earlier attempts to use them for corpus-based discourse analysis produced unreliable and unreproducible results. The paper argues that the failure is largely a prompting problem: when the prompt specifies a role, a task definition, step-by-step procedures, the co-text around each target word, and a strict output format, the same models produce acceptable keyword, collocate, and concordance analyses. The authors test this on a public corpus of COVID-19 research abstracts with three LLMs and report human-rated gains in accuracy, ethicality, reasoning, and reproducibility, with an ablation study showing scores climbing as each prompt element is added. If the claim holds, structured prompting protocols could give corpus linguists a standard, transparent way to delegate parts of qualitative analysis to LLMs while keeping human oversight.

What carries the argument

TACOMORE is the paper's central object: a prompting protocol that encodes the standard operating procedure of corpus-based discourse analysis into a fixed prompt template. Its five compulsory elements are Role Description (assign the model the persona of a corpus linguist), Task Definition (state the goal, data, and background), Task Procedures (break the task into numbered human-like steps, drawing on chain-of-thought prompting), Contextual Information (attach the same co-text—concordance lines and original text—that human analysts would see), and Output Format (compel exhaustive, machine-checkable output through delimiters and an example). The framework's work is to replace ad-hoc single-shot instructions with a replicable protocol, and the ablation isolates the contribution of each element by adding them one at a time.

What would settle it

Have two independent teams, blind to the paper's scores, re-score the recorded model outputs and run the same prompts again against a pre-registered rubric that separates 'followed the format' from 'interpreted the corpus correctly'; if format-compliant outputs still score high while interpretive quality (for example, concordance bias judgments against an expert gold standard) does not improve over baseline, the central claim fails.

Watch

Extended reading notes

Core claim

The paper's central claim is that TACOMORE—a framework built on four principles (Task, Context, Model, Reproducibility) and five prompt elements (Role Description, Task Definition, Task Procedures, Contextual Information, Output Format)—turns LLMs from unreliable statistical predictors into effective assistants for corpus-based discourse analysis. On three tasks (keyword theme grouping, collocate analysis of the word china, and bias detection in concordances of 'China virus' and 'Chinese virus'), GPT-4o and Gemini-1.5-Pro receive roughly 4-out-of-5 human Likert scores across accuracy, ethicality, reasoning, and reproducibility, while the smaller Gemini-1.5-Flash scores lower. The keyword ablation shows the total score rising monotonically from 5/20 for a baseline prompt to 16/20 when all five elements are present, which the authors read as direct evidence that each element contributes. They describe this as the first time LLM outputs in these tasks were satisfying. The paper also reports that hallucination—fabricated or mis-cited corpus context—persists even with the full framework, so the method is proposed as a complement to, not a replacement for, human validation.

Load-bearing premise

The load-bearing premise is that the researchers' own 5-point Likert rubric, scored by two human raters without reported blinding or independence, captures genuine analytical quality rather than rewarding format compliance or matching rater expectations.

Editorial extensions

If this is right

If the protocol works as claimed, corpus linguists could adopt TACOMORE as a documented default for LLM-assisted keyword, collocate, and concordance analysis, making AI involvement in qualitative work auditable step by step.
The monotonic ablation suggests each prompt element earns its place, so omitting co-text or output format forfeits measurable accuracy and completeness, arguing for always pairing LLMs with the human-in-the-loop context.
The persistence of hallucination implies that even with strong prompts, model outputs must be checked against the original corpus, so TACOMORE would complement rather than eliminate manual verification in discourse studies.
The reproducibility principle—stable and reasonable, not identical, outputs—gives the field a practical target for replicated LLM use, since identical outputs across runs are not guaranteed.
The evaluation rubric of Accuracy, Ethicality, Reasoning, and Reproducibility may transfer to other qualitative analysis tasks, giving researchers a common scoring language.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The authors do not test whether two independent teams prompting without seeing the paper's exact templates would converge on the same TACOMORE-style prompts; a multi-lab replication would clarify whether the framework's four principles alone reproduce the reported gains.
The claimed improvement may partly reflect formatting compliance: the Output Format element forces exhaustive coverage (all 83 keywords), which mechanical completeness checks would reward on Accuracy and Reproducibility even if interpretive depth is unchanged, and the paper's Likert scale cannot fully separate these.
The concordance bias task could become a quantitative benchmark by measuring the two large models' judgments against an expert gold-standard annotation set rather than the two raters' consensus, giving a testable extension beyond the current design.
If the framework generalizes, the same five elements could be tuned for other qualitative corpus tasks such as stance detection or metaphor analysis, with the COVID-19 abstract corpus serving as a reproducibility benchmark.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The paper has a genuinely useful core idea: TACOMORE, a structured prompting framework for LLM-assisted corpus analysis, built from five elements (role, task definition, procedures, contextual information, output format). The most distinctive piece is the contextual co-text principle—giving the LLM the same concordance lines a human analyst would see. That is a simple, actionable insight that could help many corpus linguists. The ablation study, though small, shows monotone improvement as elements are added, which is a nice practical demonstration. The paper also reports high inter-rater reliability (Krippendorff's alpha 0.927) and includes full prompts and many raw outputs in the appendix, which supports replication.

The soft spots are real but not fatal. The evaluation rests on a self-defined 5-point Likert rubric applied by two raters with no reported blinding. That is a moderate concern; the high inter-rater agreement helps, but it still leaves room for expectation effects. More serious is the post-hoc exclusion of concordance line 7 in Section 4.2. The authors say the line is confusing and "not taken into consideration." That is outcome-based data deletion. It directly inflates the Accuracy and Reproducibility scores for the concordance task and underpins the claim that both models "successfully detect" bias. Re-including that line could change the qualitative conclusion for that task, and the paper should report both the unstable answers and a clear rule for exclusions. The "first time ever" claim in the introduction is unsupported and should be removed. And for a paper centered on reproducibility, the lack of a released dataset or code is a gap, though the appendix partially compensates.

Overall, the framework is plausible and worth testing, but the current evidence is suggestive rather than conclusive. The central idea—especially the co-text principle—deserves independent validation.

Recommendation: send it to peer review, but expect heavy revision. The manuscript should fix the line-7 exclusion, describe the rating procedure in enough detail to allow independent replication, tone down the overclaims, and ideally release the prompts and a minimal corpus subset. who is this for: corpus linguists, social scientists using LLMs for qualitative coding, and prompt-engineering researchers. A serious referee can help this become a useful paper.

Referee Report

3 major / 4 minor

Summary. The paper proposes TACOMORE, a prompting framework for LLM-assisted corpus-based discourse analysis, built on four principles (Task, Context, Model, Reproducibility) and five prompt elements (Role Description, Task Definition, Task Procedures, Contextual Information, Output Format). The authors apply it to keyword, collocate, and concordance analysis on a corpus of COVID-19 research abstracts using GPT-4o, Gemini-1.5-Pro, and Gemini-1.5-Flash, and score the outputs on a self-defined 5-point Likert rubric (Accuracy, Ethicality, Reasoning, Reproducibility). They report that TACOMORE improves accuracy and replicability, with a monotonic improvement in an ablation study, while acknowledging persistent hallucination in model outputs.

Significance. The paper addresses a timely and practical problem: making LLM outputs more replicable and interpretable for qualitative corpus analysis. Its concrete strengths are the open corpus, the detailed TACOMORE design, and the unusually complete appendix with full prompts and raw outputs for all models and ablation conditions, which is a genuine reproducibility resource. If the evaluation were adequately controlled, this would be a useful methodological contribution for corpus linguistics. However, the current evidence is not conclusive: the outcome measure is a non-validated in-house rubric applied without reported blinding or statistical analysis, and one task excludes the only ambiguous data point from evaluation.

major comments (3)

[4.2, Concordance Analysis] The manuscript excludes the only unstable concordance line from the evaluation. Section 4.2 states that "Only concordance line 7 is confusing" and "this case is not taken into consideration in evaluation." Because Reproducibility is defined as stable output across trials (Section 3.4), dropping the one line in which the two large models disagreed (GPT-4o marked [Yes], Gemini-1.5-Pro marked [No]) mechanically raises the reported Reproducibility and Accuracy scores in Table 3. The claim that "both models could successfully detect whether a concordance is biased" is therefore supported only for 19 of 20 lines. Re-including line 7 may lower the scores and could change the qualitative conclusion; the authors should report full-set results and treat line 7 as an object of analysis rather than an exclusion.
[3.4 and 4.1] The evaluation rests entirely on a self-defined 5-point Likert rubric with no scoring anchors, no evidence of blinding, no report of per-rater or per-item scores, and no statistical tests or confidence intervals. The two raters are described only as "two human experts," and the single reported Krippendorff's alpha does not establish that the scores are valid measures of analytical quality rather than prompt-format compliance. Because every central claim—"satisfying results," the effectiveness of TACOMORE, and the monotonicity of the ablation—is derived from these scores, the paper needs a pre-specified coding protocol, blind and independent rating, and at least per-item agreement and error bars; ideally, the rubric should be validated against expert human analysis of the same corpus.
[4.3, Ablation Study] Table 4 is presented as evidence that "the score climbs up every time a new element is added to the prompt," but the design is a single fixed-order cumulative ablation on one model with one run per condition. The observed monotonic increase does not identify which element is responsible, and the baseline is an extremely terse prompt that may simply benefit from longer instructions. Furthermore, since the Reproducibility metric rewards stable output and the Output Format element explicitly constrains output structure, part of the final score gain is built into the prompt by construction. A factorial ablation or length-matched control prompts, together with variance estimates across repeated runs, are needed to support the framework-attribution claim.

minor comments (4)

[Throughout] There are several typos and infelicities: "Gemimi-1.5-Pro" in Section 1, "air and ethical" in Section 3.4, "Bellow" in the Output Format prompt, "attach attach" in Section 4.1, and "Inte-rater Reliability" in Section 4.1.
[Appendix A.2] One ablation condition is labeled "B. + R. D. + T. D. + T. D + C. I."; this should presumably be "B. + R. D. + T. D. + T. P. + C. I." to match Table 4.
[4.1, Target LLMs] The concordance experiments were run on a third-party platform while the other tasks used APIs; the possible effect of the platform on output variability is not discussed.
[3.4, Figure 3] Figure 3 is referenced but its content is not described in the text, so readers cannot see the rubric details beyond the four one-sentence definitions in Section 3.4.

Circularity Check

1 steps flagged · score 4.0 of 10

The ablation's Output Format gain is partly self-referential: the Accuracy/Ethicality rubric rewards complete instruction-following, which is exactly what the Output Format element is added to enforce.

self definitional [Section 4.3 (Ablation Study), with metric definitions in Section 3.4 and Output Format in Section 3.3]
"Accuracy (Acc) ... The LLM performs well in producing desired results, following instructions and yielding high-quality output. Ethicality (Eth) ... The LLM correctly retrieves information from the prompt and data, without any fabrication or omission ... However, all previous tryouts fail to exhaust all keywords when assigning themes, and to counter this issue, we add the element Output Format (O. F.) to compel the LLM to count each and every keyword provided. As shown in Table 4, the score climbs up every time a new element is added to the prompt, illustrating the effectiveness of TACOMORE."

The Accuracy metric is defined as following instructions and yielding high-quality output, and Ethicality is defined as working without fabrication or omission. The Output Format element is added specifically to force complete enumeration of all 83 keywords. Therefore, the observed increase in Accuracy and Ethicality when Output Format is added largely measures whether the model complied with the prompt's own completeness instruction, scored by a rubric that defines such compliance as quality. The ablation's monotonic 'effectiveness' claim is thus partly self-referential: the O.F. component and the Acc/Eth rubric reward the same behavior. The gains from Role, Task Definition, Task Procedures, and Contextual Information are not tautological, so the circularity is partial rather than total.

full rationale

The evaluation is not built on a formal derivation, so the circularity check turns on the experimental design. The main evidence for TACOMORE's effectiveness is the ablation, where the final element (Output Format) is introduced to make the LLM count and list every keyword, while the Accuracy and Ethicality metrics explicitly reward instruction-following and absence of omission. A score gain on those sub-scales from adding Output Format is therefore partly a compliance measure rather than an independent demonstration of improved corpus-analytic quality. This is a real but partial circularity: the Role, Task Definition, Task Procedures, and Contextual Information stages of the ablation improve scores in ways not baked into the metric definitions, so the central claim retains independent empirical content. There is no load-bearing self-citation chain: the baseline (Curry et al., 2024), the corpus tool (AntConc), and the SOP descriptions are external; the two human raters' high agreement (Krippendorff's alpha 0.927) is reported but blinding and independence are not documented, which is a validity concern rather than a circularity step. I also flag Section 4.2's post-hoc decision to exclude concordance line 7 ('this case is not taken into consideration in evaluation'); that is a data-selection validity issue and is not counted as circularity here. Overall, the central claim is partially circular in one of its supporting ablation steps, but not forced entirely by definition, giving a score of 4.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

There are no fitted mathematical parameters or invented physical entities. The central empirical claim rests on hand-chosen design choices (keyword list size, sample sizes, exclusion of one discordant concordance line) and on domain assumptions about corpus methods and Likert scoring. The most consequential choice is that the authors' own rubric is the sole measure of success.

free parameters (4)

Concordance lines sampled per keyword/collocate = 50 random lines per keyword/collocate
Hand-selected sample size, not justified by power analysis; it determines how much context the LLM receives and therefore affects all task scores.
Bias-task concordance set size = 20 lines
Small set of all lines containing 'China virus' or 'Chinese virus'; enough for a pilot, weak for a general bias-detection claim.
Retained keyword list size = 83
Manual intersection and curation of two top-100 keyword lists; the 'concurrent, meaningful' criterion (Section 4.1) is not operationalized, so the input set is not exactly reproducible.
Exclusion of concordance line 7 = 1 line dropped
Post hoc removal of the only disagreeing line in the bias task, which improves apparent inter-model agreement and accuracy.

assumptions (5)

domain assumption Standard corpus linguistics task definitions (keyness, collocates L5-R5, concordance windows) are the right targets for LLM evaluation.
Section 3.1 defines the three tasks following Baker and Gries; the validity of these procedures as benchmarks is taken for granted.
domain assumption Human Likert ratings of Accuracy, Ethicality, Reasoning, and Reproducibility are a valid measure of LLM performance.
Section 3.4 introduces the rubric with one-sentence definitions and no external validation or blinding protocol.
domain assumption The CDC COVID-19 abstract corpus is representative of COVID-19 discourse and was cleaned adequately.
Section 3.2; only langdetect-based language filtering is described, with no corpus representativeness argument.
domain assumption API settings and greedy decoding produce outputs stable enough for reproducibility measurements.
Section 4.1; the concordance task used a third-party platform with uncontrolled sampling, weakening this assumption.
ad hoc to paper The minimal baseline prompt from Curry et al. (2024) is a fair representative of ad-hoc prompting.
Section 4.3; the baseline is one short prompt, so the ablation may overstate TACOMORE's benefit relative to other structured methods.

how reviews work

0 comments

Cite this review

Pith. "Pith review of TACOMORE: Exploring a replicable prompting protocol for LLM-assisted corpus analysis." pith.science (2026). https://pith.science/paper/ZCPRGNHR

@misc{pith2026241210139,
  author       = {Pith},
  title        = {Pith review of: TACOMORE: Exploring a replicable prompting protocol for LLM-assisted corpus analysis},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZCPRGNHR}},
  note         = {Machine review of arXiv:2412.10139}
}

read the original abstract

As corpus linguistics continues to scale, researchers are facing a growing methodological bottleneck: while computational tools can easily count billions of words, the qualitative interpretation of these data remains a slow and labor-intensive human task. Large Language Models (LLMs) offer a promising way to automate this process, yet their integration into the field is often hindered by concerns over black-box unpredictability and a lack of replicability. This study introduces TACOMORE, a structured prompting framework designed to transform ad-hoc AI interactions into a standardized linguistic protocol. Built upon four foundational principles (Task, Context, Model, and Replicability), the framework guides LLMs to move beyond generic probability prediction to anchoring their reasoning in the specific co-occurrence patterns of a target corpus. We applied this framework to three core corpus tasks, i.e., the analysis of keywords, collocates, and concordances, using an open corpus of COVID-19 research abstracts. After testing three LLMs, we found that while structured prompting improves accuracy and replicability, inherent limitations regarding hallucination persist. This research offers a critical lens into the role of LLMs in corpus linguistics, highlighting their potential as complementary tools while emphasizing the irreplaceable role of human validation.

Figures

Figures reproduced from arXiv: 2412.10139 by the authors.

**Figure 2.** Task. Compared with an under-specified goal for the LLMs to stick to in general prompting engineering, the Task element in TACOMORE requires description of a task as detailed as possible and breaking it down to steps to allow our assistants to think and react. For qualitative corpus linguistics tasks, the first thing to determine is the definition and standardized process of a given task. Users are supposed to tell… view at source ↗

**Figure 3.** Evaluation Metrics In order to evaluate the effectiveness of TACOMORE, we also propose the metrics of evaluation in [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

Figures from the paper (4 more)

**Figure 5.** Figure 5: An example of theme categorization by GPT [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]

**Figure 4.** Figure 4: Excerpt of the prompt for keyword analysis. [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

**Figure 8.** Figure 8: An example of concordance analysis by Gemini-1.5-Pro. 4.3 Ablation Study In this section, we conduct ablation study for all three tasks with Gemini-1.5-Pro as our exemplar (see Section A.2 for all results). We extend evaluation and discussion on keyword analysis, whic…

**Figure 7.** Figure 7: Excerpt of the prompt for concordance analy [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

144 extracted references · 76 canonical work pages

[1]

**covid**: Short for COVID-19, the disease caused by the SARS-CoV-2 virus
[2]

Also refers to individuals in other medical contexts

**patients**: Individuals diagnosed with or suspected of having COVID-19. Also refers to individuals in other medical contexts. 45
[3]

**cov**: Abbreviation for Coronavirus, specifically can refer to SARS-CoV , MERS-CoV , or SARS-CoV-2
[4]

Often used in compari- son to COVID-19

**sars**: Severe Acute Respiratory Syn- drome, specifically refers to the disease caused by SARS-CoV in 2002-2003. Often used in compari- son to COVID-19

2002
[5]

**pandemic**: Refers to the global outbreak of COVID-19
[6]

Wuhan" (4),

Thread of thought unraveling chaotic contexts. arXiv preprint arXiv:2311.08734. A Appendix A.1 Discourse Analysis Tasks with TACOMORE Keyword Analysis. Prompt # Role Description You are an expert in Corpus Linguistics doing keyword analysis. # Task Definition I would give you a list of keywords from a cor- pus of the abstracts of COVID-19 research article...

arXiv 2019
[7]

**disease**: Generally refers to COVID-19 but can also refer to other medical conditions
[8]

**study**: Refers to research investigations related to COVID-19, including clinical trials, ob- servational studies, and epidemiological studies

Show all 144 references

[9]

**clinical**: Relates to the medical as- pects of COVID-19, including symptoms, diag- nosis, treatment, and patient management
[10]

**infection**: The state of being infected with the SARS-CoV-2 virus
[11]

**health**: Relates to the broader health im- pacts of COVID-19, including public health, health- care systems, and mental health
[12]

**registered**: Indicates if a clinical trial or interventional study has been registered with a recognized registry like ClinicalTrials.gov
[13]

**coronavirus**: A family of viruses that includes SARS-CoV , MERS-CoV , and SARS-CoV- 2
[14]

**virus**: Refers to SARS-CoV-2, the virus responsible for COVID-19
[15]

**severe**: Describes cases of COVID-19 with serious clinical presentations, often requiring hospitalization and intensive care
[16]

**cases**: Instances of confirmed COVID- 19 infections
[17]

**research**: Activities related to scientific investigations on COVID-19
[18]

**and**: Conjunction, carries no specific semantic meaning in this context
[19]

**Summary 2:** The corpus highlights China’s response to the COVID-19 pandemic, including its national policies and research contributions. * Collocates such as

**first emerged in China** and rapidly spread in the world causing a pandemic." **Summary 2:** The corpus highlights China’s response to the COVID-19 pandemic, including its national policies and research contributions. * Collocates such as "national" (22), "control" (85), "re...

2019
[20]

**respiratory**: Relates to the respiratory system and complications arising from COVID- 19, like pneumonia and acute respiratory distress syndrome (ARDS)
[21]

**patient**: An individual receiving medi- cal care, specifically a COVID-19 patient but can also refer to patients in other contexts
[22]

**guidelines**: Recommendations and best practices for managing COVID-19, including diag- nosis, treatment, and infection control
[23]

**acute**: Refers to the sudden onset and often severe nature of COVID-19
[24]

**viral**: Relates to the virus, SARS-CoV- 2, and its characteristics
[25]

com- peting interest,

**competing**: In the context of "com- peting interest," it refers to potential conflicts of interest that researchers may have due to financial or other ties
[26]

**results**: Outcomes and findings of re- search studies on COVID-19
[27]

**data**: Information gathered from re- search, clinical practice, and public health surveil- lance related to COVID-19
[28]

**prospective**: Describes studies that fol- low individuals forward in time to observe out- comes
[29]

**trial**: Refers to a clinical trial, a con- trolled research study to evaluate the effectiveness of interventions for COVID-19
[30]

**obtained**: Indicates data or samples collected during research
[31]

**treatment**: Medical interventions and therapies for COVID-19
[32]

**during**: Preposition indicating the time period of the COVID-19 pandemic or specific events within it
[33]

**care**: Medical treatment and support provided to COVID-19 patients
[34]

**irb**: Institutional Review Board, respon- sible for ethical oversight of research involving human subjects
[35]

**relevant**: Indicates information directly related to or applicable to the context of COVID- 19
[36]

**methods**: Procedures and techniques used in research studies on COVID-19
[37]

**risk**: The likelihood of contracting COVID-19, developing severe complications, or experiencing other adverse outcomes
[38]

**appropriate**: Indicates suitability or cor- rectness of actions or interventions in the context of COVID-19
[39]

**reported**: Indicates information that has been formally documented or communicated
[40]

**outbreak**: The initial emergence and spread of COVID-19, often used to describe spe- cific instances of clusters or geographic spread
[41]

**reporting**: The process of disseminating research findings, clinical observations, and public health data related to COVID-19
[42]

**transmission**: The spread of SARS- CoV-2 from person to person or through other routes
[43]

**necessary**: Indicates essential actions or resources required for managing COVID-19
[44]

**novel**: Emphasizes the new and pre- viously unknown nature of SARS-CoV-2 and COVID-19
[45]

transmission

**spread**: The dissemination of COVID- 19, often used interchangeably with "transmission."
[46]

**mortality**: Deaths associated with COVID-19
[47]

identi- fier

**id**: likely an abbreviation for "identi- fier" and probably used in specific data collection or research contexts. 46
[48]

**syndrome**: Refers to a collection of symptoms, specifically in the context of COVID- 19 and its potential complications like ARDS and cytokine storm
[49]

**manuscript**: A written document sum- marizing research findings or clinical observations for publication
[50]

**confirm**: To verify or validate informa- tion or a diagnosis related to COVID-19
[51]

**of**: Preposition, carries no specific se- mantic meaning in this context
[52]

**medical**: Relates to the broader med- ical context of COVID-19, including healthcare systems, medical professionals, and treatments
[53]

**studies**: A general term referring to various types of research investigations related to COVID-19
[54]

**epidemic**: Refers to the outbreak and spread of COVID-19, often used to describe spe- cific phases or geographic regions
[55]

**ci**: Confidence Interval, a statistical measure indicating the range of uncertainty around an estimate
[56]

**china**: Refers to the country where COVID-19 was first identified
[57]

**hospital**: Medical facilities where COVID-19 patients are admitted and treated
[58]

**provided**: Indicates resources or infor- mation made available
[59]

**infected**: Individuals who have con- tracted the SARS-CoV-2 virus
[60]

**symptoms**: Clinical manifestations and signs of COVID-19 infection
[61]

competing interest,

**interest**: In the context of "competing interest," it refers to potential conflicts of interest that researchers may have due to financial or other ties
[62]

**followed**: Indicates adherence to guide- lines or protocols in research or clinical practice
[63]

**healthcare**: Refers to the systems, pro- fessionals, and services involved in providing med- ical care to COVID-19 patients
[64]

**pneumonia**: A serious respiratory com- plication associated with COVID-19
[65]

**provide**: Indicates resources or infor- mation made available
[66]

**approved**: Indicates authorization or endorsement for treatments or research protocols
[67]

**p**: P-value, a statistical measure indicat- ing the probability of observing a result by chance
[68]

**authors**: Researchers or clinicians who have contributed to a scientific manuscript
[69]

**trials**: A general term for clinical tri- als, controlled studies to evaluate interventions for COVID-19
[70]

**measures**: Interventions, actions, or policies implemented to mitigate the spread or im- pact of COVID-19
[71]

**retrospectively**: Describes studies that look back at past data or events
[72]

**positive**: Indicates a confirmed diagno- sis of COVID-19 infection
[73]

**background**: Contextual information or prior knowledge relevant to COVID-19 research
[74]

**registry**: A database or system for col- lecting and tracking information, often used for clinical trials or patient data
[75]

**associated**: Indicates a connection or relationship between factors, often used to describe comorbidities or risk factors related to COVID-19
[76]

**confirmed**: Indicates validated cases of COVID-19 infection
[77]

**interventional**: Describes studies that involve active interventions or manipulations
[78]

**funding**: Financial support for research or healthcare activities related to COVID-19
[79]

**icmje**: International Committee of Med- ical Journal Editors, an organization that sets ethi- cal standards for medical publications
[80]

**wuhan**: The city in China where the COVID-19 outbreak originated
[81]

**approvals**: Ethical and regulatory au- thorizations for research or treatments
[82]

ClinicalTrials.gov,

**clinicaltrials**: Likely refers to the web- site "ClinicalTrials.gov," a registry for clinical tri- als
[83]

**countries**: Refers to nations affected by the COVID-19 pandemic
[89]

**supplementary**: Additional informa- tion or data provided as a supplement to a main manuscript. 47
[90]

Wuhan,"

**analysis**: The process of examining and interpreting data related to COVID-19. ## Themes: Here are eight potential themes for the keywords, with descriptions and assigned keywords: **1. Epidemiology and Public Health (Index 1)** * **Description:** This theme covers the stud...
[91]

**in** (preposition)
[92]

**the** (definite article)
[93]

**of** (preposition)
[94]

**to** (preposition)
[95]

**from** (preposition)
[96]

**a** (indefinite article)
[98]

**with** (preposition)
[99]

**s** (possessive marker)
[102]

**for** (preposition)
[103]

**that** (conjunction/pronoun)
[104]

**as** (conjunction/preposition)
[106]

**this** (demonstrative adjective/pronoun)
[110]

**which** (relative pronoun)
[116]

**all** (determiner/adverb)
[118]

**an** (indefinite article)
[119]

**its** (possessive pronoun) 59
[128]

**clinical** (adjective)
[137]

**hospitals** (noun)
[138]

**over** (preposition) ## Step 2: Content Word Collocates
[139]

**coronavirus** (noun)
[140]

**national** (adjective)
[141]

**first** (adjective/adverb)
[142]

**rapidly** (adverb)
[143]

**countries** (noun)
[144]

**other** (adjective)
[145]

**foundation** (noun)
[146]

**natural** (adjective)
[147]

**outside** (adverb/preposition)
[148]

**since** (preposition/conjunction)
[149]

**identified** (verb)
[150]

**novel** (adjective)
[151]

**pneumonia** (noun)
[152]

**late** (adjective/adverb)
[153]

**confirmed** (verb/adjective)
[154]

**across** (preposition)
[155]

**worldwide** (adverb)
[156]

**south** (adjective/noun)
[157]

**provinces** (noun)
[158]

**throughout** (preposition)
[159]

**originated** (verb)
[160]

**clinical** (adjective) 60
[161]

**university** (noun)
[162]

**control** (noun/verb)
[163]

**spreading** (verb)
[164]

**severe** (adjective)
[165]

**global** (adjective)
[166]

**respiratory** (adjective)
[167]

**major** (adjective)
[168]

**chinese** (adjective)
[169]

China" with

**hospitals** (noun) ## Step 3: Analysis and Summary Analyzing the content word collocate list and the concordance lines, it is clear that China was predominantly linked to COVID-19 in this corpus as **the origin point of the outbreak** and **a key player in the early stages o...

2019
[170]

63 **Summary 2:** The corpus highlights China’s response to the COVID-19 pandemic, including its national policies and research contributions. * Collocates such as

**first emerged in China** and rapidly spread in the world causing a pandemic." 63 **Summary 2:** The corpus highlights China’s response to the COVID-19 pandemic, including its national policies and research contributions. * Collocates such as "national" (22), "control" (85), ...

2020
[171]

**Descriptive of the Term Itself:** In these cases, the authors are analyzing the term itself, its prevalence, and its impact, without necessarily en- dorsing its use
[172]

**Neutral Reporting:** The authors use the term to report on its usage by others, often in the context of media analysis or social science re- search
[173]

China virus

**Potentially Biased:** The authors use the term in a way that could be interpreted as endors- ing or promoting the association of the virus with China in a negative or stigmatizing way. **Analysis of Concordance Lines:** * **Lines 1-6, 9, 11, 12, 15-20:** These lines fall und...

2020
[2000]

chinese virus

A neural probabilistic language model. Ad- vances in neural information processing systems, 13. Gavin Brookes and Tony McEnery. 2019. The utility of topic modelling for discourse studies: A critical evaluation. Discourse Studies, 21(1):3–21. Tom Brown, Benjamin Mann, Nick Ryde...

2019
[2016]

Corpora, 11(3):433–463

Clic dickens: Novel uses of concordances for the integration of corpus stylistics and cognitive poetics. Corpora, 11(3):433–463. World Health Organization et al. 2020. Social stigma associated with covid-19. Pranab Sahoo, Ayush Kumar Singh, Sriparna Saha, Vinija Jain, Samrat M...

2020
[2020]

the china virus

After “the china virus” went viral: Racially charged coronavirus coverage and trends in bias against asian americans. Health Education & Be- havior, 47(6):870–879. Yihe Deng, Weitong Zhang, Zixiang Chen, and Quan- quan Gu. 2023. Rephrase and respond: Let large language models ...

2023 arXiv
[2023]

In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, pages 1–19

Evaluating large language models in gener- ating synthetic hci research data: a case study. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, pages 1–19. Taojun Hu and Xiao-Hua Zhou. 2024. Unveiling llm evaluation focused on metrics: Challenges a...

2023 arXiv
[2024]

arXiv preprint arXiv:2402.07927

A systematic survey of prompt engineering in large language models: Techniques and applications. arXiv preprint arXiv:2402.07927. Sander Schulhoff, Michael Ilie, Nishant Balepur, Kon- stantine Kahadze, Amanda Liu, Chenglei Si, Yin- heng Li, Aayush Gupta, HyoJung Han, Sevien Sc...

2024 arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [1]

**covid**: Short for COVID-19, the disease caused by the SARS-CoV-2 virus

[2] [2]

Also refers to individuals in other medical contexts

**patients**: Individuals diagnosed with or suspected of having COVID-19. Also refers to individuals in other medical contexts. 45

[3] [3]

**cov**: Abbreviation for Coronavirus, specifically can refer to SARS-CoV , MERS-CoV , or SARS-CoV-2

[4] [4]

Often used in compari- son to COVID-19

**sars**: Severe Acute Respiratory Syn- drome, specifically refers to the disease caused by SARS-CoV in 2002-2003. Often used in compari- son to COVID-19

2002

[5] [5]

**pandemic**: Refers to the global outbreak of COVID-19

[6] [6]

Wuhan" (4),

Thread of thought unraveling chaotic contexts. arXiv preprint arXiv:2311.08734. A Appendix A.1 Discourse Analysis Tasks with TACOMORE Keyword Analysis. Prompt # Role Description You are an expert in Corpus Linguistics doing keyword analysis. # Task Definition I would give you a list of keywords from a cor- pus of the abstracts of COVID-19 research article...

arXiv 2019

[7] [7]

**disease**: Generally refers to COVID-19 but can also refer to other medical conditions

[8] [8]

**study**: Refers to research investigations related to COVID-19, including clinical trials, ob- servational studies, and epidemiological studies