Pith. sign in

REVIEW 4 major objections 5 minor 21 references

The consistent-over-inconsistent (CAI) ratio, computable without gold labels, tracks LLM annotation accuracy strongly enough to rank annotator models.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 20:05 UTC pith:S347FADG

load-bearing objection The CAI ratio is a reasonable unsupervised heuristic, but the paper's model-selection claim is vacuous and the cross-dataset correlations don't support it. the 4 major comments →

arxiv 2509.08809 v1 pith:S347FADG submitted 2025-09-10 cs.CL

Evaluating LLMs Without Oracle Feedback: Agentic Annotation Evaluation Through Unsupervised Consistency Signals

Cheng Chen , Haiyan Yin , Ivor Tsang This is my paper
classification cs.CL
keywords LLM annotation evaluationunsupervised evaluationCAI ratiostudent–teacher consistencymodel selectionmajority votingannotation qualityno oracle feedback
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tackles a practical problem: when LLMs generate annotations for text tasks, there is often no ground truth to check their work. It proposes an agentic arrangement in which a small student model acts as a referee: given 5% user-preferred examples, the student assigns labels by embedding similarity and majority vote, and an LLM teacher assigns labels either alone or with the student's label as context. Samples where both models agree are "consistent"; samples where they differ are "inconsistent." The ratio of these counts, the CAI Ratio, is claimed to be a meaningful unsupervised quality signal. Across ten datasets and four LLMs it correlates with actual annotation accuracy (0.72–0.93, p<0.02), and it picks the best LLM on six of ten datasets.

Core claim

The central discovery is that agreement between a cheap student model and an LLM teacher, without any oracle labels, is a proxy for the LLM's accuracy. On samples where the student and teacher agree, the teacher tends to be right; on samples where they disagree, the teacher is much more likely to be wrong. The CAI Ratio—the count of agreeing samples divided by the count of disagreeing samples—therefore orders LLMs by quality. The paper reports statistically significant positive correlations for four LLMs and uses the ratio for model selection, though it notes the selection is not perfect (60% exact matches, with small accuracy gaps on mismatches).

What carries the argument

The machinery is a student–teacher comparison pipeline. The student is a sentence-transformer (MiniLM) that computes an Average Similarity score between each input's embedding and the top-k examples in each cluster of the 5% user-preference set, then assigns the label of the closest cluster. The teacher LLM produces zero-shot labels and also single-shot labels conditioned on the student's label. A sample is consistent only if the student label equals both LLM outputs; otherwise inconsistent. The CAI Ratio is NC/NIC, and the paper's "Law of Consistency" claims that for optimal student and teacher, consistent samples asymptotically outnumber inconsistent ones.

Load-bearing premise

The argument rests on the student model's majority-vote labels (derived from 5% user-preferred examples) being accurate and not systematically wrong in the same way the LLM is wrong, so that agreement monotonically proxies teacher accuracy; the paper never reports the student's accuracy, varies hyperparameters across datasets (Appendix B), and provides no ablation of the student model, cluster count, or clustering method.

What would settle it

Take a held-out dataset with known labels and compare two LLMs: if the CAI ratio ranks them opposite to their measured accuracy, the monotone relationship fails. Even stronger: replace the student model with a random label assigner; if the CAI-accuracy correlation persists, the signal is not coming from the student's structure. A dataset where the worst LLM has the highest CAI would directly contradict the claimed correlation.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • LLM annotators can be compared and ranked without collecting ground-truth labels, as long as a small set of user-preference examples is available.
  • The ratio can be recomputed on new unlabeled data, so annotation quality can be monitored over time or across domains.
  • Inconsistent samples identified by student–teacher disagreement can be flagged for human review, since they carry disproportionate error.
  • The single-shot conditioning of the LLM on student labels gives a second signal that strengthens the consistency definition without extra oracle cost.
  • Because the ratio is a scalar, it offers a cheap, API-call-light way to screen candidate LLMs before committing to one for a labeling pipeline.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The method's power depends on the student reference being accurate and not sharing the LLM's error patterns; a natural extension would test whether replacing MiniLM with other encoders, or with a second LLM of a different family, preserves the correlation.
  • A per-sample consistency score could be turned into a confidence-weighting or filtering mechanism for training sets, not just a whole-model score.
  • The 5% user-preference requirement means this is not fully label-free, but it is vastly cheaper than full annotation; the paper leaves open how sensitive results are to that percentage.
  • If the consistency signal is as robust as claimed, it could serve as a drift detector in production annotation pipelines: a sudden drop in CAI would flag that the LLM's outputs are becoming unreliable before ground truth arrives.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces the Consistent-and-Inconsistent (CAI) Ratio, defined as NC/NIC, for unsupervised evaluation of LLM-generated annotations. A MiniLM student model forms clusters from 5% user-preference labels and assigns labels by average cosine similarity to the top-k cluster embeddings (Eqs. (1)–(2)). LLM outputs are compared with student labels under zero-shot and single-shot prompting; samples are 'consistent' only when the student and the LLM agree in both settings. The authors report Pearson correlations between the CAI ratio and LLM accuracy across ten datasets for four LLMs (GPT-3.5: 0.93, GPT-4o Mini: 0.86, Llama-8B: 0.81, Gemini: 0.72), and a model-selection experiment in which the CAI-best model matches the accuracy-best model in 6 of 10 datasets. The paper claims the CAI ratio is an essential unsupervised tool for both annotation evaluation and model selection.

Significance. If the claimed correlation holds, the CAI Ratio would be a practically valuable signal for monitoring LLM annotation quality without oracle labels. The paper's strengths are its multi-dataset, multi-LLM scope, the explicit definition of the metric, and the inclusion of consistency/inconsistency accuracy breakdowns. However, the current evidence does not support the stronger model-selection claim, and the central correlation results are fragile because they rely on cross-dataset variation with few data points and per-dataset hyperparameter choices made without a stated validation protocol. The idea is worth pursuing, but the load-bearing empirical arguments need substantially stronger analysis.

major comments (4)
  1. [§3, Table 1] The model-selection claim is not supported by the reported experiment. In every row of Table 1, Google Gemini is the model with the highest CAI ratio, while Gemini is the most accurate model in only 6 of 10 datasets. A trivial baseline that always selects Gemini would therefore achieve the same 60% match rate. The table provides no evidence that, for a fixed dataset, a higher CAI ratio among candidate LLMs corresponds to higher accuracy. The authors should report within-dataset rank correlations (e.g., Spearman) between CAI ratio and accuracy, or a confusion matrix of CAI-based vs accuracy-based model rankings, and compare against a 'constant model' baseline.
  2. [§3, Figure 4 and Appendix A.3] The Pearson correlations are computed per LLM over the ten datasets, so each correlation uses only n=10 points, and the points are dataset-level aggregates. This conflates dataset difficulty with LLM competence: a model may have a high CAI ratio simply because the dataset is easy, not because the LLM is good. The paper does not report within-dataset correlations, nor does it control for dataset difficulty. Additionally, no baseline is given (e.g., correlation of accuracy with self-consistency, with random rankings, or with a trivial majority-vote student). The p-values are also uncorrected across four LLM-specific tests. Without these controls, the claim of a 'strong positive correlation' is not established as evidence for the CAI ratio as a model-comparison tool.
  3. [§2.2 and Appendix B] The CAI ratio depends on several free hyperparameters: the choice of student model (MiniLM), the number of clusters k, the top-k similarity neighborhood, and the user-preference sample proportion. Appendix B shows that top-k varies from 3 to 30 and proportion varies from 0.2 to 1 across datasets, with no stated selection criterion or validation protocol. This opens the door to post-hoc selection of hyperparameters that maximize the reported correlation. The paper also never reports the student model's own accuracy, so the reader cannot judge whether the reference signal is meaningful. The authors should provide a validation protocol (e.g., internal holdout), ablations over k, top-k, and proportion, and report student accuracy on at least a subset of datasets.
  4. [Appendix A.1] The 'Law of Consistency' is stated as a formal proposition ('As |D_u| → ∞, P(N_C > N_IC) → 1') but no proof or precise conditions are given. The statement assumes T* and S* are 'optimal hypotheses', but the student model used in practice is a fixed MiniLM with approximate majority voting, and the teacher LLM is not optimal. The theoretical foundation for interpreting NC/NIC as a proxy for accuracy is therefore missing. Either provide a proof under explicit assumptions about error independence and model optimality, or soften the claim to an empirical observation.
minor comments (5)
  1. [§2.2, Eq. (1)] The notation is confusing: k is used both for the number of clusters and for the top-k neighborhood size. It is later stated that k=5, but Appendix B also varies top-k independently. Please use separate symbols (e.g., K for clusters, m for top-k).
  2. [§2.1] The definition of the user-preference set says s = 5% × |D_U|, but Appendix B reports 'proportion' values of 0.2, 0.5, and 1.0. It is unclear whether 'proportion' refers to the fraction of user-preference samples or to something else. Please define the term when first used.
  3. [§2.2] The formulas for zero-shot and single-shot generation are duplicated and typeset incorrectly (e.g., '¯yt_i = T(x_i) with ...' appears twice). Clean up the notation and formatting.
  4. [§3] Dataset names are inconsistent: 'Bank77' appears in the paragraph but 'Banking77' in tables; 'Massive (Intent)' vs 'Massive Intent'; 'FewRel Nat' vs 'FewRel'. Standardize names.
  5. [Table 3] The table formatting is broken: there is stray text ('proportion=0.2') inside the Implementation Details sentence, and table columns are misaligned. The appendix would also benefit from a note on how standard deviations over the two seeds/temperatures were aggregated.

Circularity Check

1 steps flagged

Minor self-definitional 'Law of Consistency'; central CAI-accuracy correlation is empirically anchored.

specific steps
  1. self definitional [Appendix A.1, Law of Consistency]
    "This principle states that if both the LLM and student model are optimal hypotheses, denoted as T ∗ and S∗ for a given dataset Du, the number of consistent samples should asymptotically exceed the number of inconsistent samples as the dataset size approaches infinity. [Law of Consistency] ... lim |Du|→∞ P(N C > NIC ) = 1."

    The Law's conclusion (NC > NIC) follows directly from defining both hypotheses as 'optimal', i.e., accurate. If optimality means agreement with ground truth, the theorem restates that two perfect annotators agree; it does not independently establish that agreement (CAI) tracks accuracy for the real, suboptimal models evaluated. The paper presents this as a formalization of the CAI–accuracy relationship, but the relationship is assumed inside the definition of optimality. Since the empirical correlation is measured independently, this tautology is not the central load-bearing step, but it is a self-definitional justification.

full rationale

The central empirical claim—CAI ratio correlates with measured LLM annotation accuracy—is externally anchored: the paper computes CAI from student–teacher agreement and then compares it with separately measured accuracy over ten datasets, so the correlation is not definitionally equal to the target. The student model's 5% user-preference labels are a legitimate (weakly supervised) reference; using them does not make the comparison circular, though it is a limitation. The model-selection claim is weak (CAI always selects Gemini, so the 60% match equals Gemini's base win rate, and the per-LLM Pearson correlations are across datasets rather than within a dataset), but that is a statistical validity problem rather than a circular reduction. The self-citations (Chen & Tsang 2024; Chen et al. 2024) are background citations, not load-bearing. The only genuinely circular element is the 'Law of Consistency' in Appendix A.1, which states that optimal teacher and student agree; since 'optimal' here effectively means accurate, the law is a definitional tautology rather than independent support. Because this tautology is not used in the empirical computations, the overall circularity is minor.

Axiom & Free-Parameter Ledger

3 free parameters · 5 axioms · 0 invented entities

The CAI ratio is a metric, not an entity; the paper introduces no new particles, forces, dimensions, or conserved quantities. The load-bearing structure is the student model reference, which is chosen arbitrarily (MiniLM) and fitted to a small labeled subset.

free parameters (3)
  • top-k similarity neighborhood = 3, 5, 7, 15, 20, 30 (per dataset)
    Appendix B varies top-k per dataset with no stated selection criterion; this is a free parameter affecting the student annotations and hence the CAI ratio.
  • user-preference sample proportion = 0.2, 0.5, 1.0 (per dataset)
    Proportion of labeled data used to build the student reference clusters; varies per dataset, and the choice affects student quality and the CAI ratio.
  • LLM decoding temperature = 0.5 and 1.0
    The CAI ratio depends on the stochasticity of LLM outputs; only two temperature settings are tested, treated as random seeds.
axioms (5)
  • domain assumption Semantic similarity in the MiniLM embedding space captures class structure (similar samples share labels).
    Section 2.2 relies on cosine similarity over embeddings to cluster and assign labels.
  • domain assumption The 5% user-preference samples are representative of the full data distribution and their labels are correct.
    The clusters Cj are built from H and treated as ground truth for the student model.
  • ad hoc to paper Student and LLM annotation errors are conditionally independent given the true label.
    The CAI ratio's validity as a proxy for accuracy requires that agreement is not due to shared biases; the paper does not test this.
  • ad hoc to paper Law of Consistency (Appendix A.1): if T* and S* are optimal, NC > NIC asymptotically.
    Stated without proof; it reduces to saying optimal models agree on true labels, which is definitional rather than supporting evidence.
  • domain assumption The number of clusters k equals the true number of classes, and cluster labels correspond to class labels.
    The problem definition assumes the user-preference annotations cluster into k non-overlapping clusters; no method for selecting k is given.

pith-pipeline@v1.3.0-alltime-deepseek · 9582 in / 12232 out tokens · 119609 ms · 2026-08-04T20:05:33.484266+00:00 · methodology

0 comments
read the original abstract

Large Language Models (LLMs), when paired with prompt-based tasks, have significantly reduced data annotation costs and reliance on human annotators. However, evaluating the quality of their annotations remains challenging in dynamic, unsupervised environments where oracle feedback is scarce and conventional methods fail. To address this challenge, we propose a novel agentic annotation paradigm, where a student model collaborates with a noisy teacher (the LLM) to assess and refine annotation quality without relying on oracle feedback. The student model, acting as an unsupervised feedback mechanism, employs a user preference-based majority voting strategy to evaluate the consistency of the LLM outputs. To systematically measure the reliability of LLM-generated annotations, we introduce the Consistent and Inconsistent (CAI) Ratio, a novel unsupervised evaluation metric. The CAI Ratio not only quantifies the annotation quality of the noisy teacher under limited user preferences but also plays a critical role in model selection, enabling the identification of robust LLMs in dynamic, unsupervised environments. Applied to ten open-domain NLP datasets across four LLMs, the CAI Ratio demonstrates a strong positive correlation with LLM accuracy, establishing it as an essential tool for unsupervised evaluation and model selection in real-world settings.

Figures

Figures reproduced from arXiv: 2509.08809 by Cheng Chen, Haiyan Yin, Ivor Tsang.

Figure 1
Figure 1. Figure 1: Agentic Annotation Evaluation Process: the teacher LLM generates noisy an￾notations in zero-/single-shot settings, which are compared against student model’s AS out￾put to measure the consistency score. An agentic annotation evaluation process is constructed upon an LLM teacher with noisy annotation labels and a student model with limited user preferences, incorporating a preference-based majority voting m… view at source ↗
Figure 3
Figure 3. Figure 3: Visualization of t-SNE Clustering (bet￾ter viewed in color, enlarged) comparing LLM vs Ground-Truth Annotations on Go Emotion Dataset. LLM outputs exhibit high similarity with ground￾truth labels on consistent samples, while showing significant divergence on inconsistent samples. where AS(ei , Cj ) is the average cosine similarity of ei with the embeddings in Cj . The annotation y¯j ∗ associated with Cj ∗ … view at source ↗
Figure 4
Figure 4. Figure 4: Correlation analysis between LLM annotation accuracy and the CAI ratio, evaluated across 4 principled LLMs (also see statistical test results in Sec 3). The Pearson correlation coefficients and corresponding p-values confirm the statistical significance of the positive correlation between CAI ratio and LLMs accuracy. Dataset Best CAI Model Best Accuracy Model Match Accuracy Difference Model Accuracy (%) Mo… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

21 extracted references · 15 linked inside Pith

  1. [1]

    Language models are few-shot learners.Advances in neural information processing systems, 33:1877–1901,

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners.Advances in neural information processing systems, 33:1877–1901,

    1901

  2. [3]

    Self-teaching prompting for multi-intent learning with limited super- vision

    Cheng Chen and Ivor Tsang. Self-teaching prompting for multi-intent learning with limited super- vision. InThe Second Tiny Papers Track at ICLR 2024,

    2024

  3. [7]

    An evaluation dataset for intent classification and out-of-scope prediction.arXiv preprint arXiv:1909.02027,

    Stefan Larson, Anish Mahendran, Joseph J Peper, Christopher Clarke, Andrew Lee, Parker Hill, Jonathan K Kummerfeld, Kevin Leach, Michael A Laurenzano, Lingjia Tang, et al. An evaluation dataset for intent classification and out-of-scope prediction.arXiv preprint arXiv:1909.02027,

    Pith/arXiv arXiv 1909

  4. [10]

    Large language model guided tree-of-thought.arXiv preprint arXiv:2305.08291,

    6 Jieyi Long. Large language model guided tree-of-thought.arXiv preprint arXiv:2305.08291,

    Pith/arXiv arXiv

  5. [11]

    Large language models for data annotation: A survey.arXiv preprint arXiv:2402.13446,

    Zhen Tan, Dawei Li, Song Wang, Alimohammad Beigi, Bohan Jiang, Amrita Bhattacharjee, Man- sooreh Karami, Jundong Li, Lu Cheng, and Huan Liu. Large language models for data annotation: A survey.arXiv preprint arXiv:2402.13446,

    Pith/arXiv arXiv

  6. [12]

    Codeclm: Aligning language models with tailored synthetic data.arXiv preprint arXiv:2404.05875,

    Zifeng Wang, Chun-Liang Li, Vincent Perot, Long T Le, Jin Miao, Zizhao Zhang, Chen-Yu Lee, and Tomas Pfister. Codeclm: Aligning language models with tailored synthetic data.arXiv preprint arXiv:2404.05875,

    Pith/arXiv arXiv

  7. [13]

    Finetuned language models are zero-shot learners.arXiv preprint arXiv:2109.01652,

    Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le. Finetuned language models are zero-shot learners.arXiv preprint arXiv:2109.01652,

    Pith/arXiv arXiv

  8. [14]

    Unigen: A unified framework for textual dataset generation using large language models.arXiv preprint arXiv:2406.18966,

    Siyuan Wu, Yue Huang, Chujie Gao, Dongping Chen, Qihui Zhang, Yao Wan, Tianyi Zhou, Xian- gliang Zhang, Jianfeng Gao, Chaowei Xiao, et al. Unigen: A unified framework for textual dataset generation using large language models.arXiv preprint arXiv:2406.18966,

    arXiv

  9. [15]

    Can llms express their uncertainty? an empirical evaluation of confidence elicitation in llms.arXiv preprint arXiv:2306.13063,

    Miao Xiong, Zhiyuan Hu, Xinyang Lu, Yifei Li, Jie Fu, Junxian He, and Bryan Hooi. Can llms express their uncertainty? an empirical evaluation of confidence elicitation in llms.arXiv preprint arXiv:2306.13063,

    Pith/arXiv arXiv

  10. [16]

    React: Synergizing reasoning and acting in language models.arXiv preprint arXiv:2210.03629,

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models.arXiv preprint arXiv:2210.03629,

    Pith/arXiv arXiv

  11. [17]

    Zerogen: Efficient zero-shot learning via dataset generation.arXiv preprint arXiv:2202.07922,

    Jiacheng Ye, Jiahui Gao, Qintong Li, Hang Xu, Jiangtao Feng, Zhiyong Wu, Tao Yu, and Ling- peng Kong. Zerogen: Efficient zero-shot learning via dataset generation.arXiv preprint arXiv:2202.07922,

    Pith/arXiv arXiv

  12. [19]

    Clusterllm: Large language models as a guide for text clustering

    Yuwei Zhang, Zihan Wang, and Jingbo Shang. Clusterllm: Large language models as a guide for text clustering. InThe 2023 Conference on Empirical Methods in Natural Language Processing,

    2023

  13. [20]

    Relying on the unreliable: The impact of language models’ reluctance to express uncertainty.arXiv preprint arXiv:2401.06730,

    Kaitlyn Zhou, Jena D Hwang, Xiang Ren, and Maarten Sap. Relying on the unreliable: The impact of language models’ reluctance to express uncertainty.arXiv preprint arXiv:2401.06730,

    Pith/arXiv arXiv

  14. [21]

    yi denotes the LLM annotation accuracies

    A.2 COMPARISON WITHTRADITIONALEVALUATIONMETRICS Metric Ground-Truth Labels? Data Drift? Tracks Annotation Quality Over Time? Accuracy✓ ✗ ✗ Precision/Recall✓ ✗ ✗ F1-score✓ ✗ ✗ CAI Ratio✗ ✓ ✓ Table 2: Comparison of Traditional Metrics and CAI Ratio A.3PEARSONCORRELATIONTEST FORCONSISTENT ANDINCONSISTENTRATIO We have performed a Pearson correlation, the corr...

    2018

  15. [2018]

    Large language models can self-improve.arXiv preprint arXiv:2210.11610,

    Jiaxin Huang, Shixiang Shane Gu, Le Hou, Yuexin Wu, Xuezhi Wang, Hongkun Yu, and Jiawei Han. Large language models can self-improve.arXiv preprint arXiv:2210.11610,

    Pith/arXiv arXiv

  16. [2019]

    Mtop: A comprehensive multilingual task-oriented semantic parsing benchmark.arXiv preprint arXiv:2008.09335,

    Haoran Li, Abhinav Arora, Shuohui Chen, Anchit Gupta, Sonal Gupta, and Yashar Mehdad. Mtop: A comprehensive multilingual task-oriented semantic parsing benchmark.arXiv preprint arXiv:2008.09335,

    Pith/arXiv arXiv 2008

  17. [2020]

    Efficient intent detection with dual sentence encoders.arXiv preprint arXiv:2003.04807,

    I˜nigo Casanueva, Tadas Temˇcinas, Daniela Gerz, Matthew Henderson, and Ivan Vuli ´c. Efficient intent detection with dual sentence encoders.arXiv preprint arXiv:2003.04807,

    Pith/arXiv arXiv 2003

  18. [2021]

    New intent discovery with pre-training and contrastive learning.arXiv preprint arXiv:2205.12914,

    7 Yuwei Zhang, Haode Zhang, Li-Ming Zhan, Xiao-Ming Wu, and Albert Lam. New intent discovery with pre-training and contrastive learning.arXiv preprint arXiv:2205.12914,

    Pith/arXiv arXiv

  19. [2022]

    Tweac: transformer with extendable qa agent classifiers.arXiv preprint arXiv:2104.07081,

    Gregor Geigle, Nils Reimers, Andreas R ¨uckl´e, and Iryna Gurevych. Tweac: transformer with extendable qa agent classifiers.arXiv preprint arXiv:2104.07081,

    Pith/arXiv arXiv

  20. [2023]

    Fewrel: A large-scale supervised few-shot relation classification dataset with state-of-the-art evaluation

    Xu Han, Hao Zhu, Pengfei Yu, Ziyun Wang, Yuan Yao, Zhiyuan Liu, and Maosong Sun. Fewrel: A large-scale supervised few-shot relation classification dataset with state-of-the-art evaluation. arXiv preprint arXiv:1810.10147,

    Pith/arXiv arXiv

  21. [2024]

    Graphprompt: Unifying pre-training and downstream tasks for graph neural networks

    Zemin Liu, Xingtong Yu, Yuan Fang, and Xinming Zhang. Graphprompt: Unifying pre-training and downstream tasks for graph neural networks. InProceedings of the ACM Web Conference 2023,

    2023