Pith. sign in

REVIEW 2 major objections 5 minor 40 references

Telco-GAIA: Bilingual Benchmark for Agents in Telecom Domain

T0 review · 2 major / 5 minor · reviewed 2026-08-02 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Telco-GAIA introduces 100 bilingual, multi-hop telecom tasks scored by deterministic exact matching, with the best tested model solving only 71 percent.

desk verdict A carefully built bilingual telecom agent benchmark that mostly delivers on its stated design goals, except the 'reproducible over time' claim is overstated for the 16 live-web tasks. read the letter →

arxiv 2607.20510 v1 pith:AERYTE5I submitted 2026-07-04 cs.AI cs.CL

Dmitrii Khizbullin , Zaid Alyafeai , Abdelrahman Eldesokey , Nourah AlSultan , Raghad Alshalan , David R. Pugh , Bernard Ghanem This is my paper · ORCID
classification cs.AIcs.CL
keywords telecomagentsbilingualbenchmarkmulti-hopreasoningtooluseexactstringmatchingrelationaldatabasevisualdocumentunderstandingenterpriseAIevaluation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces Telco-GAIA, a benchmark for AI agents that answer realistic telecom customer questions over a single operator's public data. Each of its 100 tasks is a human-verified multi-hop chain, averaging 4.2 hops, that forces an agent to navigate a frozen copy of the operator's website, read linked PDFs and images, query a synthetic relational database, and sometimes consult live third-party web archives. The authors claim this is the first benchmark to combine all these properties, and that scoring by normalized exact string matching makes evaluation objective, deterministic, and reproducible over time without an LLM judge. On a reference agent tested with twelve commercial and open models, the strongest reaches 71 percent, while the visually grounded categories remain the clearest bottleneck. The paper positions Telco-GAIA both as a difficulty probe for enterprise agents and as a reusable template for building closed-domain benchmarks.

What carries the argument

The load-bearing mechanism is the task-construction rule of strict causal chains: each task must form a single linear chain where the output of hop N is required for hop N+1, with no dead ends, no redundant facts, and no spoiling of intermediate answers. This is paired with a Docker-served frozen website and relational database, a normalized exact-match scorer with no LLM judge, and controlled data-quality traps in the database so that a naive SELECT SUM or COUNT returns the wrong answer and only agents that inspect and filter the data succeed. The environment is semi-closed: the operator website and database are frozen locally, while sixteen Web Archives tasks reach live third-party encyclo

What would settle it

Take any Web Archives task, edit its target third-party page so that the previously correct value changes, then rerun the unchanged evaluation script against the released ground truth. A correct environment should still pass; if the task now scores zero, the benchmark's reproducibility claim fails for that task.

Watch

Extended reading notes

Core claim

Telco-GAIA's central claim is that a closed enterprise benchmark can keep the discipline of exact-match evaluation while covering heterogeneous, realistic sources: a static website snapshot, linked PDFs and images, a synthetic SQLite customer database exposed through a REST API, and external web archives. The 100 tasks span seven categories in English and Arabic, with 83 numeric and 17 textual answers, all human-verified and scored by normalized exact string matching. The reference agent experiments show a clean accuracy spread from 13 to 71 percent across twelve backends, with the strongest model solving 71 percent of tasks; under a moderate cost budget accuracy falls to about 38 percent. C

Load-bearing premise

The reproducibility guarantee assumes that the live third-party encyclopedia and preprint pages used by the Web Archives tasks will not change their task-relevant content; if such a page is edited, the gold answer can become stale even though the served environment is unchanged.

Editorial extensions

If this is right

  • Model scores on Telco-GAIA are directly comparable across time and across labs, because the served corpus is frozen and scoring involves no judge model.
  • A moderate-cost backend can reach near-frontier accuracy at roughly half the cost of the top model, so budget, latency, and accuracy should be treated as largely independent axes when choosing an agent backend.
  • The visual categories (Images, PDF, PDF Visual) lag far behind text and database categories, pointing to document and image understanding as the current binding constraint for enterprise agents.
  • The Arabic subset is close in difficulty to the English subset, so bilingual evaluation can be carried out without one language becoming an easy out.
  • The controlled database traps make naive SQL fail, so agents must inspect and filter data rather than pattern-match, rewarding genuine tool use and data-quality awareness.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The semi-closed design has a self-hardening property: as the live operator site drifts from the frozen snapshot, parametric-memory shortcuts decay over time and later runs become harder to pass without genuine retrieval, which is an unusual direction for a benchmark.
  • The same recipe — a frozen site snapshot, a synthetic database, and exact-match scoring — could be lifted to other enterprise or regulated domains, with the catalogue of data-quality traps as a reusable component.
  • The sixteen tasks that depend on live third-party pages are the weak point of the reproducibility guarantee; freezing those pages as snapshots as well would close the remaining leak.
  • Because task difficulty is reported alongside cost and latency, the benchmark can double as a cost-calibration instrument for deployment decisions, not only as a quality probe.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. Telco-GAIA presents a 100-task bilingual benchmark for tool-using agents in telecommunications. Each task is a multi-hop QA problem requiring reasoning over a locally served snapshot of a telecom operator's website (HTML, images, PDFs), a synthetic relational SQLite database exposed via REST, and, for 16 Web Archives tasks, live Wikipedia/ArXiv pages. The paper describes human-verification and anti-shortcut design principles, adversarial database traps, and an objective exact-match scorer. The authors evaluate a purpose-built reference agent across twelve commercial/open LLMs, reporting best accuracy 71%, with lowest scores on image/PDF-visual categories.

Significance. Assuming the design and release are as described, Telco-GAIA is a useful resource and one of the few benchmarks combining multimodal retrieval, relational-database querying, bilingual content, and deterministic, LLM-judge-free scoring. The paper ships concrete infrastructure (Docker services, evaluate.py, gated ground truth), uses human-verified golden steps, and provides a broad multi-model sweep. It is a solid contribution to enterprise-agent evaluation. Two issues need to be addressed before the central claims are fully supported: reproducibility over time for the 16 live-web tasks, and the reproducibility/comparability of the reference-agent protocol given hidden per-task tool restrictions. Both are fixable within the manuscript's scope.

major comments (2)
  1. [§1, §3.2, §A.1] The reproducibility claim is not supported for the 16 Web Archives tasks. The abstract and §1 state that the environment is 'semi-closed' and runs are 'reproducible over time' because 'web archives are virtually immutable'; however §A.1 states that 'Wikipedia and ArXiv are accessed directly on the internet,' and §3.2 merely expresses a preference for 'update-resilient targets such as infobox fields and ISO codes.' Wikipedia pages are continuously edited, so an infobox value, ISO code, or article identity can change, making the gold answer absent or wrong on a later run. No stability data or snapshots are provided. Since reproducibility is Contribution 2, this is load-bearing. The fix is straightforward: snapshot and serve the specific web-archive pages inside the container, or explicitly restrict the reproducibility claim to local components and mark Web Archives tasks as time-dependent.
  2. [§A.2, §C] The reference-agent baseline uses privileged per-task tool gating that external users cannot replicate. §A.2 states that 'only task_id and question are exposed to the agent,' while the full ground-truth record includes the tools field and 'the harness additionally restricts the reference agent to each task's permitted tool subset.' Thus a third party running the public benchmark cannot know which tools are allowed per task and cannot reproduce the reference-agent conditions. This weakens the comparability of the reported 12-model sweep and the 'golden steps' anti-shortcut design. Please release the per-task tool lists as non-answer metadata (or as part of questions.json) so that any agent can be run under the same gating, or explicitly state that the reference-agent numbers use privileged metadata.
minor comments (5)
  1. [Table 2] Accuracies are from a single run. Several comparisons (e.g., 68% vs 67%) are within sampling noise for 100 binary tasks. Report confidence intervals or multi-run averages for at least the headline models.
  2. [General] No human baseline is reported. A human accuracy estimate (or a reference to one) would help calibrate the 'challenging' claim.
  3. [§A.4] For the 17 textual answers, exact string matching can be brittle even with normalization; consider documenting the set of accepted alternate phrasings or the normalization rules in more detail.
  4. [Table 3] The '‡' footnote marker is used both in the table and in the text body; this is confusing and should be cleaned up.
  5. [Section D] Section D is quite long relative to the rest of the paper; condensing would improve readability, though the content is relevant.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation found: reported accuracies are empirical measurements, not predictions derived from fitted benchmark constants.

full rationale

Telco-GAIA is a benchmark paper, not a derivation. It contains no equations, fitted parameters, or predicted quantities whose values are determined by construction from the benchmark's own definitions. The central results — model accuracies across twelve backends (Table 2) and category breakdowns (Tables 3–4) — are empirical measurements against externally fixed, human-verified ground-truth answers; the benchmark's gold answers were authored and validated independently of the evaluated models. The 'purpose-built reference agent' is an evaluation tool, not a source of benchmark labels, so there is no fitted-input-called-prediction pattern. The only self-citation is Alrashed et al. (2024) in the Limitations section, used to support the design choice of human translation over machine translation; even if that citation were removed, the Arabic subset's human-authored and human-validated construction stands on its own, so the self-citation is not load-bearing. The paper's claim that the environment is 'semi-closed' and reproducible despite Web Archives tasks accessing live Wikipedia and arXiv is a genuine internal-consistency and reproducibility concern, but it is a correctness risk, not a circularity: it does not make any result equal to its input by construction. No load-bearing step in the paper reduces a claimed prediction to its own assumptions, so the appropriate circularity score is 0.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The paper introduces no free parameters or invented physical entities. Its central claims rest on methodological assumptions about scoring validity, live-web stability, annotation correctness, and tool-gating information leakage.

assumptions (4)
  • domain assumption Exact string matching after GAIA-convention normalization correctly scores all 100 tasks.
    The evaluator (Section A.4) assumes that normalized exact match is a valid proxy for correctness for both numeric and textual answers; any brittleness (synonyms, formatting) could undercount correct answers.
  • domain assumption Web archives (Wikipedia and arXiv) are stable enough that gold answers for the 16 Web Archives tasks do not change over time.
    Section A states web archives are 'virtually immutable', but they are accessed live; if pages change, reproducibility and answer correctness are affected.
  • domain assumption The human-verified gold answers and step annotations are correct.
    The entire benchmark validity rests on annotation quality; the paper describes human verification but does not provide inter-annotator agreement or a full audit.
  • domain assumption Gating the reference agent to each task's permitted tool subset does not leak answer-relevant information.
    Section A.2 claims the tools field 'never reveals the answer', but tool subset may hint at task type (e.g., only internet_browser implies Web Archives), potentially simplifying the agentic challenge.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Telco-GAIA: Bilingual Benchmark for Agents in Telecom Domain." pith.science (2026). https://pith.science/paper/AERYTE5I

@misc{pith2026260720510,
  author       = {Pith},
  title        = {Pith review of: Telco-GAIA: Bilingual Benchmark for Agents in Telecom Domain},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AERYTE5I}},
  note         = {Machine review of arXiv:2607.20510}
}
read the original abstract

We introduce Telco-GAIA, a bilingual, multi-modal benchmark for evaluating tool-using agents on the data of a real-world telecommunications operator. Telco-GAIA comprises 100 human-verified question-answering tasks, in English and Arabic, that each demand multi-hop reasoning (4.2 hops on average) over three heterogeneous sources: a static website snapshot (HTML, images, and linked PDFs), a synthetic relational SQL database, and external web archives, spanning text, image, and tabular modalities. The benchmark is delivered as a sandboxed Docker environment and scored by normalized exact string matching, making evaluation objective, deterministic, and reproducible over time without any LLM-as-a-Judge. Evaluating a purpose-built reference agent across twelve commercial and open LLMs, we find Telco-GAIA challenging: even the strongest model solves only 71% of tasks; under a moderate cost budget, this falls to about 40%, and the visually grounded categories remain the weakest, where the average backend scores below 30%, leaving substantial headroom in document and image understanding. Telco-GAIA offers a rigorous, reproducible testbed for enterprise agents and a template for constructing closed-domain benchmarks.

Figures

Figures reproduced from arXiv: 2607.20510 by the authors.

Figure 1
Figure 1. Distribution of the tasks across categories and [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Distribution of reasoning hops per task. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Benchmark architecture. The agent reads task [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: The operator website served locally via nginx [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]
Figure 5
Figure 5. Figure 5: The Datasette interface on port 8081 exposing [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]
Figure 6
Figure 6. Figure 6: Tool usage across the 100 tasks. The grey [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

40 extracted references · 1 canonical work pages

  1. [1]

    The Twelfth International Conference on Learning Representations (ICLR) , year =

    Mialon, Gr. The Twelfth International Conference on Learning Representations (ICLR) , year =

  2. [2]

    Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP) , pages =

    Yoran, Ori and Amouyal, Samuel Joseph and Malaviya, Chaitanya and Bogin, Ben and Press, Ofir and Berant, Jonathan , title =. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP) , pages =. 2024 , url =

    2024

  3. [3]

    , title =

    Alrashed, Sultan and Khizbullin, Dmitrii and Pugh, David R. , title =. 2024 , url =

    2024

  4. [4]

    2025 , url =

    Wei, Jason and Sun, Zhiqing and Papay, Spencer and McKinney, Scott and Han, Jeffrey and Fulford, Isa and Chung, Hyung Won and Passos, Alex Tachard and Fedus, William and Glaese, Amelia , title =. 2025 , url =

    2025

  5. [5]

    and Zhu, Hao and Zhou, Xuhui and Lo, Robert and Sridhar, Abishek and Cheng, Xianyi and Ou, Tianyue and Bisk, Yonatan and Fried, Daniel and Alon, Uri and Neubig, Graham , title =

    Zhou, Shuyan and Xu, Frank F. and Zhu, Hao and Zhou, Xuhui and Lo, Robert and Sridhar, Abishek and Cheng, Xianyi and Ou, Tianyue and Bisk, Yonatan and Fried, Daniel and Alon, Uri and Neubig, Graham , title =. The Twelfth International Conference on Learning Representations (ICLR) , year =

  6. [6]

    and Del Verme, Manuel and Marty, Tom and Boisvert, L

    Drouin, Alexandre and Gasse, Maxime and Caccia, Massimo and Laradji, Issam H. and Del Verme, Manuel and Marty, Tom and Boisvert, L. Proceedings of the 41st International Conference on Machine Learning (ICML) , year =

  7. [7]

    The Twelfth International Conference on Learning Representations (ICLR) , year =

    Liu, Xiao and Yu, Hao and Zhang, Hanchen and Xu, Yifan and Lei, Xuanyu and Lai, Hanyu and Gu, Yu and Ding, Hangliang and Men, Kaiwen and Yang, Kejuan and Zhang, Shudan and Deng, Xiang and Zeng, Aohan and Du, Zhengxiao and Zhang, Chenhui and Shen, Sheng and Zhang, Tianjun and Su, Yu and Sun, Huan and Huang, Minlie and Dong, Yuxiao and Tang, Jie , title =. ...

  8. [8]

    Advances in Neural Information Processing Systems (NeurIPS) , year =

    Xie, Tianbao and Zhang, Danyang and Chen, Jixuan and Li, Xiaochuan and Zhao, Siheng and Cao, Ruisheng and Hua, Toh Jing and Cheng, Zhoujun and Shin, Dongchan and Lei, Fangyu and Liu, Yitao and Xu, Yiheng and Zhou, Shuyan and Savarese, Silvio and Xiong, Caiming and Zhong, Victor and Yu, Tao , title =. Advances in Neural Information Processing Systems (Neur...

Show all 40 references
  1. [9]

    2025 , eprint =

    Cohen, Dvir and Burg, Lin and Pykhnivskyi, Sviatoslav and Gur, Hagit and Kovynov, Stanislav and Atzmon, Olga and Barkan, Gilad , title =. 2025 , eprint =

    2025

  2. [10]

    2024 , eprint =

    Friel, Robert and Belyi, Masha and Sanyal, Atindriyo , title =. 2024 , eprint =

    2024

  3. [11]

    Advances in Neural Information Processing Systems: Datasets and Benchmarks Track (NeurIPS) , year =

    Yang, Xiao and Sun, Kai and Xin, Hao and Sun, Yushi and Bhalla, Nikita and Chen, Xiangsen and Choudhary, Sajal and Gui, Rongze Daniel and Jiang, Ziran Will and Jiang, Ziyu and Kong, Lingkun and Moran, Brian and Wang, Jiaqi and Xu, Yifan Ethan and Yan, An and Yang, Chenyu and Y...

  4. [12]

    Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL) , year =

    Saad-Falcon, Jon and Khattab, Omar and Potts, Christopher and Zaharia, Matei , title =. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL) , year =

    2024

  5. [13]

    Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations (EACL) , pages =

    Es, Shahul and James, Jithin and Espinosa-Anke, Luis and Schockaert, Steven , title =. Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations (EACL) , pages =. 2024 , url =

    2024

  6. [14]

    , title =

    Yang, Zhilin and Qi, Peng and Zhang, Saizheng and Bengio, Yoshua and Cohen, William and Salakhutdinov, Ruslan and Manning, Christopher D. , title =. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP) , pages =. 2018 , url =

    2018

  7. [15]

    Transactions of the Association for Computational Linguistics (TACL) , volume =

    Trivedi, Harsh and Balasubramanian, Niranjan and Khot, Tushar and Sabharwal, Ashish , title =. Transactions of the Association for Computational Linguistics (TACL) , volume =. 2022 , url =

    2022

  8. [16]

    Proceedings of the 2025 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL) , pages =

    Krishna, Satyapriya and Krishna, Kalpesh and Mohananey, Anhad and Schwarcz, Steven and Stambler, Adam and Upadhyay, Shyam and Faruqui, Manaal , title =. Proceedings of the 2025 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL) , ...

    2025

  9. [17]

    Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) , pages =

    Zhu, Andrew and Hwang, Alyssa and Dugan, Liam and Callison-Burch, Chris , title =. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) , pages =. 2024 , url =

    2024

  10. [18]

    Proceedings of the First Conference on Language Modeling (COLM) , year =

    Tang, Yixuan and Yang, Yi , title =. Proceedings of the First Conference on Language Modeling (COLM) , year =

  11. [19]

    and Tang, Michael and Sun, Ruoxi and Yoon, Jinsung and Arik, Sercan O

    Su, Hongjin and Yen, Howard and Xia, Mengzhou and Shi, Weijia and Muennighoff, Niklas and Wang, Han-yu and Liu, Haisu and Shi, Quan and Siegel, Zachary S. and Tang, Michael and Sun, Ruoxi and Yoon, Jinsung and Arik, Sercan O. and Chen, Danqi and Yu, Tao , title =. The Thirteen...

  12. [20]

    2023 , eprint =

    Islam, Pranab and Kannappan, Anand and Kiela, Douwe and Qian, Rebecca and Scherrer, Nino and Vidgen, Bertie , title =. 2023 , eprint =

    2023

  13. [21]

    Proceedings of the 10th Workshop on Financial Technology and Natural Language Processing (FinNLP) , pages =

    Lai, Viet Dac and Krumdick, Michael and Lovering, Charles and Reddy, Varshini and Schmidt, Craig and Tanner, Chris , title =. Proceedings of the 10th Workshop on Financial Technology and Natural Language Processing (FinNLP) , pages =. 2025 , url =

    2025

  14. [22]

    Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: Industry Track (EMNLP) , year =

    Lee, Sunwoo and Jang, Daseong and Arya, Dhammiko and Han, Gyoung-eun and Song, Injee and Kim, SaeRom and Kim, Sangjin and Lee, Seojin and Hong, Seokyoung and Sek, Sereimony and Cho, Seung-Mo and Park, Sohee and Yoon, Sungbin and Jang, Wonbeom and Davis, Eric , title =. Proceed...

    2025

  15. [23]

    Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Industry Track (EMNLP) , year =

    Lee, Sunwoo and Arya, Dhammiko and Cho, Seung-Mo and Han, Gyoung-eun and Hong, Seokyoung and Jang, Wonbeom and Lee, Seojin and Park, Sohee and Sek, Sereimony and Song, Injee and Yoon, Sungbin and Davis, Eric , title =. Proceedings of the 2024 Conference on Empirical Methods in...

    2024

  16. [24]

    International Journal of Machine Learning and Cybernetics , year =

    Li, Fei and Wang, Yanyan and Xu, Yin and Wang, Shiling and Liang, Junli and Chen, Zhengyi and Liu, Wenrui and Feng, Qiangzhong and Duan, Ticheng and Huang, Youzhi and Song, Qi and Li, Xiangyang , title =. International Journal of Machine Learning and Cybernetics , year =. doi:...

    doi

  17. [25]

    and Song, Yufan and Li, Boxuan and Tang, Yuxuan and Jain, Kritanjali and Bao, Mengxue and Wang, Zora Z

    Xu, Frank F. and Song, Yufan and Li, Boxuan and Tang, Yuxuan and Jain, Kritanjali and Bao, Mengxue and Wang, Zora Z. and Zhou, Xuhui and Guo, Zhitong and Cao, Murong and Yang, Mingyang and Lu, Hao Yang and Martin, Amaad and Su, Zhe and Maben, Leander Melroy and Mehta, Raj and ...

  18. [26]

    Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL) , year =

    Koh, Jing Yu and Lo, Robert and Jang, Lawrence and Duvvur, Vikram and Lim, Ming Chong and Huang, Po-Yu and Neubig, Graham and Zhou, Shuyan and Salakhutdinov, Ruslan and Fried, Daniel , title =. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguis...

  19. [27]

    The Thirteenth International Conference on Learning Representations (ICLR) , year =

    Yao, Shunyu and Shinn, Noah and Razavi, Pedram and Narasimhan, Karthik , title =. The Thirteenth International Conference on Learning Representations (ICLR) , year =

  20. [28]

    2025 , eprint =

    Barres, Victor and Dong, Honghua and Ray, Soham and Si, Xujie and Narasimhan, Karthik , title =. 2025 , eprint =

    2025

  21. [29]

    2025 , eprint =

    Carmel, David and Filice, Simone and Horowitz, Guy and Maarek, Yoelle and Shtoff, Alex and Somekh, Oren and Tavory, Ran , title =. 2025 , eprint =

    2025

  22. [30]

    , title =

    He, Jie and Hu, Nan and Long, Wanqiu and Chen, Jiaoyan and Pan, Jeff Z. , title =. 2024 , eprint =

    2024

  23. [31]

    Advances in Neural Information Processing Systems: Datasets and Benchmarks Track (NeurIPS) , year =

    Ma, Yubo and Zang, Yuhang and Chen, Liangyu and Chen, Meiqi and Jiao, Yizhu and Li, Xinze and Lu, Xinyuan and Liu, Ziyu and Ma, Yan and Dong, Xiaoyi and Zhang, Pan and Pan, Liangming and Jiang, Yu-Gang and Wang, Jiaqi and Cao, Yixin and Sun, Aixin , title =. Advances in Neural...

  24. [32]

    IEEE Network , year =

    Maatouk, Ali and Ayed, Fadhel and Piovesan, Nicola and De Domenico, Antonio and Debbah, Merouane and Luo, Zhi-Quan , title =. IEEE Network , year =

  25. [33]

    2026 , eprint =

    Ezzakri, Anas and Piovesan, Nicola and Sana, Mohamed and De Domenico, Antonio and Ayed, Fadhel and Zhang, Haozhe , title =. 2026 , eprint =

    2026

  26. [34]

    Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (ACL) , year =

    Ye, Junjie and Du, Zhengyin and Yao, Xuesong and Lin, Weijian and Xu, Yufei and Chen, Zehui and Wang, Zaiyuan and Zhu, Sining and Xi, Zhiheng and Yuan, Siyu and Gui, Tao and Zhang, Qi and Huang, Xuanjing and Chen, Jiecao , title =. Proceedings of the 63rd Annual Meeting of the...

  27. [35]

    2025 , howpublished =

    2025

  28. [36]

    2026 , howpublished =

    2026

  29. [37]

    and Zhang, Hao and Gonzalez, Joseph E

    Zheng, Lianmin and Chiang, Wei-Lin and Sheng, Ying and Zhuang, Siyuan and Wu, Zhanghao and Zhuang, Yonghao and Lin, Zi and Li, Zhuohan and Li, Dacheng and Xing, Eric P. and Zhang, Hao and Gonzalez, Joseph E. and Stoica, Ion , title =. Advances in Neural Information Processing ...

  30. [38]

    Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL) , year =

    Wang, Peiyi and Li, Lei and Chen, Liang and Cai, Zefan and Zhu, Dawei and Lin, Binghuai and Cao, Yunbo and Kong, Lingpeng and Liu, Qi and Liu, Tianyu and Sui, Zhifang , title =. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL) , year =

  31. [39]

    Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) , pages =

    Min, Sewon and Michael, Julian and Hajishirzi, Hannaneh and Zettlemoyer, Luke , title =. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) , pages =. 2020 , url =

    2020

  32. [40]

    Proceedings of the NeurIPS 2020 Competition and Demonstration Track, PMLR , volume =

    Min, Sewon and Boyd-Graber, Jordan and Alberti, Chris and Chen, Danqi and Choi, Eunsol and Collins, Michael and Guu, Kelvin and Hajishirzi, Hannaneh and Lee, Kenton and Palomaki, Jennimaria and others , title =. Proceedings of the NeurIPS 2020 Competition and Demonstration Tra...

    2020

Pith tools

Reviewed August 2, 2026 · model on record in the stance chip above.