Pith. sign in

REVIEW 4 major objections 5 minor 22 references

TradeVerse: A Longitudinal Benchmark of Political Negotiation in International Trade

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read TradeVerse reconstructs 1,170 WTO trade disputes into a three-task longitudinal benchmark and finds that LLMs identify Western responding countries far more accurately than non-Western ones, even when all country names are masked.

desk verdict Genuinely new longitudinal trade-negotiation benchmark, but the headline bias result currently rests on an unvalidated masking assumption and the paper's own numbers don't agree between main text and appendix. read the letter →

arxiv 2608.06549 v1 pith:VL2DF3LL submitted 2026-08-06 cs.CL cs.AIcs.CY

Debodeep Banerjee , Amitangshu Dasgupta This is my paper · ORCID
classification cs.CLcs.AIcs.CY
keywords longitudinalnegotiationWTOtradeconcernsLLMbenchmarkrespondentidentificationgeopoliticalbiasHScodepredictionstatementgenerationdiplomatictext
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

TradeVerse is a benchmark built from 1,170 World Trade Organization “specific trade concern” disputes, reconstructed from 6,933 meeting minutes into multi-party dialogues that unfold over months or years. The paper argues this is the first benchmark for longitudinal political negotiation, and it defines three tasks: predicting product categories (HS chapters), identifying the responding country from anonymized transcripts, and generating the respondent’s final statement. Across six current LLMs, the paper finds that respondent identification is accurate overall (roughly 90 percent) but systematically better for Western members than for the rest of the membership, with the gap persisting under full anonymization. It also reports that models over-predict product categories (high recall, low precision) and generate final statements that are fluent but share little specific content with the actual interventions. If the benchmark is sound, it gives a reusable measurement of how institutional and geopolitical structure enters LLM reasoning on authentic text.

What carries the argument

The central object is the TradeVerse corpus: each trade concern is represented as a chronological sequence of rounds, each round containing statements by raiser, supporter, and respondent, with labels (HS chapters, respondent identity, final statement) recovered directly from WTO proceedings rather than from human annotation. The machinery that carries the argument is the pairing of two anonymization settings with a Western/non-Western partition of respondents; this pairing lets the paper separate substance-based inference from identity-cue leakage. The history-ablation on the statement-generation task, comparing full history against the final round alone, is the complementary mechanism that shows the longitudinal information drives generation-quality gains.

What would settle it

A targeted audit would settle it: take a random sample of reconstructed concerns, check the parsed role labels against the original meeting minutes, and re-run Task 2 on a version of the transcripts from which every product-origin phrase (e.g., ‘China-made electric vehicles’), demonym, and agency name has been removed. If manual inspection finds role misassignment above a few percent, or if the Western-minus-non-Western gap shrinks to near zero when such cues are stripped, the central bias finding is not robust.

Watch

Extended reading notes

Core claim

The paper claims that longitudinal structure carries real signal that LLMs can exploit, and that a stable geopolitical skew in respondent identification exists even when all names are masked. The core evidence is the comparison between two anonymization settings: masking only the respondent versus masking every country. Accuracy stays nearly identical across these two settings, while the Western-minus-non-Western gap remains (usually 5 to 14 percentage points) and sometimes widens under full masking. The paper interprets this as models relying on the substance of the dispute—measures, products, and legal arguments—rather than on surface names, while their internal knowledge still reflects a bias toward Western respondents.

Load-bearing premise

The load-bearing premise is that the pipeline that converts WTO Word documents into structured dialogues assigns roles (raiser, supporter, respondent) correctly and that masking removes every identity cue; if role attribution is noisy, or masking leaves identifying content, the Western-bias and Task 2 results become artifacts.

Editorial extensions

If this is right

  • A model that performs well on TradeVerse must track a concern across multiple meetings rather than answer from a single document, so the benchmark directly measures longitudinal reasoning.
  • The persistent Western/non-Western gap under full masking indicates that LLMs carry internally learned knowledge about trade-dispute participants, which matters for any downstream use in institutional decision support.
  • High recall with low precision on HS chapter prediction shows that current LLMs hedge, listing plausible product categories rather than committing to a precise set.
  • The history-ablation result (five of six models improve with full history) supports the benchmark’s premise that longitudinal context carries signal.
  • Because all labels come from official proceedings without manual annotation, TradeVerse is reusable as a live evaluation suite for future models.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the Western bias is driven by training-data imbalance, the same masking design could be reused as a screening tool: any LLM deployed in trade or legal analysis could be tested for respondent-specific accuracy gaps before deployment.
  • A natural extension the paper leaves implicit is to turn Task 3 into a multi-agent negotiation setting, where separate models play raiser, supporter, and respondent, and to test whether simulated coalitions reproduce the structure of real WTO disputes.
  • Another testable extension is to control for respondent frequency, since the EU is the most frequent respondent; the paper’s EU-ablation still shows the gap, but a per-country accuracy curve would separate frequency effects from geographic ones.
  • A multilingual version of the benchmark could reveal whether the Western advantage is a property of English-language training distributions or a more general geopolitical skew.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The manuscript introduces TradeVerse, a benchmark constructed from 1,170 World Trade Organization Specific Trade Concerns reconstructed as longitudinal multi-party dialogues. It defines three tasks: (1) predicting the HS chapters of the products under discussion, (2) identifying the responding country from transcripts with either only the respondent masked (ST-1) or all country names masked (ST-2), and (3) generating the respondent's final statement. The paper evaluates six contemporary LLMs and reports high overall respondent-identification accuracy, a Western/non-Western accuracy gap that persists under ST-2 masking, over-prediction in HS-code identification, and moderate fluency but low specificity in generated statements. It claims to be the first benchmark for longitudinal political trade negotiations and releases the dataset, extraction pipeline, and evaluation code.

Significance. The benchmark addresses a genuinely underserved area: authentic institutional negotiation texts with a longitudinal structure, and all ground-truth labels are recovered directly from WTO proceedings without manual annotation. The release of the dataset, extraction pipeline, and evaluation code is a concrete and useful contribution. If the Western/non-Western gap under anonymization is real and robust, it is an important measurement of geopolitical bias in LLMs on authentic institutional text. The paper also provides useful evidence that current LLMs over-predict product categories and produce fluent but nonspecific diplomatic statements. However, the central interpretive claim about the persistence of the bias under full anonymization is not yet established, because the masking procedure does not demonstrably remove country-correlated substantive cues.

major comments (4)
  1. [Section 4.3, Task 2 (ST-2); Appendix A.3, Task 2B] The claim that accuracy remains near 90% when all country names are masked, indicating that models rely on substance rather than identity, is not supported by the design of the masking. The prompt in Appendix A.3 (Task 2B) explicitly directs the model to use 'specific trade measures, tariff bound rates, presidential decrees, product sectors, dates, and legal arguments,' and the representative concern in Figures 2 and 8 shows that a 'presidential decree imposing a 40% additional tariff only on electric vehicles originating from China' uniquely identifies Türkiye's 2023 measure even when country names are replaced by placeholders. Country-specific residue such as 'China-made electric vehicles', national programs, and unique legal frameworks therefore remains in the 'anonymized' transcripts. The stability of accuracy between ST-1 and ST-2 does not establish that models use longitudinal substance rather than identity-correlated surface cues. The authors should provide a leakage audit: identify and remove, or at least measure, residual country-specific n-grams, and report Task 2 accuracy on a subset of concerns whose measures are not uniquely attributable to a single member.
  2. [Table 4 vs. Table 8; Section 5] Tables 4 and 8 report incompatible numbers for the same model/setting combinations. For example, DeepSeek-V4-Pro under ST-2 appears as Western 96.98%, non-Western 88.21%, χ²=24.09, p=1×10⁻⁶ in Table 4, but as Western 93.53%, non-Western 88.21%, χ²=3.55, p=0.0597 in Table 8; similar discrepancies occur for every model. The main text (Section 5) claims 'every difference is significant at p < 0.01', but the appendix's EU-ablation results include p=0.0597 (DeepSeek ST-2), p=0.0278 (GLM ST-2), and p=0.0061 (GLM ST-1). The authors need to clarify which table reports the full corpus and which reports the EU-ablation, reconcile the numbers, and correct the significance claims in Section 5 and the Conclusion.
  3. [Section 3, Data collection] The only description of the reconstruction is that the authors 'parsed these documents to organise countries and their roles and corresponding statements', and the paper does not report any validation of this parsing. The respondent and raiser role labels are the ground truth for Task 2 and the input structure for Task 3, so parsing errors propagate directly into all benchmark scores. At minimum, the paper should report the precision of the role-attribution parser on a manually labeled random sample of meetings, together with any filtering rules used to resolve ambiguous or missing role assignments.
  4. [Section 6, Conclusion] The statement that the Western/non-Western gap 'remains when all participant identities are masked and cannot be attributed to surface cues' is stronger than the evidence, because the masking procedure leaves country-correlated substantive cues in place (see the major comment on Section 4.3). The conclusion should be reworded to claim only that the gap persists under the paper's masking procedure, pending the leakage audit.
minor comments (5)
  1. [Section 4.2] The sentence 'for the first task, we evaluate the models’ performance by measuring Accuracy' is inconsistent with Table 2, which reports precision, recall, and F1; it appears that 'second task' is meant.
  2. [Throughout] The benchmark name is spelled inconsistently as 'TradeVerse', 'TRADEVERSE', and 'TradeVerseis' (e.g., in the abstract); please use one consistent spelling.
  3. [Table 1 vs. Appendix A.1] Table 1 reports 1,170 concerns and 6,933 meeting records, while Appendix A.1 reports a 'cleaned corpus' of 1,101 concerns and 6,418 records; the paper should state which corpus is used for each table and why the counts differ.
  4. [Table 8 caption] The caption for Table 8 does not mention that it is the EU-ablation, so it initially appears to be a duplicate of Table 4; the ablation status should be in the caption and referenced explicitly from Section 5.
  5. [Figure 8 caption] The figure caption in the appendix ends with 'shown abridged in All statements are verbatim from the WTO proceedings, with the boilerplate opening omitted.', and the figure reference is missing; please complete the sentence and add the figure number.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: TradeVerse's labels are extracted directly from WTO proceedings, and the model evaluations are independent of label construction.

full rationale

TradeVerse's ground-truth labels are recovered directly from the WTO Trade Concerns Database rather than fitted, so none of the three tasks is a fitted parameter renamed as a prediction. Task 1 uses HS chapters recorded in the source documents; Task 2 uses the respondent country recorded in the WTO minutes; Task 3 withholds a verbatim final statement and evaluates generation against it. The ST-1/ST-2 comparison is an experimental manipulation, not a derivation, and the Western/non-Western split is applied after evaluation using an external country list, so the bias finding is not defined into existence. No load-bearing result rests on a self-citation: the benchmark comparisons are against external models and external baselines, and the data pipeline is described rather than imported from prior work by the same authors. The main caveats are validity threats rather than circularity: the paper does not demonstrate that ST-2 masking removes identity-correlated cues such as product origin (e.g., 'China-made electric vehicles' in Figure 8), and Appendix Table 8 reports DeepSeek-V4-Pro ST-2 p=0.0597, contradicting the main text's claim that 'every difference is significant at p<0.01'. These concern experimental leakage and reporting consistency, but the predictions themselves are not equivalent to their inputs by construction, so the circularity score is 0.

Assumptions & free parameters 0 free parameters · 6 assumptions · 0 invented entities

The benchmark introduces no fitted parameters or invented entities; the zero-shot evaluation has no training. All load-bearing assumptions concern the fidelity of the WTO parsing and masking pipeline and the analytical choices for cohort comparison. These are domain assumptions rather than mathematical axioms.

assumptions (6)
  • domain assumption WTO minutes are authentic verbatim records
    The entire benchmark is constructed from the WTO Trade Concerns Database; Section 3.
  • domain assumption Color-based parsing of Word documents correctly assigns statements to roles
    Section 3 Data collection states country names are written in different colors and parsed to organize countries and roles; no validation is reported.
  • domain assumption Anonymization removes all identity cues
    Section 3 Task 2 claims all country names, demonyms, and national agencies are masked; no checks for residual cues like product origins are reported.
  • ad hoc to paper Pritchard and Wallace (2011) defines an appropriate Western bloc
    Section 5 uses this classification for the geopolitical bias analysis; it includes Japan but excludes several OECD states and is taken from a health-policy context.
  • domain assumption BERTScore-F1 with deberta-large-mnli is a valid semantic metric for diplomatic statements
    Section 4.2 selects this model for semantic similarity; small differences (e.g., 57.83 vs 60.49) are treated as meaningful.
  • domain assumption HS chapters are the correct product taxonomy for TBT and SPS disputes
    Task 1 uses HS chapters as labels, but only 2 of 5 committees provide them, so the task covers a subset of the corpus.

how reviews work

0 comments
Cite this review

Pith. "Pith review of TradeVerse: A Longitudinal Benchmark of Political Negotiation in International Trade." pith.science (2026). https://pith.science/paper/VL2DF3LL

@misc{pith2026260806549,
  author       = {Pith},
  title        = {Pith review of: TradeVerse: A Longitudinal Benchmark of Political Negotiation in International Trade},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VL2DF3LL}},
  note         = {Machine review of arXiv:2608.06549}
}
abstract

LLMs are increasingly being applied to tasks involving institutional and political texts, but existing benchmarks evaluate them on isolated documents or single tasks. In realpolitik, negotiations are longitudinal data, where participating parties can align or argue over multiple iterations and each turn is an outcome of the previous turns, hence, understanding one turn requires tracking everything before it. We introduce TradeVerse, a benchmark built from the World Trade Organisation (WTO) specific trade concerns, where member states challenge one another and exchange arguments over multiple rounds, sometimes for years. We, in TradeVerse, reconstruct minutes of $1170$ meetings, spanning across 5 groups and $89$ product groups and define three tasks: first, the system has to analyze the longitudinal meeting records and predict the harmonized system codes (HS chapters) of the products under discussion in the particular meeting, second, we examine whether the system, upon analyzing the anonymized content of the meeting, can guess the name of the responding country and third, we ask the system to play the role of the responding country and provide the statement for the very last round. All labels are recovered directly from the proceedings, requiring no manual annotation. Our experiments highlight the challenges these tasks pose for current LLMs. To the best of our knowledge, TradeVerseis the first benchmark to investigate potential of LLMs in understanding longitudinal political trade negotiations.

Figures

Figures reproduced from arXiv: 2608.06549 by the authors.

Figure 1
Figure 1. Overview of the TRADEVERSE benchmark pipeline. These streams feed into three core evaluation tasks: predicting prod￾uct categories (Task 1), identifying responding members under anonymization (Task 2), and generating the concluding statement of the respondent (Task 3). to put forward its arguments – stretching the meetings to several rounds, sometimes spanning years. Therefore, in such cases, the LLMs should take in… view at source ↗
Figure 2
Figure 2. A representative TRADEVERSE concern across three meetings (abridged). China raises the same concern repeatedly while Turkiye’s position develops from acknowledgement to a substantive infant-industry defence. The product (Task 1) and ¨ responding member (Task 2) are inferred from the exchange; the respondent’s final statement (Task 3) is generated from the preceding history. • We evaluate six contemporary LLMs and id… view at source ↗
Figure 3
Figure 3. Frequency of top-15 HS chapters as per their frequency in the corpus. [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Distribution of concerns across total number of years it lasted for [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Comparison of average length of predictions against the average length of ground truth prediction. The [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Mean word count vs number of concerns. Concern threads are heavy-tailed in length. [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]
Figure 7
Figure 7. Figure 7: Distribution of the corpus across the five WTO groups. . [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]
Figure 8
Figure 8. Figure 8: The complete three-meeting dialogue for the concern shown abridged in All statements are verbatim from the WTO [PITH_FULL_IMAGE:figures/full_fig_p015_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

22 extracted references · 8 canonical work pages

  1. [1]

    Gpt-4 technical report.arXiv preprint arXiv:2303.08774,

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report.arXiv preprint arXiv:2303.08774,

    arXiv

  2. [4]

    How well can llms negotiate? negotiationarena platform and analysis.arXiv preprint arXiv:2402.05863,

    Federico Bianchi, Patrick John Chia, Mert Yuksekgonul, Jacopo Tagliabue, Dan Jurafsky, and James Zou. How well can llms negotiate? negotiationarena platform and analysis.arXiv preprint arXiv:2402.05863,

    arXiv

  3. [7]

    Bigbench: Towards an industry standard benchmark for big data analytics

    Ahmad Ghazal, Tilmann Rabl, Minqing Hu, Francois Raab, Meikel Poess, Alain Crolotte, and Hans-Arno Jacobsen. Bigbench: Towards an industry standard benchmark for big data analytics. InProceedings of the 2013 ACM SIGMOD international conference on Management of data, pages 1197–1208,

    work page 2013

  4. [10]

    Are llms effective negotiators? systematic evaluation of the multifaceted capabilities of llms in negotiation dialogues

    Deuksin Kwon, Emily Weiss, Tara Kulshrestha, Kushal Chawla, Gale Lucas, and Jonathan Gratch. Are llms effective negotiators? systematic evaluation of the multifaceted capabilities of llms in negotiation dialogues. InFindings of the Association for Computational Linguistics: EMNLP 2024, pages 5391–5413,

    work page 2024

  5. [13]

    Kimi-vl technical report.arXiv preprint arXiv:2504.07491,

    Kimi Team, Angang Du, Bohong Yin, Bowei Xing, Bowen Qu, Bowen Wang, Cheng Chen, Chenlin Zhang, Chen- zhuang Du, Chu Wei, et al. Kimi-vl technical report.arXiv preprint arXiv:2504.07491,

    arXiv

  6. [14]

    Tradingagents: Multi-agents llm financial trading framework.arXiv preprint arXiv:2412.20138,

    Yijia Xiao, Edward Sun, Di Luo, and Wei Wang. Tradingagents: Multi-agents llm financial trading framework.arXiv preprint arXiv:2412.20138,

    arXiv

  7. [15]

    Deepseek-v4: Towards highly efficient million-token context intelligence.arXiv preprint arXiv:2606.19348,

    Anyi Xu, Bangcai Lin, Bing Xue, Bingxuan Wang, Bingzheng Xu, Bochao Wu, Bowei Zhang, Chaofan Lin, Chen Dong, Chenchen Ling, et al. Deepseek-v4: Towards highly efficient million-token context intelligence.arXiv preprint arXiv:2606.19348,

    arXiv

  8. [16]

    Glm-5: from vibe coding to agentic engineering.arXiv preprint arXiv:2602.15763,

    Aohan Zeng, Xin Lv, Zhenyu Hou, Zhengxiao Du, Qinkai Zheng, Bin Chen, Da Yin, Chendi Ge, Chenghua Huang, Chengxing Xie, et al. Glm-5: from vibe coding to agentic engineering.arXiv preprint arXiv:2602.15763,

    arXiv

Show all 22 references
  1. [17]

    Bertscore: Evaluating text generation with bert.arXiv preprint arXiv:1904.09675,

    Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. Bertscore: Evaluating text generation with bert.arXiv preprint arXiv:1904.09675,

    1904 arXiv

  2. [19]

    A Appendix A.1 Additional Data Analysis This section provides further descriptive analysis of the TRADEVERSEcorpus. We first characterise the length of concern threads (Figure 6), which motivates the long-context demands of the benchmark, and then examine how the corpus is dis...

    1995

  3. [100]

    China believes this to be inconsistent with the WTO rules

    Meeting 1 — Formal Meeting of 26–27 April 2023 China (raiser):On 3 March, without explanation, T ¨urkiye sharply increased import tariffs on China-made electric vehicles. China believes this to be inconsistent with the WTO rules. First, the relevant measure violates Article II...

    2023

  4. [1994]

    The import tariff of China-made electric vehicles has reached 50%, significantly exceeding T ¨urkiye’s tariff commitment

    According to T ¨urkiye’s tariff commitment, the bound rate for electric vehicles is 20%. The import tariff of China-made electric vehicles has reached 50%, significantly exceeding T ¨urkiye’s tariff commitment. Second, the relevant measure seriously violates the WTO MFN princi...

    2023

  5. [2004]

    Can large language models trade? testing financial theories with llm agents in market simula- tions.arXiv preprint arXiv:2504.10789,

    Alejandro Lopez-Lira. Can large language models trade? testing financial theories with llm agents in market simula- tions.arXiv preprint arXiv:2504.10789,

    arXiv

  6. [2011]

    When agents trade: Live multi-market trading arena for llm agents

    Lingfei Qian, Xueqing Peng, Hanley Smith, Yi Han, Yueru He, Haohang Li, Yupeng Cao, Yangyang Yu, Guojun Xiong, Peng Lu, et al. When agents trade: Live multi-market trading arena for llm agents. InProceedings of the ACM Web Conference 2026, pages 7833–7844,

    2026

  7. [2013]

    The llama 3 herd of models.arXiv preprint arXiv:2407.21783,

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models.arXiv preprint arXiv:2407.21783,

    arXiv

  8. [2019]

    Policon: Evaluating llms on achieving diverse political consensus objectives.arXiv preprint arXiv:2505.19558,

    Zhaowei Zhang, Xiaobo Wang, Minghua Yi, Mengmeng Wang, Fengshuo Bai, Zilong Zheng, Yipeng Kang, and Yaodong Yang. Policon: Evaluating llms on achieving diverse political consensus objectives.arXiv preprint arXiv:2505.19558,

  9. [2021]

    Wenyue Hua, Ollie Liu, Lingyao Li, Alfonso Amayuelas, Julie Chen, Lucas Jiang, Mingyu Jin, Lizhou Fan, Fei Sun, William Wang, et al

    URL https://openreview.net/forum?id= XPZIaotutsD. Wenyue Hua, Ollie Liu, Lingyao Li, Alfonso Amayuelas, Julie Chen, Lucas Jiang, Mingyu Jin, Lizhou Fan, Fei Sun, William Wang, et al. Game-theoretic llm: Agent workflow for negotiation games.arXiv preprint arXiv:2411.05990,

    arXiv

  10. [2022]

    Having said that, T ¨urkiye has no intention of discriminating against any Member

    Despite the introduction of the increased MFN tariff rates, an import surge continued, predominantly from China among the non-preferential trade partners. Having said that, T ¨urkiye has no intention of discriminating against any Member. We took note of the concerns expressed ...

    2024

  11. [2023]

    gpt-oss-120b & gpt-oss-20b model card.arXiv preprint arXiv:2508.10925,

    9 TRADEVERSEA PREPRINT Sandhini Agarwal, Lama Ahmad, Jason Ai, Sam Altman, Andy Applebaum, Edwin Arbus, Rahul K Arora, Yu Bai, Bowen Baker, Haiming Bao, et al. gpt-oss-120b & gpt-oss-20b model card.arXiv preprint arXiv:2508.10925,

    arXiv

  12. [2024]

    Nemotron 3 ultra: Open, efficient mixture-of- experts hybrid mamba-transformer model for agentic reasoning.arXiv preprint arXiv:2606.15007,

    Aaron Blakeman, Aaron Thomas, Aastha Jhunjhunwala, Abhibha Gupta, Abhinav Khattar, Adam Rajfer, Adi Ren- duchintala, Adil Asif, Aditya Vavre, Adriana Flores Miranda, et al. Nemotron 3 ultra: Open, efficient mixture-of- experts hybrid mamba-transformer model for agentic reasoni...

  13. [2025]

    Longbench: A bilingual, multitask benchmark for long context understanding.arXiv preprint arXiv:2308.14508,

    Yushi Bai, Xin Lv, Jiajie Zhang, Hongchang Lyu, Jiankai Tang, Zhidian Huang, Zhengxiao Du, Xiao Liu, Aohan Zeng, Lei Hou, et al. Longbench: A bilingual, multitask benchmark for long context understanding.arXiv preprint arXiv:2308.14508,

    arXiv

  14. [2026]

    Stockbench: Can llm agents trade stocks profitably in real-world markets?arXiv preprint arXiv:2510.02209,

    Yanxu Chen, Zijun Yao, Yantao Liu, Amy Xin, Jin Ye, Jianing Yu, Lei Hou, and Juanzi Li. Stockbench: Can llm agents trade stocks profitably in real-world markets?arXiv preprint arXiv:2510.02209,

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.