MPL: Multiple Programming Languages with Large Language Models for Information Extraction

Bo Li; Gexiang Fang; Hao Cheng; Jinglei Zhang; Shikun Zhang; Wei Ye; Zhenghua Xu

REVIEW 3 major objections 6 minor 61 references

MPL: Multiple Programming Languages with Large Language Models for Information Extraction

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Fine-tuning a language model on the same information-extraction task written in Python, C++, and Java yields higher accuracy than fine-tuning on Python alone, and a new function-prompt with virtual running makes the code-style input…

desk verdict Multi-PL training beats Python-only code-style IE by about a point across backbones, and the controls make a credible case that the gain is not just ensembling; the causal story is plausible but unverified, and the paper needs a round of reporting fixes. read the letter →

arxiv 2505.16107 v1 pith:3LR3EJKS submitted 2025-05-22 cs.CL

Bo Li , Gexiang Fang , Wei Ye , Zhenghua Xu , Jinglei Zhang , Hao Cheng , Shikun Zhang This is my paper

classification cs.CL

keywords informationextractioncode-stylepromptingmultipleprogramminglanguagessupervisedfine-tuningfunction-promptvirtualrunningstructuredoutputgenerationlargelanguagemodels

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that code-style prompting for information extraction need not be tied to a single programming language. It claims that encoding the same task, label set, and input text as a function in Python, C++, and Java, and fine-tuning a language model on all three, yields better Micro-F1 than fine-tuning on Python alone. The paper also proposes a lighter prompt format, the function-prompt with virtual running, that wraps the task into one function and simulates its execution on the input text; this format cuts tokenized input length by about 20% compared with the class-prompt while slightly improving accuracy. Across eleven NER, relation, and event extraction datasets, an 8B-parameter model trained this way averages 77.6 Micro-F1, 1.2 points above its Python-only baseline and 1.1 points above a 34B-parameter rival. If the claim holds, the syntactic diversity of programming languages is itself a training signal for structured extraction, not merely a formatting convenience.

What carries the argument

The functioning engine of the method is the function-prompt with a 'virtual running' step. A named function such as Named_Entity_Recognition(InputText) is defined with the task definition and full label set inside its docstring, and the text to be analyzed is supplied as the argument; generation then proceeds as if the function had executed and returned a list, with every extracted item appended via calls such as EntityList.append(GPE("Iraq")). This replaces the class-prompt's boilerplate constructors and repeated attribute assignments, shrinking tokenized input by roughly 20% while adding 0.4 Micro-F1 on the default setting. Around that core, the framework's distinctive element is the use of three syntactic shells, Python, C++, and Java, that share identical docstring content and execution order, differing in type declarations, container syntax, and how outputs are appended. The three templates are shuffled during fine-tuning, and predictions from all three are aggregated by voting at test time.

What would settle it

A reader could test the language-diversity claim by keeping the docstrings byte-identical across Python, C++, and Java and varying only the syntactic boilerplate; if the advantage over a single-language model disappears under that control, the effect is not caused by programming-language diversity. A second observation would be to train on a Python-only dataset tripled in size under the exact hyperparameters used for MPL; the paper reports 76.1 for that setting, so reproducing that drop would support the claim, while matching the MPL score would undermine it.

Watch

Extended reading notes

Core claim

The central discovery is that fine-tuning on the same extraction task expressed in several programming languages produces a measurable, statistically significant improvement over fine-tuning on a single language. The mechanism is the function-prompt: each task becomes a single function whose docstring carries the task definition and label set, and generation proceeds as if the function had been 'virtually run' on the input text, with each extracted item appended to a returned list. The authors generate three semantically matched templates for every input, one in Python, one in C++, and one in Java, following the same design logic and execution flow, and shuffle them during training. Test-time predictions from the three languages are combined by majority voting, reaching 77.6 average Micro-F1 on eleven benchmarks; a Python-only model trained under identical settings scores 76.4, tripling the Python data does not reproduce the gain, and ensembling random seeds or reordering labels gives at most 76.63. The three languages also produce complementary errors, since the union of their predictions scores 79.5, with the largest union-voting gaps on relation extraction and event extraction.

Load-bearing premise

The load-bearing premise is that the Python, C++, and Java templates carry the same task and label information and differ only in programming-language syntax; if one template inadvertently includes richer label descriptions, the gains could come from that content rather than from language diversity.

Editorial extensions

If this is right

If MPL's claim holds, code-style IE systems can stop treating Python as the default; training on multiple languages is a cheap way to raise extraction F1 without growing the model.
Function-prompt is shorter and slightly more accurate than class-prompt, so switching prompt format offers a 20% reduction in tokenized input length, and a proportional reduction in per-example training cost, while improving performance.
The union of the three languages' outputs exceeds voting by about 2 points, implying that better aggregation, such as confidence weighting or a learned combiner, could raise results further without retraining.
The zero-shot gains on datasets absent from training suggest that multi-language supervision transfers to new schemas and domains, not just to the eleven seen benchmarks.
The reported 77.6 average comes at more than double the training cost of the single-language setting, so the method is a compute-for-accuracy trade rather than a free lunch.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the gap between MPL_sampled, which trains on one randomly chosen language per example and scores 77.1, and full MPL at 77.6 is confirmed at larger scale, then most of the benefit comes from diversity rather than token volume; sampling one language per example could capture most of the gain at a third of the training cost.
The three languages are a convenience sample; measuring the marginal F1 added by a fourth syntactic shell, such as Rust, Go, or SQL, would reveal whether the benefit saturates or continues to grow.
The virtual-running step resembles program-trace prompting; if its 0.4-point gain replicates on other structured-output tasks, the same 'define a function, run it on the input, append to a list' device may transfer to spreadsheet generation, database query construction, or configuration reasoning.
The union-voting gap of up to 16 points on relation extraction suggests the three languages are not redundant; a testable extension is to have the model output per-prediction confidence scores and train a small aggregator on them, which could beat the default voting scheme without changing the fine-tuning data.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Read the MPL paper. The headline result is that fine-tuning an LLM on code-style IE inputs written in Python, C++, and Java beats the same training with Python alone: LLaMA3-8B hits 77.6 average Micro-F1 across 11 IE datasets, +1.2 over the Python-only baseline and +1.1 over GoLLIE-34B. The idea is genuinely new—prior code-style IE work (GoLLIE, KnowCoder, CodeIE, Code4Struct) is Python-only—and the function-prompt with virtual running is a nice efficiency win (about 20% shorter inputs). The experimental breadth is impressive: multiple backbones, zero-shot tests, and a careful ablation study in Section 5.2 that attempts to rule out ensembling and prompt-perturbation explanations. That section is a real methodological contribution.

The main soft spot is the causal claim. The paper attributes the gain to “programming language diversity,” but the three templates are not pure syntactic variants: the C++ and Java versions wrap the logic in a main()/class and use typed lists, while the Python version is top-level. The authors state as a design guideline that the logic and execution flow are consistent, but they never verify information equivalence. A control that trains on, say, the C++ template plus two syntactic variants of that same template (e.g., different type spellings or variable names) would directly test whether structural diversity—not just content differences—is what matters. Without that, a skeptical reader can say the gain comes from extra boilerplate or output-format cues. The single-PL results are close (76.4/76.6/76.4), which mitigates the concern, but the p-value claim in Table 1 makes it worse: they say t-test p<0.05 but report no standard deviations or test statistics anywhere. You cannot check that.

Other reporting issues: the abstract says “we have released our code” while the conclusion says “we will release… later”; Appendix B’s baseline table has question marks instead of citations for most entries; and the appendix figures are mislabeled—Figure 4 is captioned “Python” but shows C++ code, and Figure 5 is also C++, so no Python example appears. These are all fixable.

Who is this for? Anyone working on code-style IE or structured output generation with LLMs will want to know this result. It deserves a serious referee; the empirical foundation is solid enough that the paper should not be desk-rejected. I’d recommend conditional acceptance after the authors provide variance estimates, correct the figure captions and code-release statement, and ideally run the syntactic-variant control. If they show the gain survives that control, the causal story stands.

Referee Report

3 major / 6 minor

Summary. The paper proposes MPL, a supervised fine-tuning framework for information extraction that reformulates each IE instance as code-style input in three programming languages (Python, C++, and Java) and trains an LLM on the shuffled mixture. It also introduces function-prompt with a 'virtual running' step as a more compact alternative to the class-prompt used by prior work. The authors evaluate on 11 IE datasets spanning NER, RE, EAE, and EE, using several backbone LLMs, and report that MPL-8B reaches 77.6 average Micro-F1, exceeding the GoLLIE-34B baseline by 1.1 points and its own Python-only baseline by 1.2 points. Additional experiments cover zero-shot transfer, ablations of single vs. multiple PLs, data-size controls, function-prompt vs. class-prompt efficiency, and complementarity analyses across PLs.

Significance. If the central claim holds, MPL is a useful and low-cost recipe for improving code-style IE: it works across multiple backbones, adds no external pretraining data, and comes with a more token-efficient prompt format. The paper's strengths are its breadth of evaluation (11 datasets, multiple LLMs), the explicit ablation of ensemble and data-size confounds in Section 5.2, the zero-shot assessment, and the release of code. The main weakness is that the causal interpretation—that programming-language diversity, rather than incidental template-content differences, drives the +1.2 F1 gain—is not yet backed by a matched control experiment. The statistical reporting of the claimed p-values is also incomplete.

major comments (3)

[Section 5.2, Table 5] The central attribution of the +1.2 F1 gain to 'programming language diversity' is not supported by the current controls. The paper never verifies that the Python, C++, and Java templates are semantically equivalent beyond the design guideline stated in Section 2.3. The Appendix C examples show that the templates differ not only in surface syntax but in execution scaffolding: Python returns an EntityList from a function, C++ adds a main() with push_back calls, and Java wraps everything in a class with List.add. These differences could enrich the C++/Java inputs with additional structural cues or alter output presentation. The prompt-variation baselines in Table 5 (label reordering, format variation) are all applied within Python and do not isolate language identity from template content. A decisive control would be to train on the Python template plus two syntactic transliterations of that same template into C++ and Java, preserving the label descriptions and code structure as much as the target language allows. Without this condition, the observed gain could partly reflect content differences rather than PL diversity.
[Section 4.1, Table 1 caption] The claim that results are 'statistically significant with a p-value of less than 0.05' is not verifiable from the paper. Only five-run averages are reported, with no standard deviations, no test statistic, no statement of whether the t-tests are paired or unpaired, and no correction for the multiple comparisons across 11 datasets and multiple model pairs. Since the reader cannot reconstruct the tests, the significance claim should be moved to an appendix with full details, or the averages should be accompanied by per-dataset variances. This is important because several reported differences (e.g., 93.5 vs. 93.1 on CoNLL03) are very small in absolute terms.
[Appendix C, Figures 4-6] The appendix does not currently provide the faithful per-language exemplars that the method needs. Figure 4 is captioned as the Python input but displays C++ code (vector<Entity*>, const string&, int main); Figure 5 repeats the same C++ block; Figure 6 contains the invalid token 'GPEe'. These artifacts prevent a reader from knowing the actual Python template and undermine the template-equivalence assumption at the point where it could be checked. The authors should correct the figures against the released templates, or acknowledge the duplication and provide the true Python example.

minor comments (6)

[Abstract / Conclusion] The Abstract states that code has been released, while the Conclusion says the code 'will be released later'; the GitHub link in the footnote should be reconciled with these statements.
[Appendix B, Table 8] Several SLM baseline results are listed with '?' in the reference column; full citations should be provided for all reported baseline numbers.
[Table 6] The table header contains the typo 'Avg.Socre'; also, the 'Union' aggregation is optimistic by construction (a prediction is correct if any PL is correct), so the discussion in Section 5.4 should state more clearly that this is an upper-bound style metric, not a deployable inference strategy.
[Figure 3] The text says the comparison is between class-prompt and function-prompt but does not explicitly state that both use the same language (presumably Python-only); this should be stated so the reader can relate it to the multi-PL results.
[Table 7] The Jaccard similarity computation is not defined precisely: it should specify whether it is over gold or predicted spans, and how partial matches are handled.
[Table 1] The KnowCoder column is missing OntoNotes5* and RAMS values; the footnote explains the alignment issue, but the headline 'competitive with KnowCoder' should be qualified in the text for these datasets.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: MPL's central results are empirical benchmark comparisons against external baselines, with explicit controls for data size and ensemble effects.

full rationale

The paper's main claim is that fine-tuning on Python, C++, and Java code-style inputs improves IE performance over Python-only training. This is supported by measured Micro-F1 scores across 11 datasets, compared with externally published GoLLIE and KnowCoder baselines, rather than by a derivation from a fitted quantity. The ablation study in Section 4.3 and Table 5 explicitly tests alternative explanations: Python 3x data, MPLsampled, seed ensembles, label reordering, prompt format variation, and PL ensembling. These controls address the concern that the multi-PL gain is merely an artifact of data size or voting. The only identified soft spot is the unverified assumption in Section 2.3 that the three hand-written templates are semantically equivalent, differing only in syntax. That is an experimental-design limitation or correctness risk, not circularity: no equation defines the reported F1 in terms of the templates, no fitted parameter is renamed as a prediction, and the result is not forced by construction. Self-citations appear only as background references and do not carry the load of the empirical comparison. Therefore, no circular step can be exhibited from the paper's text, and the circularity score is 0.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No free parameters are fit to data; the central claim rests on dataset assumptions and the unverified equivalence of the three PL templates. No new entities are introduced.

assumptions (4)

domain assumption Code-style input representations improve structured output generation over natural-language prompts for LLMs.
Inherited from GoLLIE, CodeIE, and Code4Struct; not proven here, and it motivates the entire approach.
ad hoc to paper The Python, C++, and Java templates for the same IE task are semantically equivalent (same task definition, label descriptions, input text) and differ only in syntax.
Section 2.3 gives transformation guidelines but provides no verification of informational equivalence. If templates differ in content, the comparison between single-PL and MPL is confounded.
domain assumption Standard IE dataset splits and Micro-F1 are appropriate measures of system quality.
The paper follows prior IE evaluation practice and does not question the validity of these benchmarks.
domain assumption The OntoNotes 5.0 dataset aligned to the GOLLIE setup is a fair benchmark for comparing MPL with systems trained on different OntoNotes splits.
The paper uses an asterisk to acknowledge the setup differs from KnowCoder's, so cross-system comparability on this dataset is limited.

how reviews work

0 comments

Cite this review

Pith. "Pith review of MPL: Multiple Programming Languages with Large Language Models for Information Extraction." pith.science (2026). https://pith.science/paper/3LR3EJKS

@misc{pith2026250516107,
  author       = {Pith},
  title        = {Pith review of: MPL: Multiple Programming Languages with Large Language Models for Information Extraction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3LR3EJKS}},
  note         = {Machine review of arXiv:2505.16107}
}

read the original abstract

Recent research in information extraction (IE) focuses on utilizing code-style inputs to enhance structured output generation. The intuition behind this is that the programming languages (PLs) inherently exhibit greater structural organization than natural languages (NLs). This structural advantage makes PLs particularly suited for IE tasks. Nevertheless, existing research primarily focuses on Python for code-style simulation, overlooking the potential of other widely-used PLs (e.g., C++ and Java) during the supervised fine-tuning (SFT) phase. In this research, we propose \textbf{M}ultiple \textbf{P}rogramming \textbf{L}anguages with large language models for information extraction (abbreviated as \textbf{MPL}), a novel framework that explores the potential of incorporating different PLs in the SFT phase. Additionally, we introduce \texttt{function-prompt} with virtual running to simulate code-style inputs more effectively and efficiently. Experimental results on a wide range of datasets demonstrate the effectiveness of MPL. Furthermore, we conduct extensive experiments to provide a comprehensive analysis. We have released our code for future research.

Figures

Figures reproduced from arXiv: 2505.16107 by the authors.

**Figure 2.** Our framework utilizes multiple programming languages, i.e, Python, C++, and Java, to convert elements [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** Performance and training statistics for differ [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Figures from the paper (3 more)

**Figure 4.** Figure 4: The detailed input using Python and function-prompt on ACE05-NER dataset. vector<Entity*> Named_Entity_Recognition(const string& InputText) { /* Task: Named entity recognition (NER) involves identifying and classifying named entities in text into predefined categories.…

**Figure 5.** Figure 5: The detailed input using C++ and function-prompt on ACE05-NER dataset [PITH_FULL_IMAGE:figures/full_fig_p015_5.png]

**Figure 6.** Figure 6: The detailed input using Java and function-prompt on ACE05-NER dataset [PITH_FULL_IMAGE:figures/full_fig_p016_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

61 extracted references · 21 canonical work pages

[1]

online" 'onlinestring :=

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...
[2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
[3]

Zhen Bi, Jing Chen, Yinuo Jiang, Feiyu Xiong, Wei Guo, Huajun Chen, and Ningyu Zhang. 2024. Codekgc: Code language model for generative knowledge graph construction. ACM Transactions on Asian and Low-Resource Language Information Processing, 23(3):1--16

work page 2024
[4]

Bruno Blanchet. 1999. Escape analysis for object-oriented languages: application to java. Acm Sigplan Notices, 34(10):20--34

work page 1999
[5]

Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert - Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litw...

2020
[7]

Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. 2021 b . Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374

arXiv 2021
[8]

Leon Derczynski, Kalina Bontcheva, and Ian Roberts. 2016. Broad twitter corpus: A diverse named entity recognition resource. In Proceedings of COLING 2016, the 26th international conference on computational linguistics: technical papers, pages 1169--1179

work page 2016
[9]

Leon Derczynski, Eric Nichols, Marieke Van Erp, and Nut Limsopatham. 2017. Results of the wnut2017 shared task on novel and emerging entity recognition. In Proceedings of the 3rd Workshop on Noisy User-generated Text, pages 140--147

2017

Show all 61 references

[10]

Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2024. Qlora: Efficient finetuning of quantized llms. Advances in Neural Information Processing Systems, 36

2024
[11]

Jacob Devlin, Ming - Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. https://doi.org/10.18653/v1/n19-1423 BERT: pre-training of deep bidirectional transformers for language understanding . In NAACL-HLT 2019 , pages 4171--4186. Association for Computational Linguistics

2019 doi
[12]

Rezarta Islamaj Dogan and Zhiyong Lu. 2012. An improved corpus of disease mentions in pubmed citations. In BioNLP: Proceedings of the 2012 Workshop on Biomedical Natural Language Processing, pages 91--99

2012
[13]

J Michael Dunn. 2019. Natural language versus formal language. New Essays on Belnap-Dunn Logic, pages 13--19

2019
[14]

Seth Ebner, Patrick Xia, Ryan Culkin, Kyle Rawlins, and Benjamin Van Durme. 2019. Multi-sentence argument linking. arXiv preprint arXiv:1911.03766

2019 arXiv
[15]

Hermenegildo Fabregat, Juan Martinez-Romo, and Lourdes Araujo. 2018. Overview of the diann task: Disability annotation task. In IberEval@ SEPLN, pages 1--14

2018
[16]

Chufan Gao, Xulin Fan, Jimeng Sun, and Xuan Wang. 2024. Promptre: Weakly-supervised document-level relation extraction via prompting-based data programming. In Proceedings of the 1st Workshop on Towards Knowledgeable Language Models (KnowLLM 2024), pages 132--145

2024
[17]

Yucan Guo, Zixuan Li, Xiaolong Jin, Yantao Liu, Yutao Zeng, Wenxuan Liu, Xiang Li, Pan Yang, Long Bai, Jiafeng Guo, et al. 2023. Retrieval-augmented code generation for universal information extraction. arXiv preprint arXiv:2311.02962

2023 arXiv
[18]

Ridong Han, Tao Peng, Chaohao Yang, Benyou Wang, Lu Liu, and Xiang Wan. 2023. Is information extraction solved by chatgpt? an analysis of performance, evaluation criteria, robustness and errors. arXiv preprint arXiv:2305.14450

2023 arXiv
[19]

Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021. https://openreview.net/forum?id=d7KBjmI3GmQ Measuring massive multitask language understanding . In 9th International Conference on Learning Representations, ICLR 2021,...

2021
[20]

Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, et al. 2022. Training compute-optimal large language models. arXiv preprint arXiv:2203.15556

2022 arXiv
[21]

Gustaf Holst and Felix Dobslaw. 2021. On the importance and shortcomings of code readability metrics: A case study on reactive programming. arXiv preprint arXiv:2110.15246

2021 arXiv
[22]

Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Keming Lu, et al. 2024. Qwen2. 5-coder technical report. arXiv preprint arXiv:2409.12186

2024 arXiv
[23]

Paul Jaccard. 1912. The distribution of the flora in the alpine zone. 1. New phytologist, 11(2):37--50

1912
[24]

Pengcheng Jiang, Jiacheng Lin, Zifeng Wang, Jimeng Sun, and Jiawei Han. 2024. Genres: Rethinking evaluation for generative relation extraction in the era of large language models. arXiv preprint arXiv:2402.10744

2024 arXiv
[25]

Bo Li, Gexiang Fang, Yang Yang, Quansen Wang, Wei Ye, Wen Zhao, and Shikun Zhang. 2023 a . Evaluating chatgpt's information extraction capabilities: An assessment of performance, explainability, calibration, and faithfulness. arXiv preprint arXiv:2304.11633

2023 arXiv
[26]

Bo Li, Wei Ye, Quansen Wang, Wen Zhao, and Shikun Zhang. 2024 a . Labels need prompts too: mask matching for natural language understanding tasks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 18453--18461

2024
[27]

Bo Li, Wei Ye, Jinglei Zhang, and Shikun Zhang. 2023 b . Reviewing labels: Label graph network with top-k prediction set for relation extraction. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 13051--13058

2023
[28]

Bo Li, Dingyao Yu, Wei Ye, Jinglei Zhang, and Shikun Zhang. 2023 c . Sequence generation with label augmentation for relation extraction. In Proceedings of the AAAI conference on artificial intelligence, volume 37, pages 13043--13050

2023
[29]

Guozheng Li, Zijie Xu, Ziyu Shang, Jiajun Liu, Ke Ji, and Yikai Guo. 2024 b . Empirical analysis of dialogue relation extraction with large language models. arXiv preprint arXiv:2404.17802

2024 arXiv
[30]

Peng Li, Tianxiang Sun, Qiong Tang, Hang Yan, Yuanbin Wu, Xuanjing Huang, and Xipeng Qiu. 2023 d . Codeie: Large code generation models are better few-shot information extractors. arXiv preprint arXiv:2305.05711

2023 arXiv
[31]

Raymond Li, Loubna Ben Allal, Yangtian Zi, Niklas Muennighoff, Denis Kocetkov, Chenghao Mou, Marc Marone, Christopher Akiki, Jia Li, Jenny Chim, et al. 2023 e . Starcoder: may the source be with you! arXiv preprint arXiv:2305.06161

2023 arXiv
[32]

Sha Li, Heng Ji, and Jiawei Han. 2021. Document-level event argument extraction by conditional generation. arXiv preprint arXiv:2104.05919

2021 arXiv
[33]

Xian Li, Ping Yu, Chunting Zhou, Timo Schick, Luke Zettlemoyer, Omer Levy, Jason Weston, and Mike Lewis. 2023 f . Self-alignment with instruction backtranslation. arXiv preprint arXiv:2308.06259

2023 arXiv
[34]

Zixuan Li, Yutao Zeng, Yuxin Zuo, Weicheng Ren, Wenxuan Liu, Miao Su, Yucan Guo, Yantao Liu, Xiang Li, Zhilei Hu, et al. 2024 c . Knowcoder: Coding structured knowledge into llms for universal information extraction. arXiv preprint arXiv:2403.07969

2024 arXiv
[35]

Jingjing Liu, Panupong Pasupat, Scott Cyphers, and Jim Glass. 2013. Asgard: A portable architecture for multilingual dialogue systems. In 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, pages 8386--8390. IEEE

2013
[36]

Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized BERT pretraining approach. CoRR, abs/1907.11692

2019 arXiv
[37]

Anton Lozhkov, Raymond Li, Loubna Ben Allal, Federico Cassano, Joel Lamy-Poirier, Nouamane Tazi, Ao Tang, Dmytro Pykhtar, Jiawei Liu, Yuxiang Wei, et al. 2024. Starcoder 2 and the stack v2: The next generation. arXiv preprint arXiv:2402.19173

2024 arXiv
[38]

Yaojie Lu, Hongyu Lin, Jin Xu, Xianpei Han, Jialong Tang, Annan Li, Le Sun, Meng Liao, and Shaoyi Chen. 2021. Text2event: Controllable sequence-to-structure generation for end-to-end event extraction. In Proceedings of the 59th Annual Meeting of the Association for Computation...

2021
[39]

Navapat Nananukul and Wichayaporn Wongkamjan. 2024. What if red can talk? dynamic dialogue generation using large language models. arXiv preprint arXiv:2407.20382

2024 arXiv
[40]

Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 3...

2022
[41]

Danny CC Poo, Derek Kiong, and Swarnalatha Ashok. 2008. Object-oriented programming and Java. Springer Science & Business Media

2008
[42]

Sameer Pradhan, Alessandro Moschitti, Nianwen Xue, Hwee Tou Ng, Anders Bj \"o rkelund, Olga Uryupina, Yuchen Zhang, and Zhi Zhong. 2013. Towards robust linguistic analysis using ontonotes. In Proceedings of the Seventeenth Conference on Computational Natural Language Learning,...

2013
[43]

Baptiste Roziere, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Tal Remez, J \'e r \'e my Rapin, et al. 2023. Code llama: Open foundation models for code. arXiv preprint arXiv:2308.12950

2023 arXiv
[44]

Oscar Sainz, Iker Garc \' a-Ferrero, Rodrigo Agerri, Oier Lopez de Lacalle, German Rigau, and Eneko Agirre. 2023. Gollie: Annotation guidelines improve zero-shot information-extraction. arXiv preprint arXiv:2310.03668

2023 arXiv
[45]

Erik F Sang and Fien De Meulder. 2003. Introduction to the conll-2003 shared task: Language-independent named entity recognition. arXiv preprint cs/0306050

2003 arXiv
[46]

Dipanjan Sarkar. 2016. Text analytics with python, volume 2. Springer

2016
[47]

Shaden Smith, Mostofa Patwary, Brandon Norick, Patrick LeGresley, Samyam Rajbhandari, Jared Casper, Zhun Liu, Shrimai Prabhumoye, George Zerveas, Vijay Korthikanti, et al. 2022. Using deepspeed and megatron to train megatron-turing nlg 530b, a large-scale generative language m...

2022 arXiv
[48]

Bjarne Stroustrup. 2013. The C++ programming language. Pearson Education

2013
[49]

Bjarne Stroustrup. 2014. Programming: principles and practice using C++. Pearson Education

2014
[50]

Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288

2023 arXiv
[51]

Sowmya Vajjala and Ramya Balasubramaniam. 2022. What do we really know about state of the art ner? arXiv preprint arXiv:2205.00034

2022 arXiv
[52]

Peter Van Roy and Seif Haridi. 2004. Concepts, techniques, and models of computer programming. MIT press

2004
[53]

Somin Wadhwa, Silvio Amir, and Byron C Wallace. 2023. Revisiting relation extraction in the era of large language models. In Proceedings of the conference. Association for Computational Linguistics. Meeting, volume 2023, page 15566. NIH Public Access

2023
[54]

Sijia Wang, Mo Yu, Shiyu Chang, Lichao Sun, and Lifu Huang. 2022 a . Query and extract: Refining event extraction as type-oriented binary decoding. In ACL (Findings)

2022
[55]

Xingyao Wang, Sha Li, and Heng Ji. 2022 b . Code4struct: Code generation for few-shot event structure prediction. arXiv preprint arXiv:2210.12810

2022 arXiv
[56]

Chih-Hsuan Wei, Yifan Peng, Robert Leaman, Allan Peter Davis, Carolyn J Mattingly, Jiao Li, Thomas C Wiegers, and Zhiyong Lu. 2016. Assessing the state of the art in biomedical relation extraction: overview of the biocreative v chemical-disease relation (cdr) task. Database, 2...

2016
[57]

Derong Xu, Wei Chen, Wenjun Peng, Chao Zhang, Tong Xu, Xiangyu Zhao, Xian Wu, Yefeng Zheng, and Enhong Chen. 2023. Large language models for generative information extraction: A survey. arXiv preprint arXiv:2312.17617

2023 arXiv
[58]

An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. 2024 a . Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115

2024 arXiv
[59]

Songhua Yang, Hanjie Zhao, Senbin Zhu, Guangyu Zhou, Hongfei Xu, Yuxiang Jia, and Hongying Zan. 2024 b . https://doi.org/10.1609/AAAI.V38I17.29907 Zhongjing: Enhancing the chinese medical capabilities of large language model through expert feedback and real-world multi-turn di...

2024 doi
[60]

Biao Zhang, Barry Haddow, and Alexandra Birch. 2023. Prompting large language model for machine translation: A case study. In International Conference on Machine Learning, pages 41092--41110. PMLR

2023
[61]

Qinkai Zheng, Xiao Xia, Xu Zou, Yuxiao Dong, Shan Wang, Yufei Xue, Zihan Wang, Lei Shen, Andi Wang, Yang Li, et al. 2023. Codegeex: A pre-trained model for code generation with multilingual evaluations on humaneval-x. arXiv preprint arXiv:2303.17568

2023 arXiv
[62]

Yuqi Zhu, Xiaohan Wang, Jing Chen, Shuofei Qiao, Yixin Ou, Yunzhi Yao, Shumin Deng, Huajun Chen, and Ningyu Zhang. 2023. Llms for knowledge graph construction and reasoning: Recent capabilities and future opportunities. arXiv preprint arXiv:2305.13168

2023 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

online" 'onlinestring :=

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

[2] [2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

[3] [3]

Zhen Bi, Jing Chen, Yinuo Jiang, Feiyu Xiong, Wei Guo, Huajun Chen, and Ningyu Zhang. 2024. Codekgc: Code language model for generative knowledge graph construction. ACM Transactions on Asian and Low-Resource Language Information Processing, 23(3):1--16

work page 2024

[4] [4]

Bruno Blanchet. 1999. Escape analysis for object-oriented languages: application to java. Acm Sigplan Notices, 34(10):20--34

work page 1999

[5] [5]

Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert - Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litw...

2020

[6] [7]

Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. 2021 b . Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374

arXiv 2021

[7] [8]

Leon Derczynski, Kalina Bontcheva, and Ian Roberts. 2016. Broad twitter corpus: A diverse named entity recognition resource. In Proceedings of COLING 2016, the 26th international conference on computational linguistics: technical papers, pages 1169--1179

work page 2016

[8] [9]

Leon Derczynski, Eric Nichols, Marieke Van Erp, and Nut Limsopatham. 2017. Results of the wnut2017 shared task on novel and emerging entity recognition. In Proceedings of the 3rd Workshop on Noisy User-generated Text, pages 140--147

2017