GenerationPrograms: Fine-grained Attribution with Executable Programs

David Wan; Elias Stengel-Eskin; Eran Hirsch; Ido Dagan; Mohit Bansal

REVIEW 4 major objections 5 minor 1 cited by

GenerationPrograms: Fine-grained Attribution with Executable Programs

T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read GENERATION PROGRAMS claims that splitting LLM generation into an executable program plan and a modular execution stage yields accurate, exhaustive, and interpretable citations, and reports large attribution-F1 gains over direct-citation…

desk verdict A credible program-then-execute framework for attribution that likely generalizes, but the headline numbers rest on thin metric validation and small test sets. read the letter →

arxiv 2506.14580 v1 pith:KY4VY3XX submitted 2025-06-17 cs.CL cs.AI

David Wan , Eran Hirsch , Elias Stengel-Eskin , Ido Dagan , Mohit Bansal This is my paper · ORCID

classification cs.CLcs.AI

keywords attributedgenerationfine-grainedcitationexecutableprogramsmodulartextoperationslong-formquestionansweringmulti-documentsummarizationcontributiveattributionpost-hoc

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that LLMs can get much better at citing their sources if they generate text differently: first write an executable 'program' that selects source sentences and specifies text operations (paraphrase, compression, fusion, extraction), then run that program to produce the answer. Because each operation records which sentences it consumed, every output sentence carries an exact citation to its inputs, and the program itself explains why the model wrote what it wrote. Across two long-form question-answering tasks and a multi-document summarization task, the method reports large attribution-F1 gains over a strong generate-with-citations baseline—up to 39 points at document level and 29 at sentence level—while reducing missing citations to zero. The same program structure also serves as a post-hoc attribution tool and enables module-level refinement that further improves attribution quality, at a small cost in standard answer-correctness metrics that the paper argues is mostly stylistic.

What carries the argument

The executable program itself is the load-bearing object: a collection of trees, one per output sentence, whose leaves are sentences extracted from source documents, whose internal nodes are outputs of neural modules, and whose root is the final generated sentence. Edges are module applications carrying optional natural-language instructions (for example, 'Highlight the current trend of robots replacing humans'). The mechanism that guarantees attribution is the principle of local attribution: every module's output must be entailed by its inputs, so that the program trace is transitive—a claim can be verified by checking each step against its named sources. Because the trace records exactly which sentences fed each operation, document- and sentence-level citations are read off the program rather than generated heuristically, and the program doubles as an explicit explanation of the model's reasoning.

What would settle it

Run a human adjudication on a larger sample—say 150 outputs per task from both ALCE and GenerationPrograms, with two independent annotators judging, for each sentence, whether the cited sources entail it—and compare human-judged attribution F1. If the gap between methods shrinks to a few points or reverses on LFQA or MDS, the headline improvement is an artifact of the NLI oracle rather than a real gain. A second check: on nested programs, verify locally that each fusion/paraphrase output is entailed by its inputs; if even a few percent of module outputs break entailment, the transitive attribution chain the method relies on fails for those sentences.

Watch

Extended reading notes

Core claim

GENERATION PROGRAMS claims that the central obstacle to verifiable LLM output is not a lack of citation skill but a collapsed process: generating text and citations at once conflates deciding what to say with tracking where it came from. The paper's solution is to separate planning from execution. In the planning stage, a language model reads the query and retrieved documents and writes a Python-style program built from modular text operations, where each step names the exact source sentences it operates on and an optional instruction describing what to emphasize. In the execution stage, dedicated neural modules (paraphrase, compression, fusion, extract) apply those operations; the output is the program's result, and the citations are the operation inputs, recovered directly from the program trace. The paper reports that this design raises attribution F1 on ASQA from 62.7 to 87.1 at document level and from 54.2 to 79.4 at sentence level; on LFQA from 55.4 to 94.4 and from 54.0 to 82.8; and on a MultiNews-based MDS task from 63.8 to 94.4 document-level and 55.2 to 90.0 sentence-level, while eliminating sentences without citations entirely. It further shows the program trace can reconstruct attributions for already-written text in a post-hoc setting, beating a prompting baseline by a large margin, and that checking each module's output for entailment of its inputs, then reranking only the failing steps, adds further attribution gains at low latency.

Load-bearing premise

The load-bearing premise is that the NLI-based oracle (AutoAIS/TRUE for ASQA, GPT-4o for LFQA and MDS) correctly decides whether a cited source supports a generated sentence—validated on only 50 examples at 78.1% agreement—and that the small test sets, notably 45 LFQA examples, represent the tasks.

Editorial extensions

If this is right

On all three tasks, GenerationPrograms eliminates sentences with no citation, while the ALCE baseline leaves 25.8% (ASQA docs), 40% (LFQA docs), and 25.4% (MDS docs) of sentences uncited.
The program trace provides contributive, not just corroborative, attributions: it shows which source sentences were actually used and how they were transformed, supporting per-step verification by a reader.
Because the program can be regenerated from an existing output, attribution can be added post hoc to black-box generations without access to model internals or logits, and the reconstructed output preserves content (ROUGE-L 50.7 on LFQA).
Module-level entailment checking with reranking of only failing steps raises attribution F1 by 4.0 points on ASQA and 2.7 on LFQA, at an added latency of about 1.8 seconds per example, whereas reranking entire ALCE outputs costs roughly 8x runtime.
Combining program-based generation with extractive summarization of retrieved sources balances attribution quality against answer correctness, mitigating the correctness drop the method otherwise shows on style-sensitive metrics.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The program trace is an inverse map from output to context, so it could be used to prune retrieved documents: removing sources that never appear in any tree and re-generating should preserve the answer while cutting context length, a use the paper mentions only as future direction.
The reported correctness gap between ALCE and GenerationPrograms (e.g., ROUGE-L 39.9 vs 32.3 on LFQA) may be largely a length and style artifact: the paper's own LLM-based correctness judge and a 25-example human rating found the methods close or reversed, which suggests standard metrics understate the method's answer quality.
If local attribution holds reliably across modules, the same tree structure could support adversarial-robustness checks—detecting when a small change to a cited source alters a downstream output sentence—since every node names its upstream sources.
A testable extension: applying GenerationPrograms to tasks with finer granularity (phrase-level or paragraph-level operations, with pointers to sentences) should preserve attribution fidelity; the paper asserts this extension is straightforward but does not evaluate it.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Dear colleague,

The thing to know about this paper: the core design is genuinely useful. GenerationPrograms decomposes RAG answers into a program of modular text operations (paraphrase, compression, fusion) with explicit instructions, then executes it, so each output sentence is traceable to its source sentences by construction. That's a real step beyond the direct-prompt-with-citations approach, and it's an honest extension of SummarizationPrograms to long-form QA and multi-document summarization. The qualitative examples look right, and the code is public.

What's new: applying program-then-execute to attribution-heavy tasks, adding instruction-conditioned modules, showing it can be used post hoc, and showing that the program trace enables targeted, cheap refinement. The post-hoc attribution result is interesting even if the baselines are not the strongest available. The modular refinement is a nice bonus, not the main event.

Where I'd push back: the quantitative claims rest on model-based NLI judges. The authors validate AutoAIS on 50 ASQA examples and get 78.1% agreement, which is reasonable but thin, and they never validate GPT-4o as a judge on LFQA or MDS. Since GenerationPrograms outputs share more surface wording with the sources (because modules operate on the source sentences), an NLI judge that rewards lexical overlap could inflate the attribution F1 advantage even if the true support relation is only modestly better. That's a real measurement concern, not a fatal one. The LFQA test set has 45 examples, so a 39-point document-level F1 gain is fragile. No error bars anywhere. The weaknesses don't undercut the qualitative claim—the mechanism is sound, and the Llama-3.3 results show the trend holds with a different generator—but they should temper how the numbers are read and reported.

Also, for the 'post-hoc attribution' table, they compare against Semantic Similarity and a simple LLM prompt, and they don't include ContextCite, which is the obvious state-of-the-art comparison they already cite. That should be fixed. And in the refinement section, the same AutoAIS metric is used both to select refinements and to evaluate them; that's a circularity, but it's confined to that one experiment and the gains are small.

Bottom line: this is a solid, well-engineered paper that deserves a serious referee. The method is a real contribution, the code is out, and the central mechanism is transparent. It needs a stronger evaluation section—error bars, larger or at least acknowledged test sets, a human-validated judge for the non-ASQA settings, and stronger baselines—before the magnitude of the gains should be taken at face value. I'd engage with it, and I'd probably cite it for the program-trace idea.

Recommendation: send to reviewers, with a request that they focus on the metric validity and baseline choices.

Referee Report

4 major / 5 minor

Summary. The paper proposes GenerationPrograms, a two-stage framework for attributed long-form generation: a planner LLM first produces an executable program composed of modular text operations (paraphrase, compression, fusion, extract) over retrieved source sentences, and the program is then executed to produce an answer whose sentence-level citations are automatically the module input sentences. The authors evaluate the framework on ASQA, LFQA, and MultiNews MDS, reporting large attribution F1 improvements over the ALCE baseline at both document and sentence level (e.g., document-level F1 of 87.1 vs 62.7 on ASQA and 94.4 vs 55.4 on LFQA in Table 1). They also present a post-hoc attribution experiment on LFQA using human gold sentence annotations, and a module-level refinement study that reranks module outputs. The paper additionally reports answer-correctness trade-offs with EM/ROUGE-L, which they argue are reduced when evaluated with an LLM-based correctness metric or human annotation.

Significance. If the headline attribution gains hold, the paper makes a strong practical contribution: it offers an interpretable, executable program trace that provides contributive rather than merely corroborative attribution, and the framework naturally supports localized refinement at low computational cost. The post-hoc attribution experiment is evaluated against human-annotated sentence IDs rather than an NLI judge, which is a notable strength, and the authors provide an open-source implementation, deterministic decoding settings, a replication with an open-source model, and a human evaluation of answer correctness. These elements support the credibility of the core mechanism. The main risks are the reliance on model-based NLI judges for the central comparisons and the small LFQA test set, which the major comments address.

major comments (4)

[§3.2, Table 4, Appendix C.8] The central attribution comparisons are evaluated with model-based NLI judges (AutoAIS for ASQA, GPT-4o for LFQA and MDS), but the only human validation of an NLI judge is 50 ASQA/AutoAIS examples with 78.1% agreement; no human validation is reported for the GPT-4o judge on LFQA or MDS, and the LFQA test set contains only 45 examples (Table 4), while the MDS test-set size is not stated. Because GenerationPrograms executes paraphrase/compression/fusion directly on source sentences, its outputs likely have higher lexical overlap with the source than ALCE's free-form generations, and an NLI judge sensitive to lexical overlap could inflate GenerationPrograms' F1 without a genuine improvement in support. The authors should report human agreement with the GPT-4o judge on a stratified sample from LFQA and MDS, provide a lexical-overlap or judge-bias analysis, or conduct a human re-evaluation of attributions on a subset of all three datasets, together with error bars.
[§4.3, Table 3] The module-level refinement experiment uses AutoAIS both to select the 'entailed' candidate among five sampled module outputs and to compute the reported attribution F1, so the refinement procedure directly optimizes the metric on which the +4.0 (ASQA) and +2.7 (LFQA) F1 improvements are reported; the same coupling applies to the ALCE+reranking comparison. This selection-evaluation overlap can inflate the apparent refinement gains. The authors should evaluate refined outputs with an independent metric or human annotations rather than the same AutoAIS judge used for selection, or clearly justify why the coupling does not affect the comparison.
[Tables 1, 2, 3, 5, 7, 8; Table 4] No confidence intervals or significance tests are reported anywhere in the paper, while the text repeatedly uses 'significantly' to describe improvements. Given that the LFQA evaluation is based on only 45 examples (Table 4) and the MDS sample size is not stated, a few examples could shift F1 by several points; the paper should report bootstrap confidence intervals or significance tests for the main comparisons, and should state the MDS test-set size.
[Figure 5 vs §2.2/Table 1] The ALCE baseline is prompted to cite at most three documents per sentence and to cite only a minimum sufficient subset (Figure 5), whereas GenerationPrograms cites every source sentence consumed by the modules with no explicit upper bound. This asymmetry in the citation instruction means part of the large recall/F1 gap may reflect a difference in citation policy rather than attribution quality. The authors should either adopt a comparable citation policy for ALCE (e.g., an ALCE variant without the three-citation cap) or report a citation-budget-controlled analysis.

minor comments (5)

[Table 1] The table header labels the MDS columns as 'Correct Attribution (AutoAIS)', but §3.2 states that GPT-4o is used for LFQA and MDS; the caption or header should be corrected.
[§4.1 and Abstract/Introduction] Section 4.1 reports document-level F1 improvements of 20.4% on ASQA and 39.0% on LFQA, but Table 1 shows absolute differences of 24.4 and 39.0 points; the abstract and introduction report gains of 31.6% and 27%, which are inconsistent with both §4.1 and Table 1.
[Table 3] The name 'GENERATIONPROGRAMS' appears without the space used elsewhere in the paper; please make the name consistent.
[Appendix C.4] The text states that extractive summarization enhances accuracy with Llama 3.3 70B, but Table 8 shows ASQA correctness actually decreases from 46.1 to 45.0 for GENPROG and from 51.8 to 48.7 for ALCE; the claim should be qualified to LFQA or to the average.
[Table 4] The table omits statistics for the multi-document summarization dataset even though MDS results are part of the central claims; please include the number of examples, average source tokens, and average sentences for that dataset.

Circularity Check

1 steps flagged · score 6.0 of 10

Main attribution gains are empirical, but the module-level refinement result is partly forced because AutoAIS is both the selection criterion and the evaluation metric.

fitted input called prediction [Section 4.3 (Fine-grained Module-level Detection and Refinement), Table 3; also Appendix C.3]
"We leverage AutoAIS to measure entailment between the generated module outputs and their respective input sentences. If a module generates a non-attributable output, we refine it using a reranking approach. Specifically, we sample five candidate outputs with a temperature of 1.0 and select the first candidate considered entailed by AutoAIS. ... targeted module-level refinements further enhance performance, increasing attribution F1 by 4 points on ASQA and 2.7 points on LFQA."

The refinement procedure and the evaluation use the same AutoAIS judge. A candidate is selected precisely because AutoAIS rates it entailed, and the reported attribution F1 is then computed by that same AutoAIS judge. Thus the measured gain is not an independent estimate: selecting by a metric and then scoring with that metric forces the selected output to count as correct by construction. The ALCE reranking baseline uses the same AutoAIS selection, so the relative comparison is fairer, but the claimed module-level refinement improvement is a fitted-input-called-prediction artifact rather than an externally validated result. The companion '%Entail.' measure is likewise self-referential, since the selected candidate is by definition AutoAIS-entailed.

full rationale

The central GenerationPrograms pipeline is not circular. Citations are defined as the module input sentences, and the main attribution-quality claims are checked by an external NLI judge (AutoAIS for ASQA, GPT-4o NLI for LFQA and MDS) rather than by the mechanism that produced the citations; the post-hoc attribution experiment in Section 4.2 uses human gold annotations, not the NLI metric. That core comparison against ALCE is an empirical result, even if one worries about judge leniency or small test sets. The genuinely circular spot is Section 4.3: the same AutoAIS model selects refined module outputs and evaluates them, so the reported 4.0 and 2.7 point F1 gains from module-level refinement are in part an artifact of optimizing the evaluation objective. The self-citations to Saha et al. (2023) and Wan et al. (2025) are ordinary prior-work citations and are not load-bearing. The human-validation of the attribution metric is limited (50 examples, 78.1% agreement, ASQA/AutoAIS only), and LFQA uses only 45 test examples, but these are measurement-validity concerns rather than circularity. Overall, one presented result reduces by construction, while the main empirical claim retains independent content.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

No parameters are fitted to data; the method uses zero-shot LLM prompting with temperature 0. The listed free parameters are hand-chosen experimental settings. The main assumptions are that the NLI-based attribution metrics are valid oracles, that the modules preserve entailment in the large majority of executions, and that the LLM planner produces valid programs. No new unobservable entities are introduced.

free parameters (3)

Number of retrieved passages (top 10) for ASQA = 10
Hand-chosen standard setting from prior work (Section 3.1); not fitted to the data.
Number of sentences in extractive summarization (num_sent) = not reported in main text
Hand-chosen experimental setting for the summarization preprocessing step (Section 2.4, Appendix B).
Number of reranking candidates in module-level refinement = 5
Sampling temperature 1.0 with five candidates, then selecting the first entailed candidate (Section 4.3).

assumptions (4)

domain assumption NLI-based attribution metrics (AutoAIS for ASQA, GPT-4o for LFQA/MDS) are valid and unbiased measures of whether a cited source supports a generated sentence.
Section 3.2 defines attribution quality via these metrics; Appendix C.8 validates on 50 examples with 78.1% human agreement. If the metrics are biased, the reported attribution improvements are not meaningful.
domain assumption The neural modules (paraphrase, compression, fusion, extract) preserve the meaning and entailment of their input sentences in the large majority of executions, making local attribution transitive.
Section 2.3 states this 'local attribution' principle; Appendix D.1 reports per-module entailment rates above 80-90%. If modules frequently hallucinate, the automatic sentence-level citations become invalid.
domain assumption The LLM planner reliably produces valid, executable programs with correct sentence selections and instructions.
Section 2.2 relies on GPT-4o to generate Python-style programs; Appendix D.1 reports invalid trees are extremely rare (1 per dataset). If the planner frequently chooses wrong sentences or malformed programs, the whole pipeline fails.
domain assumption The LFQA test set (45 examples) is a representative and sufficiently large evaluation set for the reported post-hoc attribution and refinement results.
Table 4 lists 45 LFQA examples. With such a small sample, the large numerical gaps (e.g., 47.4 vs 7.8 F1 in Table 2) could be unstable; no confidence intervals are provided.

how reviews work

0 comments

Cite this review

Pith. "Pith review of GenerationPrograms: Fine-grained Attribution with Executable Programs." pith.science (2026). https://pith.science/paper/KY4VY3XX

@misc{pith2026250614580,
  author       = {Pith},
  title        = {Pith review of: GenerationPrograms: Fine-grained Attribution with Executable Programs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KY4VY3XX}},
  note         = {Machine review of arXiv:2506.14580}
}

read the original abstract

Recent large language models (LLMs) achieve impressive performance in source-conditioned text generation but often fail to correctly provide fine-grained attributions for their outputs, undermining verifiability and trust. Moreover, existing attribution methods do not explain how and why models leverage the provided source documents to generate their final responses, limiting interpretability. To overcome these challenges, we introduce a modular generation framework, GenerationPrograms, inspired by recent advancements in executable "code agent" architectures. Unlike conventional generation methods that simultaneously generate outputs and attributions or rely on post-hoc attribution, GenerationPrograms decomposes the process into two distinct stages: first, creating an executable program plan composed of modular text operations (such as paraphrasing, compression, and fusion) explicitly tailored to the query, and second, executing these operations following the program's specified instructions to produce the final response. Empirical evaluations demonstrate that GenerationPrograms significantly improves attribution quality at both the document level and sentence level across two long-form question-answering tasks and a multi-document summarization task. We further demonstrate that GenerationPrograms can effectively function as a post-hoc attribution method, outperforming traditional techniques in recovering accurate attributions. In addition, the interpretable programs generated by GenerationPrograms enable localized refinement through modular-level improvements that further enhance overall attribution quality.

Figures

Figures reproduced from arXiv: 2506.14580 by the authors.

**Figure 2.** Illustration of GENERATIONPROGRAMS. First, an executable program is generated from the question and retrieved documents. Next, the program is executed using dedicated text-based operations to produce the final sentences. The sentences used are automatically treated as sentence-level attributions. This design promotes both interpretability and reliable attribution by tracing the program execution and citing every sen… view at source ↗

**Figure 3.** Example on LFQA. 23 [PITH_FULL_IMAGE:figures/full_fig_p023_3.png] view at source ↗

Figures from the paper (5 more)

**Figure 4.** Figure 4: Example of post-hoc attribution with GENERATIONPROGRAMS and its re-generated program with Rouge-L score. ASQA LFQA Avg. Word Avg. Sent Avg. Word Avg. Sent ALCE 31.4 1.7 87.1 4.7 GENERATIONPROGRAMS 41.9 2.3 140.6 4.8 Reference 71.8 3.8 47.5 2.4 [PITH_FULL_IMAGE:figures…

**Figure 5.** Figure 5: Prompt for ALCE. E Prompts We include the prompt for ALCE in [PITH_FULL_IMAGE:figures/full_fig_p025_5.png]

**Figure 6.** Figure 6: Prompt for running GENERATIONPROGRAMS. 26 [PITH_FULL_IMAGE:figures/full_fig_p026_6.png]

**Figure 7.** Figure 7: Prompt for modules. Extractive Given the following document and the question ”question”, extract numsent sentences from the passage that can answer the question. Do not change the sentences and copy the sentences exactly as they are. You should format your output as a…

**Figure 8.** Figure 8: Prompt for summarization. 27 [PITH_FULL_IMAGE:figures/full_fig_p027_8.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

ProvenanceGuard: Source-Aware Factuality Verification for MCP-Based LLM Agents
cs.AI 2026-06 unverdicted novelty 7.0 of 10

ProvenanceGuard detects when a claim in an MCP-based agent answer is supported somewhere but attributed to the wrong source, with block F1 0.802 and perfect detection on 50 controlled swaps.

Reference graph

Works this paper leans on

16 extracted references · 5 canonical work pages · cited by 1 Pith paper

[1]

Optional:* You can specify an instruction for a desired style or syntactic structure (e.g., instruction=”YOUR INSTRUCTION”)

**paraphrase(sentence, instruction=None)** Purpose:* Rephrase the given sentence while preserving its original meaning. Optional:* You can specify an instruction for a desired style or syntactic structure (e.g., instruction=”YOUR INSTRUCTION”)

work page
[2]

Optional:* Include an instruction detailing which parts to preserve (e.g., instruc- tion=”YOUR INSTRUCTION”)

**compression(sentence, instruction=None)** Purpose:* Compress the given sentence to produce a shorter version that retains the essential content and syntactic structure. Optional:* Include an instruction detailing which parts to preserve (e.g., instruc- tion=”YOUR INSTRUCTION”)

work page
[3]

sentence n, instruction=None)** Purpose:* Merge multiple sentences into a single sentence

**fusion(sentence 1, sentence 2, ... sentence n, instruction=None)** Purpose:* Merge multiple sentences into a single sentence. The sentences might convey similar or complementary information. Optional:* Provide an instruction on how to combine the sentences, such as which parts to prioritize (e.g., instruction=”YOUR INSTRUCTION”). *Careful:** - **[Format...

work page
[5]

doi: 10.18653/v1/2022.emnlp-main.396

Association for Computational Linguistics. doi: 10.18653/v1/2022.emnlp-main.396. URL https://aclanthology.org/2022.emnlp-main.396/. Matthew Honnibal, Ines Montani, Sofie Van Landeghem, and Adriane Boyd. spaCy: Industrial-strength Natural Language Processing in Python, 2020. Or Honovich, Roee Aharoni, Jonathan Herzig, Hagai Taitelbaum, Doron Kukliansy, Ver...

arXiv 2022
[7]

doi: 10.18653/v1/P19-1209

Association for Computational Linguistics. doi: 10.18653/v1/P19-1209. URL https://aclanthology.org/P19-1209/. Logan Lebanoff, John Muchovej, Franck Dernoncourt, Doo Soon Kim, Lidan Wang, Wal- ter Chang, and Fei Liu. Understanding points of correspondence between sentences for abstractive summarization. In Shruti Rijhwani, Jiangming Liu, Yizhong Wang, and ...

work page doi:10.18653/v1/p19-1209
[8]

URL https://arxiv.org/abs/2203.11147. Gr´egoire Mialon, Roberto Dessi, Maria Lomeli, Christoforos Nalmpantis, Ramakanth Pa- sunuru, Roberta Raileanu, Baptiste Roziere, Timo Schick, Jane Dwivedi-Yu, Asli Ce- likyilmaz, Edouard Grave, Yann LeCun, and Thomas Scialom. Augmented language models: a survey. Transactions on Machine Learning Research, 2023. ISSN 2...

arXiv 2023
[9]

Ramesh Nallapati, Bowen Zhou, Cicero dos Santos, C ¸a˘glar Gu˙lc ¸ehre, and Bing Xiang

URL https://arxiv.org/abs/2112.09332. Ramesh Nallapati, Bowen Zhou, Cicero dos Santos, C ¸a˘glar Gu˙lc ¸ehre, and Bing Xiang. Abstractive text summarization using sequence-to-sequence RNNs and beyond. In Stefan Riezler and Yoav Goldberg (eds.), Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning , pp. 280–290, Berlin, Germ...

arXiv 2016
[10]

doi: 10.18653/v1/2024.emnlp-main.347

Association for Computational Linguistics. doi: 10.18653/v1/2024.emnlp-main.347. URL https://aclanthology.org/2024.emnlp-main.347/. Hannah Rashkin, Vitaly Nikolaev, Matthew Lamm, Lora Aroyo, Michael Collins, Dipanjan Das, Slav Petrov, Gaurav Singh Tomar, Iulia Turc, and David Reitter. Measuring attribution in natural language generation models.Computation...

doi:10.18653/v1/2024.emnlp-main.347 2024

Show all 16 references

[11]

11 Junxian He, Wojciech Kryscinski, Bryan McCann, Nazneen Rajani, and Caiming Xiong

URL https://doi.org/10.1016/j.eswa.2018.12.011. 11 Junxian He, Wojciech Kryscinski, Bryan McCann, Nazneen Rajani, and Caiming Xiong. CTRLsum: Towards generic controllable text summarization. In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang (eds.), Proceedings of the 2022 Con...

2018 doi
[12]

URL https://aclanthology.org/2023.cl-4.2/

doi: 10.1162/coli a 00486. URL https://aclanthology.org/2023.cl-4.2/. Nils Reimers and Iryna Gurevych. Sentence-BERT: Sentence embeddings using Siamese BERT-networks. In Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan (eds.), Proceed- ings of the 2019 Conference on Empir...

2023 doi
[13]

retrieve-then-generate

URL https://arxiv.org/abs/2311.12233. Fangyuan Xu, Weijia Shi, and Eunsol Choi. RECOMP: Improving retrieval-augmented LMs with context compression and selective augmentation. In The Twelfth International Conference on Learning Representations , 2024. URL https://openreview.net...

2023 arXiv
[182]

Ivan Stelmakh, Yi Luan, Bhuwan Dhingra, and Ming-Wei Chang

URL https://aclanthology.org/2024.acl-long.182/. Ivan Stelmakh, Yi Luan, Bhuwan Dhingra, and Ming-Wei Chang. ASQA: Factoid ques- tions meet long-form answers. In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang (eds.), Proceedings of the 2022 Conference on Empirical Methods in ...

2024 arXiv
[2019]

doi: 10.18653/v1/P19-1102

Association for Computational Linguistics. doi: 10.18653/v1/P19-1102. URL https://aclanthology.org/P19-1102/. Luyu Gao, Zhuyun Dai, Panupong Pasupat, Anthony Chen, Arun Tejasvi Chaganty, Yicheng Fan, Vincent Zhao, Ni Lao, Hongrae Lee, Da-Cheng Juan, and Kelvin Guu. RARR: Resea...

arXiv
[2022]

doi: 10.18653/v1/2022.naacl-main.128

Association for Computational Linguistics. doi: 10.18653/v1/2022.naacl-main.128. URL https://aclanthology.org/2022.naacl-main.128/. Ori Ernst, Ori Shapira, Aviv Slobodkin, Sharon Adar, Mohit Bansal, Jacob Goldberger, Ran Levy, and Ido Dagan. The power of summary-source alignme...

2022 doi
[2023]

Yen-Chun Chen and Mohit Bansal

URL https://arxiv.org/abs/2305.14908. Yen-Chun Chen and Mohit Bansal. Fast abstractive summarization with reinforce-selected sentence rewriting. In Iryna Gurevych and Yusuke Miyao (eds.), Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (...

2018 arXiv
[2024]

doi: 10.18653/v1/2024.emnlp-main.552

Association for Computational Linguistics. doi: 10.18653/v1/2024.emnlp-main.552. URL https://aclanthology.org/2024.emnlp-main.552/. Logan Lebanoff, Kaiqiang Song, Franck Dernoncourt, Doo Soon Kim, Seokhwan Kim, Walter Chang, and Fei Liu. Scoring sentence singletons and pairs f...

2024 doi

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.

[1] [1]

Optional:* You can specify an instruction for a desired style or syntactic structure (e.g., instruction=”YOUR INSTRUCTION”)

**paraphrase(sentence, instruction=None)** Purpose:* Rephrase the given sentence while preserving its original meaning. Optional:* You can specify an instruction for a desired style or syntactic structure (e.g., instruction=”YOUR INSTRUCTION”)

work page

[2] [2]

Optional:* Include an instruction detailing which parts to preserve (e.g., instruc- tion=”YOUR INSTRUCTION”)

**compression(sentence, instruction=None)** Purpose:* Compress the given sentence to produce a shorter version that retains the essential content and syntactic structure. Optional:* Include an instruction detailing which parts to preserve (e.g., instruc- tion=”YOUR INSTRUCTION”)

work page

[3] [3]

sentence n, instruction=None)** Purpose:* Merge multiple sentences into a single sentence

**fusion(sentence 1, sentence 2, ... sentence n, instruction=None)** Purpose:* Merge multiple sentences into a single sentence. The sentences might convey similar or complementary information. Optional:* Provide an instruction on how to combine the sentences, such as which parts to prioritize (e.g., instruction=”YOUR INSTRUCTION”). *Careful:** - **[Format...

work page

[4] [5]

doi: 10.18653/v1/2022.emnlp-main.396

Association for Computational Linguistics. doi: 10.18653/v1/2022.emnlp-main.396. URL https://aclanthology.org/2022.emnlp-main.396/. Matthew Honnibal, Ines Montani, Sofie Van Landeghem, and Adriane Boyd. spaCy: Industrial-strength Natural Language Processing in Python, 2020. Or Honovich, Roee Aharoni, Jonathan Herzig, Hagai Taitelbaum, Doron Kukliansy, Ver...

arXiv 2022

[5] [7]

doi: 10.18653/v1/P19-1209

Association for Computational Linguistics. doi: 10.18653/v1/P19-1209. URL https://aclanthology.org/P19-1209/. Logan Lebanoff, John Muchovej, Franck Dernoncourt, Doo Soon Kim, Lidan Wang, Wal- ter Chang, and Fei Liu. Understanding points of correspondence between sentences for abstractive summarization. In Shruti Rijhwani, Jiangming Liu, Yizhong Wang, and ...

work page doi:10.18653/v1/p19-1209

[6] [8]

URL https://arxiv.org/abs/2203.11147. Gr´egoire Mialon, Roberto Dessi, Maria Lomeli, Christoforos Nalmpantis, Ramakanth Pa- sunuru, Roberta Raileanu, Baptiste Roziere, Timo Schick, Jane Dwivedi-Yu, Asli Ce- likyilmaz, Edouard Grave, Yann LeCun, and Thomas Scialom. Augmented language models: a survey. Transactions on Machine Learning Research, 2023. ISSN 2...

arXiv 2023

[7] [9]

Ramesh Nallapati, Bowen Zhou, Cicero dos Santos, C ¸a˘glar Gu˙lc ¸ehre, and Bing Xiang

URL https://arxiv.org/abs/2112.09332. Ramesh Nallapati, Bowen Zhou, Cicero dos Santos, C ¸a˘glar Gu˙lc ¸ehre, and Bing Xiang. Abstractive text summarization using sequence-to-sequence RNNs and beyond. In Stefan Riezler and Yoav Goldberg (eds.), Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning , pp. 280–290, Berlin, Germ...

arXiv 2016

[8] [10]

doi: 10.18653/v1/2024.emnlp-main.347

Association for Computational Linguistics. doi: 10.18653/v1/2024.emnlp-main.347. URL https://aclanthology.org/2024.emnlp-main.347/. Hannah Rashkin, Vitaly Nikolaev, Matthew Lamm, Lora Aroyo, Michael Collins, Dipanjan Das, Slav Petrov, Gaurav Singh Tomar, Iulia Turc, and David Reitter. Measuring attribution in natural language generation models.Computation...

doi:10.18653/v1/2024.emnlp-main.347 2024