Pith. sign in

REVIEW 4 major objections 4 minor 116 references

SuperWriter: Reflection-Driven Long-Form Generation with Large Language Models

T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A 7B model trained to plan, draft, and revise approaches the quality of far larger writing models.

desk verdict A useful 7B long-form writing recipe with a clean ablation story, but the headline claim overreaches and the hierarchical DPO reward is not validated against humans. read the letter →

arxiv 2506.04180 v1 pith:ONX2URWC submitted 2025-06-04 cs.CL

Yuhao Wu , Yushi Bai , Zhiqiang Hu , Juanzi Li , Roy Ka-Wei Lee This is my paper
classification cs.CL
keywords long-formtextgenerationhierarchicaldirectpreferenceoptimizationMonteCarloTreeSearchplan-write-refineagentframeworkthinkingdataLLM-as-a-judgeWritingBench7Blanguagemodel
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

SuperWriter tackles the problem that language models generate long texts in one pass and their coherence decays as output length grows. The paper's proposal is to teach a small model the deliberate process a human writer uses—first plan an outline, then think before writing each paragraph, then check and revise—rather than to rely on scale alone. To do this the authors generate a plan–write–refine dataset with an agent, train a 7B model on it, and add a hierarchical DPO stage that converts final-output quality scores into preferences at every step using Monte Carlo Tree Search. On WritingBench the resulting SuperWriter-LM scores 8.51/10, second only to DeepSeek-R1 and ahead of all same-size models, and it wins or ties most pairwise comparisons against several larger open models on roughly 200 real user queries. If the result holds, a 7B model can reach near-frontier long-form writing quality by learning how to structure its generation, with consequences for efficiency and deployment.

What carries the argument

The load-bearing mechanism is the three-stage SuperWriter pipeline plus a hierarchical DPO procedure over an MCTS tree. For each writing prompt the agent produces five plans, four drafts per plan, and three refinements per draft, giving sixty leaf outputs; a six-dimension rubric scored by QwQ-32B (average of three runs) rates each leaf. Leaf scores are converted into ordinal rewards by percentile, then averaged upward first across refinements and then across drafts, so every stage gets a preference pair that reflects final quality; the policy is then optimized with the standard DPO loss on preferences harvested at the plan, draft, and refine levels. The design assumption is that good plans cause good drafts and good refinements cause good finals, so averaging leaf rewards down the tree is a valid credit-assignment signal.

What would settle it

Show a set of the sixty-leaf MCTS trees to human raters and measure agreement between their rankings and the QwQ-32B leaf scores; if agreement is near chance or driven by output length or formatting rather than coherence, the preference pairs built from those scores are misaligned. A cleaner test is to retrain the hierarchical DPO with a judge score that has been validated against human preferences and see whether the WritingBench and win-rate gains persist.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that a 7B model can internalize an agent-style writing process and thereby match or beat much larger models at long-form generation. SuperWriter-LM, trained on the three-stage SuperWriter-Agent data and then aligned with hierarchical DPO, achieves an average WritingBench score of 8.51, second among all evaluated models behind DeepSeek-R1's 8.55 and first among models of its own size; it also matches DeepSeek-R1 on English and outperforms it in four of the six benchmark domains. In pairwise win-rate evaluations on roughly 200 real user queries, SuperWriter-LM beats long-form specialist LongWriter-8B and writing-tuned 7B models, and remains competitive against much larger systems including DeepSeek-R1. The ablation path—7.43 base, 8.21 with agent final outputs, 8.47 after stage-wise SFT, 8.51 after hierarchical DPO—is the paper's evidence that the structured thinking data, not scale, drives the gain.

Load-bearing premise

The load-bearing premise is that the judge's averaged six-dimension score on a final output is a true measure of writing quality and that averaging it down through refinements and drafts correctly identifies which earlier plan or draft decisions were good.

Editorial extensions

If this is right

  • A single 7B model can replace a 30-to-40-call agent pipeline, so the quality gains of multi-stage reasoning can be distilled into one efficient inference-time model.
  • Because the SFT data is segmented into plan, write, and refine stages, users could be given intermediate outlines to edit before the model writes, aligning generation with human review workflows.
  • The ablation numbers imply that structured thinking data accounts for most of the gain (7.43 to 8.47 on WritingBench) and hierarchical DPO adds a further, smaller improvement (to 8.51).
  • Same-size and writing-specialist baselines are overtaken, suggesting that explicit planning plus step-level preference learning is a more effective use of a fixed model budget than scaling parameters.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the QwQ-32B judge is partly rewarding length, formatting, or stylistic fluency rather than coherence, the DPO pairs inherit that bias; a direct test is to compare judge scores against human rankings on the same sixty-leaf trees.
  • The paper's comparison bundles plan, write, and refine into one three-stage package, so it does not isolate whether the planning stage alone or the refinement stage alone drives the gain; an ablation that trains on only two of the three stages would answer that.
  • The same recipe—agent-generated thinking data plus hierarchical DPO—should transfer to other base models and languages; testing it on an 8B or 14B backbone would show whether the gain is specific to Qwen2.5-7B or a general property of the method.
  • Three sequential forward passes cost latency, so the practical question the paper leaves implicit is whether users prefer the quality gain over a single-pass model when both are given the same wall-clock budget.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes SuperWriter-Agent, a three-stage (Plan-Write-Refine) agent pipeline that produces thought-annotated long-form writing data, and SuperWriter-LM, a 7B model trained on this data via stage-wise SFT plus a hierarchical DPO procedure in which MCTS propagates final-output quality scores to intermediate plan, draft, and refine decisions. On WritingBench, SuperWriter-LM scores 8.51 on average, making it the best same-size model and second overall to DeepSeek-R1; the paper also reports pairwise win-rate evaluations against several baselines and a human evaluation against Writing-Model-Qwen-7B. The authors argue that these results demonstrate the value of explicit thinking steps and hierarchical preference learning for long-form generation.

Significance. If the central claims were fully supported, the paper would be a useful demonstration that a 7B model can match or approach much larger systems on long-form writing by distilling an agentic planning-writing-refining process, and that hierarchical DPO over tree-structured rollouts can improve multi-stage generation. Strengths include the explicit three-stage data format, the MCTS-based credit-assignment idea, the inclusion of a human evaluation, and the release of code and models, which are reproducibility pluses. However, the headline 'state-of-the-art' claim is not supported by the paper's own Table 1, and the validity of the training-time reward and the statistical significance of the key ablation are not yet established, so the strength of the claims currently exceeds the evidence.

major comments (4)
  1. [Abstract and §4.2, Table 1] The abstract claims that SuperWriter-LM 'achieves state-of-the-art performance, surpassing even larger-scale baseline models in both automatic evaluation and human evaluation,' but Table 1 shows SuperWriter-LM is second to DeepSeek-R1 on WritingBench (8.51 vs. 8.55), and the human evaluation in §4.3 compares only against Writing-Model-Qwen-7B, a same-size 7B model. The abstract's claim about surpassing larger-scale models in human evaluation is therefore not supported by any reported experiment. Please revise the claims to match the evidence, e.g., 'best among same-size models and competitive with much larger models,' and clearly state that the human evaluation was against a 7B baseline.
  2. [§3.2, Steps 1–3 (Write-judge reward)] The entire hierarchical DPO signal is built on the Write-judge reward: QwQ-32B scores each leaf three times on six LLM-selected dimensions, the scores are averaged and discretized into ordinal rewards, then propagated upward by averaging over children to construct preference pairs P1, P2, and P3. The paper reports no validation of this reward against human judgments, no inter-annotator or judge agreement statistics, and no check that child-averaging correctly ranks intermediate plans and drafts. If the judge is biased toward length, formatting, or stylistic fluency rather than coherence and planning quality, all three preference sets encode that bias, and the +0.04 gain attributed to hierarchical DPO in Figure 6 could reflect alignment to the judge rather than improved writing. Please add a human-agreement study for the Write-judge scores, report agreement/reliability metrics, analyze correlations with length and surface features, and validate the parent-node averaging assumption on human judgments of plans and drafts.
  3. [§4.4, Figure 6] The ablation improvement from hierarchical DPO is reported as a single point estimate: 8.47 with three-stage SFT versus 8.51 with hierarchical DPO, a difference of 0.04 on the WritingBench scale. No confidence intervals, repeated evaluation runs, bootstrapped standard errors, or significance tests are provided, and the evaluation itself uses an LLM critic. Given that the entire claimed benefit of the paper's central training contribution rests on this small increment, the current presentation does not establish that the improvement is real rather than noise. Please report variance or significance estimates, or temper the claim accordingly.
  4. [§4.2, Table 1 caption and §4.1 Benchmark Setup] Table 1 is captioned 'evaluated with our critic model,' which is ambiguous: it could mean the official WritingBench critic (a Qwen2.5-7B model described in the benchmark) or a custom critic built for this paper. If a custom critic was used, the scores are not directly comparable to published WritingBench numbers, and the comparison to baselines may not be meaningful unless the custom critic is validated against the official benchmark. Please clarify whether the official WritingBench evaluation harness was used, and if not, report the agreement between the custom critic and the official benchmark's critic, ideally reporting both sets of scores.
minor comments (4)
  1. [§4.4] The text says 'As shown in Table 6' but the cited result is Figure 6; please correct the cross-reference.
  2. [§7, Limitation (3)] The phrase 'Lack of online reinforcement learning. Lack of online reinforcement learning.' is duplicated; please remove the repetition.
  3. [§4.3, Human Evaluation] The sentence 'we conduct an human supplementary assessment' contains a grammatical error ('an human' should be 'a human'), and the description of annotators as 'with undergraduate degrees.' should be made into a complete clause with details about their language proficiency and evaluation instructions.
  4. [§3.1, SFT Training] The phrase 'it better accommodate to real-world user workflows' should be 'it better accommodates real-world user workflows' or 'it better accommodates the workflows of real users.'

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the training signal (QwQ-32B judge) is distinct from the evaluation signal (WritingBench critic and human judges), and self-citations are not load-bearing.

full rationale

The paper's derivation chain is self-contained with respect to the definitions of circularity used here. The SFT data are produced by a GPT-4o-based agent from WildChat/LMSYS instructions, not from the evaluation benchmark. The hierarchical DPO preferences are built from QwQ-32B scores on final outputs, while the main claimed results are measured on WritingBench, whose Qwen2.5-7B critic was independently fine-tuned on 50K human-labeled samples with 83% human agreement, and by human pairwise evaluation. The +0.04 gain attributed to hierarchical DPO in Figure 6 is therefore evaluated on a different judge than the one used to construct the preference pairs, so no equation-level reduction makes the result equivalent to its own input. The paper's self-citations (e.g., LongAlign for packing-based training, LongWriter as a baseline) are procedural or comparative, not the source of the central claim. The lack of human validation for the QwQ-32B Write-judge is a legitimate correctness and validity concern, but it is not circularity: an unvalidated reward model is an assumption about credit assignment, not a case where the prediction is forced by construction. No fitted parameter is renamed as a prediction, and no uniqueness theorem or ansatz is imported from the authors' prior work to forbid alternatives. The evaluation is external to the training objective, so the central derivation is not circular.

Assumptions & free parameters 5 free parameters · 6 assumptions · 0 invented entities

The central claim rests on automated LLM judges for reward generation, training-data filtering, and evaluation; none of these judges is validated against human ratings in the reported experiments. Training and MCTS hyperparameters are chosen by hand without sensitivity analysis. The two structural assumptions in Section 3.2, good plans lead to good drafts and good drafts refine to good finals, are stated but not independently tested.

free parameters (5)
  • MCTS branching factors (5 plans, 4 drafts, 3 refinements) = 5, 4, 3
    Figure 4 fixes the tree shape used to generate DPO preference pairs; no sensitivity analysis is reported.
  • Ordinal reward percentile thresholds = 1/6, 2/6, 4/6, 5/6
    Section 3.2 Step 1 discretizes leaf scores into +2..-2 using hand-chosen percentile cut points.
  • DPO loss coefficient beta = not reported
    Section 3.2 uses the 'standard DPO loss' but beta is not stated; the result depends on this hyperparameter.
  • Number of SFT instructions and stage samples = 4,000 instructions, 12,000 stage samples
    Section 3.1; dataset size is chosen by hand and not varied.
  • Six-dimension rubric selection = 3 general plus 3 special criteria from a pool of 20
    Appendix A.4; rubric dimensions are selected per query by an LLM, not independently validated.
assumptions (6)
  • domain assumption QwQ-32B's averaged six-dimension scores are a valid reward for long-form writing quality.
    Section 3.2 Write-judge; all DPO preferences derive from these scores, with no human validation of the reward model.
  • ad hoc to paper Averaging leaf reward scores over children gives correct intermediate node quality for plans and drafts.
    Section 3.2 Step 2; no evidence that plan or draft quality is a linear average of refined output scores.
  • domain assumption Well-structured plans lead to higher-quality drafts and well-refined drafts yield better final outputs.
    Section 3.2, explicitly stated as 'two key assumptions'; this motivates back-propagation.
  • domain assumption DeepSeek-R1-Distill-Qwen-32B accurately identifies long-form instructions from WildChat and LMSYS.
    Section 3.1; the filtering method in Appendix A.7 is heuristic and not evaluated for precision or recall.
  • domain assumption WritingBench's Qwen2.5-7B critic and GPT-4.1 judge provide valid quality measurements.
    Sections 4.1 to 4.3; the paper relies on these automated judges for headline results, while noting critic limitations.
  • domain assumption GPT-4o agent generated SFT data is high quality and suitable supervision for writing.
    Section 3.1; no human quality check of the 12,000 stage samples is reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SuperWriter: Reflection-Driven Long-Form Generation with Large Language Models." pith.science (2026). https://pith.science/paper/ONX2URWC

@misc{pith2026250604180,
  author       = {Pith},
  title        = {Pith review of: SuperWriter: Reflection-Driven Long-Form Generation with Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ONX2URWC}},
  note         = {Machine review of arXiv:2506.04180}
}
read the original abstract

Long-form text generation remains a significant challenge for large language models (LLMs), particularly in maintaining coherence, ensuring logical consistency, and preserving text quality as sequence length increases. To address these limitations, we propose SuperWriter-Agent, an agent-based framework designed to enhance the quality and consistency of long-form text generation. SuperWriter-Agent introduces explicit structured thinking-through planning and refinement stages into the generation pipeline, guiding the model to follow a more deliberate and cognitively grounded process akin to that of a professional writer. Based on this framework, we construct a supervised fine-tuning dataset to train a 7B SuperWriter-LM. We further develop a hierarchical Direct Preference Optimization (DPO) procedure that uses Monte Carlo Tree Search (MCTS) to propagate final quality assessments and optimize each generation step accordingly. Empirical results across diverse benchmarks demonstrate that SuperWriter-LM achieves state-of-the-art performance, surpassing even larger-scale baseline models in both automatic evaluation and human evaluation. Furthermore, comprehensive ablation studies demonstrate the effectiveness of hierarchical DPO and underscore the value of incorporating structured thinking steps to improve the quality of long-form text generation.

Figures

Figures reproduced from arXiv: 2506.04180 by the authors.

Figure 1
Figure 1. Current LLMs directly generate long text in a single pass, while human writers follow an [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 1
Figure 1. Moreover, such writing processes are often accompanied by extensive cognitive activities, [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. This figure illustrates a three-stage agent framework for long-form generation. In Stage [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figures from the paper (4 more)
Figure 3
Figure 3. Figure 3: Format of the SFT dataset constructed by the [PITH_FULL_IMAGE:figures/full_fig_p005_3.png]
Figure 4
Figure 4. Figure 4: The MCTS begins with 5 distinct writing plans, each leading to 4 written drafts, totaling 20 [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: This figure presents eight donut charts comparing the win rates of our model against seven [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Performance comparison on Writing￾Bench [60] – Avg, ZH and EN. Finally, we conduct an ablation study compris￾ing four different setups evaluated on the Writ￾ingBench benchmark. The first setup uses the base model, Qwen2.5-Instruct, as the perfor￾mance baseline. The sec…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

116 extracted references · 38 canonical work pages

  1. [1]

    The llama 4 herd: The beginning of a new era of natively multimodal ai innovation, April 2025

    Meta AI. The llama 4 herd: The beginning of a new era of natively multimodal ai innovation, April 2025. URL https://ai.meta.com/blog/ llama-4-multimodal-intelligence/

    2025

  2. [2]

    Training-free long-context scaling of large language models

    Chenxin An, Fei Huang, Jun Zhang, Shansan Gong, Xipeng Qiu, Chang Zhou, and Ling- peng Kong. Training-free long-context scaling of large language models. arXiv preprint arXiv:2402.17463, 2024

    arXiv 2024

  3. [3]

    Training a helpful and harmless assistant with reinforcement learning from human feedback

    Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862, 2022

    arXiv 2022

  4. [4]

    Benchmarking foundation models with language-model-as-an- examiner

    Yushi Bai, Jiahao Ying, Yixin Cao, Xin Lv, Yuze He, Xiaozhi Wang, Jifan Yu, Kaisheng Zeng, Yijia Xiao, Haozhe Lyu, et al. Benchmarking foundation models with language-model-as-an- examiner. Advances in Neural Information Processing Systems, 36:78142–78167, 2023. 10

    2023

  5. [5]

    LongAlign: A recipe for long context alignment of large language models

    Yushi Bai, Xin Lv, Jiajie Zhang, Yuze He, Ji Qi, Lei Hou, Jie Tang, Yuxiao Dong, and Juanzi Li. LongAlign: A recipe for long context alignment of large language models. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 1376–1395, Miami, Florida, USA, November 2024. Association for Computational Linguistics. doi: 10.18653/v1/2...

    doi:10.18653/v1/2024 2024

  6. [6]

    Longwriter: Unleashing 10,000+ word generation from long context llms

    Yushi Bai, Jiajie Zhang, Xin Lv, Linzhi Zheng, Siqi Zhu, Lei Hou, Yuxiao Dong, Jie Tang, and Juanzi Li. Longwriter: Unleashing 10,000+ word generation from long context llms. arXiv preprint arXiv:2408.07055, 2024

    arXiv 2024

  7. [7]

    Revision: History, theory, and practice, pages 25–49, 2006

    Anne Becker. Revision: History, theory, and practice, pages 25–49, 2006

    2006

  8. [8]

    Extending context window of large language models via positional interpolation

    Shouyuan Chen, Sherman Wong, Liangjian Chen, and Yuandong Tian. Extending context window of large language models via positional interpolation. arXiv preprint arXiv:2306.15595, 2023

    arXiv 2023

Show all 116 references
  1. [9]

    Longlora: Efficient fine-tuning of long-context large language models

    Yukang Chen, Shengju Qian, Haotian Tang, Xin Lai, Zhijian Liu, Song Han, and Jiaya Jia. Longlora: Efficient fine-tuning of long-context large language models. arXiv preprint arXiv:2309.12307, 2023

    2023 arXiv

  2. [10]

    Brown, Miljan Martic, Shane Legg, and Dario Amodei

    Paul Christiano, Jan Leike, Tom B. Brown, Miljan Martic, Shane Legg, and Dario Amodei. Deep reinforcement learning from human preferences, 2023. URL https://arxiv.org/ abs/1706.03741

    2023 arXiv

  3. [11]

    Smith, and Matt Gardner

    Pradeep Dasigi, Kyle Lo, Iz Beltagy, Arman Cohan, Noah A. Smith, and Matt Gardner. A dataset of information-seeking questions and answers anchored in research papers. In Proc. of NAACL, pages 4599–4610, 2021

    2021

  4. [12]

    DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei F...

    2025 arXiv

  5. [13]

    Zhang, Han Bao, Hanwei Xu, Haocheng Wang, Haowei Zhang, Honghui Ding, Huajian Xin, Huazuo Gao, Hui Li, Hui Qu, J

    DeepSeek-AI, Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Daya Guo, Dejian 11 Yang, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangbo Hao, Guanting Chen, Guowe...

  6. [14]

    A cognitive process theory of writing

    L Flower. A cognitive process theory of writing. Composition and communication, 1981

    1981

  7. [15]

    Chip: Cross-modal hierarchical direct preference optimization for multimodal llms

    Jinlan Fu, Shenzhen Huangfu, Hao Fei, Xiaoyu Shen, Bryan Hooi, Xipeng Qiu, and See-Kiong Ng. Chip: Cross-modal hierarchical direct preference optimization for multimodal llms. arXiv preprint arXiv:2501.16629, 2025

    2025 arXiv

  8. [16]

    Data engineering for scaling language models to 128k context

    Yao Fu, Rameswar Panda, Xinyao Niu, Xiang Yue, Hannaneh Hajishirzi, Yoon Kim, and Hao Peng. Data engineering for scaling language models to 128k context. arXiv preprint arXiv:2402.10171, 2024

    2024 arXiv

  9. [17]

    Plan, write, and revise: an interactive system for open-domain story generation, 2019

    Seraphina Goldfarb-Tarrant, Haining Feng, and Nanyun Peng. Plan, write, and revise: an interactive system for open-domain story generation, 2019. URL https://arxiv.org/ abs/1904.02357

    2019 arXiv

  10. [18]

    A framework for a cognitive theory of writing

    Allan Gollins and Dedre Gentner. A framework for a cognitive theory of writing. In Cognitive processes in writing, pages 51–72. Routledge, 2016

    2016

  11. [19]

    Lm-infinite: Simple on-the-fly length generalization for large language models

    Chi Han, Qifan Wang, Wenhan Xiong, Yu Chen, Heng Ji, and Sinong Wang. Lm-infinite: Simple on-the-fly length generalization for large language models. arXiv preprint arXiv:2308.16137, 2023

    2023 arXiv

  12. [20]

    360-llama-factory, 2024

    Shousheng Jia Haosheng Zou, Xiaowei Lv and Xiangzheng Zhang. 360-llama-factory, 2024. URL https://github.com/Qihoo360/360-LLaMA-Factory

    2024

  13. [21]

    Pair: Planning and iterative refinement in pre-trained transformers for long text generation, 2020

    Xinyu Hua and Lu Wang. Pair: Planning and iterative refinement in pre-trained transformers for long text generation, 2020. URL https://arxiv.org/abs/2010.02301

    2020 arXiv

  14. [22]

    Llm maybe longlm: Self-extend llm context window without tuning

    Hongye Jin, Xiaotian Han, Jingfeng Yang, Zhimeng Jiang, Zirui Liu, Chia-Yuan Chang, Huiyuan Chen, and Xia Hu. Llm maybe longlm: Self-extend llm context window without tuning. arXiv preprint arXiv:2401.01325, 2024

    2024 arXiv

  15. [23]

    Rossi, Franck Dernoncourt, Hanieh Deilamsalehy, Xiang Chen, Ruiyi Zhang, Shubham Agarwal, Nedim Lipka, Chien Van Nguyen, Thien Huu Nguyen, and Hamed Zamani

    Ishita Kumar, Snigdha Viswanathan, Sushrita Yerra, Alireza Salemi, Ryan A. Rossi, Franck Dernoncourt, Hanieh Deilamsalehy, Xiang Chen, Ruiyi Zhang, Shubham Agarwal, Nedim Lipka, Chien Van Nguyen, Thien Huu Nguyen, and Hamed Zamani. Longlamp: A benchmark 12 for personalized lon...

    2024

  16. [24]

    QASA: Advanced question answering on scientific articles

    Yoonjoo Lee, Kyungjae Lee, Sunghyun Park, Dasol Hwang, Jaehyeon Kim, Hong-In Lee, and Moontae Lee. QASA: Advanced question answering on scientific articles. In Proc. of ICML, pages 19036–19052, 2023

    2023

  17. [25]

    Teach llms to personalize - an approach inspired by writing education

    Cheng Li, Mingyang Zhang, Qiaozhu Mei, Yaqing Wang, Spurthi Amba Hombaiah, Yi Liang, and Michael Bendersky. Teach llms to personalize - an approach inspired by writing education. ArXiv, 2023

    2023

  18. [26]

    TPO: Aligning large language models with multi- branch & multi-step preference trees

    Weibin Liao, Xu Chu, and Yasha Wang. TPO: Aligning large language models with multi- branch & multi-step preference trees. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=O0sQ9CPzai

    2025

  19. [27]

    Task- adaptive tokenization: Enhancing long-form text generation efficacy in mental health and beyond

    Siyang Liu, Naihao Deng, Sahand Sabour, Yilin Jia, Minlie Huang, and Rada Mihalcea. Task- adaptive tokenization: Enhancing long-form text generation efficacy in mental health and beyond. In Proc. of EMNLP, pages 15264–15281, 2023

    2023

  20. [28]

    Large language models: A survey, 2024

    Shervin Minaee, Tomas Mikolov, Narjes Nikzad, Meysam Chenaghlu, Richard Socher, Xavier Amatriain, and Jianfeng Gao. Large language models: A survey, 2024. URL https:// arxiv.org/abs/2402.06196

    2024 arXiv

  21. [29]

    Learning to reason with llms, 2024

    OpenAI. Learning to reason with llms, 2024. URL https://openai.com/index/ learning-to-reason-with-llms/

    2024

  22. [30]

    Introducing gpt-4.1 in the api, April 2025

    OpenAI. Introducing gpt-4.1 in the api, April 2025. URL https://openai.com/index/ gpt-4-1/. Accessed: 2025-05-02

    2025

  23. [31]

    OpenAI, :, Aaron Hurst, Adam Lerer, Adam P. Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, Aleksander M ˛ adry, Alex Baker-Whitcomb, Alex Beutel, Alex Borzunov, Alex Carney, Alex Chow, Alex Kirillov, Alex Nichol, Alex ...

    2024 arXiv

  24. [32]

    Training language models to follow instructions with human feedback

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:2773...

    2022

  25. [33]

    Data, data everywhere: A guide for pretraining dataset construction

    Jupinder Parmar, Shrimai Prabhumoye, Joseph Jennings, Bo Liu, Aastha Jhunjhunwala, Zhilin Wang, Mostofa Patwary, Mohammad Shoeybi, and Bryan Catanzaro. Data, data everywhere: A guide for pretraining dataset construction. arXiv preprint arXiv:2407.06380, 2024

    2024 arXiv

  26. [34]

    Yarn: Efficient context window extension of large language models

    Bowen Peng, Jeffrey Quesnelle, Honglu Fan, and Enrico Shippole. Yarn: Efficient context window extension of large language models. arXiv preprint arXiv:2309.00071, 2023

    2023 arXiv

  27. [35]

    Suri: Multi-constraint instruction following for long-form text generation

    Chau Minh Pham, Simeng Sun, and Mohit Iyyer. Suri: Multi-constraint instruction following for long-form text generation. arXiv preprint arXiv:2406.19371, 2024

    2024 arXiv

  28. [36]

    Language models can self-lengthen to generate long texts

    Shanghaoran Quan, Tianyi Tang, Bowen Yu, An Yang, Dayiheng Liu, Bofei Gao, Jianhong Tu, Yichang Zhang, Jingren Zhou, and Junyang Lin. Language models can self-lengthen to generate long texts. arXiv preprint arXiv:2410.23933, 2024

    arXiv

  29. [37]

    Hellobench: Evaluating long text generation capabilities of large language models, 2024

    Haoran Que, Feiyu Duan, Liqun He, Yutao Mou, Wangchunshu Zhou, Jiaheng Liu, Wenge Rong, Zekun Moore Wang, Jian Yang, Ge Zhang, Junran Peng, Zhaoxiang Zhang, Songyang Zhang, and Kai Chen. Hellobench: Evaluating long text generation capabilities of large language models, 2024. U...

    2024 arXiv

  30. [38]

    Qwen2.5 technical report, 2025

    Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, ...

    2025 arXiv

  31. [39]

    Direct preference optimization: Your language model is secretly a reward model

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL https://openr...

    2023

  32. [40]

    Direct preference optimization: Your language model is secretly a reward model

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36, 2024

    2024

  33. [41]

    Large language models for code generation: The practitioners perspective, 2025

    Zeeshan Rasheed, Muhammad Waseem, Kai Kristian Kemell, Aakash Ahmad, Malik Abdul Sami, Jussi Rasku, Kari Systä, and Pekka Abrahamsson. Large language models for code generation: The practitioners perspective, 2025. URL https://arxiv.org/abs/2501. 16998

    2025

  34. [42]

    Deepspeed: System optimizations enable training deep learning models with over 100 billion parameters

    Jeff Rasley, Samyam Rajbhandari, Olatunji Ruwase, and Yuxiong He. Deepspeed: System optimizations enable training deep learning models with over 100 billion parameters. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 3...

    2020

  35. [43]

    Expert: Effective and explainable evaluation of personalized long-form text generation, 2025

    Alireza Salemi, Julian Killingback, and Hamed Zamani. Expert: Effective and explainable evaluation of personalized long-form text generation, 2025. URL https://arxiv.org/ abs/2501.14956

    2025 arXiv

  36. [44]

    Reasoning-enhanced self-training for long-form personalized text generation, 2025

    Alireza Salemi, Cheng Li, Mingyang Zhang, Qiaozhu Mei, Weize Kong, Tao Chen, Zhuowan Li, Michael Bendersky, and Hamed Zamani. Reasoning-enhanced self-training for long-form personalized text generation, 2025. URL https://arxiv.org/abs/2501.04167

    2025 arXiv

  37. [45]

    Agent laboratory: Using llm agents as research assistants,

    Samuel Schmidgall, Yusheng Su, Ze Wang, Ximeng Sun, Jialian Wu, Xiaodong Yu, Jiang Liu, Zicheng Liu, and Emad Barsoum. Agent laboratory: Using llm agents as research assistants,

  38. [46]

    The story workshop method: Writing from start to finish

    John Schultz. The story workshop method: Writing from start to finish. College English, 39(4): 381–389, 1977

    1977

  39. [47]

    URL https://arxiv.org/abs/2501.04227

    arXiv

  40. [48]

    SlimPajama: A 627B token cleaned and dedu- plicated version of RedPajama

    Daria Soboleva, Faisal Al-Khateeb, Robert Myers, Jacob R Steeves, Joel Hestness, and Nolan Dey. SlimPajama: A 627B token cleaned and dedu- plicated version of RedPajama. https://www.cerebras.net/blog/ slimpajama-a-627b-token-cleaned-and-deduplicated-version-of-redpajama ,

  41. [49]

    Story workshop as a method of teaching writing.College English, 35(2):141–160, 1973

    Betty Sheflett. Story workshop as a method of teaching writing.College English, 35(2):141–160, 1973

    1973

  42. [50]

    ProxyQA: An alternative framework for evaluating long-form text generation with large language models

    Haochen Tan, Zhijiang Guo, Zhan Shi, Lu Xu, Zhili Liu, Yunlong Feng, Xiaoguang Li, Yasheng Wang, Lifeng Shang, Qun Liu, and Linqi Song. ProxyQA: An alternative framework for evaluating long-form text generation with large language models. In Lun-Wei Ku, Andre Martins, and Vive...

    2024

  43. [51]

    The collaborative discussion model: Developing writing skills through prewriting discussion

    Pospelova Tatiana. The collaborative discussion model: Developing writing skills through prewriting discussion. Journal of Language and Education, 7(1 (25)):156–170, 2021

    2021

  44. [52]

    ASQA: Factoid questions meet long-form answers

    Ivan Stelmakh, Yi Luan, Bhuwan Dhingra, and Ming-Wei Chang. ASQA: Factoid questions meet long-form answers. In Proc. of EMNLP, pages 8273–8288, 2022

    2022

  45. [53]

    Qwq-32b: Embracing the power of reinforcement learning, March 2025

    Qwen Team. Qwq-32b: Embracing the power of reinforcement learning, March 2025. URL https://qwenlm.github.io/blog/qwq-32b/

    2025

  46. [54]

    Math-shepherd: Verify and reinforce LLMs step-by-step without human annotations

    Peiyi Wang, Lei Li, Zhihong Shao, Runxin Xu, Damai Dai, Yifei Li, Deli Chen, Yu Wu, and Zhifang Sui. Math-shepherd: Verify and reinforce LLMs step-by-step without human annotations. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Proceedings of the 62nd Annual Meeti...

    2024 doi

  47. [55]

    Qwen2.5: A party of foundation models, September 2024

    Qwen Team. Qwen2.5: A party of foundation models, September 2024. URL https: //qwenlm.github.io/blog/qwen2.5/

    2024

  48. [56]

    Autosurvey: Large language models can automatically write surveys

    Yidong Wang, Qi Guo, Wenjin Yao, Hongbo Zhang, Xin Zhang, Zhen Wu, Meishan Zhang, Xinyu Dai, Min Zhang, Qingsong Wen, Wei Ye, Shikun Zhang, and Yue Zhang. Autosurvey: Large language models can automatically write surveys. In Proc. of NeurIPS, 2024

    2024

  49. [57]

    Maurice Weber, Daniel Y . Fu, Quentin Anthony, Yonatan Oren, Shane Adams, Anton Alexan- drov, Xiaozhong Lyu, Huu Nguyen, Xiaozhe Yao, Virginia Adams, Ben Athiwaratkun, Rahul Chalamala, Kezhen Chen, Max Ryabinin, Tri Dao, Percy Liang, Christopher Ré, Irina Rish, and Ce Zhang. R...

    2024

  50. [58]

    Generating long-form story using dynamic hierarchical outlining with memory-enhancement,

    Qianyue Wang, Jinwu Hu, Zhengping Li, Yufeng Wang, daiyuan li, Yu Hu, and Mingkui Tan. Generating long-form story using dynamic hierarchical outlining with memory-enhancement,

  51. [59]

    Shifting long-context llms research from input to output, 2025

    Yuhao Wu, Yushi Bai, Zhiqing Hu, Shangqing Tu, Ming Shan Hee, Juanzi Li, and Roy Ka- Wei Lee. Shifting long-context llms research from input to output, 2025. URL https: //arxiv.org/abs/2503.04723

    2025 arXiv

  52. [60]

    Writingbench: A comprehensive benchmark for generative writing, 2025

    Yuning Wu, Jiahao Mei, Ming Yan, Chenliang Li, Shaopeng Lai, Yuran Ren, Zijia Wang, Ji Zhang, Mengyue Wu, Qin Jin, and Fei Huang. Writingbench: A comprehensive benchmark for generative writing, 2025. URL https://arxiv.org/abs/2503.05244

    2025

  53. [61]

    Efficient streaming language models with attention sinks

    Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, and Mike Lewis. Efficient streaming language models with attention sinks. arXiv preprint arXiv:2309.17453, 2023

    2023 arXiv

  54. [62]

    Longgenbench: Benchmarking long-form generation in long context llms

    Yuhao Wu, Ming Shan Hee, Zhiqing Hu, and Roy Ka-Wei Lee. Longgenbench: Benchmarking long-form generation in long context llms. arXiv preprint arXiv:2409.02076, 2024

    2024 arXiv

  55. [63]

    Qwen2 technical report

    An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma, Jin Xu, Jingren Zhou, Jinze Bai, Jinzheng...

    2024 arXiv

  56. [64]

    Re3: Generating longer stories with recursive reprompting and revision

    Kevin Yang, Yuandong Tian, Nanyun Peng, and Dan Klein. Re3: Generating longer stories with recursive reprompting and revision. In Proc. of EMNLP, pages 4393–4479, 2022. 16

    2022

  57. [65]

    DOC: Improving long story coherence with detailed outline control

    Kevin Yang, Dan Klein, Nanyun Peng, and Yuandong Tian. DOC: Improving long story coherence with detailed outline control. In Proc. of ACL, pages 3378–3465, 2023

    2023

  58. [66]

    Effective long-context scaling of foundation models

    Wenhan Xiong, Jingyu Liu, Igor Molybog, Hejia Zhang, Prajjwal Bhargava, Rui Hou, Louis Martin, Rashi Rungta, Karthik Abinav Sankararaman, Barlas Oguz, et al. Effective long-context scaling of foundation models. In Proceedings of the 2024 Conference of the North American Chapte...

    2024

  59. [67]

    Soar- ing from 4k to 400k: Extending llm’s context with activation beacon

    Peitian Zhang, Zheng Liu, Shitao Xiao, Ninglu Shao, Qiwei Ye, and Zhicheng Dou. Soar- ing from 4k to 400k: Extending llm’s context with activation beacon. arXiv preprint arXiv:2401.03462, 2024

    2024 arXiv

  60. [68]

    Wildchat: 1m chatgpt interaction logs in the wild

    Wenting Zhao, Xiang Ren, Jack Hessel, Claire Cardie, Yejin Choi, and Yuntian Deng. Wildchat: 1m chatgpt interaction logs in the wild. arXiv preprint arXiv:2405.01470, 2024

    2024 arXiv

  61. [69]

    P Xing, Joseph E

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Tianle Li, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zhuohan Li, Zi Lin, Eric. P Xing, Joseph E. Gonzalez, Ion Stoica, and Hao Zhang. Lmsys-chat-1m: A large-scale real-world llm conversation dataset, 2023

    2023

  62. [70]

    Plan- and-write: Towards better automatic storytelling, 2019

    Lili Yao, Nanyun Peng, Ralph Weischedel, Kevin Knight, Dongyan Zhao, and Rui Yan. Plan- and-write: Towards better automatic storytelling, 2019. URL https://arxiv.org/abs/ 1811.05701

    2019 arXiv

  63. [71]

    Gonzalez, Clark Barrett, and Ying Sheng

    Lianmin Zheng, Liangsheng Yin, Zhiqiang Xie, Chuyue Sun, Jeff Huang, Cody Hao Yu, Shiyi Cao, Christos Kozyrakis, Ion Stoica, Joseph E. Gonzalez, Clark Barrett, and Ying Sheng. Sglang: Efficient execution of structured language model programs. arXiv preprint arXiv:2312.07104,

    arXiv

  64. [74]

    Judging llm-as-a-judge with mt-bench and chatbot arena

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in Neural Information Processing Systems, 36:46595–46623, 2023

    2023

  65. [76]

    URL https://arxiv.org/abs/2312.07104

    doi: 10.48550/arXiv.2312.07104. URL https://arxiv.org/abs/2312.07104. 17 A Appendix A.1 Stage-1 Plan Prompt This appendix provides a brief overview of the prompt modules used in SuperWriter-Agent Stage-1 Plan. There are a total of 6 modules, each serving a specific role in the...

    doi arXiv

  66. [77]

    - Type: Choose an appropriate writing style (e.g., argumentative, expository, or business writing) that aligns with the intended format and structure

    Define the Purpose and Type of Writing - Purpose: Clearly establish the objective of the piece (e.g., to inform, persuade, or inspire), setting the tone and direction accordingly. - Type: Choose an appropriate writing style (e.g., argumentative, expository, or business writing...

  67. [78]

    - Structure: Develop a coherent framework that maintains a logical flow throughout the piece

    Plan Content and Structure - Key Points: Outline the essential information to ensure a clear and focused topic. - Structure: Develop a coherent framework that maintains a logical flow throughout the piece

  68. [79]

    Provide detailed descriptions, including specifics such as names, gender, and relationships

    Characters and Plot (for Narrative Writing) - Character Development: Define the traits and motivations of all characters. Provide detailed descriptions, including specifics such as names, gender, and relationships. - Plot Development: Establish pivotal plot points and emotiona...

  69. [80]

    - Other Key Elements: Include any genre- or task-specific components

    Additional Guidelines - Formatting Requirements: Automatically select an appropriate output format (e.g., Markdown, bullet points) based on content and presentation needs to enhance visual clarity and appeal. - Other Key Elements: Include any genre- or task-specific components...

  70. [81]

    Purpose: What is the main objective of this paragraph? What message or emotion should it convey?

  71. [82]

    Structure: How should the content of this paragraph be organized? What logical sequence would best ensure clarity and coherence, and how will it connect tightly with the previous content?

  72. [83]

    Transitions: How will this paragraph naturally link to the one before it? Are there specific transition sentences or bridging techniques that can be used?

  73. [84]

    Details and Examples: What details, facts, or examples are needed to support the main idea? How should these be arranged for maximum impact?

  74. [85]

    Language Style and Techniques: What kind of language style should be used to achieve the goal? Are there rhetorical devices (such as metaphors or analogies) that could enhance the paragraph’s impact—while still being clear, readable, and easy to understand for the audience?

  75. [86]

    Based on the outline and the key point for this paragraph, construct a detailed writing plan

    Markdown Format: Use Markdown to structure the output neatly, including headings, bullet points, or bold text to improve readability. Based on the outline and the key point for this paragraph, construct a detailed writing plan. Add any other relevant considerations as needed, ...

  76. [87]

    Logical Consistency: Is this paragraph logically consistent with the rest of the document? Are there any illogical transitions or abrupt shifts?

  77. [88]

    Completeness: Does this paragraph provide enough information to support its main idea? Are there any important missing details?

  78. [89]

    Focus on offering detailed revision suggestions for paragraph idx+1

    Coherence: Does this paragraph connect smoothly with the surrounding paragraphs? Would transitional sentences help improve the flow? Please provide at least two specific improvement suggestions. Focus on offering detailed revision suggestions for paragraph idx+1. Only provide ...

  79. [90]

    This rubric is reused across all queries

    Rubric Definition (evaluation_criteria): Defines the complete set of General and Special evaluation dimensions. This rubric is reused across all queries

  80. [91]

    Criterion Selection Schema (format_query): Specifies the JSON format for selecting six criteria (three General, three query-relevant Special) and rewriting their Definitions and Standards to match the specific query context

  81. [92]

    The output is a JSON object referred to as evaluate_standard

    Criterion Selection Prompt: Combines the rubric and schema to instruct the model to select and customize criteria. The output is a JSON object referred to as evaluate_standard

  82. [93]

    Scoring Format Schema (format_eval): Specifies the expected evaluation output format: for each selected criterion, the model must return an Analysis string and a numeric Score

  83. [94]

    The model performs criterion-wise evaluation and outputs a structured JSON

    Final Scoring Prompt: Provides the model with a query, its generated result, the customized evaluate_standard, and the format_eval schema. The model performs criterion-wise evaluation and outputs a structured JSON. Outcome: This pipeline yields structured, query-specific evalu...

  84. [95]

    Standards: 10: Fully aligned with the user’s needs, highly relevant to the request

    General Criteria (Applicable to All Genres) 1.1 Relevance Definition: How well the content matches the user’s request, and whether it addresses the intended purpose or topic. Standards: 10: Fully aligned with the user’s needs, highly relevant to the request. 7–9: Mostly releva...

  85. [96]

    Criterion 1

    Special Criteria (Applicable to Specific Genres) 2.1 Creativity and Uniqueness Definition: Whether the content is innovative, offering new perspectives, or showcasing original expression. Standards: 10: Highly creative and unique, presenting entirely new or unconventional idea...

  86. [97]

    Think step-by-step about why each criterion is relevant to the query

  87. [98]

    Think step-by-step through the query and how each criterion applies

  88. [99]

    Provide a brief analysis for each selected criterion on how it applies to the query

  89. [100]

    Integrate the above reasoning into the Definition and Standards sections of each criterion. {format_query} prompt ### Query: {query} ### Result: <start> {clean_res} <end> ### Evaluation Standard: {json.dumps(evaluate_standard, ensure_ascii=False)} Based on the provided info, p...

  90. [101]

    Relevance and Completeness: Does the assistant fully respond to the writing prompt? Does the length meet the user’s query expectations? Is the content relevant to the topic, and does it provide sufficient depth, length, and detail, rather than drifting off-topic or simplistic?

  91. [102]

    The overall quality of the writing is high, with elegant

    Writing Quality : Evaluate whether the assistant’s writing is clear, fluent, and free of obvious grammatical errors. The overall quality of the writing is high, with elegant

  92. [103]

    Does the assistant offer fresh perspectives, unique insights, or demonstrate a certain level of originality?

    Creativity and Originality: If applicable, assess the creativity of the response. Does the assistant offer fresh perspectives, unique insights, or demonstrate a certain level of originality?

  93. [104]

    Properly justified repetition is permissible

    Specificity and Detail : Determine whether the assistant provides concrete examples or detailed explanations. Properly justified repetition is permissible

  94. [105]

    岩灵祭祀需三色岩盐

    Tone and Style : Is the tone appropriate for the writing prompt? Is the writing style consistent throughout? Consider whether it aligns with the expectations of the intended audience or writing purpose. After evaluating each response, determine which one is superior based on t...

    2020

  95. [106]

    above 2K

    Step 1: Predicting Length Exceedance (Prompt 1): The first prediction is made by checking whether the input exceeds 2,000 words. A carefully crafted prompt (Prompt 1) is provided to the model to predict if the content’s expected word count will surpass the 2K threshold. The mo...

  96. [107]

    Based on the result from Step 1, Prompt 2 is designed to predict whether the content is in the 2K-4K, 4K-8K, 8K-16K, or 16K+ category

    Step 2: Predicting Exact Length Requirement (Prompt 2): Once the model predicts whether the task exceeds 2,000 words, a second prediction is made to determine the exact length category. Based on the result from Step 1, Prompt 2 is designed to predict whether the content is in ...

    2000

  97. [108]

    Depth and Complexity: Does the task require detailed explanations, in-depth analysis, or comprehensive coverage of complex topics?

  98. [109]

    Scope and Breadth: Does the task cover multiple subtopics, perspectives, or extensive subject matter?

  99. [110]

    Structure and Sections: Does the output need to include multiple sections such as introduc- tions, literature reviews, methodologies, results, discussions, and conclusions?

  100. [111]

    #*# Yes” or “#*# No

    Research and References: Does the task require extensive research, citations, and referenc- ing of multiple sources? Response Format: • Answer with either “#*# Yes” or “#*# No”. • Provide a concise justification based on the guidelines above. Example 1: Query: Is Sanskrit the ...

    2000

  101. [112]

    Depth and Complexity: Does the task require detailed explanations, in-depth analysis, or complex reasoning?

  102. [113]

    Scope and Breadth: Does the task cover multiple subtopics, perspectives, or an extensive subject matter? 26

  103. [114]

    Structure and Sections: Does the output require multiple sections (e.g., introduction, literature review, methodologies, results, discussions, conclusions)?

  104. [115]

    Research and References: Does the task require significant research, citations, or references to multiple sources?

  105. [116]

    Less than 2000 words

    Detail Level: Is the task expected to be highly detailed, or can it be summarized concisely? Response Format: - Choose the most likely word count category: “Less than 2000 words”, “2000 words”, “4000 words”, “8000 words”, or “16000 words”. Using (### Category: “Chosen category...

    2000

  106. [2023]

    URL https://huggingface.co/datasets/cerebras/ SlimPajama-627B

  107. [2024]

    URL https://arxiv.org/abs/2412.13575

    arXiv

  108. [2025]

    URL https://arxiv.org/abs/2412.19437

    arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.