{"id":"5527302f-94ea-41d2-9161-2ba23cffc496","arxiv_id":"2608.03138","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"RL with per-section rewards and a revision penalty lets a single LLM pass generate structured paper introductions comparable to GPT-5.1 in human preference.","lead":"StructPO is a training method that lets a language model write a paper introduction in a single pass, using section markers instead of chaining many prompts. If the results hold, structured academic writing becomes much cheaper and closer in quality to top closed models.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Automated structural/section metrics share the GPT-4o decomposition used as training reward; independent human segmentation is needed before claiming structural gains.","rationale":"The reader's weakest assumption correctly identifies the fixed GPT-4o decomposition as load-bearing. I agree, and I would add that the circularity is not merely about training-data bias: the same GPT-4o-derived labels feed the reward functions (Eqs. 3–4), the DeBERTa structure classifier, and the automated Section/Structure evaluation. This means the headline automated superiority on structural metrics is partly by construction. The paper deserves credit for a coherent RL framework, detailed ablations, and for holding out judge metrics and human evaluation; however, the human evaluation is small (30 samples, 3 annotators) and is only claimed to show competitiveness, not structural superiority. Independent human annotation of rhetorical structure would settle whether the structural gains are real. Since this is the same concern the reader raised, the conditional verdict should remain unchanged until such a test is run.","tokens_in":22223,"tokens_out":4759,"duration_ms":60619,"concrete_test":"Take 50 ACL test inputs; have two expert annotators manually segment each generated introduction (StructPO-8B/32B, STIG, AutoSurvey, GPT-5.1) into background/problem/method/contribution and rate rhetorical purity. Recompute Section Similarity and Structure Score using these human labels instead of GPT-4o's decomposition, and report inter-annotator agreement. If StructPO's advantage over baselines disappears or shrinks to within noise, the automated structural claim is an artifact of shared GPT-4o supervision.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing concern is that the automated evidence for the central structural claim is circular. Stage-level supervision is obtained by using GPT-4o with a fixed decomposition prompt (§Experiments, Appendix), and the same GPT-4o-derived segmentation is used to compute Section Similarity and to train the DeBERTa classifier behind Structure Score. The training rewards (Eqs. 3–4) include rmatch and rstruct, so StructPO is explicitly optimized against the same labels that later define its headline metrics. Consequently, Table 1's large StructPO-vs-baseline gaps in Sec./Struc. may reflect fitting to GPT-4o's decomposition preferences rather than objective rhetorical quality. The LLM-judge metrics (AWQ/SFC/etc.) are not used in training, but they are still not independent human ground truth; the only human evaluation is 30 papers × 3 annotators with no significance testing, and it is used only for the GPT-5.1 competitiveness claim, not for the structural-advantage claim. Without an external annotation of rhetorical structure, the paper's quantitative superiority on structural/section metrics is not established.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes StructPO, a structure-aware policy optimization framework for generating academic paper introductions in a single pass using explicit stage tokens. The method decomposes an introduction into eight units (four rhetorical sections × outline/content), computes stage-level rewards for semantic similarity, length, and structural rationality, and uses a Struct-aware Relative Advantage (SRA) estimator with a refinement-guided penalty to train a policy. The authors evaluate on 1,176 ACL 2025 papers and 141 CVPR papers, comparing against prompt-based, workflow-based, and closed-source LLM baselines. They report that StructPO improves semantic alignment, structural rationality, and inference efficiency, and that a Qwen3-32B variant is competitive with GPT-5.1 in a small blind human evaluation.","tokens_in":22563,"tokens_out":4842,"duration_ms":59833,"significance":"If the results held, the paper would make a useful contribution: it formulates structured introduction generation as a single-pass policy optimization problem, provides a concrete dense-reward design, and demonstrates a substantial reduction in inference overhead relative to multi-stage workflows. The paper also includes detailed training configurations, dataset splits, and reproduction details (e.g., Verl, SGLang, hyperparameters), which are valuable strengths. The central limitation is that the automated evaluation metrics are essentially the same functions as the training rewards, so the headline automated gains over closed-source models are partly by construction. The only independent human evidence is a small, non-significant win-rate comparison against GPT-5.1, and it does not address the structural-advantage claim. The paper's own conclusion wisely notes the fixed eight-stage template limitation, but this does not resolve the metric-circularity concern. With an external evaluation of rhetorical structure and appropriate statistical reporting, the central claims could be supported.","major_comments":[{"comment":"The automated metrics Sem., Sec., Len., and Struc. are, by the paper's own equations, the quantities the policy is trained to maximize. Sem. corresponds to the embedding similarity r_sim used in content rewards, Sec. corresponds to the section-level similarity derived from the GPT-4o decomposition (the same supervision used for outline rewards r_match), Len. corresponds to the Gaussian/linear length regularization r_gau/r_lin, and Struc. corresponds to r_struct computed by the DeBERTa classifier. Eq. (4) and the global reward R_global therefore train against these exact functions. Tables 1 and 2 then report gains on these same functions as 'Auto Overall.' The paper notes that Soundness, Presentation, and Contribution are not used in training, but these are LLM-judge metrics, not independent human ground truth. To support the structural-advantage claim, the authors should add an external","section":"§Evaluation Metrics / §Reward Design (Eq. 4)"},{"comment":"The ground truth for both reward computation and evaluation is a single GPT-4o decomposition with a fixed prompt. The paper states: 'Stage-level supervision is obtained by using GPT-4o with a fixed decomposition prompt to segment each introduction' and 'The resulting section-content and section-outline pairs are used for supervised training, reward computation and evaluation.' A systematic bias in GPT-4o's segmentation (e.g., misclassifying method sentences as background, or over-partitioning contributions) will be learned by the reward model and then reported as high Sec./Struc. The reported 94.76% classifier accuracy is on the same GPT-4o label distribution, so it does not break the circle. An independent human-annotated segmentation (with inter-annotator agreement) is needed before one can conclude that StructPO produces objectively more rational rhetorical structure.","section":"§Experiments / Appendix 'Prompt for Structural Decomposition'"},{"comment":"The only human evaluation is 30 papers × 3 annotators, with no significance testing. The win rate is 53.3% (16/30) and vote share 55.6% (50/90); a two-sided binomial test of 50/90 gives p ≈ 0.36, so the result is not statistically distinguishable from chance. The abstract's claim that StructPO 'remains competitive with GPT-5.1' is appropriately cautious, but the text later says StructPO 'can be preferred over GPT-5.1 in human evaluation,' which is stronger than the data support. Also, this evaluation only compares against GPT-5.1; it does not test the claimed structural advantage over workflow baselines. Please report a proper significance test and, ideally, additional annotation sample size.","section":"§Human Evaluation / Table 4"}],"minor_comments":[{"comment":"The ablation table's 'Overall' is defined as a rescaled composite of four automated and two judge metrics, which differs from 'Auto Overall' in Table 2. Although the caption explains this, the shared term 'Overall' is confusing and should be renamed (e.g., 'Composite') to avoid misleading comparisons across tables.","section":"Table 3 caption / 'Overall' definition"},{"comment":"The vertical axis mixes 0–100 normalized scores with 1–5 judge scores. The parenthetical values clarify the mapping, but a dual-axis or separate panels would improve readability.","section":"Fig. 3 caption"},{"comment":"The variance expression Var(g) ∝ Var(r_k) + Σ_{j≠k} Var(r_j) is simply the variance of a sum under independence assumptions. It is not incorrect, but the notation overstates the contribution; the more relevant point is the attribution noise, not the variance decomposition itself.","section":"Eq. (6)"},{"comment":"The paper honestly states that StructPO relies on a fixed eight-stage template and may require adaptation for theoretical, survey, or non-standard papers. This limitation should be discussed earlier, since it qualifies the 'generalization' claim and the CVPR transfer result.","section":"Conclusion / fixed template limitation"}],"recommendation":"major_revision","confidential_remarks":"The core technical idea (single-pass structured policy with dense, stage-level advantages) is interesting and the reproduction detail is strong. My main concern is the circularity between training rewards and automated evaluation metrics; this needs to be addressed with independent human annotation or a clearly external metric before the paper's central structural claims can be accepted. The human evaluation is too small to support the 'preferred over GPT-5.1' statement. I recommend major revision, not rejection, because the issues are fixable within the manuscript's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Read the paper, and the stress-test note lands. The load-bearing structural result — StructPO beats closed-source LLMs on Sec. and Struc. — rests on metrics that are the training rewards themselves. Eq. (4) optimizes rmatch, rsim, rstruct, and length; Tables 1 and 2 report Sem., Sec., Len., and Struc. built from the same GPT-4o decomposition and the same DeBERTa classifier. The large gaps in Sec./Struc. are partly by construction. To the paper's credit, it doesn't hide this: the appendix states clearly that GPT-4o's fixed prompt produces the stage labels and the classifier training data. The real question is whether that decomposition tracks human judgment of rhetorical quality, and the paper provides no external check. The only human eval is 30 papers × 3 annotators with no significance test, and it's used for the GPT-5.1 competitiveness claim, not for the structural-advantage claim. So the central structural claim is not established.\n\nWhat's actually new: combining explicit stage tokens with stage-level advantage estimation and a refinement penalty. STIG used stage tokens with SFT only; fine-grained credit assignment in prior work targeted math/coding. Internalizing a multi-stage workflow into a single-pass policy with dense credit assignment is a legitimate and useful extension. The token-efficiency analysis is concrete and shows a real benefit over workflow-based systems. The method section is clear, and the writing is honest about the fixed-template limitation.\n\nSoft spots besides circularity: no artifacts released, which hurts reproducibility; ablations only run on 7B/8B, with 32B results incomplete; judge metrics are LLM-as-a-judge and could share preferences with the training signal, though AWQ/SFC are not training rewards. The 'surpasses all evaluated closed-source LLMs on automated metrics' line should be tempered.\n\nWho this is for: people working on RLHF/GRPO for long-form structured generation, and anyone designing evaluation for writing tasks. It deserves a serious referee, but the referee should ask for independent annotation of rhetorical structure and significance testing on the human eval. As is, I'd treat the structural results as suggestive, not decisive.","headline":"The method is a plausible extension of stage-token RL for structured writing, but the headline structural metrics are the same functions the policy is trained on, so the claimed edge over closed-source LLMs is only partly supported.","tokens_in":22971,"tokens_out":2567,"would_cite":true,"duration_ms":29890,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A single RL-trained policy can generate a structured paper introduction in one pass and rival multi-stage writing workflows and large closed-source models.","keywords":["reinforcement learning","policy optimization","credit assignment","introduction generation","academic writing","stage tokens","single-pass generation","LLM writing workflow"],"falsifier":"Take a set of introductions with non-canonical rhetoric (theory papers or surveys that skip an explicit contributions section), have human experts judge quality, and compare human rankings to StructPO's structure-score rankings. If introductions the paper's classifier calls structurally correct are systematically rated worse by experts—or if a workflow baseline wins human preference while losing on the automated metrics—the fixed decomposition, not the writing capability, would be carrying the result.","tokens_in":22186,"feed_emoji":"📝","tokens_out":8430,"duration_ms":86394,"temperature":0.7,"pith_summary":"Today's best systems for writing a paper introduction split the work into stages—outline, draft, revise—orchestrated through prompts and agent calls. This paper argues that the whole workflow can instead be trained into a single policy that produces an introduction in one pass, with explicit markers separating background, problem, method, and contribution sections and their outlines. The key move is a reward design that judges each stage separately, so a good background is not punished for a weak method paragraph; a global alignment signal still keeps the whole introduction coherent. The authors report that this one-pass model beats workflow-based pipelines on automated metrics, lowers inference tokens by roughly 74%, transfers to a new domain, and at 32B scale is judged about as good as GPT-5.1 by human readers. If right, the result suggests that structured academic writing does not need external orchestration once the policy itself has internalized the workflow.","feed_headline":"One trained pass beats multi-step writing agents on paper intros","feed_subtitle":"Stage-level rewards internalize outlining and revision; overhead drops 74% and 32B matches GPT-5.1.","key_machinery":"The central mechanism is the Struct-aware Relative Advantage (SRA) estimator. Instead of assigning one scalar reward to the entire generated introduction (as in GRPO/PPO), SRA computes a normalized advantage per stage: local advantages compare the same stage across sampled trajectories, a global advantage compares whole introductions, and the two are fused with weight λ. A refinement-gap term subtracts a penalty from draft-stage tokens when the revision version's stage reward is much higher, which pushes the first-pass policy to behave like its own reviser. These stage-level advantages are projected back to the tokens inside the corresponding stage boundaries, so the gradient can reward one","core_discovery":"StructPO treats an introduction as eight structural units—outline and content for each of four rhetorical sections (background, problem, method, contributions)—generated between explicit stage tokens in a single pass. Training uses draft–revision trajectories: the model first drafts, then revises. Rewards are computed per stage: outlines are scored by bidirectional semantic matching against reference outlines plus a length penalty; content is scored by embedding similarity, a trained classifier's estimate of whether each sentence belongs to its assigned rhetorical section, and Gaussian length regularization. Struct-aware Relative Advantage (SRA) then normalizes these stage rewards within a g","pith_inferences":["If the fixed four-section template is relaxed, the same SRA machinery could in principle learn section count and ordering as part of the policy, replacing the hand-set eight-stage grammar.","The refinement-penalty trick suggests a general recipe for 'amortized revision' in any long-form generation task: train the first pass by penalizing exactly the drafts that a later pass would fix, without running the reviser at inference.","Because the structure reward relies on a classifier, a natural stress test is to measure reward hacking over longer training: whether structure scores keep rising while human judges see repetitive or template-following text.","For neighboring tasks like related-work sections or survey generation, the same stage-token plus dense-advantage design could internalize their multi-stage pipelines, shrinking the cost of agentic writing tools generally."],"forward_implications":["A single-pass policy with stage tokens can replace handcrafted multi-stage prompts and agent workflows for introduction generation, avoiding cross-stage drift.","Distilling revision into the first pass cuts deployment cost: overhead tokens drop by about 74% compared with a workflow baseline while retaining fuller content than SFT stage-token models.","Stage-level credit assignment makes policy optimization stable for long-form writing: the ablation shows standard GRPO degrades factual consistency on the 7B backbone, while StructPO does not.","The learned rhetorical structure transfers out of domain: an ACL-trained checkpoint applied to CVPR papers keeps structural control and beats GPT-5.1 on structure and section alignment.","Scaling the trained policy to 32B parameters brings it to parity with strong closed-source models in blind human evaluation (53.3% win rate), suggesting policy structure and backbone scale are complementary."],"supporting_citations":[{"why":"GPT-4 is used with a fixed prompt to decompose each introduction into the four rhetorical sections and outlines that serve as training supervision and evaluation ground truth.","marker":"(Achiam et al. 2023)"},{"why":"GRPO is the base group-relative policy-optimization method that StructPO extends; it also supplies the trajectory-level baseline that motivates stage-level advantages.","marker":"(Shao et al. 2024)"},{"why":"PPO provides the clipped surrogate objective and clipping parameters reused in the StructPO loss.","marker":"(Schulman et al. 2017)"},{"why":"DeBERTa-v3 is trained into the structural-rationality classifier whose per-sentence section labels produce the r_struct reward.","marker":"(He, Gao, and Chen 2021; He et al. 2021)"},{"why":"Qwen3 Embedding provides the dense encoder used to compute semantic similarity rewards for content stages.","marker":"(Zhang et al. 2025b)"},{"why":"AutoSurvey is the main workflow-based baseline; its external orchestration is the approach StructPO internalizes and beats.","marker":"(Wang et al. 2024b)"},{"why":"STIG supplies the stage-token supervised baseline that shares StructPO's output format but lacks RL, isolating the effect of policy optimization.","marker":"(Zhang, su, and He 2025)"},{"why":"SurveyForge is the second workflow-based baseline for the comparison of external multi-stage generation.","marker":"(Yan et al. 2025)"},{"why":"Provides the LLM-as-a-judge methodology whose rubric metrics are used to evaluate writing quality, factual consistency, and soundness.","marker":"(Zheng et al. 2023)"}],"fun_headline_variants":["Single pass beats multi-step agents for paper intros","Internalized writing: one pass, 74% less overhead, matches GPT-5.1","StructPO: single-pass paper intros with stage-level rewards","One trained pass: intro generation with 74% lower cost","No more agent drift: single-pass intros rival GPT-5.1"],"cache_read_input_tokens":2816,"weakest_assumption_plain":"The gains rest on the fixed GPT-4o segmentation of each introduction into the four rhetorical sections being a correct, sufficiently rich ground truth; if that decomposition is biased, the reward signal and the automated metrics are biased in the same direction with no external check.","fun_headline_variants_meta":{"raw":{"variants":["Single pass beats multi-step agents for paper intros","Internalized writing: one pass, 74% less overhead, matches GPT-5.1","StructPO: single-pass paper intros with stage-level rewards","One trained pass: intro generation with 74% lower cost","No more agent drift: single-pass intros rival GPT-5.1"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000219,"raw_usage":{"total_tokens":1248,"prompt_tokens":680,"completion_tokens":568,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":424,"completion_tokens_details":{"reasoning_tokens":474}},"tokens_in":424,"tokens_out":568,"duration_ms":8087,"temperature":1.0,"reasoning_tokens":474,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T00:51:56.445292+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a set of introductions with non-canonical rhetoric (theory papers or surveys that skip an explicit contributions section), have human experts judge quality, and compare human rankings to StructPO's structure-score rankings. If introductions the paper's classifier calls structurally correct are systematically rated worse by experts—or if a workflow baseline wins human preference while losing on the automated metrics—the fixed decomposition, not the writing capability, would be carrying the result.","supporting_citations":[],"review_version":1}