{"id":"766586e7-88d6-4b69-b584-c95117c98a75","arxiv_id":"2607.06175","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":10,"one_line_summary":"Equal reward weighting outperforms targeted weighting in RL-based BPMN generation across 48 configurations, with design choices producing effects as large as applying RL itself.","lead":"This paper systematically tests how to design reward functions for reinforcement learning when generating BPMN process models with LLMs, finding that equal weighting of quality dimensions beats targeted weighting. The findings offer practical guidance for anyone using RL to optimize multi-dimensional structured outputs.","discovery_kind":"unclear","skeptic_critique":{"model":"glm-5.2","headline":"The R_avg vs R1 natural experiment reveals substantial uncontrolled training variance (Δ=0.059, p<0.00002 for Llama syntactic between mathematically equivalent configurations), undermining confidence in single-run pairwise comparisons for moderate effect sizes.","rationale":"The reader correctly identified a variance-related concern but focused on generation-time variance (single output per test sample, Section 4.4) rather than training-time variance, which is the more pressing issue. The R_avg vs R1 comparison is the paper's own natural experiment that directly reveals the magnitude of uncontrolled training variance: two mathematically equivalent configurations producing a statistically significant 0.059-point difference for Llama. This is more load-bearing than the single-generation concern because (a) it is directly evidenced in the paper's own data rather than being a hypothetical risk, and (b) it affects the reliability of all pairwise comparisons, not just the precision of point estimates. The reader did flag the R_avg/R1 discrepancy as concern (2) in their rationale, so they were aware of it, but they treated it as a separate issue about 'training stability' rather than recognizing it as a direct estimate of the noise floor that should calibrate interpretation of all other effects. The reader's other concerns — the 3:1:1 ratio being potentially too aggressive, and the validity conditioning — are valid but secondary. The 3:1:1 ratio concern is acknowledged by the authors (Section 3.5, Section 6.5) and doesn't undermine the core finding that naive weighting can fail badly. The validity conditioning concern is real but the paper handles it transparently by reporting validity counts alongside all quality scores. The CONDITIONAL verdict is appropriate: the large effects (policy collapse under R3, pragmatic improvement under RL, the penalty's role for Qwen) are likely robust to training noise, but the broader generalization — that equal weighting 'consistently' outperforms and that reward composition effects are 'as large as RL itself' — requires the training variance check to be fully established. The paper's released code (GitHub link in abstract) makes the proposed re-run with multiple seeds straightforward to execute.","tokens_in":31983,"tokens_out":3780,"duration_ms":274223,"concrete_test":"Re-run R1, R_avg, and R3 for Llama SFT with at least 3 independent random seeds each (varying only the seed, keeping all hyperparameters identical). Compute between-seed standard deviation for each configuration on each quality dimension. If the between-seed SD for R1 approaches or exceeds 0.03 (half the observed R_avg–R1 gap of 0.059), then effects smaller than ~0.06 cannot be reliably distinguished from training noise, weakening the 'consistently outperforms' claim for moderate effects. If R1's between-seed SD is below 0.01 and the R1–R3 difference remains stable across all seeds, the central claim is robust.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim that equal reward weighting consistently outperforms targeted weighting rests on pairwise comparisons between configurations trained once each. The paper includes R_avg and R1 as a control: they are mathematically equivalent (both compute the unweighted mean of three quality scores with p=-1) but implemented through separate pipelines. For Llama, these equivalent configurations produce significantly different outcomes: Δ_syntactic = +0.059 (p_adj < 2×10^-5, favoring R1), with R_avg scoring 0.869 vs R1's 0.926 (Table 5). This difference cannot be attributed to reward design and must reflect training run variance — stochasticity from random seeds, sampling during GSPO candidate generation (K=4 at temperature 0.7), and optimization noise. This natural experiment provides a lower bound on uncontrolled training variance. The largest weighting effects (e.g., R1 vs R3 for Llama syntactic: Δ=0.264) are ~4.5x larger than this variance estimate and are likely robust. However, several moderate effects that contribute to the 'consistently outperforms' claim are closer to the noise floor: R1 vs R5 for Llama semantic (Δ=0.027, Table 6), R1 vs R4 for Qwen pragmatic (Δ=0.023), R1 vs R5 for Qwen semantic (Δ=0.019). These are within 0.3–0.5x of the R_avg–R1 gap, making it unclear whether they reflect genuine reward design effects or training instability. The claim that 'reward composition effects are as large as the decision to apply RL itself' (Section 5.2, abstract) is particularly sensitive: the RL-vs-SFT comparison for Llama syntactic yields Δ=0.092, while the R_avg–R1 noise floor is 0.059 — these are within the same order of magnitude, suggesting that some portion of attributed reward design effects may be training noise rather than systematic composition effects. The paper acknowledges the R_avg/R1 difference in Section 6.3 but interprets it 'cautiously' without recognizing its implications for the reliability of all other single-run pairwise tests.","agreement_with_reader":"partial"},"referee_report":{"model":"glm-5.2","summary":"This paper investigates reward function design for reinforcement learning (RL) applied to LLM-based BPMN process model generation. Using Group Sequence Policy Optimization (GSPO) with rewards derived from the BEF4LLM quality framework (38 metrics across syntactic, pragmatic, and semantic dimensions), the authors train Llama 3.1 8B and Qwen 2.5 14B under 48 configurations varying reward weighting, invalidity penalty, base model initialization, and grammar-constrained decoding. Three main findings emerge: (1) GSPO significantly improves pragmatic and syntactic quality while preserving semantic fidelity, with dramatic variability reduction; (2) equal reward weighting consistently outperforms targeted (3:1:1) weighting, which can cause policy collapse; (3) design choices interact with model architecture in non-trivial ways. The experimental design is systematic, with paired permutation tests and Bonferroni correction. The paper releases code and experimental artifacts.","tokens_in":32335,"tokens_out":1822,"duration_ms":336976,"significance":"The paper addresses a genuine gap: systematic investigation of multi-dimensional reward function design for structured generation, where most prior RLVR work uses single-dimensional rewards. The finding that equal weighting outperforms targeted weighting is counter-intuitive and practically useful. The inclusion of mathematically equivalent configurations (R_avg vs R1) as a natural experiment on training variance is a methodological strength, as is the release of reproducible code. The cross-architecture analysis (two model families, two initialization strategies) provides useful generality evidence. The work is relevant beyond BPMN to any structured generation task with multi-dimensional quality assessment.","major_comments":[{"comment":"§4.4 and Table 5: The R_avg vs R1 natural experiment reveals substantial uncontrolled training variance. These configurations are mathematically equivalent (both compute the unweighted mean of three quality scores with p=-1), yet for Llama they produce significantly different syntactic outcomes (Δ=0.059, p_adj<2×10^-5, favoring R1: 0.926 vs 0.869). The authors acknowledge this in §6.3 but interpret it cautiously. However, several moderate effects that support the central 'equal weighting consistently outperforms' claim are within 0.3-0.5× of this variance estimate: R1 vs R5 for Llama semantic (Δ=0.027, Table 6), R1 vs R4 for Qwen pragmatic (Δ=0.023), R1 vs R5 for Qwen semantic (Δ=0.019). The claim of 'consistently outperforms' and the abstract's statement that 'reward composition effects are as large as the decision to apply RL itself' rest on single-run pairwise comparisons where some效应","section":null},{"comment":"§4.4: The single generation per evaluation sample (no repeated runs) is a load-bearing limitation. The paper states this 'limits the ability to estimate within-configuration variance' but the statistical significance claims via paired permutation tests assume the single observed score per sample is a reliable point estimate. Given that the R_avg-R1 gap demonstrates training-run variance of ~0.06, the authors should either (a) explicitly bound which effect sizes are robust given this variance floor, or (b) temper the 'consistently outperforms' language for effects near the noise floor. The largest effects (e.g., R1 vs R3 for Llama syntactic: Δ=0.264) are clearly robust, but the blanket claim needs qualification.","section":null},{"comment":"§3.3 and §3.4: The reward functions are linear combinations of BEF4LLM metrics, and evaluation uses the same BEF4LLM framework. The authors note this is not a logical circularity (the reward is a weighted composite; evaluation reports per-dimension scores). This is correct, but the concern is deeper: the reward signal and evaluation metric share the same underlying 38 metrics, meaning RL optimization directly improves the measured quantities. The paper does not discuss whether improvements on BEF4LLM metrics correspond to genuine process model quality improvements as perceived by human modelers, or whether RL could exploit metric artifacts. The Qwen R2 template collapse (§5.2.2, syntactic std=0.002) is actually evidence of such metric exploitation. The authors should discuss this risk more explicitly and note that human evaluation would strengthen the claims.","section":null},{"comment":"Table 5: Several configurations have substantially different validity counts (e.g., Llama SFT-only n=36 vs R1 n=102; Llama R5 n=38 vs R1 n=102). The paired permutation tests are conditioned on joint validity (§4.5), meaning comparisons between configurations with very different validity rates are conducted on restricted, potentially biased subsets. The paper notes this but does not report how many samples are in the joint validity set for each comparison. For the R1 vs R5 comparison (Llama), R5 has only 38 valid outputs — the joint set could be as small as ~35, substantially reducing statistical power. The authors should report joint validity counts for each pairwise comparison in Table 6.","section":null}],"minor_comments":[{"comment":"§3.4, Table 2: The distinction between R_avg ('aggregated scalar') and R1 ('equal, decomposed') is unclear. The text states they are 'mathematically equivalent' but 'correspond to separate implementation pipelines,' but the implementation difference is not described. What specifically differs in the reward computation pipeline?","section":null},{"comment":"§5.2.1: The term 'policy collapse' is introduced informally. While the phenomenon is well-described, a brief formal characterization (e.g., convergence to a low-entropy output distribution) would strengthen the analysis.","section":null},{"comment":"Figure 1: The violin plots for SFT-only Llama show n=36 for syntactic and pragmatic but n=35 for semantic. This discrepancy is unexplained. Is this due to additional translation failures for semantic evaluation?","section":null},{"comment":"§6.4: The grammar-constrained decoding analysis is described as 'supplementary' and summarized briefly. Given that it is one of the four experimental factors in the 48-configuration matrix, the results should be reported more fully, at least in an appendix or supplementary material.","section":null},{"comment":"§4.1: The training set (1,552 samples) is described as a mix of German and English, assembled from 'multiple publicly available BPMN datasets, supplemented with synthetically generated samples.' The proportion of synthetic data and its potential impact on generalization are not discussed.","section":null},{"comment":"Table 6: The R_avg vs R1 comparison is not included in the consolidated pairwise tests table, despite being discussed in §6.3. It should be added for completeness.","section":null},{"comment":"§3.5: The choice of 3:1:1 weighting ratio is justified as balancing 'experimental discriminability with training stability,' but no sensitivity analysis is provided. The authors acknowledge this in §6.5 but it weakens the claim that targeted weighting 'consistently fails' — gentler ratios might succeed.","section":null},{"comment":"Abstract: 'reducing output variability by more than sixfold' — this refers specifically to pragmatic quality standard deviation for Llama (0.085 to 0.013). The abstract should clarify this is dimension-specific rather than a general variability reduction.","section":null}],"recommendation":"major_revision","confidential_remarks":"The paper is a solid empirical contribution with a well-designed experimental matrix. The main concern is the tension between the strength of the central claim ('equal weighting consistently outperforms,' 'effects as large as RL itself') and the evidence base: single-run training with demonstrated training-run variance of ~0.06 for equivalent configurations. The authors are transparent about limitations (§6.5), which is commendable, but the claims in the abstract and conclusion are not sufficiently qualified relative to these limitations. I believe the paper can be strengthened to publication quality with: (1) explicit effect-size robustness analysis using the R_avg-R1 gap as a variance floor, (2) reporting joint validity counts for all pairwise comparisons, (3) tempering the 'consistently outperforms' language for moderate effects, and (4) fuller reporting of grammar-constrained decoding results. The code release is a positive signal for reproducibility. The paper fits the journal's scope well."},"author_rebuttal":{"model":"glm-5.2","summary":"We thank the referee for a careful and substantive review. The comments identify genuine limitations in our statistical claims and evaluation methodology. We agree with the core of each comment and will revise accordingly. Below we address each point in turn.","responses":[{"response":"The referee is correct that the R_avg–R1 gap (Δ=0.059 for Llama syntactic) provides a lower bound on training-run variance, and that several effects supporting the 'consistently outperforms' claim are of comparable or smaller magnitude. We have re-examined each pairwise comparison in Table 6 against this variance floor. The largest effects are clearly robust: R1 vs R3 for Llama syntactic (Δ=0.264), R1 vs R4 for Llama pragmatic (Δ=0.120), R1 vs R5 for Llama syntactic (Δ=0.128), and the R1-vs-SFT-only comparisons (Δ≥0.092 for syntactic, Δ≥0.116 for pragmatic). These exceed the variance estimate by a factor of 2–4× and are unlikely to be artifacts. However, the referee correctly identifies effects near the noise floor: R1 vs R5 for Llama semantic (Δ=0.027), R1 vs R4 for Qwen pragmatic (Δ=0.023), and R1 vs R5 for Qwen semantic (Δ=0.019) are all within 0.3–0.5× of the R_avg–R1 gap. We will revise the manuscript to explicitly classify effects as robust (Δ ≥ 2× the R_avg–R1 variance estimate) versus tentative (Δ < 2×), and to qualify the 'consistently outperforms' language accordingly. The abstract's claim that 'reward composition effects are as large as the decision to apply RL itself' is supported by the robust effects (e.g., R3 collapse at Δ=0.264 vs R1-vs-SFT Δ=0.092), but we will add a qualifier noting that this holds for the largest effects, not all pairwise comparisons.","revision_made":"yes","referee_comment":"§4.4 and Table 5: The R_avg vs R1 natural experiment reveals substantial uncontrolled training variance. Several moderate effects supporting 'equal weighting consistently outperforms' are within 0.3-0.5× of this variance estimate. The claim of 'consistently outperforms' and the abstract's statement rest on single-run pairwise comparisons."},{"response":"We agree. The single-generation protocol means we cannot estimate within-configuration variance from repeated sampling, and the R_avg–R1 comparison is our only direct evidence of training-run variance. We will take approach (a): we will add a subsection to Section 6.3 that explicitly establishes the R_avg–R1 gap as a variance floor, classifies each Table 6 comparison as robust or tentative relative to this floor, and states which specific claims are robust versus which require confirmation through multi-seed replication. We will also temper the blanket 'consistently outperforms' language in the abstract, Section 5.2.1 summary, and conclusion to 'consistently outperforms for the largest effects; smaller effects require multi-seed confirmation.' We acknowledge that multi-seed training (at minimum 3 seeds per configuration) would be the proper remedy, and we will note this as the most important future work item. Given the 48-configuration experimental matrix, this was not computationally feasible for the current study, but we recognize it as essential for confirming the smaller effects.","revision_made":"yes","referee_comment":"§4.4: Single generation per evaluation sample is a load-bearing limitation. The authors should either (a) explicitly bound which effect sizes are robust given this variance floor, or (b) temper the 'consistently outperforms' language for effects near the noise floor."},{"response":"This is a fair and important concern. We agree that the shared metric basis between reward and evaluation constitutes a form of evaluation–reward coupling that is stronger than typical RLVR settings (where, e.g., training rewards and test metrics are the same pass/fail signal, but the concern is less salient because the metric is unambiguous). The Qwen R2 template collapse (syntactic std=0.002, pragmatic spike at 0.964) is indeed a concrete example of metric exploitation: the model discovered a degenerate structure that scores well on the metrics while being useless as a process model. We will add a dedicated discussion in Section 6.2 that (1) explicitly acknowledges the reward–evaluation coupling as a limitation, (2) uses the R2 collapse as a case study of metric exploitation, (3) notes that the equal-weight R1 configuration's preserved output diversity (visible in the violin plots) provides indirect evidence against exploitation for the main results, but is not a substitute for independent evaluation, and (4) states that human evaluation would substantially strengthen the claims and should be conducted in future work. We will also note that the BEF4LLM metrics are rule-based and inspectable rather than learned proxies, which limits (but does not eliminate) the risk of exploitation compared to learned reward models. We cannot fully resolve this concern without human evaluation, which is beyond the scope of the current revision.","revision_made":"partial","referee_comment":"§3.3 and §3.4: The reward signal and evaluation metric share the same underlying 38 metrics, meaning RL optimization directly improves the measured quantities. The paper does not discuss whether improvements correspond to genuine process model quality improvements as perceived by human modelers, or whether RL could exploit metric artifacts. The Qwen R2 template collapse is evidence of such exploitation."},{"response":"The referee is correct that joint validity counts are essential for interpreting the statistical tests, particularly for comparisons involving configurations with very different validity rates (e.g., R1 n=102 vs R5 n=38 for Llama SFT). We will add a column to Table 6 reporting the joint validity count (n_joint) for each pairwise comparison. We will also add a brief discussion of the potential selection bias introduced by conditioning on joint validity: when one configuration has much lower validity, the joint set is restricted to the 'easier' samples where both models succeed, which may attenuate or amplify observed differences. For the R1 vs R5 (Llama) comparison specifically, we expect n_joint to be approximately 35–38 (the intersection of 102 and 38 valid outputs), which substantially reduces statistical power and means the Δ=0.027 semantic difference should be interpreted very cautiously—consistent with our response to the first comment. We will flag comparisons where n_joint is substantially smaller than either configuration's individual validity count as having reduced reliability.","revision_made":"yes","referee_comment":"Table 5: Several configurations have substantially different validity counts. The paired permutation tests are conditioned on joint validity, meaning comparisons between configurations with very different validity rates are conducted on restricted, potentially biased subsets. The authors should report joint validity counts for each pairwise comparison in Table 6."}],"tokens_in":32082,"tokens_out":1703,"duration_ms":90545,"standing_objections":["The reward–evaluation coupling concern (Comment 3) cannot be fully resolved without human evaluation of generated process models, which is beyond the scope of this revision. We can acknowledge the limitation and provide indirect evidence (output diversity under R1, inspectability of rule-based metrics), but we cannot provide direct evidence that BEF4LLM improvements correspond to human-perceived quality improvements.","Multi-seed training to establish within-configuration variance (Comments 1 and 2) is the proper remedy for the training-variance concern, but is computationally infeasible within the revision timeframe given the 48-configuration matrix. We will bound effect sizes using the R_avg–R1 variance floor as an interim measure."]},"desk_editor":{"model":"glm-5.2","letter":"Here's what you need to know: this paper systematically tests how reward function composition (weighting, penalty, aggregation) affects RL-based BPMN process model generation across 48 configurations and two model families. The central finding — that equal reward weighting outperforms targeted weighting — is well-supported for the large effects but shakier for the moderate ones. Code is released at GitHub, which is good practice and lets readers verify the pipeline. The statistical methodology is solid: paired permutation tests with Bonferroni correction and 99,999 resamples, applied across three quality dimensions. The experimental design is genuinely systematic — six reward functions, two initialization strategies, two decoding conditions, two architectures. The finding that targeted weighting can cause policy collapse (Llama R3 dropping to 0.662 syntactic quality) is a real result with a clear mechanism: over-weighting one dimension reduces the information content of group-relative advantage signals. The penalty analysis (R1 vs R2) showing template collapse in Qwen without the penalty is also well-documented, with the near-zero syntactic standard deviation (0.002) making the case concretely. The architecture-dependent findings for SFT initialization (indispensable for Llama, counterproductive for Qwen's semantic quality) are worth reporting. Now the soft spots. The stress-test concern about the R_avg vs R1 gap is legitimate and the paper underplays it. These are mathematically equivalent configurations producing a 0.059 syntactic difference for Llama (p < 0.00002). The authors acknowledge this in Section 6.3 but interpret it 'cautiously' without recognizing that it sets a noise floor for all other single-run pairwise comparisons. The largest effects (R1 vs R3 for Llama: Δ=0.264) are ~4.5x above this floor and are robust. But several moderate effects contributing to the 'consistently outperforms' claim are within 0.3–0.5x of the gap: R1 vs R5 for Llama semantic (Δ=0.027), R1 vs R4 for Qwen pragmatic (Δ=0.023), R1 vs R5 for Qwen semantic (Δ=0.019). These could be training noise rather than reward design effects. The claim that 'reward composition effects are as large as the decision to apply RL itself' is particularly sensitive: the RL-vs-SFT comparison for Llama syntactic is Δ=0.092, while the noise floor is 0.059 — same order of magnitude. The single-generation-per-sample protocol (no repeated runs) compounds this: within-configuration variance is unestimable, so the permutation tests assume each single score is a reliable point estimate. The 3:1:1 weighting ratio is acknowledged as potentially too aggressive, so the failure of targeted weighting may not generalize to gentler ratios like 2:1:1. The circularity concern (reward and evaluation both use BEF4LLM) is real but not fatal — the paper is measuring whether RL can optimize for a given quality metric, not whether that metric is correct. The conditioning of quality scores on validity means comparisons between configurations with very different validity rates (e.g., SFT-only with 36 valid vs R1 with 102) are on restricted subsets, which the paper acknowledges but could discuss more carefully. This paper is for researchers and practitioners working on RL for structured generation tasks with multi-dimensional quality metrics. The practical takeaway — start with equal weighting, match penalty to model behavior, test whether SFT is needed — is useful and well-grounded. The paper deserves a serious referee. The core findings are sound for large effects; the referee should push the authors to (1) acknowledge the noise floor explicitly and temper claims about moderate effects, (2) soften the 'as large as RL itself' claim given the R_avg/R1 evidence, and (3) discuss whether single-run comparisons are adequate for the effect sizes they report.","headline":"Solid empirical study of multi-dimensional reward design for RL-based process model generation; main findings hold but some moderate effects are near the noise floor","tokens_in":32965,"tokens_out":896,"would_cite":false,"duration_ms":178529,"reading_group":"no","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"glm-5.2","headline":"Equal reward weights beat targeted weights in multi-dimensional RL","keywords":[],"falsifier":"If repeated generation runs revealed that the within-configuration variance is large relative to the between-configuration differences reported, particularly for the targeted-weighting failure and the penalty effect on Qwen, the central claims about equal weighting superiority and penalty importance would be weakened.","tokens_in":32054,"feed_emoji":"⚖️","tokens_out":1155,"duration_ms":195397,"temperature":0.7,"pith_summary":"This paper investigates how to design reward functions when quality is multi-dimensional, using the generation of BPMN business process models from natural language as a testbed. The authors train two LLM families (Llama 3.1 8B and Qwen 2.5 14B) under 48 configurations using Group Sequence Policy Optimization (GSPO), with rewards derived from 38 automated metrics spanning syntactic correctness, pragmatic comprehensibility, and semantic fidelity. The central finding is that equal weighting of quality dimensions consistently outperforms targeted weighting: emphasizing a specific dimension in the reward function not only fails to improve that dimension but can cause the model to collapse into a narrow, low-quality output mode. This effect is as large as the decision to apply reinforcement learning at all. The paper also finds that design choices interact strongly with model architecture: an invalidity penalty is essential for one model but irrelevant for the other, and supervised fine-tuning initialization is indispensable for one architecture but counterproductive for another. The authors argue that reward composition is a primary determinant of optimization outcomes, not a secondary tuning detail, and that these findings generalize to any structured generation task where quality is assessed along multiple automated dimensions.","feed_headline":"Equal reward weights beat targeted weights in multi-dimensional RL","feed_subtitle":"When quality has multiple dimensions, emphasizing one in the reward function backfires. The effect is as large as applying RL at all.","key_machinery":"The central object carrying the argument is the composite reward function R(w, p; y) = w_syn·r_syn + w_pra·r_pra + w_sem·r_sem for valid outputs, with penalty p for invalid outputs. The six reward configurations (R_avg through R_5) systematically vary the weight vector w and the penalty p, allowing the authors to isolate the effects of dimensional weighting, penalty structure, and implementation pathway. GSPO's group-relative normalization is the mechanism through which reward composition exerts its effects: by ranking candidates within each group rather than against an absolute baseline, the structure of the reward function directly shapes which quality distinctions are detectable and thus,","core_discovery":"The paper's core discovery is that in multi-dimensional reward optimization for structured generation, equal weighting of quality dimensions is not merely a safe default but actively superior to targeted weighting. Targeted weighting destabilizes the group-relative advantage signal that GSPO relies on: when one dimension dominates the reward, candidate rankings are driven by variation along a single axis, which is often insufficient to produce informative gradients, causing the policy to drift or collapse. The authors identify a recurring failure mode they call policy collapse, where the model converges to a narrow, low-diversity output template regardless of input. This collapse appears in三","pith_inferences":["The failure of targeted weighting may reflect a general property of group-relative RL methods: when the reward signal is dominated by a single dimension, the within-group ranking becomes uninformative if candidates show little variation on that dimension. This could extend to other group-relative methods beyond GSPO.","The observation that pragmatic quality is the most easily optimized dimension (because it rewards simpler outputs) suggests that reward accessibility is partly determined by the direction of the gradient in policy space: dimensions that can be improved by moving toward simpler, more compact outputs are more accessible than those requiring specific structural changes.","The template collapse observed under the no-penalty condition for Qwen resembles reward hacking, but with the distinction that the template achieves high reward under the defined objective. This suggests that multi-dimensional reward functions may need explicit diversity-preserving mechanisms beyond penalty terms, particularly when the model's validity behavior is variable.","The architecture-dependent effects (penalty essential for one model but not the other, SFT indispensable for one but counterproductive for the other) suggest that the interaction between reward design and model architecture may be governed by the model's prior disposition toward the target task, which could potentially be assessed before training through zero-shot evaluation."],"forward_implications":["Practitioners applying RL to any structured generation task with multiple quality metrics should default to equal reward weighting unless they have specific evidence that targeted weighting helps their model and task.","The invalidity penalty should be treated not only as a validity enforcement mechanism but as an implicit diversity regularizer whose effect depends on the base model's prior validity behavior.","The standard SFT-then-RL pipeline should not be assumed universally optimal; a pilot evaluation comparing SFT-initialized and untrained bases may reveal that skipping SFT yields better results for some architectures.","Reward function design choices should be co-tuned with model architecture and initialization strategy rather than transferred across model families.","The finding that reward composition effects can be as large as the decision to apply RL suggests that future work on RL for structured generation should report and ablate reward configurations as systematically as the authors do here."],"fun_headline_variants":["Equal reward weights beat targeted weights for RL process model generation","Targeting reward weights collapses LLM output diversity in process modeling","Reward composition matters as much as applying RL for structured generation","RL cuts process model output variability sixfold with equal reward weights","Best reward function for multi-dimensional quality: weight all dimensions equally"],"cache_read_input_tokens":0,"weakest_assumption_plain":"The evaluation uses a single generation per sample across 105 held-out instances with no repeated runs, meaning within-configuration variance cannot be estimated. This is load-bearing because the statistical significance claims rest on paired permutation tests that assume each single observed score is a reliable point estimate, yet the paper reports dramatic distributional changes (such as a syntactic standard deviation of 0.002 for one Qwen configuration) that could be arte","fun_headline_variants_meta":{"raw":{"variants":["Equal reward weights beat targeted weights for RL process model generation","Targeting reward weights collapses LLM output diversity in process modeling","Reward composition matters as much as applying RL for structured generation","RL cuts process model output variability sixfold with equal reward weights","Best reward function for multi-dimensional quality: weight all dimensions equally"]},"model":"glm-5.2","effort":"high","cost_usd":0.0,"raw_usage":{"total_tokens":723,"prompt_tokens":640,"completion_tokens":83,"prompt_tokens_details":null},"tokens_in":640,"tokens_out":83,"duration_ms":25949,"temperature":1.0,"reasoning_tokens":null,"cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-08T14:15:11.193242+00:00","model_set":{"reader":"glm-5.2"},"falsifier":"If repeated generation runs revealed that the within-configuration variance is large relative to the between-configuration differences reported, particularly for the targeted-weighting failure and the penalty effect on Qwen, the central claims about equal weighting superiority and penalty importance would be weakened.","supporting_citations":[],"review_version":1}