{"id":"d5135aab-5da5-4b16-a7f4-7acda2b287b2","arxiv_id":"1908.11571","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A hierarchical pointer-network decoder that conditions on parent and sibling states improves discourse parsing relation F1 to 82.77 and gives marginal dependency parsing gains.","lead":"This paper adds hierarchical connections to pointer-network parsers, letting each decoding step see its parent and sibling states. It reports small gains on dependency parsing benchmarks and larger gains on discourse parsing, approaching human agreement on relation labeling.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Dependency-parsing SOTA claim is contradicted by the paper's own Table 2: best proposed PTB system (96.09 UAS) is below the published StackPtr baseline (96.12 UAS).","rationale":"The reader's rationale explicitly flags that the PTB table contradicts the SOTA claim, but the reader's stated weakest assumption is about whether the parent/sibling states supply informative context and whether gating controls error accumulation. My concern is narrower and more decisive: the paper's own published-baseline row refutes the dependency-parsing portion of the central claim, irrespective of whether the hierarchical context is informative. I therefore keep the reader's CONDITIONAL verdict: the discourse results may justify a revised claim, and the dependency claims could be restored with significance testing and a fair comparison to the published StackPtr numbers, but the current abstract and conclusion overstate the evidence. No ad hominem is intended; this is an internal consistency issue in the reported tables.","tokens_in":12076,"tokens_out":5990,"duration_ms":55281,"concrete_test":"Rerun the released StackPtr code on PTB with the published hyperparameters and evaluation configuration; if the rerun reproduces 96.12/95.06, or any value above 96.09/95.03, the dependency-parsing comparison collapses. In the same run, compute paired bootstrap 95% confidence intervals over the PTB test sentences for the difference between H-PtrNet-PS (Gate) and the rerun baseline for UAS and LAS; if the intervals include zero, the claimed effectiveness gain over the baseline is not established.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim in the abstract and Section 5 is that the parsers outperform existing methods and set a new state-of-the-art on both tasks. Table 2 does not support this for dependency parsing. On PTB, the published StackPtr result is 96.12 +/- 0.03 UAS and 95.06 +/- 0.05 LAS, while the best proposed variant, H-PtrNet-PS (Gate), reaches only 96.09 +/- 0.05 and 95.03 +/- 0.03, and H-PtrNet-PST (Gate) reaches 96.03 and 94.99. Every proposed variant is below the published baseline. The apparent improvement comes from comparing against StackPtr (code), a rerun that scores 95.94 and 94.91, lower than the published system. For PTB, no explanation is offered for this gap; the note in Section 4.1 about using UD Treebanks 2.3 applies only to the UD experiments. Moreover, Section 4.1 states that H-PtrNet-PS performs worse than StackPtr on all seven UD treebanks, and the PTB improvements over the rerun baseline are under 0.15 points, within the reported standard deviations and with no significance testing. The discourse-parsing results are stronger, but the paper's two-task SOTA statement is not internally consistent.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes hierarchical pointer network parsers (H-PtrNet) for dependency parsing and sentence-level RST discourse parsing. The decoder conditions each decoding state on the parent decoder state, the immediate sibling decoder state, the previous decoder state, and the parent encoder state, with gated fusion variants. The authors evaluate on English Penn Treebank and seven UD treebanks for dependency parsing, and on RST-DT for discourse parsing, reporting improvements over rerun baselines and claiming new state-of-the-art results on both tasks. The paper also includes ablations over parent/sibling/temporal connections and a sentence-length breakdown analysis.","tokens_in":12353,"tokens_out":4119,"duration_ms":39304,"significance":"If the claims were fully supported, the paper would be a useful contribution: the hierarchical decoder is a plausible inductive bias for tree-structured outputs, the code is released, and the discourse parsing results are strong, with the relation F1 of 82.77 approaching the reported human agreement of 83.0. The paper also does a fair amount of controlled comparison by rerunning the StackPtr and Pointer Net baselines under identical settings. However, the headline claim of state-of-the-art dependency parsing is not supported by the paper's own Table 2, where every proposed variant is below the published StackPtr result on PTB. The UD improvements are mostly within one standard deviation and lack significance testing, and the un-gated model degrades on long sentences. These issues are fixable by recalibrating the claims and adding appropriate statistical analysis, but they are central to the current presentation.","major_comments":[{"comment":"The abstract and Section 5 claim that the parsers 'outperform existing methods and set a new state-of-the-art' on both tasks, but Table 2 contradicts this for dependency parsing. The best proposed model, H-PtrNet-PS (Gate), reaches 96.09 UAS and 95.03 LAS on English PTB, below the published StackPtr result of 96.12 UAS and 95.06 LAS reported in the same table. The gains described in §4.1 are relative to the rerun StackPtr (code) baseline (95.94/94.91), not to the published system, and the paper does not explain why the code rerun is lower. The two-task state-of-the-art claim is therefore not supported by the dependency results.","section":"Abstract; §4.1; §5"},{"comment":"The UD dependency results do not support the claimed 'consistent improvements' without significance testing. In Table 1, H-PtrNet-PST (Gate) is worse than StackPtr (code) on ca (93.78 vs 93.82 UAS) and it (93.73 vs 93.76), and H-PtrNet-PST (SGate) is worse on en (90.94 vs 90.97) and ro (91.09 vs 91.15). The positive differences are mostly within 0.2 points, which is comparable to the reported standard deviations (0.05–0.26), and no paired significance test is provided. The text also states that H-PtrNet-PS is worse than StackPtr on all seven languages, so the claim of a consistent gain should be qualified substantially.","section":"§4.1, Table 1"},{"comment":"The paper's own analysis in Figure 4(a) shows that the un-gated H-PtrNet-PST performs better on short sentences but its accuracy drops on longer sentences, which the text attributes to error accumulation. This directly undercuts the general conclusion in §5 that the parsers 'perform better for long sequences.' The gated version improves some long-sentence cases, but the improvement is not shown to be statistically significant and is demonstrated only on selected languages (German, French, Italian). The limitation should be acknowledged and the conclusion should be scoped to the gated variants with supporting significance tests.","section":"§3.3, §4.1 Performance Analysis, §5"}],"minor_comments":[{"comment":"The sentence 'Our model H-PtrNet-PST (Gate) outperforms the baseline by 0.09 and 0.08 in terms of UAS and LAS' should explicitly state that the baseline is StackPtr (code), not StackPtr (paper), because the latter comparison is negative for all proposed variants.","section":"§4.1, Table 2"},{"comment":"The sentence claiming 'consistent improvements against the baseline on bg, en, de, fr, it and ro' is inaccurate: H-PtrNet-PST (Gate) is below the baseline on ca and it, and H-PtrNet-PST (SGate) is below on en and ro. The wording should be corrected or the table should be described more precisely.","section":"§4.1"},{"comment":"There are small typographical errors: 'as the the hidden state' in §3.4 should be 'as the hidden state', and 'sibing' in §4.2 should be 'sibling'.","section":"§3.4 and §4.2"},{"comment":"The 'partial tree information' provided to the decoder is mentioned but not formally specified; for reproducibility, the exact parent and sibling representations fed into the fusion function should be described or referenced precisely.","section":"§3.4 and §3.5"}],"recommendation":"major_revision","confidential_remarks":"The dependency-parsing state-of-the-art claim is not supported by the paper's own numbers, and the UD gains are not properly significance-tested. The discourse-parsing results are more promising and the code release is a plus. I recommend major revision rather than rejection because the central claims can be corrected by fairer baseline comparisons, significance testing, and more careful wording of the conclusions. The authors should also explain the discrepancy between StackPtr (paper) and StackPtr (code) or avoid claiming superiority over the published system."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea here is clean and worth taking seriously: give a pointer-network parser's decoder direct access to the parent and sibling decoder states, instead of only the sequential previous state. That is a simple, motivated extension over StackPointer and the Lin et al. discourse parser, and the authors apply it consistently across two tasks. The discourse parsing results are the real highlight: relation F1 of 82.77, within a hair of the 83.0 human agreement, with a ~1.5 point gain over the rerun baseline. That is a meaningful empirical step, and the analysis showing SGate recovers performance on long sentences supports the architectural motivation.\n\nThe dependency parsing story is where the paper trips. The abstract and conclusion claim a new state-of-the-art for both tasks, but Table 2 shows the best proposed PTB model (H-PtrNet-PS, 96.09 UAS) below the published StackPtr number (96.12). The paper instead compares against StackPtr (code), a rerun at 95.94, and reports a 0.15 point gain. That is a fair comparison for identical settings, but it does not support the SOTA claim. On UD, most gains are under 0.2 points, within one standard deviation, with no significance testing, and the paper itself notes H-PtrNet-PS is worse than StackPtr on all seven treebanks. So the dependency contribution is unclear; the discourse contribution is solid.\n\nThe writing is honest about some limitations—the gating analysis is good—but the overreach in the abstract and conclusion should be fixed. I would also want paired significance tests or at least a more careful statement about effect size. The code for the dependency parser is released, which is good, though discourse code is not.\n\nSerious thinking is evident: the idea, the experiments, and the analysis hang together. The paper deserves a proper referee, but not in its current form. The authors need to either drop the SOTA claim for dependency parsing, present it as an improvement over a rerun baseline, or explain why the published comparison is not applicable. If the discourse results stand up, this could be a useful contribution for parsing researchers, especially those working on RST discourse parsing.","headline":"Plausible hierarchical decoder idea with genuinely promising discourse results, but the dependency-parsing SOTA claim is contradicted by the paper's own Table 2 and needs reframing.","tokens_in":12865,"tokens_out":2215,"would_cite":false,"duration_ms":22151,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Parent-sibling decoder links raise parsing accuracy","keywords":["hierarchical pointer networks","dependency parsing","discourse parsing","Rhetorical Structure Theory","transition-based parsing","decoder inductive bias","top-down parsing"],"falsifier":"A decisive test would be an ablation that feeds the decoder the parent and sibling encoder states but not their decoder states; if the reported gains on UD and RST-DT disappear, the hierarchical decoder-state connections are the source, whereas if the gains remain, the improvement comes from extra input information rather than the hierarchical inductive bias.","tokens_in":11893,"feed_emoji":"🌳","tokens_out":6718,"duration_ms":58889,"temperature":0.7,"pith_summary":"Parsers for dependency and discourse trees are often built with pointer networks whose decoder is a flat sequence, even though the output is a tree. This paper proposes hierarchical pointer networks (H-PtrNet), in which each decoder state is conditioned not only on the previous decoding step but also on the decoder states of the current node's parent and immediate sibling. The paper argues that this gives the decoder the right inductive bias for constructing hierarchical structures, and reports that the resulting models outperform the existing pointer-network baselines on dependency parsing across most of seven Universal Dependencies languages and on English Penn Treebank, and set new state-of-the-art numbers on sentence-level RST discourse parsing. In discourse parsing, relation labeling reaches 82.77 F1, close to the 83.0 human-agreement figure.","feed_headline":"Parent-sibling decoder links raise parsing accuracy","feed_subtitle":"Dependency and RST discourse parsers gain accuracy when each decoding step sees its tree parent and sibling.","key_machinery":"Pointer networks are encoder-decoder models that produce outputs by attending over input positions, effectively pointing to elements of the input. The central mechanism introduced here is the hierarchical decoder connection defined by Eq. (2), $d_t = f(d_{p(t)}, d_{s(t)}, d_{t-1}, h_{p(t)})$, where $d_{p(t)}$ and $d_{s(t)}$ are the decoder states of the current node's parent and immediate sibling, $d_{t-1}$ is the previous decoder state, and $h_{p(t)}$ is the parent's encoder state. In dependency parsing, $f$ is implemented as an LSTM with a learned gate (Eqs. 5-9) that controls information flow from the parent, sibling, and previous decoder states; in discourse parsing, a GRU variant is used without the gating layer because gating did not help there. The resulting decoder state feeds the pointer layer, which for dependency parsing points from a head word to a dependent and for discourse parsing points from a span to a split position. The gating mechanism is load-bearing because it suppresses error accumulation from earlier parsing mistakes on long sentences.","core_discovery":"The central claim is that a sequential decoder is the wrong inductive bias for tree construction. When a pointer-network parser decodes a word such as a dependent noun, the most relevant context is not the token decoded immediately before it but the head word that dominates it and the sibling that shares that head. The paper implements this by replacing the decoder recurrence with a hierarchical fusion, Eq. (2), that combines the parent decoder state, the immediate sibling decoder state, the previous decoder state, and the parent encoder state. In dependency parsing the full H-PtrNet-PST model with gating improves over the StackPointer baseline on six of seven UD languages and on English Penn Treebank, while an ablation without gating degrades on long sentences, showing that the gate is what prevents error accumulation from earlier parsing mistakes. In sentence-level RST discourse parsing, the parent-only variant improves span F1 beyond the human-agreement baseline, and the full variant improves relation labeling to near human level. The paper concludes that hierarchical decoder connections are more appropriate than purely sequential ones for deriving tree structures.","pith_inferences":["The same parent-sibling conditioning could apply to other tree-generating decoders, such as constituency parsers or semantic graph readers, wherever node hierarchy defines the contextual dependency.","Because the dependency-parsing comparison relies on a rerun of the baseline under identical settings, some of the gain could stem from training conditions rather than the hierarchical inductive bias; an ablation feeding the same extra encoder states without the parent and sibling decoder states would separate the two.","The gating result suggests a depth-adaptive gate: in shallow trees gating may be unnecessary, while in deep trees it is crucial, so a gate whose strength grows with subtree depth would be a direct testable extension."],"forward_implications":["The hierarchical decoder preserves the $O(n^2)$ decoding complexity of the baseline, so the accuracy improvements come without an asymptotic speed penalty.","Parent connections alone carry most of the benefit in discourse parsing; sibling and temporal connections add little for span identification but improve relation labeling.","The gating mechanism is essential on long sentences: without it, conditioning on parent and sibling states amplifies error accumulation and accuracy drops.","For sentence-level RST discourse parsing, relation labeling reaches 82.77 F1, within 0.23 of the 83.0 human-agreement figure, when the model is selected on relation performance."],"supporting_citations":[{"why":"Introduces pointer networks, the encoder-decoder architecture whose decoder this paper makes hierarchical.","marker":"Vinyals et al. (2015)"},{"why":"Provides the StackPointer dependency parser baseline and the top-down pointer decoding setup that H-PtrNet extends.","marker":"Ma et al. (2018)"},{"why":"Provides the pointer-network discourse parser baseline and the RST sentence-level parsing framework and settings.","marker":"Lin et al. (2019)"},{"why":"Supplies the RST Discourse Treebank used as the discourse parsing benchmark.","marker":"Carlson et al. (2002)"},{"why":"Defines the Span, Nuclearity, and Relation evaluation metrics used for RST parsing results.","marker":"Marcu (2000)"}],"fun_headline_variants":["Parent-sibling decoder beats sequential baseline","Tree-structured decoder improves dependency parsing","Hierarchical decoder links lift parsing accuracy","Why tree decoders beat chains for parsing","Decoder that sees its family tree wins on parsing"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the parent and sibling decoder states carry structural context that helps the next decoding decision, and that the gating mechanism can keep that context from being corrupted by earlier parsing errors.","fun_headline_variants_meta":{"raw":{"variants":["Parent-sibling decoder beats sequential baseline","Tree-structured decoder improves dependency parsing","Hierarchical decoder links lift parsing accuracy","Why tree decoders beat chains for parsing","Decoder that sees its family tree wins on parsing"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000183,"raw_usage":{"total_tokens":1252,"prompt_tokens":820,"completion_tokens":432,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":436,"completion_tokens_details":{"reasoning_tokens":369}},"tokens_in":436,"tokens_out":432,"duration_ms":4897,"temperature":1.0,"reasoning_tokens":369,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T10:10:57.932187+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A decisive test would be an ablation that feeds the decoder the parent and sibling encoder states but not their decoder states; if the reported gains on UD and RST-DT disappear, the hierarchical decoder-state connections are the source, whereas if the gains remain, the improvement comes from extra input information rather than the hierarchical inductive bias.","supporting_citations":[{"cited_title":"A Unified Linear-Time Framework for Sentence-Level Discourse Parsing","cited_arxiv_id":"1905.05682","evidence_quote":"Provides the pointer-network discourse parser baseline and the RST sentence-level parsing framework and settings."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the RST Discourse Treebank used as the discourse parsing benchmark."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the Span, Nuclearity, and Relation evaluation metrics used for RST parsing results."}],"review_version":1}