{"id":"bb4be35b-0d19-423b-a393-fae358cd95cc","arxiv_id":"2607.05689","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":3.5,"correctness_risk":"low","formal_verification":"none","parameter_count":7,"one_line_summary":"Boundary-aware multi-label span classification with hard negatives and containment NMS, plus independent RoBERTa document classification, ranks 7th/11-12th on PsyCoMark conspiracy tasks.","lead":"UCSC NLP systems extract conspiracy narrative roles via boundary-aware span classification and classify documents with RoBERTa. Mid-pack shared-task rankings and role-type analysis show entity spans are far more reliable than abstract ones.","discovery_kind":"incremental","skeptic_critique":{"model":"grok-4.5","headline":"No significant objection identified beyond the reader's already-noted L=32 span-length ceiling.","rationale":"The reader's strongest claim accurately restates the paper's empirical contribution and analysis. The weakest assumption the reader flags is real but already self-reported by the authors and does not invalidate the mid-pack rankings or the entity-vs-abstract role distinction. Because the paper is a standard SemEval system description whose claims are modest, empirical, and checkable from the full text, no further load-bearing concern arises that would move the verdict. A simple length-distribution check on gold spans would quantify the already-acknowledged limitation without altering the CONDITIONAL acceptance recommendation.","tokens_in":7610,"tokens_out":530,"duration_ms":4489,"concrete_test":"On the validation set, compute the fraction of gold markers whose token length exceeds 32 and the fraction of those long markers that are recovered after the ≤3-character gap-merging step; if >15% of abstract-role gold spans remain unrecovered solely because of the L=32 hard cutoff, the recall-ceiling claim in §4.2 is quantitatively confirmed and the mid-pack ranking should be interpreted with that caveat.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central empirical claim is modest and well-supported by the reported numbers: a boundary-aware span classifier (IoU≥0.95 positives, hard negatives, containment NMS) plus an independent RoBERTa document classifier yields mid-pack SemEval rankings and shows entity-like roles (Actor/Victim) more robust than abstract roles. The reader's weakest assumption (fixed L=32 plus post-NMS gap merging may miss multi-clause Action/Effect/Evidence spans) is already stated by the authors themselves in §3.1.1 and Limitations §4.2; it is a genuine recall ceiling but does not undermine the reported results or the qualitative analysis of boundary sensitivity. No internal inconsistency, circular derivation, or unacknowledged methodological flaw is present. Ranking numbers differ slightly between abstract and body (11th vs 12th for Subtask 2), yet this is cosmetic. The large validation-to-test drop is explained by the metric mismatch the authors themselves document (character IoU≥0.3 vs token IoU≥0.5). The argument therefore holds under the paper's own stated scope as a workshop system description.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"The paper describes the UCSC NLP systems for SemEval-2026 Task 10 (PsyCoMark). Subtask 1 (conspiracy-marker extraction) is cast as multi-label classification over enumerated candidate spans (max length 32) with boundary-aware representations (start/end tokens, mean pool, width embedding, left/right context), IoU ≥ 0.95 positive labeling, hard-negative sampling in the [0.5, 0.75) band, positive class weighting, and containment-based NMS plus adjacent-span merging. Subtask 2 (document-level conspiracy classification) is an independent RoBERTa-large sequence classifier with label smoothing (α = 0.05) and a stratified split. The systems obtain mid-pack official rankings (0.2251 macro F1 / 7th on Subtask 1; 0.7694 weighted F1 on Subtask 2) and report that entity-like roles (Actor, Victim) are substantially more robust than abstract roles (Action, Effect, Evidence) under strict token-level IoU.","tokens_in":7940,"tokens_out":770,"duration_ms":12425,"significance":"As a shared-task system description the contribution is appropriately modest and useful: it supplies a clear, fully specified baseline for boundary-sensitive multi-label span extraction on psycholinguistic conspiracy markers, documents the practical impact of hard negatives and containment NMS, and isolates a reproducible qualitative finding (entity-like vs. abstract role difficulty) that is of interest beyond the leaderboard. Hyper-parameters, sampling tiers, decoding thresholds, and per-role tables are given in sufficient detail for re-implementation. The independent modeling of the two subtasks cleanly avoids error propagation and makes the reported numbers easy to interpret. No theoretical novelty is claimed, nor is any required for this venue; the empirical package is solid and the boundary-sensitivity analysis is a genuine takeaway.","major_comments":[],"minor_comments":[{"comment":"Abstract states 11th place for Subtask 2 while §1 and §5 state 12th place (same score 0.7694). Please reconcile the ranking number throughout the manuscript.","section":null},{"comment":"§3.1.5 / Table 1 vs. Table 2: the large validation-to-test drop is correctly attributed to the change from character IoU ≥ 0.3 to token IoU ≥ 0.5, yet a short quantitative sensitivity table (same model, both metrics on the same split) would make the explanation fully transparent.","section":null},{"comment":"§3.1.1: the six-component span representation is well-motivated; a one-sentence ablation note (or citation to prior span-classification work that used the same concatenation) would strengthen the architectural claim.","section":null},{"comment":"§4.2 Limitations already flags the L = 32 ceiling; adding the empirical fraction of gold markers longer than 32 tokens (if available) would quantify the recall ceiling more precisely.","section":null},{"comment":"Minor typographic inconsistencies: “11th”/“12th”, “Can’t tell” vs. “Can't tell”, and the arXiv header date versus the SemEval-2026 venue year. A final proof-reading pass will remove them.","section":null}],"recommendation":"minor_revision","confidential_remarks":"Standard SemEval system paper; the ranking discrepancy is cosmetic and the technical content is sound. Suitable for the workshop proceedings after the minor clean-ups. No novelty or ethical concerns."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"This is a clean workshop system description for PsyCoMark. The one thing worth knowing is the empirical observation that entity-like roles (Actor, Victim) hold up under strict token IoU while abstract roles (Action, Effect, Evidence) collapse on boundary criteria; everything else is competent application of known tools.\n\nWhat they actually did: multi-label classification over enumerated spans (L≤32) with a six-part boundary-aware representation (start/end/mean/width/left/right context), IoU≥0.95 positives, hard-negative mining in the 0.5–0.75 band, and containment-based NMS with role-specific thresholds. Document classification is a separate RoBERTa-large sequence classifier with light label smoothing. They report official scores (0.2251 macro F1 / 7th on Subtask 1; 0.7694 weighted F1 / ~12th on Subtask 2), per-role tables, and a clear validation-to-test gap that they correctly attribute to the metric mismatch (char IoU 0.3 vs token IoU 0.5). The hard-negative and NMS choices are well-motivated and the error analysis (boundary drift vs under-segmentation) is useful.\n\nSoft spots are minor and mostly already flagged by the authors. L=32 plus post-NMS gap merging is a real recall ceiling for multi-clause abstract roles; the ranking number flips between abstract and body (11th vs 12th); no code or significance tests. None of these break the claims. The modeling ingredients are standard (Lee et al. 2017 lineage); the novelty is the concrete application and the role-type robustness finding, not a new architecture.\n\nThis is for people working on the same shared task or on social-media span extraction who want a reproducible mid-pack baseline and a clear statement of where boundary sensitivity bites. It is not a high-significance scientific advance, but it is honest, internally consistent, and properly scoped. I would send it to peer review for the workshop proceedings without hesitation; a light revision pass on the ranking inconsistency and a code release would be enough. Worth a quick look if you are in the area; skip if you are not.","headline":"Competent mid-pack SemEval system paper: solid span-classification engineering and honest role-type analysis, nothing methodologically new.","tokens_in":8517,"tokens_out":544,"would_cite":false,"duration_ms":4529,"reading_group":"no","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"Boundary-aware span classification finds conspiracy roles well for people, poorly for abstract actions and evidence.","keywords":["conspiracy marker extraction","span classification","boundary-aware representations","hard-negative sampling","non-maximum suppression","RoBERTa","SemEval-2026","PsyCoMark"],"falsifier":"Measure recall of gold markers longer than 32 tokens (or multi-clause abstract roles) on held-out data; if a large fraction of those gold spans are systematically missed even after gap merging, the length-cap premise fails.","tokens_in":8521,"feed_emoji":"🔍","tokens_out":886,"duration_ms":7051,"temperature":0.7,"pith_summary":"This shared-task system paper shows how to extract conspiracy-narrative roles (Actor, Action, Effect, Evidence, Victim) from Reddit text and separately decide whether a whole document is conspiratorial. The authors treat marker extraction as multi-label classification over every candidate span up to 32 tokens, labeling a span positive only when it overlaps a gold span almost exactly (IoU at least 0.95), mining hard near-miss negatives, and cleaning overlaps with containment-based non-maximum suppression plus a width-aware span vector that includes start, end, mean pool, length, and left/right context. Document-level detection is an independent RoBERTa classifier with label smoothing. Entity-like roles (Actor, Victim) emerge as comparatively reliable; abstract roles stay boundary-sensitive. The practical payoff is a mid-pack competitive pipeline that makes the boundary problem concrete rather than treating all roles as equally hard.","feed_headline":"Entity conspiracy roles extract cleanly; abstract ones stay fuzzy","feed_subtitle":"Strict IoU labeling and hard negatives help, but Action and Evidence spans remain boundary-sensitive","key_machinery":"Boundary-aware span vector: the concatenation of start token, end token, mean-pooled interior, learned width embedding, and immediate left/right context tokens, scored by a two-layer MLP and decoded by per-role thresholds plus containment NMS.","core_discovery":"Formulating conspiracy-marker extraction as multi-label span classification over enumerated candidates, with strict IoU≥0.95 positive labeling, hard-negative sampling of partial overlaps, and containment-based NMS on boundary-aware span representations, yields competitive official scores while exposing a clear difficulty split: entity-like roles are robust and abstract roles (Action, Effect, Evidence) remain highly sensitive to boundary criteria.","pith_inferences":["The same boundary-sensitivity pattern should appear in other narrative-role or SRL-style extraction tasks that mix named entities with clause-level predicates.","Raising the maximum span length or replacing enumeration with a learned span proposal step is the most direct route to higher abstract-role recall.","Containment NMS with role-specific thresholds is a lightweight substitute for full structured decoding when compute is limited.","Document-level conspiracy signals appear strong enough that simple sequence classification already captures much of the stance, so the main remaining gain is in precise marker boundaries rather than pure classification accuracy."],"forward_implications":["Entity roles can be extracted with higher confidence than abstract roles under the same pipeline.","Development metrics must match the official token-level IoU threshold; relaxed character IoU produces optimistic validation scores that do not transfer.","Hard-negative near-boundary examples are necessary for sharp localization; without them the model prefers overly broad spans.","Independent document classification can reach competitive weighted F1 without using predicted markers as features.","Future joint models can safely condition document stance on high-precision Actor/Victim spans while remaining cautious about abstract-role noise."],"fun_headline_variants":["Entity roles extract robustly; Action and Evidence stay boundary-sensitive","Strict IoU and hard negatives expose split: entities clean, abstracts fuzzy","Boundary-aware spans detect Actors well; abstract conspiracy roles lag","Multi-label span method ranks competitive, reveals entity vs abstract gap","Containment NMS helps entities but abstract markers remain boundary-fragile"],"cache_read_input_tokens":128,"weakest_assumption_plain":"The authors assume that limiting candidate spans to 32 tokens and then merging only tiny character gaps is enough to cover the real lengths of multi-clause Action, Effect, and Evidence markers.","fun_headline_variants_meta":{"raw":{"variants":["Entity roles extract robustly; Action and Evidence stay boundary-sensitive","Strict IoU and hard negatives expose split: entities clean, abstracts fuzzy","Boundary-aware spans detect Actors well; abstract conspiracy roles lag","Multi-label span method ranks competitive, reveals entity vs abstract gap","Containment NMS helps entities but abstract markers remain boundary-fragile"]},"model":"grok-4.5","effort":"low","cost_usd":0.004568,"raw_usage":{"total_tokens":1282,"prompt_tokens":733,"num_sources_used":0,"completion_tokens":93,"cost_in_usd_ticks":45680000,"prompt_tokens_details":{"text_tokens":733,"audio_tokens":0,"image_tokens":0,"cached_tokens":128},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":456,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":733,"tokens_out":93,"duration_ms":3624,"temperature":1.0,"reasoning_tokens":456,"cache_read_input_tokens":128,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-11T03:43:29.919148+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"Measure recall of gold markers longer than 32 tokens (or multi-clause abstract roles) on held-out data; if a large fraction of those gold spans are systematically missed even after gap merging, the length-cap premise fails.","supporting_citations":[],"review_version":1}