{"id":"f479e23b-6d78-49e3-8bf4-7051fa86f285","arxiv_id":"1908.05691","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"FROBES splits the inside tag into front and rear tags and improves F1 by roughly 0.2 points for biomedical entities longer than two words on JNLPBA and i2b2 datasets.","lead":"This paper proposes FROBES, a new tagging scheme for biomedical named entity recognition that replaces the inside tag with front and rear tags for entities longer than two words. Evaluations on two biomedical datasets show F1 gains of about 0.2 points for such entities, but the gains are not tested for statistical significance.","discovery_kind":"incremental","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Claimed FROBES advantage for entities longer than two words rests on single-run F1 gaps of 0.18–0.20 over aggregated N≥3 entities, with no variance reporting or significance testing; the aggregation also hides N=3, where FROBES adds no front/rear split.","rationale":"The paper makes a comparative empirical claim, and its only direct support is the N≥3 rows of Tables 4 and 5. Those differences are small, and the paper gives no statistical evidence that they are reproducible rather than artifacts of a single random initialization. The reader's weakest assumption identifies this correctly. I add a design-level observation that strengthens rather than replaces the concern: FROBES's tag rule assigns B-F-E to length-3 entities, leaving no R tag, so the proposed front/rear distinction does not apply at N=3. Aggregating N=3 with N>3 therefore risks attributing to the proposed mechanism a gain that could come from N>3 alone, or from noise. This does not make the paper's result false; it makes the central claim under-supported at the current confidence level. The appropriate disposition is the same conditional acceptance the reader already recommended, pending a stratified, multi-seed rerun with uncertainty estimates. I would not move to reject or accept outright without that test.","tokens_in":7876,"tokens_out":7274,"duration_ms":71687,"concrete_test":"Re-run the same BiLSTM-CRF baseline with IOBES and FROBES tag sets on JNLPBA and i2b2 using at least 10 random seeds, fixed hyperparameters, and identical pretrained embeddings. Report F1 separately for N=3, N>3, and N≥3, with paired bootstrap 95% confidence intervals for the FROBES−IOBES difference. If the N≥3 interval includes 0, or if the N=3 difference is not near 0 while the N>3 difference is significant, the central claim as stated is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that FROBES outperforms other SR models for multi-word entities with length greater than two. The supporting evidence is Tables 4 and 5, where FROBES beats IOBES by 0.20 F1 (65.83 vs 65.63) on JNLPBA and 0.18 F1 (77.00 vs 76.82) on i2b2 for N≥3. These are single runs: the paper reports no random seeds, no confidence intervals, no paired tests, and no code or hyperparameter details. For a BiLSTM-CRF model, seed and optimizer variance on a subset of test entities is usually comparable to or larger than 0.2 F1, so the reported advantage may be entirely noise. A second, reinforcing issue concerns the mechanism: FROBES replaces I with F and R only for entities of length greater than three. For n=3, the rule gives B-F-E, which encodes the same segmentation as B-I-E but with no front/rear distinction; the proposed mechanism therefore cannot explain an N=3 improvement, and Tables 4 and 5 aggregate N=3 with N>3. Thus the headline claim conflates a length where the representation is unchanged with the N>3 cases where F/R tagging applies.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes FROBES, a new segment representation (SR) tag set for biomedical named entity recognition. FROBES extends IOBES by replacing the inner tag I with two tags, F and R, for tokens in the front and rear halves of multi-word entities of length greater than two. The authors evaluate FROBES against IOB2 and IOBES using the same BiLSTM+CRF baseline on the JNLPBA 2004 and i2b2/VA 2010 datasets, and additionally combine the three taggers with majority voting. The central empirical claim is that FROBES outperforms IOB2 and IOBES for entities of length N >= 3, supported by Tables 4 and 5 with F1 gains of about 0.2 points, and that the ensemble further improves performance. The paper motivates the tag split by the observation that many biomedical entities share common ending expressions.","tokens_in":8164,"tokens_out":6799,"duration_ms":61890,"significance":"If the reported gains are statistically reliable, the contribution is a simple, parameter-free modification of the tag scheme that may improve recognition of long biomedical entities and can be plugged into existing neural architectures. The evaluation is internally consistent: all tag schemes share the same BiLSTM+CRF pipeline, and FROBES itself has no fitted parameters. These strengths are offset by the very small reported improvements and the complete absence of variance estimates or significance tests, so the central claim is not yet established. The paper is an incremental contribution to BioNER, with the main value being the possible demonstration that richer positional tags can help for long entities.","major_comments":[{"comment":"The central claim rests on F1 differences of 0.18–0.20 points for N>=3 entities (65.83 vs. 65.63 on JNLPBA and 77.00 vs. 76.82 on i2b2). The paper appears to report a single run per configuration and gives no random seeds, confidence intervals, error bars, or significance tests, although BiLSTM+CRF models are known to be sensitive to initialization and optimization noise. At this effect size, the reported advantage of FROBES over IOBES may be within run-to-run variation. I ask the authors to run multiple restarts or otherwise provide variance estimates, and to report a paired significance test (e.g., McNemar's test or a bootstrap over entities) for the N>=3 comparison.","section":"Section 5.3, Tables 4 and 5"},{"comment":"The headline claim concerns entities of length greater than two, but Tables 4 and 5 report only the aggregate N>=3. For N=3, the FROBES rule yields B-F-E with zero R tags, as follows from the counting formula in Section 4.1, and the single inner token's position is already fully determined by the B and E boundaries in IOBES's B-I-E encoding. The proposed front/rear distinction therefore adds no new segmentation information at N=3; the R tag first appears at N=4. To substantiate the claimed improvement for 'length greater than two,' the authors must report N=3 and N>3 separately; otherwise the improvement could be driven entirely by N>3 cases while N=3 is unchanged or worse.","section":"Section 4.1, Tables 4 and 5"}],"minor_comments":[{"comment":"The precision formula reads P = TP/(TP+TF); the denominator should be TP+FP (false positives), not TF. Please correct this typo.","section":"Section 5.1"},{"comment":"The text attributes the CNN multi-task model to 'Gamal et al. [24]', but reference [24] is Crichton et al.; please align the in-text citation with the bibliography entry.","section":"Section 3"},{"comment":"The acronym is spelled 'FORBES' in the sentence 'It is clear that, FORBES outperforms the other two models...'; use 'FROBES' consistently throughout.","section":"Section 5.3"},{"comment":"The term 'Polysomy' in the list of BioNER challenges should probably be 'polysemy'.","section":"Section 1"},{"comment":"Please report the hyperparameters of the BiLSTM+CRF baseline (embedding dimensions, hidden sizes, dropout, optimizer, epochs, and model selection criterion) and state whether the reported numbers are single runs or averages; without this information the experiments cannot be reproduced or the variance assessed.","section":"Section 5"}],"recommendation":"major_revision","confidential_remarks":"The paper is in scope and the idea is simple and easy to replicate, but the main risk is that the reported improvement is noise. I would be willing to review a revision that adds multi-seed experiments, significance testing, and disaggregated N=3 / N>3 results. The citation/reference mismatch and the typos should also be fixed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The new thing here is FROBES, a tweak to IOBES that replaces I with F and R for multi-word entities longer than two tokens. The paper does a clean, internally consistent comparison: the same BiLSTM-CRF pipeline is run with IOB2, IOBES, and FROBES on JNLPBA and i2b2, and the tag schemes are described clearly. Credit where due: the evaluation is a genuine head-to-head, not a fitting exercise, and the authors are upfront that the gain is for multi-word entities, not for single tokens. The F/R split is a simple idea, and the common-endings motivation is post hoc but not circular.\n\nThe soft spots are real and about evidence, not about the idea. First, every number comes from a single run. No seeds, no variance, no significance test. A 0.2 F1 gap on a subset of test entities is exactly the size of seed-level noise in a BiLSTM-CRF, so the headline advantage could easily vanish with more runs. Second, and more specific, the paper claims improvement for entities \"with length greater than two\" but reports results for N≥3 aggregated. For N=3, FROBES assigns B-F-E with no R tag — the split reduces to a renaming of I, so the mechanism does not apply. The aggregate hides whether N=3 helps, hurts, or is flat, and Table 2 shows N=3 is a sizable fraction (about 14% of entities in both test sets). The real claim should be about N>3, and the paper does not present that breakdown. Third, on i2b2 the overall F1 for FROBES (83.15) is below IOBES (83.30), so the benefit is confined to a subset that is not cleanly isolated. No code or hyperparameters are provided, which makes the single-run issue harder to evaluate.\n\nNone of this sinks the paper. The comparison is honest, the idea is testable, and with multi-seed runs, confidence intervals, and a separate N>3 column, the result could be a modest but useful contribution to the tag-scheme literature. The stress-test note about N=3 is correct and should be taken seriously.\n\nWho is this for? Researchers working on sequence-labeling tag schemes or BioNER pipelines. It is not a major advance, but it is a legitimate empirical data point. I would not cite it in my own work unless the authors tighten the evaluation, but I would send it to peer review rather than desk-reject it: the claim is specific, the experiments are reproducible in principle, and the flaw is fixable.","headline":"Small tag-scheme tweak with an honest but statistically weak claim; worth a careful referee, not a desk reject.","tokens_in":8680,"tokens_out":2484,"would_cite":false,"duration_ms":24108,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"FROBES, a tag-set extension of IOBES that splits the Inside tag into Front and Rear halves, improves F1 for biomedical entities of length three or more on two benchmark datasets.","keywords":["Biomedical Named Entity Recognition","Segment Representations","FROBES","IOBES","BiLSTM","multi-word entities","majority voting ensemble","sequence labeling"],"falsifier":"Run both IOBES and FROBES on JNLPBA and i2b2 with the same BiLSTM-CRF code and, say, ten random seeds each, and compare the N >= 3 F1 distributions; if the distributions overlap or the median difference is not positive, the reported 0.18- and 0.20-point gaps are random initialization noise.","tokens_in":7703,"feed_emoji":"🧬","tokens_out":6883,"duration_ms":58239,"temperature":0.7,"pith_summary":"The paper tries to establish that the way we label the inside of a long named entity matters for recognizing it, and that a modest change to the standard IOBES tag set can move performance on biomedical texts. It introduces FROBES, which replaces the single Inside tag with two tags, Front and Rear, for entities of more than two words, so the tagger is told which half of the entity a token sits in. In a BiLSTM-CRF system on the JNLPBA and i2b2 datasets, FROBES improves F1 for entities of length three or more compared with IOB2 and IOBES, while staying competitive overall. The reader should care because biomedical names are often long and multi-word, and a tag-set change costs nothing at inference time.","feed_headline":"New tag set FROBES beats IOBES on long biomedical names","feed_subtitle":"Splitting the 'inside' tag into front and rear parts nudges F1 up on JNLPBA and i2b2.","key_machinery":"The machinery is the FROBES tag set: tags F, R, O, B, E, S stand for front, rear, outside, begin, end, and single. For an entity of n words with n > 2, the first token gets B, the last gets E, single-word entities get S, and the n - 2 interior tokens are split so that the front half is tagged F and the rear half is tagged R (if n is even, F and R each occur (n-2)/2 times; if n is odd, F occurs (n-1)/2 times and R occurs (n-3)/2 times). This gives the decoder a positional signal about which side of a long entity a token belongs to, and it is the only difference between FROBES and IOBES; the underlying word/character BiLSTM and CRF decoder are unchanged. The paper motivates the split by noting that many biomedical names share class-diagnostic endings such as 'transcription factor' or 'binding site', which land in the rear half.","core_discovery":"On the paper's own terms, the discovery is that replacing the single 'I' (inside) tag of IOBES with two positional tags, F (front) and R (rear), for any entity longer than two words improves the F1 of biomedical named entity recognition on long entities. For entities of length three or more, FROBES scores 65.83 versus 65.63 for IOBES on JNLPBA and 77.00 versus 76.82 on i2b2; for entities of length two or one the differences are smaller and not consistently in its favor. The same paper reports that a majority-voting ensemble over IOB2, IOBES, and FROBES outputs reaches an overall F1 of 71.99 on JNLPBA and 83.62 on i2b2, above every individual tag set.","pith_inferences":["The paper does not test this, but the F/R split should reduce end-boundary errors specifically, since the rear half of every long entity is tagged distinctly; an error analysis separating start and end boundary errors would show this directly.","The paper does not test this, but the same positional split could transfer to other sequence-labeling tasks with long spans and class-diagnostic suffixes, such as adverse drug event extraction or chemical entity recognition.","The paper does not test this, but if the N >= 3 gain is replicated across random seeds, a natural extension is to make the front/rear split adaptive to entity class or to learn the split point, rather than fixing it at the midpoint."],"forward_implications":["For any sequence-labeling system already using IOBES, switching to FROBES is a drop-in change to the tag layer only; no architectural retraining beyond the new label set is required.","The representation concentrates its benefit on entities of at least three words, so domains with many long entity names, such as biomedical abstracts and clinical notes, are where the gain should appear.","Because the F/R split creates tag diversity, majority voting over IOB2, IOBES, and FROBES yields a further gain, reaching 71.99 F1 on JNLPBA and 83.62 on i2b2.","Entities that share a common ending now share the same positional tag pattern, which the paper argues helps the model assign the correct semantic class as well as the correct boundary."],"supporting_citations":[{"why":"Provides the direct earlier comparison of segment representation models on JNLPBA and i2b2 that this paper extends.","marker":"[27]"},{"why":"Prior extension of IOBES for biomedical named entity recognition; FROBES is positioned against it.","marker":"[28]"},{"why":"Supplies the multiple-segment-representation approach and the prior study of how tag sets affect NER.","marker":"[26]"},{"why":"Defines the JNLPBA 2004 shared-task data and its entity classes, one of the two evaluation sets.","marker":"[29]"},{"why":"Defines the i2b2/VA 2010 clinical notes and its three entity classes, the second evaluation set.","marker":"[30]"},{"why":"Supplies the GENIA corpus, source of the JNLPBA training abstracts.","marker":"[31]"},{"why":"Introduces the LSTM unit the baseline encoder is built on.","marker":"[9]"},{"why":"Introduces bidirectional LSTM, giving the left and right context the tagger uses.","marker":"[10]"},{"why":"Supplies CRF decoding, the final layer that turns BiLSTM outputs into tag sequences.","marker":"[22]"}],"fun_headline_variants":["FROBES tag set boosts long biomedical entity recognition","Split I tag improves F1 for long biomedical names","Front-rear tags edge out IOBES on lengthy entities","FROBES: better tags for multi-word biomedical terms"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the F1 gaps of 0.18 points on JNLPBA and 0.20 points on i2b2 for entities of length three or more are real and repeatable rather than artifacts of a single random initialization, because the paper reports one run per configuration with no variance, confidence interval, or significance test.","fun_headline_variants_meta":{"raw":{"variants":["FROBES tag set boosts long biomedical entity recognition","Split I tag improves F1 for long biomedical names","Front-rear tags edge out IOBES on lengthy entities","FROBES: better tags for multi-word biomedical terms"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000198,"raw_usage":{"total_tokens":1369,"prompt_tokens":945,"completion_tokens":424,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":561,"completion_tokens_details":{"reasoning_tokens":356}},"tokens_in":561,"tokens_out":424,"duration_ms":4612,"temperature":1.0,"reasoning_tokens":356,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T13:06:12.831198+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run both IOBES and FROBES on JNLPBA and i2b2 with the same BiLSTM-CRF code and, say, ten random seeds each, and compare the N >= 3 F1 distributions; if the distributions overlap or the median difference is not positive, the reported 0.18- and 0.20-point gaps are random initialization noise.","supporting_citations":[{"cited_title":"and Lu, B.-L.: Effective tag set selection in Chinese word segmentation via conditional random field modeling","cited_arxiv_id":null,"evidence_quote":"Provides the direct earlier comparison of segment representation models on JNLPBA and i2b2 that this paper extends."},{"cited_title":"\"Design challenges and misconceptions in named entity recognition","cited_arxiv_id":null,"evidence_quote":"Prior extension of IOBES for biomedical named entity recognition; FROBES is positioned against it."},{"cited_title":"Computational Linguistics and Chinese Language Processing vol","cited_arxiv_id":null,"evidence_quote":"Supplies the multiple-segment-representation approach and the prior study of how tag sets affect NER."},{"cited_title":"C., Okazaki N., Miwa M","cited_arxiv_id":null,"evidence_quote":"Defines the JNLPBA 2004 shared-task data and its entity classes, one of the two evaluation sets."},{"cited_title":"Marcus: Text chunking using transformation-based learning","cited_arxiv_id":null,"evidence_quote":"Defines the i2b2/VA 2010 clinical notes and its three entity classes, the second evaluation set."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the GENIA corpus, source of the JNLPBA training abstracts."},{"cited_title":"The T cell surface molecule CD28 binds to ligands on accessory cells and APCs ,","cited_arxiv_id":null,"evidence_quote":"Introduces the LSTM unit the baseline encoder is built on."},{"cited_title":"These approaches depend es sentially on extracting feature set used for training the appropriate algorithm","cited_arxiv_id":null,"evidence_quote":"Introduces bidirectional LSTM, giving the left and right context the tagger uses."},{"cited_title":"and Jürgen Schmidhuber","cited_arxiv_id":null,"evidence_quote":"Supplies CRF decoding, the final layer that turns BiLSTM outputs into tag sequences."}],"review_version":1}