REVIEW 2 major objections 5 minor 44 references
Improving Multi-Word Entity Recognition for Biomedical Texts
T0 review · 2 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read FROBES, a tag-set extension of IOBES that splits the Inside tag into Front and Rear halves, improves F1 for biomedical entities of length three or more on two benchmark datasets.
desk verdict Small tag-scheme tweak with an honest but statistically weak claim; worth a careful referee, not a desk reject. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The machinery is the FROBES tag set: tags F, R, O, B, E, S stand for front, rear, outside, begin, end, and single. For an entity of n words with n > 2, the first token gets B, the last gets E, single-word entities get S, and the n - 2 interior tokens are split so that the front half is tagged F and the rear half is tagged R (if n is even, F and R each occur (n-2)/2 times; if n is odd, F occurs (n-1)/2 times and R occurs (n-3)/2 times). This gives the decoder a positional signal about which side of a long entity a token belongs to, and it is the only difference between FROBES and IOBES; the underlying word/character BiLSTM and CRF decoder are unchanged. The paper motivates the split by noting that many biomedical names share class-diagnostic endings such as 'transcription factor' or 'binding site', which land in the rear half.
What would settle it
Run both IOBES and FROBES on JNLPBA and i2b2 with the same BiLSTM-CRF code and, say, ten random seeds each, and compare the N >= 3 F1 distributions; if the distributions overlap or the median difference is not positive, the reported 0.18- and 0.20-point gaps are random initialization noise.
Extended reading notes
Core claim
On the paper's own terms, the discovery is that replacing the single 'I' (inside) tag of IOBES with two positional tags, F (front) and R (rear), for any entity longer than two words improves the F1 of biomedical named entity recognition on long entities. For entities of length three or more, FROBES scores 65.83 versus 65.63 for IOBES on JNLPBA and 77.00 versus 76.82 on i2b2; for entities of length two or one the differences are smaller and not consistently in its favor. The same paper reports that a majority-voting ensemble over IOB2, IOBES, and FROBES outputs reaches an overall F1 of 71.99 on JNLPBA and 83.62 on i2b2, above every individual tag set.
Load-bearing premise
The load-bearing premise is that the F1 gaps of 0.18 points on JNLPBA and 0.20 points on i2b2 for entities of length three or more are real and repeatable rather than artifacts of a single random initialization, because the paper reports one run per configuration with no variance, confidence interval, or significance test.
Editorial extensions
If this is right
- For any sequence-labeling system already using IOBES, switching to FROBES is a drop-in change to the tag layer only; no architectural retraining beyond the new label set is required.
- The representation concentrates its benefit on entities of at least three words, so domains with many long entity names, such as biomedical abstracts and clinical notes, are where the gain should appear.
- Because the F/R split creates tag diversity, majority voting over IOB2, IOBES, and FROBES yields a further gain, reaching 71.99 F1 on JNLPBA and 83.62 on i2b2.
- Entities that share a common ending now share the same positional tag pattern, which the paper argues helps the model assign the correct semantic class as well as the correct boundary.
Reading between the lines
- The paper does not test this, but the F/R split should reduce end-boundary errors specifically, since the rear half of every long entity is tagged distinctly; an error analysis separating start and end boundary errors would show this directly.
- The paper does not test this, but the same positional split could transfer to other sequence-labeling tasks with long spans and class-diagnostic suffixes, such as adverse drug event extraction or chemical entity recognition.
- The paper does not test this, but if the N >= 3 gain is replicated across random seeds, a natural extension is to make the front/rear split adaptive to entity class or to learn the split point, rather than fixing it at the midpoint.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes FROBES, a new segment representation (SR) tag set for biomedical named entity recognition. FROBES extends IOBES by replacing the inner tag I with two tags, F and R, for tokens in the front and rear halves of multi-word entities of length greater than two. The authors evaluate FROBES against IOB2 and IOBES using the same BiLSTM+CRF baseline on the JNLPBA 2004 and i2b2/VA 2010 datasets, and additionally combine the three taggers with majority voting. The central empirical claim is that FROBES outperforms IOB2 and IOBES for entities of length N >= 3, supported by Tables 4 and 5 with F1 gains of about 0.2 points, and that the ensemble further improves performance. The paper motivates the tag split by the observation that many biomedical entities share common ending expressions.
Significance. If the reported gains are statistically reliable, the contribution is a simple, parameter-free modification of the tag scheme that may improve recognition of long biomedical entities and can be plugged into existing neural architectures. The evaluation is internally consistent: all tag schemes share the same BiLSTM+CRF pipeline, and FROBES itself has no fitted parameters. These strengths are offset by the very small reported improvements and the complete absence of variance estimates or significance tests, so the central claim is not yet established. The paper is an incremental contribution to BioNER, with the main value being the possible demonstration that richer positional tags can help for long entities.
major comments (2)
- [Section 5.3, Tables 4 and 5] The central claim rests on F1 differences of 0.18–0.20 points for N>=3 entities (65.83 vs. 65.63 on JNLPBA and 77.00 vs. 76.82 on i2b2). The paper appears to report a single run per configuration and gives no random seeds, confidence intervals, error bars, or significance tests, although BiLSTM+CRF models are known to be sensitive to initialization and optimization noise. At this effect size, the reported advantage of FROBES over IOBES may be within run-to-run variation. I ask the authors to run multiple restarts or otherwise provide variance estimates, and to report a paired significance test (e.g., McNemar's test or a bootstrap over entities) for the N>=3 comparison.
- [Section 4.1, Tables 4 and 5] The headline claim concerns entities of length greater than two, but Tables 4 and 5 report only the aggregate N>=3. For N=3, the FROBES rule yields B-F-E with zero R tags, as follows from the counting formula in Section 4.1, and the single inner token's position is already fully determined by the B and E boundaries in IOBES's B-I-E encoding. The proposed front/rear distinction therefore adds no new segmentation information at N=3; the R tag first appears at N=4. To substantiate the claimed improvement for 'length greater than two,' the authors must report N=3 and N>3 separately; otherwise the improvement could be driven entirely by N>3 cases while N=3 is unchanged or worse.
minor comments (5)
- [Section 5.1] The precision formula reads P = TP/(TP+TF); the denominator should be TP+FP (false positives), not TF. Please correct this typo.
- [Section 3] The text attributes the CNN multi-task model to 'Gamal et al. [24]', but reference [24] is Crichton et al.; please align the in-text citation with the bibliography entry.
- [Section 5.3] The acronym is spelled 'FORBES' in the sentence 'It is clear that, FORBES outperforms the other two models...'; use 'FROBES' consistently throughout.
- [Section 1] The term 'Polysomy' in the list of BioNER challenges should probably be 'polysemy'.
- [Section 5] Please report the hyperparameters of the BiLSTM+CRF baseline (embedding dimensions, hidden sizes, dropout, optimizer, epochs, and model selection criterion) and state whether the reported numbers are single runs or averages; without this information the experiments cannot be reproduced or the variance assessed.
Circularity Check
No circularity: FROBES is a fixed tag transformation evaluated head-to-head on external benchmarks; there is no fitted parameter or self-citation chain that forces the reported result.
full rationale
The paper proposes FROBES, a deterministic extension of the IOBES tag set in which the inner tag I is split into F and R depending on whether a token lies in the front or rear half of a multi-word entity. The central claim, that FROBES improves recognition for entities of length greater than two, is an empirical claim tested on the standard JNLPBA and i2b2/VA 2010 datasets using a BiLSTM-CRF baseline. There is no derivation chain, no fitted parameter that is later renamed as a prediction, and no mathematical identity that makes the conclusion equal to an input by construction. The tag transformation itself is defined independently of the test labels: the F/R split is a one-to-one refinement of I based solely on token position within the entity, so the reported F1 differences are genuine measurements rather than consequences of how the model is defined. The observed common-ending motivation (e.g., protein names ending in "transcription factor") is plausibly post hoc, but post hoc hypothesis formation does not make an evaluation on held-out test data circular. The only reference to the authors' own prior work, [27], is a related-work comparative study and is not load-bearing for the central claim. The absence of multiple runs, confidence intervals, or significance tests is a correctness and reproducibility concern, not a circularity concern. The evaluation is therefore self-contained against external benchmarks, and no circular step can be identified.
Assumptions & free parameters
free parameters (1)
- BiLSTM+CRF hyperparameters
assumptions (3)
- standard math LSTM and CRF formulations are correct and applicable to sequence labeling
- domain assumption Multi-word biomedical entities often share common endings that carry class information
- domain assumption JNLPBA and i2b2/VA 2010 annotations and standard splits are used without modification
Cite this review
Pith. "Pith review of Improving Multi-Word Entity Recognition for Biomedical Texts." pith.science (2026). https://pith.science/paper/R5KJB7UF
@misc{pith2026190805691,
author = {Pith},
title = {Pith review of: Improving Multi-Word Entity Recognition for Biomedical Texts},
year = {2026},
howpublished = {\url{https://pith.science/paper/R5KJB7UF}},
note = {Machine review of arXiv:1908.05691}
}
read the original abstract
Biomedical Named Entity Recognition (BioNER) is a crucial step for analyzing Biomedical texts, which aims at extracting biomedical named entities from a given text. Different supervised machine learning algorithms have been applied for BioNER by various researchers. The main requirement of these approaches is an annotated dataset used for learning the parameters of machine learning algorithms. Segment Representation (SR) models comprise of different tag sets used for representing the annotated data, such as IOB2, IOE2 and IOBES. In this paper, we propose an extension of IOBES model to improve the performance of BioNER. The proposed SR model, FROBES, improves the representation of multi-word entities. We used Bidirectional Long Short-Term Memory (BiLSTM) network; an instance of Recurrent Neural Networks (RNN), to design a baseline system for BioNER and evaluated the new SR model on two datasets, i2b2/VA 2010 challenge dataset and JNLPBA 2004 shared task dataset. The proposed SR model outperforms other models for multi-word entities with length greater than two. Further, the outputs of different SR models have been combined using majority voting ensemble method which outperforms the baseline models performance.
Figures
Reference graph
Works this paper leans on
-
[1]
Introduction Named Entity Recognition (NER) is defined as identifying the named entities (NEs) in the text and classifying them into predefined semantic categories [1]. Names of p laces, organizations and persons are examples of NEs in general 2 domain, while RNA, DNA, proteins, treatment and medical test are examples of NEs in biomedical domain (BioNEs)....
-
[2]
A single abbreviation can be interpreted as two different entities according to the context
Ambiguity: abbreviations are the major source of ambiguity. A single abbreviation can be interpreted as two different entities according to the context. For example, “ EGFR” corresponds to epidermal growth factor receptor or estimated glomerular filtration rate
-
[3]
For example, “myc-c” refers to the name of a gene or protein
Polysomy: a word refers to different entities . For example, “myc-c” refers to the name of a gene or protein
-
[4]
For example, CASP3, caspase-3, and CPP32 denote the same entity [2]
Synonyms: an entity can be denoted by multiple names or aliases. For example, CASP3, caspase-3, and CPP32 denote the same entity [2]
-
[5]
Out of dictiona ry: the overwhelming growth rate and the frequent insertion of new names into the dictionary [3]
-
[6]
Multi-word BioNEs: most of BioNEs have multiple words , for example, CD28 surface receptor
-
[7]
BP” (blood pressure) corresponding to laboratory test is a BioNE that occurs in “control BP
Nested BioNEs: a BioNE may occur as part of longer BioNE as a proper string. For example, “BP” (blood pressure) corresponding to laboratory test is a BioNE that occurs in “control BP” which is a treatment
-
[8]
Lack of standard nomenclature for BioNEs of the same class. Approaches for Bio NER varies from dictionary -based, rule -based, Machine Learning (ML) to hybrid approaches. The widely used ML approaches use annotated data to train a learning model which is then used to classify the unseen BioNEs. Combining the output of different classifiers using ensemble ...
Show all 44 references
-
[9]
The T cell surface molecule CD28 binds to ligands on accessory cells and APCs ,
Segment Representation (SR) Models One of the major requirements of learning algorithms is a n annotated corpus. Segment representation (SR) models which have been applied for different NLP tasks such as Noun Phrase chunking (NP -chunking) [ 11-12], word segmentation [13-14], ...
-
[10]
These approaches depend es sentially on extracting feature set used for training the appropriate algorithm
Related Work Different ML algorithms such as Support Vector machines (SVMs) [ 21], Conditional Random Fields (CRFs) [ 22] and Maximum Entropy (ME) [ 23] have been used for BioNER . These approaches depend es sentially on extracting feature set used for training the appropriate...
2010
-
[11]
human proximal sequence element -binding transcription factor
Proposed model 4.1 FROBES We propose FROBES, an extension of IOBES model, used to represent multi- word entities using the tags (F/R/O/B/E/S) for ( front, rear, outside, begin, end, single) respectively. In this model, the tag I in IOBES model is replaced by the tags F and R f...
-
[12]
human",
Experiments We conducted experiments using ANN model which contains a bi -LSTM for character representation and a bi -LSTM for word context representation and CRF for decoding the results to tags. 5.1 Performance Evaluation We used f1-measure as a performance evaluation for Bi...
2004
-
[13]
To evaluate FROBES, we used a Bi-LSTM based model as a baseline system on JNLPBA and i2b2 datasets
Conclusion We have proposed a new SR model, FROBES, to improve multi -word BioNEs representation. To evaluate FROBES, we used a Bi-LSTM based model as a baseline system on JNLPBA and i2b2 datasets . Experimental results show that, FROBES has improved performance of BioNER for ...
-
[14]
and Sundheim B.: Message Understanding Conference -6: a brief history
Grishman R. and Sundheim B.: Message Understanding Conference -6: a brief history. In proceedings of the 16th International Conference on Computational Linguistics, vol. 1, (1996)
1996
-
[15]
Tanabe and W
L. Tanabe and W. J. Wilbur: Tagging gene and protein names in biomedical text. Bioinformatics, vol. 18(8), pp. 1124–1132, (2002)
2002
-
[16]
Ananiadou and J
S. Ananiadou and J. Mcnaught: Text Mining for Biology and Biomedicine. Norwood, MA, USA: Artech House, Inc., 2005
2005
-
[17]
and Zhang Z.: A generic classifier -ensemble approach for biomedical named entity recognition, Advances in Knowledge Discovery and Data Mining, 86-97
Liao Z. and Zhang Z.: A generic classifier -ensemble approach for biomedical named entity recognition, Advances in Knowledge Discovery and Data Mining, 86-97
-
[18]
BMC bioinformatics, 18(11), 385, (2017)
Haode Li, Qingcai Chen, Buzhou Tang, Xiaolong Wang, Hua Xu, Baohua Wang and Dong Huang: CNN -based ranking for biomedical entity normalization. BMC bioinformatics, 18(11), 385, (2017)
2017
-
[19]
In International Conference on Advanced Intelligent Systems and Informatics (pp
Xu K., Zhou Z., Hao T., and Liu W.: A Bidirectional LSTM and Conditional Random Fields Approach to Medical Named Entity Recognition. In International Conference on Advanced Intelligent Systems and Informatics (pp. 355-365). Springer, (2017)
2017
-
[20]
IEEE transactions on neural networks , 5(2), 157-166, (1994)
Bengio, Y., Simard, P., and Frasconi, P.: Learning long -term dependencies with gradient descent is difficult. IEEE transactions on neural networks , 5(2), 157-166, (1994)
1994
-
[21]
I n International Conference on Machine Learning, pp
Pascanu Razvan, Tomas Mikolov and Yoshua Bengio.: On the difficulty of training recurrent neural networks. I n International Conference on Machine Learning, pp. 1310-1318, (2013). 12
2013
-
[22]
and Jürgen Schmidhuber
Hochreiter S. and Jürgen Schmidhuber. : Long short -term memory. Neural computation 9, no. 8 (1997): 1735-1780
1997
-
[23]
Neural Networks, 18(5), 602-610, (2005)
Graves Alex and Jurgen Schmidhuber.: Framewise phoneme classifica tion with bidirectional LSTM and other neural network architectures. Neural Networks, 18(5), 602-610, (2005)
2005
-
[24]
and Frank G.: Tagging unknown proper names using decision trees
Béchet F., Alexis N . and Frank G.: Tagging unknown proper names using decision trees. In proceedings of the 38th Annual Meeting on ACL, 77-84, 2000
2000
-
[25]
173-179, (1999)
S ang, Erik F., and Jorn Veenstra: Representing text chunks ., In Proceedings of the ninth conference on European chapter of the Association for Computational Linguistics, ACL, pp. 173-179, (1999)
1999
-
[26]
Computational Linguistics and Chinese Language Processing vol
Xue N.: Chinese word segmentation as character tagging. Computational Linguistics and Chinese Language Processing vol. 8(1), pp. 29-48, (2003)
2003
-
[27]
and Lu, B.-L.: Effective tag set selection in Chinese word segmentation via conditional random field modeling
Zhao, H., Huang, C.-N., Li, M. and Lu, B.-L.: Effective tag set selection in Chinese word segmentation via conditional random field modeling. I n Proceedings of the 20 th Asian pacific conference on Language, Information and Computation, pp. 87-94, (2006)
2006
-
[28]
"Design challenges and misconceptions in named entity recognition
Ratinov, Lev, and Dan Roth. "Design challenges and misconceptions in named entity recognition. In Proceedings of the Thirteenth Conference on Computational Natural Language Learning, ACL, pp. 147-155, 2009
2009
-
[29]
C., Okazaki N., Miwa M
Cho H. C., Okazaki N., Miwa M. and Tsujii J. I.: Named entity recognition with multiple segment representations. Information Processing and Management, 49(4), pp. 954-965, (2013)
2013
-
[30]
Marcus: Text chunking using transformation-based learning
Ramshaw Lance and Mitchell P. Marcus: Text chunking using transformation-based learning. In Natural language processing using very large corpora, pp. 157-176. Springer Netherlands, (1999)
1999
-
[31]
Adwait Ratnaparkhi: Maximum entropy models for natural la nguage ambiguity resolution. Ph.D. thesis, University of Pennsylvania, PA, USA, (1998)
1998
-
[32]
In Proceedings of the second meeting of the North American Chapter of ACL on Language Technologies, pp
Taku Kudo and Yuji Matsumoto: Chunking with support vector machines. In Proceedings of the second meeting of the North American Chapter of ACL on Language Technologies, pp. 1-8, (2001)
2001
-
[33]
Sun Jiashen, Tianmin Wang, Li Li, and Xing Wu.: Person name disambiguation based on topic model." In CIPS-SIGHAN Joint Conference on Chinese Language Processing, p. 391. 2010
2010
-
[34]
Z. Ju, J. Wang, and F. Zhu: Named entity recognition from biomedical text using SVM. In 5 th International Conference on Bioinformatics and Biomedical Engineering, (ICBBE) pp. 1–4, (2011)
2011
-
[35]
He and M
Y. He and M. Kayaalp : Biological entity recognition with conditional random fields. In the Proceedings AMIA Annual Symposium Proceedings, 2008
2008
-
[36]
Y. Lin, T. Tsai, W. Chou, K. Wu, T. Sung, and W. Hsu : A maximum entropy approach to biomedical named entity recognition . Proceedings of the Workshop on Data Mining in Bioinformatics BIOKDD, USA, 56–61, 2004
2004
-
[37]
and Korhonen A.: A neural network multi-task learning approach to biomedical named entity recognition
Crichton G., Pyysalo S., Chiu B. and Korhonen A.: A neural network multi-task learning approach to biomedical named entity recognition. BMC bioinformatics, 18(1), 368-381. 13
-
[38]
In Pacific symposium on biocomputing, vol
Leaman Robert, and Graciela Gonzalez.: BANNER: an executable survey of advances in biomedical named entity recognition. In Pacific symposium on biocomputing, vol. 13, pp. 652-663, (2008)
2008
-
[39]
Information Processing & Management, vol
Cho, Han -Cheol, Naoaki Okazaki, Makoto Miwa, and Jun’ichi Tsujii: Named entity recognition with multiple segment representations. Information Processing & Management, vol. 49(4), pp. 954-965, (2013)
2013
-
[40]
L., and Hamada A
Shashirekha H. L., and Hamada A. Nayel.: A comparative study of segment representation for biomedical named entity recognition. In International Conference on Advances in Computing, Communica tions and Informatics (ICACCI), pp. 1046-1052. IEEE, 2016
2016
-
[41]
Computer methods and programs in biomedicine vol
Keretna Sara, Chee Peng Lim, Doug Creighton and Khaled Bashir Shaban.: Enhancing medical named entity recognition with an extended segment representation technique. Computer methods and programs in biomedicine vol. 119(2), pp. 88-100, (2015)
2015
-
[42]
Kim J -D, Ohta T, Tsuruoka Y and Tateisi Y : Introduction to the Bio - Entity Recognition Task at JNLPBA. In Proc. of the Joint Workshop on Natural Language Processing in Biomedicine and its Application, pp. 70-75, (2004)
2004
-
[43]
South, Shuying Shen and Scott L
Özlem Uzuner, Brett R. South, Shuying Shen and Scott L. DuVall: 2010 i2b2/va challenge on concepts, assertions and relations in clinical test. Journal of the American Medical Informatics Association, vol. 18(2), pp. 552-556
2010
-
[44]
D., Ohta T., Tateisi Y
Kim J. D., Ohta T., Tateisi Y. and Tsujii, J. I. : GENIA corpus -- a semantically annotated corpus for bio -text mining. Bioinformatics, 19(1), 180- 182, (2003)
2003
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.