An Advanced NLP Framework for Automated Medical Diagnosis with DeBERTa and Dynamic Contextual Positional Gating

Mohammad Ali Labbaf Khaniki; Mohammad Manthouri; Sahabeh Saadati

REVIEW 4 major objections 5 minor 52 references

An Advanced NLP Framework for Automated Medical Diagnosis with DeBERTa and Dynamic Contextual Positional Gating

T0 review · 4 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A pipeline of back-translation, gated DeBERTa, and an attention classifier reports 99.78% accuracy on symptom-to-disease classification, outperforming MCN-BERT.

desk verdict Near-perfect accuracy on a public benchmark, but the missing train/test and augmentation protocol makes the headline claim unverifiable as written. read the letter →

arxiv 2502.07755 v1 pith:INKO5XLB submitted 2025-02-11 cs.CL cs.AI

Mohammad Ali Labbaf Khaniki , Sahabeh Saadati , Mohammad Manthouri This is my paper · ORCID

classification cs.CLcs.AI

keywords medicaldiagnosisnaturallanguageprocessingDeBERTadynamiccontextualpositionalgatingback-translationattentionmechanismSymptom2diseasetextclassification

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes a pipeline for classifying free-text symptom descriptions into 24 diseases: German round-trip back-translation to paraphrase each input, DeBERTa extended with a learned gate that reweights positional information by semantic context, and an attention-based feedforward classifier. On the Symptom2disease dataset, the pipeline reports 99.78% accuracy, 99.72% recall, 99.79% precision, and 99.75% F1-score, beating the MCN-BERT baselines from the same study. The authors attribute the gain to the dynamic contextual positional gating mechanism, which modulates the content-to-position attention term instead of leaving it fixed as in DeBERTa. If the result holds, it means a lightweight modification to a pretrained encoder, plus paraphrase augmentation and attention-focused classification, can bring a small medical text benchmark close to a perfect score.

What carries the argument

The load-bearing object is the dynamic contextual positional gating scalar $\alpha_{i,j}$, a value between 0 and 1 produced by a sigmoid over a learned bilinear interaction between a token's content query and another token's relative-position key. Multiplying this scalar into the content-to-position attention term is what distinguishes the proposed model from DeBERTa; the paper credits this addition for the jump from 99.66% accuracy for BT + DeBERTa + ABFNN to 99.78% for the full pipeline. The classifier, an attention-based feedforward network with two parallel branches, residual connections, and multi-scale integration, sits on top of the gated encoder and selects the features that drive the 24-way disease decision.

What would settle it

Run the same pipeline on Symptom2disease with a documented stratified split, apply back-translation only to the training folds, and report test accuracy; if the number falls materially below 99.78% or moves widely across folds, the headline result depended on test-set contamination rather than on the gating mechanism.

Watch

Extended reading notes

Core claim

The paper's central claim is that the combination of back-translation, DeBERTa with DCPG, and ABFNN outperforms all compared approaches on Symptom2disease, with metrics clustered near 99.8%. The DCPG mechanism changes DeBERTa's disentangled attention by computing a per-pair gate $\alpha_{i,j}=\sigma(\mathbf{Q}^C_i \mathbf{W}_g \mathbf{K}^P_{|i-j|})$ and multiplying it into the content-to-position attention term, so the influence of relative word position depends on the semantics of the two tokens rather than being static. The authors present Table 2, a confusion matrix, and an attention heatmap as evidence, and they frame the improvement specifically against MCN-BERT with AdamW and AdamP and against ablated variants without DCPG or ABFNN.

Load-bearing premise

The paper never specifies where the train/test split occurs relative to back-translation, so the central claim rests on the unstated premise that no paraphrased version of a training text appears in the test set.

Editorial extensions

If this is right

If the reported numbers reproduce under a clean split, Symptom2disease classification is effectively saturated, and the remaining errors concentrate in symptom-overlap pairs such as Common Cold versus Pneumonia and Typhoid versus Urinary Tract Infection, as the paper's confusion matrix indicates.
The DCPG gate is a drop-in reweighting of an existing attention term, so the same mechanism could be applied to other disentangled-attention encoders and to tasks where word order matters in some contexts but not others, such as negation scope or temporal symptom descriptions.
Because the ABFNN adds only attention and residual layers over the encoder's embedding, the classifier is comparatively cheap to train, so the pipeline's cost is dominated by the pretrained encoder and the two MarianMT translation passes used for augmentation.
The reported AUC-ROC of 99.88% leaves little room for further model-side gains on this dataset; the paper's own future-work list points to multilingual data, rare diseases, multi-label output, temporal data, and lightweight architectures as the next sources of improvement.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A reader should treat the 99.78% number as contingent on the unstated split: on a balanced 1,200-example dataset the gap to 100% is only a handful of examples, so fold choice and random seed can move the metric materially.
The contribution of DCPG could be isolated by freezing the gate to 1 and comparing against the learned gate over several seeds; the 0.12-point accuracy gap between the full model and BT + DeBERTa + ABFNN is small enough that seed noise alone could explain it.
A stronger generalization test than an attention heatmap would be a held-out set of clinician-written paraphrases that were not produced by the same back-translation pipeline, which would reveal whether the model learned the disease vocabulary or merely the augmentation style.
The gate's claimed behavior is directly testable with synthetic symptom pairs whose medical meaning depends on order, such as 'rash after fever' versus 'fever after rash'; a context-aware positional gate should produce measurably different attention patterns on those pairs.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The headline result is a 99.78% accuracy on the 1,200-sample Symptom2disease benchmark, but the paper never defines the train/test split or the augmentation protocol, so the number is not yet supported. The DCPG idea is real: a learned sigmoid gate on DeBERTa's content-to-position attention term (Eq. 8). It is a small modification, and the paper says so indirectly by showing the gate adds 0.12% accuracy over DeBERTa+ABFNN. The ABFNN is a standard MLP with self-attention. The writing is mostly clear, and the confusion matrix and attention map are useful sanity checks.

The soft spots are proportional. The missing split details are load-bearing. Section 3.4 lists back-translation as stage 2 and data collection/preprocessing as stage 3, before any split is mentioned. If the whole corpus is back-translated before splitting, paraphrases of training texts can appear in the test set, inflating all metrics. The paper also reports accuracy to two decimal places, yet with 1,200 samples and a standard 80/20 or 90/10 split, 99.78% is not reachable (240 test samples allow 99.58 or 99.67; 120 allow 99.17 or 100). So either the split is nonstandard (e.g., 40% test gives 479/480 = 99.79) or the metric is computed on an augmented corpus, and neither is disclosed. No seeds, runs, or error bars are given. The comparison baseline is taken from [11] rather than reproduced, so the improvement over MCN-BERT could be partly due to differences in splits and augmentation. The paper does not ship code.

If the authors supply the exact split, confirm that back-translation is confined to the training fold, and release the script, the claims are checkable on the public dataset. As it stands, the paper is a plausible but unverified data point. It is not a breakthrough and does not organize the field, but it is not nonsense either. A serious referee could request the missing protocol and the paper might be salvageable after major revision. I would not cite it in my own work without the protocol, and I would not bring it to a reading group except as a cautionary example of benchmark hygiene. But I would not desk-reject it: the method is concrete, the dataset is public, and the authors have to answer for their numbers.

Referee Report

4 major / 5 minor

Summary. The paper proposes an NLP pipeline for 24-class disease classification from symptom descriptions, combining MarianMT-based back-translation (English-German-English), DeBERTa with a proposed Dynamic Contextual Positional Gating (DCPG) mechanism, and an Attention-Based Feedforward Neural Network (ABFNN) classifier. On the Symptom2disease dataset (1200 samples, 24 classes) the paper reports 99.78% accuracy, 99.72% recall, 99.79% precision, 99.75% F1-score, and 99.88% AUC-ROC, claiming superiority over MCN-BERT and other baselines. The main evidence is Table 2, a confusion matrix, and a single attention heatmap; no train/test split, training hyperparameters, seeds, number of runs, or error bars are reported.

Significance. If the reported results were reproducible and leakage-free, the paper would provide a useful benchmark on a small medical text classification task and a modest architectural variant of DeBERTa. The back-translation models are clearly specified, the equations for DCPG and ABFNN are explicit, and the comparison includes external baselines from Hassan et al. [11]. The proposed DCPG gate is a reasonable, lightweight addition to disentangled attention, and the ABFNN is described with enough detail to reimplement. However, the central empirical claim is currently unsupported because the evaluation protocol is underspecified and the reported accuracy is arithmetically inconsistent with standard splits of the 1200-sample dataset. The paper does not ship code, seeds, or run statistics; its strength is the clarity of the proposed architecture, not yet its demonstrated performance.

major comments (4)

[Section 3.4] The pipeline applies back-translation to the input texts before any train/test split is defined, and no sentence restricts augmented paraphrases to the training fold. If the full 1200-example corpus is back-translated and then split, near-duplicate paraphrases of training texts can appear in the test set, making the reported 99.78% accuracy a measure of paraphrase retrieval rather than generalization. The paper must state the exact split (seed, sizes, stratification), whether augmentation was applied only to the training fold, and how many unique original texts appear in the test set.
[Section 4, Table 2] No number of runs, seeds, learning rate, batch size, epochs, optimizer, or max sequence length are reported, so the two-decimal metrics cannot be interpreted. Moreover, with the standard 80/20 or 90/10 split of the 1200-sample Symptom2disease corpus, 99.78% is not an attainable accuracy: 240 test samples allow 99.58% or 100%, and 300 test samples allow 99.67% or 100%. The reported value therefore must arise from a nonstandard or augmented evaluation set; the paper must report the exact test-set size and demonstrate that no augmented paraphrase of a training text appears in the test set.
[Section 3.2.4, Eqs. (8)-(10)] The central architectural contribution is the gating scalar alpha_{i,j}, but the only evidence for it is the comparison in Table 2 between BT + DeBERTa + ABFNN (99.66%) and BT + DeBERTa-DCPG + ABFNN (99.78%). On a 1200-sample dataset, a 0.12-point accuracy gap can be one or two examples, and without error bars, seeds, or significance testing the claim that DCPG 'outperforms' DeBERTa is not supported. The authors should report multiple runs with variance, per-class performance, and a statistical test, or explicitly concede that the DCPG contribution is not statistically separable on this benchmark.
[Section 4.4, Eqs. (19)-(23)] The evaluation metrics are defined only for binary classification, but the task is 24-class classification. The paper must state whether accuracy, precision, recall, and F1 are macro-averaged, micro-averaged, or weighted across the 24 classes, and how TP/FP/TN/FN are aggregated. This matters for interpreting the reported 99.72% recall and 99.79% precision, and it is load-bearing for the central empirical claim.

minor comments (5)

[Section 4.1] The accuracy equation is numbered (17), duplicating the equation number already used for the ABFNN multi-scale integration in Section 3.3; renumber the equations.
[Table 2 and Figure 3] The model names are inconsistent: Table 2 uses 'BT + DeBERTa - DCPG + ABFNN' while Figure 3 uses 'BT + DeBERTa + DCPG + ABFNN'; clarify whether the hyphen denotes 'with' or 'minus'.
[Author affiliations and Section 3.2.4] The author affiliation contains a duplicated word 'Department Department', and 'Gating Scaler' should be 'Gating Scalar' for consistency with the mathematical definition of alpha_{i,j}.
[Section 4.3] The attention heatmap is a single qualitative example; the claims that DCPG 'suppresses' or 'amplifies' positional bias for specific terms should be supported by quantitative gate-value statistics across examples, not by one sentence.
[References [33], [34]] The MarianMT models are cited without version identifiers or download dates; for reproducibility, specify the exact model revisions and the back-translation implementation details.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the DCPG and ABFNN components are defined by explicit model equations and evaluated empirically against an external benchmark, so the accuracy claim is a measured result rather than a derived prediction.

full rationale

The paper's central claim (99.78% accuracy on Symptom2disease) is an empirical evaluation result, not a quantity derived from first principles or from fitted parameters disguised as predictions. The DCPG mechanism is defined by explicit model equations (8)-(10): alpha_{i,j} = sigma(Q_{C,i} W_g K_{P,|i-j|}^T), A~_{i,j}^{(cp)} = alpha_{i,j} A_{i,j}^{(cp)}, and A_{i,j} = (A_{i,j}^{(cc)} + A~_{i,j}^{(cp)} + A_{i,j}^{(pc)}) / sqrt(3d). These are construction choices, not circular derivations. The ABFNN is likewise specified by equations (13)-(18). No fitted parameter is renamed as a prediction; the reported metrics are measured outcomes of a trained pipeline. The comparison baseline MCN-BERT is external [11], and no load-bearing argument reduces to a self-citation or to a uniqueness theorem by the authors. Although Section 3.4 lists back-translation before defining the train/test split and omits the exact evaluation-set size, that is a methodological reproducibility concern rather than equation-level circularity, and no quote in the paper establishes that augmented paraphrases of training texts enter the test set. Under the specified circularity criteria, the derivation chain is self-contained.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

No new physical or symbolic entities are postulated. The only additions are learned parameters and a gating formula, which are listed as free parameters. The central assumptions are the correctness and representativeness of the dataset, the label-preserving property of back-translation, and the unstated disjointness of train and test sets after augmentation.

free parameters (5)

DCPG gate matrix W_g = learned, values not reported
This matrix defines the gating scalar alpha_{i,j} in Eq. 8 and is central to the claimed improvement over DeBERTa.
ABFNN weight matrices = learned, values not reported
Classifier weights W1, W2, Wr, Wm, and Wout are fit to the data; only the layer equations are given.
Relative distance clipping kmax = not reported
The maximum relative positional distance in Eq. 2 is a design choice that controls position encoding capacity.
Back-translation language pair = English to German and back
The choice of intermediate language affects the paraphrases and therefore the augmented training distribution.
Training hyperparameters = not reported
Epochs, learning rate, batch size, optimizer, and split ratio are not given, so all results depend on unspecified settings.

assumptions (5)

domain assumption Symptom2disease labels are correct and symptom descriptions are representative of the 24 diseases
The entire evaluation treats the Kaggle dataset labels as ground truth (Section 2.1).
domain assumption Back-translation preserves the disease label and semantic meaning
Section 3.1 asserts semantic preservation is critical and relies on MarianMT to provide it, but no human or automated verification is reported.
domain assumption Train and test sets are disjoint after augmentation
Section 3.4 describes augmenting the input texts without specifying that augmented samples are confined to the training fold, so metric validity depends on this unstated assumption.
ad hoc to paper The gating scalar adds useful capacity beyond DeBERTa's existing positional attention
Eq. 8 is presented as a contribution, but no analysis shows that it is not redundant with the learned relative position projections already in DeBERTa.
standard math Standard transformer training and evaluation assumptions
Softmax, backpropagation, and DeBERTa's pretrained weights are used as black boxes (Sections 3.2 and 3.3).

how reviews work

0 comments

Cite this review

Pith. "Pith review of An Advanced NLP Framework for Automated Medical Diagnosis with DeBERTa and Dynamic Contextual Positional Gating." pith.science (2026). https://pith.science/paper/INKO5XLB

@misc{pith2026250207755,
  author       = {Pith},
  title        = {Pith review of: An Advanced NLP Framework for Automated Medical Diagnosis with DeBERTa and Dynamic Contextual Positional Gating},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/INKO5XLB}},
  note         = {Machine review of arXiv:2502.07755}
}

read the original abstract

This paper presents a novel Natural Language Processing (NLP) framework for enhancing medical diagnosis through the integration of advanced techniques in data augmentation, feature extraction, and classification. The proposed approach employs back-translation to generate diverse paraphrased datasets, improving robustness and mitigating overfitting in classification tasks. Leveraging Decoding-enhanced BERT with Disentangled Attention (DeBERTa) with Dynamic Contextual Positional Gating (DCPG), the model captures fine-grained contextual and positional relationships, dynamically adjusting the influence of positional information based on semantic context to produce high-quality text embeddings. For classification, an Attention-Based Feedforward Neural Network (ABFNN) is utilized, effectively focusing on the most relevant features to improve decision-making accuracy. Applied to the classification of symptoms, clinical notes, and other medical texts, this architecture demonstrates its ability to address the complexities of medical data. The combination of data augmentation, contextual embedding generation, and advanced classification mechanisms offers a robust and accurate diagnostic tool, with potential applications in automated medical diagnosis and clinical decision support. This method demonstrates the effectiveness of the proposed NLP framework for medical diagnosis, achieving remarkable results with an accuracy of 99.78%, recall of 99.72%, precision of 99.79%, and an F1-score of 99.75%. These metrics not only underscore the model's robust performance in classifying medical texts with exceptional precision and reliability but also highlight its superiority over existing methods, making it a highly promising tool for automated diagnostic systems.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

52 extracted references · 50 canonical work pages

[11]

Output Layer: 𝑦 = softmax(𝑊𝑜𝑢𝑡ℎ𝑚𝑢𝑙𝑡𝑖−𝑠𝑐𝑎𝑙𝑒 + 𝑏𝑜𝑢𝑡) (18) This design enhances feature representation and classification, making it suitable for complex datasets, such as outputs from models like BERT. The Figure 1 illustrates the architecture of the ABFNN, highlighting its integrated attention mechanism designed to focus on the most relevant features for i...

work page
[1]

Introduction Accurate and timely medical diagnosis is essential for effective healthcare, directly impacting patient outcomes and treatment quality. Delays or errors in diagnosis, especially for critical conditions like sepsis, cancer, or myocardial infarction, can worsen prognoses and lead to unnecessary treatments or fatalities. Traditional diagnostic m...

work page 2017
[2]

This process involves translating text from English to German and subsequently re-translating it back to English, thereby generating paraphrased versions of the original data

Back-Translation for Data Augmentation: The model utilizes a back -translation technique to augment data diversity. This process involves translating text from English to German and subsequently re-translating it back to English, thereby generating paraphrased versions of the original data. This appr oach aids in mitigating overfitting and enhances the ro...

work page
[3]

This allows the model to generate high-quality embeddings that are crucial for downstream classification tasks

DeBERTa with Dynamic Contextual Positional Gating (DCPG): The DeBERTa model serves as the backbone for feature extraction, leveraging its disentangled attention mechanism and enhanced decoding to capture fine -grained contextual and positional relationships in the input text. This allows the model to generate high-quality embeddings that are crucial for d...

work page
[4]

Attention-Based Feedforward Neural Network (ABFNN) for Classification: The ABFNN serves as the classifier within the proposed pipeline. This innovative architecture integrates attention mechanisms with a feedforward neural network, enabling the model to selectively focus on the most pertinent features of the input data. By effect ively leveraging the hier...

work page
[5]

Additionally, a comprehensive literature review is presented, summarizing key studies and methodologies in the field of medical diagnosis using NLP techniques

Background and Related Work This section provides an overview of the medical diagnosis dataset, detailing its structure, features, and relevance to the task of disease classification. Additionally, a comprehensive literature review is presented, summarizing key studies and methodologies in the field of medical diagnosis using NLP techniques. 2.1. Medical ...

work page
[6]

bank" near

Methodology In this section, we outline the methodologies employed in the proposed framework for enhancing medical diagnosis using NLP. The approach integrates three key techniques: back - translation for data augmentation, DeBERTa with DCPG for feature extraction, and an ABFNN for classification. Each of these components plays a crucial role in improving...

work page
[7]

Retains DeBERTa's three terms but adaptively reweights C→P via a learned gate 𝛼𝑖,𝑗

P→C. Retains DeBERTa's three terms but adaptively reweights C→P via a learned gate 𝛼𝑖,𝑗. Positional Encoding Absolute positional embeddings. Relative positional embeddings (distance-based). Relative positional embeddings + dynamic gating. Gating Mechanism None. None. Introduces 𝛼𝑖,𝑗 = 𝜎 (𝑸𝐶𝑖𝑾𝑔𝐾𝑃|i−j| 𝑇 ). Attention Score 𝐴𝑖,𝑗 = (𝑐𝑖 + 𝑝𝑖)𝑾𝑄(𝑐𝑖 + 𝑝𝑖)𝑊𝑘 𝑇 √𝑑...

work page

Show all 52 references

[8]

Hierarchical Feature Extraction: Two branches extract diverse features: ℎ𝑏𝑟𝑎𝑛𝑐ℎ1 = ReLU(𝑊1𝑥 + 𝑏1), ℎ𝑏𝑟𝑎𝑛𝑐ℎ2 = ReLU(𝑊2𝑥 + 𝑏2) (13) ℎ𝑐𝑜𝑚𝑏𝑖𝑛𝑒𝑑 = ℎ𝑏𝑟𝑎𝑛𝑐ℎ2 + ℎ𝑏𝑟𝑎𝑛𝑐ℎ1 (14)
[9]

(15) with 𝑄, 𝐾, 𝑉 = 𝑊𝑄ℎ𝑐𝑜𝑚𝑏𝑖𝑛𝑒𝑑, 𝑊𝐾ℎ𝑐𝑜𝑚𝑏𝑖𝑛𝑒𝑑, 𝑊𝑉ℎ𝑐𝑜𝑚𝑏𝑖𝑛𝑒𝑑

Self-Attention Mechanism: Focuses on important features: Attention(𝑄, 𝐾, 𝑉) = Softmax (𝑄𝐾𝑇 √𝑑𝑘 ) × 𝑉. (15) with 𝑄, 𝐾, 𝑉 = 𝑊𝑄ℎ𝑐𝑜𝑚𝑏𝑖𝑛𝑒𝑑, 𝑊𝐾ℎ𝑐𝑜𝑚𝑏𝑖𝑛𝑒𝑑, 𝑊𝑉ℎ𝑐𝑜𝑚𝑏𝑖𝑛𝑒𝑑
[10]

Residual Connection and Multi-Scale Integration: ℎ𝑟𝑒𝑠𝑖𝑑𝑢𝑎𝑙 = ReLU(𝑊𝑟Attention + 𝑏𝑟) + Attention (16) ℎ𝑚𝑢𝑙𝑡𝑖−𝑠𝑐𝑎𝑙𝑒 = ReLU(𝑊𝑚[ℎ𝑟𝑒𝑠𝑖𝑑𝑢𝑎𝑙, ℎ𝑐𝑜𝑚𝑏𝑖𝑛𝑒𝑑] + 𝑏𝑚) (17)
[12]

Symptom Descriptions: Patients’ symptoms are provided in textual form, serving as the primary input for the diagnosis model
[13]

This introduces lexical variety while preserving semantic meaning, improving the model's ability to generalize

Back-Translation for Data Augmentation : To enhance the diversity and robustness of the dataset, the input texts are translated to German and back to English using the MarianMT model. This introduces lexical variety while preserving semantic meaning, improving the model's abil...
[14]

Data Collection and Preprocessing : The collected symptom descriptions are cleaned, tokenized, and prepared for input into the model, ensuring high -quality and standardized data for training
[15]

DeBERTa with DCPG : The input data is processed using the DeBERTa model, which employs disentangled attention and positional gating to capture contextual relationships and subtle nuances in the text for accurate feature representation
[16]

The schematic representation of the proposed method is illustrated in Figure 2

ABFNN: The output embeddings from DeBERTa are fed into the ABFNN, which uses an integrated attention mechanism to focus on the most relevant features and improve classification accuracy for medical diagnosis. The schematic representation of the proposed method is illustrated i...
[17]

fever for 5 days

Simulations This section presents a comprehensive evaluation of the performance of several deep learning models, including the BERT, DeBERTa, and the methods which is presented in [11]. we utilized the model with the following hyper parameters: • Number of Layers: 12 layers in...
[18]

Our innovative approach combines Back -Translation techniques, an enhanced version of DeBERTa featuring DCPG , and an ABFNN

Conclusion This research introduces a groundbreaking NLP architecture that improved automated medical diagnosis. Our innovative approach combines Back -Translation techniques, an enhanced version of DeBERTa featuring DCPG , and an ABFNN. This synergistic integration results in...
[19]

A systematic review of natural language processing in healthcare,

O. G. Iroju and J. O. Olaleke, “A systematic review of natural language processing in healthcare,” Int. J. Inf. Technol. Comput. Sci., vol. 8, no. 8, pp. 44–50, 2015

2015
[20]

Desiderata for delivering NLP to accelerate healthcare AI advancement and a Mayo Clinic NLP-as-a-service implementation,

A. Wen et al., “Desiderata for delivering NLP to accelerate healthcare AI advancement and a Mayo Clinic NLP-as-a-service implementation,” NPJ Digit. Med., vol. 2, no. 1, p. 130, 2019

2019
[21]

Deep learning techniques on text classification using Natural language processing (NLP) in social healthcare network: A comprehensive survey,

P. M. Lavanya and E. Sasikala, “Deep learning techniques on text classification using Natural language processing (NLP) in social healthcare network: A comprehensive survey,” in 2021 3rd international conference on signal processing and communication (ICPSC), IEEE, 2021, pp. 603–609

2021
[22]

Attention is all you need,

A. Vaswani et al., “Attention is all you need,” Adv. Neural Inf. Process. Syst., vol. 30, 2017

2017
[23]

Comparison of DEEP- LSTM and MLP Models in Estimation of Evaporation Pan for Arid Regions.,

A. Samii, H. Karami, H. Ghazvinian, A. Safari, and Y. D. Ajirlou, “Comparison of DEEP- LSTM and MLP Models in Estimation of Evaporation Pan for Arid Regions.,” J. Soft Comput. Civ. Eng., vol. 7, no. 2, 2023

2023
[24]

Predicting clinical diagnosis from patients electronic health records using BERT-based neural networks,

P. Blinov, M. Avetisian, V. Kokh, D. Umerenkov, and A. Tuzhilin, “Predicting clinical diagnosis from patients electronic health records using BERT-based neural networks,” in Artificial Intelligence in Medicine: 18th International Conference on Artificial Intelligence in Medici...

2020
[25]

Entity-enhanced BERT for medical specialty prediction based on clinical questionnaire data,

S. Lee et al., “Entity-enhanced BERT for medical specialty prediction based on clinical questionnaire data,” PLoS One, vol. 20, no. 1, p. e0317795, 2025

2025
[26]

GPT-4 assistance for improvement of physician performance on patient care tasks: a randomized controlled trial,

E. Goh et al., “GPT-4 assistance for improvement of physician performance on patient care tasks: a randomized controlled trial,” Nat. Med., pp. 1–6, 2025

2025
[27]

Auggpt: Leveraging chatgpt for text data augmentation,

H. Dai et al., “Auggpt: Leveraging chatgpt for text data augmentation,” IEEE Trans. Big Data, 2025

2025
[28]

Diffusion data augmentation for enhancing Norberg hip angle estimation,

S. Yueh, F. Higgins, Z. Lin, R. J. Todhunter, and Y. Zhang, “Diffusion data augmentation for enhancing Norberg hip angle estimation,” Vet. Radiol. Ultrasound, vol. 66, no. 1, p. e13463, 2025

2025
[29]

Optimizing classification of diseases through language model analysis of symptoms,

E. Hassan, T. Abd El-Hafeez, and M. Y. Shams, “Optimizing classification of diseases through language model analysis of symptoms,” Sci. Rep., vol. 14, no. 1, pp. 1–24, 2024, doi: 10.1038/s41598-024-51615-5

2024 doi
[30]

Advancements in AI-Driven Customer Service,

M. Esmaeili, M. Ahmadi, M. D. Ismaeil, S. Mirzaei, and J. Canales Verdial, “Advancements in AI-Driven Customer Service,” in 2024 IEEE World AI IoT Congress (AIIoT), IEEE, 2024, pp. 1–5

2024
[31]

A machine learning-based prediction of hospital mortality in mechanically ventilated ICU patients,

H. Li, N. Ashrafi, C. Kang, G. Zhao, Y. Chen, and M. Pishgar, “A machine learning-based prediction of hospital mortality in mechanically ventilated ICU patients,” PLoS One, vol. 19, no. 9, p. e0309383, 2024

2024
[32]

A Multi-Sensor Integrated with Augmented Reality System for Precise Nursing Education and Analysis,

K. Sepanloo et al., “A Multi-Sensor Integrated with Augmented Reality System for Precise Nursing Education and Analysis,” in IISE Annual Conference. Proceedings, Institute of Industrial and Systems Engineers (IISE), 2023, pp. 1–6

2023
[33]

COMPARISON OF SEVERAL MACHINE LEARNING METHODS IN CREDIT CARD DEFAULT CLASSIFICATION,

M. Khodayari Gharanchaei, “COMPARISON OF SEVERAL MACHINE LEARNING METHODS IN CREDIT CARD DEFAULT CLASSIFICATION,” J. Strateg. Int. Stud., vol. 18, no. 1, pp. 24–31, 2024

2024
[34]

Exploring Eye Tracking to Detect Cognitive Load in Complex Virtual Reality Training,

M. Nasri, M. Kosa, L. Chukoskie, M. Moghaddam, and C. Harteveld, “Exploring Eye Tracking to Detect Cognitive Load in Complex Virtual Reality Training,” in 2024 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), 2024, pp. 51–54. doi: 10.1109/IS...

2024
[35]

Safe Navigation in Unmapped Environments for Robotic Systems with Input Constraints,

A. Safari and J. B. Hoagg, “Safe Navigation in Unmapped Environments for Robotic Systems with Input Constraints,” arXiv Prepr. arXiv2410.02106, 2024

2024 arXiv
[36]

Evaluation of LLM-based chatbots for OSINT- based Cyber Threat Awareness,

S. Shafee, A. Bessani, and P. M. Ferreira, “Evaluation of LLM-based chatbots for OSINT- based Cyber Threat Awareness,” Expert Syst. Appl., p. 125509, 2024

2024
[37]

Mri brain cancer image detection: Application of an integrated u-net and resnet50 architecture,

M. Benchari and M. W. Totaro, “Mri brain cancer image detection: Application of an integrated u-net and resnet50 architecture,” in International Conference on Artificial Intelligence in Medicine, Springer, 2024, pp. 104–108

2024
[38]

From natural language to sql: Review of llm-based text-to-sql systems,

A. Mohammadjafari, A. S. Maida, and R. Gottumukkala, “From natural language to sql: Review of llm-based text-to-sql systems,” arXiv Prepr. arXiv2410.01066, 2024

2024 arXiv
[39]

Transportation mode detection through spatial attention-based transductive long short-term memory and off- policy feature selection,

M. Merikhipour, S. Khanmohammadidoustani, and M. Abbasi, “Transportation mode detection through spatial attention-based transductive long short-term memory and off- policy feature selection,” Expert Syst. Appl., vol. 267, p. 126196, 2025

2025
[40]

Comparative analysis of diabetes diagnosis: WE-LSTM networks and WizardLM-powered DiabeTalk chatbot,

D. Rossi, A. A. Citarella, F. De Marco, L. Di Biasi, and G. Tortora, “Comparative analysis of diabetes diagnosis: WE-LSTM networks and WizardLM-powered DiabeTalk chatbot,” in 2024 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), IEEE, 2024, pp. 6859–6866

2024
[41]

Graph attention-based curriculum learning for mental healthcare classification,

U. Ahmed, J. C.-W. Lin, and G. Srivastava, “Graph attention-based curriculum learning for mental healthcare classification,” IEEE J. Biomed. Heal. Informatics, 2023

2023
[42]

Predicting medical specialty from text based on a domain-specific pre-trained BERT,

Y. Kim, J.-H. Kim, Y.-M. Kim, S. Song, and H. J. Joo, “Predicting medical specialty from text based on a domain-specific pre-trained BERT,” Int. J. Med. Inform., vol. 170, p. 104956, 2023

2023
[43]

Using the contextual language model BERT for multi-criteria classification of scientific articles,

A. K. Ambalavanan and M. V Devarakonda, “Using the contextual language model BERT for multi-criteria classification of scientific articles,” J. Biomed. Inform., vol. 112, p. 103578, 2020

2020
[44]

A role distinguishing Bert model for medical dialogue system in sustainable smart city,

S. Wang, S. Wang, Z. Liu, and Q. Zhang, “A role distinguishing Bert model for medical dialogue system in sustainable smart city,” Sustain. Energy Technol. Assessments, vol. 55, p. 102896, 2023

2023
[45]

Automatic symptoms identification from a massive volume of unstructured medical consultations using deep neural and BERT models,

H. Faris, M. Faris, M. Habib, and A. Alomari, “Automatic symptoms identification from a massive volume of unstructured medical consultations using deep neural and BERT models,” Heliyon, vol. 8, no. 6, 2022

2022
[46]

Biomedical Natural Language Inference on Clinical trials using the BERT-based Models,

A. Seerat, S. Nasir, M. Wasim, and N. M. Garcia, “Biomedical Natural Language Inference on Clinical trials using the BERT-based Models,” Procedia Comput. Sci., vol. 241, pp. 576–581, 2024

2024
[47]

DeBERTa-BiLSTM: A multi-label classification model of Arabic medical questions using pre-trained models and deep learning,

B. S. Al-Smadi, “DeBERTa-BiLSTM: A multi-label classification model of Arabic medical questions using pre-trained models and deep learning,” Comput. Biol. Med., vol. 170, p. 107921, 2024

2024
[48]

Role of natural language processing in automatic detection of unexpected findings in radiology reports: a comparative study of RoBERTa, CNN, and ChatGPT,

P. López-Úbeda, T. Martín-Noguerol, J. Escartín, and A. Luna, “Role of natural language processing in automatic detection of unexpected findings in radiology reports: a comparative study of RoBERTa, CNN, and ChatGPT,” Acad. Radiol., vol. 31, no. 12, pp. 4833–4842, 2024

2024
[49]

A cross attention approach to diagnostic explainability using clinical practice guidelines for depression,

S. Dalal, D. Tilwani, M. Gaur, S. Jain, V. L. Shalin, and A. P. Sheth, “A cross attention approach to diagnostic explainability using clinical practice guidelines for depression,” IEEE J. Biomed. Heal. Informatics, 2024

2024
[50]

Automatic Medical Report Generation Based on Detector Attention Module and GPT-Based Word LSTM,

Y. Gu, J. Sun, X. Wang, R. Li, and Z. Zhou, “Automatic Medical Report Generation Based on Detector Attention Module and GPT-Based Word LSTM,” in 2024 6th International Conference on Data-driven Optimization of Complex Systems (DOCS), IEEE, 2024, pp. 7–15

2024
[51]

Grammatical versus Spelling Error Correction: An Investigation into the Responsiveness of Transformer-Based Language Models Using BART and MarianMT,

R. Raju, P. B. Pati, S. A. Gandheesh, G. S. Sannala, and K. S. Suriya, “Grammatical versus Spelling Error Correction: An Investigation into the Responsiveness of Transformer-Based Language Models Using BART and MarianMT,” J. Inf. Knowl. Manag., p. 2450037, 2024

2024
[52]

Comparative Analysis of Neural Translation Models based on Transformers Architecture.,

A. V Smirnov, N. Teslya, N. Shilov, D. Frank, E. Minina, and M. Kovacs, “Comparative Analysis of Neural Translation Models based on Transformers Architecture.,” in ICEIS (1), 2022, pp. 586–593

2022

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.

[1] [11]

Output Layer: 𝑦 = softmax(𝑊𝑜𝑢𝑡ℎ𝑚𝑢𝑙𝑡𝑖−𝑠𝑐𝑎𝑙𝑒 + 𝑏𝑜𝑢𝑡) (18) This design enhances feature representation and classification, making it suitable for complex datasets, such as outputs from models like BERT. The Figure 1 illustrates the architecture of the ABFNN, highlighting its integrated attention mechanism designed to focus on the most relevant features for i...

work page

[2] [1]

Introduction Accurate and timely medical diagnosis is essential for effective healthcare, directly impacting patient outcomes and treatment quality. Delays or errors in diagnosis, especially for critical conditions like sepsis, cancer, or myocardial infarction, can worsen prognoses and lead to unnecessary treatments or fatalities. Traditional diagnostic m...

work page 2017

[3] [2]

This process involves translating text from English to German and subsequently re-translating it back to English, thereby generating paraphrased versions of the original data

Back-Translation for Data Augmentation: The model utilizes a back -translation technique to augment data diversity. This process involves translating text from English to German and subsequently re-translating it back to English, thereby generating paraphrased versions of the original data. This appr oach aids in mitigating overfitting and enhances the ro...

work page

[4] [3]

This allows the model to generate high-quality embeddings that are crucial for downstream classification tasks

DeBERTa with Dynamic Contextual Positional Gating (DCPG): The DeBERTa model serves as the backbone for feature extraction, leveraging its disentangled attention mechanism and enhanced decoding to capture fine -grained contextual and positional relationships in the input text. This allows the model to generate high-quality embeddings that are crucial for d...

work page

[5] [4]

Attention-Based Feedforward Neural Network (ABFNN) for Classification: The ABFNN serves as the classifier within the proposed pipeline. This innovative architecture integrates attention mechanisms with a feedforward neural network, enabling the model to selectively focus on the most pertinent features of the input data. By effect ively leveraging the hier...

work page

[6] [5]

Additionally, a comprehensive literature review is presented, summarizing key studies and methodologies in the field of medical diagnosis using NLP techniques

Background and Related Work This section provides an overview of the medical diagnosis dataset, detailing its structure, features, and relevance to the task of disease classification. Additionally, a comprehensive literature review is presented, summarizing key studies and methodologies in the field of medical diagnosis using NLP techniques. 2.1. Medical ...

work page

[7] [6]

bank" near

Methodology In this section, we outline the methodologies employed in the proposed framework for enhancing medical diagnosis using NLP. The approach integrates three key techniques: back - translation for data augmentation, DeBERTa with DCPG for feature extraction, and an ABFNN for classification. Each of these components plays a crucial role in improving...

work page

[8] [7]

Retains DeBERTa's three terms but adaptively reweights C→P via a learned gate 𝛼𝑖,𝑗

P→C. Retains DeBERTa's three terms but adaptively reweights C→P via a learned gate 𝛼𝑖,𝑗. Positional Encoding Absolute positional embeddings. Relative positional embeddings (distance-based). Relative positional embeddings + dynamic gating. Gating Mechanism None. None. Introduces 𝛼𝑖,𝑗 = 𝜎 (𝑸𝐶𝑖𝑾𝑔𝐾𝑃|i−j| 𝑇 ). Attention Score 𝐴𝑖,𝑗 = (𝑐𝑖 + 𝑝𝑖)𝑾𝑄(𝑐𝑖 + 𝑝𝑖)𝑊𝑘 𝑇 √𝑑...

work page