Enhancing Plagiarism Detection in Marathi with a Weighted Ensemble of TF-IDF and BERT Embeddings for Low-Resource Language Processing

Aditya Choudhary; Atharva Mutsaddi

REVIEW 3 major objections 5 minor 4 references

Enhancing Plagiarism Detection in Marathi with a Weighted Ensemble of TF-IDF and BERT Embeddings for Low-Resource Language Processing

T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A weighted ensemble of TF-IDF vectors and BERT embeddings achieves 82.04% accuracy on Marathi plagiarism detection, outperforming BERT-only (80.64%) and TF-IDF-only (58.68%) models.

desk verdict Useful Marathi dataset resource, but the headline accuracy gain is very likely a test-set tuning artifact and the label preservation of the translation is unvalidated. read the letter →

arxiv 2501.05260 v1 pith:RACX2KG4 submitted 2025-01-09 cs.CL cs.AIcs.LG

Atharva Mutsaddi , Aditya Choudhary This is my paper · ORCID

classification cs.CLcs.AIcs.LG MSC 68T50

keywords Marathiplagiarismdetectionlow-resourcelanguageBERTembeddingsTF-IDFweightedensembleparaphrasemachinetranslationsemanticsimilarity

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that for Marathi, a low-resource language, a weighted ensemble combining TF-IDF vectors and BERT sentence embeddings detects plagiarism more accurately than either representation alone. The proposed system reaches 82.04% accuracy on a machine-translated Marathi dataset, versus 80.64% for BERT-only embeddings and 58.68% for TF-IDF only. This matters because most existing Marathi plagiarism detection relies on similarity heuristics or English-centric models, which miss paraphrase structure in under-resourced languages. The paper also contributes a translated Marathi paraphrase corpus built from the MIT Plagiarism Detection Dataset.

What carries the argument

The weighted voting ensemble is the load-bearing mechanism. Each classifier outputs a plagiarism probability; the BERT-set probability is the weighted average of XGBoost (weight 0.7) and SVC (weight 0.3) predictions, and the TF-IDF-set probability is the weighted average of Logistic Regression (weight 0.1) and LightGBM (weight 0.9) predictions. The final probability is P = 0.6*PBERT + 0.4*PTF-IDF, and a pair is flagged as plagiarized when P > 0.5. The pair-wise representation is created by element-wise subtraction of the reference embedding from the input embedding, which forces the classifier to learn relationship features rather than absolute text content.

What would settle it

Have native Marathi speakers label a random sample of 1,000 pairs from the translated dataset, then compare the ensemble's predictions to those labels; if accuracy is below 82% on that human-validated subset, the reported result is inflated by translation label noise.

Watch

Extended reading notes

Core claim

The paper's central discovery is that the combination of statistical and semantic text representations improves plagiarism detection in Marathi. Using element-wise subtraction of MahaSBERT-STS embeddings (768 dimensions) and TF-IDF vectors (400 dimensions) for each reference-input pair, the authors train four classifiers: XGBoost and SVC on BERT features, Logistic Regression and LightGBM on TF-IDF features. A weighted vote with WBERT=0.6 and WTF-IDF=0.4 yields 82.04% accuracy, 85.32% recall, and an F1 of 82.69%, outperforming every single model and representation. The result is presented as evidence that hybrid representations compensate for the weak semantic coverage of BERT models in low-resource languages.

Load-bearing premise

The machine-translated Marathi dataset preserves the original English plagiarism labels, meaning translation errors do not silently change whether a test pair should be considered plagiarized.

Editorial extensions

If this is right

If accurate, the 82.04% result shows that TF-IDF's statistical signal complements BERT's semantic signal for Marathi, so hybrid representations should be preferred in low-resource plagiarism detection.
The translated MIT Plagiarism Detection dataset becomes a reusable benchmark for Marathi paraphrase and plagiarism research, enabling direct comparison of future methods.
The same weighted ensemble configuration (0.6 BERT, 0.4 TF-IDF) is a candidate starting point for other low-resource languages with a sentence-BERT model.
The system's high recall (85.32%) suggests it is better at catching true plagiarized pairs than BERT alone, which matters for screening large text collections.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

We infer that the 1.4-point accuracy gap between the ensemble and BERT-only sits within the range that machine-translation label noise could produce; a human-validated test set is needed to confirm the hybrid's advantage.
We infer that element-wise subtraction may underuse the embeddings; concatenating difference and product features, or using attention-based pair modeling, could increase the gap further.
We infer that the approach extends naturally to other low-resource Indian languages (e.g., Hindi, Gujarati) that have SBERT variants, provided a labeled paraphrase corpus is translated with similar care.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: this paper is a useful resource contribution wrapped around a central accuracy claim that does not hold up. The new Marathi translation of the MIT plagiarism detection dataset, with code released and a first application of MahaSBERT-STS to the task, is genuinely of interest to the low-resource NLP crowd. The authors also do some things right: they compare two translation models, evaluate on a translated MRPC set, and include an honest limitations section.

The problem is that the headline 82.04% vs 80.64% gain is very likely a selection artifact. Section 3.4 says the model combinations and weights were "iteratively refined to achieve optimal performance," and Section 4.1 reports the winning configuration as the one that "achieved optimal results" — both on the 20% test split. With no validation set, no nested CV, and no error bars, the extra degrees of freedom in the ensemble (which classifiers, per-classifier weights, WBERT/WTF-IDF) mean the reported peak at WBERT=0.6 is fitted, not predicted. A single 80/20 split cannot support a 1.4-point claim under those conditions. This is load-bearing, not a minor quibble: the paper's main contribution, the hybrid ensemble recipe, is not actually demonstrated.

The second soft spot is the translated data. BERTScore and TransQuest tell you translation quality, not whether the paraphrase relation survived translation. Systematic translator errors would add label noise to both training and testing, and the same translator is used for the MRPC validation set, so that evaluation shares the noise. The authors acknowledge the lack of standardized Marathi datasets, but they don't validate label preservation at all.

What is genuinely new: the dataset itself and the exploration of MahaSBERT-STS for this task. The ensemble of TF-IDF and BERT is well known in English (the authors cite Arabi and Akbari 2022 and Agarwal et al. 2018), so the novelty is narrow but real for Marathi. If the dataset is cleaned and re-evaluated with proper held-out tuning, it could be a useful benchmark. As is, I'd treat the 82% number with suspicion.

Who this is for: people working on Marathi NLP who need a starting dataset, and anyone teaching evaluation pitfalls. It deserves a serious referee — the flaws are addressable and the resource is worth vetting — but the paper should not be accepted without a reworked evaluation.

Recommendation: send to peer review, but expect major revision: proper validation split, error bars or significance testing, and some check on label preservation.

Referee Report

3 major / 5 minor

Summary. The paper presents a Marathi plagiarism/paraphrase detection system that combines TF-IDF vectors and MahaSBERT-STS BERT embeddings in a weighted ensemble of classifiers. The authors construct a Marathi dataset by machine-translating the MIT Plagiarism Detection Dataset (an SNLI-derived corpus), compare two translation models, and evaluate several classifiers plus the proposed ensemble. The headline result is 82.04% accuracy for the ensemble with WBERT = 0.6 versus 80.64% for BERT-only, with an additional comparison on a translated version of MRPC used as validation.

Significance. If the result were established under a proper out-of-sample evaluation, the contribution would be modest but useful for low-resource Marathi NLP: it shows that a hybrid of classical TF-IDF features and modern sentence embeddings can be assembled with publicly available tools, and it releases code and a translated corpus. The paper is reproducible in principle: the code and dataset are public, the models are named, and the hyperparameters are given in Table 7. However, the central numerical claim currently rests on a test-set selection procedure and a single split, so the significance of the reported improvement cannot be assessed from the present evidence.

major comments (3)

[Section 3.4 and Section 4.1 (with Section 4.2, Table 4)]
[Section 3.1 and Section 3.2 (with Tables 2-3)]
[Section 4.2, Table 4]

minor comments (5)

[Table 2 caption]
[References]
[Figure 3]
[Section 4.2]
[Section 3.4]

Circularity Check

1 steps flagged · score 6.0 of 10

The reported 82.04% ensemble accuracy is not an independent prediction: the ensemble weights and model combination were selected by iterative refinement on the same data split that produced the headline number.

fitted input called prediction [Section 3.3, Section 3.4, Section 4.1, Section 4.2]
"We utilized 80% of the extracted vectors for training the classifier and reserved 20% for testing. ... Model combinations and weights were iteratively refined to achieve optimal performance by leveraging complementary insights from each classifier set, as documented in subsection 4.1. ... The ensemble system achieved optimal results with WBERT and WT F−IDF values of 0.6 and 0.4, respectively. ... Our proposed system ... achieved the highest accuracy of 82.04%, compared to 80.64% accuracy when using only BERT embeddings."

The only held-out split described before tuning is the 20% test split; no validation split is introduced for model selection. The text says model combinations and weights were 'iteratively refined to achieve optimal performance' and that the 0.6/0.4 ensemble weights gave 'optimal results,' while the accuracy reported in Section 4.2 is measured on that same 20% test set. Therefore the headline 82.04% is the value of a fitted configuration on the test set, not an out-of-sample prediction, and the 1.4-point gain over BERT-only is a selected maximum rather than an independent comparison.

full rationale

The paper's core building blocks are not circular: MahaSBERT-STS and TF-IDF are independent representation methods, the classifiers are standard, and the data is a translation of an existing English corpus. The circularity concern is concentrated in the evaluation protocol. Section 3.3 fixes an 80/20 train/test split; Section 3.4 states that model combinations and weights were 'iteratively refined to achieve optimal performance'; Section 4.1 reports the winning configuration; Section 4.2 then quotes 82.04% for that configuration. Because no validation split is described, the reported test accuracy is not shown to be independent of the weight-selection procedure. This makes the central claim—that the weighted ensemble beats BERT-only by 1.4 points—partially circular as an empirical claim. The MRPC validation results in Table 5 provide some independent evidence that the final configuration generalizes beyond the translated MIT dataset, and the representations themselves are not defined in terms of the target result, so the score is 6 rather than 8. No load-bearing self-citation, uniqueness theorem, or ansatz-by-citation is present.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the validity of machine-translated labels, the reliability of MahaSBERT-STS embeddings for Marathi, and the standard machine-learning background. No new entities are introduced; the main free parameters are the ensemble weights tuned on the test set.

free parameters (4)

Ensemble weight for BERT (WBERT) = 0.6
Tuned on the test set; reported optimum in Section 4.1 and Figure 3.
Ensemble weight for TF-IDF (WTF-IDF) = 0.4
Tuned alongside WBERT; complements the BERT weight.
Per-classifier weights in BERT set (wBi) = XGBoost 0.7, SVC 0.3
Tuned on the test set; listed in Table 7.
Per-classifier weights in TF-IDF set (wTj) = Logistic Regression 0.1, LightGBM 0.9
Tuned on the test set; listed in Table 7.

assumptions (5)

domain assumption The MIT Plagiarism Detection Dataset (derived from SNLI) provides valid ground-truth labels for plagiarism/paraphrase detection, with entailment pairs labeled as plagiarized.
Section 3.1; the labels are taken as given and translated.
domain assumption Machine translation with aryaumesh/english-to-marathi preserves the semantic relations needed for the plagiarism labels.
Section 3.1; quality verified via BERTScore and TransQuest, not via label-preservation checks.
domain assumption MahaSBERT-STS produces reliable sentence embeddings for Marathi semantic similarity.
Section 3.3; model is used as a frozen feature extractor, cited from Joshi et al. 2022.
domain assumption Element-wise subtraction of embeddings yields a representation where classifiers can learn the semantic relation between input and reference.
Section 3.3; this is a standard heuristic for sentence pair modeling, but unproven for Marathi.
standard math Standard ML background (scikit-learn, XGBoost, LightGBM, PCA) works as claimed.
Used throughout; accepted computational tools.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Enhancing Plagiarism Detection in Marathi with a Weighted Ensemble of TF-IDF and BERT Embeddings for Low-Resource Language Processing." pith.science (2026). https://pith.science/paper/RACX2KG4

@misc{pith2026250105260,
  author       = {Pith},
  title        = {Pith review of: Enhancing Plagiarism Detection in Marathi with a Weighted Ensemble of TF-IDF and BERT Embeddings for Low-Resource Language Processing},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RACX2KG4}},
  note         = {Machine review of arXiv:2501.05260}
}

read the original abstract

Plagiarism involves using another person's work or concepts without proper attribution, presenting them as original creations. With the growing amount of data communicated in regional languages such as Marathi -- one of India's regional languages -- it is crucial to design robust plagiarism detection systems tailored for low-resource languages. Language models like Bidirectional Encoder Representations from Transformers (BERT) have demonstrated exceptional capability in text representation and feature extraction, making them essential tools for semantic analysis and plagiarism detection. However, the application of BERT for low-resource languages remains under-explored, particularly in the context of plagiarism detection. This paper presents a method to enhance the accuracy of plagiarism detection for Marathi texts using BERT sentence embeddings in conjunction with Term Frequency-Inverse Document Frequency (TF-IDF) feature representation. This approach effectively captures statistical, semantic, and syntactic aspects of text features through a weighted voting ensemble of machine learning models.

Figures

Figures reproduced from arXiv: 2501.05260 by the authors.

**Figure 2.** Proposed Weighted Ensemble Voting System for Plagiarism Detection [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

4 extracted references · 2 canonical work pages

[2005]

UNDL foun- dation, 2

Universal networking language. UNDL foun- dation, 2. Chi Wang, Qingyun Wu, Markus Weimer, and Erkang Zhu. 2021. Flaml: A fast and lightweight automl library. In MLSys. Tianyi Zhang*, V arsha Kishore*, Felix Wu*, Kilian Q. Weinberger, and Y oav Artzi. 2020.Bertscore: Eval- uating text generation with bert . In International Conference on Learning Represent...

work page 2021
[2017]

Advances in neural information processing systems, 30

Lightgbm: A highly efﬁcient gradient boost- ing decision tree. Advances in neural information processing systems, 30. Leilei Kong, Kaisheng Chen, Liuyang Tian, Zhenyuan Hao, Zhongyuan Han, and Haoliang Qi. 2016. Hit2016@ dpil-ﬁre2016: Detecting paraphrases in indian languages based on gradient tree boosting. In FIRE (Working Notes), pages 260–265. Citesee...

work page 2016
[2019]

arXiv preprint arXiv:1908.06951

Gradient boosting machine: a survey. arXiv preprint arXiv:1908.06951. Paul Jaccard. 1901. Étude comparative de la distribution ﬂorale dans une portion des alpes et des jura. Bull Soc V audoise Sci Nat, 37:547–579. Ananya Joshi, Aditi Kajale, Janhavi Gadre, Samruddhi Deode, and Raviraj Joshi. 2022. L3cube-mahasbert and hindsbert: Sentence bert models and b...

arXiv 1908
[2021]

In Proceedings of the 4th International Conference on Advances in Science & Technology (ICAST2021)

Analysis of plagiarism detection tools and methods. In Proceedings of the 4th International Conference on Advances in Science & Technology (ICAST2021). V Levenshtein. 1966. Binary codes capable of correct- ing deletions, insertions, and reversals. Proceedings of the Soviet physics doklady . David D Lewis. 1998. Naive (bayes) at forty: The independence ass...

arXiv 1966

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.

[1] [2005]

UNDL foun- dation, 2

Universal networking language. UNDL foun- dation, 2. Chi Wang, Qingyun Wu, Markus Weimer, and Erkang Zhu. 2021. Flaml: A fast and lightweight automl library. In MLSys. Tianyi Zhang*, V arsha Kishore*, Felix Wu*, Kilian Q. Weinberger, and Y oav Artzi. 2020.Bertscore: Eval- uating text generation with bert . In International Conference on Learning Represent...

work page 2021

[2] [2017]

Advances in neural information processing systems, 30

Lightgbm: A highly efﬁcient gradient boost- ing decision tree. Advances in neural information processing systems, 30. Leilei Kong, Kaisheng Chen, Liuyang Tian, Zhenyuan Hao, Zhongyuan Han, and Haoliang Qi. 2016. Hit2016@ dpil-ﬁre2016: Detecting paraphrases in indian languages based on gradient tree boosting. In FIRE (Working Notes), pages 260–265. Citesee...

work page 2016

[3] [2019]

arXiv preprint arXiv:1908.06951

Gradient boosting machine: a survey. arXiv preprint arXiv:1908.06951. Paul Jaccard. 1901. Étude comparative de la distribution ﬂorale dans une portion des alpes et des jura. Bull Soc V audoise Sci Nat, 37:547–579. Ananya Joshi, Aditi Kajale, Janhavi Gadre, Samruddhi Deode, and Raviraj Joshi. 2022. L3cube-mahasbert and hindsbert: Sentence bert models and b...

arXiv 1908

[4] [2021]

In Proceedings of the 4th International Conference on Advances in Science & Technology (ICAST2021)

Analysis of plagiarism detection tools and methods. In Proceedings of the 4th International Conference on Advances in Science & Technology (ICAST2021). V Levenshtein. 1966. Binary codes capable of correct- ing deletions, insertions, and reversals. Proceedings of the Soviet physics doklady . David D Lewis. 1998. Naive (bayes) at forty: The independence ass...

arXiv 1966