REVIEW 3 major objections 5 minor 31 references
A Novel Word Pair-based Gaussian Sentence Similarity Algorithm For Bengali Extractive Text Summarization
T0 review · 3 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read A word-pair Gaussian similarity measure followed by spectral clustering and TF-IDF selection improves extractive summarization in Bengali and other low-resource languages.
desk verdict The word-pair similarity idea and the Bengali dataset are real, but the stated sigma makes the algorithm mathematically impossible as written; reject and invite a corrected resubmission. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The mechanism that carries the argument is the two-sided nearest-word-pair distance set $D=\{D_{\mathrm{msw}}(x_i,Y)\}\cup\{D_{\mathrm{msw}}(y_j,X)\}$, which the paper calls the Most Similar Word Distance. Each distance is mapped to a similarity by the Gaussian kernel $W_{\mathrm{sim}}=\exp(-D_i^2/(2\sigma^2))$, and the geometric mean of those values gives the sentence similarity, exactly $Sim(X,Y)=\exp(-\sum_i D_i^2/(2n\sigma^2))$. This object replaces the sentence-average vector used by the SASbSC baseline, and the affinity matrix it produces is what spectral clustering operates on. The second mechanism is the two-step extraction rule: group sentences with spectral clustering into $k=\lceil Np\rceil$ clusters, pick the maximum-TF-IDF sentence from each cluster, then restore document order.
What would settle it
Take a sample of sentence pairs from the released dataset, compute the one-sided nearest-word distances $D_{\mathrm{msw}}$ with the same FastText vectors, and evaluate $\exp(-D_i^2/(2(5\times10^{-11})^2))$. If almost every value rounds to zero, the affinity matrix is all zeros and spectral clustering has no signal; a second check is to count nonzero entries in the affinity matrix and verify that the resulting clusters are not trivial. That observation would settle whether the reported results can come from WGSS as written.
Extended reading notes
Core claim
On the paper's own terms, the central discovery is that a sentence is better compared to another sentence through its individual word correspondences than through a single averaged vector. The proposed function is $Sim(X,Y)=\exp\left(-\frac{1}{2n\sigma^2}\sum_{i=1}^{n}D_i^2\right)$, where the $D_i$ come from the two-sided nearest-neighbor distances $D_{\mathrm{msw}}(x,Y)=\min_{y\in Y}\|x-y\|$ and $D_{\mathrm{msw}}(y,X)=\min_{x\in X}\|y-x\|$, and $\sigma$ is set to $5\times10^{-11}$. This symmetric word-pair distance set is meant to preserve local semantic links that averaging loses, and the geometric-mean Gaussian form is meant to damp outlier words. Feeding the resulting affinity matrix into spectral clustering with $k=\lceil Np\rceil$ clusters and choosing one sentence per cluster by TF-IDF yields summaries that the authors report are consistently better than the three comparison models on all four Bengali datasets and competitive on Hindi, Marathi, and Turkish.
Load-bearing premise
The load-bearing numerical premise is that with $\sigma=5\times10^{-11}$, the Gaussian factor $\exp(-D_i^2/(2\sigma^2))$ gives nonzero similarities for the FastText distances actually found in Bengali sentences; if real distances make these exponentials underflow to zero, the affinity matrix becomes degenerate and the claimed ROUGE gains cannot follow from the described computation.
Editorial extensions
If this is right
- If WGSS works as claimed, unsupervised extractive summarization in Bengali needs no training data, only pretrained word vectors, a tokenizer, and a stopword list.
- The approach should transfer to any language with FastText embeddings; the paper demonstrates this on Hindi, Marathi, and Turkish with scores close to Bengali.
- Selecting one sentence per spectral cluster should reduce redundancy and improve topic coverage relative to ranking-only methods like LexRank.
- The released 250-article, 500-summary Bengali dataset gives future work a common evaluation ground for extractive and abstractive summarization.
- The reported 43.2% average ROUGE improvement implies that word-level matching, not additional training data, is the main source of the gain over the three baselines.
Reading between the lines
- Beyond the paper, the same word-pair similarity could be plugged into any graph-based NLP task that needs sentence-level affinity, such as keyphrase extraction or question-answer retrieval, since it only requires embeddings.
- A testable extension the authors do not explore is making $\sigma$ depend on the data, for example scaling it to the median nearest-word distance; that would test whether the fixed $5\times10^{-11}$ value is essential or an artifact.
- Because WGSS ignores word order, a natural follow-up is to add position-sensitive weighting or a small n-gram component and check whether ROUGE-LCS gains further; the paper itself concedes word order is a limitation.
- If the reported numbers reproduce, the method's simplicity makes it a credible baseline for future Bengali summarization work; if the underflow issue is real, correcting the scaling would still leave the word-pair idea testable.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes WGSS, an unsupervised extractive summarization method for Bengali and other low-resource languages. Sentence similarity is computed by taking the geometric mean of Gaussian kernel similarities over word pairs, where each word is matched to its nearest counterpart in the other sentence via FastText embeddings (Eqs. 1-4). This similarity populates an affinity matrix (Eq. 5), spectral clustering groups sentences into topics, and the highest-TF-IDF sentence from each cluster is selected for the summary (Algorithms 1-2). The method is evaluated against BenSumm, SASbSC, and LexRank on four Bengali datasets using ROUGE, with additional experiments in Hindi, Marathi, and Turkish. The authors report an average improvement of 43.2% and release a new dataset of 250 articles with two human summaries each, along with code. The paper itself acknowledges limitations in Section 5, including reliance on pre-trained embeddings and the absence of word-order information.
Significance. If the reported results were reproducible, WGSS would be a simple, language-agnostic addition to extractive summarization for low-resource languages, and the released 500-summary Bengali dataset would be a useful community resource. The authors should be credited for making the implementation and data public and for testing across four languages. However, the central numerical claim is not reproducible as stated: the fixed value of sigma in Eq. (3) forces the Gaussian similarity to underflow to zero for realistic word-vector distances, so the affinity matrix degenerates. In addition, sigma is tuned on the evaluation sets themselves, which introduces circularity into the headline performance numbers. These issues bear directly on the paper's central claims, so the positive potential of the method and dataset does not translate into a publishable result in the current form.
major comments (3)
- [Section 3.2, Eqs. (3)-(4), Algorithm 1] The stated sigma value makes the proposed similarity computation degenerate. With sigma fixed at 5e-11, 2*sigma^2 is 5e-21, so the exponent -D_i^2/(2*sigma^2) is numerically below about -745 for any squared distance greater than roughly 3.7e-18. In IEEE double precision, exp() of such values underflows to zero. FastText word-vector distances between distinct words are many orders of magnitude larger (typically O(1) or more in squared Euclidean norm), so every Wsim term for distinct word pairs is exactly zero, and the geometric mean in Eq. (4) is zero for all non-identical sentence pairs. The affinity matrix in Eq. (5) therefore has zero off-diagonal entries, and spectral clustering on this matrix is degenerate. The ROUGE scores around 0.4-0.5 in Table 3 cannot follow from the described computation. The manuscript provides no rescaling or normalization that would make sigma = 5e-11 meaningful; if this value is a typo, the paper must state the intended value and recompute all results.
- [Section 4.4.1 and Table 3] The standard deviation sigma is selected by maximizing ROUGE-F1 on the evaluation datasets, including the Self-Curated dataset that is later reported as a test set in Table 3. Section 4.4.1 says that 63 values were 'experimented on' and that 5e-11 'was fixed as the value for sigma that gives the best similarity measurement,' with Fig. 4 showing ROUGE scores as a function of sigma. Tuning a hyperparameter on the same data that is used for final evaluation makes the reported improvements partly a product of test-set selection. An independent validation split, nested cross-validation, or a fixed sigma chosen without reference to the evaluation labels is required before the comparative claims can be accepted.
- [Table 2 versus Table 3] The evaluation setup is internally inconsistent. Table 2 reports that TF-IDF ranking on the Self-Curated dataset with summary proportion 0.2 achieves ROUGE-1 0.50, ROUGE-2 0.40, and ROUGE-LCS 0.46. Table 3 reports the proposed WGSS on the same Self-Curated dataset, also with summary proportion 0.2, as achieving 0.47, 0.36, and 0.43. Since the TF-IDF ranking in Table 2 is the extraction strategy used inside WGSS, these two tables describe the same configuration and should not disagree. This discrepancy undermines confidence in the reproducibility of the reported results and must be reconciled.
minor comments (5)
- [Section 4.3, item 1] The description of ROUGE-1 says 'It calculates the total common characters between the summaries'; this should read 'common unigrams' rather than characters.
- [Figures 5 and 6] The legend in Figs. 5 and 6 labels the proposed method as 'WSbSC' instead of 'WGSS'; please correct the label.
- [Table 4] The 'Bengali (Average)' row lists ROUGE-2 as 0.38, but the mean of the four preceding Bengali ROUGE-2 values (0.36, 0.43, 0.34, 0.41) is 0.385, which rounds to 0.39; the averaging should be checked and reported consistently.
- [Footnote 13] The URL for the Turkish dataset is given as 'https://wwww.github.com/xtinge/turkish-extractive-summarization-dataset'; the extra 'w' and the use of 'github.com' rather than 'www.github.com' should be corrected.
- [Figure 4] The x-axis labels of Fig. 4 are difficult to read and inconsistently formatted (e.g., 'E-12', 'E-1', '5E-1'); using a logarithmic axis with clear scientific notation would make the fine-tuning curve interpretable.
Circularity Check
Partial circularity: WGSS's key hyperparameters (sigma and the cluster-sentence picker) are selected by ROUGE-F1 on the Self-Curated evaluation dataset, and the resulting ROUGE scores on that same dataset are then reported as validation.
-
fitted input called prediction
[Section 4.4.1-4.4.2 and Table 3; Equations 3-4]
"After experimentation, 5× 10− 11 was fixed as the value for σ that gives the most representative semantic relation between sentences. The result of the fine-tuning process is shown in Fig. 4. ... We examined the two methods on our Self-Curated dataset with 0.2 as a summary proportion. In Table 2, the TF-IDF ranking is shown to perform better than the lead extraction method in the Self-Curated dataset."
The paper tunes sigma by comparing ROUGE-F1 curves (Fig. 4) and tunes the sentence-extraction rule by comparing ROUGE scores on the Self-Curated dataset (Table 2). The final validation table (Table 3) then reports WGSS's ROUGE scores on that same Self-Curated dataset as evidence of the model's quality. As described, no held-out split is mentioned, so the Self-Curated validation numbers are the direct output of the ROUGE-based selection procedure rather than an independent prediction from the algorithm. The other datasets and languages are not used for this fit on the paper's own account, so the circularity is partial rather than total.
full rationale
The derivation chain is mostly an engineering proposal: compute word-pair Gaussian similarities, build an affinity matrix, run spectral clustering, and pick TF-IDF-best sentences. There is no self-citation chain or imported uniqueness theorem; the cited baselines and the FastText embeddings are external. The central circular component is hyperparameter selection: Section 4.4.1 selects sigma by inspecting ROUGE-F1 values, and Section 4.4.2 selects the TF-IDF picking rule by comparing ROUGE scores on the Self-Curated dataset, after which Table 3 reports ROUGE scores on that same dataset as validation. That makes part of the headline improvement an artifact of fitting the reported metric rather than an out-of-sample prediction. The fixed sigma is then applied to three other Bengali datasets and to Hindi, Marathi, and Turkish, giving independent empirical content that justifies a moderate rather than a high circularity score. Separately, the stated sigma = 5e-11 appears numerically implausible: in double precision, exp(-D^2/(2*sigma^2)) underflows for ordinary FastText word-vector squared distances, so the described computation would produce zero affinities; this is a serious correctness/consistency issue, not itself a circularity finding. Overall circularity score: 4.
Assumptions & free parameters
free parameters (2)
- sigma =
5e-11
- p =
0.2
assumptions (4)
- domain assumption FastText pretrained vectors capture semantic similarity for Bengali, Hindi, Marathi, and Turkish.
- standard math Spectral clustering of the affinity matrix produces meaningful topic clusters for short documents.
- ad hoc to paper A good extractive summary is one sentence per topic cluster, selected by TF-IDF.
- domain assumption Geometric mean of Gaussian kernel values is a valid sentence similarity measure.
Cite this review
Pith. "Pith review of A Novel Word Pair-based Gaussian Sentence Similarity Algorithm For Bengali Extractive Text Summarization." pith.science (2026). https://pith.science/paper/MRLQ3UCH
@misc{pith2026241117181,
author = {Pith},
title = {Pith review of: A Novel Word Pair-based Gaussian Sentence Similarity Algorithm For Bengali Extractive Text Summarization},
year = {2026},
howpublished = {\url{https://pith.science/paper/MRLQ3UCH}},
note = {Machine review of arXiv:2411.17181}
}
read the original abstract
Extractive Text Summarization is the process of selecting the most representative parts of a larger text without losing any key information. Recent attempts at extractive text summarization in Bengali, either relied on statistical techniques like TF-IDF or used naive sentence similarity measures like the word averaging technique. All of these strategies suffer from expressing semantic relationships correctly. Here, we propose a novel Word pair-based Gaussian Sentence Similarity (WGSS) algorithm for calculating the semantic relation between two sentences. WGSS takes the geometric means of individual Gaussian similarity values of word embedding vectors to get the semantic relationship between sentences. It compares two sentences on a word-to-word basis which rectifies the sentence representation problem faced by the word averaging method. The summarization process extracts key sentences by grouping semantically similar sentences into clusters using the Spectral Clustering algorithm. After clustering, we use TF-IDF ranking to pick the best sentence from each cluster. The proposed method is validated using four different datasets, and it outperformed other recent models by 43.2% on average ROUGE scores (ranging from 2.5% to 95.4%). It is also experimented on other low-resource languages i.e. Turkish, Marathi, and Hindi language, where we find that the proposed method performs as similar as Bengali for these languages. In addition, a new high-quality Bengali dataset is curated which contains 250 articles and a pair of summaries for each of them. We believe this research is a crucial addition to Bengali Natural Language Processing (NLP) research and it can easily be extended into other low-resource languages. We made the implementation of the proposed model and data public on https://github.com/FMOpee/WGSS.
Figures
Figures from the paper (3 more)
Reference graph
Works this paper leans on
-
[1]
A KTER , S., A SA, A. S., U DDIN , M. P., H OSSAIN , M. D., R OY, S. K., AND AFJAL , M. I. An extractive text summarization technique for bengali document(s) using k-m eans clustering algorithm. In 2017 IEEE Interna- 14 A PREPRINT - N OVEMBER 28, 2024 tional Conference on Imaging, Vision & Pattern Recognition (icIVPR) (2017)
work page 2017
-
[2]
A LGULIYEV , R. M., A LIGULIYEV , R. M., I SAZADE , N. R., A BDI , A., AND IDRIS , N. Cosum: Text summa- rization based on clustering and optimization. Expert Systems 36 , 1 (2019), e12340
work page 2019
-
[3]
P., B AUDIN , M., AND DUDA , R
B ABAUD , J., W ITKIN , A. P., B AUDIN , M., AND DUDA , R. O. Uniqueness of the gaussian kernel for scale- space filtering. IEEE Transactions on Pattern Analysis and Machine Intellig ence PAMI-8, 1 (1986), 26–33
work page 1986
-
[4]
B AXENDALE , P. B. Machine-made index for technical literature—an expe riment. IBM Journal of Research and Development 2, 4 (1958), 354–361
work page 1958
-
[5]
C HOWDHURY , R. R., N AYEEM , M. T., M IM, T. T., C HOWDHURY , M. S. R., AND JANNAT, T. Unsupervised abstractive summarization of Bengali text documents. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main V olume(Apr. 2021), Association for Computa- tional Linguistics, pp. 2612–2619
work page 2021
-
[6]
D ASH , S. R., G UHA , P., M ALLICK , D. K., AND PARIDA , S. Summarizing bengali text: An extractive approach. In Intelligent Data Engineering and Analytics (2022), Springer Nature Singapore, pp. 133–140
work page 2022
-
[7]
Extractive summarization data sets generated with meas urable anal- yses
D EMIR , I., K ¨UPC¸ ¨U, E., AND K ¨UPC¸ ¨U, A. Extractive summarization data sets generated with meas urable anal- yses. In Proceedings of the 32nd IEEE Conference on Signal Processin g and Communications Applications (2024)
work page 2024
-
[8]
E DMUNDSON , H. P. New methods in automatic extracting. J. ACM 16, 2 (apr 1969), 264–285
work page 1969
Show all 31 references
-
[9]
S., S ALAMA , C
E L-K ASSAS , W. S., S ALAMA , C. R., R AFEA , A. A., AND MOHAMED , H. K. Automatic text summarization: A comprehensive survey. Expert Systems with Applications 165 (2021), 113679
2021
-
[10]
E RKAN , G., AND RADEV, D. R. Lexrank: graph-based lexical centrality as salience in text summarization. J. Artif. Int. Res. 22 , 1 (dec 2004), 457–479
2004
-
[11]
G OWER , J. C. Generalized procrustes analysis. Psychometrika 40, 1 (Mar 1975), 33–51
1975
-
[12]
Learning word vectors for 157 languages
G RAVE, E., B OJANOWSKI , P., G UPTA , P., J OULIN , A., AND MIKOLOV , T. Learning word vectors for 157 languages. In Proceedings of the Eleventh International Conference on La nguage Resources and Evaluation (LREC 2018) (May 2018), European Language Resources Association (ELRA )
2018
-
[13]
A survey of text summarization extractive techniques
G UPTA , V., AND LEHAL , G. A survey of text summarization extractive techniques. Journal of Emerging T echnologies in W eb Intelligence 2(08 2010)
2010
-
[14]
M., P ERVIN , S., AND BEGUM , Z
H AQUE , M. M., P ERVIN , S., AND BEGUM , Z. Automatic bengali news documents summarization by intr o- ducing sentence frequency and clustering. In 2015 18th International Conference on Computer and Informa tion T echnology (ICCIT)(2015), pp. 156–160
2015
-
[15]
Grundz¨uge der mengenlehre, vol
H AUSDORFF , F. Grundz¨uge der mengenlehre, vol. 7. von V eit, 1914
1914
-
[16]
J AIN , A., B HATIA , D., AND THAKUR , M. K. Extractive text summarization using word vector embe dding. In 2017 International Conference on Machine Learning and Data Science (MLDS) (2017), pp. 51–55
2017
-
[17]
ROUGE: A package for automatic evaluation of summari es
L IN, C.-Y. ROUGE: A package for automatic evaluation of summari es. In T ext Summarization Branches Out (July 2004), Association for Computational Linguistics, p p. 74–81
2004
-
[18]
TextRank: Bringing order into text
M IHALCEA , R., AND TARAU , P. TextRank: Bringing order into text. In Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing (July 2004), Association for Computational Linguistics, pp. 404–411
2004
-
[19]
R., J ONNALAGADDA , S., M OSTAFA , J., AND DEL FIOL , G
M ISHRA , R., B IAN , J., F ISZMAN , M., W EIR , C. R., J ONNALAGADDA , S., M OSTAFA , J., AND DEL FIOL , G. Text summarization in the biomedical domain: A systematic r eview of recent research. Journal of Biomedical Informatics 52 (2014), 457–467. Special Section: Methods in ...
2014
-
[20]
B., AND KUMAR , R
M OHAN , G. B., AND KUMAR , R. P. A comprehensive survey on topic modeling in text summa rization. In Micro-Electronics and T elecommunication Engineering(2022), Springer Nature Singapore, pp. 231–240
2022
-
[21]
A survey on abstractive text summarization
M ORATANCH , N., AND CHITRAKALA , S. A survey on abstractive text summarization. In 2016 International Conference on Circuit, Power and Computing T echnologies (ICCPCT) (2016), pp. 1–7
2016
-
[22]
A survey on extractive text summarization
M ORATANCH , N., AND CHITRAKALA , S. A survey on extractive text summarization. In 2017 International Conference on Computer , Communication and Signal Processing (ICCCSP) (2017), pp. 1–6
2017
-
[23]
The pagerank citation ranking : Bringing order to the web
P AGE , L., B RIN , S., M OTWANI , R., AND WINOGRAD , T. The pagerank citation ranking : Bringing order to the web. In The W eb Conference(1999)
1999
-
[24]
Unsupervised Bengali text summarization using sentenc e embedding and spectral clustering
R OYCHOWDHURY , S., S ARKAR , K., AND MAJI , A. Unsupervised Bengali text summarization using sentenc e embedding and spectral clustering. In Proceedings of the 19th International Conference on Natura l Language Processing (ICON) (Dec. 2022), Association for Computational Ling...
2022
-
[25]
A metric for distributions with applications to image da tabases
R UBNER , Y., T OMASI , C., AND GUIBAS , L. A metric for distributions with applications to image da tabases. In Sixth International Conference on Computer Vision (IEEE Ca t. No.98CH36271) (1998), pp. 59–66
1998
-
[26]
S ALTON , G., W ONG , A., AND YANG , C. S. A vector space model for automatic indexing. Commun. ACM 18, 11 (nov 1975), 613–620
1975
-
[27]
An approach to summarizing bengali news documents
S ARKAR , K. An approach to summarizing bengali news documents. In Proceedings of the International Confer- ence on Advances in Computing, Communications and Informat ics (2012), ICACCI ’12, Association for Com- puting Machinery, p. 857–862
2012
-
[28]
Bengali text summarization by sentence extraction
S ARKAR , K. Bengali text summarization by sentence extraction. CoRR abs/1201.2240 (2012)
2012 arXiv
-
[29]
A survey automatic text summarization
T AS, O., AND KIYANI , F. A survey automatic text summarization. PressAcademia Procedia 5 , 1 (2017), 205–213
2017
-
[30]
A tutorial on spectral clustering
VON LUXBURG , U. A tutorial on spectral clustering. Statistics and Computing 17 , 4 (Dec 2007), 395–416
2007
-
[31]
P., R USTAD , S., S HIDIK , G
W IDYASSARI , A. P., R USTAD , S., S HIDIK , G. F., N OERSASONGKO , E., S YUKUR , A., A FFANDY, A., AND SETIADI , D. R. I. M. Review of automatic text summarization techniqu es & methods. Journal of King Saud University - Computer and Information Sciences 34 , 4 (2022), 1029–1046. 16
2022
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.