A Comparative Study of Text Embedding Models for Semantic Text Similarity in Bug Reports

Aryan Jadon; Avinash Patil; Kihwan Han

A Comparative Study of Text Embedding Models for Semantic Text Similarity in Bug Reports

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2308.09193 v2 pith:RET4E2TG submitted 2023-08-17 cs.SE cs.CLcs.LG

A Comparative Study of Text Embedding Models for Semantic Text Similarity in Bug Reports

Avinash Patil , Kihwan Han , Aryan Jadon This is my paper

classification cs.SE cs.CLcs.LG

keywords reportsmodelssoftwareembeddingretrievingsimilarsimilaritybert

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

0 comments

read the original abstract

Bug reports are an essential aspect of software development, and it is crucial to identify and resolve them quickly to ensure the consistent functioning of software systems. Retrieving similar bug reports from an existing database can help reduce the time and effort required to resolve bugs. In this paper, we compared the effectiveness of semantic textual similarity methods for retrieving similar bug reports based on a similarity score. We explored several embedding models such as TF-IDF (Baseline), FastText, Gensim, BERT, and ADA. We used the Software Defects Data containing bug reports for various software projects to evaluate the performance of these models. Our experimental results showed that BERT generally outperformed the rest of the models regarding recall, followed by ADA, Gensim, FastText, and TFIDF. Our study provides insights into the effectiveness of different embedding methods for retrieving similar bug reports and highlights the impact of selecting the appropriate one for this task. Our code is available on GitHub.

discussion (0)

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

Semantic Drift in Bug Resolution: How Behavioral Signals Propagate from Reports to Tests and Patches
cs.SE 2026-07 conditional novelty 7.0

Behavioral signals in bug reports propagate only partially into tests and fixes; alignment is measurable but representation-dependent, and LLM judges are systematically optimistic versus human ratings.