Stack Trace Deduplication: Faster, More Accurately, and in More Realistic Scenarios

Aleksandr Khvorov; Denis Sushentsev; Egor Shibaev; Yaroslav Golubev

REVIEW 3 major objections 4 minor 1 cited by

Stack Trace Deduplication: Faster, More Accurately, and in More Realistic Scenarios

T0 review · 3 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that a two-stage embedding-plus-reranker model outperforms all compared stack-trace deduplication methods on every tested dataset while remaining fast enough for real-time use.

desk verdict A practical stack-trace dedup system with a valuable new dataset and a realistic evaluation; the accuracy claims are plausible but rest on a possible tokenizer leakage path and single runs, so condition acceptance on those being fixed. read the letter →

arxiv 2412.14802 v1 pith:HSIGISFO submitted 2024-12-19 cs.SE cs.AIcs.LG

Egor Shibaev , Denis Sushentsev , Yaroslav Golubev , Aleksandr Khvorov This is my paper · ORCID

classification cs.SEcs.AIcs.LG

keywords stacktracededuplicationcrashreportgroupingbytepairencodingcross-encoderrerankerapproximatenearestneighborROC-AUCindustrialdatasetbiLSTM

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that automatic deduplication of error reports can be made both more accurate and faster than current alternatives by splitting the task into two stages: a lightweight embedding model retrieves the most plausible matching stack traces, and a reranker re-scores those candidates using shared frames. To test this in realistic conditions, the authors introduce SlowOps, an industrial dataset with an order of magnitude more stack traces per category than existing open-source datasets, and measure not only category-assignment accuracy but also the ability to create new categories and per-report latency. Across Ubuntu, Eclipse, NetBeans, Gnome, and SlowOps, the two-stage model reports the highest Acc@1, ahead of the strongest baseline text-embedding-3-small, and reaches this accuracy in 144.5 ms per report (8.7 ms without reranking). The paper's central claim is that a practical stack-trace deduplication system can have both leading accuracy and real-time speed.

What carries the argument

The load-bearing machinery is the two-stage retrieve-then-rerank pipeline. The first stage uses BPE tokenization (a fixed 10,000-token subword vocabulary) plus a biLSTM encoder and FAISS-style approximate nearest neighbor search so that unseen package names still tokenize and candidate retrieval is fast. The second stage is a cross-encoder reranker: it encodes each frame of a query-candidate pair, and whenever a frame occurs in both traces it adds a learned significance vector V to that frame's embedding on both sides, so the two traces interact before the pair is scored by an MLP. That shared-frame signal is what lets the reranker correct mistakes made by the embedding-only stage; the paper reports that even this limited interaction improves accuracy.

What would settle it

Retrain the BPE tokenizer on only the training split, rerun the Acc@1 and ROC-AUC evaluations, and compare; if the scores drop materially, test-set leakage through tokenization was inflating the reported gains.

Watch

Extended reading notes

Core claim

The central discovery is that a two-stage retrieval architecture outperforms every compared method on all five datasets. The embedding stage tokenizes stack frames with byte-pair encoding (BPE), encodes each trace with a biLSTM, and uses approximate nearest neighbor search to return the top K candidates; the reranker stage is a biLSTM cross-encoder that adds a learned significance vector to the embeddings of frames that appear in both the query and the candidate, then scores the pair with an MLP. With the reranker, Acc@1 is 0.65 on Ubuntu, 0.75 on Eclipse, 0.52 on NetBeans, 0.45 on Gnome, and 0.98 on SlowOps, beating text-embedding-3-small on every benchmark. The same model also earns the best or tied-best ROC-AUC for creating new categories on all datasets except SlowOps, where the embedding-only version leads, and it runs in 144.5 ms with reranking or 8.7 ms without, compared with 1021.2 ms for the LLM baseline.

Load-bearing premise

The byte-pair tokenizer is trained on the same dataset as the model, and the paper does not state that only the training split is used, so tokenization could encode information from test reports and inflate the reported accuracy.

Editorial extensions

If this is right

Deployments can precompute embeddings of known reports and only rerank the top K candidates, so the per-report latency of a high-accuracy system is tens of milliseconds rather than seconds.
BPE tokenization removes the need for fixed frame and package vocabularies, so new packages added after training still receive meaningful token vectors instead of random initializations.
The high accuracy on SlowOps suggests that datasets with many reports per category are substantially easier, and that benchmark results on small open-source datasets may understate real-world performance.
The threshold T that balances attaching to an existing category versus creating a new one must be tuned per dataset, since category structures differ by orders of magnitude.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

An implication the authors leave implicit: the reranker's accuracy gain is probably driven by the learned significance vector for shared frames, so ablating that vector would isolate how much of the improvement comes from cross-trace interaction rather than from a larger model.
Because SlowOps contains only Slow Operation Assertion reports, where the informative frames sit in the middle of the trace, methods that weight top frames heavily may be unfairly disadvantaged, and rankings could differ on other error types.
The speed comparison assumes embeddings of known reports are pre-computed; in a live stream, new reports must periodically be re-embedded, so an end-to-end ingestion test would clarify real throughput beyond per-report scoring time.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Here's the short version: this is a genuinely useful, practice-oriented paper on stack trace dedup, and the SlowOps dataset plus the multi-faceted evaluation are the real contributions. The architecture—BPE tokenization, a biLSTM embedding model with FAISS, then a reranker that adds a learned significance vector to frames appearing in both traces—is a modest but sensible twist on retrieve-and-rerank, and it works. The head-to-head with text-embedding-3-small is fair and informative. I'd send this to review, with one condition: the BPE tokenizer split needs to be clarified.

What the paper does well: it evaluates on five datasets, including a new industrial one with about 50 unique reports per category versus around 1-3 for the open-source ones. It measures accuracy, new-category creation (ROC-AUC), and time assuming pre-computed embeddings, which is the right setting for large systems. It releases code and data. On Ubuntu, Eclipse, NetBeans, Gnome, and SlowOps, the full pipeline beats the strongest baseline (text-embedding-3-small) by 0.02 to 0.06 in Acc@1, and does it at 144.5 ms per report versus 1021.2 ms. That is a legitimate result.

The soft spots are real but not fatal. The paper says the BPE tokenizer is 'trained using the same dataset as the model' without stating that only the training split is used. If the tokenizer sees test reports, its vocab encodes test-specific subwords, and with margins this small, that could flip the comparison. The released code makes this checkable, but until it is checked, the 'outperforms on all datasets' claim is conditional. Second, there are no error bars or repeated runs; the margins over text-embedding-3-small are 0.01-0.06, so single seeds are thin. Third, the threshold T is fit per dataset and several hyperparameters (K, vocab size, max pairs, temperature) are chosen in preliminary experiments; the paper admits this in the threats section, but it still means the headline numbers are somewhat post-hoc. Fourth, the speed comparison is only on Ubuntu, which is fine as a starting point, not as a general claim. Minor: on SlowOps, the reranker's ROC-AUC (0.96) is below the embedding-only model (0.99) and ties text-embedding-3-small only if you use the embedding-only variant; the text glosses over this.

Bottom line: the dataset and evaluation framework deserve a serious referee. I'd accept for review and ask for the tokenizer split, error bars, and a clearer statement of which variant is being compared on ROC-AUC. The central claim is plausible; it just needs tightening.

Referee Report

3 major / 4 minor

Summary. The paper proposes a two-stage stack trace deduplication system: a biLSTM-based embedding model trained with InfoNCE on BPE-tokenized stack frames, followed by a cross-encoder reranker that adds a learned significance vector to frames appearing in both traces. The authors introduce SlowOps, a new industrial dataset from JetBrains with an order of magnitude more reports per category than existing open-source datasets. They evaluate the approach on Ubuntu, Eclipse, NetBeans, Gnome, and SlowOps using Acc@1 for attachment accuracy, ROC-AUC for new-category creation, and wall-clock time per report. The main claim is that the proposed approach outperforms existing methods (including text-embedding-3-small) on all datasets in accuracy while being faster than most baselines. Code and data are released.

Significance. If the accuracy and speed claims hold, the paper makes a practical contribution to crash report deduplication: the two-stage design (fast ANN retrieval plus a more accurate reranker) is a sensible architecture that other practitioners could adopt, and the SlowOps dataset fills a clear gap by providing a large, industrial, manually curated benchmark with many reports per category. The paper also improves on prior evaluations by measuring ROC-AUC for new-category decisions and by timing the retrieval pipeline with precomputed embeddings. The explicit release of code, data, and a Zenodo dataset is a strength that should be credited. The main risk is that several small accuracy margins over the strongest baseline rest on a preprocessing choice (BPE tokenizer training set) that is not fully specified, and on single-run measurements without variance estimates.

major comments (3)

[Section III-A1 and Section IV-B] The paper states in Section III-A1 that "The BPE tokenizer is trained using the same dataset as the model," but it does not specify that this means only the training split of the chronological 70/10/20 split described in Section IV-B. If the BPE vocabulary and token frequencies are learned from the full dataset including test reports, then the subword representation of test frames could be artificially well-formed, giving the proposed model an advantage over baselines that use pre-defined tokenizers (e.g., text-embedding-3-small). The reported Acc@1 margins in Table II over text-embedding-3-small are only 0.02 on Eclipse, 0.03 on Ubuntu, 0.05 on Gnome, 0.06 on NetBeans, and 0.05 on SlowOps, so a small leakage effect could change the outcome on some datasets. Please clarify that the tokenizer is fit only on the training split, or retrain the tokenizer accordingly and rerun the experiments if this was not already the case.
[Tables II and III] All Acc@1 and ROC-AUC values are reported as single point estimates with no confidence intervals, standard deviations, or significance tests. The margins over the strongest baseline (text-embedding-3-small) are small on several datasets (for example, 0.02 on Eclipse and 0.03 on Ubuntu in Table II). Without repeated runs or bootstrap intervals, it is not possible to assess whether these differences are reliable or within noise. Please provide results over multiple random seeds, or at least bootstrap confidence intervals for the test-set metrics, and, where appropriate, a statistical significance test for the pairwise comparison against the strongest baseline.
[Section IV-B] The threshold T used for the new-category creation decision is described as "trained for each particular dataset" by selecting the value that gives the best F1 score, but the paper does not state whether this selection is made on the validation split or on the test split. If T is chosen using test labels, the reported ROC-AUC numbers in Table III are not directly affected because ROC-AUC is threshold-free, but any claim about the operational behavior of the new-category decision in a deployed system would be optimistic. Please explicitly state which split is used for threshold selection and, ideally, report precision/recall (or F1) on the test split for the chosen T.

minor comments (4)

[Introduction and Section IV-E2] The statement that "for creating new categories, our approach ... ties with an LLM text-embedding-3-small on SlowOps" is only true for the embedding-model-only variant (ROC-AUC 0.99 vs. 0.99); with the reranker, Table III shows 0.96 vs. 0.99, which is not a tie. Please clarify which variant this claim refers to.
[Table IV] The speed comparison reports average time per report on Ubuntu but does not state how many repetitions were performed or give any measure of variance. Please report the number of runs and, if possible, a standard deviation or confidence interval.
[References] Reference [24] is cited as "text-embedding-ada-003" but the baseline is called text-embedding-3-small elsewhere; please correct the reference title/identifier to match the actual model used.
[Section IV-D2] The name "Rodriguez et al." appears in the text while the reference list uses "Rodrigues et al."; please unify the spelling.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity; the paper's claims rest on empirical evaluation against external datasets and baselines.

full rationale

The paper is an empirical systems paper: it trains an embedding model and reranker, evaluates on held-out chronological test splits of five datasets, and compares against external baselines including an LLM embedding service. No claimed 'prediction' is defined in terms of a fitted value. Acc@1 is computed only on reports attached to existing categories, and ROC-AUC is threshold-free; the per-dataset threshold T is tuned for the new-category decision but does not enter the threshold-free ROC-AUC or the attachment accuracy numbers, so threshold tuning does not force the headline results. The BPE tokenizer is stated to be 'trained using the same dataset as the model,' which could raise a data-leakage concern if the full dataset including test reports was used, but that is a methodological validity risk, not a circularity in the derivation chain: the tokenizer is not the target quantity being predicted. The authors' own prior S3M work appears as a baseline and as related work, but it is not used as evidence or justification for the new model's superiority; the comparison is external and reproducible from the released code and data. There is no invocation of a uniqueness theorem, no ansatz smuggled in via citation, and no renaming of a known result as a new derivation. The central accuracy and speed claims stand or fall on the experimental setup rather than on any definitional equivalence, so no significant circularity is present.

Assumptions & free parameters 5 free parameters · 4 assumptions · 1 invented entities

The central claims are empirical and rest on standard machine learning components. The free parameters are few but several are tuned on the evaluated data without a dedicated held-out tuning set. The BPE training split is a potential leakage risk. No new physical or domain entities are introduced; the significance vector is an internal model parameter.

free parameters (5)

BPE vocabulary size = 10,000
Selected in preliminary experiments (Section III-A1).
K (number of candidates passed to reranker) = 10
Selected in preliminary experiments; larger values gave negligible improvement (Section IV-B).
Threshold T = per dataset
Chosen per dataset to maximize F1 score on historical category-creation data (Section IV-B).
Max pairs per category = per dataset
Set depending on average number of unique reports per category to avoid dominance in training (Section III-A4).
InfoNCE temperature tau = not reported
Temperature parameter of the loss function; its value is not specified in the paper (Section III-A4).

assumptions (4)

domain assumption Ground-truth categories in all datasets are correct and consistent.
The evaluation uses manual labels as ground truth (Section IV-A1).
domain assumption Identical reports are always assigned to the same category in practice.
The evaluation skips exact duplicates based on this reasoning (Section IV-B).
ad hoc to paper BPE tokenizer is trained only on the training split, or leakage from test data does not affect conclusions.
Section III-A1 says the tokenizer is trained on the same dataset as the model without specifying the split.
domain assumption Pre-computed embeddings are available for all previously seen reports.
Time evaluation relies on this assumption (Section IV-D3).

invented entities (1)

Significance vector V
purpose: Added to frame embeddings of frames that occur in both stack traces to encode repeated-frame information in the cross-encoder.
A learned model parameter; its contribution is demonstrated only by the paper's own experiments, with no external evidence or ablation isolating it.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Stack Trace Deduplication: Faster, More Accurately, and in More Realistic Scenarios." pith.science (2026). https://pith.science/paper/HSIGISFO

@misc{pith2026241214802,
  author       = {Pith},
  title        = {Pith review of: Stack Trace Deduplication: Faster, More Accurately, and in More Realistic Scenarios},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HSIGISFO}},
  note         = {Machine review of arXiv:2412.14802}
}

read the original abstract

In large-scale software systems, there are often no fully-fledged bug reports with human-written descriptions when an error occurs. In this case, developers rely on stack traces, i.e., series of function calls that led to the error. Since there can be tens and hundreds of thousands of them describing the same issue from different users, automatic deduplication into categories is necessary to allow for processing. Recent works have proposed powerful deep learning-based approaches for this, but they are evaluated and compared in isolation from real-life workflows, and it is not clear whether they will actually work well at scale. To overcome this gap, this work presents three main contributions: a novel model, an industry-based dataset, and a multi-faceted evaluation. Our model consists of two parts - (1) an embedding model with byte-pair encoding and approximate nearest neighbor search to quickly find the most relevant stack traces to the incoming one, and (2) a reranker that re-ranks the most fitting stack traces, taking into account the repeated frames between them. To complement the existing datasets collected from open-source projects, we share with the community SlowOps - a dataset of stack traces from IntelliJ-based products developed by JetBrains, which has an order of magnitude more stack traces per category. Finally, we carry out an evaluation that strives to be realistic: measuring not only the accuracy of categorization, but also the operation time and the ability to create new categories. The evaluation shows that our model strikes a good balance - it outperforms other models on both open-source datasets and SlowOps, while also being faster on time than most. We release all of our code and data, and hope that our work can pave the way to further practice-oriented research in the area.

Figures

Figures reproduced from arXiv: 2412.14802 by the authors.

**Figure 2.** The architecture of the cross-encoder with biLSTM encoder and searching for identical frames. The identical frames are connected by dotted lines. [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Finding Important Stack Frames in Large Systems
cs.SE 2025-01 conditional novelty 3.0 of 10

An IDF-based tool that highlights the three rarest stack-trace frames was integrated into JetBrains' bug processing and received a usefulness score of 3.6/5 from 18 developers.

Reference graph

Works this paper leans on

36 extracted references · 30 canonical work pages · cited by 1 Pith paper

[1]

Design, discussion, and dissent in open bug reports,

A. J. Ko and P. K. Chilana, “Design, discussion, and dissent in open bug reports,” in Proceedings of the 2011 IConference , 2011, p. 106–113

work page 2011
[2]

What makes a good bug report?

T. Zimmermann, R. Premraj, N. Bettenburg, S. Just, A. Schroter, and C. Weiss, “What makes a good bug report?” IEEE Transactions on Software Engineering, vol. 36, no. 5, pp. 618–643, 2010

work page 2010
[3]

Mining version histories to guide software changes,

T. Zimmermann, P. Weibgerber, S. Diehl, and A. Zeller, “Mining version histories to guide software changes,” in Proceedings. 26th International Conference on Software Engineering , 2004, pp. 563–572

work page 2004
[4]

Automated duplicate detection for bug track- ing systems,

N. Jalbert and W. Weimer, “Automated duplicate detection for bug track- ing systems,” in 2008 IEEE International Conference on Dependable Systems and Networks With FTCS and DCC (DSN) , 2008, pp. 52–61

work page 2008
[5]

Detection of duplicate defect reports using natural language processing,

P. Runeson, M. Alexandersson, and O. Nyholm, “Detection of duplicate defect reports using natural language processing,” in 29th International Conference on Software Engineering (ICSE’07) , 2007, pp. 499–510

work page 2007
[6]

Automatically identifying known software problems,

N. Modani, R. Gupta, G. Lohman, T. Syeda-Mahmood, and L. Mignet, “Automatically identifying known software problems,” in 2007 IEEE 23rd International Conference on Data Engineering Workshop , 2007, pp. 433–441

work page 2007
[7]

Finding similar failures using callstack similarity,

K. Bartz, J. W. Stokes, J. C. Platt, R. Kivett, D. Grant, S. Calinoiu, and G. Loihle, “Finding similar failures using callstack similarity,” in Proceedings of the Third Conference on Tackling Computer Systems Problems with Machine Learning Techniques , 2008, p. 1

work page 2008
[8]

Classifying field crash reports for fixing bugs: A case study of Mozilla Firefox,

T. Dhaliwal, F. Khomh, and Y . Zou, “Classifying field crash reports for fixing bugs: A case study of Mozilla Firefox,” in 2011 27th IEEE International Conference on Software Maintenance (ICSM) , 2011, pp. 333–342

work page 2011

Show all 36 references

[9]

Towards more accurate retrieval of duplicate bug reports,

C. Sun, D. Lo, S.-C. Khoo, and J. Jiang, “Towards more accurate retrieval of duplicate bug reports,” in2011 26th IEEE/ACM International Conference on Automated Software Engineering (ASE 2011) , 2011, pp. 253–262

2011
[10]

DURFEX: A feature extraction technique for efficient detection of duplicate bug reports,

K. K. Sabor, A. Hamou-Lhadj, and A. Larsson, “DURFEX: A feature extraction technique for efficient detection of duplicate bug reports,” in 2017 IEEE International Conference on Software Quality, Reliability and Security (QRS) , 2017, pp. 240–250

2017
[11]

Finding duplicates of your yet unwritten bug report,

J. Lerch and M. Mezini, “Finding duplicates of your yet unwritten bug report,” in 2013 17th European Conference on Software Maintenance and Reengineering, 2013, pp. 69–78

2013
[12]

TraceSim: a method for calculating stack trace similarity,

R. Vasiliev, D. Koznov, G. Chernishev, A. Khvorov, D. Luciv, and N. Povarov, “TraceSim: a method for calculating stack trace similarity,” in Proceedings of the 4th ACM SIGSOFT International Workshop on Machine-Learning Techniques for Software-Quality Evaluation , 2020, p. 25–30

2020
[13]

S3M: Siamese stack (trace) similarity measure,

A. Khvorov, R. Vasiliev, G. Chernishev, I. M. Rodrigues, D. Koznov, and N. Povarov, “S3M: Siamese stack (trace) similarity measure,” in 2021 IEEE/ACM 18th International Conference on Mining Software Repositories (MSR), 2021, pp. 266–270

2021
[14]

Aggregation of stack trace similarities for crash report deduplication,

N. Karasov, A. Khvorov, R. Vasiliev, Y . Golubev, and T. Bryksin, “Aggregation of stack trace similarities for crash report deduplication,” arXiv preprint arXiv:2205.00212 , 2022

2022 arXiv
[15]

Deep- Crash: deep metric learning for crash bucketing based on stack trace,

L. Chao, X. Qiaoluan, L. Yong, X. Yang, and C. Hyun-Deok, “Deep- Crash: deep metric learning for crash bucketing based on stack trace,” in Proceedings of the 6th International Workshop on Machine Learning Techniques for Software Quality Evaluation , 2022, pp. 29–34

2022
[16]

FaST: A linear time stack trace alignment heuristic for crash report deduplication,

I. M. Rodrigues, D. Aloise, and E. R. Fernandes, “FaST: A linear time stack trace alignment heuristic for crash report deduplication,” in 2022 IEEE/ACM 19th International Conference on Mining Software Repositories (MSR), 2022, pp. 549–560

2022
[17]

Long short-term memory,

S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural computation, vol. 9, pp. 1735–80, 12 1997

1997
[18]

Passage re-ranking with BERT,

R. Nogueira and K. Cho, “Passage re-ranking with BERT,” arXiv preprint arXiv:1901.04085, 2019

1901 arXiv
[19]

The unreasonable effectiveness of traditional information retrieval in crash report dedupli- cation,

J. C. Campbell, E. A. Santos, and A. Hindle, “The unreasonable effectiveness of traditional information retrieval in crash report dedupli- cation,” in Proceedings of the 13th International Conference on Mining Software Repositories, 2016, pp. 269–280

2016
[20]

TraceSim: An alignment method for computing stack trace similarity,

I. M. Rodrigues, A. Khvorov, D. Aloise, R. Vasiliev, D. Koznov, E. R. Fernandes, G. Chernishev, D. Luciv, and N. Povarov, “TraceSim: An alignment method for computing stack trace similarity,” Empirical Software Engineering, vol. 27, no. 2, p. 53, 2022

2022
[21]

Neural machine translation of rare words with subword units,

R. Sennrich, B. Haddow, and A. Birch, “Neural machine translation of rare words with subword units,” in Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2016, pp. 1715–1725

2016
[22]

Billion-scale similarity search with GPUs,

J. Johnson, M. Douze, and H. Jegou, “Billion-scale similarity search with GPUs,” IEEE Transactions on Big Data , vol. 7, no. 03, pp. 535– 547, 2021

2021
[23]

The Intel- liJ Platform: A framework for building plugins and mining software data,

Z. Kurbatova, Y . Golubev, V . Kovalenko, and T. Bryksin, “The Intel- liJ Platform: A framework for building plugins and mining software data,” in 2021 36th IEEE/ACM International Conference on Automated Software Engineering Workshops (ASEW) , 2021, pp. 14–17

2021
[24]

(2022) text-embedding-ada-003

OpenAI. (2022) text-embedding-ada-003. Accessed 10.12.2024. [On- line]. Available: https://platform.openai.com/docs/guides/embeddings

2022
[25]

Shibaev, D

E. Shibaev, D. Sushentsev, Y . Golubev, and A. Khvorov. (2024) SlowOps: An industrial dataset of stack traces. Accessed 10.12.2024. [Online]. Available: https://doi.org/10.5281/zenodo.14364857

2024 doi
[26]

(2024) Repl

——. (2024) Repl. package. Accessed 10.12.2024. [Online]. Available: https://github.com/JetBrains-Research/stack-trace-deduplication

2024
[27]

Quickly finding known software problems via automated symptom matching,

M. Brodie, S. Ma, G. Lohman, L. Mignet, N. Modani, M. Wilding, J. Champlin, and P. Sohn, “Quickly finding known software problems via automated symptom matching,” in Second International Conference on Autonomic Computing (ICAC’05) , 2005, pp. 101–110

2005
[28]

A general method applicable to the search for similarities in the amino acid sequence of two proteins,

S. B. Needleman and C. D. Wunsch, “A general method applicable to the search for similarities in the amino acid sequence of two proteins,” Journal of molecular biology , vol. 48, no. 3, pp. 443–453, 1970

1970
[29]

Efficient estimation of word representations in vector space,

T. Mikolov, K. Chen, G. S. Corrado, and J. Dean, “Efficient estimation of word representations in vector space,” in International Conference on Learning Representations, 2013

2013
[30]

Scaffle: Bug localization on millions of files,

M. Pradel, V . Murali, R. Qian, M. Machalica, E. Meijer, and S. Chandra, “Scaffle: Bug localization on millions of files,” in Proceedings of the 29th ACM SIGSOFT International Symposium on Software Testing and Analysis, 2020, pp. 225–236

2020
[31]

Smoothed con- trastive learning for unsupervised sentence embedding,

X. Wu, C. Gao, Y . Su, J. Han, Z. Wang, and S. Hu, “Smoothed con- trastive learning for unsupervised sentence embedding,” in Proceedings of the 29th International Conference on Computational Linguistics , 2022, pp. 4902–4906

2022
[32]

Rethinking InfoNCE: How many negative samples do you need?

C. Wu, F. Wu, and Y . Huang, “Rethinking InfoNCE: How many negative samples do you need?” arXiv preprint arXiv:2105.13003 , 2021

2021 arXiv
[33]

Representation learning with contrastive predictive coding,

A. v. d. Oord, Y . Li, and O. Vinyals, “Representation learning with contrastive predictive coding,” arXiv preprint arXiv:1807.03748 , 2018

2018 arXiv
[34]

Poly-encoders: Architectures and pre-training strategies for fast and accurate multi- sentence scoring,

S. Humeau, K. Shuster, M.-A. Lachaux, and J. Weston, “Poly-encoders: Architectures and pre-training strategies for fast and accurate multi- sentence scoring,” in International Conference on Learning Representa- tions, 2019

2019
[35]

BERT with history answer embedding for conversational question answering,

C. Qu, L. Yang, M. Qiu, W. B. Croft, Y . Zhang, and M. Iyyer, “BERT with history answer embedding for conversational question answering,” in Proceedings of the 42nd international ACM SIGIR conference on research and development in information retrieval, 2019, pp. 1133–1136

2019
[36]

ColBERT: Efficient and effective passage search via contextualized late interaction over BERT,

O. Khattab and M. Zaharia, “ColBERT: Efficient and effective passage search via contextualized late interaction over BERT,” in Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval , 2020, pp. 39–48

2020

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [1]

Design, discussion, and dissent in open bug reports,

A. J. Ko and P. K. Chilana, “Design, discussion, and dissent in open bug reports,” in Proceedings of the 2011 IConference , 2011, p. 106–113

work page 2011

[2] [2]

What makes a good bug report?

T. Zimmermann, R. Premraj, N. Bettenburg, S. Just, A. Schroter, and C. Weiss, “What makes a good bug report?” IEEE Transactions on Software Engineering, vol. 36, no. 5, pp. 618–643, 2010

work page 2010

[3] [3]

Mining version histories to guide software changes,

T. Zimmermann, P. Weibgerber, S. Diehl, and A. Zeller, “Mining version histories to guide software changes,” in Proceedings. 26th International Conference on Software Engineering , 2004, pp. 563–572

work page 2004

[4] [4]

Automated duplicate detection for bug track- ing systems,

N. Jalbert and W. Weimer, “Automated duplicate detection for bug track- ing systems,” in 2008 IEEE International Conference on Dependable Systems and Networks With FTCS and DCC (DSN) , 2008, pp. 52–61

work page 2008

[5] [5]

Detection of duplicate defect reports using natural language processing,

P. Runeson, M. Alexandersson, and O. Nyholm, “Detection of duplicate defect reports using natural language processing,” in 29th International Conference on Software Engineering (ICSE’07) , 2007, pp. 499–510

work page 2007

[6] [6]

Automatically identifying known software problems,

N. Modani, R. Gupta, G. Lohman, T. Syeda-Mahmood, and L. Mignet, “Automatically identifying known software problems,” in 2007 IEEE 23rd International Conference on Data Engineering Workshop , 2007, pp. 433–441

work page 2007

[7] [7]

Finding similar failures using callstack similarity,

K. Bartz, J. W. Stokes, J. C. Platt, R. Kivett, D. Grant, S. Calinoiu, and G. Loihle, “Finding similar failures using callstack similarity,” in Proceedings of the Third Conference on Tackling Computer Systems Problems with Machine Learning Techniques , 2008, p. 1

work page 2008

[8] [8]

Classifying field crash reports for fixing bugs: A case study of Mozilla Firefox,

T. Dhaliwal, F. Khomh, and Y . Zou, “Classifying field crash reports for fixing bugs: A case study of Mozilla Firefox,” in 2011 27th IEEE International Conference on Software Maintenance (ICSM) , 2011, pp. 333–342

work page 2011