Text Tells the Cost: Predicting and Analyzing Repayment Effort of Self-Admitted Technical Debt

Jiakun Liu; Jie Tan; Mohamed Soliman; Paris Avgeriou; Yikun Li

arxiv: 2309.06020 · v3 · submitted 2023-09-12 · 💻 cs.SE · cs.AI

Text Tells the Cost: Predicting and Analyzing Repayment Effort of Self-Admitted Technical Debt

Yikun Li , Mohamed Soliman , Paris Avgeriou , Jie Tan , Jiakun Liu This is my paper

Pith reviewed 2026-05-24 07:08 UTC · model grok-4.3

classification 💻 cs.SE cs.AI

keywords self-admitted technical debtrepayment efforttext classificationBERTTextCNNsoftware maintenancetechnical debt prioritizationcommit analysis

0 comments

The pith

Textual descriptions of self-admitted technical debt can predict its repayment effort.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper assembles records of 341,740 SATD items drawn from more than 2.5 million commits in 1,060 Apache projects and measures actual repayment effort by the elapsed time and code changes between the commit that introduces each item and the commit that removes it. It reports that code, requirement, and test debt require more effort than non-SATD changes while documentation debt requires less. The authors then train models that take only the text of the SATD comment or commit message as input and show that BERT- and TextCNN-based versions outperform both traditional machine-learning classifiers and a simple baseline at estimating effort level. A reader would care because the ability to rank debts by predicted cost from their descriptions alone would let teams decide which items to repay first without first inspecting the surrounding code.

Core claim

The authors claim that repayment effort for SATD can be inferred from textual descriptions alone, that effort differs systematically by SATD type, and that deep-learning models trained on those descriptions achieve higher accuracy than conventional machine-learning methods when tested on the curated dataset of 341,740 items.

What carries the argument

The PRESTI approach, which trains text classifiers on SATD descriptions to predict effort levels measured from commit intervals and diff sizes.

If this is right

Code, requirement, and test debt require greater repayment effort than non-SATD changes.
Documentation debt requires less repayment effort than non-SATD changes.
BERT- and TextCNN-based models outperform traditional machine-learning methods at estimating repayment effort from text.
Distinct keywords appear in SATD items that demand high versus low repayment effort.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

Teams could embed the models in issue trackers to surface high-effort SATD during sprint planning.
The same text-based prediction idea could be tested on technical debt that is not self-admitted.
Effort estimates might improve if models also receive project size or language as additional features.

Load-bearing premise

The time and code changes between the commit that adds an SATD item and the commit that removes it give a clean measure of repayment effort.

What would settle it

Manual review of a random sample of repaid SATD items that finds the measured commit intervals and diff sizes bear no consistent relation to the actual developer hours spent fixing the debt.

Figures

Figures reproduced from arXiv: 2309.06020 by Jiakun Liu, Jie Tan, Mohamed Soliman, Paris Avgeriou, Yikun Li.

**Figure 1.** Figure 1: An Overview of PRESTI. ered during the prioritization process. This, in turn, can lead to more informed decision-making and better management of technical debt in software projects. To address our research questions, we use the PRESTI approach outlined in [PITH_FULL_IMAGE:figures/full_fig_p009_1.png] view at source ↗

read the original abstract

Technical debt refers to the consequences of sub-optimal decisions made during software development that prioritize short-term benefits over long-term maintainability. Self-Admitted Technical Debt (SATD) is a specific form of technical debt, explicitly documented by developers within software artifacts such as source code comments and commit messages. As SATD can hinder software development and maintenance, it is crucial to estimate the effort required to repay it so that we can effectively prioritize it. However, we currently lack an understanding of SATD repayment, and more importantly, we lack approaches that can automatically estimate the repayment effort of SATD based on its textual description. To bridge this gap, we have curated a comprehensive dataset of 341,740 SATD items from 2,568,728 commits across 1,060 Apache repositories and analyzed the repayment effort comparing SATD vs. non-SATD items, as well as different types of SATD items. Furthermore, we proposed an innovative approach for Predicting Repayment Effort of SATD using Textual Information, named PRESTI. Our findings show that different types of SATD require varying levels of repayment effort, with code/design, requirement, and test debt demanding greater effort compared to non-SATD items, while documentation debt requires less. We have evaluated our approaches, particularly BERT- and TextCNN-based models, which outperform traditional machine learning methods and the baseline in estimating repayment effort. Additionally, we summarize keywords associated with varying levels of repayment effort that occur during SATD repayment. Our work aims to enhance SATD repayment prioritization and resource allocation, thereby improving software development and maintainability.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

Large new SATD dataset from Apache commits with text-based effort prediction models, but the commit-interval labels are too noisy for the performance claims to hold.

read the letter

Colleague, the paper curates 341k SATD items across 1060 Apache repos and trains BERT and TextCNN models to predict repayment effort from the debt comment text alone. That dataset and the new prediction task are the actual new pieces. They also break out effort by SATD type and find code/design and requirement debt cost more than non-SATD while documentation debt costs less; that analysis is straightforward from the commit data and looks reproducible. The models beat their traditional ML baselines and a simple baseline on held-out data. The soft spot is the target variable. Effort is measured by elapsed time plus code-change volume between the introducing commit and the removing commit. That interval routinely mixes in unrelated changes, parallel work, and project batching, so the labels carry substantial noise unrelated to the SATD text. Training models to predict that noisy proxy means the reported gains may not reflect real repayment cost. The abstract gives no error bars, no validation of the proxy, and no discussion of selection effects in which SATD items get removed. This is for the mining-software-repositories and technical-debt crowd who want to explore automated prioritization. It deserves peer review because the scale and task are fresh, even though the labeling step will need tightening before the results can be trusted.

Referee Report

3 major / 2 minor

Summary. The paper curates a dataset of 341,740 SATD items from 2,568,728 commits across 1,060 Apache repositories, analyzes differences in repayment effort between SATD and non-SATD items and across SATD types (code/design, requirement, test, documentation), and proposes the PRESTI approach in which BERT- and TextCNN-based models predict repayment effort from SATD textual descriptions, outperforming traditional ML methods and a baseline.

Significance. If the commit-interval proxy for repayment effort is shown to be reliable, the work supplies both large-scale empirical comparisons of effort by SATD type and deployable text-based predictors that could improve prioritization of debt repayment in practice.

major comments (3)

[§4, §5] §4 (dataset construction) and §5 (effort quantification): repayment effort is defined via elapsed time plus code-change volume between the introducing commit and the removing commit; no validation against actual developer-reported effort or controls for unrelated changes within the interval are described, directly affecting the validity of the target variable used to train and evaluate all models.
[§6] §6 (model evaluation): the claim that BERT- and TextCNN-based models outperform traditional ML and the baseline is presented without error bars, statistical significance tests across folds, or ablation on the effect of noisy labels, making it impossible to assess whether the reported superiority is robust to the acknowledged selection and measurement issues in the labeling process.
[§3, §4] §3 and §4: the curation process for identifying SATD-removal commits relies on commit messages and code diffs; potential selection bias (e.g., only easily detectable removals are labeled) is not quantified or mitigated, which is load-bearing for both the comparative analysis of effort by SATD type and the downstream prediction task.

minor comments (2)

[Table 1, Figure 2] Table 1 and Figure 2: axis labels and legends use inconsistent abbreviations for effort metrics; add explicit definitions in the caption.
[§2] §2 (related work): several recent SATD repayment studies are cited but the discussion does not contrast the commit-interval proxy against alternative effort measures used in those works.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for their detailed and constructive feedback on our manuscript. We address each of the major comments point by point below, indicating where revisions will be made to strengthen the paper.

read point-by-point responses

Referee: [§4, §5] §4 (dataset construction) and §5 (effort quantification): repayment effort is defined via elapsed time plus code-change volume between the introducing commit and the removing commit; no validation against actual developer-reported effort or controls for unrelated changes within the interval are described, directly affecting the validity of the target variable used to train and evaluate all models.

Authors: We acknowledge that our proxy for repayment effort, based on the time interval and code changes between the SATD introduction and removal commits, has not been validated against actual developer-reported effort metrics, and we do not control for unrelated changes in the interval. This is a known limitation when relying on commit history data at this scale, as direct effort reports are not available in the repositories studied. In the revised manuscript, we will expand the Threats to Validity section to explicitly discuss this proxy's limitations and potential confounding factors. We note that similar proxies have been employed in prior MSR studies on technical debt. revision: partial
Referee: [§6] §6 (model evaluation): the claim that BERT- and TextCNN-based models outperform traditional ML and the baseline is presented without error bars, statistical significance tests across folds, or ablation on the effect of noisy labels, making it impossible to assess whether the reported superiority is robust to the acknowledged selection and measurement issues in the labeling process.

Authors: We agree that the evaluation section would benefit from additional statistical rigor. In the revised version, we will include error bars representing standard deviations across cross-validation folds, conduct statistical significance tests (such as paired t-tests or Wilcoxon tests) to compare model performances, and perform an ablation study to examine the sensitivity to noisy labels in the dataset. revision: yes
Referee: [§3, §4] §3 and §4: the curation process for identifying SATD-removal commits relies on commit messages and code diffs; potential selection bias (e.g., only easily detectable removals are labeled) is not quantified or mitigated, which is load-bearing for both the comparative analysis of effort by SATD type and the downstream prediction task.

Authors: The SATD identification and removal detection indeed depends on patterns in commit messages and code diffs, which may introduce selection bias toward more explicitly documented cases. To address this, we will add a manual validation study on a sample of detected SATD items to quantify precision and recall of our curation process, and discuss the implications for our findings in the revised manuscript. revision: partial

Circularity Check

0 steps flagged

No circularity: standard supervised prediction on independently derived labels

full rationale

The paper curates SATD items and derives repayment-effort labels from commit intervals and code changes, then trains text-based models (BERT, TextCNN) to predict those labels on held-out data. This is ordinary supervised learning with no self-definitional loop, no fitted parameter renamed as a prediction, and no load-bearing self-citation or uniqueness theorem. Performance numbers are not forced by construction; they reflect generalization on unseen instances. The skeptic concern about label noise is a validity issue, not a circularity issue.

Axiom & Free-Parameter Ledger

1 free parameters · 1 axioms · 0 invented entities

The central claim rests on an unvalidated mapping from commit history to repayment effort and on standard supervised learning assumptions that are not detailed in the abstract.

free parameters (1)

effort quantification rule
The exact formula or threshold used to turn commit diffs into an effort label is not stated.

axioms (1)

domain assumption Subsequent commits that modify the same code region can be treated as repayment of the admitted debt.
This labeling rule is required to create the training targets from raw commit data.

pith-pipeline@v0.9.0 · 5834 in / 1159 out tokens · 30127 ms · 2026-05-24T07:08:32.955406+00:00 · methodology

discussion (0)

Reference graph

Works this paper leans on

32 extracted references · 32 canonical work pages · 1 internal anchor

[1]

Cunningham, The wycash portfolio management system, addendum to the proceedings on object-oriented programming systems, languages, and applications (addendum), 1992

W. Cunningham, The wycash portfolio management system, addendum to the proceedings on object-oriented programming systems, languages, and applications (addendum), 1992

work page 1992
[2]

Avgeriou, P

P. Avgeriou, P. Kruchten, I. Ozkaya, C. Seaman, Managing Techni- cal Debt in Software Engineering (Dagstuhl Seminar 16162), Dagstuhl Reports 6 (2016) 110–138. doi: 10.4230/DagRep.6.4.110

work page doi:10.4230/dagrep.6.4.110 2016
[3]

Potdar, E

A. Potdar, E. Shihab, An exploratory study on self-admitted technical debt, in: 2014 IEEE International Conference on Software Maintenance and Evolution, IEEE, 2014, pp. 91–100

work page 2014
[4]

Y. Li, M. Soliman, P. Avgeriou, Identification and Remediation of Self-Admitted Technical Debt in Issue Trackers, Proceedings - 46th Euromicro Conference on Software Engineering and Advanced Appli- cations, SEAA 2020 (2020) 495–503. doi: 10.1109/SEAA51224.2020. 00083. arXiv:2007.01568. 32

work page doi:10.1109/seaa51224.2020 2020
[5]

X. Ren, Z. Xing, X. Xia, D. Lo, X. Wang, J. Grundy, Neural network- based detection of self-admitted technical debt: From performance to explainability, ACM Transactions on Software Engineering and Method- ology (TOSEM) 28 (2019) 1–45

work page 2019
[6]

Y. Li, M. Soliman, P. Avgeriou, Identifying self-admitted technical debt in issue tracking systems using machine learning, Empirical Software Engineering 27 (2022). doi: 10.1007/s10664-022-10128-3

work page doi:10.1007/s10664-022-10128-3 2022
[7]

Y. Li, M. Soliman, P. Avgeriou, Automatic identification of self- admitted technical debt from four different sources, Empirical Software Engineering 28 (2023). doi: 10.1007/s10664-023-10297-9

work page doi:10.1007/s10664-023-10297-9 2023
[8]

Y. Li, M. Soliman, P. Avgeriou, L. Somers, Self-admitted technical debt in the embedded systems industry: An exploratory case study, IEEE Transactions on Software Engineering (2022) 1–22. doi: 10.1109/TSE. 2022.3224378

work page doi:10.1109/tse 2022
[9]

Fluri, H

B. Fluri, H. C. Gall, Classifying change types for qualifying change couplings, in: 14th IEEE International Conference on Program Com- prehension (ICPC’06), IEEE, 2006, pp. 35–45

work page 2006
[10]

Fluri, M

B. Fluri, M. Wursch, M. PInzger, H. Gall, Change distilling: Tree dif- ferencing for fine-grained source code change extraction, IEEE Trans- actions on software engineering 33 (2007) 725–743

work page 2007
[11]

Y. Li, M. Soliman, P. Avgeriou, Replication package for presti: Pre- dicting repayment effort of self-admitted technical debt using tex- tual information, https://www.dropbox.com/sh/4or1uixlakgcq2z/ AAAsYl67aK-r_VUDBB6XDDsBa?dl=0, 2024

work page 2024
[12]

E. d. S. Maldonado, E. Shihab, Detecting and quantifying different types of self-admitted technical debt, in: 2015 IEEE 7th International Workshop on Managing Technical Debt (MTD), IEEE, 2015, pp. 9–15

work page 2015
[13]

E. d. S. Maldonado, R. Abdalkareem, E. Shihab, A. Serebrenik, An empirical study on the removal of self-admitted technical debt, in: 2017 IEEE International Conference on Software Maintenance and Evolution (ICSME), IEEE, 2017, pp. 238–248. 33

work page 2017
[14]

Zampetti, A

F. Zampetti, A. Serebrenik, M. Di Penta, Was self-admitted techni- cal debt removal a real removal? an in-depth perspective, in: 2018 IEEE/ACM 15th International Conference on Mining Software Reposi- tories (MSR), IEEE, 2018, pp. 526–536

work page 2018
[15]

L. Xiao, Y. Cai, R. Kazman, R. Mo, Q. Feng, Identifying and quantifying architectural debt, in: Proceedings of the 38th international conference on software engineering, 2016, pp. 488–498

work page 2016
[16]

Martini, E

A. Martini, E. Sikander, N. Madlani, A semi-automated framework for the identification and estimation of architectural technical debt: A comparative case-study on the modularization of a software component, Information and Software Technology 93 (2018) 264–279

work page 2018
[17]

Nugroho, J

A. Nugroho, J. Visser, T. Kuipers, An empirical model of technical debt and interest, in: Proceedings of the 2nd workshop on managing technical debt, 2011, pp. 1–8

work page 2011
[18]

Mensah, J

S. Mensah, J. Keung, J. Svajlenko, K. E. Bennin, Q. Mi, On the value of a prioritization scheme for resolving self-admitted technical debt, Jour- nal of Systems and Software 135 (2018) 37–54

work page 2018
[19]

Wehaibi, E

S. Wehaibi, E. Shihab, L. Guerrouj, Examining the impact of self- admitted technical debt on software quality, in: 2016 IEEE 23Rd inter- national conference on software analysis, evolution, and reengineering (SANER), volume 1, IEEE, 2016, pp. 179–188

work page 2016
[20]

van Solingen, V

R. van Solingen, V. Basili, G. Caldiera, H. D. Rombach, Goal Question Metric (GQM) approach, in: Encyclopedia of Software Eng., John Wiley & Sons, Inc., Hoboken, NJ, USA, 2002, pp. 528–532

work page 2002
[21]

Q. Liu, Z. Liu, H. Zhu, H. Fan, B. Du, Y. Qian, Generating commit mes- sages from diffs using pointer-generator network, in: 2019 IEEE/ACM 16th International Conference on Mining Software Repositories (MSR), IEEE, 2019, pp. 299–309

work page 2019
[22]

Brown, Y

N. Brown, Y. Cai, Y. Guo, R. Kazman, M. Kim, P. Kruchten, E. Lim, A. MacCormack, R. Nord, I. Ozkaya, et al., Managing technical debt in software-reliant systems, in: Proceedings of the FSE/SDP workshop on Future of software engineering research, 2010, pp. 47–52. 34

work page 2010
[23]

Nguyen, S

V. Nguyen, S. Deeds-Rubin, T. Tan, B. Boehm, A sloc counting stan- dard, in: Cocomo ii forum, volume 2007, Citeseer, 2007, pp. 1–16

work page 2007
[24]

Herraiz, G

I. Herraiz, G. Robles, J. M. Gonz´ alez-Barahona, A. Capiluppi, J. F. Ramil, Comparison between slocs and number of files as size metrics for software evolution analysis, in: Conference on Software Maintenance and Reengineering (CSMR’06), IEEE, 2006, pp. 8–pp

work page 2006
[25]

Y. Li, M. Soliman, P. Avgeriou, Automatically identifying relations between self-admitted technical debt across different sources, arXiv preprint arXiv:2303.07079 (2023)

work page arXiv 2023
[26]

Kim, Convolutional neural networks for sentence classification, in: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2014, pp

Y. Kim, Convolutional neural networks for sentence classification, in: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2014, pp. 1746–1751

work page 2014
[27]

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

J. Devlin, M.-W. Chang, K. Lee, K. Toutanova, Bert: Pre-training of deep bidirectional transformers for language understanding, arXiv preprint arXiv:1810.04805 (2018)

work page internal anchor Pith review Pith/arXiv arXiv 2018
[28]

da Silva Maldonado, E

E. da Silva Maldonado, E. Shihab, N. Tsantalis, Using natural language processing to automatically detect self-admitted technical debt, IEEE Transactions on Software Engineering 43 (2017) 1044–1062

work page 2017
[29]

Kitchenham, L

B. Kitchenham, L. Madeyski, D. Budgen, J. Keung, P. Brereton, S. Charters, S. Gibbs, A. Pohthong, Robust statistical methods for em- pirical software engineering, Empirical Software Engineering 22 (2017) 579–630

work page 2017
[30]

Schmoch, Mean values of skewed distributions in the bibliometric assessment of research units, Scientometrics 125 (2020) 925–935

U. Schmoch, Mean values of skewed distributions in the bibliometric assessment of research units, Scientometrics 125 (2020) 925–935

work page 2020
[31]

H. B. Mann, D. R. Whitney, On a test of whether one of two random variables is stochastically larger than the other, The annals of mathe- matical statistics (1947) 50–60

work page 1947
[32]

Tantithamthavorn, S

C. Tantithamthavorn, S. McIntosh, A. E. Hassan, K. Matsumoto, The impact of automated parameter optimization on defect prediction mod- els, IEEE Transactions on Software Engineering 45 (2018) 683–711. 35

work page 2018

[1] [1]

Cunningham, The wycash portfolio management system, addendum to the proceedings on object-oriented programming systems, languages, and applications (addendum), 1992

W. Cunningham, The wycash portfolio management system, addendum to the proceedings on object-oriented programming systems, languages, and applications (addendum), 1992

work page 1992

[2] [2]

Avgeriou, P

P. Avgeriou, P. Kruchten, I. Ozkaya, C. Seaman, Managing Techni- cal Debt in Software Engineering (Dagstuhl Seminar 16162), Dagstuhl Reports 6 (2016) 110–138. doi: 10.4230/DagRep.6.4.110

work page doi:10.4230/dagrep.6.4.110 2016

[3] [3]

Potdar, E

A. Potdar, E. Shihab, An exploratory study on self-admitted technical debt, in: 2014 IEEE International Conference on Software Maintenance and Evolution, IEEE, 2014, pp. 91–100

work page 2014

[4] [4]

Y. Li, M. Soliman, P. Avgeriou, Identification and Remediation of Self-Admitted Technical Debt in Issue Trackers, Proceedings - 46th Euromicro Conference on Software Engineering and Advanced Appli- cations, SEAA 2020 (2020) 495–503. doi: 10.1109/SEAA51224.2020. 00083. arXiv:2007.01568. 32

work page doi:10.1109/seaa51224.2020 2020

[5] [5]

X. Ren, Z. Xing, X. Xia, D. Lo, X. Wang, J. Grundy, Neural network- based detection of self-admitted technical debt: From performance to explainability, ACM Transactions on Software Engineering and Method- ology (TOSEM) 28 (2019) 1–45

work page 2019

[6] [6]

Y. Li, M. Soliman, P. Avgeriou, Identifying self-admitted technical debt in issue tracking systems using machine learning, Empirical Software Engineering 27 (2022). doi: 10.1007/s10664-022-10128-3

work page doi:10.1007/s10664-022-10128-3 2022

[7] [7]

Y. Li, M. Soliman, P. Avgeriou, Automatic identification of self- admitted technical debt from four different sources, Empirical Software Engineering 28 (2023). doi: 10.1007/s10664-023-10297-9

work page doi:10.1007/s10664-023-10297-9 2023

[8] [8]

Y. Li, M. Soliman, P. Avgeriou, L. Somers, Self-admitted technical debt in the embedded systems industry: An exploratory case study, IEEE Transactions on Software Engineering (2022) 1–22. doi: 10.1109/TSE. 2022.3224378

work page doi:10.1109/tse 2022

[9] [9]

Fluri, H

B. Fluri, H. C. Gall, Classifying change types for qualifying change couplings, in: 14th IEEE International Conference on Program Com- prehension (ICPC’06), IEEE, 2006, pp. 35–45

work page 2006

[10] [10]

Fluri, M

B. Fluri, M. Wursch, M. PInzger, H. Gall, Change distilling: Tree dif- ferencing for fine-grained source code change extraction, IEEE Trans- actions on software engineering 33 (2007) 725–743

work page 2007

[11] [11]

Y. Li, M. Soliman, P. Avgeriou, Replication package for presti: Pre- dicting repayment effort of self-admitted technical debt using tex- tual information, https://www.dropbox.com/sh/4or1uixlakgcq2z/ AAAsYl67aK-r_VUDBB6XDDsBa?dl=0, 2024

work page 2024

[12] [12]

E. d. S. Maldonado, E. Shihab, Detecting and quantifying different types of self-admitted technical debt, in: 2015 IEEE 7th International Workshop on Managing Technical Debt (MTD), IEEE, 2015, pp. 9–15

work page 2015

[13] [13]

E. d. S. Maldonado, R. Abdalkareem, E. Shihab, A. Serebrenik, An empirical study on the removal of self-admitted technical debt, in: 2017 IEEE International Conference on Software Maintenance and Evolution (ICSME), IEEE, 2017, pp. 238–248. 33

work page 2017

[14] [14]

Zampetti, A

F. Zampetti, A. Serebrenik, M. Di Penta, Was self-admitted techni- cal debt removal a real removal? an in-depth perspective, in: 2018 IEEE/ACM 15th International Conference on Mining Software Reposi- tories (MSR), IEEE, 2018, pp. 526–536

work page 2018

[15] [15]

L. Xiao, Y. Cai, R. Kazman, R. Mo, Q. Feng, Identifying and quantifying architectural debt, in: Proceedings of the 38th international conference on software engineering, 2016, pp. 488–498

work page 2016

[16] [16]

Martini, E

A. Martini, E. Sikander, N. Madlani, A semi-automated framework for the identification and estimation of architectural technical debt: A comparative case-study on the modularization of a software component, Information and Software Technology 93 (2018) 264–279

work page 2018

[17] [17]

Nugroho, J

A. Nugroho, J. Visser, T. Kuipers, An empirical model of technical debt and interest, in: Proceedings of the 2nd workshop on managing technical debt, 2011, pp. 1–8

work page 2011

[18] [18]

Mensah, J

S. Mensah, J. Keung, J. Svajlenko, K. E. Bennin, Q. Mi, On the value of a prioritization scheme for resolving self-admitted technical debt, Jour- nal of Systems and Software 135 (2018) 37–54

work page 2018

[19] [19]

Wehaibi, E

S. Wehaibi, E. Shihab, L. Guerrouj, Examining the impact of self- admitted technical debt on software quality, in: 2016 IEEE 23Rd inter- national conference on software analysis, evolution, and reengineering (SANER), volume 1, IEEE, 2016, pp. 179–188

work page 2016

[20] [20]

van Solingen, V

R. van Solingen, V. Basili, G. Caldiera, H. D. Rombach, Goal Question Metric (GQM) approach, in: Encyclopedia of Software Eng., John Wiley & Sons, Inc., Hoboken, NJ, USA, 2002, pp. 528–532

work page 2002

[21] [21]

Q. Liu, Z. Liu, H. Zhu, H. Fan, B. Du, Y. Qian, Generating commit mes- sages from diffs using pointer-generator network, in: 2019 IEEE/ACM 16th International Conference on Mining Software Repositories (MSR), IEEE, 2019, pp. 299–309

work page 2019

[22] [22]

Brown, Y

N. Brown, Y. Cai, Y. Guo, R. Kazman, M. Kim, P. Kruchten, E. Lim, A. MacCormack, R. Nord, I. Ozkaya, et al., Managing technical debt in software-reliant systems, in: Proceedings of the FSE/SDP workshop on Future of software engineering research, 2010, pp. 47–52. 34

work page 2010

[23] [23]

Nguyen, S

V. Nguyen, S. Deeds-Rubin, T. Tan, B. Boehm, A sloc counting stan- dard, in: Cocomo ii forum, volume 2007, Citeseer, 2007, pp. 1–16

work page 2007

[24] [24]

Herraiz, G

I. Herraiz, G. Robles, J. M. Gonz´ alez-Barahona, A. Capiluppi, J. F. Ramil, Comparison between slocs and number of files as size metrics for software evolution analysis, in: Conference on Software Maintenance and Reengineering (CSMR’06), IEEE, 2006, pp. 8–pp

work page 2006

[25] [25]

Y. Li, M. Soliman, P. Avgeriou, Automatically identifying relations between self-admitted technical debt across different sources, arXiv preprint arXiv:2303.07079 (2023)

work page arXiv 2023

[26] [26]

Kim, Convolutional neural networks for sentence classification, in: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2014, pp

Y. Kim, Convolutional neural networks for sentence classification, in: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2014, pp. 1746–1751

work page 2014

[27] [27]

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

J. Devlin, M.-W. Chang, K. Lee, K. Toutanova, Bert: Pre-training of deep bidirectional transformers for language understanding, arXiv preprint arXiv:1810.04805 (2018)

work page internal anchor Pith review Pith/arXiv arXiv 2018

[28] [28]

da Silva Maldonado, E

E. da Silva Maldonado, E. Shihab, N. Tsantalis, Using natural language processing to automatically detect self-admitted technical debt, IEEE Transactions on Software Engineering 43 (2017) 1044–1062

work page 2017

[29] [29]

Kitchenham, L

B. Kitchenham, L. Madeyski, D. Budgen, J. Keung, P. Brereton, S. Charters, S. Gibbs, A. Pohthong, Robust statistical methods for em- pirical software engineering, Empirical Software Engineering 22 (2017) 579–630

work page 2017

[30] [30]

Schmoch, Mean values of skewed distributions in the bibliometric assessment of research units, Scientometrics 125 (2020) 925–935

U. Schmoch, Mean values of skewed distributions in the bibliometric assessment of research units, Scientometrics 125 (2020) 925–935

work page 2020

[31] [31]

H. B. Mann, D. R. Whitney, On a test of whether one of two random variables is stochastically larger than the other, The annals of mathe- matical statistics (1947) 50–60

work page 1947

[32] [32]

Tantithamthavorn, S

C. Tantithamthavorn, S. McIntosh, A. E. Hassan, K. Matsumoto, The impact of automated parameter optimization on defect prediction mod- els, IEEE Transactions on Software Engineering 45 (2018) 683–711. 35

work page 2018