Pith. sign in

REVIEW 2 major objections 4 minor 81 references

Negativity in Self-Admitted Technical Debt: How Sentiment Influences Prioritization

T0 review · 2 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that negative sentiment in self-admitted technical debt comments causally raises the priority that a sizable minority of developers assign to fixing the debt, even though most developers say such a cue should not be used.

desk verdict A transparent vignette experiment that gives the first controlled behavioral evidence that negative sentiment in SATD comments raises prioritization for a subset of developers, but the manipulation is too confounded to support the abstract's causal wording. read the letter →

arxiv 2501.01068 v1 pith:NMR73CFP submitted 2025-01-02 cs.SE

Nathan Cassee , Neil Ernst , Nicole Novielli , Alexander Serebrenik This is my paper · ORCID
classification cs.SE
keywords self-admittedtechnicaldebtSATDsentimentanalysisnegativityprioritizationvignetteexperimentBayesianordered-logitmodeldeveloperperception
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether negative sentiment in self-admitted technical debt (SATD) comments—comments where developers admit a suboptimal implementation—actually changes how developers prioritize fixing it. In a vignette experiment with 59 developers and students, each participant rated four realistic SATD snippets, two phrased neutrally and two with negative emotion. The authors report that between one-third and half of respondents gave higher priority scores to the negative versions, and that affected developers were 1.4 to 2.0 times more likely to raise rather than lower their ratings of urgency, importance, or effort. At the same time, two-thirds of the same developers said using negativity as a priority signal is unacceptable. The finding matters because it suggests technical debt can end up prioritized for emotional rather than technical reasons, potentially distorting backlog decisions.

What carries the argument

The machinery is a between-person experimental vignette design combined with Bayesian ordered-logit models. Four real SATD comments from the Poor Implementation Choices category were paired with ChatGPT-generated counterparts that flip the sentiment while trying to preserve meaning; each participant saw two negative and two neutral vignettes, with whether a comment was ChatGPT-generated (Manipulated) treated as a blocking factor. Priority was operationalized as three Likert-scaled constructs—urgency, importance, and effort—and each outcome was modeled with an ordered logit adjusted for sentiment, manipulation, perception, and experience, following a directed acyclic graph that identifies the confounders. The model's posterior contrasts produce the odds ratios and evidence ratios that quantify how often negativity changes a score.

What would settle it

Have an independent panel rate the four neutral and four negative vignettes for technical severity, problem size, and code quality without being told the hypothesis; if the negative versions are systematically rated as worse or more extensive problems, the observed prioritization shift cannot be cleanly attributed to negativity alone.

Watch

Extended reading notes

Core claim

The paper's central claim is that negativity in a SATD comment is not neutral packaging: it changes how the same technical debt is judged. In an experiment where 59 developers and students rated four realistic SATD vignettes, two neutral and two negative, between one-third and half of respondents assigned higher priority to negative versions. Developers who agreed that negativity signals importance were 1.4 times as likely to raise effort, 1.95 times as likely to raise urgency, and 1.5 times as likely to raise importance as to lower those scores; respondents with no opinion were 1.56 times as likely to raise effort, while respondents who disagreed showed no effect. The authors conclude that negativity acts as an additional, largely implicit communication channel for priority, and they highlight the gap between this behavior and the 67% of developers who consider using negativity as a proxy for priority unacceptable.

Load-bearing premise

The causal conclusion assumes each neutral and negative version of a vignette differs only in emotional tone, not in how severe, extensive, or blameworthy the technical problem appears.

Editorial extensions

If this is right

  • A negative SATD comment can make the same underlying technical issue look more urgent, important, or effortful to a substantial minority of developers.
  • The effect is concentrated among developers who believe negativity signals importance; those without an opinion still raise effort estimates, while those who disagree show no measurable shift.
  • Because roughly a quarter of SATD comments in the wild carry negative sentiment, emotional tone may be silently shaping which technical debt gets addressed first.
  • Most developers find the practice unacceptable, so negativity is not a reliable or agreed-upon coordination signal for team prioritization; the paper advises against using it as a proxy.
  • The authors expect the effect to generalize beyond source-code comments to technical debt described in issue trackers, because the underlying judgment mechanism is the same.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the effect replicates in field settings, triage processes that strip or downweight sentiment-laden wording before prioritization could reduce emotion-driven bias; the paper does not test such interventions.
  • A natural next test is whether negative SATD comments are actually resolved faster in repositories; the paper lists this as future work, and a large-scale mining study of issue-tracker resolution times would extend the causal claim to the field.
  • The manipulated negative comments contain words like 'junk,' 'mess,' and 'ugly code' that may carry severity information, so an independent rating of the perceived severity of the paired vignettes would separate emotional tone from content; this is an open question the experiment does not fully close.
  • Because the design is between-person, developers never compare the two versions side by side; a within-subject replication with the same developer rating both variants could show whether the effect strengthens or weakens when the sentiment contrast is explicit.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper asks whether negative sentiment expressed in self-admitted technical debt (SATD) comments causally affects developers' prioritization judgments. The authors conduct a vignette experiment with 59 respondents, using four real SATD comments with ChatGPT-generated variants that swap sentiment while attempting to preserve meaning, and they ask respondents to rate urgency, importance, and effort. They fit Bayesian ordered-logit models adjusting for sentiment, perception, manipulated status, and experience, and they report evidence ratios and odds-ratio contrasts. They conclude that one-third to half of developers assign higher priority to negatively phrased SATD, that affected developers are 1.4-2.0 times more likely to increase than decrease their scores, and that most developers nonetheless consider the practice unacceptable.

Significance. If the causal claim holds, the paper provides novel controlled experimental evidence on a topic that has previously been studied mainly through correlational survey and repository work; it could inform tooling and team practices for SATD triage. The manuscript is transparent about its design and limitations, provides a replication package, uses explicit Bayesian priors with prior predictive checks, and reports evidence ratios alongside parameter estimates. The main unresolved issue is treatment integrity: the sentiment manipulation appears to vary more than sentiment alone, and the statistical model does not fully account for vignette- and participant-level structure.

major comments (2)
  1. [§2.2, Table 2; §2.3, Model 1] The causal reading of the Sentiment coefficient requires that each negative–neutral pair differ only in expressed sentiment. Table 2 shows this is not established: for example, Vignette #1 contrasts "obviously get rid of all this junk" with "clearly remove all this unnecessary code"; Vignette #3 contrasts "Ugh, ConstDecl is a mess ... never in sync" with "could be two separate classes ... never exist at the same time"; Vignette #4 adds "really ugly code" and "I should have written something much cooler" alongside neutral wording about manual updates. These changes plausibly alter perceived severity and required effort, not only emotional tone. The binary Manipulated regressor in Model 1 cannot absorb pair-specific semantic drift, and the Technical Debt node shown in Figure 2 is not included in Model 1 as a predictor or random effect. A manipulation check (for example, independent ratings of perceived negativity, severity, and effort of the comments) is needed before the effect can be attributed to negativity rather than to other properties of the reworded comments.
  2. [§2.2, Table 3; §2.3, Model 1, Tables 6–7] The paper describes the design as between-person, but Table 3 shows that each participant sees two negative and two neutral vignettes, so the sentiment contrast is within-subject. Model 1 nevertheless treats the 236 vignette ratings as independent observations and includes no participant-level or vignette-level random effects. Each respondent contributes four ratings and each vignette is rated by many respondents, so the posterior intervals and evidence ratios in Tables 6 and 7 are likely too narrow. The authors should fit multilevel ordered-logit models with random intercepts for participant and vignette, or otherwise account for clustering, and report whether the odds-ratio conclusions survive this correction.
minor comments (4)
  1. [§2.3, paragraph after Table 4] The sentence "if the priors are unbiased, we expect the odds ratios to be close to zero" should read "close to one," since Table 4 reports values near 1.0 and an odds ratio of zero would mean an effect in only one direction.
  2. [Table 2 and Figure 4] There are small presentation errors: "seperate" in Vignette #3 should be "separate," and the Figure 4 captions render "Distribution" as "Di tribution."
  3. [§3.2, Table 6] Table 6 is labeled "Evidence ratio," while the text says "Bayes factor"; these are not interchangeable without defining prior odds, so the terminology should be made consistent.
  4. [Abstract and §1] The abstract states that "about a quarter" of SATD descriptions express negativity, while the introduction says "roughly 20%"; the figures should be aligned or the discrepancy explained.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the causal estimate is produced from new experimental data, and reused prior material is not load-bearing.

full rationale

The paper's load-bearing claim—that negative sentiment raises some developers' prioritization scores—is estimated from newly collected experimental data, not from the authors' earlier survey or from the modeling assumptions. Participants were shown manipulated and neutral SATD vignettes and gave urgency, importance, and effort ratings; the Bayesian ordered-logit model (Model 1) then estimates the Sentiment contrast while adjusting for Manipulated, Perception, and Experience. The odds ratios in Table 7 and the 1.4–2.0x claim are posterior contrasts of these ratings, not algebraic restatements of any input. The prior predictive check confirms the priors are centered at zero, so the effect is not baked into the model. Reused material from Cassee et al. (2022)—the SATD dataset, sentiment labels, and perception questions—serves as motivation and covariate construction, not as the outcome; the causal estimate depends on the experimental responses rather than on those earlier results being true. The ChatGPT manipulation may introduce semantic confounds (e.g., "junk" versus "unnecessary code"), but the authors acknowledge this risk and include Manipulated as a blocking factor; this is a construct-validity threat, not circularity. No equation defines the result in terms of its inputs, and no load-bearing claim reduces to a self-citation. Verdict: no significant circularity.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

No new theoretical entities are introduced. The causal estimate rests on the experimental design, the Bayesian ordered-logit model with weakly informative priors, the validity of the ChatGPT sentiment manipulation, and the representativeness of the convenience sample. The coefficients listed are estimated from the data, not derived from first principles.

free parameters (5)
  • Ordered-logit cutpoints (kappa) = not reported
    Thresholds of the ordinal regression estimated from the 236 vignette ratings; they map the linear predictor to the five Likert categories.
  • alpha_effects (Sentiment by Perception) = not reported
    Coefficients for each combination of sentiment and participant perception; the posteriors in Figure 5 and odds ratios in Table 7 are functions of these fitted values.
  • alpha_manipulated = not reported
    Adjustment coefficient for whether the comment was generated by ChatGPT; estimated from data to account for wording effects.
  • alpha_experience = not reported
    Coefficient for years of programming experience; estimated from data.
  • Weakly informative prior scales for Normal priors = Normal(0, 0.5) for coefficients; Normal(0, 2) for cutpoints
    Chosen by the authors, not estimated from data; the prior predictive check is meant to show they do not bias the outcome.
assumptions (5)
  • domain assumption The DAG in Figure 2 correctly identifies the adjustment set for the causal effect of Sentiment on Prioritization.
    Section 2.3: the total effect is estimated by conditioning on Manipulated, Perception, and Experience; if the graph is misspecified, the estimated effect is biased.
  • domain assumption The ChatGPT-generated comments preserve the semantics of the original SATD and differ only in sentiment polarity.
    Section 2.2 Alternative generation; manual author voting is the only validation, and the paper acknowledges manipulation may affect scores.
  • domain assumption Respondents' Likert ratings of urgency, importance, and effort are valid proxies for real technical debt prioritization.
    Section 2.2 and Section 5 Construct Validity; the paper admits these constructs are not the ones used in practice by developers.
  • domain assumption The convenience sample of 59 developers is representative enough to support statements about developers in general.
    Section 3.1 compares age and experience to Stack Overflow survey respondents, but sampling was non-random and self-selected.
  • standard math The ordered-logit likelihood is appropriate for the five-point Likert outcomes.
    Section 2.3 Model 1; ordinal regression is standard for ordinal responses but the proportional-odds assumption is not tested.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Negativity in Self-Admitted Technical Debt: How Sentiment Influences Prioritization." pith.science (2026). https://pith.science/paper/NMR73CFP

@misc{pith2026250101068,
  author       = {Pith},
  title        = {Pith review of: Negativity in Self-Admitted Technical Debt: How Sentiment Influences Prioritization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NMR73CFP}},
  note         = {Machine review of arXiv:2501.01068}
}
read the original abstract

Self-Admitted Technical Debt, or SATD, is a self-admission of technical debt present in a software system. To effectively manage SATD, developers need to estimate its priority and assess the effort required to fix the described technical debt. About a quarter of descriptions of SATD in software systems express some form of negativity or negative emotions when describing technical debt. In this paper, we report on an experiment conducted with 59 respondents to study whether negativity expressed in the description of SATD \textbf{actually} affects the prioritization of SATD. The respondents are a mix of professional developers and students, and in the experiment, we asked participants to prioritize four vignettes: two expressing negativity and two expressing neutral sentiment. To ensure realism, vignettes were based on existing SATD. We find that negativity causes between one-third and half of developers to prioritize SATD, in which negativity is expressed as having more priority. Developers affected by negativity when prioritizing SATD are twice as likely to increase their estimation of urgency and 1.5 times as likely to increase their estimation of importance and effort for SATD compared to the likelihood of decreasing these prioritization scores. Our findings show how developers actively use negativity in SATD to determine how urgently a particular instance of TD should be addressed. However, our study also describes a gap in the actions and belief of developers. Even if 33% to 50% use negativity to prioritize SATD, 67% of developers believe that using negativity as a proxy for priority is unacceptable. Therefore, we would not recommend using negativity as a proxy for priority. However, we also recognize that developers might unavoidably express negativity when describing technical debt.

Figures

Figures reproduced from arXiv: 2501.01068 by the authors.

Figure 1
Figure 1. A Directed Acyclic Graph (DAG) illustrating the relation we are ana [PITH_FULL_IMAGE:figures/full_fig_p011_1.png] view at source ↗
Figure 2
Figure 2. The theoretical model visualized as a DAG. The color blue denotes the [PITH_FULL_IMAGE:figures/full_fig_p012_2.png] view at source ↗
Figure 3
Figure 3. Distributions of the priors for each of the outcome variables. [PITH_FULL_IMAGE:figures/full_fig_p016_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Histograms for respondent age and experience. [PITH_FULL_IMAGE:figures/full_fig_p017_4.png]
Figure 5
Figure 5. Figure 5: Distributions of the posterior for each of the outcome variables. [PITH_FULL_IMAGE:figures/full_fig_p018_5.png]
Figure 6
Figure 6. Figure 6: Responses to survey questions Q4, Q5, Q6. [PITH_FULL_IMAGE:figures/full_fig_p021_6.png]
Figure 7
Figure 7. Figure 7: Responses to survey questions Q7, Q8, Q9. [PITH_FULL_IMAGE:figures/full_fig_p021_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

81 extracted references · 25 canonical work pages

  1. [1]

    Organizational Research Methods 17(4):351--371, doi:10.1177/1094428114547952

    Aguinis H, Bradley KJ (2014) Best practice recommendations for designing and implementing experimental vignette methodology studies. Organizational Research Methods 17(4):351--371, doi:10.1177/1094428114547952

    work page doi:10.1177/1094428114547952 2014

  2. [2]

    Information and Software Technology 64:52--73, doi:10.1016/j.infsof.2015.04.001

    Ampatzoglou A, Ampatzoglou A, Chatzigeorgiou A, Avgeriou P (2015) The financial aspect of managing technical debt: A systematic literature review. Information and Software Technology 64:52--73, doi:10.1016/j.infsof.2015.04.001

    work page doi:10.1016/j.infsof.2015.04.001 2015

  3. [3]

    IEEE, pp 298--308, doi:10.1109/ICSE.2009.5070530

    Aranda J, Venolia G (2009) The secret life of bugs: Going past the errors and omissions in software repositories. IEEE, pp 298--308, doi:10.1109/ICSE.2009.5070530

    arXiv 2009

  4. [4]

    Information and Software Technology 114:37--54, doi:10.1016/j.infsof.2019.06.005

    Asri IE, Kerzazi N, Uddin G, Khomh F, Idrissi MAJ (2019) An empirical study of sentiments in code reviews. Information and Software Technology 114:37--54, doi:10.1016/j.infsof.2019.06.005

    doi:10.1016/j.infsof.2019.06.005 2019

  5. [5]

    Methodology 6(3):128--138, doi:10.1027/1614-2241/a000014

    Atzm\" u ller C, Steiner PM (2010) Experimental vignette studies in survey research. Methodology 6(3):128--138, doi:10.1027/1614-2241/a000014

    doi:10.1027/1614-2241/a000014 2010

  6. [6]

    Geography 91:43--54, doi:10.1080/00167487.2006.12094149

    Barr S (2006) Environmental action in the home: Investigating the ‘value-action’ gap. Geography 91:43--54, doi:10.1080/00167487.2006.12094149

    work page arXiv 2006

  7. [7]

    ACM, pp 735--742, doi:10.1145/985692.985785

    Bellotti V, Dalal B, Good N, Flynn P, Bobrow DG, Ducheneaut N (2004) What a to-do: studies of task management towards the design of a personal task list manager. ACM, pp 735--742, doi:10.1145/985692.985785

    arXiv 2004

  8. [8]

    Journal of Systems and Software 167:110586, doi:10.1016/j.jss.2020.110586

    Besker T, Ghanbari H, Martini A, Bosch J (2020) The influence of technical debt on software developer morale. Journal of Systems and Software 167:110586, doi:10.1016/j.jss.2020.110586

    work page arXiv 2020

Show all 81 references
  1. [9]

    Information and Software Technology 94:186--207, doi:10.1016/j.infsof.2017.10.009

    Calefato F, Lanubile F, Novielli N (2018) How to ask for technical help? evidence-based guidelines for writing questions on stack overflow. Information and Software Technology 94:186--207, doi:10.1016/j.infsof.2017.10.009

    2018 doi

  2. [10]

    Empirical Software Engineering 27:139, doi:10.1007/s10664-022-10183-w

    Cassee N, Zampetti F, Novielli N, Serebrenik A, Penta MD (2022) Self-admitted technical debt and comments’ polarity: an empirical study. Empirical Software Engineering 27:139, doi:10.1007/s10664-022-10183-w

    2022 doi

  3. [11]

    IEEE, pp 29--33, doi:10.1109/SEmotion.2019.00013

    Cheruvelil J, da Silva BC (2019) Developers' sentiment and issue reopening. IEEE, pp 29--33, doi:10.1109/SEmotion.2019.00013

    2019

  4. [12]

    SIGPLAN OOPS Mess 4(2):29--30, doi:10.1145/157710.157715

    Cunningham W (1992) The wycash portfolio management system. SIGPLAN OOPS Mess 4(2):29--30, doi:10.1145/157710.157715

    1992

  5. [13]

    IEEE, pp 537--548, doi:10.1109/ICSE43902.2021.00057

    Danilova A, Naiakshina A, Horstmann S, Smith M (2021) Do you really code? designing and evaluating screening questions for online surveys with programmers. IEEE, pp 537--548, doi:10.1109/ICSE43902.2021.00057

    2021

  6. [14]

    doi:10.1007/978-94-007-6094-3_13

    Elwert F (2013) Graphical causal models. doi:10.1007/978-94-007-6094-3_13

    2013 doi

  7. [15]

    2401.01154

    Frattini J, Fucci D, Torkar R, Montgomery L, Unterkalmsteiner M, Fischbach J, Mendez D (2024) Applying bayesian data analysis for causal inference about requirements quality: A replicated experiment. 2401.01154

    2024 arXiv

  8. [16]

    ACM Transactions on Software Engineering and Methodology 31:1--38, doi:10.1145/3490953

    Furia CA, Torkar R, Feldt R (2022) Applying bayesian analysis guidelines to empirical software engineering data: The case of programming languages and code quality. ACM Transactions on Software Engineering and Methodology 31:1--38, doi:10.1145/3490953

    2022 doi

  9. [17]

    IEEE, pp 11--14, doi:10.1109/ICSE-NIER.2017.18

    Gachechiladze D, Lanubile F, Novielli N, Serebrenik A (2017) Anger and its direction in collaborative software development. IEEE, pp 11--14, doi:10.1109/ICSE-NIER.2017.18

    2017 doi

  10. [18]

    IEEE Transactions on Software Engineering 47(10):2143--2161, doi:10.1109/TSE.2019.2944608

    Gavidia-Calderon C, Sarro F, Harman M, Barr ET (2021) The assessor's dilemma: Improving bug repair via empirical game theory. IEEE Transactions on Software Engineering 47(10):2143--2161, doi:10.1109/TSE.2019.2944608

    2021

  11. [19]

    Chapman and Hall/CRC, doi:10.1201/b16018

    Gelman A, Carlin JB, Stern HS, Dunson DB, Vehtari A, Rubin DB (2013) Bayesian Data Analysis. Chapman and Hall/CRC, doi:10.1201/b16018

    2013 doi

  12. [20]

    ://arxiv.org/abs/2011.01808

    Gelman A, Vehtari A, Simpson D, Margossian CC, Carpenter B, Yao Y, Kennedy L, Gabry J, Bürkner PC, Modrák M (2020) Bayesian workflow. ://arxiv.org/abs/2011.01808

    2020 arXiv

  13. [21]

    IEEE, pp 1405--1417, doi:10.1109/ICSE48619.2023.00123

    Ghorbani A, Cassee N, Robinson D, Alami A, Ernst NA, Serebrenik A, Wasowski A (2023) Autonomy is an acquired taste: Exploring developer preferences for github bots. IEEE, pp 1405--1417, doi:10.1109/ICSE48619.2023.00123

    2023

  14. [22]

    ACM Transactions on Software Engineering and Methodology 30:1--56, doi:10.1145/3447247

    Guo Z, Liu S, Liu J, Li Y, Chen L, Lu H, Zhou Y (2021) How far have we progressed in identifying self-admitted technical debts? a comprehensive empirical study. ACM Transactions on Software Engineering and Methodology 30:1--56, doi:10.1145/3447247

    2021 doi

  15. [23]

    IEEE, pp 371--376, doi:10.1109/REW53955.2021.00065

    Herrmann M, Klunder J (2021) From textual to verbal communication: Towards applying sentiment analysis to a software project meeting. IEEE, pp 371--376, doi:10.1109/REW53955.2021.00065

    2021

  16. [24]

    Psi Chi Journal of Psychological Research 21:138--151, doi:10.24839/2164-8204.JN21.3.138

    Hughes JL, Camden AA, Yangchen T (2016) Rethinking and updating demographic questions: Guidance to improve descriptions of research samples. Psi Chi Journal of Psychological Research 21:138--151, doi:10.24839/2164-8204.JN21.3.138

    2016 doi

  17. [26]

    Psychological Bulletin 114:3--28, doi:10.1037/0033-2909.114.1.3

    Johnson MK, Hashtroudi S, Lindsay DS (1993) Source monitoring. Psychological Bulletin 114:3--28, doi:10.1037/0033-2909.114.1.3

    1993 doi

  18. [27]

    doi:10.1007/978-1-4757-3304-4_9

    Juristo N, Moreno AM (2001) Experiments with undesired variations. doi:10.1007/978-1-4757-3304-4_9

    2001 doi

  19. [28]

    Information and Software Technology 146:106855, doi:10.1016/j.infsof.2022.106855

    Kashiwa Y, Nishikawa R, Kamei Y, Kondo M, Shihab E, Sato R, Ubayashi N (2022) An empirical study on self-admitted technical debt in modern code review. Information and Software Technology 146:106855, doi:10.1016/j.infsof.2022.106855

    2022

  20. [29]

    Current Directions in Psychological Science 18:184--188, doi:10.1111/j.1467-8721.2009.01633.x

    Kleef GAV (2009) How emotions regulate social life. Current Directions in Psychological Science 18:184--188, doi:10.1111/j.1467-8721.2009.01633.x

    2009

  21. [30]

    Psychonomic Bulletin & Review 25:178--206, doi:10.3758/s13423-016-1221-4

    Kruschke JK, Liddell TM (2018) The bayesian new statistics: Hypothesis testing, estimation, meta-analysis, and power analysis from a bayesian perspective. Psychonomic Bulletin & Review 25:178--206, doi:10.3758/s13423-016-1221-4

    2018 doi

  22. [31]

    Journal of Systems and Software 171:110827, doi:10.1016/j.jss.2020.110827

    Lenarduzzi V, Besker T, Taibi D, Martini A, Fontana FA (2021) A systematic literature review on technical debt prioritization: Strategies, processes, factors, and tools. Journal of Systems and Software 171:110827, doi:10.1016/j.jss.2020.110827

    2021

  23. [32]

    Journal of Systems and Software 101:193--220, doi:10.1016/j.jss.2014.12.027

    Li Z, Avgeriou P, Liang P (2015) A systematic mapping study on technical debt and its management. Journal of Systems and Software 101:193--220, doi:10.1016/j.jss.2014.12.027

    2015 doi

  24. [33]

    IEEE Software 29:22--27, doi:10.1109/MS.2012.130

    Lim E, Taksande N, Seaman C (2012) A balancing act: What software practitioners have to say about technical debt. IEEE Software 29:22--27, doi:10.1109/MS.2012.130

    2012 doi

  25. [34]

    ACM, pp 94--104, doi:10.1145/3180155.3180195

    Lin B, Zampetti F, Bavota G, Penta MD, Lanza M, Oliveto R (2018) Sentiment analysis for software engineering. ACM, pp 94--104, doi:10.1145/3180155.3180195

    2018

  26. [35]

    ACM Transactions on Software Engineering and Methodology 31:1--41, doi:10.1145/3490388

    Lin B, Cassee N, Serebrenik A, Bavota G, Novielli N, Lanza M (2022) Opinion mining for software development: A systematic literature review. ACM Transactions on Software Engineering and Methodology 31:1--41, doi:10.1145/3490388

    2022 doi

  27. [36]

    ACM, pp 9--12, doi:10.1145/3183440.3183478

    Liu Z, Huang Q, Xia X, Shihab E, Lo D, Li S (2018) Satd detector: a text-mining-based self-admitted technical debt detection tool. ACM, pp 9--12, doi:10.1145/3183440.3183478

    2018

  28. [37]

    2015 IEEE 7th International Workshop on Managing Technical Debt, MTD 2015 - Proceedings pp 9--15, doi:10.1109/MTD.2015.7332619

    Maldonado EDS, Shihab E (2015) Detecting and quantifying different types of self-admitted technical debt. 2015 IEEE 7th International Workshop on Managing Technical Debt, MTD 2015 - Proceedings pp 9--15, doi:10.1109/MTD.2015.7332619

    2015

  29. [38]

    IEEE, pp 238--248, doi:10.1109/ICSME.2017.8

    Maldonado EDS, Abdalkareem R, Shihab E, Serebrenik A (2017) An empirical study on the removal of self-admitted technical debt. IEEE, pp 238--248, doi:10.1109/ICSME.2017.8

    2017 doi

  30. [39]

    a ntyl \

    M \"a ntyl \"a M, Adams B, Destefanis G, Graziotin D, Ortu M (2016) Mining valence, arousal, and dominance - possibilities for detecting burnout and productivity? Proceedings - 13th Working Conference on Mining Software Repositories, MSR 2016 pp 247--258, doi:10.1145/2901739.2901752

    2016

  31. [40]

    Chapman and Hall/CRC, doi:10.1201/9781315372495

    McElreath R (2018) Statistical Rethinking. Chapman and Hall/CRC, doi:10.1201/9781315372495

    2018 doi

  32. [41]

    Future Healthc J 5(2):138--142

    Middleton S, Charnock A, Forster S, Blakey J (2018) Factors affecting -individual task prioritisation in a workplace setting. Future Healthc J 5(2):138--142

    2018

  33. [42]

    did you miss my comment or what?

    Miller C, Cohen S, Klug D, Vasilescu B, KaUstner C (2022) "did you miss my comment or what?": understanding toxicity in open source discussions. ACM, pp 710--722, doi:10.1145/3510003.3510111

    2022

  34. [43]

    IEEE Comput

    Molokken K, Jorgensen M (2003) A review of software surveys on software effort estimation. IEEE Comput. Soc, pp 223--230, doi:10.1109/ISESE.2003.1237981

    2003 arXiv

  35. [44]

    doi:10.1007/978-3-031-36060-2_5

    Novielli N, Serebrenik A (2023) Emotion analysis in software ecosystems. doi:10.1007/978-3-031-36060-2_5

    2023 doi

  36. [45]

    Frontiers in Human Neuroscience 9, doi:10.3389/fnhum.2015.00058

    Okon-Singer H, Hendler T, Pessoa L, Shackman AJ (2015) The neurobiology of emotion-cognition interactions: Fundamental questions and strategies for future research. Frontiers in Human Neuroscience 9, doi:10.3389/fnhum.2015.00058

    2015

  37. [46]

    Empirical Software Engineering 26:105, doi:10.1007/s10664-021-09998-w

    Olsson J, Risfelt E, Besker T, Martini A, Torkar R (2021) Measuring affective states from technical debt. Empirical Software Engineering 26:105, doi:10.1007/s10664-021-09998-w

    2021 doi

  38. [47]

    issue fixing time

    Ortu M, Adams B, Destefanis G, Tourani P, Marchesi M, Tonelli R (2015) Are bullies more productive? empirical study of affectiveness vs. issue fixing time. IEEE, vol 2015-Augus, pp 303--313, doi:10.1109/MSR.2015.35

    2015 doi

  39. [48]

    IEEE, pp 176--185, doi:10.1109/ICPC.2017.38

    Palomba F, Zaidman A, Oliveto R, Lucia AD (2017) An exploratory study on the relationship between changes and refactoring. IEEE, pp 176--185, doi:10.1109/ICPC.2017.38

    2017 doi

  40. [49]

    IEEE, pp 227--237, doi:10.1109/MSR.2017.63

    Pascarella L, Bacchelli A (2017) Classifying code comments in java open-source software systems. IEEE, pp 227--237, doi:10.1109/MSR.2017.63

    2017 doi

  41. [50]

    ACM, pp 127--131, doi:10.1145/3524842.3528527

    Peruma A, AlOmar EA, Newman CD, Mkaouer MW, Ouni A (2022) Refactoring debt: myth or reality? an exploratory study on the relationship between technical debt and refactoring. ACM, pp 127--131, doi:10.1145/3524842.3528527

    2022

  42. [51]

    IEEE, pp 91--100, doi:10.1109/ICSME.2014.31

    Potdar A, Shihab E (2014) An exploratory study on self-admitted technical debt. IEEE, pp 91--100, doi:10.1109/ICSME.2014.31

    2014 doi

  43. [52]

    ACM, pp 57--60, doi:10.1145/3377816.3381732

    Raman N, Cao M, Tsvetkov Y, Kästner C, Vasilescu B (2020) Stress and burnout in open source. ACM, pp 57--60, doi:10.1145/3377816.3381732

    2020

  44. [53]

    Journal of Experimental Psychology: General 152:3459--3475, doi:10.1037/xge0001462

    Raykov PP, Varga D, Bird CM (2023) False memories for ending of events. Journal of Experimental Psychology: General 152:3459--3475, doi:10.1037/xge0001462

    2023 doi

  45. [54]

    ACM Transactions on Software Engineering and Methodology 28:1--45, doi:10.1145/3324916

    Ren X, Xing Z, Xia X, Lo D, Wang X, Grundy J (2019) Neural network-based detection of self-admitted technical debt. ACM Transactions on Software Engineering and Methodology 28:1--45, doi:10.1145/3324916

    2019 doi

  46. [55]

    ACM Transactions on Software Engineering and Methodology doi:10.1145/3649598

    Robillard MP, Arya DM, Ernst NA, Guo JL, Lamothe M, Nassif M, Novielli N, Serebrenik A, Steinmacher I, Stol KJ (2024) Communicating study design trade-offs in software engineering. ACM Transactions on Software Engineering and Methodology doi:10.1145/3649598

    2024 doi

  47. [56]

    IEEE, pp 1--10, doi:10.1109/CHASE52884.2021.00009

    Sanei A, Cheng J, Adams B (2021) The impacts of sentiments and tones in community-generated issue discussions. IEEE, pp 1--10, doi:10.1109/CHASE52884.2021.00009

    2021

  48. [57]

    Empir Softw Eng 19(5):1299--1334, doi:10.1007/S10664-013-9286-4

    Siegmund J, K \" a stner C, Liebig J, Apel S, Hanenberg S (2014) Measuring and modeling programming experience. Empir Softw Eng 19(5):1299--1334, doi:10.1007/S10664-013-9286-4

    2014 doi

  49. [58]

    Proceedings - Asia-Pacific Software Engineering Conference, APSEC 2017-Decem:648--653, doi:10.1109/APSEC.2017.79

    Singh N, Singh P (2018) How do code refactoring activities impact software developers' sentiments? - an empirical investigation into github commits. Proceedings - Asia-Pacific Software Engineering Conference, APSEC 2017-Decem:648--653, doi:10.1109/APSEC.2017.79

    2018 doi

  50. [59]

    doi:10.1109/MSR59073.2023.00063

    Sridharan M, Rantala L, Mäntylä M (2023) Pentacet data-23 million contextual code comments and 250,000 satd comments. doi:10.1109/MSR59073.2023.00063

    2023

  51. [60]

    Journal of Personality and Social Psychology 69:797--811, doi:10.1037/0022-3514.69.5.797

    Steele CM, Aronson J (1995) Stereotype threat and the intellectual test performance of african americans. Journal of Personality and Social Psychology 69:797--811, doi:10.1037/0022-3514.69.5.797

    1995 doi

  52. [61]

    Behavior Research Methods 51:1042--1058, doi:10.3758/s13428-018-01189-8

    Stefan AM, Gronau QF, Schönbrodt FD, Wagenmakers EJ (2019) A tutorial on bayes factor design analysis using an informed prior. Behavior Research Methods 51:1042--1058, doi:10.3758/s13428-018-01189-8

    2019 doi

  53. [62]

    ACM Transactions on Software Engineering and Methodology 27, doi:10.1145/3241743

    Stol KJ, Fitzgerald B (2018) The abc of software engineering research. ACM Transactions on Software Engineering and Methodology 27, doi:10.1145/3241743

    2018 doi

  54. [63]

    Empirical Software Engineering 25:4097--4129, doi:10.1007/s10664-020-09858-z

    Storey MA, Ernst NA, Williams C, Kalliamvakou E (2020) The who, what, how of software engineering research: a socio-technical framework. Empirical Software Engineering 25:4097--4129, doi:10.1007/s10664-020-09858-z

    2020 doi

  55. [64]

    Journal of Systems and Software 205:111804, doi:10.1016/j.jss.2023.111804

    Swillus M, Zaidman A (2023) Sentiment overflow in the testing stack: Analyzing software testing posts on stack overflow. Journal of Systems and Software 205:111804, doi:10.1016/j.jss.2023.111804

    2023

  56. [65]

    Tan J, Feitosa D, Avgeriou P (2021) Do practitioners intentionally self-fix technical debt and why? IEEE, pp 251--262, doi:10.1109/ICSME52107.2021.00029

    2021

  57. [66]

    Journal of Systems and Software 86:1498--1516, doi:10.1016/j.jss.2012.12.052

    Tom E, Aurum A, Vidgen R (2013) An exploration of technical debt. Journal of Systems and Software 86:1498--1516, doi:10.1016/j.jss.2012.12.052

    2013 doi

  58. [67]

    IEEE Transactions on Software Engineering 48:2053--2065, doi:10.1109/TSE.2020.3048991

    Torkar R, Furia CA, Feldt R, de Oliveira Neto FG, Gren L, Lenberg P, Ernst NA (2022) A method to assess and argue for practical significance in software engineering. IEEE Transactions on Software Engineering 48:2053--2065, doi:10.1109/TSE.2020.3048991

    2022

  59. [68]

    IEEE, pp 254--259, doi:10.1109/CONISOFT50191.2020.00043

    Valdez A, Oktaba H, Gomez H, Vizcaino A (2020) Sentiment analysis in jira software repositories. IEEE, pp 254--259, doi:10.1109/CONISOFT50191.2020.00043

    2020

  60. [69]

    IEEE, pp 179--188, doi:10.1109/SANER.2016.72

    Wehaibi S, Shihab E, Guerrouj L (2016) Examining the impact of self-admitted technical debt on software quality. IEEE, pp 179--188, doi:10.1109/SANER.2016.72

    2016 doi

  61. [70]

    doi:10.1007/978-3-642-29044-2_8

    Wohlin C, Runeson P, Höst M, Ohlsson MC, Regnell B, Wesslén A (2012) Planning. doi:10.1007/978-3-642-29044-2_8

    2012 doi

  62. [71]

    ACM, pp 137--146, doi:10.1145/3379597.3387459

    Xavier L, Ferreira F, Brito R, Valente MT (2020) Beyond the code: Mining self-admitted technical debt in issue tracker systems. ACM, pp 137--146, doi:10.1145/3379597.3387459

    2020

  63. [72]

    Empirical Software Engineering 27:163, doi:10.1007/s10664-022-10203-9

    Xavier L, Montandon JE, Ferreira F, Brito R, Valente MT (2022) On the documentation of self-admitted technical debt in issues. Empirical Software Engineering 27:163, doi:10.1007/s10664-022-10203-9

    2022 doi

  64. [73]

    Aging & Mental Health 15:414--418, doi:10.1080/13607863.2010.536136

    Yeung DY, Wong CK, Lok DP (2011) Emotion regulation mediates age differences in emotions. Aging & Mental Health 15:414--418, doi:10.1080/13607863.2010.536136

    2011

  65. [74]

    Zampetti F, Serebrenik A, Penta MD (2018) Was self-admitted technical debt removal a real removal? ACM, pp 526--536, doi:10.1145/3196398.3196423

    2018

  66. [75]

    IEEE, pp 355--366, doi:10.1109/SANER48275.2020.9054868

    Zampetti F, Serebrenik A, Penta MD (2020) Automatically learning patterns for self-admitted technical debt removal. IEEE, pp 355--366, doi:10.1109/SANER48275.2020.9054868

    2020

  67. [76]

    Empirical Software Engineering 26:131, doi:10.1007/s10664-021-10031-3

    Zampetti F, Fucci G, Serebrenik A, Penta MD (2021) Self-admitted technical debt practices: a comparison between industry and open-source. Empirical Software Engineering 26:131, doi:10.1007/s10664-021-10031-3

    2021 doi

  68. [77]

    , " * write output.state after.block = add.period write newline

    ENTRY address archive author booktitle chapter doi edition editor eid eprint howpublished institution journal key month note number organization pages publisher school series title type url volume year label extra.label sort.label short.list INTEGERS output.state before.all mi...

  69. [78]

    write newline

    " write newline "" before.all 'output.state := FUNCTION add.period duplicate empty 'skip "." * add.blank if FUNCTION if.digit duplicate "0" = swap duplicate "1" = swap duplicate "2" = swap duplicate "3" = swap duplicate "4" = swap duplicate "5" = swap duplicate "6" = swap dupl...

  70. [79]

    , " * write output.state after.block = add.period write newline

    ENTRY address author booktitle chapter doi edition editor eid howpublished institution journal key month note number organization pages publisher school series title type url volume year label INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION in...

  71. [80]

    write newline

    " write newline "" before.all 'output.state := FUNCTION if.digit duplicate "0" = swap duplicate "1" = swap duplicate "2" = swap duplicate "3" = swap duplicate "4" = swap duplicate "5" = swap duplicate "6" = swap duplicate "7" = swap duplicate "8" = swap "9" = or or or or or or...

  72. [81]

    , " * write output.state after.block = add.period write newline

    ENTRY address author booktitle chapter doi edition editor eid howpublished institution journal key month note number organization pages publisher school series title type url volume year label INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION in...

  73. [82]

    write newline

    " write newline "" before.all 'output.state := FUNCTION if.digit duplicate "0" = swap duplicate "1" = swap duplicate "2" = swap duplicate "3" = swap duplicate "4" = swap duplicate "5" = swap duplicate "6" = swap duplicate "7" = swap duplicate "8" = swap "9" = or or or or or or...

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.