REVIEW 3 major objections 6 minor 2 references
An Empirical Study on Technical Debt in a Finnish SME
T0 review · 3 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read A Finnish micro-enterprise's worst technical debt was a supplier dispute and missing automated tests.
desk verdict A useful qualitative case study whose headline numbers don't add up: Table I's vote sums and type classifications are internally inconsistent, and the abstract's ranking of Test and Requirements debt doesn't survive a re-tally. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the structured focus group combined with a classification scheme and a voting procedure. Participants first wrote postponed activities on post-it notes, then assigned each activity to one of eleven technical-debt categories (ten from the cited taxonomy plus a new Organizational Debt category), then attached causes using the 5-Whys technique. Severity was measured by dot-voting: each of the five participants had ten adhesive dots to place on the problems they considered most harmful, producing the 30-point total that drives the ranking. The moderation was deliberately hands-off, with an author who did not vote or propose items, and the whole session was limited to about two hours.
What would settle it
Go to the company's records and check the two highest-ranked items directly: inspect the version history and test suite to see whether automated tests were really absent during the period the participants described, and interview the external supplier to see whether the MVP disagreement and its costs are corroborated. If the supplier disputes the account, or if test automation existed earlier than claimed, the severity ranking loses its factual basis. A second check would be a replication in the same company using anonymous individual interviews rather than a group session; a different top-two ranking would show that the focus-group setting shaped the result.
Extended reading notes
Core claim
On the paper's own terms, the central discovery is an empirical ranking produced by the company's own staff: the most harmful technical debt items were a disagreement with the supplier about what the Minimum Viable Product should contain, and the lack of automatic testing, each receiving 7 of the 30 voting dots. Grouped by type, test debt received 11 points and requirements debt 10, well above code debt, which received 5. The root causes named most often were budget constraints and time constraints, each attached to five of the ten debt items, followed by estimation issues with three. The authors interpret this as evidence that postponement is not automatically debt: it becomes debt when the deferred work later costs more to repair than it bought in deadline relief, and the company's costly episodes were tied to weak validation of requirements, underestimated testing effort, and supplier relations.
Load-bearing premise
The ranking depends on five employees accurately remembering, in one two-hour session with the CTO, CFO, and CMO present, which postponed activities later became costly and why; if those memories or answers were biased, the top-ranked debt items would not be the true ones.
Editorial extensions
If this is right
- If these results hold for the case company, small software firms should expect requirement and test debt, rather than code debt, to be their most expensive backlog items.
- The dominance of budget and time constraints as causes implies that better estimation and earlier customer validation are the most direct mitigation levers, matching the paper's three recommendations.
- The finding that not all postponement creates debt suggests that companies should judge deferral by whether it accrues future interest, not by the mere existence of shortcuts.
- The supplier-disagreement item implies that outsourcing the first version of a product shifts technical debt risk to the requirements boundary, where contractually fixed specifications can block iterative learning.
- Because test debt received the highest type-level score, investing in automated testing early is presented as a way to reduce both regression risk and the manual effort that later consumes developer time.
Reading between the lines
- The authors do not say this, but their data suggest a coupling: missing automated tests made requirement changes riskier, so the supplier's refusal to iterate became more expensive than it would have been with a safety net.
- A testable extension of the root-cause ranking is that companies using prototype-based validation with customers before contracting will report lower requirements debt; this could be checked with a multi-company focus group replication.
- The two-hour, mixed-hierarchy focus group design may systematically undercount debt that developers are reluctant to admit in front of the CTO; an anonymous survey or diary study could reveal a different distribution.
- If the pattern generalizes beyond this firm, technical debt dashboards for SMEs should include requirement and test debt indicators, not only code smells and coverage metrics, which aligns with the paper's own call for continuous quality monitoring.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents an exploratory case study of technical debt (TD) in a Finnish micro-enterprise developing a SaaS sales-channel management tool. The authors conducted a two-hour focus group with five company members (CTO, CFO, CMO, and two developers), moderated by a co-author. Participants listed postponed activities, classified them into TD types using a 10+1 category scheme, identified causes and effects, and voted on the most harmful items. The authors conclude that supplier disagreement and lack of test automation are the most significant TD items, that Test and Requirements TD are the most significant types, and that budget and time constraints are the main root causes. The paper also proposes mitigation strategies: learning from customers, careful estimation, and continuous improvement.
Significance. If the data are reliable, the study offers a rare qualitative account of TD in a small company, showing that non-code-level debt (requirements conflicts, missing test automation, and supplier issues) can dominate code-level shortcuts. The focus-group design with mixed roles is appropriate for surfacing organizational causes, and the authors explicitly acknowledge several threats to validity, including management presence and the two-hour time limit. The paper reports raw vote counts and quotes participant statements, which adds transparency. However, the significance is reduced by the small number of participants, the lack of inter-rater reliability checks or independent coding, and the absence of outcome data linking postponed activities to actual costs. As a single-case exploratory study, the contribution is modest but potentially reusable for future comparisons if the data presentation is corrected.
major comments (3)
- [Section IV, Table I(b) and Section IV.A] The type-level point totals are inconsistent with the item-level data. The sum of the values in Table I(b) is 33, while the table's stated Grand Total is 30; this appears to result from counting TD1 (3 votes) under both Requirements TD and Organizational TD, but the table does not say so. More seriously, TD5 is defined in Section IV.A as 'Lack of automatic testing costs more in the future (Infrastructure TD)', yet the Test TD total of 11 in Table I(b) is only attainable if TD5's 7 votes are counted under Test TD. If TD5 is classified as Infrastructure TD as labeled, then Test TD totals 4 (TD6+TD7) and Infrastructure TD totals 9 (TD3+TD5), which would contradict the abstract's claim that 'Specification and test TD are the most significant types of TD'. The authors must either correct the TD5 classification, revise the type totals, or qualify the conclusion accordingly.
- [Section IV, RQ2 and Table I(c)] The cause-count table reports 'Design issues' with a count of 1, but no TD item in the text identifies design issues as a cause; for instance, TD4 (Design TD) is attributed only to Time constraints. The mapping between the TD items and the cause counts in Table I(c) is not reproducible from the text. The authors should provide the explicit mapping from each TD item to its causes or reconcile Table I(c) with the item descriptions.
- [Section III.B (T6) and Table I(a)] The voting procedure states that each of the five participants received ten votes, which implies up to 50 votes, but Table I(a) sums to 30. The paper does not explain whether participants were allowed to use fewer than ten votes or whether some votes were invalid. If all ten votes per participant were expected, the missing 20 votes could bias the ranking of the most significant TD items; if abstentions were permitted, this should be stated explicitly and considered in the interpretation.
minor comments (6)
- [Section IV, RQ1] 'As reported in Table 1a' should reference Table 1b, since the type-level totals appear in Table 1b.
- [Section IV.A, TD5] The sentence beginning 'The testing budget was too low ...' contains a fragment ('since the company did not even have enough time to the prioritization of the features and tasks...') that appears to be a copy-paste error from TD1.
- [Section IV.A, Table I(a)] Table I(a) lists 'TD0. Technical shortcuts', but the text defines 'TD9 Technical shortcuts (Code TD)'; the numbering is inconsistent.
- [Section III] The company is described as 'a micro-enterprise (less than 10 person)', while the title and abstract use 'SME'; the terminology should be harmonized.
- [Table I caption] The caption states that 'Motivations are counted once for each TD', but Table I(b) appears to count TD1 twice (under Requirements and Organizational TD); the caption or the table should be corrected.
- [Section III.C] The Data Analysis subsection gives no detail on how the TD items were assigned to the eleven categories or whether multiple participants independently validated the classifications; a sentence on this would help the reader assess the reliability of the type-level totals.
Circularity Check
No circularity: the focus-group findings are direct participant codings, not outputs of a model fitted to the same data or of self-cited prior work.
full rationale
This paper is a qualitative focus-group study, and its central results (TD2 and TD5 as the most significant debt items, Test and Requirements TD as the most significant types, and budget/time constraints as root causes) are direct codings and vote tallies of participant statements rather than the output of a mathematical derivation or a model fitted to those same statements. The TD taxonomy is adopted from the external Li et al. classification plus one new category, and the assignment of debt items to categories is an interpretive step performed on the focus-group data, not a self-definitional equivalence. The authors do cite their own prior work, but those self-citations appear in related-work, mitigation-suggestion, and future-tool-proposal contexts (e.g., references [9], [14]-[16], [23]-[27]) and none supplies the evidence for RQ1-RQ3. The arithmetic inconsistencies in Table I are a data-reporting and validity concern, not a circular derivation: the reported totals do not align with the classification labels, but this does not make the conclusions equivalent to the paper's inputs by construction. No load-bearing circular reduction can be exhibited from the text, so the appropriate finding is no significant circularity.
Assumptions & free parameters
assumptions (4)
- domain assumption Technical debt is a meaningful and shared construct in a micro-company context.
- domain assumption The eleven-category taxonomy (Li et al. plus Organizational TD) is complete and correctly applied by participants.
- domain assumption The 5-Whys technique uncovers true root causes from group discussion.
- domain assumption Vote counts by five participants are a valid measure of TD significance.
invented entities (1)
-
Organizational TD category
Cite this review
Pith. "Pith review of An Empirical Study on Technical Debt in a Finnish SME." pith.science (2026). https://pith.science/paper/WIDFMRWW
@misc{pith2026190801502,
author = {Pith},
title = {Pith review of: An Empirical Study on Technical Debt in a Finnish SME},
year = {2026},
howpublished = {\url{https://pith.science/paper/WIDFMRWW}},
note = {Machine review of arXiv:1908.01502}
}
read the original abstract
Objective. In this work, we report the experience of a Finnish SME in managing Technical Debt (TD), investigating the most common types of TD they faced in the past, their causes, and their effects. Method. We set up a focus group in the case-company, involving different roles. Results. The results showed that the most significant TD in the company stems from disagreements with the supplier and lack of test automation. Specification and test TD are the most significant types of TD. Budget and time constraints were identified as the most important root causes of TD. Conclusion. TD occurs when time or budget is limited or the amount of work are not understood properly. However, not all postponed activities generated "debt". Sometimes the accumulation of TD helped meet deadlines without a major impact, while in other cases the cost for repaying the TD was much higher than the benefits. From this study, we learned that learning, careful estimations, and continuous improvement could be good strategies to mitigate TD. These strategies include iterative validation with customers, efficient communication with stakeholders, meta-cognition in estimations, and value orientation in budgeting and scheduling.
Reference graph
Works this paper leans on
-
[2]
Careful estimation. The whole organization should under- stand the technical boundaries to avoid estimation errors. They should use previous tasks to improve their effort estimation regarding the development of new tasks. Un- derestimation can cause additional expenses for company. Customers should pay for the overall costs of the system; they tend to pay...
-
[2015]
and it is based on JavaScript and NoSQL and it’s devel- oped with the MEAN stack (MongoDB, Express.js, AngularJS and Node.js). A. Research Questions Based on the aforementioned goal, we derived the following Research Questions (RQs): • RQ1: What are the most common types of TD? • RQ2: What are the main causes of the accumulated TD? • RQ3: How to mitigate ...
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.