On the Relationship Between Coupling and Refactoring: An Empirical Viewpoint

Davide Taibi; Giuseppe Destefanis; Mahir Arzoky; Steve Counsell

REVIEW 4 major objections 5 minor 19 references

On the Relationship Between Coupling and Refactoring: An Empirical Viewpoint

T0 review · 4 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Developers do not refactor highly coupled classes differently from loosely coupled classes, according to more than five thousand refactorings from three open-source systems.

desk verdict New quartile analysis of a public refactoring dataset, but the 'developer indifference' claim depends on an unstated timing assumption that could flip the result. read the letter →

arxiv 1908.01501 v1 pith:LBD4DCKW submitted 2019-08-05 cs.SE

Steve Counsell , Mahir Arzoky , Giuseppe Destefanis , Davide Taibi This is my paper · ORCID

classification cs.SE

keywords refactoringcouplingmetricsCBOmetricCCBCquartileanalysisempiricalsoftwareengineeringopen-sourceinheritance

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Refactoring is supposed to improve internal structure, and excessive coupling is usually treated as a reason to refactor. This paper asks whether developers actually behave that way: do classes with high coupling receive different refactoring types than classes with low coupling? The authors analyze more than 5,200 refactorings from releases of three open-source systems, split into lower and upper quartiles by two coupling measures (CBO, a count of coupled classes, and CCBC, a text-similarity measure). They find no meaningful difference in the types of refactoring applied in either quartile for either metric, and they find that move-method and move-field refactorings, typically associated with coupling removal, appear in the lower quartile too, sometimes more often. If this result holds, it undercuts the intuition that high coupling triggers coupling-reducing refactoring and raises the question of why so few of the standard seventy-two refactorings ever get used.

What carries the argument

The argument is carried by a quartile comparison. Each class is ranked by CBO (the number of other classes to which it is coupled) and separately by CCBC (the textual similarity of a class's tokens to other classes); the classes in the bottom 25% and top 25% of each ranking define the lower and upper quartiles. Refactorings assigned to classes in each quartile are then tallied by type, and the top-ten lists are compared by overlap and frequency. The refactoring data themselves were produced by a reconstruction tool with reported 95% recall and 79% precision, operating on the release history of three open-source systems; the paper treats those reconstructed refactorings as the record of what developers did, then checks whether the type mix changes with coupling quartile.

What would settle it

A reader could test this by taking a random sample of classes from the same three systems and manually reconstructing refactorings from version-control diffs, independent of the automated tool. If move-method and move-field refactorings concentrate in the high-coupling sample rather than appearing equally across quartiles, the paper's reported indifference would fail.

Watch

Extended reading notes

Core claim

The central claim is reported as a negative result: refactoring choices do not appear to track class coupling. For each of three systems, the paper splits classes into lower and upper quartiles by CBO and by CCBC, then compares the ten most frequent refactoring types in each quartile. The lists overlap heavily, with seven, six, and eight of the top ten common across the systems for CBO and comparable overlap for CCBC, and the same types (rename method, move method, move field, add or remove parameter, replace magic number with symbolic constant) dominate both quartiles. Move method and move field, which the paper identifies as the refactorings most strongly associated with coupling reduction, are numerous in the low-coupling quartile as well, and in some systems more numerous there. The paper also reports that only 112 of 5,200 analyzed refactorings (2.15%) were inheritance-related, mostly in one system. Its conclusion is that developers are largely indifferent to coupling when choosing refactoring types.

Load-bearing premise

The whole result assumes that the automated refactoring reconstruction finds refactorings equally well in highly coupled and loosely coupled classes, since the paper does not revalidate the tool on these three systems.

Editorial extensions

If this is right

High coupling alone will not predict where coupling-removal refactorings such as move method and move field occur; in this dataset they are common in low-coupling classes too.
The overlap in refactoring types across quartiles implies that developers' refactoring decisions are driven by something other than coupling level, possibly code smells like long methods, duplication, or parameter-list awkwardness.
Because only a handful of the seventy-two catalogued refactorings appear in either quartile, claims about refactoring practice based on the full catalog may describe an ideal rather than actual developer behavior.
The similar behavior of CBO and CCBC across quartiles suggests the two metrics may be interchangeable for this kind of refactoring study, as the paper notes.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Left implicit in the paper: refactoring-recommendation systems that rank classes by coupling are likely to mismatch real developer attention; adding size, change frequency, or defect history as co-predictors is a natural test the authors did not run.
The paper reports that upper-quartile classes are much larger on average than lower-quartile classes, yet refactoring types look the same. That makes size a plausible confound: a follow-up should compare coupling quartiles within narrow size bands to see whether size, not coupling, drives the observed choices.
The authors interpret the result as developer indifference to coupling. A competing interpretation is that high-coupling classes had already been refactored or are stable enough that no coupling-related refactoring was needed; distinguishing these would require developer interviews or commit-level rationale, which the dataset cannot provide.
If replication on the larger 33-project dataset the paper names as future work shows the same overlap, then the absence of coupling-driven refactoring may be a general property of open-source maintenance rather than a quirk of these three systems.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Coupling/refactoring paper: new quartile analysis of the Bavota et al. data, but the headline interpretation is shaky until the timing question is answered. The descriptive overlap between high- and low-coupling quartiles is a genuine observation, and the lack of inheritance refactorings is worth noting. The authors are honest about the exploratory scope and list relevant threats.

The main soft spot is exactly the one the stress-test flags. The paper never states whether CBO and CCBC are measured before the refactorings or after. Since Move Method and Move Field reduce source-class coupling, post-treatment measurement would push those very refactorings into the low-coupling quartile, which is what they see in some systems. That would make the 'developers treat high and low coupling the same' conclusion an artifact. The dataset is public, so the authors can check this; as written, the central claim is not supported. I'd call this the one serious issue, not because the result is wrong but because the ordering of measurement and events is silently doing work.

Two smaller issues. First, 'no meaningful difference' is an interpretation, not a statistical result – no tests, intervals, or effect sizes, and some of their own percentages (ArgoUML MM+MF 35% LQ vs 10% UQ) look like differences worth talking about. Second, the refactoring counts come from Ref-Finder, 79% precision, and the authors don't validate its output on these three systems. That's a known limitation, but it reinforces the need for cautious wording.

What's genuinely new: the quartile split, the side-by-side CBO/CCBC comparison, and the observation that the same few refactoring types dominate both quartiles, which was not in the original Bavota et al. paper. Credit is due for reusing a public dataset and for keeping the analysis simple enough to reproduce from the paper's tables.

Recommendation: yes, send it to a serious referee. The timing issue is addressable and the paper's contribution is a useful empirical note, but it needs a revision that either verifies the temporal alignment or weakens the 'indifference' framing. I would not cite it in its current form, but I'd read a revised version.

Referee Report

4 major / 5 minor

Summary. The paper analyzes 5,200 refactoring operations from three open-source Java systems (Xerces, Apache Ant, ArgoUML), dividing classes into upper and lower quartiles of the CBO and CCBC coupling metrics and comparing the types of refactorings applied in each quartile. The authors report substantial overlap in refactoring types across quartiles, observe that coupling-removal refactorings such as Move Method and Move Field are sometimes more numerous in the lower quartile, and note a scarcity of inheritance-related refactorings. They conclude that developers appear largely indifferent to high coupling when selecting refactoring types, and question the relevance of Fowler's full catalog of refactorings.

Significance. If the central finding is robust, the paper challenges a widely held assumption that high coupling motivates coupling-reduction refactorings, and it contributes descriptive evidence about which refactorings are actually used in practice. The use of an existing external dataset (Bavota et al.) and the clear presentation of quartile-level counts are strengths, as is the explicit comparison of two coupling metrics. However, the paper's main conclusion is currently supported only by informal comparisons of proportions, with no statistical tests or confidence intervals, and several potentially confounding aspects (notably the timing of metric measurement) are not addressed. The work is an interesting exploratory study but does not yet meet the standards for a definitive empirical contribution.

major comments (4)

[Section III, Table 2] The paper never states whether the CBO and CCBC values used to construct the quartiles are measured before or after the refactorings, or at a single release snapshot. This timing is load-bearing: Move Method and Move Field reduce the coupling of the source class, so if coupling values are computed after these refactorings, originally high-coupling classes that were refactored would systematically appear in the lower quartile. That mechanism would mechanically produce the paper's key observation that coupling-removal refactorings are numerous in the lower quartile, without implying developer indifference. The authors must specify the measurement timing, and ideally re-analyze the data using pre-refactoring coupling values or at least discuss how this could change the results.
[Section III, Figs. 1 and 2] The conclusion 'no meaningful difference' is asserted without any statistical comparison. The reported proportions include substantial differences, for example ArgoUML CBO data show MM and MF together accounting for 35.17% of refactorings in the LQ versus 10.39% in the UQ (Section III-A). The authors should provide confidence intervals, effect sizes, or formal tests (e.g., chi-square or bootstrap-based comparisons) to support the claim of no difference. Without such analysis, the impression of overlap may be misleading.
[Section III-C, Table 3] The size confounding is acknowledged in the paper's own Table 3, which shows that UQ classes are much larger than LQ classes (e.g., Xerces mean LOC 1781 vs. 249; ArgoUML mean WMC 57.72 vs. 15.00). The analysis compares raw counts and proportions of refactorings without normalizing by class size or method count. Larger classes naturally offer more refactoring opportunities, so the comparison of refactoring type mixes across quartiles is confounded; the authors should either normalize the data or control for size statistically.
[Section II] The analysis relies entirely on Ref-Finder's reconstructed refactorings, which the paper reports as having 79% precision and 95% recall. If Ref-Finder's detection accuracy varies with class coupling (e.g., if move operations are easier to detect in smaller, low-coupling classes), the observed overlap between quartiles could be a measurement artifact. The authors do not validate the tool's output on these three systems, and the threats-to-validity section does not address this. At minimum, a sensitivity analysis or an explicit argument about why detection bias is unlikely would strengthen the claim.

minor comments (5)

[Abstract and Introduction] The phrase 'Results showed no very little difference' is grammatically incorrect and appears to convey the opposite of the intended meaning; it should read 'no difference' or 'very little difference.'
[Section III-A] There is an inconsistent abbreviation: 'UP' should be 'UQ' when referring to the upper quartile (in the sentence about RMNwSC percentages).
[Section III-B, Figs. 2a-2f] The text states that Figs. 2a, 2c, and 2e represent the UQ, but the figure captions label Fig. 2a and 2c as 'LQ' and Fig. 2b and 2d as 'UQ.' This mismatch between text and figures must be corrected.
[Abstract and Table 1] The abstract claims 'over six thousand refactoring operations,' but Table 1 lists 7,502 refactorings for Xerces alone, and the total across all three systems is 12,046. After quartile filtering only 5,200 are used; the wording should specify the number analyzed after quartile selection.
[Section V] The threats-to-validity section does not mention the timing of metric measurement, which is a potential major threat as discussed in the major comments. It should be explicitly addressed there.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the study is an observational analysis of an external dataset with independent metrics, and its conclusion is not defined into existence.

full rationale

No load-bearing circular step was found. The paper analyzes a previously published refactoring dataset from Bavota et al. [3], computed coupling metrics CBO and CCBC from the code, split classes into quartiles, and compared observed refactoring-type frequencies between upper and lower quartiles. No parameter is fitted to the data and then renamed as a prediction. The central empirical claim, that refactoring types overlap heavily between high- and low-coupling quartiles, is a statistical observation from the dataset, not a quantity defined in terms of itself. The only self-citations are [18], an earlier coupling-metrics paper by one of the present authors, mentioned in the threats section as one possible alternative metric that could have been used, and [19], cited only as planned future work based on the Technical Debt Dataset. Neither is load-bearing for the main result, and neither is used to forbid alternatives or to justify the conclusion. The skeptical concern about whether coupling was measured before or after refactoring is a validity threat about temporal alignment, not a circularity: even if the measurement timing were wrong, the observed overlap would be an artifact of the measurement design, not a case of the paper deriving its conclusion from its own assumptions by construction. Thus the appropriate finding is no significant circularity, score 0.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No free parameters are fitted and no invented entities are introduced. The central claim rests on the external Bavota et al. dataset, the external CBO and CCBC metrics, and the assumed validity of Ref-Finder's refactoring extraction.

assumptions (4)

domain assumption Ref-Finder correctly reconstructs refactorings with 95% recall and 79% precision, and the extracted refactorings can be treated as ground truth for comparing quartiles.
Invoked in Section II; the analysis is based entirely on the externally extracted refactoring set, and the authors do not revalidate the tool's output on these systems.
domain assumption CBO and CCBC are valid and sufficient operationalizations of class coupling for the research question.
Used throughout Sections I and III; the conclusion about coupling depends on these two metrics.
domain assumption Quartile decomposition into lower, middle, and upper coupling values is a meaningful partition for detecting refactoring differences.
Defined in Section I; the mid-range 50% of data is discarded, so conclusions rest on this partition.
domain assumption The set of refactorings identified as the top ten in each quartile is sufficient to characterize refactoring behavior.
Used in Section III; the analysis compares only the most frequent refactorings, and no aggregate statistical comparison is performed.

how reviews work

0 comments

Cite this review

Pith. "Pith review of On the Relationship Between Coupling and Refactoring: An Empirical Viewpoint." pith.science (2026). https://pith.science/paper/LBD4DCKW

@misc{pith2026190801501,
  author       = {Pith},
  title        = {Pith review of: On the Relationship Between Coupling and Refactoring: An Empirical Viewpoint},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LBD4DCKW}},
  note         = {Machine review of arXiv:1908.01501}
}

read the original abstract

[Background] Refactoring has matured over the past twenty years to become part of a developer's toolkit. However, many fundamental research questions still remain largely unexplored. [Aim] The goal of this paper is to investigate the highest and lowest quartile of refactoring-based data using two coupling metrics - the Coupling between Objects metric and the more recent Conceptual Coupling between Classes metric to answer this question. Can refactoring trends and patterns be identified based on the level of class coupling? [Method] In this paper, we analyze over six thousand refactoring operations drawn from releases of three open-source systems to address one such question. [Results] Results showed no meaningful difference in the types of refactoring applied across either lower or upper quartile of coupling for both metrics; refactorings usually associated with coupling removal were actually more numerous in the lower quartile in some cases. A lack of inheritance-related refactorings across all systems was also noted. [Conclusions] The emerging message (and a perplexing one) is that developers seem to be largely indifferent to classes with high coupling when it comes to refactoring types - they treat classes with relatively low coupling in almost the same way.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

19 extracted references · 19 canonical work pages

[1]

A validation of object -oriented design metrics as quality indicators

V. Basili, L. Briand, W. Melo. “A validation of object -oriented design metrics as quality indicators. ” IEEE Transactions on Software Engineering 22, 10, 751–761. 1995

work page 1995
[2]

An empirical study on the developers' perception of software coupling

G. Bavota, B. Dit, R. Oliveto, M. Di Penta, D. Poshyvanyk, A. De Lucia . “An empirical study on the developers' perception of software coupling. ” ICSE 2013. pp 692-701. 2013

work page 2013
[3]

An experimental investigation on the innate relationship between quality and refactoring

G. Bavota, A. De Lucia, M. Di Penta,, R. Oliveto, F. Palomba . “An experimental investigation on the innate relationship between quality and refactoring.” J. Syst. Software. 107, pp 1-14. 2015

work page 2015
[4]

An Investigation into Coupling Measures for C++

L. Briand, P., Devanbu, W. Melo . “An Investigation into Coupling Measures for C++.” Proceedings of International Conference on Software Engineering, Los Angeles, USA. 1999

work page 1999
[5]

A Unified Framework for Coupling Measurement in Object -Oriented Systems

L.C. Briand, J. Daly, J. Wüst. “A Unified Framework for Coupling Measurement in Object -Oriented Systems.” IEEE Trans. Softw. Eng. 25, 1, pp 91-121. 1999

work page 1999
[6]

A Metrics Suite for Object Oriented Design

S. R. Chidamber, C. F. Kemerer . “A Metrics Suite for Object Oriented Design” IEEE Transaction s on Software Engineering, 20(6):476 -493, 1994

work page 1994
[7]

Refactoring: Improving the Design of Existing Code

M. Fowler . “Refactoring: Improving the Design of Existing Code ” Addison-Wesley, 1999

work page 1999
[8]

Ref-Finder: A refactoring reconstruction tool based on logic query temp lates

M. Kim, M. Gee, A. Loh, N. Rachatasumrit, Napol . “ Ref-Finder: A refactoring reconstruction tool based on logic query temp lates.” Proceedings of the ACM SIGSOFT Symposium on the Foundations of Software Engineering. 371 -372. 2010

work page 2010

Show all 19 references

[9]

A survey of software refactoring

T. Mens, T. Tourwe . “ A survey of software refactoring. ” IEEE Transactions on Software Engineering 30, 2, 126 –139. 2004

2004
[10]

Refactoring object-oriented frameworks

W. Opdyke, “Refactoring object-oriented frameworks .” PhD Thesis, University of Illinois, Urbana-Champaign, 1992

1992
[11]

Using infor- mation retrieval based coupling measures for impact analysis

D. Poshyvanyk, A., Marcus, R., Ferenc, and T., Gyimothy . “Using infor- mation retrieval based coupling measures for impact analysis.” Empirical Software Engineering 14, 1, 5–32. 2009

2009
[12]

Structured design

W. Stevens, G. Myers, L. Constantine. “Structured design” IBM Systems Journal. 13 (2): 115–13, 1974

1974
[13]

Identification of move method refactoring opportunities

N. Tsantalis, A., Chatzigeorgiou, A. “Identification of move method refactoring opportunities. ” IEEE Transactions on Software Engineering 35, 3, 347–367. 2009

2009
[14]

https://refactoring.guru/move- method
[15]

http://ant.apache.org/
[16]

http://argouml.tigris.org/
[17]

http://xerces.apache.org/xerces-j/
[18]

Coupling metrics for object - oriented design

R. Harrison, S. Counsell, and R. Nithi. "Coupling metrics for object - oriented design." Proceedings Fifth Internati onal Software Metrics Symposium. Metrics. IEEE, 1998

1998
[19]

”The Technical Debt Dataset

Valentina Lenarduzzi, Nyyti Saarim äki, and Davide Taibi. ”The Technical Debt Dataset. ” Proceedings of the 15th International Conference on Predictive Models and Data Analytics in Software Engineering (PROMISE’19), September 18, 2019, Recife, Brazil

2019

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

A validation of object -oriented design metrics as quality indicators

V. Basili, L. Briand, W. Melo. “A validation of object -oriented design metrics as quality indicators. ” IEEE Transactions on Software Engineering 22, 10, 751–761. 1995

work page 1995

[2] [2]

An empirical study on the developers' perception of software coupling

G. Bavota, B. Dit, R. Oliveto, M. Di Penta, D. Poshyvanyk, A. De Lucia . “An empirical study on the developers' perception of software coupling. ” ICSE 2013. pp 692-701. 2013

work page 2013

[3] [3]

An experimental investigation on the innate relationship between quality and refactoring

G. Bavota, A. De Lucia, M. Di Penta,, R. Oliveto, F. Palomba . “An experimental investigation on the innate relationship between quality and refactoring.” J. Syst. Software. 107, pp 1-14. 2015

work page 2015

[4] [4]

An Investigation into Coupling Measures for C++

L. Briand, P., Devanbu, W. Melo . “An Investigation into Coupling Measures for C++.” Proceedings of International Conference on Software Engineering, Los Angeles, USA. 1999

work page 1999

[5] [5]

A Unified Framework for Coupling Measurement in Object -Oriented Systems

L.C. Briand, J. Daly, J. Wüst. “A Unified Framework for Coupling Measurement in Object -Oriented Systems.” IEEE Trans. Softw. Eng. 25, 1, pp 91-121. 1999

work page 1999

[6] [6]

A Metrics Suite for Object Oriented Design

S. R. Chidamber, C. F. Kemerer . “A Metrics Suite for Object Oriented Design” IEEE Transaction s on Software Engineering, 20(6):476 -493, 1994

work page 1994

[7] [7]

Refactoring: Improving the Design of Existing Code

M. Fowler . “Refactoring: Improving the Design of Existing Code ” Addison-Wesley, 1999

work page 1999

[8] [8]

Ref-Finder: A refactoring reconstruction tool based on logic query temp lates

M. Kim, M. Gee, A. Loh, N. Rachatasumrit, Napol . “ Ref-Finder: A refactoring reconstruction tool based on logic query temp lates.” Proceedings of the ACM SIGSOFT Symposium on the Foundations of Software Engineering. 371 -372. 2010

work page 2010