pith. sign in

arxiv: 1907.08076 · v1 · pith:QOHPJD5Vnew · submitted 2019-07-18 · 💻 cs.SE

Systematic Mapping Protocol Feature Modeling Tools

Samuel Sep\'ulveda , Andrea Rivera This is my paper

Pith reviewed 2026-05-24 19:48 UTC · model grok-4.3

classification 💻 cs.SE
keywords feature modeling toolssoftware product linessystematic mapping studySPL toolsprotocol definitiontool assessmentvariability management
0
0 comments X

The pith

This paper defines a protocol for a systematic mapping study on feature modeling tools in software product lines.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The authors lay out the plan for a systematic mapping study that will synthesize literature on feature modeling tools used in software product lines. The review will cover each tool's application domain, underlying model, origin, degree of empirical validation, and overall quality. A reader would care because the resulting map can show which tools are well-supported by evidence and where research gaps remain in SPL development. The protocol itself specifies search strings, data sources, inclusion rules, and quality assessment steps to guide the mapping.

Core claim

The paper establishes a complete protocol for conducting a systematic mapping study to identify and assess relevant papers on feature modeling tools, with extraction focused on application domain, underlying model, origin, empirical validation level, and tool quality.

What carries the argument

The systematic mapping study protocol, which defines the search process, paper selection criteria, and data extraction form for reviewing feature modeling tools in SPL.

If this is right

  • The mapping will produce a classification of tools by their primary application domains.
  • It will quantify how many tools have undergone empirical validation studies.
  • Quality scores will indicate which tools meet basic research standards.
  • The protocol can serve as a reusable template for future reviews of SPL tooling.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Execution of the protocol may expose that most tools originate from academic rather than industrial settings.
  • The results could highlight under-explored domains where feature modeling support is still weak.
  • Practitioners might gain a shortlist of validated tools to consider for adoption.

Load-bearing premise

The chosen search strings and assessment criteria will locate a representative sample of papers describing existing feature modeling tools.

What would settle it

Executing the search and finding that major known tools such as FeatureIDE are absent from the retrieved papers would show the protocol failed to capture relevant literature.

Figures

Figures reproduced from arXiv: 1907.08076 by Andrea Rivera, Samuel Sep\'ulveda.

Figure 1
Figure 1. Figure 1: SMS steps 2.1 Protocol Definition (Planning) In this section we present the main steps performed in the protocol definition for this SMS. 3 [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: SMS Primary Study Selection Steps [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗
Figure 4
Figure 4. Figure 4: Example for a data extraction form for the tools in selected papers. [PITH_FULL_IMAGE:figures/full_fig_p009_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Dropbox as a shared repository for resources. [PITH_FULL_IMAGE:figures/full_fig_p011_5.png] view at source ↗
Figure 7
Figure 7. Figure 7: Screenshot from website of Publish or Perish tool. [PITH_FULL_IMAGE:figures/full_fig_p012_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Overleaf for writing this report and the paper. [PITH_FULL_IMAGE:figures/full_fig_p012_8.png] view at source ↗
read the original abstract

The customers and users need for new products and services according to high-quality standards have increased in the last time. In that sense, the production processes must be aligned with the organization and development process in order to achieve this goal. The aim of this paper is to synthesize the current state of the research reported in the literature regarding the application domain, underlying model, origin, degree of empirical validation and quality of existing feature modeling tools used in SPL. Therefore, this technical report presents the protocol definition for a systematic mapping study (SMS) that we will conduct to identify and assess the set of relevant papers on feature model tools.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

0 major / 2 minor

Summary. The manuscript presents the protocol definition for a future systematic mapping study (SMS) on feature modeling tools used in Software Product Lines (SPL). The stated aim is to synthesize literature regarding application domain, underlying model, origin, degree of empirical validation, and quality of existing tools; the contribution is the protocol itself rather than any executed search, data extraction, or synthesis results.

Significance. Defining the SMS protocol in advance supports transparency and reduces the risk of post-hoc bias in the eventual study. If executed, the mapping could usefully catalog the state of feature modeling tools and highlight gaps in empirical validation. The protocol format itself is a recognized contribution in evidence-based software engineering when the design is sufficiently detailed.

minor comments (2)
  1. [Abstract] Abstract: the phrasing 'in the last time' is nonstandard and should be replaced with 'recently' or 'in recent years' for clarity.
  2. The title is terse and could be revised to 'Protocol for a Systematic Mapping Study of Feature Modeling Tools' to better indicate the manuscript type and scope.

Simulated Author's Rebuttal

0 responses · 0 unresolved

We thank the referee for the positive review and the recommendation of minor revision. We appreciate the recognition that publishing the SMS protocol in advance promotes transparency and helps mitigate post-hoc bias in the planned mapping study on feature modeling tools.

Circularity Check

0 steps flagged

Protocol definition with no derivations or load-bearing claims

full rationale

The paper is explicitly a protocol definition for a future systematic mapping study (SMS) on feature modeling tools in SPL. It states its aim as synthesizing literature but clarifies that the contribution is the protocol itself, with no results, data extraction, synthesis, equations, predictions, or fitted parameters presented. No self-citations are load-bearing because no empirical assertions depend on them. The document contains no derivation chain that could reduce to its own inputs by construction, making it self-contained as a forward-looking methodological plan.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

The paper relies on standard systematic mapping study practices from software engineering literature without introducing new parameters, axioms, or entities.

pith-pipeline@v0.9.0 · 5616 in / 880 out tokens · 21650 ms · 2026-05-24T19:48:00.878346+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

19 extracted references · 19 canonical work pages

  1. [1]

    Clements and L

    P. Clements and L. Northrop, Software Product Lines: Practices and Pat- terns, 3rd ed. Addison-Wesley Professional, Aug. 2001

    work page 2001

  2. [2]

    Best practices of rup R⃝ in software product line development,

    F. Ahmed and L. F. Capretz, “Best practices of rup R⃝ in software product line development,” in Proceedings of the International Conference on Com- puter and Communication Engineering (ICCCE 2008) . IEEE, 2008, pp. 1363–1366

    work page 2008

  3. [3]

    Guest editors’ introduction: Successful software product line practices,

    J. D. McGregor, D. Muthig, K. Yoshimura, and P. Jensen, “Guest editors’ introduction: Successful software product line practices,” IEEE Software, vol. 27, no. 3, pp. 16–21, May/June 2010

    work page 2010

  4. [4]

    Domain Specific Languages for Managing Feature Models: Ad- vances and Challenges,

    P. Collet, “Domain Specific Languages for Managing Feature Models: Ad- vances and Challenges,” in Proceedings of the 6th International Symposium on Leveraging Applications of Formal Methods, Verification and Validation (ISoLA 2014) , ser. Lecture Notes in Computer Science, T. Margaria and B. Steffen, Eds., vol. 8802. Springer, Oct. 2014, pp. 273–288

    work page 2014

  5. [5]

    Towards a Unified Feature Meta- model: a Systematic Comparison of Feature Languages,

    S. Sep´ ulveda, C. Cares, and C. Cachero, “Towards a Unified Feature Meta- model: a Systematic Comparison of Feature Languages,” in Proceedings of the 7th Iberian Conference on Information Systems and Technologies (CISTI 2012). IEEE Computer Society, Jun. 2012, pp. 1–7

    work page 2012

  6. [6]

    Feature Modeling Languages: Denotations and Semantic Differ- ences,

    ——, “Feature Modeling Languages: Denotations and Semantic Differ- ences,” in Proceedings of the 7th Iberian Conference on Information Sys- tems and Technologies (CISTI 2012). IEEE Computer Society, Jun. 2012, pp. 1–6

    work page 2012

  7. [7]

    Requirements modeling lan- guages for software product lines: A systematic literature review,

    S. Sep´ ulveda, A. Cravero, and C. Cachero, “Requirements modeling lan- guages for software product lines: A systematic literature review,” Infor- mation and Software Technology, vol. 69, pp. 16–36, Jan. 2016

    work page 2016

  8. [8]

    The value of map- ping studies: a participant observer case study,

    B. A. Kitchenham, D. Budgen, and O. P. Brereton, “The value of map- ping studies: a participant observer case study,” in Proceedings of the 14th international conference on Evaluation and Assessment in Software Engi- neering. BCS Learning & Development Ltd., Apr. 2010, pp. 25–33

    work page 2010

  9. [9]

    Systematic map- ping studies in software engineering,

    K. Petersen, R. Feldt, S. Mujtaba, and M. Mattsson, “Systematic map- ping studies in software engineering,” in Proceedings of the 12th Interna- tional Conference on Evaluation and Assessment in Software Engineering (EASE’08). BCS Learning & Development Ltd., Jun. 2008, pp. 1–10

    work page 2008

  10. [10]

    Guidelines for performing Systematic Literature Reviews in Software Engineering,

    B. Kitchenham and S. Charters, “Guidelines for performing Systematic Literature Reviews in Software Engineering,” Keele University and Durham University, Tech. Rep. EBSE 2007-01, Jul. 2007. 14

    work page 2007

  11. [11]

    Evaluation and measurement of software process im- provement - a systematic literature review,

    M. Unterkalmsteiner, T. Gorschek, A. M. Islam, C. K. Cheng, R. B. Per- madi, and R. Feldt, “Evaluation and measurement of software process im- provement - a systematic literature review,” IEEE Transactions on Soft- ware Engineering, vol. 38, no. 2, pp. 398–424, March-April 2012

    work page 2012

  12. [12]

    Petticrew and H

    M. Petticrew and H. Roberts, Systematic Reviews in the Social Sciences: A Practical Guide. John Wiley & Sons, Dec. 2008

    work page 2008

  13. [13]

    Guidelines for conducting systematic mapping studies in software engineering: An update,

    K. Petersen, S. Vakkalanka, and L. Kuzniarz, “Guidelines for conducting systematic mapping studies in software engineering: An update,” Informa- tion and Software Technology, vol. 64, pp. 1–18, Aug. 2015

    work page 2015

  14. [14]

    Lessons from applying the systematic literature review process within the software engineering domain,

    P. Brereton, B. A. Kitchenham, D. Budgen, M. Turner, and M. Khalil, “Lessons from applying the systematic literature review process within the software engineering domain,” Journal of Systems and Software , vol. 80, no. 4, pp. 571–583, Apr. 2007

    work page 2007

  15. [15]

    Inter-rater Reliability: Dependency on Trait Prevalence and Marginal Homogeneity,

    K. Gwet, “Inter-rater Reliability: Dependency on Trait Prevalence and Marginal Homogeneity,” Statistical Methods for Inter-Rater Reliability As- sessment Series, vol. 2, pp. 1–9, May 2002

    work page 2002

  16. [16]

    J. L. Fleiss, Statistical methods for rates and proportions . John Wiley & Sons, 1981

    work page 1981

  17. [17]

    Inside dropbox: Understanding personal cloud storage services,

    I. Drago, M. Mellia, M. M. Munaf` o, A. Sperotto, R. Sadre, and A. Pras, “Inside dropbox: Understanding personal cloud storage services,” in Pro- ceedings of the 2012 ACM SIGCOMM Internet Measurement Conference . ACM, Nov. 2012, pp. 481–494

    work page 2012

  18. [18]

    Vaidhyanathan, M

    V. Vaidhyanathan, M. Moore, K. A. Loper, J. Van Schaik, and D. Goolabs- ingh, “Making bibliographic researchers more efficient: Tools for organizing and downloading pdfs, part 1: icyte, mendeley desktop, papers, pdf stacks, pubget paperplane, wizfolio, and zoterot,” Journal of Electronic Resources in Medical Libraries, vol. 9, no. 1, pp. 47–55, 2012

    work page 2012

  19. [19]

    Publish or perish, version 3,

    A.-W. Harzing, “Publish or perish, version 3,” Available at www.harzing.com/pop.htm, 2007. A Evaluation of the SMS process Here we include a self evaluation for the work that will be done according to [13]. Table 7 shows the activities considered for conducting a SMS. We declare using a check-mark ( ✓) the activities that will be performed. We used the ev...

    work page 2007