Pith. sign in

REVIEW 1 major objections 1 minor 18 references

Handling Numerous Stakeholders Requirements A Research Agenda and Preliminary Outcomes

T0 review · 1 major / 1 minor · reviewed 2026-05-24 · grok-4.3

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Gaps remain in techniques for prioritizing large numbers of crowdsourced stakeholder requirements.

desk verdict This is a PhD research agenda paper that flags gaps in large-scale crowdsourced requirements prioritization but presents no new method, data, or completed analysis. read the letter →

arxiv 1907.10192 v1 pith:VX2M5J77 submitted 2019-07-24 cs.SE

Saurabh Malgaonkar , Sherlock A. Licorish , Bastin Tony Roy Savarimuthu This is my paper
classification cs.SE
keywords requirementsprioritizationcrowdsourcedstakeholderfeedbacksoftwareengineeringresearchagenda
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper presents a PhD research agenda aimed at designing a new approach to analyze and prioritize requirements mentioned in feedback for software products. Early analysis finds that requirements prioritization has been studied widely, yet few methods address the scale of input from many crowdsourced stakeholders. The problem extends to domains outside software engineering, which suggests that practices from those areas may transfer to improve software engineering methods. A roadmap for the proposed research is outlined to develop the targeted outcomes.

What carries the argument

The PhD research agenda and roadmap for developing a new requirements prioritization approach that analyzes and prioritizes large volumes of stakeholder feedback.

What would settle it

A review that identifies no meaningful gaps in existing techniques for large-scale crowdsourced requirements or finds no transferable knowledge from other domains would undermine the case for the proposed new approach.

Watch

Extended reading notes

Core claim

The requirements prioritization problem has been researched extensively, however, gaps still remain when considering techniques that handle a large number of crowdsourced requirements. Furthermore, requirements prioritization as a problem affects many domains beyond software engineering. Hence, knowledge from other fields could be useful for informing requirements prioritization practice in the software engineering space.

Load-bearing premise

Knowledge from other fields can be transferred to improve requirements prioritization practice in software engineering.

Editorial extensions

If this is right

  • New prioritization techniques can be developed to scale to large sets of crowdsourced requirements.
  • Cross-domain insights can be integrated into software engineering practices for handling stakeholder input.
  • The research will produce a methodology that addresses prioritization needs across multiple application domains.
  • Preliminary findings establish the basis for further empirical work on crowdsourced feedback analysis.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Methods from data analytics or decision science fields could be adapted to automate parts of the prioritization process.
  • Validation on real software product feedback datasets would clarify whether the new approach outperforms current methods.
  • The agenda implies potential benefits for companies managing user reviews on platforms where volume is high.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

1 major / 1 minor

Summary. This manuscript presents a PhD research agenda for designing a new requirements prioritization approach to handle large volumes of crowdsourced stakeholder requirements extracted from software product feedback. It reports preliminary outcomes from early-stage literature analysis, states that requirements prioritization has been extensively studied yet gaps persist for large-scale crowdsourced cases, notes the problem's relevance beyond software engineering, and outlines a methodology roadmap.

Significance. If executed, the agenda could address a practical gap in requirements engineering for user-feedback-driven software development by potentially incorporating cross-domain techniques. The interdisciplinary motivation is a positive aspect, as is the explicit roadmap for future work. The contribution is primarily agenda-setting rather than empirical.

major comments (1)
  1. [Abstract] Abstract: the central motivation that 'gaps still remain when considering techniques that handle a large number of crowdsourced requirements' is stated without any cited references, search protocol, or concrete examples of the reviewed literature; this claim is load-bearing for justifying the proposed agenda yet cannot be assessed from the provided description.
minor comments (1)
  1. [Title] Title: missing possessive apostrophe and colon (suggested: 'Handling Numerous Stakeholders' Requirements: A Research Agenda and Preliminary Outcomes').

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for their constructive review of our research agenda manuscript. We address the single major comment below.

read point-by-point responses

  1. Referee: [Abstract] Abstract: the central motivation that 'gaps still remain when considering techniques that handle a large number of crowdsourced requirements' is stated without any cited references, search protocol, or concrete examples of the reviewed literature; this claim is load-bearing for justifying the proposed agenda yet cannot be assessed from the provided description.

    Authors: We agree that the abstract presents the motivation in summary form without supporting details. Because the manuscript reports only preliminary outcomes from an early-stage literature analysis (as stated in the abstract and introduction), the full search protocol and exhaustive examples were not included. In a revised version we will expand the abstract with 1-2 concrete examples of existing prioritization techniques and their documented limitations with large crowdsourced sets, plus a one-sentence description of the preliminary search approach. This will make the claim assessable while preserving the paper's character as an agenda rather than a completed review. revision: yes

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity

full rationale

The paper is explicitly a PhD research agenda and literature survey presenting planned methodology and early observations rather than any derivations, predictions, fitted parameters, or load-bearing claims. The statements about research gaps and cross-domain applicability function as motivation for future work and do not reduce to self-referential definitions, fitted inputs renamed as predictions, or self-citation chains. No equations or quantitative results exist that could create circularity.

Assumptions & free parameters 0 free parameters · 0 assumptions · 0 invented entities

No mathematical model, empirical claims, or technical derivations are present; the document is a planning statement.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Handling Numerous Stakeholders Requirements A Research Agenda and Preliminary Outcomes." pith.science (2026). https://pith.science/paper/VX2M5J77

@misc{pith2026190710192,
  author       = {Pith},
  title        = {Pith review of: Handling Numerous Stakeholders Requirements A Research Agenda and Preliminary Outcomes},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VX2M5J77}},
  note         = {Machine review of arXiv:1907.10192}
}
read the original abstract

This research aims to design and develop a new requirements prioritization approach for analyzing and prioritizing stakeholders requirements which are mentioned in the feedback for software products. This paper presents a PhD research agenda and preliminary outcomes from the early analysis. A roadmap to the proposed research methodology that is to be followed to achieve the targeted outcomes is also outlined. Outcomes to date show that the requirements prioritization problem has been researched extensively, however, gaps still remain when considering techniques that handle a large number of crowdsourced requirements. Furthermore, requirements prioritization as a problem affects many domains beyond software engineering. Hence, knowledge from other fields could be useful for informing requirements prioritization practice in the software engineering space.

Discussion (0). Sign in to comment.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

18 extracted references · 18 canonical work pages

  1. [1]

    Sommervile, I., Software Engineering. 9 ed. 2009

    work page 2009

  2. [2]

    cited 2017; Available from: https://www.business2community.com/mobile-apps/2017-mobile-app-market- statistics-trends-analysis-01750346

    Saifi, R. cited 2017; Available from: https://www.business2community.com/mobile-apps/2017-mobile-app-market- statistics-trends-analysis-01750346

    work page 2017

  3. [3]

    2013, ACM: Chicago, Illinois, USA

    Fu, B., et al., Why people hate your app: making sense of user feedback in a mobile app store, in Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining. 2013, ACM: Chicago, Illinois, USA. p. 1276-1284

    work page 2013

  4. [4]

    2017, ACM: Limassol, Cyprus

    Su'a, T., et al., QuickReview: A Novel Data-Driven Mobile User Interface for Reporting Problematic App Features, in Proceedings of the 22nd International Conference on Intelligent User Interfaces. 2017, ACM: Limassol, Cyprus. p. 517-522

    work page 2017

  5. [5]

    Configuring crowdsourcing for requirements elicitation

    Hosseini, M., et al. Configuring crowdsourcing for requirements elicitation. in 2015 IEEE 9th International Conference on Research Challenges in Information Science (RCIS). 2015

    work page 2015

  6. [6]

    Information and Software Technology, 2014

    Achimugu, P., et al., A systematic literature review of software requirements prioritization research. Information and Software Technology, 2014. 56(6): p. 568-585

    work page 2014

  7. [7]

    Iacob, C. and R. Harrison. Retrieving and analyzing mobile apps feature requests from online reviews. in 2013 10th Working Conference on Mining Software Repositories (MSR). 2013

    work page 2013

  8. [8]

    Khan, and A

    Malgaonkar, S., A. Khan, and A. Vichare. Mixed bilingual social media analytics: case study: Live Twitter data. in 2017 International Conference on Advances in Computing, Communications and Informatics (ICACCI) . 2017

    work page 2017

Show all 18 references
  1. [9]

    Berander, P. and A. Andrews, Requirements Prioritization, in Engineering and Managing Software Requirements, A. Aurum and C. Wohlin, Editors. 2005, Springer Berlin Heidelberg: Berlin, Heidelberg. p. 69-94

    2005

  2. [10]

    cited 2017; Available from: https://play.google.com/store?hl=en

    Store, G.P. cited 2017; Available from: https://play.google.com/store?hl=en

    2017

  3. [11]

    cited 2017; Available from: https://www.apple.com/nz/ios/app- store/

    Store, A.A. cited 2017; Available from: https://www.apple.com/nz/ios/app- store/

    2017

  4. [12]

    Savarimuthu, and S.A

    Keertipati, S., B.T.R. Savarimuthu, and S.A. Licorish, Approaches for prioritizing feature improvements extracted from app reviews , in Proceedings of the 20th International Conference on Evaluation and Assessment in Softwar e Engineering. 2016, ACM: Limerick, Ireland. p. 1 -6

    2016

  5. [13]

    Savarimuthu, and S

    Licorish, S.A., B.T.R. Savarimuthu, and S. Keertipati, Attributes that Predict which Features to Fix: Lessons for App Store Mining, in Proceedings of the 21st International Conference on Evaluation and Assessment in Software Engineering. 2017, ACM: Karlskrona, Sweden. p. 108-117

    2017

  6. [14]

    Information and Software Technology, 2007

    Karlsson, L., et al., Requirements engineering challenges in market-driven software development – An interview study with practitioners. Information and Software Technology, 2007. 49(6): p. 588-604

    2007

  7. [15]

    Rayward-Smith, and I.M

    Bagnall, A.J., V.J. Rayward-Smith, and I.M. Whittley, The next release problem. Information and Software Technology, 2001. 43(14): p. 883-890

    2001

  8. [16]

    Cleland-Huang, and C

    Laurent, P., J. Cleland-Huang, and C. Duan. Towards Automated Requirements Triage. in 15th IEEE International Requirements Engineering Conference (RE 2007). 2007

    2007

  9. [17]

    2008, BCS Learning & Development Ltd.: Italy

    Petersen, K., et al., Systematic mapping studies in software engineering, in Proceedings of the 12th international conference on Evaluation and Assessment in Software Engineering. 2008, BCS Learning & Development Ltd.: Italy. p. 68-77

    2008

  10. [18]

    Requirements prioritization techniques and different aspects for prioritization a systematic literature review protocol

    Sher, F., et al. Requirements prioritization techniques and different aspects for prioritization a systematic literature review protocol . in 2014 8th. Malaysian Software Engineering Conference (MySEC). 2014

    2014

Pith tools

Reviewed May 24, 2026 · model on record in the stance chip above.