{"id":"de444b5c-5022-44ce-a4fe-73670cd6e104","arxiv_id":"2509.06012","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A rapid review finds that requirements engineering research lacks a shared definition of legal requirements and that common claims about them are not backed by empirical evidence.","lead":"This paper reviews 21 open-access papers on how requirements engineering research defines and describes legal requirements. It finds conflicting definitions and little empirical evidence behind commonly repeated claims.","discovery_kind":"review","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Open-access-only sampling frames the literature; Section 4's 'no robust empirical evidence' conclusion is a sampling artifact risk, not a demonstrated field property.","rationale":"After reading the paper, I find the reader's weakest assumption is indeed the most load-bearing. The sampling procedure (Section 2) restricts to open-access English-language papers and adds subjective 'classical' literature. This affects both central conclusions, but especially the second: the claim that the questions about LRs 'seem to lack empirical answers' is an inference from a sample that structurally excludes many empirical venues. The paper's own transparency about the small sample does not neutralize this selection bias. I do not find internal contradictions sufficient to reject the paper; the confusion claim is supported by quoted contradictions, and the author uses cautious language. Thus CONDITIONAL remains appropriate.","tokens_in":6830,"tokens_out":6886,"duration_ms":73757,"concrete_test":"Audit the 21 sampled papers for primary empirical data (surveys, interviews, experiments, case studies) that directly address any Table 1 theme (e.g., whether practitioners perceive LRs as ambiguous or implement them reluctantly). If such data exist in [7], [9], or others, the Section 4 claim is internally contradicted. If none exists, re-run the Scopus query without the open-access filter and screen the full set for empirical studies; if any emerge, the 'lack of empirical answers' conclusion is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central conclusion in Section 4—that 'none of what is present in Table 1 is backed by robust empirical evidence'—rests on a sample that excludes all closed-access papers by design (Section 2: 602 Scopus hits reduced to 154 open-access, then to 21 after exclusions, plus 68 metadata exclusions). Since many empirical RE studies appear in closed venues, the sampling frame is biased against finding empirical evidence; the absence is expected even if the field is replete with such studies. The paper's own sample may even contradict the claim: [7] is an empirical perceptions study, [9] includes an evaluation, and [3] is an Empirical Software Engineering article. At minimum, the blanket assertion requires an audit of empirical content within the sampled papers before generalizing to the field. The author acknowledges the small sample but does not address this specific selection bias against empirical evidence.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper conducts a rapid review of the concept of legal requirements (LRs) in requirements engineering (RE) research. Based on a Scopus search restricted to open-access English-language papers, 21 papers were included after excluding those that did not offer a definition or characterization of LRs. The author augments this sample with subjectively selected 'classical' literature, including his own prior work. The review identifies themes under three research questions: what LRs are (RQ.1), their characteristics (RQ.2), and their effect on engineers' work beyond elicitation (RQ.3). The paper reports that LRs are often treated normatively, sometimes as functional and sometimes as non-functional, and are characterized as ambiguous, complex, changing, overlapping, and reluctantly implemented. The main conclusions are that there is enduring conceptual confusion about LRs and that the recurring claims about them are not backed by robust empirical evidence.","tokens_in":7004,"tokens_out":3471,"duration_ms":39775,"significance":"If the conclusions are accepted, the paper provides a useful critical synthesis of how a core but poorly defined concept in RE research is used, and it identifies a potential gap: many claims about LRs rest on anecdote or opinion rather than empirical data. The review is transparent about its scope and limitations, and it offers concrete quotes from the sampled papers, which supports the observation of heterogeneous conceptualizations. However, the strength of the claimed empirical gap is not warranted by the sampling method, which excludes closed-access venues by design. The paper also raises a plausible and practically important call for empirical studies of how software engineers perceive and handle legal requirements. The contribution is modest but potentially valuable for RE researchers and practitioners seeking a starting point for a more rigorous conceptual foundation.","major_comments":[{"comment":"The claim 'none of what is present in Table 1 is backed by robust empirical evidence' overreaches relative to the sampling frame. Section 2 deliberately restricts the corpus to open-access papers (602 hits reduced to 154 open access, then to 21 after exclusions). This design biases against finding empirical evidence, as many empirical RE studies appear in closed-venue journals and conferences. Moreover, the sample itself contains at least three papers with empirical components: [7] is an empirical perceptions study, [9] includes an evaluation of design patterns, and [3] is published in Empirical Software Engineering. The author should either restrict the conclusion explicitly to the sampled papers or conduct an audit of the 154 open-access papers for empirical content before generalizing to the field.","section":"Section 4"},{"comment":"The augmentation of the systematic sample with 'subjectively selected' classical literature is a source of potential selection bias and mild circularity. The author's own prior work [15,26,27] is included and then used to support themes such as overlaps and non-negotiability. This can inflate support for themes the author already believes in. The paper should specify the criteria for selecting the classical literature and ideally show that the core themes identified from the 21-paper sample do not depend on these additional works. Without this, the conclusion of conceptual confusion may partly reflect the author's own framing.","section":"Section 2"},{"comment":"The thematic coding is not described with a formal protocol. The paper says a thematic approach is used for feasibility, but does not explain how themes were derived, how disagreements were handled, or whether any inter-rater reliability was assessed. For a review that draws critical conclusions about the field, the lack of a transparent coding scheme makes it difficult for readers to verify the mapping from papers to themes. This is a methodological limitation that should be acknowledged and at least partially mitigated by providing example quotes for each theme and clarifying the inclusion/exclusion decision rules for each theme.","section":"Section 3 / Table 1"},{"comment":"In Table 1, the Theme 5 'Reluctance' is associated with references [5,9,10]. However, in the text, the reluctance theme is supported only by quotations from [10] and [9], while [5] is used for the 'timing' theme. This appears to be a citation mismatch. If [5] does indeed support reluctance, a specific quotation or explanation should be provided; otherwise, the table entry should be corrected. Such inconsistencies undermine the reliability of the theme enumeration as presented.","section":"Section 3, 'Reluctance' theme"}],"minor_comments":[{"comment":"The search string and inclusion criteria are described only vaguely: 'from all fields of papers written in English' and the keyword search. Please specify the exact Scopus query, including whether 'legal requirement' was searched as an exact phrase, and list the exclusion criteria (e.g., non-English, non-article) more explicitly for reproducibility.","section":"Section 2"},{"comment":"The footnote about 68 papers being mislabeled as open access is important for transparency. Consider moving this information into the main text so that readers understand the full reduction from 154 to 21 papers, as it affects the credibility of the sample description.","section":"Section 2, footnote"},{"comment":"The phrase 'These characterizations supposedly correlate with knowledge gaps' is vague. Consider rephrasing to clarify whether the correlation is claimed in the literature, inferred by the author, or hypothesized, as this distinction is central to the review's critical argument.","section":"Abstract"},{"comment":"Some quotations include page numbers (e.g., [10, p. 2322]) while others do not (e.g., [6, p. 3] is given but many others are not). For a rapid review, this is acceptable, but consistency would be helpful if the authors want to align with systematic review conventions.","section":"Section 3"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a relevant gap in RE research and is transparent about its limitations. The main technical issue is the overstatement of the empirical-evidence gap given the open-access-only sample. The author should also tighten the method description for the thematic analysis and reconsider the inclusion of self-authored work in the classical augmentation. If these concerns are addressed, the paper could become a useful contribution; as it stands, it overclaims in a way that would likely attract criticism from empirical software engineering reviewers."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick one: this is a useful, honestly hedged rapid review of a real conceptual muddle in RE research, but its headline claim about 'no robust empirical evidence' is too strong for a sample that excludes closed-access venues by design. The thematic table and the quoted contradictions are the actual contribution; the sampling frame is the soft underbelly.\n\nWhat is new: Ruohonen actually goes looking for how 'legal requirements' are defined and characterized, and finds the expected chaos—normative vs non-normative, functional vs non-functional, ambiguous vs stable. The paper does the field a small service by collecting quotations that show these tensions side by side (e.g., [3] vs [10] vs [30]), and by identifying recurring themes (effort, knowledge, reluctance, validation) that are mostly asserted rather than demonstrated. It also credits two existing reviews [17,30] and positions itself as complementing them rather than replacing them. The method is transparent: open-access Scopus, keyword search, manual inclusion, plus a disclosed 'classical' augmentation. The author repeatedly flags the small sample, so the paper does not pretend to be a systematic review.\n\nWhere it goes soft: the stress-test is right. The concluding claim that 'none of what is present in Table 1 is backed by robust empirical evidence' is a sampling artifact risk, not a field-level fact. By restricting to open-access papers, the review systematically excludes many empirical RE studies that appear in closed venues (conferences like RE, journals behind paywalls). That makes the absence of empirical evidence largely an expected consequence of the filter. Worse, even within the sample, [7] is a perceptions survey, [9] includes an evaluation, and [3] is an Empirical Software Engineering article. The author could still defend a weaker claim—that the recurring characterizations are rarely backed by robust, generalizable evidence—but that requires actually auditing empirical content in the sample and stating the limits of the sampling frame explicitly. The citation of the author's own prior work [15,26,27] is fine here because the central observations come from the independently sampled papers, not from those self-citations.\n\nBottom line: this is a candid, short position paper that deserves a serious referee, but the referee should push for a more careful empirical-evidence claim (either audit the sample or qualify the conclusion). It is a good reading-group piece on methodological pitfalls in rapid reviews.","headline":"A candid, small-scale rapid review that documents conceptual disarray about legal requirements, but overreaches when it claims 'no robust empirical evidence' on the strength of an open-access-only sample.","tokens_in":7444,"tokens_out":2177,"would_cite":true,"duration_ms":22782,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Legal requirements in software engineering have no shared definition, and the common claims about them rest on weak evidence, a rapid review argues.","keywords":["legal requirements","requirements engineering","regulatory compliance","rapid review","conceptual confusion","empirical evidence","functional vs non-functional requirements","deontic norms"],"falsifier":"A broader systematic review that searches all 602 initially retrieved papers (not just the open-access subset) for explicit definitions of legal requirements; if many definitions with a shared core are found, the claim of conceptual confusion would be weakened. Alternatively, a large representative survey of requirements engineers asking whether they implement LRs reluctantly and minimally; a clear majority reporting diligent implementation would undercut the folklore claim.","tokens_in":6698,"feed_emoji":"⚖️","tokens_out":4140,"duration_ms":44588,"temperature":0.7,"pith_summary":"The paper is a rapid review of how requirements engineering (RE) research defines and characterizes legal requirements (LRs). It argues that the literature shows a persistent conceptual confusion: LRs are treated as normative but are also placed in both functional and non-functional camps, with few actual definitions or operationalizations. It also argues that recurring claims about LRs being vague, complex, effortful, changing, overlapping, and reluctantly implemented are not backed by robust empirical evidence. A sympathetic reader cares because, if true, RE research and its tools for legal compliance are built on unexamined assumptions rather than measured facts.","feed_headline":"Legal requirements in software lack a shared definition","feed_subtitle":"A rapid review says claims about their vagueness, complexity, and reluctant implementation lack empirical backing.","key_machinery":"The central mechanism is a rapid review design: a Boolean keyword search in a literature database, restricted to open-access English papers, then reduced to the 21 papers that actually define or characterize LRs, and finally coded thematically across three research questions (what LRs are, what characteristics they have, how they affect engineers). The thematic table organizing 31 themes under these three questions is the paper's main conceptual device, augmented by a 'classical' selection of additional literature.","core_discovery":"On its own terms, the paper establishes that among a systematically sampled set of 21 open-access papers plus subjectively chosen classical literature, there is no consistent definition of legal requirements. A normative understanding is common but rarely operationalized, and LRs are variously framed as functional or non-functional requirements. The review catalogs a recurring set of characteristics: ambiguity and complexity, knowledge gaps among engineers, change and overlap, high effort, minimal and reluctant implementation, prioritization driven by penalty risk, late consideration, and validation by regulators or supervisors. Its critical claim is that none of these recurring characteriza","pith_inferences":["The absence of empirical evidence may reflect publication incentives: LRs often appear only as motivation for a proposed tool or technique, so the recurrence of the same characterizations across papers might indicate citation chains rather than independent observation.","A natural test would be a preregistered survey of requirements engineers in regulated industries asking whether they implement LRs reluctantly and minimally; if most report diligent, value-driven implementation, the 'folklore' claim would be weakened.","The paper's hint about deontic versus constitutive norms suggests one constructive path forward: borrowing more rigorously from legal theory categories to give LRs a firmer conceptual footing.","The stabilization hypothesis—that LRs stabilize over time and recur in similar forms—could be tested with longitudinal case studies of how specific legal requirements migrate into software systems across projects and years."],"forward_implications":["If there is no shared conceptual framework for LRs, then tools and methods for legal compliance in RE rest on unstable foundations; a common terminology would be a necessary first step toward stability.","The recurring claims about vagueness, complexity, effort, and reluctance should be treated as hypotheses, not established facts, until tested by empirical studies such as industry surveys.","The unresolved functional versus non-functional classification of LRs has practical consequences for how engineers model, prioritize, and trace legal requirements.","Laws themselves can be measured directly, for example through readability metrics and cross-reference analysis, giving empirical content to claims about legal ambiguity and complexity.","Because this is a rapid review with a deliberately small sample, it does not settle the questions it raises; it reframes them as research gaps for the community."],"supporting_citations":[{"why":"Supplies the definition of rapid review that grounds the paper's methodological choices and feasibility arguments.","marker":"[13]"},{"why":"An existing exhaustive systematic mapping study on RE for regulatory compliance, used as a baseline that does not answer the paper's research questions and as evidence for the complexity causal logic.","marker":"[17]"},{"why":"A systematic literature review on legal contracts to formal specifications, providing an example of normative/deontic treatment of LRs.","marker":"[30]"},{"why":"A heavily cited source for multiple themes (vagueness, knowledge gaps, effort, reluctance, prioritization, stabilization) that the review's critical argument depends on.","marker":"[9]"},{"why":"Provides an alternative conceptualization of LRs as real-world facts and the minimal-baseline compliance claim.","marker":"[10]"},{"why":"Earlier ontology-building research on LRs that motivates the paper's second research question about characteristics.","marker":"[1]"},{"why":"Supplies the 'folklore turned into facts' framing that underpins the paper's critical argument about the lack of empirical evidence.","marker":"[11]"}],"fun_headline_variants":["Legal requirements: no shared definition in RE","Review finds legal reqs lack evidence, definition","Legal reqs are vague, but proof is missing","RE can't agree on what a legal requirement is"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The conclusions depend on the assumption that the 21 open-access English-language papers from one literature database, plus the author's subjectively chosen classical literature, fairly represent how requirements engineering research thinks about legal requirements.","fun_headline_variants_meta":{"raw":{"variants":["Legal requirements: no shared definition in RE","Review finds legal reqs lack evidence, definition","Legal reqs are vague, but proof is missing","RE can't agree on what a legal requirement is"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000184,"raw_usage":{"total_tokens":1115,"prompt_tokens":662,"completion_tokens":453,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":406,"completion_tokens_details":{"reasoning_tokens":393}},"tokens_in":406,"tokens_out":453,"duration_ms":5285,"temperature":1.0,"reasoning_tokens":393,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T04:37:35.937916+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A broader systematic review that searches all 602 initially retrieved papers (not just the open-access subset) for explicit definitions of legal requirements; if many definitions with a shared core are found, the claim of conceptual confusion would be weakened. Alternatively, a large representative survey of requirements engineers asking whether they implement LRs reluctantly and minimally; a clear majority reporting diligent implementation would undercut the folklore claim.","supporting_citations":[{"cited_title":"Journal of Clinical Epidemiology 129, 74–85 (2021)","cited_arxiv_id":null,"evidence_quote":"Supplies the definition of rapid review that grounds the paper's methodological choices and feasibility arguments."},{"cited_title":"Information and Software Technology 178, 107622 (2025)","cited_arxiv_id":null,"evidence_quote":"An existing exhaustive systematic mapping study on RE for regulatory compliance, used as a baseline that does not answer the paper's research questions and as evidence for the complexity causal logic."},{"cited_title":"SN Computer Science 3, 1–25 (2022)","cited_arxiv_id":null,"evidence_quote":"A systematic literature review on legal contracts to formal specifications, providing an example of normative/deontic treatment of LRs."},{"cited_title":"European Journal of Information Systems 33(4), 441–468 (2024)","cited_arxiv_id":null,"evidence_quote":"A heavily cited source for multiple themes (vagueness, knowledge gaps, effort, reluctance, prioritization, stabilization) that the review's critical argument depends on."},{"cited_title":"Electronic Markets 32, 2311–2331 (2022)","cited_arxiv_id":null,"evidence_quote":"Provides an alternative conceptualization of LRs as real-world facts and the minimal-baseline compliance claim."},{"cited_title":"Legal Requirements Analysis","cited_arxiv_id":"2311.13871","evidence_quote":"Earlier ontology-building research on LRs that motivates the paper's second research question about characteristics."},{"cited_title":"The Journal of Systems and Software 148, 170–179 (2019)","cited_arxiv_id":null,"evidence_quote":"Supplies the 'folklore turned into facts' framing that underpins the paper's critical argument about the lack of empirical evidence."}],"review_version":1}