{"id":"17fc43da-6009-42c4-b7bb-7e4c6415bee4","arxiv_id":"2506.05836","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A systematic literature review extracts 17 parameters affecting serverless cost savings, though its study count is inconsistent across the paper (34 vs 23).","lead":"This paper surveys published research on the cost savings of serverless cloud computing and lists 17 parameters that influence those savings. It is a reference checklist for engineers and researchers, not a new experiment.","discovery_kind":"review","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Paper's own methodology contradicts the stated count of studies: Section III says 23 relevant papers, while the abstract and conclusion say 34, so the basis of the parameter catalog is inconsistently reported.","rationale":"The reader's weakest_assumption focused on the single-engine search protocol and potential incompleteness. That is a valid methodological threat, but the more immediate and load-bearing concern is the paper's own numerical inconsistency: the abstract and conclusion say 34 studies, while the methodology says 23. The central claim explicitly cites the count of related studies, so a contradiction in that count undermines the claim's factual basis. The reader did note this contradiction in the rationale, but chose the search protocol as the weakest assumption. I partially agree: both are real issues, but the internal contradiction is more concrete and directly testable. It does not necessarily invalidate the whole parameter taxonomy — the 17 parameters may still be plausible — but it must be resolved before the systematic-review claim can be accepted. Since the reader's verdict is CONDITIONAL, and this concern is consistent with that verdict (it is a correctable reporting/validation issue), no change to the verdict is needed.","tokens_in":9726,"tokens_out":2624,"duration_ms":27444,"concrete_test":"Count the unique references cited in Table III for the 17 parameters. If the union of these references is 23, then the abstract's '34 related studies' is unsupported. If the union is 34, then Section III's '23 relevant papers' is incorrect. Also check whether any parameter cites a source that would have been excluded by EC1/EC3 (e.g., survey papers such as [24], or industrial experience reports), which would reveal whether the inclusion/exclusion criteria were actually applied consistently.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that the authors 'identified 34 related studies, from which we extracted 17 parameters.' However, Section III (Methodology) states: 'Altogether, we identified 23 relevant papers, from which we extracted 17 parameters.' This is a difference of 11 studies, with no explanation or reconciliation anywhere in the paper. The abstract and conclusion repeat 34, while the methodology section — the only place where the refinement outcome is described — says 23. If the true number is 23, the stated central claim is factually wrong. If the true number is 34, the methodology's final count is wrong and the selection process is misreported. The paper does not provide a list of the included studies, so a reader cannot determine which count is accurate or reproduce the exclusion decisions. This inconsistency directly affects the claimed basis of the parameter catalog: the catalog is presented as derived from a systematic review, but the size of the underlying study set is contradictory. The search-protocol limitations (single engine, one search string, no snowballing) are secondary; this internal contradiction is a concrete, observable flaw in the paper's own reporting of its core evidence base.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper reports a systematic literature review (SLR) of research on cost effectiveness and cost savings of serverless computing, following Kitchenham's method. The authors state they identified 34 related studies (23 in the methodology section) and extracted 17 parameters that influence relative cost savings, grouped into core resources, code, execution, and data/architecture. For each parameter, Table III provides supporting references, and Figure 1 gives a qualitative correlation map. The paper also describes the search protocol, inclusion/exclusion criteria, and limitations.","tokens_in":10052,"tokens_out":5503,"duration_ms":53278,"significance":"If the parameter catalog is accepted, it provides a practically useful checklist of cost-relevant factors for serverless systems and a qualitative causal map. The paper's strength is that each parameter is linked to external sources, providing a starting point for evidence-based cost modeling. However, the reliability of the catalog as a systematic-review product is currently limited by the inconsistent study count and the absence of a machine-readable list of included studies and screening artifacts; these issues are fixable and do not necessarily invalidate the parameter set.","major_comments":[{"comment":"The manuscript reports two different counts of the underlying study set: the abstract and conclusion state '34 related studies,' while Section III (Methodology) states 'Altogether, we identified 23 relevant papers.' Since the central claim is that 17 parameters were extracted from those studies, an inconsistent count directly affects the credibility of the catalog. The authors should correct the count, or provide a reconciliation (e.g., a typo) and, crucially, list the included studies so the reader can audit which count is correct.","section":"III (Methodology) and Abstract/Conclusion"},{"comment":"The search protocol is not fully reproducible: the exact date of the Google Scholar search is absent, the claim that the search used 'title, abstract and keywords fields' is not supported by Google Scholar's search behavior, and no screening log (number of records assessed, excluded at each stage) is provided. Section V states that the methodology 'would allow us to reproduce our SLR,' but the description does not allow an independent replication. In particular, the stop rule of three consecutive irrelevant pages was reached at page 25, meaning only about 250 of the 11,000 results were examined; this coverage limitation should be justified or explicitly acknowledged as a threat to completeness.","section":"III (Search protocol) and V (Limitations)"},{"comment":"The process of extracting and consolidating the 17 parameters is not documented. The paper states the selected papers were 'analysed to extract the parameters' but does not describe the extraction form, the naming and merging rules, or how disagreements were resolved. Similarly, Figure 1 is presented as a correlation map without describing how the correlations were determined or whether they are based on the cited studies or on the authors' reasoning. Without this detail, the 'systematic' status of the parameter set and its correlation map is weakened.","section":"IV (Analysis of collected data)"},{"comment":"The limitations section does not address the internal inconsistency of the study count (23 vs. 34). It also does not discuss the implication of the relevance-sorted stop rule: if relevant studies appear beyond the first three consecutive irrelevant pages, the parameter list could be incomplete. The authors should either add these as threats or provide evidence that saturation was achieved (e.g., a cumulative plot of new parameters per page).","section":"V (Limitations and threats to validity)"}],"minor_comments":[{"comment":"The outline says 'SRL' but the correct acronym is 'SLR' (Systematic Literature Review).","section":"I (Outline)"},{"comment":"The conclusion contains 'the the relative cost savings' with a duplicated article, and it repeats the '34' count without reconciling it with the '23' stated in Section III.","section":"VI (Conclusions)"},{"comment":"The subsection on 'Research Embedded in Teaching' is not related to serverless cost efficiency and reads as a program description; consider moving it to an acknowledgments-like section or removing it from the related work.","section":"II.C (Research Embedded in Teaching)"},{"comment":"The phrase 'the search has been conducted using the title, abstract and keywords fields' should be clarified, as Google Scholar does not support field-restricted searching in the same way as publisher databases.","section":"III (Methodology)"},{"comment":"Some parameters (e.g., P12, P14, P16) are supported by a single reference; for a systematic review, the authors should at least comment on the weight of evidence for each parameter.","section":"IV (Table III)"},{"comment":"Figure 1 has no legend or explanation of the notation (e.g., arrows, signs); a brief caption or footnote would help interpret the correlations.","section":"IV (Figure 1)"}],"recommendation":"major_revision","confidential_remarks":"The paper appears to be an early version of a student-led SLR. The inconsistency in study count suggests that the final editing was not careful. I recommend the editor ask the authors to provide a supplementary file with the full list of included studies and a PRISMA-style flow diagram. If the authors cannot provide such artifacts, the paper's contribution as a systematic review should be reclassified as a less formal survey. Given the journal's standards, major revision is appropriate."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"This is a modest, clearly written systematic review that gives practitioners a usable checklist of 17 parameters influencing serverless cost savings, grouped into core resources, code, execution, and data/architecture. The taxonomy is plausible and each parameter carries references to primary studies; the correlation diagram is a reasonable qualitative summary of how the parameters interact. If you need a compact overview of what to watch when estimating serverless costs, this is a fine starting point.\n\nThe soft spot is not the taxonomy—it is the reporting. The abstract and conclusion both say the authors identified 34 related studies, but the methodology section, which is the only place the selection outcome is described, says 23. That is an 11-paper discrepancy with no explanation anywhere in the paper. It directly undermines the central claim because the 17-parameter catalog is presented as derived from a systematic review, and the reader cannot tell which count is accurate. The paper provides no list of included studies, no screening logs, and no exclusion details, so the selection process is not reproducible from the text alone. The search protocol is also thin: one search string, one engine, no snowballing, and a stop rule of three consecutive irrelevant pages. These are not fatal to the taxonomy, but they are exactly the kinds of omissions that make a review hard to trust.\n\nA few smaller issues: several parameters are supported by only one or two references (P14, P16, P17), so their status as general cost factors is weaker than the rest. The correlation diagram appears to be an informal judgment rather than a derived model, which is fine as exposition but should be labeled as such. The limitations section mentions reproducibility threats but does not acknowledge the count mismatch.\n\nOverall, the paper is a legitimate synthesis effort, not a flawed idea. The flaws are concrete and fixable: the authors need to reconcile the count, provide the full list of included studies, and ideally publish their screening data. Without those, the review cannot be verified. With them, it would be a useful contribution to a small but active subfield.\n\nFor peer review: yes, I would send this to referees. The topic is real, the parameter set is useful, and the problems are addressable in revision. It would be a disservice to desk-reject outright, but it should not be accepted in its current form.","headline":"Useful 17-parameter taxonomy for serverless cost factors, but the paper's own methodology contradicts the central study count (34 vs 23) and the evidence base is not reproducible as reported.","tokens_in":10458,"tokens_out":1587,"would_cite":false,"duration_ms":17327,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A systematic review identifies 17 parameters that determine whether serverless computing delivers cost savings.","keywords":["serverless computing","cost efficiency","cost savings","systematic literature review","cloud computing","function-as-a-service","cold start","parameter catalog"],"falsifier":"Run the same review protocol in a second scholarly search engine and with backward snowballing from the 34 selected studies; if reasonable new parameters beyond the 17 appear, the catalog is incomplete. Alternatively, an empirical measurement where a parameter not in the list, such as provider-specific pricing tiers, changes the relative cost enough to flip a serverless-versus-server-based recommendation would also falsify the completeness claim.","tokens_in":9550,"feed_emoji":"☁️","tokens_out":3592,"duration_ms":31640,"temperature":0.7,"pith_summary":"This paper is a systematic literature review asking what determines whether serverless computing actually saves money compared with running servers. After screening papers published between 2010 and 2024, the authors identify 34 relevant studies and extract from them 17 parameters that can influence the relative cost savings of the serverless approach. These parameters are organized into four groups: core resources (memory, computing power), code characteristics (reuse, packaging, complexity, language), execution behavior (number of hits, auto-scaling, cold versus warm starts, execution time, concurrency), and data/architecture choices (data location, scheduling, clustering, payload, scale, extra resources). The paper also gives a qualitative map of how these parameters push against one another, such as more memory raising the cost per invocation but lowering execution time.","feed_headline":"17 parameters decide when serverless really saves money","feed_subtitle":"A review of 34 studies maps the cost levers of serverless computing.","key_machinery":"The central object is the 17-parameter catalog (Table III), grouped into core resource, code-related, execution, and data/architecture parameters. The catalog is produced by a four-phase systematic literature review: selecting a primary literature source, running an automated search with a single search string, refining the results with inclusion and exclusion criteria, and then extracting parameters from the remaining papers. The parameter list carries the argument because the review's answer to its research question is exactly this list, with a qualitative correlation map describing how the parameters interact.","core_discovery":"The central claim is that the cost-effectiveness of serverless is not one question but a constellation of 17 parameters, and that these parameters can be meaningfully catalogued from existing research. The paper presents Table III as the identified set, P1 through P17, each tied to the studies that reported it, and Figure 1 as the correlation sketch. On the paper's own terms, anyone deciding whether to adopt serverless, or how to optimize an existing serverless application, should reason through these parameters rather than assume serverless is inherently cheaper.","pith_inferences":["The 17 parameters could be turned into a quantitative cost model if each were assigned a weight based on measured cloud pricing; the paper stops at the qualitative stage.","Because the list comes from a single search strategy, a different search protocol might surface additional parameters, so the catalog should be treated as a starting taxonomy rather than a closed set.","The correlation graph implies directional relationships that could be tested empirically, for example by measuring how execution time varies with memory allocation on a real function-as-a-service platform.","The same parameter lens could be applied to neighbouring cost questions, such as managed databases or container-based platforms, to see whether the four parameter groups generalize."],"forward_implications":["Adoption decisions can be framed as a cost model: an application with many cold starts, heavy payloads, or large concurrency demands will erode the savings that serverless offers.","Optimization efforts now have a checklist: memory size, packaging, code reuse, start type, and scheduling are all independently addressable levers.","The correlation map suggests trade-offs such as memory versus execution time, meaning cost tuning is a balancing act rather than a single setting.","The list gives a common vocabulary for comparing studies of serverless cost, which is a direct product of the categorization."],"supporting_citations":[{"why":"Supplies the systematic literature review methodology the paper follows.","marker":"[34]"},{"why":"Reports allocated memory as a cost-influencing parameter.","marker":"[41]"},{"why":"Reports memory, computing power, number of hits, code complexity, and auto-scaling as cost parameters.","marker":"[18]"},{"why":"Reports cold start behavior and programming language runtime as factors affecting cost.","marker":"[29]"},{"why":"Provides expert-interview evidence that application scale influences cost-effectiveness.","marker":"[26]"},{"why":"Reports memory, number of hits, and execution time as parameters in cost-aware resource procurement.","marker":"[25]"},{"why":"Supplies the data location, decentralized scheduling, and task clustering parameters.","marker":"[11]"}],"fun_headline_variants":["Serverless costs hinge on 17 parameters, survey finds","17 levers decide if serverless pays off: review of 34 studies","17 parameters dictate serverless cost savings, says 34-study review","Serverless cost efficiency: 17 parameters from 34 studies","What makes serverless cheap? 17 parameters, 34 studies"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The assumption that a single relevance-sorted search using one search string and a three-page stop rule finds enough of the relevant literature to define a comprehensive parameter set.","fun_headline_variants_meta":{"raw":{"variants":["Serverless costs hinge on 17 parameters, survey finds","17 levers decide if serverless pays off: review of 34 studies","17 parameters dictate serverless cost savings, says 34-study review","Serverless cost efficiency: 17 parameters from 34 studies","What makes serverless cheap? 17 parameters, 34 studies"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000617,"raw_usage":{"total_tokens":2736,"prompt_tokens":685,"completion_tokens":2051,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":301,"completion_tokens_details":{"reasoning_tokens":1961}},"tokens_in":301,"tokens_out":2051,"duration_ms":15083,"temperature":1.0,"reasoning_tokens":1961,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T10:12:50.553978+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same review protocol in a second scholarly search engine and with backward snowballing from the 34 selected studies; if reasonable new parameters beyond the 17 appear, the catalog is incomplete. Alternatively, an empirical measurement where a parameter not in the list, such as provider-specific pricing tiers, changes the relative cost enough to flip a serverless-versus-server-based recommendation would also falsify the completeness claim.","supporting_citations":[{"cited_title":"Kitchenham, Tore Dyba, and Magne Jorgensen","cited_arxiv_id":null,"evidence_quote":"Supplies the systematic literature review methodology the paper follows."},{"cited_title":"Serverless computing: Design, implementation, and performance","cited_arxiv_id":null,"evidence_quote":"Reports allocated memory as a cost-influencing parameter."},{"cited_title":"Be wary of the economics of”” serverless”” cloud computing.IEEE Cloud Computing, 4(2):6–12, 2017","cited_arxiv_id":null,"evidence_quote":"Reports memory, computing power, number of hits, code complexity, and auto-scaling as cost parameters."},{"cited_title":"An investigation of the impact of language runtime on the performance and cost of serverless functions","cited_arxiv_id":null,"evidence_quote":"Reports cold start behavior and programming language runtime as factors affecting cost."},{"cited_title":"Understanding cost dynamics of serverless computing: An empirical study","cited_arxiv_id":null,"evidence_quote":"Provides expert-interview evidence that application scale influences cost-effectiveness."},{"cited_title":"Spock: Ex- ploiting serverless functions for slo and cost aware resource procurement in public cloud","cited_arxiv_id":null,"evidence_quote":"Reports memory, number of hits, and execution time as parameters in cost-aware resource procurement."},{"cited_title":"Wukong: a scalable and locality-enhanced framework for serverless parallel computing","cited_arxiv_id":null,"evidence_quote":"Supplies the data location, decentralized scheduling, and task clustering parameters."}],"review_version":1}