{"id":"d79777cc-fa81-4708-a549-e67d913dda4a","arxiv_id":"1908.06492","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":2,"one_line_summary":"SAM is an unevaluated proposal to detect and repair API misuses by flagging usage patterns that are statistically rare in a large corpus of production code.","lead":"This paper describes a proposed system, SAM, that would learn typical ways to use programming APIs from large code collections and then flag and automatically repair code that uses APIs in unusual ways. If it worked, it could reduce debugging time caused by API misuse bugs, but the paper contains no experiments showing that it actually works.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central claim depends on the unvalidated assumption that statistical rarity equals API misuse; without an evaluation against a ground-truth benchmark, detection and repair are unsupported.","rationale":"The reader's verdict is REJECT with high confidence. The paper is a proposal with no implementation, no evaluation, no formal proof of the claimed optimal search, and key parameters left unspecified. The weakest assumption identified by the reader is exactly the one I find most load-bearing: statistical rarity of a usage factor is treated as synonymous with API misuse, with no ground-truth validation. The paper's own references to MuBench and the 144-misuse dataset make the absence of evaluation particularly conspicuous. I considered whether the lack of a proof for the 'optimal search' claim is the more fundamental issue, but even a correct repair search would be useless if the detection signal does not identify real misuses; conversely, if detection were reliable, the repair claim would still need proof but could be tested empirically. Thus the rarity-to-misuse mapping is the single most load-bearing concern. Because the reader already rejects the paper on this basis and my analysis adds no new reason to accept it, the verdict should remain unchanged.","tokens_in":5644,"tokens_out":2039,"duration_ms":21543,"concrete_test":"Implement the Section II counting models over a large code corpus, then run the proposed detector on MuBench's 144 documented API misuses and on a matched sample of correct usages from the same projects. Tune the 'sufficiently low probability' threshold by cross-validation and report precision and recall at the operating point. If precision at usable recall is not clearly above the misuse base rate, the rarity-as-misuse assumption is not reliable. As a secondary check on the repair claim, apply CORRECT-API-MISUSES to the misuses it detects and verify whether it terminates within MaxLength and whether its output matches the developer fix or passes the project's tests, comparing against ExAssist.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The claim that SAM can 'detect and repair API misuses automatically' (Abstract) requires that a low-probability usage factor be a reliable signal of actual misuse. Section II defines P(m_i|m) approximately as (N(m_i m)+1)/(N(m)+1) and states that any factor with a 'sufficiently low probability' is treated as an API misuse, but it never specifies a threshold and never validates this mapping against ground truth. The training corpus is described only as 'high-quality production code'; if that corpus contains misuses, or if rare usages are mostly domain-specific idioms, the counts conflate correct specialized usage with errors. The paper itself cites MuBench [8] and reports a study of 144 real API misuses, yet provides no precision/recall evaluation on that benchmark or any other. Separately, Figure 2's CORRECT-API-MISUSES is called an 'optimal search' but no proof is given that the search space is finite, that MaxLength is chosen appropriately, or that GENERATE-REPAIR-ACTIONS enumerates all relevant repairs; the repair subclaim is therefore also unsupported. The most load-bearing issue is the rarity-to-misuse mapping: if that mapping fails, both detection and the repair loop built on detection fail.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SAM, a statistical approach for automatically detecting and repairing API misuses. SAM trains statistical models on a large corpus of high-quality production code for five usage factors of an API method call: temporal order, preconditions, postconditions, argument values, and exception handling. At detection time, any factor with a \"sufficiently low probability\" is treated as an API misuse, and the repair step is framed as an optimal search over editing operations that remove these low-probability factors. The manuscript motivates the five factors using a dataset of 144 real API misuses from MuBench and defines a bigram-style probability estimate for the temporal-order factor, but it does not describe an implementation, report experiments, or provide formal guarantees about the search procedure.","tokens_in":6038,"tokens_out":3318,"duration_ms":35296,"significance":"If the central assumptions were validated, this work could meaningfully address a known limitation of API-misuse detectors, namely their low accuracy and their inability to repair detected misuses. The paper appropriately draws on the public MuBench benchmark and a 144-case study to motivate the five-factor taxonomy, which is a reasonable step toward organizing the problem. However, as submitted, the manuscript is essentially a research proposal: it contains no implementation, no evaluation, no parameter settings, and no formal analysis of the search procedure. The main scientific claim that statistical rarity equates to API misuse is plausible but entirely unvalidated, and the repair claim is unsupported by any termination, completeness, or optimality argument. The paper's significance cannot be assessed on the evidence presented.","major_comments":[{"comment":"The load-bearing claim that a usage factor with a \"sufficiently low probability\" constitutes an API misuse is never validated. The paper neither specifies the threshold nor reports any precision/recall evaluation against MuBench [8] or any other ground-truth dataset, despite citing a study of 144 real API misuses in Section I that could have been used to set or test the threshold. Because the trained models are the only detection criterion, the entire approach fails if the training corpus contains misuses or if rare usage is mostly domain-specific idiom rather than error; the manuscript provides no evidence against those failure modes. At minimum, the authors should report detection precision/recall on MuBench and compare with existing detectors.","section":"Section II (Detection)"},{"comment":"The repair procedure is described as an \"optimal search\", but no optimality criterion, cost function, or completeness argument is provided. The paper does not prove that the search space is finite, that GENERATE-REPAIR-ACTIONS enumerates all relevant edits, or that MaxLength is chosen appropriately; moreover, the recursion shown in line 10 discards the returned candidates without selecting the best one, so the algorithm as written cannot be expected to return an optimal corrected program. These gaps are load-bearing because automatic repair is one of the paper's two central contributions.","section":"Figure 2 / Section II (Repair algorithm)"},{"comment":"The statistical model is underspecified. The equation P(mi|m) ≈ (N(mim)+1)/(N(m)+1) is not a standard smoothed conditional-probability estimate as written (no vocabulary-size term appears), and the paper does not explain how the probabilities of the five usage factors are combined into a single detection decision, how preconditions and postconditions are represented in the count-based models, or what concretely constitutes \"high-quality production code\" for training. These details are necessary for reproducibility and for assessing whether low probability can be equated with misuse.","section":"Section II (Probability model)"}],"minor_comments":[{"comment":"In the definition of temporal order, the sentence \"The method mi appears right before mi\" should read \"The method mi appears right before m\".","section":"Section II (Temporal order definition)"},{"comment":"The caption mentions FileOutputStream while the code snippet creates a FileInputStream; the caption and code should be consistent.","section":"Figure 1 caption"},{"comment":"The recursive call CORRECT-API-MISUSES(C', L+1) has no mechanism to collect or compare the returned candidates, which makes the labeling of the search as \"optimal\" misleading; a selection step should be shown or described.","section":"Figure 2 line 10"},{"comment":"The paper gives no high-level description of how precondition, postcondition, argument-value, and exception factors are extracted and counted from source code, so a reader cannot tell whether the approach is implementable as described.","section":"Section II (Training)"},{"comment":"The bibliography contains formatting inconsistencies, such as reference [10] lacking volume and page details and references [9] and [21] having identical author lists despite different titles; the reference list should be cleaned up.","section":"References"}],"recommendation":"reject","confidential_remarks":"This manuscript is an extended-abstract-style proposal that would be better suited to a workshop or a new-ideas track. The absence of any implementation or evaluation, combined with the unproven optimality and unvalidated rarity-to-misuse mapping, places it below the bar for a full journal paper. The authors should first implement SAM, calibrate the threshold on MuBench, and report precision/recall and repair-success rates before resubmitting."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a four-page proposal, not a demonstrated result. The authors propose SAM, which uses five statistical usage factors to detect API misuses and then repairs them via search. The idea is coherent and the taxonomy is well grounded in their 144-misuse analysis, but the paper provides no implementation, no evaluation, no threshold, and no proof of the search claims. The right venue for this is a workshop or a position statement, not a peer-reviewed venue.\n\nWhat is good: the five factors (temporal order, preconditions, postconditions, argument values, exceptions) map naturally to the root causes in the MuBench data, and framing correction as a search over edits is a sensible direction that goes beyond detection alone. The bigram probability estimate P(mi|m) ≈ (N(mim)+1)/(N(m)+1) is standard smoothed counting, and the claim that this is easy to train is fair.\n\nWhere it falls down: the central mapping from low probability to misuse is asserted, not validated. The paper cites MuBench and a prior study of 144 real misuses, but offers no precision/recall on that benchmark or any ground-truth set. Rarity could just as easily be a domain-specific idiom. The training corpus is only described as 'high-quality production code,' so the counts themselves may contain misuses. No probability threshold is specified, which makes the detector parameter-dependent. The repair algorithm in Figure 2 is a sketch: there is no termination proof, no bound on the search space, no argument that MaxLength is enough, and no proof that GENERATE-REPAIR-ACTIONS covers the relevant edits. 'Optimal' is never defined. On novelty, the five factors overlap with JADET, Alattin, ExAssist, and the authors' own HMM work, so the only new piece is the repair search, and that is the thinnest part.\n\nThe citation pattern is fine; the self-citations are to closely related prior work and are not being used to hide a flaw. The paper reads as an honest research proposal, just a very early one.\n\nWho is this for? Someone curating a reading list on API misuse tools might skim it for the taxonomy, but I wouldn't cite it as a result. If it came to me for peer review, I would desk reject it in its current form and invite the authors to resubmit once they have an implementation, a threshold policy, and an evaluation on MuBench. The idea is not bad; the evidence is absent.","headline":"A plausible but entirely unevaluated proposal: the five-factor misuse taxonomy is reasonable, yet without implementation or evaluation the rarity-to-misuse mapping and the 'optimal repair' claims are unsupported.","tokens_in":6408,"tokens_out":2245,"would_cite":false,"duration_ms":22976,"reading_group":"no","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Statistical rarity in API usage is treated as misuse and repaired automatically.","keywords":["API misuse detection","API misuse repair","statistical program analysis","temporal order of method calls","exception handling","preconditions and postconditions","argument values","corpus-trained code models"],"falsifier":"Run the detection rule on a clean corpus of known-correct code and count how many calls have at least one usage factor below the proposed threshold; a substantial false-positive rate would show that rarity is not equivalent to misuse.","tokens_in":5469,"feed_emoji":"🔧","tokens_out":11235,"duration_ms":103017,"temperature":0.7,"pith_summary":"This paper argues that API misuse can be detected and repaired automatically by treating correct usage as a statistical tendency in a large body of production code. It defines five usage factors for any method call—the temporal order of surrounding calls, exception handling, preconditions, postconditions, and argument values—and trains a probability model for each by counting occurrences in the corpus. A call with any factor below a low-probability threshold is labeled an API misuse, and correction becomes an optimal search for editing operations that removes all low-probability factors. If the approach holds, developers would get a single detector that both locates API errors and suggests concrete fixes, an ability the paper says existing detectors lack.","feed_headline":"Rare API use is treated as misuse—and repaired automatically","feed_subtitle":"One trained model covers call order, exceptions, pre/postconditions, and argument values to detect and fix errors","key_machinery":"The working machinery is a set of five statistical usage-factor models attached to each API method call, trained purely by occurrence counting over a large corpus of production code. Each factor has a probability: temporal order $P_\\alpha$, precondition $P_\\beta$, postcondition $P_\\gamma$, argument value $P_\\delta$, and exception handling $P_\\epsilon$. The central identity is the smoothed bigram estimate $P(m_i|m)\\approx(N(m_i m)+1)/(N(m)+1)$, which converts raw co-occurrence counts into a usable probability for the order factor; the other factors are assigned analogous count-based distributions. Correction is driven by a recursive optimal-search procedure that detects the low-probability factors, generates repair actions for each, applies each action, and recurses until no factor is low or a maximum edit length is reached.","core_discovery":"SAM's central claim is that API misuse is a measurable deviation from statistically normal usage, not a property that must be specified by hand. For each method call $m$, the paper models five usage factors: the probability $P(m_i|m)$ that another method call $m_i$ appears immediately before $m$ in the temporal order, the probability that a parameter is null-checked before the call (precondition), the probability that the return value or object state is checked after the call (postcondition), the probability distribution of argument values, and the probability of the exception-handling context. The temporal-order estimate is $P(m_i|m)\\approx(N(m_i m)+1)/(N(m)+1)$, a smoothed count ratio, and analogous count-based estimates are proposed for the other factors. Once trained, the detector flags any call with at least one factor below threshold; the repair phase searches over editing operations, up to a maximum edit length, until no factor remains below threshold. The intended result is a unified, corpus-trained approach that covers five different misuse categories in one pipeline.","pith_inferences":["A natural next step the paper does not take is to condition the probability estimates on the specific API or application domain, so that rare-but-correct idioms in a niche library are not flagged as misuses.","The optimal search could be made practical by ranking repair actions according to how much they raise the lowest probability factor, rather than exploring all edits up to a fixed maximum edit length.","The five-factor claim is directly testable against a public benchmark of documented misuses: if most documented misuses do not have a low-probability factor, the rarity assumption would need revision."],"forward_implications":["A tool built on this design could take a failing code snippet and return both a warning and an automatically edited version with the low-probability factors removed.","Because training is counting, the models can be retrained on larger or more specialized corpora, and the paper claims accuracy would improve with more production code.","The five-factor decomposition means one pipeline can address wrong call order, missing preconditions, missing postconditions, bad argument values, and exception-handling mistakes that earlier tools targeted separately.","The same counting-and-search recipe transfers, in principle, to any API or programming language where a sufficiently large corpus of high-quality code is available."],"supporting_citations":[{"why":"Supplies the 144-misuse dataset whose root-cause breakdown motivates the five usage factors.","marker":"[8]"},{"why":"Documents the low accuracy of existing static API-misuse detectors, motivating a new statistical approach.","marker":"[10]"},{"why":"Introduces object-usage anomaly detection, the precursor that SAM generalizes into probability-based factors.","marker":"[11]"},{"why":"Provides graph-based mining of object usage patterns, the basis for the temporal-order factor.","marker":"[12]"},{"why":"Establishes the majority-rule violation idea for missing method calls, a direct predecessor of rarity-as-misuse.","marker":"[3]"},{"why":"Prior work on detecting, correcting, and explaining API misuses, the closest existing goal to SAM's repair objective.","marker":"[24]"}],"fun_headline_variants":["Five API call factors, one statistical model: misuse auto-fixed","Detect and repair API misuse with statistical models trained on good code","API misuse: a statistical deviation, now automatically repaired","SAM: statistical models turn API misuse into a search-and-fix problem","From code stats to automated API misuse repair"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that statistically rare usage of an API is usually a mistake: a call with a low-probability factor is treated as a misuse, so if rare usages are actually correct but unusual coding styles, or if the training corpus itself contains misuses, the detector will flag the wrong code.","fun_headline_variants_meta":{"raw":{"variants":["Five API call factors, one statistical model: misuse auto-fixed","Detect and repair API misuse with statistical models trained on good code","API misuse: a statistical deviation, now automatically repaired","SAM: statistical models turn API misuse into a search-and-fix problem","From code stats to automated API misuse repair"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000625,"raw_usage":{"total_tokens":2884,"prompt_tokens":927,"completion_tokens":1957,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":543,"completion_tokens_details":{"reasoning_tokens":1883}},"tokens_in":543,"tokens_out":1957,"duration_ms":14113,"temperature":1.0,"reasoning_tokens":1883,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T12:43:47.902126+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the detection rule on a clean corpus of known-correct code and count how many calls have at least one usage factor below the proposed threshold; a substantial false-positive rate would show that rarity is not equivalent to misuse.","supporting_citations":[{"cited_title":"Mubench: A benchmark for api-misuse detectors,","cited_arxiv_id":null,"evidence_quote":"Supplies the 144-misuse dataset whose root-cause breakdown motivates the five usage factors."},{"cited_title":"A systematic evaluation of static api-misuse detectors,","cited_arxiv_id":null,"evidence_quote":"Documents the low accuracy of existing static API-misuse detectors, motivating a new statistical approach."},{"cited_title":"Mad-api: Detection, correction and explanation of api misuses in distributed android applications,","cited_arxiv_id":null,"evidence_quote":"Prior work on detecting, correcting, and explaining API misuses, the closest existing goal to SAM's repair objective."}],"review_version":1}