{"id":"540e658e-9cd9-41d3-8dae-0706eb5c25ff","arxiv_id":"2505.24079","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"PCD-DAug uses a diffusion model guided by program-slicing and PCA-derived contexts to generate synthetic failing test cases, improving fault localization effectiveness across six methods.","lead":"This paper introduces PCD-DAug, a data augmentation method that uses program slicing, principal component analysis, and a diffusion model to create synthetic failing test cases for software fault localization. The method is tested on 262 faulty program versions and improves the ranking of faulty statements for six fault localization techniques.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Generated 'failing test cases' are never executed or even explicitly binarized; the reported FL gains may be artifacts of invalid synthetic rows.","rationale":"The reader's weakest assumption identifies the same load-bearing issue: the paper treats synthetic rows as valid failing test cases without executing them, and it omits the conversion from continuous diffusion outputs to binary coverage vectors. I reviewed the manuscript for other potential vulnerabilities, including the misleading average improvement of 383.83% in the abstract, which is obtained by averaging relative improvements across six methods and is dominated by DLFL baselines with very small Top-1 counts. I do not make that the primary concern because even a corrected effect size would still depend on the synthetic-label validity. The absence of binarization and oracle validation is more fundamental: it threatens the basic semantics of every spectrum used in the evaluation, not merely the magnitude of the reported improvement. The reader's conditional verdict already captures this uncertainty, so I recommend keeping the verdict unchanged pending the proposed check. If the check confirms that generated rows are not executable or do not fail, the appropriate verdict would harden to reject because the central empirical claim would then be unsupported.","tokens_in":18062,"tokens_out":7798,"duration_ms":95486,"concrete_test":"Use the replication package on one Defects4J version and instrument the generation script to dump the raw diffusion output before it enters the FL pipeline. First, record the exact binarization/rounding rule and verify that every generated entry is exactly 0 or 1; if no rule exists, the pipeline is invalid as described. Second, for at least 20 generated rows, either reconstruct the corresponding concrete test input and execute it on the faulty program using the Defects4J oracle, or demonstrate that no such mapping exists. Count how many of the executed rows actually fail. If fewer than 100% (or a pre-specified high threshold) of the generated rows fail, or if no row-to-input mapping exists, then 'synthesized failing test cases' is unsupported and the experiments should be rerun with only executable, oracle-confirmed failing rows.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing step is the claim in §3.4 and §4.3 that the diffusion model synthesizes failing test cases. For that claim to support the experimental results, two conditions are needed: (i) the continuous output of the reverse diffusion process must be converted into the binary statement-execution rows that Dstar, Ochiai, Barinel, MLP-FL, CNN-FL, and RNN-FL consume; and (ii) each generated row must actually correspond to a failing execution of the faulty program. Neither condition is demonstrated. Algorithm 2 returns X_fusion by taking columns from the original binary coverage matrix, but the generated rows are never described as being thresholded, rounded, or otherwise binarized; no such rule appears in §3.4 or in the parameter table (Table 2). If raw float values are fed directly to the FL methods, the spectra are not valid coverage vectors. If a threshold is silently applied, the threshold choice is an unstated parameter that can dominate the augmentation effect. Moreover, because a generated row carries no concrete test input, it cannot be executed; the 'failing' label is assigned by construction, and no oracle check is reported. Every Top-K, MFR, and MAR result in Tables 3–6 could in principle come from plausible-looking but semantically meaningless rows. The diffusion model is trained on one or two real failing rows and will tend to emit near-duplicates; whether those near-duplicates are genuinely failure-inducing is exactly what is asserted, not verified. This is load-bearing because without label validity the central claim reduces to 'changing the dataset changes rankings,' not 'PCD-DAug improves fault localization.'","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes PCD-DAug, a data augmentation approach for fault localization that combines dynamic program slicing with a revised PCA to form a 'principal context', and then trains a conditional diffusion model to generate synthetic failing test cases. The augmented, class-balanced dataset is fed to six FL methods (Dstar, Ochiai, Barinel, MLP-FL, CNN-FL, RNN-FL) and compared against six baselines on 262 Defects4J versions. The paper reports large average improvements in Top-1, Top-3, Top-5, MFR, and MAR metrics, but the central pipeline from continuous diffusion outputs to binary coverage vectors, and the validity of the generated failing test cases, is not demonstrated.","tokens_in":18328,"tokens_out":3312,"duration_ms":30744,"significance":"If the reported effects are genuine, incorporating semantic and statistical contexts into a diffusion-based generator would be a useful contribution to FL under class imbalance. The paper's strengths include a large-scale Defects4J evaluation, comparison with several augmentation baselines, and a stated open-source replication package. However, the central empirical claim is currently not supported because the synthetic samples are not validated as failing executions and the conversion of continuous generated data into binary coverage vectors is unspecified. These issues are load-bearing rather than cosmetic, so the paper needs substantial revision before the results can be trusted.","major_comments":[{"comment":"The paper never specifies how the continuous output of the reverse diffusion process is converted into the binary statement-coverage vectors consumed by Dstar, Ochiai, Barinel, MLP-FL, CNN-FL, and RNN-FL. Algorithm 2 returns X_fusion by selecting existing binary columns from X, but the generated rows are not described as being thresholded, rounded, or otherwise binarized, and no such rule appears in §3.4 or in Table 2. If raw float values are passed to the FL methods, the spectra are invalid; if a threshold is applied silently, the threshold choice is an unstated parameter that can dominate the augmentation effect.","section":"§3.4, Table 2"},{"comment":"The generated 'failing test cases' are never executed on the faulty program, and no oracle check is reported. Because a generated row carries no concrete test input, the label 'failing' is assigned by construction. The reported Top-K, MFR, and MAR improvements in Tables 3–6 therefore do not establish that the augmentation adds genuine failure-inducing executions; they may reflect plausible-looking but semantically meaningless rows.","section":"§3.4 and §4.3"},{"comment":"The diffusion model is trained on the same failing test cases that are later combined with synthetic rows and used to compute FL effectiveness. This gives the evaluation a self-referential character: the generator can only recombine information already present in the benchmark. The paper should assess whether the gains persist when the original failing test cases are held out from the augmented set, or when synthetic rows are validated by execution.","section":"§3.4 and §4.3"},{"comment":"The reported average improvements (e.g., 383.83%, 227.08%, and 224.19%) are presented without any statistical significance tests or variance estimates. Since the diffusion model is stochastic and the augmentation procedure depends on random sampling, the paper should report standard deviations or confidence intervals across repeated runs and, where appropriate, significance tests against the baselines.","section":"Tables 3–6 and RQ1–RQ4"}],"minor_comments":[{"comment":"The text uses 'principle contexts' where 'principal contexts' is meant, and 'metris' instead of 'metrics'.","section":"Abstract and §1"},{"comment":"Several cells in Table 3 have numbers run together (e.g., '3 317 8 79 910 10' and '3 31 7 8 7 9 9 10 11'), making the table difficult to read.","section":"Table 3"},{"comment":"The program name 'Mokito' should be 'Mockito'; the same typo appears in Tables 3–5.","section":"§4.3 RQ1"},{"comment":"The summary says 'three data augmentation methods' although four are compared (Aeneas, Lamont, CGAN4FL, and PRAM).","section":"§4.3 RQ4 summary"},{"comment":"Equation (9) describes classifier-based guidance, but the paper later states that classifier-free guidance is used; clarify whether the classifier-based strategy is implemented or is only discussed as an alternative.","section":"§3.3"},{"comment":"The condition 'if StmFusion matches the dimensional requirements of PCD-DAug or DLFL' is vague and should refer to a specific dimension or parameter.","section":"Algorithm 2"},{"comment":"The captions state 'RImp' but the figures plot ratios rather than a clearly defined relative improvement; please define the exact calculation of RImp in the text.","section":"Figures 4–8"},{"comment":"The reference for Mark Weiser's 'Program slicing' lists the year 2009, but the original paper was published in 1984.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The claimed improvements are extremely large, and the lack of any validation that generated rows correspond to real failing executions means the headline numbers should be treated with caution. I would encourage the editor to ask for an explicit description of the binarization step and an execution-based validation of the synthetic tests before further consideration."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the idea—use a conditional diffusion model, conditioned on a fused dynamic-slice plus revised-PCA context, to synthesize minority failing spectra—is new in the FL augmentation niche and worth a look. The replication package is a plus. But the load-bearing claim is unsupported: the paper never says how continuous diffusion outputs become the binary coverage vectors that Dstar, Ochiai, etc. consume, and never executes or oracle-checks a single generated 'failing test case.' The label 'failing' is assigned by construction. Without that, the reported gains (including the 383.83% number in the abstract) could just be the effect of duplicating near-copies of real failing rows—something prior work has already shown can help. So the central experimental claim, as written, is not established.\n\nWhat is genuinely new: the specific fusion of dynamic slicing with revised PCA as a conditioning context, and the move to diffusion models (rather than GAN/CVAE) for FL augmentation. The training only needs the reverse process, which is a reasonable practical advantage. The benchmark setup is broad: 262 Defects4J versions, six FL methods, six augmentation baselines. That is a solid evaluation scaffold.\n\nThe soft spots, in order of seriousness:\n1. The binarization step is missing (Section 3.4, Algorithm 2). If the raw float rows are fed to SFL formulas, the spectra are invalid; if a threshold is used, it's an unstated free parameter that could dominate.\n2. No validation of synthetic failing tests. A coverage row with no concrete input cannot be executed. The paper should either restrict claims to 'synthetic spectra' (and compare against cloning/resampling baselines fairly) or run the generated tests on the faulty program and report that they actually fail.\n3. No significance tests or variance across seeds. Some improvements over the best baseline are tiny (e.g., Top-3 Dstar 74 vs 74; Top-5 98 vs 99), yet the text says 'consistently outperforms.'\n4. The abstract's 383.83% figure is an average of per-method improvements dominated by the DLFL baselines, which start near zero; a reader should compute the actual Top-K counts (Tables 3–4) before believing the headline.\n\nNone of these are fatal to the underlying idea; they are fixes that a serious revision can make. But the paper as submitted does not support 'significantly improves FL effectiveness' for any specific method.\n\nRecommendation: send to peer review. The novelty is real within the subfield, the evaluation scope is right, and the flaw is identifiable and fixable. The reviewers should demand the binarization rule, a validity check on synthetic tests, and honest aggregate reporting.","headline":"A new diffusion-based FL augmentation idea, undermined by an unsupported claim: synthetic 'failing tests' are never binarized or executed, so the headline gains are not yet evidence.","tokens_in":18886,"tokens_out":3209,"would_cite":true,"duration_ms":32229,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that synthetic failing test cases generated by a program-context-aware diffusion model can rebalance imbalanced test suites and improve fault localization, reporting average Top-1 improvements of 383.83% across six…","keywords":["fault localization","data augmentation","class imbalance","diffusion model","program slicing","principal component analysis","spectrum-based fault localization","deep learning fault localization"],"falsifier":"Pick one Defects4J version, rerun PCD-DAug, and attempt to realize each generated coverage row as an actual execution of the faulty program: if most rows do not correspond to any real failing run, or if the 383.83 percent Top-1 improvement disappears when only executed-and-verified failing cases are kept, the central claim is refuted.","tokens_in":17820,"feed_emoji":"🐞","tokens_out":7132,"duration_ms":67283,"temperature":0.7,"pith_summary":"Fault localization relies on execution data from passing and failing tests, but real test suites are heavily skewed toward passing tests, and that skew degrades the rankings of faulty statements. This paper proposes a data-augmentation method, PCD-DAug, that uses a conditional diffusion model to synthesize additional failing test cases so the test suite becomes class-balanced. To keep the synthesized tests meaningful, the method first builds a principal context from dynamic program slicing (which statements influence the faulty output) and a revised PCA (statistical dependencies among statements), then feeds that fused context into a diffusion model trained to generate failing test cases. On 262 fault versions from five benchmark programs, the paper reports that six fault-localization methods improve on average by 383.83% at Top-1, 227.08% at Top-3, and 224.19% at Top-5 after augmentation, and that PCD-DAug outperforms six data-augmentation baselines. The value of the claim, if true, is that localization accuracy can be raised without changing the localization algorithm at all, only by balancing the test data it consumes.","feed_headline":"Synthetic failing tests lift fault localization by up to 384%","feed_subtitle":"A diffusion model trained on program-slice and PCA contexts rebalances test suites and improves six localization methods.","key_machinery":"The central object is the principal context: a fused, dimensionality-reduced coverage matrix that keeps the statements most likely to explain a failure. It is built by dynamic program slicing over multiple failing test cases, which collects statements that influence the faulty output through data and control dependencies; by a revised PCA that ranks original statement columns by summed contributions to the top eigenvectors; and by a fusion algorithm that intersects the two statement index sets and then expands the intersection with PCA-ranked statements that also appear in the slice. The second component is the conditional diffusion model: a forward process adds Gaussian noise according to a fixed schedule, and a trainable reverse process denoises under classifier-free guidance to emit synthetic failing rows, with DPM-Solver replacing the slow denoising chain by a high-order ODE solver so that 25 sampling steps suffice. The key work of these components is to let the generator learn failure-relevant structure without the generator-discriminator instability of GAN-based augmentation and without needing per-dataset retuning of hyperparameters.","core_discovery":"The paper's central claim is that class imbalance between passing and failing test cases is a removable obstacle: a diffusion model conditioned on a program-specific context can fabricate the missing failing cases well enough that spectrum-based and deep-learning fault localizers rank real faults much higher. PCD-DAug constructs the conditioning context in two halves: dynamic program slicing yields a semantic context of statements that drive the faulty output, and a revised PCA extracts a statistical context of the most influential coverage dimensions; an intersection-and-expansion fusion merges the two into a reduced coverage matrix. A simplified U-Net diffusion model with classifier-free guidance learns the distribution of failing and passing rows in that reduced space, and DPM-Solver sampling turns random noise into synthetic failing test cases until the failing class matches the passing class. The paper reports the resulting balanced datasets improve all six evaluated fault-localization approaches on Top-1, Top-3, Top-5, mean first rank, and mean average rank, with the largest relative gains on the deep-learning methods.","pith_inferences":["Beyond the paper, the decisive unverified step is whether the generated rows are genuinely failing test cases: because the diffusion model outputs continuous coverage-like vectors and no thresholding or execution check is reported, the 383.83 percent Top-1 figure is an upper-bound claim until that step is validated.","Beyond the paper, the same principal-context-plus-diffusion recipe could be applied to other imbalanced software-engineering datasets, such as defect prediction or crash triage, though the paper does not evaluate those settings.","Beyond the paper, a direct extension would be to replace continuous-output binarization with a learned threshold, or to generate in a latent space as the paper names for future work, and then test whether the gains survive binarization at different cutoffs."],"forward_implications":["Any of the six tested fault-localization methods can be upgraded simply by feeding in a balanced dataset; the augmentation does not touch the suspiciousness formula or network.","The reported mean-first-rank and mean-average-rank drops imply developers inspect far fewer statements: relative-improvement values for deep-learning methods fall below 16 percent of the original inspection effort for both metrics.","Balanced data from this generator beats undersampling, resampling, and four earlier augmentation approaches on most Top-K and ranking metrics.","A single hyperparameter configuration works across 262 versions and five programs, suggesting the method generalizes without per-dataset tuning."],"supporting_citations":[{"why":"Supplies dynamic program slicing, which PCD-DAug uses to build the fault semantic context.","marker":"[29]"},{"why":"Supplies the revised PCA feature-selection procedure used to build the statistical context.","marker":"[30]"},{"why":"Defines the denoising diffusion probabilistic model whose forward and reverse processes PCD-DAug adapts.","marker":"[32]"},{"why":"Provides classifier-free diffusion guidance, the strategy PCD-DAug uses to steer generation toward failing test cases.","marker":"[31]"},{"why":"Supplies the DPM-Solver sampler that makes the diffusion generation efficient with 25 sampling steps.","marker":"[39]"},{"why":"Describes CGAN4FL, a context-aware GAN augmentation baseline that PCD-DAug must outperform.","marker":"[10]"},{"why":"Describes PRAM, a semantics-aware Mixup augmentation baseline compared in the experiments.","marker":"[11]"},{"why":"Describes Lamont, a lightweight SMOTE-based augmentation baseline compared in the experiments.","marker":"[12]"},{"why":"Describes Aeneas, a universal data augmentation baseline compared in the experiments.","marker":"[13]"},{"why":"Supplies the coverage matrices for the 262 faulty versions that all experiments are run on.","marker":"[42]"}],"fun_headline_variants":["Diffusion-made failing tests boost fault localization up to 384%","Diffusion crafts failing tests to rebalance debug data, boosting FL","Synthetic failing tests via diffusion lift fault localization by 384%","Diffusion model generates failing tests to fix class imbalance in debugging"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that every synthesized row the diffusion model emits is a real failing test case for the faulty program, even though the paper never executes those rows or shows how its continuous outputs are converted into the binary coverage vectors that fault localization consumes.","fun_headline_variants_meta":{"raw":{"variants":["Diffusion-made failing tests boost fault localization up to 384%","Diffusion crafts failing tests to rebalance debug data, boosting FL","Synthetic failing tests via diffusion lift fault localization by 384%","Diffusion model generates failing tests to fix class imbalance in debugging"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000228,"raw_usage":{"total_tokens":1488,"prompt_tokens":968,"completion_tokens":520,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":584,"completion_tokens_details":{"reasoning_tokens":446}},"tokens_in":584,"tokens_out":520,"duration_ms":5399,"temperature":1.0,"reasoning_tokens":446,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T12:35:55.606450+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Pick one Defects4J version, rerun PCD-DAug, and attempt to realize each generated coverage row as an actual execution of the faulty program: if most rows do not correspond to any real failing run, or if the 383.83 percent Top-1 improvement disappears when only executed-and-verified failing cases are kept, the central claim is refuted.","supporting_citations":[{"cited_title":"Dynamic program slicing.ACM SIGPlan Notices, 25(6):246–256, 1990","cited_arxiv_id":null,"evidence_quote":"Supplies dynamic program slicing, which PCD-DAug uses to build the fault semantic context."},{"cited_title":"Feature selection using principal component analysis","cited_arxiv_id":null,"evidence_quote":"Supplies the revised PCA feature-selection procedure used to build the statistical context."},{"cited_title":"Mitigating the effect of class imbalance in fault localization using context-aware generative adversarial network","cited_arxiv_id":null,"evidence_quote":"Describes CGAN4FL, a context-aware GAN augmentation baseline that PCD-DAug must outperform."},{"cited_title":"A deep semantics-aware data augmentation method for fault localization.Information and Software Technology, 168:107409, 2024","cited_arxiv_id":null,"evidence_quote":"Describes PRAM, a semantics-aware Mixup augmentation baseline compared in the experiments."},{"cited_title":"A light-weight data augmentation method for fault localization","cited_arxiv_id":null,"evidence_quote":"Describes Lamont, a lightweight SMOTE-based augmentation baseline compared in the experiments."},{"cited_title":"A universal data augmentation approach for fault localization","cited_arxiv_id":null,"evidence_quote":"Describes Aeneas, a universal data augmentation baseline compared in the experiments."},{"cited_title":"Evaluating & improving fault localization techniques.University of Washington Department of Computer Science and Engineering, Seattle, WA, USA, Tech","cited_arxiv_id":null,"evidence_quote":"Supplies the coverage matrices for the 262 faulty versions that all experiments are run on."}],"review_version":1}