{"id":"3d76c230-1fbb-4968-93ab-19bb4d13ed2b","arxiv_id":"2608.11422","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"COGENT proposes counterfactual explanations in 3D Gaussian parameter space for Sybil, but its localization claim is undermined by using the tumour mask to pre-select the optimized Gaussians.","lead":"A new method called COGENT explains a lung cancer risk AI by editing the 3D Gaussian representation of a CT scan until the risk score changes. Its reported tumour localization is largely predetermined because the known tumour mask is used to choose which Gaussians are allowed to move.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Localization claim is confounded: mask filtering restricts counterfactual optimization to tumour-mask Gaussians, so RRA against the same mask is favorable by construction and does not establish that COGENT finds Sybil's evidence.","rationale":"The reader's weakest assumption identifies exactly the load-bearing flaw: mask-based preselection of Gaussians predetermines the localization score measured against the same mask. The manuscript text confirms this in Figure 2 and in the Method section, and Table 1 is then used as the primary quantitative support for the central claim. Because the optimization domain is restricted to the tumour, the experiment cannot distinguish 'COGENT finds the lesion' from 'COGENT can only move the lesion.' The comparison against unrestricted pixel-space baselines is therefore unfair, and the expert evaluation, although suggestive, is too small and too coarsely reported to carry the claim on its own. The framework may be salvageable with an unrestricted evaluation or with a control using a misregistered mask, but as written the central claim is not established. I find no additional independent objection beyond this one; the concern is concrete, testable, and verdict-changing, so REJECT is the appropriate current verdict.","tokens_in":9764,"tokens_out":3893,"duration_ms":38777,"concrete_test":"Rerun the COGENT pipeline on the same CT volumes with the mask-filtering stage disabled: optimize a random matched-size subset of all Gaussians (or all Gaussians), select the top 10% most-changed Gaussians, and recompute RRA/RRA_abs against the same ground-truth tumour mask. If the scores fall to the range of the pixel-space baselines (roughly 0.02–0.08), the reported 0.2837 is an artifact of preselecting in-mask Gaussians. As a control, repeat the original mask-filtered pipeline with the mask shifted to a non-lesion region; if RRA remains high, the metric is not measuring Sybil's evidence but merely the preselected optimization region.","verdict_should_be":"REJECT","load_bearing_attack":"The central claim—that Gaussian-space counterfactual optimization localizes Sybil's prediction evidence to the ground-truth tumour—rests on Table 1, where COGENT reports RRA/RRA_abs = 0.2837. But the evaluation is circular. Figure 2 states: 'mask filtering keeps only the Gaussians inside the tumour. Their parameters are optimized with PGD using gradients from the frozen Sybil classifier.' The Method section similarly says the optimization is performed on 'the Gaussians intersecting regions defined by anatomical masks.' The top-changed Gaussians used for localization are therefore, by construction, a subset of the ground-truth tumour mask. Overlapping that subset with the same mask does not test where Sybil's evidence is; it tests only whether PGD moved the preselected in-mask Gaussians. The pixel-space baselines (GradCAM, Saliency, IG, Input×Gradient, Kernel SHAP) are not given this mask pre-filtering, so the comparison is not apples-to-apples and the reported superiority is not evidence for the claimed behavior. In fact, the experiment cannot falsify the hypothesis that Sybil relies on features outside the visible lesion, because those Gaussians are never eligible to be selected. The statement that COGENT 'effectively localize[s] radiologically relevant disease features' is therefore unsupported by the current evaluation.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces COGENT, a counterfactual explanation method for volumetric medical images that operates directly in the parameter space of 3D Gaussian splatting representations (MedGS). COGENT first selects Gaussian primitives via mask filtering, then optimizes their parameters with projected gradient descent to shift the logits of the Sybil lung cancer risk prediction model, and finally uses the most-changed Gaussians as the explanation. The authors evaluate localization by comparing the resulting attribution map to ground-truth tumour masks (reporting RRA/RRA_abs) and supplement this with expert ratings. The central claim is that Gaussian-space counterfactual explanations localize the evidence driving Sybil's prediction to the tumour region.","tokens_in":10032,"tokens_out":5349,"duration_ms":45967,"significance":"If the localization claim were valid, COGENT would offer a novel and potentially useful representation-space explainability paradigm for volumetric medical imaging, with publicly released code and a fresh integration of MedGS with a clinical risk model. However, the evaluation design contains a fundamental circularity: the method's candidate Gaussians are preselected using the ground-truth tumour mask, and the localization score is computed against that same mask. This makes the central claim unfalsifiable in its current form. The expert evaluation, which shows only 40% positive ratings, does not rescue the claim. The underlying idea is interesting, but the manuscript's evidence is not.","major_comments":[{"comment":"The optimization is restricted to Gaussians preselected via mask filtering; Figure 2's caption states that 'mask filtering keeps only the Gaussians inside the tumour,' and the Method section describes identifying 'the Gaussians intersecting regions defined by anatomical masks.' Table 1 then scores the resulting explanation against the same ground-truth lesion mask (RRA/RRA_abs). Because the candidate Gaussians are confined to the tumour mask, the reported RRA = 0.2837 is largely determined by the preselection step rather than by whether Sybil's evidence actually lies in the lesion. The pixel-space baselines are not given this mask-based preselection, so the comparison in Table 1 is not apples-to-apples. This circularity invalidates the central claim that COGENT localizes Sybil's evidence; the experiment cannot falsify the possibility that Sybil relies on features outside the visible lesion, because outside-mask Gaussians are never eligible for selection.","section":"Figure 2 and Method description"},{"comment":"The counterfactual objective is computed on V(Theta), the volume rendered from MedGS slices, but the explanation is claimed to explain Sybil's prediction on the original CT volume. The paper does not report whether Sybil's raw hazard logits on V(Theta) are close to those on the original CT volume. Without such a fidelity check, the optimized gradients may be explaining the reconstructed volume rather than the original scan, and the counterfactual may succeed only because of rendering artifacts. The authors should report a concordance measure (e.g., logit correlation or prediction agreement) between original and reconstructed volumes for the evaluation dataset, and restrict or qualify the claims accordingly.","section":"Preliminary studies / MedGS and Method description"},{"comment":"The paper reports that only 40% of the evaluated cases received positive expert ratings (+1/+2), with 40% neutral and 20% negative. The statement that 'these findings demonstrate that Gaussian-space counterfactual explanations effectively localize radiologically relevant disease features' is not supported by these numbers; a 40% positive rate is inconclusive, and the conclusion is overstated. The expert evaluation should be presented as a preliminary qualitative observation, not as confirmatory evidence for the localization claim.","section":"Expert Qualitative Evaluation and Results"}],"minor_comments":[{"comment":"No error bars, confidence intervals, or statistical significance tests are reported, and the number of scans in the evaluation is not stated.","section":"Table 1"},{"comment":"RRA and RRA_abs are not formally defined in the main text; a precise definition with the exact formula is needed to interpret the reported values.","section":"Experiments"},{"comment":"The pixel-space baseline is not identified; the text only refers to 'a pixel-space counterfactual baseline' without specifying which method produced it.","section":"Figure 3"},{"comment":"The hyperparameters used for the main experiments (number of PGD steps, learning rate, epsilon, number of evaluated scans) are not given in the main text or the appendix; only a limited epsilon sensitivity analysis appears in the appendix.","section":"Method description"},{"comment":"The manuscript contains numerous typographical and formatting issues (e.g., missing spaces in the abstract, inconsistent capitalization of 'Gaussian Splatting'), which should be corrected in revision.","section":"General"}],"recommendation":"reject","confidential_remarks":"The circular evaluation is a load-bearing flaw that cannot be fixed with local edits. The central claim of the paper is unsupported by the current experimental design, and the expert evaluation is weak. The authors would need to substantially redesign the experiments (e.g., run COGENT without mask-based preselection or apply the same preselection to all baselines) and validate the reconstruction fidelity before the claim could be taken seriously. I therefore recommend rejection in its current form."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First, the useful thing to know: the core idea—explaining a volumetric risk model by counterfactual optimization in the parameter space of a 3D Gaussian scene representation—is genuinely new as far as I can tell, and the paper is clearly written. The problem is that the central quantitative claim is invalidated by the evaluation design. Figure 2's caption says that mask filtering keeps only the Gaussians inside the tumour, and the optimization is restricted to those. Table 1 then scores COGENT's attribution map against that same ground-truth tumour mask. Of course the overlap is high: the only Gaussians eligible to be 'most changed' are already inside the lesion. The comparison baselines (GradCAM, Saliency, etc.) do not get this pre-filtering, so the 0.2837 RRA is not apples-to-apples and does not establish that COGENT localizes Sybil's evidence. The experiment cannot falsify the hypothesis that Sybil relies on features outside the visible lesion.\n\nCredit where due: the idea of moving explainability into Gaussian parameter space is worth exploring, the MedGS-to-Sybil differentiable bridge is a nice engineering contribution, and code is public. The qualitative examples look cleaner than the pixel-space baseline, and the authors are appropriately modest about the mixed expert ratings (40% positive, 40% neutral, 20% negative).\n\nThe other soft spots are minor by comparison. There is no check that Sybil's prediction on the rendered volume matches its prediction on the original CT; if rendering introduces a domain gap, the whole pipeline is explaining a different model. The top-k threshold for 'most changed Gaussians' is fixed and unablated. And the expert evaluation is a single reader with no error bars. But these are all fixable. The mask-filtering issue is the load-bearing one.\n\nWho this is for: anyone building or reviewing XAI methods for volumetric medical AI would benefit from reading it as a cautionary example of an evaluation that predetermines the answer. The framework may be salvageable—an unrestricted optimization, or a causal sufficiency test that perturbs only inside the lesion and measures risk change, would be a more honest design.\n\nMy recommendation: send it to peer review, but expect a major revision. The idea deserves referee attention; the current evidence does not support the claim.","headline":"Clever idea, but the central localization experiment is circular: the counterfactual is only allowed to move Gaussians inside the tumour mask and is then scored against that same mask.","tokens_in":10640,"tokens_out":2659,"would_cite":false,"duration_ms":23466,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Counterfactual edits in Gaussian space find Sybil's tumour evidence","keywords":["counterfactual explanations","3D Gaussian splatting","volumetric medical imaging","lung cancer risk prediction","Sybil","MedGS","explainable AI","computed tomography"],"falsifier":"Run COGENT with the mask-filtering step disabled, allowing all Gaussians to be optimized, and compute RRA against the ground-truth mask; if the top-changed Gaussians fall largely outside the lesion while risk still flips, the localization claim is refuted. Separately, feed Sybil both the original CT volume and the MedGS-rendered volume of the same scan and compare risk predictions; material divergence would mean the counterfactual is explaining the rendering, not the scan.","tokens_in":9589,"feed_emoji":"🫁","tokens_out":5825,"duration_ms":62750,"temperature":0.7,"pith_summary":"COGENT asks a different question from standard saliency maps: instead of highlighting voxels, it treats the CT scan as a 3D scene built from Gaussian primitives and searches for the smallest parameter changes to those primitives that flip Sybil's predicted lung-cancer risk. The paper claims that the Gaussians that change most mark the evidence driving the model, and that this localization lands on the ground-truth tumour: COGENT scores 0.2837 on Relative Random Area, far above pixel-space baselines like GradCAM (0.0805) and Saliency (0.0702). If the claim holds, representation-space counterfactual optimization gives clinicians a way to see which part of a scan the model relies on, in a form that stays anatomically coherent.","feed_headline":"Counterfactual edits in Gaussian space find Sybil's tumour evidence","feed_subtitle":"Instead of voxel heatmaps, COGENT rewrites 3D scene parameters and checks which changes flip the risk prediction.","key_machinery":"The load-bearing object is the Gaussian primitive set Θ = {θ_i} with θ_i = (position, color, opacity, scale, rotation), rendered by MedGS into 2D slices that are assembled into V(Θ), the volume fed to Sybil. Because the entire chain Θ → R(Θ, c_j) → V(Θ) → f_m(V(Θ)) → L_cf is differentiable, gradients from Sybil's raw logits flow back to individual Gaussians. COGENT selects a subset Ω of Gaussians via anatomical-mask filtering, applies random-start projected gradient descent with an L∞ budget ε around the initial parameters, and ranks Gaussians by the magnitude of their parameter changes; the top 10% are highlighted as the explanation. Raw logits instead of calibrated probabilities are used to avoid gradient saturation.","core_discovery":"The paper's central claim is that counterfactual optimization performed directly in the parameter space of a 3D Gaussian scene representation—not in voxel space—identifies the radiological features that drive Sybil's risk prediction. The pipeline reconstructs each LDCT volume with MedGS, renders slices that Sybil scores, and runs projected gradient descent on the Gaussian parameters (position, opacity, scale, color) under an L∞ bound, using raw risk logits as the objective so gradients stay strong. The most-changed Gaussians then form the explanation. COGENT reports the best localization among compared methods (RRA and RRA_abs = 0.2837, versus 0.0805 for GradCAM, 0.0702 for Saliency, 0.0254 for Integrated Gradients, 0.0218 for Input×Gradient, 0.0134 for Kernel SHAP) while staying competitive on sparsity and perturbation metrics, and expert review rated 40% of counterfactual edits as attenuating malignancy markers, 40% neutral, and 20% as exacerbating them.","pith_inferences":["The reported localization advantage is partly by construction: Figure 2 shows that mask filtering restricts optimization to Gaussians inside the tumour, so RRA measures how well COGENT uses an already-tumour-confined set rather than whether the model's evidence is the lesion. Running COGENT without the mask filter would test the stronger claim.","The method implicitly assumes that Sybil's risk judgment on MedGS-rendered slices equals its judgment on the original CT; if the rendering pipeline distorts low-dose CT appearance, the counterfactual may explain the rendering rather than the scan.","A natural testable extension is to use the same Gaussian-space optimization to locate evidence for other volumetric biomarkers (e.g., emphysema or fibrosis) where the relevant mask is not predefined, letting the optimizer discover regions of interest without annotation."],"forward_implications":["If COGENT is correct, explainability for volumetric medical AI no longer needs to operate on pixels or voxels; any differentiable scene representation can host counterfactual explanations.","Clinicians could audit Sybil-type risk models by asking which localized 3D region, when altered, moves the prediction, rather than by inspecting 2D heatmaps.","Because the counterfactual volume stays close to the original under an L∞ bound, the explanation corresponds to a minimal, anatomically plausible edit, which is closer to what a radiologist would consider a cause.","The framework is presented as a general paradigm: it should transfer to other volumetric predictors as long as the rendering pipeline is differentiable."],"supporting_citations":[{"why":"Supplies MedGS, the differentiable Gaussian rendering pipeline that turns CT volumes into optimizable primitives.","marker":"(Marzol et al. 2025)"},{"why":"Provides Sybil, the frozen lung cancer risk predictor whose logits serve as the optimization objective.","marker":"(Mikhael et al. 2023)"},{"why":"Introduces 3D Gaussian Splatting, the explicit scene representation COGENT operates in.","marker":"(Kerbl et al. 2023)"},{"why":"Supplies the projected gradient descent method and L∞-constrained perturbation framework used for counterfactual optimization.","marker":"(Madry et al. 2018)"},{"why":"Defines counterfactual explanations as minimal input changes that alter a model's decision, the framing COGENT adopts.","marker":"(Wachter, Mittelstadt, and Russell 2017)"},{"why":"Provides Grad-CAM, one of the principal pixel-space baselines COGENT compares against in localization.","marker":"(Selvaraju et al. 2017)"}],"fun_headline_variants":["Gaussian-space counterfactuals pinpoint Sybil's decision drivers","Editing 3D Gaussians reveals tumor evidence to Sybil","Counterfactual scene edits explain lung cancer risk","Beyond voxels: Gaussian counterfactuals for CT explainability","Sparse Gaussian edits localize Sybil's risk factors"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The localization score is measured against a tumour mask that was already used to decide which Gaussians COGENT may modify, so high RRA may be a consequence of that pre-selection rather than proof that Sybil's evidence is the visible lesion.","fun_headline_variants_meta":{"raw":{"variants":["Gaussian-space counterfactuals pinpoint Sybil's decision drivers","Editing 3D Gaussians reveals tumor evidence to Sybil","Counterfactual scene edits explain lung cancer risk","Beyond voxels: Gaussian counterfactuals for CT explainability","Sparse Gaussian edits localize Sybil's risk factors"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000464,"raw_usage":{"total_tokens":2328,"prompt_tokens":966,"completion_tokens":1362,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":582,"completion_tokens_details":{"reasoning_tokens":1280}},"tokens_in":582,"tokens_out":1362,"duration_ms":26148,"temperature":1.0,"reasoning_tokens":1280,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T14:12:08.897874+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run COGENT with the mask-filtering step disabled, allowing all Gaussians to be optimized, and compute RRA against the ground-truth mask; if the top-changed Gaussians fall largely outside the lesion while risk still flips, the localization claim is refuted. Separately, feed Sybil both the original CT volume and the MedGS-rendered volume of the same scan and compare risk predictions; material divergence would mean the counterfactual is explaining the rendering, not the scan.","supporting_citations":[],"review_version":1}