{"id":"2ef0883a-149a-4e25-be69-cbee940bd5a1","arxiv_id":"2506.19871","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"A GAN-based adversarial attack is claimed to achieve 99% success in making fraudulent insurance claims look legitimate to LSTM and XGBoost fraud detectors, but the evidence is undermined by missing dataset and implementation details.","lead":"This paper trains a GAN to generate insurance claims that fool fraud detection models, reporting a 99% attack success rate against LSTM and XGBoost classifiers. It highlights that state-of-the-art fraud detectors can be bypassed by adversarial examples, but the experimental setup is too incomplete to verify the claim.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 99% ASR is computed against the same LSTM/XGBoost models that provide the generator's training reward, so the claimed black-box, no-training-data attack is not established.","rationale":"The paper's strongest claim is the black-box, no-training-data 99% ASR. To support it, the experiment must show that the generator succeeds on a target it was not optimized against. The text in Section 2.2 shows the opposite design: S provides the reward, and Section 3.4 plus Table 2 report attacks on LSTM and XGBoost. No split between surrogate and target is described, so the headline number is uninterpretable as an attack success rate against unknown models. The reader's dimension mismatch is real and would prevent the pipeline from running at the described dimensions, but the evaluation leak is more decisive because even a perfectly implemented pipeline would not establish the abstract claim. No code, data, or reproducible artifacts are provided, so independent verification requires reimplementation. This concern does not change the reader's REJECT verdict; it reinforces it. The critique is directed at the experimental design as reported, not at the authors' intent.","tokens_in":10073,"tokens_out":3660,"duration_ms":42417,"concrete_test":"Re-run the experiment with strict separation: train the GAN using only LSTM as the surrogate (Eqs. 3-4), freeze the generator, and evaluate ASR on an independently trained XGBoost model; then repeat with roles reversed. If the held-out ASR is substantially below 0.99, the reported number is an artifact of training and testing on the same model. As a sanity check, verify that generated samples have 38 features before scoring them.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 2.2 defines the generator update using the surrogate model S: Eq. (3) derives a binary prediction from S(x~), the step-wise reward compares it with y_target, and Eq. (4) minimizes BCE(S(x~), y_target). Table 2 then reports that 'our' attack reduces LSTM and XGBoost accuracy to 0.01. If the LSTM and XGBoost entries in Table 2 are the same models used as S during training, this result mainly confirms that the generator was optimized on those models' outputs; it does not support the abstract's claim that an attacker with no knowledge of training data or internal model details can achieve 99% ASR. The paper never states that the attacked models are held out from GAN training, nor does it evaluate transfer to an unseen target. For LSTM, Section 3.4 additionally says gradient-based optimization is used, which is stronger access than 'model output only.' There is also a dimension inconsistency: the generator's final layer has 12 units (Section 3.2) while the dataset has 38 features (Section 3.1), with no projection or feature-selection step described, making it unclear how generated samples are fed to the classifiers.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a GAN-based adversarial attack against insurance claim fraud detectors. A generator is trained with a surrogate-guided reinforcement-learning reward (Eq. 2–4) to produce samples classified as non-fraudulent; experiments report that the attack reduces LSTM and XGBoost accuracy to 0.01, corresponding to a 99% attack success rate, and the authors argue this demonstrates black-box vulnerability. The paper also provides baseline comparisons with FGSM/BIM/PGD/random noise and SHAP analyses. The central claim—that an attacker with no knowledge of training data or model details can achieve 99% ASR—is not supported by the evidence as presented.","tokens_in":10375,"tokens_out":6085,"duration_ms":60116,"significance":"The topic is timely and the problem of adversarial robustness of insurance fraud detection is worth studying. The GAN-with-reinforcement-learning formulation is a reasonable design choice, and the paper includes a useful descriptive SHAP comparison of LSTM and XGBoost feature attributions. If the central claim were properly supported, this would be a useful contribution. However, the current evidence is insufficient: the 99% ASR is computed against models that appear to be the same ones used to train the generator, the generator's output dimension does not match the input space, and the dataset appears to be auto-insurance rather than medical claims. The paper provides no code or dataset link, so the experiments are not independently checkable.","major_comments":[{"comment":"The central black-box claim is not established. In Section 2.2 the generator's reward is computed from the surrogate model S via Eq. (3) (thresholded prediction) and Eq. (4) (BCE against y_target), and Section 3.4 reports that the proposed attack reduces LSTM and XGBoost accuracy to 0.01 in Table 2. The paper never states that the LSTM and XGBoost models reported in Table 2 are held out from GAN training; if they are the same models used as S, the result only demonstrates that the generator was optimized on those models' outputs. Moreover, Section 3.4 says that gradient-based optimization is used for differentiable surrogates, which is stronger than the claimed 'accessing only the model output' scenario, and no transfer experiment to an unseen target model is reported. Without a held-out target, the abstract's claim of 99% ASR with no knowledge of training data or model details is unsupported.","section":"§2.2, Eq. (3)–(4), §3.4, Table 2"},{"comment":"Section 3.1 describes a dataset with 1,000 samples and 38 features, while Section 3.2 states that the generator's final fully connected layer has 12 units. No projection, padding, feature-selection, or inverse-normalization step is described that would map the generator's 12-dimensional output to the 38-feature input space of the classifiers. As written, the attack pipeline cannot be executed, because the generated samples cannot be fed to the fraud detection models.","section":"§3.1 vs. §3.2"},{"comment":"The paper's title and abstract frame the contribution as a medical insurance claim fraud attack, but the features shown in Figures 3 and 5 (vehicle_claim, property_claim, injury_claim, insured_zip, policy_annual_premium, months_as_customer, capital-gains, capital-loss) are characteristic of an auto insurance claims dataset, not a medical claims dataset. The dataset is never named or cited, so the reader cannot resolve this mismatch. This undermines the relevance of the results to the stated application domain.","section":"§3.1, Figures 3 and 5"},{"comment":"The test split is inconsistent with the reported confusion matrices. Section 3.1 states a 75/5/20 split of 1,000 samples, which gives a 200-sample test set, but every confusion matrix in Figure 4 is labeled '250 Test Samples' and its entries sum to 250. In addition, the XGBoost baseline accuracy in Table 2 is 0.814, whereas Table 1 reports 0.825 for the same model. These numerical inconsistencies prevent the reader from verifying the attack success rate.","section":"§3.1, Figure 4, Tables 1–2"},{"comment":"The attack success rate is not defined in a way that can be reproduced. Eq. (13) defines ASR as '# batches deceiving stability / # batches sent', but 'deceiving stability' is undefined and no batch-level protocol is described. The claimed 99% ASR is inferred from the accuracy 0.01 in Table 2, yet the manuscript does not explain how the accuracy under attack is computed (e.g., whether generated samples replace all test samples, how labels are assigned, or how many samples are used).","section":"§3, Eq. (13), Table 2"}],"minor_comments":[{"comment":"The rendered title contains 'F raud' with a spurious space; please fix the typo.","section":"Title"},{"comment":"The terms 'LightBoost' and 'LightGBM' are used interchangeably; standardize on one name.","section":"§2.3 and §3.2"},{"comment":"The dataset should be named and a URL or citation provided; currently it is described only as 'a publicly available insurance dataset'.","section":"§3.1"},{"comment":"Several panels in Figure 3 have '1e 5' in the y-axis label; the exponent formatting should be corrected.","section":"§3.1, Figure 3"},{"comment":"The latent variable is introduced as z ∈ R^{B×T×F} in Section 2.2, but the generator is later described as fully connected; the relationship between the tensor shape and the 12-unit output is not explained.","section":"§2.2"}],"recommendation":"reject","confidential_remarks":"The manuscript is, in its current form, not suitable for publication. The main experimental pipeline is underspecified and internally inconsistent (generator output dimension vs. feature space, test-set size, dataset identity), and the evaluation does not support the black-box transfer claim. I would not oppose a future resubmission if the authors provide a correctly named dataset, align the generator output with the feature space, and evaluate against held-out target models."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the central claim is not credible as stated. The paper applies a standard GAN-based adversarial attack, trained with a surrogate model's feedback, to a small insurance claims dataset, and reports 99% ASR against LSTM and XGBoost. But the evaluation is circular: the same LSTM and XGBoost that provide the training reward are the ones attacked in Table 2. That's measuring fit, not transfer. The black-box framing is further undermined by Section 3.4's admission that gradient-based optimization is used for LSTM.\n\nWhat's genuinely new is narrow: a GAN generator trained with an RL-style TD update to target a fraud classifier, plus a comparison against FGSM/BIM/PGD on this domain. That's not a new attack paradigm; it's an application of known surrogate-guided black-box attack ideas to insurance data. The SHAP analysis is a nice touch but doesn't rescue the evaluation.\n\nThe soft spots are hard to ignore. The dataset is described as medical insurance but the features (incident_hour, vehicle_claim, insured_zip, property_claim) are from a standard auto-insurance fraud dataset. The generator's final layer outputs 12 units while the data has 38 features; no projection or feature selection is described. The test split says 20% of 1,000 (200 samples) but the confusion matrices report 250 test samples. The ASR definition is nonstandard and unclear. No code or data is released, so none of this can be checked. The paper's contribution claim—'first GAN-based attack against insurance fraud'—is questionable given existing adversarial ML work on tabular data.\n\nThe citation pattern is sloppy (adversarial examples attributed to Chakraborty et al. rather than Szegedy et al.), but that's a minor issue relative to the experimental problems.\n\nWho is this for? Possibly a practitioner wanting a cautionary data point, but the data/model mismatch makes even that unreliable. A serious referee would be able to identify the flaws quickly, but I don't think this deserves referee time in its current form—the load-bearing result is an artifact of the evaluation setup. I'd desk reject with an invitation to resubmit if the authors fix the dimension mismatch, evaluate on a held-out target model, and name the dataset.","headline":"The 99% ASR claim is an artifact of circular evaluation; the dataset is mislabeled, the generator's output dimension doesn't match the features, and no held-out target is tested.","tokens_in":10845,"tokens_out":3573,"would_cite":false,"duration_ms":35131,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a GAN trained on model outputs alone can generate fraudulent insurance claims that evade detection with 99% success.","keywords":["insurance fraud detection","generative adversarial network","adversarial attack","medical insurance claims","LSTM","XGBoost","black-box attack","attack success rate"],"falsifier":"Reproduce the attack on the stated 1,000-sample, 38-feature insurance dataset using the described generator architecture with an output layer of 12 units, feed the generated samples directly into the LSTM and XGBoost fraud classifiers, and measure the attack success rate. If the classifiers reject the 12-dimensional inputs or the accuracy does not drop to about 0.01 once the dimension mismatch is handled, the 99% ASR claim is falsified.","tokens_in":9875,"feed_emoji":"🎭","tokens_out":5840,"duration_ms":50485,"temperature":0.7,"pith_summary":"This paper claims that a generative adversarial network, trained with a surrogate-guided reinforcement-learning loop that only queries a fraud detector's outputs, can produce fraudulent medical insurance claims that the detector labels as legitimate. On a publicly available dataset of 1,000 claims with 38 features, the attack drives LSTM accuracy from 0.750 to 0.01 and XGBoost accuracy from 0.814 to 0.01, corresponding to a 99% attack success rate. The finding matters because it suggests that deployed insurance fraud detection systems are vulnerable to a practical black-box attack that requires no training data and no knowledge of the model's internals. The paper also reports that this GAN attack outperforms gradient-based baselines such as FGSM, BIM, and PGD, which need more access and achieve lower success.","feed_headline":"GAN attack slips 99% of fake claims past fraud detectors","feed_subtitle":"With only model output queries and no training data, the attack crushes LSTM and XGBoost detectors.","key_machinery":"The machinery is a generative adversarial network whose generator maps random noise $z$ through five fully connected layers with 128, 256, 512, 64, and 12 units to produce synthetic samples $\\tilde{x}=G(z)$. Training follows a surrogate-guided reinforcement-learning loop: a surrogate model $S$ classifies each generated sample, the binary prediction $\\hat{y}=1[S(\\tilde{x})>0.5]$ is compared against a target label $y_{\\text{target}}$, and the reward feeds a temporal-difference update $z_{t+1}=z_t+\\alpha\\,\\delta_t\\,\\gamma_t\\,\\mathcal{N}(0,I)$; the generator then minimizes a binary cross-entropy loss $L_G=\\text{BCE}(S(\\tilde{x}), y_{\\text{target}})$ with Adam. The same trained generator is applied to fraud detection models that it did not see during training, which is what enables the claimed black-box transfer.","core_discovery":"The central claim is that a GAN can be trained to generate adversarial insurance claims that are classified as legitimate by fraud detectors, even when the attacker only has access to the model's output. The proposed method trains a generator through a reinforcement-learning scheme where a surrogate classifier supplies rewards; the generator is updated to produce samples that push the surrogate's prediction toward the legitimate class. Experiments on a public insurance dataset show the attack reduces the accuracy of an LSTM model from 0.750 to 0.01 and of an XGBoost model from 0.814 to 0.01, giving an attack success rate of 0.99 for both models. The authors interpret this as evidence that existing fraud detection systems, without adversarial defenses, can be effectively bypassed by an attacker who can query the model and does not know its training data or internal architecture.","pith_inferences":["If the 99% figure holds, any fraud detector with a public query interface becomes exploitable without data or model access, which would push the security burden toward output-level defenses and rate-limiting rather than model secrecy.","The unstated mapping from the generator's 12-dimensional output to the dataset's 38 features is a likely gap; a repaired description would either select 12 features, pad to 38, or add a reconstruction layer, and the reported ASR would need to be re-verified under that mapping.","The black-box claim could be tested more forcefully by training the surrogate on a different model family from the target and measuring transfer; the paper does not report this, but it is a natural next experiment.","The same surrogate-guided GAN recipe should transfer to other tabular fraud domains, such as credit card or property claims, where the feature space is structured and classifiers are served behind an API."],"forward_implications":["Insurance fraud detectors that expose a prediction API without adversarial defenses can be bypassed by an attacker who only queries the model.","The attack transfers across different model families (LSTM and XGBoost), suggesting the vulnerability is architectural rather than model-specific.","The GAN attack outperforms gradient-based methods (FGSM, BIM, PGD) and random noise under the tested threat model, so defenses should be evaluated against this class of attack.","Deployed fraud detection pipelines should incorporate robustness measures such as adversarial training, input anomaly detection, or output-level filtering.","The demonstrated vulnerability implies that the financial losses from fraudulent claims could increase if attackers adopt GAN-based generation."],"supporting_citations":[{"why":"Defines the FGSM gradient-based attack used as a baseline that the proposed GAN attack must outperform.","marker":"[12]"},{"why":"Defines the BIM iterative attack used as a baseline in the comparison.","marker":"[19]"},{"why":"Defines the PGD projected-gradient attack used as a baseline in the comparison.","marker":"[21]"},{"why":"Supplies the Adversarial Robustness Toolbox used to launch the gradient-based baseline attacks against the LSTM detector.","marker":"[22]"}],"fun_headline_variants":["GAN attack slips 99% of fake claims past detectors","Training-free GAN attack fools fraud detectors 99%","Query-only GAN attack evades insurance fraud detection","GAN attack beats LSTM and XGBoost with 99% success","Blind GAN generates fraudulent claims detectors approve"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the generator's output, a 12-dimensional vector, can be treated as a complete 38-feature insurance claim that the fraud detectors will accept as input; if that mapping does not exist, the reported attack pipeline cannot run as written.","fun_headline_variants_meta":{"raw":{"variants":["GAN attack slips 99% of fake claims past detectors","Training-free GAN attack fools fraud detectors 99%","Query-only GAN attack evades insurance fraud detection","GAN attack beats LSTM and XGBoost with 99% success","Blind GAN generates fraudulent claims detectors approve"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000617,"raw_usage":{"total_tokens":2825,"prompt_tokens":870,"completion_tokens":1955,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":486,"completion_tokens_details":{"reasoning_tokens":1873}},"tokens_in":486,"tokens_out":1955,"duration_ms":16952,"temperature":1.0,"reasoning_tokens":1873,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T23:23:23.503353+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Reproduce the attack on the stated 1,000-sample, 38-feature insurance dataset using the described generator architecture with an output layer of 12 units, feed the generated samples directly into the LSTM and XGBoost fraud classifiers, and measure the attack success rate. If the classifiers reject the 12-dimensional inputs or the accuracy does not drop to about 0.01 once the dimension mismatch is handled, the 99% ASR claim is falsified.","supporting_citations":[],"review_version":1}