{"id":"4a5d1d5f-eca7-4257-bc1e-1193a106c1dd","arxiv_id":"1908.05725","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"A compact binary model format is proposed for edge scoring, and a Capsule Network based synthetic-data generator is claimed to improve fraud detection with limited training data.","lead":"This paper describes two additions to a blockchain-based machine learning framework: saving trained models in a compact binary format for scoring on edge devices, and using a Capsule Network to generate synthetic fraud training data. The fraud-detection experiments show only curves, not numbers or code, so the effectiveness claim is unverified.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No experimental support for the central claim: numeric metrics, distributional fidelity, and the unspecified 'parameter ratio' are all absent, so 'very effective' is not established.","rationale":"The reader's verdict is REJECT with moderate confidence, and my stress-test pass agrees. The strongest claim, stated in the abstract and Section 4, is that the proposed CapsNet-based synthetic data generation is 'very effective' for fraud detection. This claim depends on two unverified conditions. First, the generated synthetic fraud samples must be realistic enough to improve a downstream classifier; the paper provides no distributional fidelity check or similarity analysis, despite this being a standard expectation for synthetic data work. Second, the experimental comparison must be quantitative and reproducible; the paper provides only curves, with no numeric metrics, no error bars, no hypothesis tests, and no code or data. The undefined 'parameter ratio' is a particularly concrete gap: it is the central tweak of the generation method, yet the manuscript never says what it is, how it was chosen, or how sensitive the results are to it. I also note the paper's own text says 'during scoring, we change the parameters of the trained model to generate synthetic data', but the mechanism is not specified, and no ablation isolates the effect of this tweak from simple reconstruction. The ASTORE contribution, while more clearly described with pseudocode, is not the paper's central claim and is asserted without formal security analysis; my concern is focused on the experimental claim, which is the load-bearing contribution. Given the absence of numeric evidence and the inability to reproduce the experiment from the manuscript, REJECT is the appropriate verdict. I do not find evidence of fraud or misconduct; the problem is insufficient support for the claim as written. A concrete re-implementation and quantitative comparison, including a distributional distance check and the missing parameter value, would settle whether the concern lands.","tokens_in":12320,"tokens_out":2268,"duration_ms":26270,"concrete_test":"Reproduce Section 4 on the public Kaggle credit-card fraud data set: implement the exact CapsNet (100-neuron fully connected layer, reshape to 10 capsules by 10 neurons, 2 ClassCaps caps by 16 neurons, three fully connected layers, 29 outputs), ask the authors for the 'parameter ratio' value, then generate synthetic fraud samples. Compare the gradient-boosted tree trained on oversampled data alone, with CapsNet-generated fraud added, and with random synthetic fraud added, reporting PR-AUC, F1, and ROC-AUC over cutoffs 0-10% across at least 10 random seeds with standard errors. Also compute a distributional distance (e.g., MMD) between real and synthetic fraud features. If the CapsNet-augmented model is not significantly better than the random-augmented model, or if the synthetic distribution strongly mismatches real fraud, the central claim fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim (abstract and Section 4) is that capsule-network reconstruction, tweaked by 'the same parameter ratio', generates synthetic fraud records that improve a gradient-boosted tree detector. For this to hold, the generated synthetic fraud must lie close to real fraud in the 29-dimensional tabular space. The paper never tests that: there is no MMD or other distributional comparison between real and synthetic fraud, no per-feature statistics, no PCA/t-SNE visualization, and no check that reconstructed samples retain the rare-event structure. The 'parameter ratio' itself is undefined: the text only says 'we also introduced the same parameter ratio to tweak the capsule embedding', without specifying its value, its range, or how it modifies the learned embedding. The experimental evidence is limited to Figs. 3-5, with no numeric PR-AUC, F1, or ROC-AUC values, no error bars, no repeated runs, and no statistical test; the statement that the CapsNet-augmented model 'performed better' is qualitative. The architecture is described at a high level, but key details for replication are missing: how tabular inputs are normalized, how the ClassCaps mask is applied to tabular data, how the three fully connected layers map capsule vectors to 29 outputs, and how many synthetic samples are added. Finally, because the training data are oversampled before training, the baseline already balances rare events; there is no ablation separating the effect of oversampling from the effect of synthetic data. The claimed effectiveness is therefore unsupported, not merely underreported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper makes two contributions within a blockchain-based 'trustable and automated' machine-learning framework. First, it proposes using the ASTORE binary format to serialize machine-learning models on a server layer and then deploy them on resource-limited streaming (edge) devices, arguing that this format is unique, immutable, and resistant to reverse engineering. Second, it proposes a Capsule Network (CapsNet) based synthetic data generation method that is said to enrich scarce training data for fraud detection. The method trains a CapsNet on an oversampled credit-card fraud dataset, then 'tweaks' capsule embeddings during scoring with an unspecified 'same parameter ratio' to generate synthetic fraud samples. These samples are added to the training set of a gradient-boosted tree, and the resulting model is compared against a no-synthetic baseline and a random-synthetic baseline using precision-recall, F1, and ROC curves. The paper reports qualitative improvements but provides no numerical metrics, confidence intervals, or statistical tests.","tokens_in":12574,"tokens_out":4448,"duration_ms":46993,"significance":"If the synthetic data generation method were rigorously validated, it would offer a practical tool for rare-event classification with limited labeled data, which is a genuine and widely relevant problem. The ASTORE contribution, however, is mostly a descriptive account of a proprietary binary format; the paper provides no formal security analysis, no benchmark comparisons with existing formats such as PMML/PFA/ONNX, and no evidence of the claimed uniqueness or reverse-engineering resistance. The most valuable part of the paper, the CapsNet-based synthetic data generation, is currently under-specified and under-validated. The paper includes pseudocode for ASTORE scoring and a clear architecture diagram for the CapsNet, which are helpful for understanding the intended pipeline, but these do not compensate for the lack of empirical support for the central claim.","major_comments":[{"comment":"The 'same parameter ratio' introduced to tweak the capsule embedding is never defined. This parameter is load-bearing: the paper states that during scoring 'we also introduced the same parameter ratio to tweak the capsule embedding in the model so that the generated data sets can present more diversity,' yet no value, range, or operational definition is given. Without this definition, the method is not reproducible, and the claimed improvement cannot be attributed to a well-specified procedure.","section":"Section 4, architecture description"},{"comment":"The experimental evidence consists solely of precision-recall, F1, and ROC curves with no numeric values, no error bars, no repeated runs, and no statistical tests. The statement that 'overall the model that used synthetic data from Capsule Network performed better' is qualitative. Furthermore, the paper does not state how many synthetic samples were added to the training set for Model 2 or Model 3, so it is impossible to determine whether any apparent advantage comes from the quality of the generated samples or simply from the increased size of the minority class.","section":"Section 4, experimental results (Figs. 3-5)"},{"comment":"The paper never checks whether the synthetic fraud data resemble real fraud data in the 29-dimensional feature space. There is no distributional comparison (e.g., MMD, per-feature statistics, PCA/t-SNE visualization) between real and generated samples. The claim that tweaking capsule embeddings produces samples that 'retain much of the learned features' is therefore an assertion without supporting evidence. This is a central gap because the entire hypothesis is that the synthetic samples are useful for training a fraud detector.","section":"Section 4, synthetic data fidelity"},{"comment":"The claims that ASTORE is 'unique and immutable' and that it is 'almost impossible to reverse-engineer' are not backed by formal analysis. The paper states that 'if one bit of data is changed, the store key is changed too,' but this property alone does not establish cryptographic uniqueness or immutability; no hash function, collision-resistance argument, or threat model is provided. Similarly, the assertion that binary formats prevent reverse engineering is a security claim that requires analysis, especially in financial applications where adversarial reverse engineering is a realistic concern.","section":"Section 3, ASTORE security and uniqueness claims"},{"comment":"The baseline model is trained after oversampling the rare events, and Model 3 adds random synthetic rare events. There is no ablation that separates the effect of adding CapsNet-generated samples from the effect of adding more minority-class samples in general (which Model 3 also does). Without such an ablation, the comparison does not demonstrate that the CapsNet generation is superior to simpler oversampling or random synthesis; it may simply be adding more data, an effect that is confounded with sample quality.","section":"Section 4, baseline and ablation"}],"minor_comments":[{"comment":"The manuscript contains several typos and inconsistent spellings: 'ASOTRE' for ASTORE in Section 3, 'retrived' in references [32] and [54], 'Thampson' for Thompson in [56], 'MINST' for MNIST in [32], and 'reciever' in the title of reference [44]. Please correct these.","section":"Throughout"},{"comment":"The text references 'Fig. 1' to visualize the analytical framework, but the figure is not included in the manuscript. Either include the figure or adjust the reference.","section":"Section 2 and Figure 1"},{"comment":"There is an incomplete sentence: 'Some We have seen some efforts such as using the IPFS...' This appears to be a typographical error and should be rephrased.","section":"Section 5"},{"comment":"The connection between the synthetic data generation experiments and the blockchain framework is not made explicit. The experiments are run on a standard Kaggle dataset, not on data retrieved from a blockchain, so the reader is left to infer how this method integrates with the trustable/automation framework proposed in the introduction.","section":"Section 4"},{"comment":"The paper claims that ASTORE is 'unique' but does not specify the universe over which uniqueness is defined. If the store key is a hash of the model contents, uniqueness is a collision-resistance property, but this is not stated. Clarify the intended meaning.","section":"Section 3"}],"recommendation":"reject","confidential_remarks":"The paper reads like a workshop paper that describes a proprietary system (ASTORE) and a preliminary idea for synthetic data generation. The ASTORE contribution is not a scientific contribution in the usual sense: it is a product description with unsubstantiated security claims. The synthetic data method is interesting but the current manuscript does not provide the experimental rigour or methodological detail needed to support its central claim. Even allowing for workshop length, the absence of numeric metrics, undefined parameter, and lack of any distributional fidelity check are serious gaps. I would not be confident that a minor revision could fix these within the scope of the paper; a substantial rewrite with new experiments, a formal security analysis, and a full specification of the method would be needed. For a journal, the threshold for acceptance is not met."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague, this paper pairs two things that don't quite connect: a note on using SAS's ASTORE binary format to ship models to edge devices, and a CapsNet-based synthetic data generator for tabular fraud data. The second is the real contribution, and it's a novel combination of known parts: masked reconstruction from the ClassCaps layer, then tweaking the embedding to create synthetic minority-class samples. That's a sensible idea worth a proper test.\n\nWhat the paper does well: the literature review is honest, the seven-step ML framing is clear, and the authors state the open questions precisely. The Kaggle experiment setup is described well enough to see the intent: oversample, train a CapsNet autoencoder-like model, generate synthetic fraud, add to training set, compare a gradient-boosted tree against baselines on ROC/F1/precision curves.\n\nNow the soft spots, in proportion to how bad they are. The experiments are the load-bearing piece, and they are essentially absent. There are no numeric metrics anywhere—no PR-AUC, F1 at a given cutoff, or ROC-AUC values, no confidence intervals, no repeated runs, no statistical test. We only see three curves that are visually compared. The 'same parameter ratio' used to tweak the capsule embedding is never defined; neither are the architecture details for mapping capsules to 29 tabular outputs, the oversampling ratio, or how many synthetic samples were added. Most importantly, the baseline already gets oversampled training data, so we cannot tell whether the improvement comes from the synthetic data or just from extra minority-class samples. There is no distributional check that the synthetic fraud resembles real fraud, and no ablation.\n\nThe ASTORE half is thin but honestly presented as a proprietary format. The claim that binary formats are 'almost impossible to reverse-engineer' is hand-waving; binary formats can be disassembled. That's a minor overstatement, not a fatal flaw.\n\nThe citation pattern is reasonable; the paper builds on its own prior work [24] and that's fine. The main issue is that the central claim—'very effective'—is not established. This is a solid idea in search of an experiment. A proper revision with numeric results, ablations, and parameter details could make it useful. I'd send it to a serious referee if the authors commit to that, but as is, I would not trust the conclusions. For a reading group, it's a decent case study in how not to report ML results, but not otherwise worth your time.","headline":"A plausible CapsNet synthetic-data idea with no numeric evidence behind its central claim; the ASTORE half is an honest but thin application note.","tokens_in":13127,"tokens_out":2594,"would_cite":false,"duration_ms":25626,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A Capsule Network's masked reconstruction, tweaked during scoring, manufactures synthetic fraud records that improve a gradient-boosted fraud detector on real credit-card transactions.","keywords":["blockchain","trustable machine learning","automation","synthetic data generation","fraud detection","Capsule Network","ASTORE","gradient boosting"],"falsifier":"Compare the capsule-generated fraud samples with held-out real fraud samples in feature space using a two-sample distance or a classifier trained to tell synthetic from real; if the two sets are easily separable, or if a detector trained only on synthetic fraud performs near chance on real fraud, the claimed mechanism fails. Rerunning the experiment while sweeping the undefined same parameter ratio would also show whether the reported gain depends on a fine-tuned value rather than on the method itself.","tokens_in":12068,"feed_emoji":"💳","tokens_out":10262,"duration_ms":100225,"temperature":0.7,"pith_summary":"This paper is trying to establish a complete route from immutable blockchain data to automated fraud scoring, with two load-bearing pieces. First, a model trained on a server layer can be saved as an ASTORE binary blob—a compact, cryptographically keyed object—and loaded onto memory-limited edge devices that score incoming transactions. Second, a Capsule Network trained on tabular transactions can generate synthetic fraud records by masking the label-specific capsule and tweaking its embedding during scoring, and adding those records to the training set improves a gradient-boosted fraud detector compared with both a no-synthetic baseline and random synthetic data. The experiments use a public credit-card transaction data set in which fraud is only 0.172 percent of the 284,807 transactions and report better precision-recall, F1, and ROC behavior at low cutoff values. If the second claim holds, practitioners get a cheap, automated way to train rare-event classifiers when labeled fraud is scarce.","feed_headline":"Fake fraud data from capsule embeddings improve real fraud detection","feed_subtitle":"Adding capsule-generated synthetic fraud to credit-card training data lifts recall and precision at low alert thresholds.","key_machinery":"The machinery that carries the argument is the Capsule Network used as a reconstruction autoencoder. A capsule is a group of neurons whose vector output represents the instantiation parameters of an entity; dynamic routing determines coupling coefficients, and a mask in the ClassCaps layer suppresses all activity vectors except the one for the target label. That masked embedding is decoded back to input space, and at scoring time the embedding is tweaked with the same parameter ratio so the decoded output is not the original record but a synthetic variant of the rare class. These variants are what enrich the downstream training set. The companion mechanism for deployment is ASTORE, a platform-independent binary model format with a cryptographic store key, which packs the trained model, score functions, and variable metadata into a blob that an edge device can load and score online.","core_discovery":"On its own terms, the paper's central discovery is that a Capsule Network's reconstruction path can be turned into a synthetic-data engine for rare-event tabular data. A capsule is a group of neurons whose activity vector encodes instantiation parameters, and dynamic routing sends lower-level outputs to higher-level capsules according to agreement; the ClassCaps mask keeps only the activity vector for the target label, so the decoder learns a label-specific embedding of the input. During scoring, the authors tweak the trained model's capsule embedding by the same parameter ratio to make the reconstruction diverge from the exact input, generating new samples that retain much of the learned features. On the Credit Card Fraud Detection data set, the authors train a gradient-boosted tree on the original oversampled data, on data enriched with capsule-generated fraud samples, and on data enriched with random fraud samples; they report that the capsule-enriched model performs better than the other two in balancing catching frauds and avoiding false alerts across precision-recall, F1, and ROC curves for cutoffs between 0 and 10 percent. The paper also claims ASTORE, a compact binary model format, solves the edge-deployment half of the problem by letting a server-trained model score streaming data directly.","pith_inferences":["If the reported gain is real, the same masked-reconstruction trick should transfer to other rare-event tabular problems, such as network intrusion, loan default, or rare disease, whenever a capsule can learn a class-specific embedding; rerunning the pipeline on a second public data set would test that directly.","The paper leaves the same parameter ratio undefined, so a natural next experiment is to sweep that ratio and check whether the fraud-detection gain is stable or depends on one carefully chosen value; practitioners would need this before trusting the method.","Because an ASTORE blob's store key changes whenever any bit changes, the blob could double as an on-chain model fingerprint that records which model version scored a transaction; the paper describes compatibility with blockchain but does not implement this audit trail."],"forward_implications":["A fraud detector can be improved without collecting new labeled fraud cases: capsule-generated minority-class samples are added to the training set, and the reported experiments show better precision-recall, F1, and ROC behavior than no-synthetic and random-synthetic baselines.","A model trained on a powerful server can be serialized as an ASTORE blob and score incoming transactions in real time on memory-limited streaming or edge devices, without retraining there.","The capsule reconstruction architecture accepts image, text, speech, and tabular inputs, so the same synthetic-data mechanism can in principle enrich training sets beyond financial fraud.","Synthetic labels from the generator are exact by construction, which removes the labeling bottleneck that makes rare-event training data scarce.","For blockchain-stored financial transactions, the two contributions connect into a pipeline: immutable data feed a server-trained model, the model travels as an ASTORE blob to the streaming layer, and capsule-generated synthetic fraud addresses the small-label problem."],"supporting_citations":[{"why":"Defines the unified framework with server, streaming, and smart-contract layers that motivates both open problems addressed here.","marker":"[24]"},{"why":"Introduces the ASTORE binary model format used to move trained models from the server layer to edge scoring.","marker":"[21]"},{"why":"Supplies the Capsule Network, dynamic-routing equations, and masked reconstruction that the synthetic-data method is built on.","marker":"[30]"},{"why":"Supplies the public credit-card transaction data set used in all experiments.","marker":"[38]"},{"why":"Supplies the gradient-boosted tree used as the fraud detector and baseline.","marker":"[40]"},{"why":"Supplies the Adam optimizer used to train the capsule network.","marker":"[41]"}],"fun_headline_variants":["Capsule-packed fake fraud data sharpen real fraud detection","Fake fraud from capsule embeddings beats random oversampling","Capsule decoder synthesizes rare fraud data to lift precision","Compact binary blobs put server-trained ML on edge devices","Capsule synthetic data plus compact blobs stretch edge ML"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that slightly changing the internal feature descriptions learned from real fraud transactions produces new fake fraud records that look enough like real fraud to help a detector; the paper does not verify that resemblance directly.","fun_headline_variants_meta":{"raw":{"variants":["Capsule-packed fake fraud data sharpen real fraud detection","Fake fraud from capsule embeddings beats random oversampling","Capsule decoder synthesizes rare fraud data to lift precision","Compact binary blobs put server-trained ML on edge devices","Capsule synthetic data plus compact blobs stretch edge ML"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001039,"raw_usage":{"total_tokens":4442,"prompt_tokens":1086,"completion_tokens":3356,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":702,"completion_tokens_details":{"reasoning_tokens":3288}},"tokens_in":702,"tokens_out":3356,"duration_ms":27207,"temperature":1.0,"reasoning_tokens":3288,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T13:28:56.206248+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compare the capsule-generated fraud samples with held-out real fraud samples in feature space using a two-sample distance or a classifier trained to tell synthetic from real; if the two sets are easily separable, or if a detector trained only on synthetic fraud performs near chance on real fraud, the claimed mechanism fails. Rerunning the experiment while sweeping the undefined same parameter ratio would also show whether the reported gain depends on a fine-tuned value rather than on the method itself.","supporting_citations":[{"cited_title":"A unified analytical framework for trustable machine learning and automation running with blockchain","cited_arxiv_id":null,"evidence_quote":"Defines the unified framework with server, streaming, and smart-contract layers that motivates both open problems addressed here."},{"cited_title":"An overview of SAS Visual Data Mining and Machine Learning on SAS Viya,","cited_arxiv_id":null,"evidence_quote":"Introduces the ASTORE binary model format used to move trained models from the server layer to edge scoring."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the public credit-card transaction data set used in all experiments."},{"cited_title":"XGBoost: A Scalable Tree Boosting System","cited_arxiv_id":null,"evidence_quote":"Supplies the gradient-boosted tree used as the fraud detector and baseline."},{"cited_title":"ADAM: a method for stochastic optimization","cited_arxiv_id":null,"evidence_quote":"Supplies the Adam optimizer used to train the capsule network."}],"review_version":1}