{"id":"e9a86466-5d76-4d02-8d85-92c646b91b50","arxiv_id":"2412.17069","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":1,"one_line_summary":"SALN selects within-batch training samples using the Fiedler vector of a spectral similarity graph, reporting large speedups and accuracy gains that are only partly supported by the tabulated results.","lead":"This preprint proposes SALN, a data-selection heuristic that scores images inside each training batch by the Fiedler vector of a cosine-similarity Laplacian and trains only on the highest-scoring samples. The paper claims up to 8x faster training and 5% higher accuracy than standard training, and better results than DeepMind's JEST, but its own tables contradict parts of that claim.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"SALN's reported gains are not attributable to spectral selection because every comparison confounds it with a 20% per-batch sample reduction and no random-subset control is run; the abstract's 8x speedup is also not supported by the paper's own timing tables.","rationale":"The paper proposes a clear, simple heuristic, and Algorithm 1 is easy to follow. However, the load-bearing premise is that the Fiedler-vector ranking identifies more informative samples than random selection. The experiments do not test this premise: the only control is standard training, which passes over the full dataset, whereas SALN processes only 80% of each batch. Thus the reported speedup and accuracy differences are confounded with the reduced number of samples per epoch. This is not an objection to the heuristic being outside current consensus; it is a correctness risk because the paper's own data cannot distinguish the proposed mechanism from a trivial random-subset baseline. The internal mismatch between the abstract's '8x reduction' and the reported timing ratios (roughly 3.9x and 1.7x) reinforces that the headline claim is not carefully supported. The JEST comparison is also inconsistent: on the Pet dataset, JEST has higher test accuracy than SALN, so the claim 'SALN achieves better performance ... compared to JEST' fails on the paper's own Table 7. These issues all touch the central empirical claim, so the reader's REJECT verdict is appropriate. My read does not change that verdict; it sharpens the reason by emphasizing the missing random-subset control as the decisive experiment that would settle attribution.","tokens_in":8578,"tokens_out":3560,"duration_ms":34648,"concrete_test":"Run the exact SALN training loop for 25 epochs on Pet and CIFAR-10 with filter_ratio=0.8 under three conditions: (a) SALN spectral selection, (b) uniform random selection of the same number of samples per batch, and (c) full-batch standard training, with at least 5 seeds each. Report mean +/- std test accuracy and wall-clock time including per-batch eigendecomposition overhead. If (a) and (b) agree within seed noise, the spectral selection mechanism is not load-bearing and the central claim must be revised.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim requires that the spectral ranking in Algorithm 1 lines 27-30 (top absolute Fiedler-vector entries) is what makes SALN faster and more accurate. But with filter_ratio=0.8, SALN trains on only 80% of each batch (n_draws = int(n_images * (1 - filter_ratio))), so every SALN run differs from standard training in two ways: the selection rule and the number of samples processed per epoch. The paper never compares against random selection at the same filter_ratio, so a simpler mechanism—fewer forward/backward passes, altered batch statistics, or implicit regularization—can explain the speed and the Pet accuracy gap. Additionally, the abstract's '8x reduction' is not supported by Tables 4 and 12: 24.48/6.31 ≈ 3.9x on Pet and 39.41/22.66 ≈ 1.7x on CIFAR-10. Table 7 also shows JEST (87.55%) beating SALN (86.8%) on Pet test accuracy, directly contradicting the claim that SALN achieves better performance than JEST. Because the key independent variable—the selection rule itself—is never isolated from the reduced sample count, the empirical central claim is unverified even if Algorithm 1 is implemented correctly.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"SALN is a within-batch data curation method that scores samples by the absolute values of the Fiedler vector of the cosine-similarity Laplacian of each batch, then trains only on the top-scoring samples. The paper compares SALN with standard training and with Google DeepMind's JEST on Oxford-IIIT Pet and CIFAR-10 using a fixed pretrained ResNet-18 and reports large training-time reductions and accuracy gains. The abstract claims up to 8x training-time reduction, up to 5% accuracy improvement, and better performance than JEST. Those headline claims are not supported by the paper's own tables: the largest measured speedup is roughly 3.9x, CIFAR-10 test accuracy is slightly lower for SALN than for standard training, and on Pet JEST achieves higher test accuracy than SALN. The core experimental design also confounds the spectral selection rule with a large per-batch sample reduction, because no random-subset baseline is run.","tokens_in":8803,"tokens_out":4604,"duration_ms":38973,"significance":"If the spectral ranking mechanism were responsible for the reported gains, SALN would be a simple and potentially useful addition to the data-curation toolbox: the scoring rule is fixed and external to the target metric, the method has a single hyperparameter (filter_ratio), and the code and notebooks are publicly released. The pretrained feature extractor being fixed before training also avoids an obvious circularity. However, the presented evidence does not isolate the spectral mechanism from the confounding effect of subsampling, and several stated claims are contradicted by the paper's own results tables, so the significance of the contribution cannot be assessed as it stands.","major_comments":[{"comment":"The abstract's quantitative claims are not supported by the paper's tables. The largest training-time reduction in Tables 4 and 12 is 24.48/6.31 ≈ 3.9x on Pet and 39.41/22.66 ≈ 1.7x on CIFAR-10, not the claimed 'up to an 8x reduction.' The claim of 'up to a 5% increase in accuracy' is contradicted by Table 11, where SALN test accuracy on CIFAR-10 is 82.46% versus 82.56% for standard training. The claim that SALN achieves better performance than JEST is contradicted by Table 7 on Pet, where JEST test accuracy is 87.55% versus SALN's 86.8%.","section":"Abstract, Section 4.1.1, Section 4.2.1"},{"comment":"With the default filter_ratio=0.8, line 12 sets n_draws = int(n_images * (1 - filter_ratio)) = 0.2 * n_images, and line 30 selects exactly the top n_draws absolute Fiedler-vector entries. SALN therefore trains on only 20% of each batch, contrary to the docstring's phrasing that filter_ratio 'specify how many data should be selected from this batch.' If the intended keep ratio was 80%, the code implements the opposite; if the intended keep ratio was 20%, the hyperparameter name and Section 3.4 description are misleading. In either case, every comparison to standard training or JEST changes both the selection rule and the number of samples processed per batch.","section":"Algorithm 1, lines 12 and 30"},{"comment":"No random-subset baseline is reported. The experiments vary the selection criterion and the number of training samples per batch simultaneously, so the observed speedups and the Pet accuracy gain could be explained by fewer forward/backward passes, altered batch statistics, or implicit regularization rather than by the spectral ranking. A control that trains on a randomly chosen subset of size n_draws from each batch under otherwise identical conditions is required to attribute the results to SALN's selection heuristic.","section":"Section 4 (all experiments)"},{"comment":"The central heuristic—that samples with the largest absolute entries in the Fiedler vector of the within-batch cosine-similarity Laplacian are the most informative—is asserted without derivation, a toy example, or an ablation against alternative selection rules (e.g., random, loss-based, coreset, or degree-based selection). No evidence is provided that this particular spectral scoring rule, rather than the act of subsampling itself, carries the reported effects.","section":"Section 3.5 and Algorithm 1"},{"comment":"All accuracy, loss, and training-time values appear to come from single runs with no error bars, no seeds, and no significance tests. Given the small differences involved (e.g., 82.46% versus 82.56% on CIFAR-10 test accuracy), the results do not currently support the strength of the conclusions drawn in Section 5.","section":"Section 4, experimental methodology"}],"minor_comments":[{"comment":"The sentence 'represented in Table 16' refers to the training-time comparison that is actually labeled Table 8; this cross-reference should be corrected.","section":"Section 4.1.2"},{"comment":"Figure 10 is captioned as a 'Weights Heatmap' but the surrounding text describes a weight distribution histogram; the caption and the figure content should be reconciled.","section":"Section 4.4 and Figure 10"},{"comment":"There are numerous typographical errors, including 'CIF AR-10', 'V anishing', 'T ransfer', and 'Accessability'; the manuscript would benefit from a careful proofreading pass.","section":"Section 3.1 and throughout"},{"comment":"Reference [23] is a blog post ('Processing Paradigms: Stream vs Batch in the ML Era', Airbyte, 2023); for a journal submission it should be replaced or supplemented with a scholarly source on batch processing in machine learning.","section":"References"},{"comment":"The paper states that features are extracted from the whole dataset with a pretrained ResNet-50, but does not state whether this extraction is performed once before training or repeatedly; specifying this would clarify the method's computational cost.","section":"Section 3.5, step 1"}],"recommendation":"reject","confidential_remarks":"The manuscript reads like a workshop-style extended abstract with an overclaimed abstract. The missing random-subset control and the contradiction between Table 7 and the abstract's JEST comparison are substantive, not merely presentational. I would look favorably on a revised version that adds controlled experiments, corrects the filter_ratio ambiguity, and tempers the claims to match the measured results."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You asked what I think of the SALN paper. The short version: the spectral batch-selection idea is simple and worth a coffee conversation, but the empirical case as written does not hold up, and the abstract overstates what the tables show.\n\nWhat's genuinely new: using the Fiedler vector of a within-batch cosine-similarity Laplacian to rank and drop samples is a clean, easy-to-implement heuristic. I don't see it in the cited core-set, BatchBALD, or JEST lines of work. The algorithm is specified clearly enough that someone could reproduce the mechanics from the pseudocode alone, and the GitHub link is a plus even if no commit-verified code is in the paper.\n\nWhere it falls apart: the claims and the data disagree. The abstract promises up to 8x training-time reduction; the paper's own tables cap out at about 3.9x on Pet and 1.7x on CIFAR-10. On CIFAR-10 the SALN test accuracy is slightly worse than standard training (82.46 vs 82.56), and on Pet the JEST test accuracy is higher than SALN (87.55 vs 86.8). Those are not just quibbles; they are direct contradictions of the central claims. The stress-test note is right: every SALN run trains on only 80% of each batch, and there is no random-subset baseline at the same filter ratio. So the observed speedup and any accuracy gain could come from simply seeing fewer examples per epoch, or from implicit regularization, rather than from spectral informativeness. That missing control is load-bearing. Also, no error bars or multiple seeds, the JEST implementation is underspecified (filter ratio not reported for the comparisons), and the filter_ratio hyperparameter is never ablated.\n\nIs the paper hopeless? No. The heuristic is plausible and the write-up is honest about its heuristic nature. But the current experiments do not isolate the selection rule, and the abstract needs a rewrite to match the reported numbers. If the author adds a random-subset control, runs multiple seeds, reports the JEST configuration, and recalibrates the claims, the idea might become publishable at a workshop or a short paper. As it stands, I would not send this to a serious peer review, because the core empirical claim is unverified and the internal contradictions would waste referee time. My recommendation: desk reject with an invitation to resubmit after proper controls and corrected claims.","headline":"Clean algorithmic idea, but the paper's own tables contradict its headline speed/accuracy claims and the missing random-subset baseline means the core empirical result is unverified.","tokens_in":9363,"tokens_out":1671,"would_cite":false,"duration_ms":16770,"reading_group":"no","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper proposes SALN, a within-batch spectral selection method that keeps the samples with the largest Fiedler-vector entries of a cosine-similarity Laplacian, and reports up to an 8x training-time reduction and up to a 5% accuracy…","keywords":["spectral analysis","data curation","joint batch selection","Fiedler vector","Laplacian matrix","training time reduction","batch prioritization","ResNet-18"],"falsifier":"Train the same pre-trained ResNet-18 on the primary cat-and-dog dataset for the same 25 epochs and filter ratio, but choose the kept samples uniformly at random within each batch; if the random-subset run matches SALN's accuracy and training time, the Fiedler-vector ranking carries none of the reported benefit.","tokens_in":8315,"feed_emoji":"⚡","tokens_out":9978,"duration_ms":79221,"temperature":0.7,"pith_summary":"This paper proposes SALN, a training-time data-curation method that scores each sample inside a batch by spectral analysis rather than by individual loss values. For every batch it builds a cosine-similarity matrix, forms the graph Laplacian, and keeps the samples whose entries in the Fiedler vector (the second-smallest eigenvector) have the largest absolute values. The paper reports that training a pre-trained ResNet-18 on these selected subsets for 25 epochs reduces training time by up to 8x and raises test accuracy by up to 5% on the primary cat-and-dog dataset compared with standard full-data training, while also running faster than JEST. On CIFAR-10 the reported gains are mainly speed (training time drops from about 39 to 23 minutes) with test accuracy nearly unchanged.","feed_headline":"Spectral selection cuts training time 8x, lifts accuracy 5%","feed_subtitle":"Choosing the most informative images per batch trains faster, with accuracy gains on the cat-and-dog dataset.","key_machinery":"The load-bearing object is the within-batch cosine-similarity Laplacian and its Fiedler vector, defined as the eigenvector for the second-smallest eigenvalue. The algorithm computes $S_{ij}$ as the cosine similarity between feature vectors in the batch, sets $D_{ii}=\\sum_j S_{ij}$, forms $L=D-S$, and selects the $n_{\\mathrm{draws}}$ indices $\\mathrm{argsort}(|v_{\\mathrm{Fiedler}}|)[-n_{\\mathrm{draws}}:]$. The Fiedler vector is a standard spectral-clustering tool; here it serves as a cheap per-batch ranking of which samples sit at structurally significant positions in the similarity graph, and the selected fraction becomes the effective training batch.","core_discovery":"On its own terms, the paper's discovery is that the Fiedler vector of a batch's Laplacian is a usable importance score for joint sample selection. The procedure extracts features for all images in a batch with a fixed pre-trained ResNet-50, computes pairwise cosine similarities, forms the degree matrix and Laplacian $L=D-S$, and takes the eigenvector corresponding to the second-smallest eigenvalue. The indices with the largest absolute entries in that Fiedler vector are declared the most informative and are the only samples passed to the model, with the fraction controlled by a filter ratio. The paper argues that this spectral ranking captures structural significance that individual-sample heuristics miss, and that jointly selecting such batches makes training both faster and at least as accurate as standard training on the datasets tested.","pith_inferences":["An extension the paper does not explore is replacing Fiedler-vector ranking with random selection at the same filter ratio and the same number of gradient steps, which would isolate whether spectral structure or merely training on fewer samples explains the speed-up.","A testable prediction of the spectral heuristic is that selected samples are the ones straddling cluster boundaries in feature space, so visualizations of selected batches should show more boundary and outlier images than a uniform sample.","If the ranking is informative, the benefit should grow when the reference features are semantically meaningful and shrink when features are random or class-agnostic, giving a concrete probe of the mechanism."],"forward_implications":["If the spectral ranking is doing the work, any training loop that already draws batches can prepend SALN's selection step and train on a smaller effective batch without changing the loss, optimizer, or model.","On the primary cat-and-dog dataset, the reported numbers imply that 25 epochs of SALN (6.31 minutes) train a ResNet-18 to higher validation accuracy than 25 epochs of standard training (24.48 minutes), so wall-clock savings come with accuracy gains on this dataset.","The method's dependence on a fixed pre-trained feature extractor means it can be applied before training starts, without per-epoch re-embedding, keeping the overhead to a batch-sized eigendecomposition.","Because selection is within-batch rather than over the whole dataset, the method scales to streaming or distributed settings where a global coreset is not available."],"supporting_citations":[{"why":"It supplies the joint-example-selection baseline (JEST) that SALN adapts and compares against.","marker":"[8]"},{"why":"It introduces the Fiedler value and Fiedler vector that the method uses as the informativeness score.","marker":"[9]"},{"why":"It supplies the spectral-clustering account of the Laplacian that justifies reading structure from eigenvectors.","marker":"[20]"},{"why":"It provides the spectral-clustering algorithm and analysis used to argue that eigenvectors reveal data structure.","marker":"[21]"},{"why":"It supplies the primary 37-class cat-and-dog image dataset used in the experiments.","marker":"[22]"},{"why":"It supplies the ResNet-18 architecture that is fine-tuned in all reported experiments.","marker":"[12]"}],"fun_headline_variants":["Spectral batch selection: 8x faster training, 5% better accuracy","Fiedler vector batch selection: 8x faster, 5% more accurate","SALN: spectral selection speeds training 8x, boosts accuracy 5%","Joint batch spectral selection: 8x speedup, 5% accuracy lift","Prioritize batches via Fiedler vector: 8x faster, 5% accurate"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper assumes, without a derivation or ablation, that the samples with the largest absolute Fiedler-vector entries inside a batch's similarity Laplacian are the most informative for training.","fun_headline_variants_meta":{"raw":{"variants":["Spectral batch selection: 8x faster training, 5% better accuracy","Fiedler vector batch selection: 8x faster, 5% more accurate","SALN: spectral selection speeds training 8x, boosts accuracy 5%","Joint batch spectral selection: 8x speedup, 5% accuracy lift","Prioritize batches via Fiedler vector: 8x faster, 5% accurate"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000546,"raw_usage":{"total_tokens":2567,"prompt_tokens":856,"completion_tokens":1711,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":472,"completion_tokens_details":{"reasoning_tokens":1600}},"tokens_in":472,"tokens_out":1711,"duration_ms":11979,"temperature":1.0,"reasoning_tokens":1600,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T05:48:36.609490+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the same pre-trained ResNet-18 on the primary cat-and-dog dataset for the same 25 epochs and filter ratio, but choose the kept samples uniformly at random within each batch; if the random-subset run matches SALN's accuracy and training time, the Fiedler-vector ranking carries none of the reported benefit.","supporting_citations":[{"cited_title":"Fiedler, Algebraic connectivity of graphs","cited_arxiv_id":null,"evidence_quote":"It introduces the Fiedler value and Fiedler vector that the method uses as the informativeness score."},{"cited_title":"von Luxburg, A Tutorial on Spectral Clustering","cited_arxiv_id":null,"evidence_quote":"It supplies the spectral-clustering account of the Laplacian that justifies reading structure from eigenvectors."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It provides the spectral-clustering algorithm and analysis used to argue that eigenvectors reveal data structure."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It supplies the primary 37-class cat-and-dog image dataset used in the experiments."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It supplies the ResNet-18 architecture that is fine-tuned in all reported experiments."}],"review_version":1}