{"id":"8f5ed436-a2e8-4c16-99a9-d9b07bb516ff","arxiv_id":"1908.09258","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A sparse auto-encoder called RandNet learns image dictionaries and classifies MNIST from random compressed measurements, reaching 1.56% test error at 50% compression.","lead":"RandNet trains sparse-coding auto-encoders on randomly compressed images, cutting memory and computation needs. It learns image dictionaries and classifies MNIST digits from half-size random projections with small accuracy loss.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Simulated dictionary recovery is tested only from a warm start: Sec. 4.3.1 initializes the estimated dictionary by perturbing the true A to err≈0.5, so Fig. 2 shows refinement rather than recovery from compressed data.","rationale":"The reader's weakest_assumption concerned the existence and identifiability of a sparse dictionary from the specific compressed measurements. The present concern is closely related but distinct: even if such a dictionary exists and is identifiable in principle, the paper's quantitative evidence for recovery is compromised because the simulation starts within the basin of attraction of the true dictionary. This makes the unsupervised claim weaker than the text suggests. I do not see this as grounds for rejection, because the flaw is in the demonstration rather than in the proposed architecture, and a random-initialization rerun could settle it. The supervised MNIST result may also have evaluation-protocol ambiguities, but the warm-start issue is the single most load-bearing weakness for the paper's central claim about dictionary learning. The verdict remains CONDITIONAL pending the requested experiment.","tokens_in":8838,"tokens_out":16301,"duration_ms":179421,"concrete_test":"Rerun the simulation of Sec. 4.1.1/4.3.1 with the dictionary initialized from i.i.d. N(0,1/500) entries with unit-normalized columns, without using the true A, for CRsAE and for RandNet with Gaussian and row-sparse Phi at beta=0.1, 0.3, and 0.5. Report the final err(A, hat A) after the same 10 epochs. As an additional identifiability control, run B=1 (a single measurement matrix) alongside B=40. If final err does not approach the levels shown in Fig. 2, the unsupervised dictionary-recovery claim is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central unsupervised claim—that RandNet performs dictionary learning using compressed data—rests on the simulation in Sec. 4.3.1, but that experiment initializes the estimated dictionary by randomly perturbing the true dictionary A so that err(A, hat A) ≈ 0.5. With err=0.5, each estimated atom is within roughly 30 degrees of its ground-truth atom, placing the optimization in the basin of attraction of the true solution. The figure therefore demonstrates that the unrolled FISTA autoencoder can refine a near-oracle dictionary from random projections; it does not demonstrate that the non-convex compressive DL objective in Eq. 5 can be solved from a random start. The MNIST experiment does initialize A randomly (Sec. 4.2.2), but there is no ground-truth dictionary, so it cannot supply quantitative recovery evidence. As a result, the paper's central claim that RandNet learns dictionaries from randomly compressed data is not actually established by the reported experiments.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces RandNet, a recurrent sparse auto-encoder whose encoder implements T iterations of FISTA on the compressed-data dictionary-learning problem (Eq. 5), with a tied decoder ΦA; for supervised tasks, the resulting sparse code feeds a linear-softmax classifier. The authors provide a memory and computational efficiency analysis for Gaussian and row-sparse measurement matrices (Table 1), a simulation of dictionary recovery at β ∈ {0.1, 0.3, 0.5}, and MNIST classification experiments at β = 0.5 reporting 1.56% test error with Gaussian and 3.16% with row-sparse measurements.","tokens_in":8990,"tokens_out":3567,"duration_ms":37824,"significance":"If established, the ability to learn dictionaries and to classify from random projections with only a modest accuracy loss would be valuable in memory- and compute-limited settings. The architecture is clearly described, the extension of CRsAE to dense dictionaries and to compressed inputs is natural, and Table 1 gives a useful accounting of where the savings occur. The MNIST result at β = 0.5 is suggestive and, if reproduced with honest evaluation, would be a worthwhile contribution. However, the central unsupervised claim is not currently established because the recovery experiment starts from a near-oracle initialization, and the classification results are weakened by test-set tuning and selective reporting; these issues are fixable within the scope of the manuscript.","major_comments":[{"comment":"The dictionary-recovery experiment initializes  by randomly perturbing A so that err(A, Â) ≈ 0.5, which places each estimated atom within roughly 30 degrees of the true atom. Fig. 2 therefore demonstrates that the unrolled FISTA auto-encoder can refine a near-oracle dictionary from compressed data; it does not demonstrate that the non-convex objective in Eq. 5 can be solved from a random initialization. Since the paper's central claim that RandNet performs dictionary learning from compressed data rests on this experiment, a random-start recovery experiment with the same simulation setup is needed, and the final err values for each β should be reported.","section":"Sec. 4.3.1, Fig. 2"},{"comment":"The regularization parameter λ is selected by sweeping on the test set: the text reports that the classification error reaches its minimum at λ = 2.2 for Gaussian and λ = 2 for row-sparse measurements, and these values are then used for the reported 1.56% and 3.16% test errors. This is test-set tuning, which makes the reported generalization performance optimistic. A validation split should be used for λ selection, and the test error should be reported once for the chosen λ.","section":"Sec. 4.3.2, Fig. 4"},{"comment":"The paper states that 'the accuracy for more efficient networks (lower β) was lower and hence not reported.' This omits exactly the data needed to support the memory/computation-versus-accuracy tradeoff that motivates RandNet. Please report the test errors for lower β values, or explicitly state that these experiments were not run, so that readers can judge whether the framework remains accurate at meaningful compression levels.","section":"Sec. 4.3.2, Table 2"},{"comment":"It is not clear from the text whether the DrSAE and SDL baselines are trained on compressed inputs or on the original images. If these baselines see the original data while RandNet sees only random projections, Table 2 does not establish RandNet's performance relative to alternatives under the same compressed-data constraint. Please clarify the input modality for each baseline and, if necessary, add compressed-input versions of the baselines.","section":"Sec. 4.3.2, Table 2"}],"minor_comments":[{"comment":"The figure shows learning curves but no final numeric err values and no indication of variability across random seeds; reporting the mean and standard deviation over several runs would make the comparison with CK-SVD more informative.","section":"Fig. 2"},{"comment":"The choices T = 400 for the simulation and T = 60 for MNIST are stated without sensitivity analysis; a brief discussion or ablation of the dependence on the number of FISTA iterations would help the reader assess the practical robustness of the architecture.","section":"Sec. 4.2.1 and 4.2.2"},{"comment":"The claim that this is the first time CRsAE is trained to learn a dense dictionary is strong and is not supported by a comparison with prior dense dictionary-learning auto-encoder methods; please soften the claim or add appropriate references.","section":"Sec. 2.2"},{"comment":"The definition of the softmax output in the text is garbled ('û = eq / ∑ i e qi'); the vector notation for the cross-entropy loss should be written out cleanly so that the dimensions of C, d, and u are unambiguous.","section":"Eq. (7)"}],"recommendation":"major_revision","confidential_remarks":"The paper is closer to a strong workshop contribution than to a fully supported journal article: the architecture and efficiency analysis are useful, but the warm-start recovery experiment undermines the central claim, and the MNIST evaluation has avoidable methodological weaknesses. I would be happy to see a revised version in which the recovery experiment is run from a random initialization, λ is selected on a validation set, and the lower-β results are reported."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The real contribution here is the integration: an unrolled FISTA auto-encoder with tied weights, trained end-to-end on random projections of images, with a memory/computation story that is honest and well quantified. Extending CRsAE from convolutional to dense dictionaries and to compressed inputs is natural, and the shared-seed trick for avoiding explicit storage of the measurement matrices is a nice practical touch. The MNIST results at beta=0.5 (1.56% Gaussian, 3.16% row-sparse) are genuinely interesting, especially given that the input dimension is cut in half. If I work on compressed-domain representation learning, this is the kind of result I want to know about.\n\nThe soft spots are real but mostly fixable. The one that matters most is the simulation in Sec. 4.3.1. The estimated dictionary is initialized by randomly perturbing the true A so that err(A, A_hat) ≈ 0.5. For unit-norm columns, that means each atom is within roughly 30 degrees of ground truth. That is a warm start, not a random start. So Fig. 2 shows that the unrolled network can refine a near-oracle dictionary from compressed data, but it does not show that RandNet can solve the non-convex compressive DL problem from scratch. The MNIST experiment does initialize A randomly, but there is no ground-truth dictionary there, so it cannot supply quantitative recovery evidence. The paper's claim to 'demonstrate that RandNet performs dictionary learning using compressed data' is therefore stronger than what the experiments support. This is not a fatal flaw in the architecture, but it should be stated as a gap and fixed with a random-start simulation.\n\nThe evaluation also has the usual conference-paper issues: lambda is chosen by test-set grid search (Fig. 4), results for beta<0.5 are omitted because accuracy was lower, there are no error bars, and no code is shipped. Calling the MNIST accuracy 'rivals the state-of-the-art' is a stretch — DrSAE and SDL see uncompressed data — though 'close, given half the input dimension' is fair. The compressed-sensing explanation is post hoc but plausible, and it does not make the result circular.\n\nOn balance, the integration is sound and the memory argument is the strongest part. The self-citation of CRsAE is legitimate; this is a direct extension of that line. The paper deserves a serious referee, especially if the authors can address the warm-start issue and run from random initializations. I would not cite it in its current form, but I would read a revised version carefully.","headline":"A sensible integration of unrolled auto-encoders with compressed measurements, but the unsupervised recovery experiment starts from a near-oracle warm start and the MNIST evaluation has selection issues; still worth a real review.","tokens_in":9563,"tokens_out":1992,"would_cite":false,"duration_ms":24414,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A recurrent sparse auto-encoder can be trained on random compressed measurements of images to perform dictionary learning and MNIST classification with minimal accuracy loss.","keywords":["random projections","dictionary learning","sparse coding","compressed measurements","constrained recurrent sparse auto-encoder","FISTA","MNIST classification","compressive sensing"],"falsifier":"Run the identical RandNet pipeline on a dataset that provably has no sparse dictionary (for instance, pixel-wise i.i.d. Gaussian noise images): if classification error under 50% Gaussian projections stays near 1.56%, the sparse-dictionary explanation is wrong. Alternatively, fix beta=0.5 on MNIST and sweep the row sparsity of Phi from 1 to 392; the RIP explanation predicts error increases monotonically as rows get sparser, so a flat or non-monotonic error curve would contradict it.","tokens_in":8577,"feed_emoji":"🔢","tokens_out":5930,"duration_ms":56024,"temperature":0.7,"pith_summary":"The paper claims that a neural network can learn useful representations from random compressed measurements of data, rather than from the data themselves. It introduces RandNet, a modification of the constrained recurrent sparse auto-encoder in which the encoder operates on randomly projected images and the decoder reconstructs the projection. In an unsupervised setting, RandNet learns the true underlying dictionary from compressed simulations. In a supervised setting, the sparse code from the encoder serves as features for a classifier, reaching 1.56% MNIST test error with Gaussian random projections and 3.16% with row-sparse projections, at a 50% input reduction. If true, this means deep learning can proceed on data that are never fully acquired or stored, with memory and compute savings.","feed_headline":"Half-size random projections train a 1.56% MNIST classifier","feed_subtitle":"Compressed measurements cut memory and compute while keeping accuracy near full-resolution models.","key_machinery":"The central object is the unrolled FISTA encoder: a recurrent network that repeats T iterations of a gradient step w_t + (1/L) A^T Phi^T (r - Phi A w_t) followed by a two-sided ReLU shrinkage nonlinearity eta_epsilon, with momentum coefficients s_t = (1 + $\\sqrt$(1+4s_{t-1}^2))/2. The decoder applies Phi A, and A is shared (tied) between encoder and decoder, so backpropagation through the unrolled iterations updates only the dictionary A. The measurement matrix Phi is random and fixed; examples are divided into B blocks that share Phi_b, and only the random seeds are stored, yielding memory cost O($\\beta$ N) and, for row-sparse Phi, matrix-operation cost O(gamma N) with $\\beta$ = M/N and gamma = $\\beta$ s.","core_discovery":"RandNet demonstrates, for the first time, that a constrained recurrent sparse auto-encoder can be trained on compressed random measurements to perform dictionary learning and classification. The architecture solves the compressive dictionary-learning objective min_{x,A} sum_j 1/2 || r_j - Phi A x_j ||$_2^{2}$ + $\\lambda$ ||x_j||_1 with unit-norm columns of A, using an encoder that unrolls T FISTA iterations and a tied linear decoder. On MNIST with 392 random measurements per 784-pixel image, the learned sparse codes classify digits at 1.56% error with Gaussian Phi and 3.16% with row-sparse Phi, compared with 3.72% and 5.20% for the CK-SVD baseline, and 1.08% and 1.05% for uncompressed DrSAE and SDL. The results are evidence that MNIST images admit a dictionary in which they are sparse and that this dictionary is identifiable from random projections, supporting a compressed-sensing explanation.","pith_inferences":["If the result generalizes beyond MNIST, any dataset known to be sparsely representable in a fixed basis (wavelets, curvelets, or a learned dictionary) could be trained on directly from compressive-camera measurements, bypassing image reconstruction entirely.","The two-stage training (dictionary first, classifier second) leaves a small accuracy gap to jointly trained baselines; a testable extension is to train the classifier and dictionary jointly, which may recover that gap.","The paper's RIP-based explanation predicts that error rises as measurement rows become sparser; this is directly testable by sweeping row sparsity s at fixed beta, and a flat error curve would falsify that explanation.","The use of stored random seeds for shared projection matrices suggests a distributed or privacy-preserving variant in which each node only shares compressed projections and a seed, never raw data."],"forward_implications":["Training on compressed data with Gaussian Phi cuts memory storage by a factor beta; with row-sparse Phi it cuts both memory-access and matrix-operation costs by gamma, allowing larger effective batch sizes on fixed hardware.","Unsupervised dictionary learning no longer requires access to full-resolution examples, so dictionaries can be learned from compressive sensors or from data that are only available as random projections.","The sparse codes obtained from compressed measurements are directly usable as classification features, indicating that reconstruction of the image is not a necessary intermediate step.","Because the measurement matrices are fixed and shared, the same framework applies to any block-structured dataset, with only random seeds stored rather than the full projection matrices."],"supporting_citations":[{"why":"Supplies the constrained recurrent sparse auto-encoder architecture that RandNet extends to dense dictionaries.","marker":"[2]"},{"why":"Provides the recurrent sparse auto-encoder formulation and the tied encoder-decoder template used by RandNet.","marker":"[3]"},{"why":"Establishes the memory and computation cost model for very sparse random projections, which RandNet adopts for row-sparse Phi.","marker":"[6]"},{"why":"Introduces very sparse random projection dictionary learning and the block-shared measurement matrices that RandNet uses.","marker":"[7]"},{"why":"Shows that a dictionary can be learned from random Gaussian projections with fewer measurements than original dimension.","marker":"[9]"},{"why":"FISTA, whose iterations the RandNet encoder unrolls to compute sparse codes.","marker":"[10]"},{"why":"Discriminative recurrent sparse auto-encoder, the supervised architecture RandNet extends and compares against.","marker":"[11]"},{"why":"Supervised dictionary learning baseline that reaches near-state-of-the-art MNIST error without compression.","marker":"[13]"},{"why":"Compressed sensing restricted-isometry guarantee used to argue that sparse codes are recoverable from few random projections.","marker":"[14]"}],"fun_headline_variants":["RandNet learns from half-size random projections with near-full accuracy","Compressed measurements: RandNet trains with 50% less data, minimal loss","Random projections shrink inputs, RandNet keeps MNIST error under 2%","Half-size random measurements: RandNet's compressed learning","RandNet: deep learning on compressed random projections"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The data must have a dictionary in which they are sparse, and the particular block-random projections used must preserve enough information to recover that dictionary and its sparse codes.","fun_headline_variants_meta":{"raw":{"variants":["RandNet learns from half-size random projections with near-full accuracy","Compressed measurements: RandNet trains with 50% less data, minimal loss","Random projections shrink inputs, RandNet keeps MNIST error under 2%","Half-size random measurements: RandNet's compressed learning","RandNet: deep learning on compressed random projections"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000388,"raw_usage":{"total_tokens":2045,"prompt_tokens":939,"completion_tokens":1106,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":555,"completion_tokens_details":{"reasoning_tokens":1018}},"tokens_in":555,"tokens_out":1106,"duration_ms":8031,"temperature":1.0,"reasoning_tokens":1018,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T11:17:06.658536+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the identical RandNet pipeline on a dataset that provably has no sparse dictionary (for instance, pixel-wise i.i.d. Gaussian noise images): if classification error under 50% Gaussian projections stays near 1.56%, the sparse-dictionary explanation is wrong. Alternatively, fix beta=0.5 on MNIST and sweep the row sparsity of Phi from 1 to 392; the RIP explanation predicts error increases monotonically as rows get sparser, so a flat or non-monotonic error curve would contradict it.","supporting_citations":[{"cited_title":"RandNet: deep learning with compressed measurements of images","cited_arxiv_id":"1908.09258","evidence_quote":"Supplies the constrained recurrent sparse auto-encoder architecture that RandNet extends to dense dictionaries."},{"cited_title":"First, we in- troduce the architecture in unsupervised and supervised set- tings","cited_arxiv_id":null,"evidence_quote":"Provides the recurrent sparse auto-encoder formulation and the tied encoder-decoder template used by RandNet."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Establishes the memory and computation cost model for very sparse random projections, which RandNet adopts for row-sparse Phi."},{"cited_title":"Learning sparsely used overcomplete dic- tionaries via alternating minimization,","cited_arxiv_id":null,"evidence_quote":"Introduces very sparse random projection dictionary learning and the block-shared measurement matrices that RandNet uses."},{"cited_title":"Deep Residual Autoencoders for Expectation Maximization-inspired Dictionary Learning","cited_arxiv_id":"1904.08827","evidence_quote":"Shows that a dictionary can be learned from random Gaussian projections with fewer measurements than original dimension."},{"cited_title":"Convolutional neu- ral networks analyzed via convolutional sparse coding,","cited_arxiv_id":null,"evidence_quote":"FISTA, whose iterations the RandNet encoder unrolls to compute sparse codes."},{"cited_title":"Efficient dictionary learning via very sparse random projections,","cited_arxiv_id":null,"evidence_quote":"Supervised dictionary learning baseline that reaches near-state-of-the-art MNIST error without compression."},{"cited_title":"Gradient-based learning applied to document recogni- tion,","cited_arxiv_id":null,"evidence_quote":"Compressed sensing restricted-isometry guarantee used to argue that sparse codes are recoverable from few random projections."}],"review_version":1}