{"id":"f546b493-d0c8-4b46-808f-e3acb2751c40","arxiv_id":"2501.15603","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"An improved fixation point generator for TDFN is trained to match the difference between the network's reconstructed image and the input image, yielding better MNIST accuracy with fewer fixations than the prior RL approach.","lead":"Wang and Wang replace reinforcement learning in their Task-Driven Fixation Network with a self-supervised target: the fixation point generator is trained to predict where the network's own reconstruction fails, and the next fixation is placed at the largest error. On MNIST, the new method beats their earlier RL baseline in accuracy and in fixations needed, but lacks comparisons to standard classifiers.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central claim depends on an unproven equivalence: pixel-wise reconstruction error (Eq. 7) is treated as classification-relevant saliency, but the reconstructor is trained with MSE (Eq. 10), not a task loss, and the Taylor argument in §3.1 is only a heuristic.","rationale":"The reader's verdict is conditional for the right reason. I independently find the weakest assumption to be the equivalence between reconstruction error and task-relevant difference. My reading adds a concrete reason this equivalence is suspect: the Taylor expansion in §3.1 is an analogy applied to a loss that is not defined on the full input image, and the reconstructor is optimized with a pixel-wise objective that is agnostic to classification. The empirical comparison to FPG1 is meaningful but narrow: FPG1 is the authors' earlier RL method on a 4×4 grid, so the improvement may reflect the trivial advantage of high-resolution over coarse fixation selection rather than the reconstruction-error principle. No external baselines, error bars, or repeated-seed results are provided, so the generality claim is unsupported. However, the paper is internally coherent, the method is clearly specified, and the visualizations are consistent with the reported behavior on MNIST. I therefore do not request rejection; the paper should be accepted only conditionally on demonstrating that the reconstruction-error proxy transfers beyond MNIST or on narrowing the claim.","tokens_in":6510,"tokens_out":5147,"duration_ms":49876,"concrete_test":"Compute, on held-out MNIST and CIFAR-10 images, the Spearman rank correlation between the reconstruction error map (Eq. 7) and a task-relevance map such as |∂L/∂x| or occlusion sensitivity. Additionally, run the same TDFN with FPG2, random fixation, and center fixation at equal fixation counts on CIFAR-10. If the correlation is low or FPG2 does not beat random/center fixation, the proxy assumption fails; if FPG2 still wins, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing step is the identification in §3.1–3.3 of the reconstruction error map |ReconImage − InputImage| (Eq. 7) with the classification-relevant term (x* − x) of Eq. (1). This identification is not justified. Eq. (1) concerns a loss L(x) defined on the full high-resolution input x, but TDFN's classifier never receives the full image: it receives a 4×4 low-resolution global view plus selected 8×8 ROIs, so L as a function of the missing full image is not the object being optimized. Moreover, the reconstructor is trained only to minimize pixel-wise MSE (Eq. 10); nothing ties its errors to classification loss. Hence the saliency map, even if it perfectly predicts reconstruction error, may simply point to regions where the autoencoder is uncertain (high-frequency edges, texture, background clutter). On MNIST these regions coincide with digit strokes, which explains the reported gains, but the central claim is stated generally. The absence of external baselines and error bars makes it impossible to tell whether FPG2 beats even a random or center-fixation baseline on more complex images.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes FPG2, a supervised replacement for the reinforcement-learning-based fixation point generator (FPG1) in the Task-Driven Fixation Network (TDFN). The core idea is to train a fixation generator to predict the absolute difference between an internally reconstructed image and the input image (Eqs. 6–7), and to select the argmax of the resulting saliency map as the next fixation point. The method is motivated by a Taylor-expansion argument in §3.1 in which the reconstructed image is identified with a loss-minimizing 'template' x*. Experiments on MNIST (§4) report that FPG2 achieves higher classification accuracy than FPG1 for the same number of fixations and higher accuracy with fewer fixation steps under dynamic termination.","tokens_in":6773,"tokens_out":13293,"duration_ms":104840,"significance":"If its central premise is accepted — that the reconstruction error map is a task-relevant saliency signal — the paper's engineering contribution is genuine: it replaces an unstable RL procedure with a straightforward supervised MSE regression (Eq. 6), enables fixation selection at full 32×32 resolution, and reuses TDFN's existing reconstruction module. The paper is commendably honest, stating in the Conclusion that the method 'is not the optimal solution' and is tailored to TDFN. The dynamic-termination analysis in Table 2 is a useful addition, and the premise is falsifiable via ablations. However, the theoretical justification is at present only a heuristic, the load-bearing identification of reconstruction error with classification-relevant difference is unproven, and the empirical support is limited to single-run MNIST results against the authors' own baseline. The paper releases no code; its strengths are the clarity and simplicity of the proposal rather than the strength of the evidence.","major_comments":[{"comment":"The Taylor-expansion argument does not, as written, support the fixation-selection criterion. TDFN's classifier loss is a function of the low-resolution global input and the sequence of selected ROIs, not of the full-resolution image x; the ROI selection is discrete (argmax in Eq. 5), so the gradient L'(x) with respect to the full image is not the derivative of the actually optimized function. The step 'Assuming L(x*) is very small and can be ignored' also conflates the minimum loss value with the truncation error of the expansion. As it stands, Eq. (1) yields at most a heuristic; the authors should either label it as such and support it empirically, or derive the criterion from the classifier's true inputs.","section":"§3.1, Eq. (1)"},{"comment":"The substitution of the reconstructed image for the template x* is unproven and load-bearing. The reconstructor is trained only with pixel-wise MSE (Eq. 10), which measures pixel fidelity; a reconstructor can have small MSE while being wrong at classification-critical pixels, and its largest errors may lie on textures, edges, or background clutter with no class evidence. Since the training target ReconErrMap (Eq. 7) is produced by the network's own reconstruction module, FPG2's quality is entirely inherited from the unproven task-relevance of that internally generated map. The paper should test the premise directly by replacing ReconErrMap with alternative targets (gradient-based saliency, edge magnitude, random, or center fixations) under identical training and showing that the reconstruction-difference target is what drives the gains. This ablation is essential on MNIST, where digit strokes dominate both reconstruction error and class-relevant pixels.","section":"§3.2–§3.3, Eqs. (7), (10)"},{"comment":"The experimental evidence is too thin for the 'significantly enhances' claims. All accuracies come from a single run, with no error bars, no variance over seeds, and no significance test. The only comparator is FPG1, the authors' own RL method; there are no trivial baselines such as random fixations, center fixation, or a classical saliency method. Since FPG1 selects on a 4×4 grid while FPG2 selects on the full 32×32 image, the gains could partly reflect the finer decision grid rather than the proposed target. In Table 2, matched MCP thresholds do not equate operating points because FPG1 and FPG2 operate at very different coverage levels (e.g., 38.54% versus 7.78% at MCP 0.99), and the abstract's phrase 'achieve a predefined accuracy level' does not match the protocol, which terminates on the classifier's maximum probability rather than on achieved accuracy. Multi-seed results, at least one non-MNIST dataset, and fixed operating points are needed.","section":"§4.3–§4.4, Tables 1–2"},{"comment":"The claim of 'highly accurate fixation points' is operationalized only through downstream classification accuracy and subjective visualization; no quantitative measure of fixation precision is provided. Without ground-truth fixation annotations or digit-part masks, statements such as fixations being 'more accurately located at stroke endings, corners, intersections, and edges' (§4.2) cannot be checked. The authors should either quantify precision (e.g., distance to digit-stroke pixels, or agreement with human fixations) or rephrase the claim as task-performance improvement rather than spatial precision.","section":"§4.2, Fig. 3"}],"minor_comments":[{"comment":"The running title on the first page reads 'P RECISE FIXATION POINT GENERATION'; it should read 'PRECISE FIXATION POINT GENERATION'.","section":"Title/page 1"},{"comment":"The expansion L(x*) ≈ L(x) + L'(x)^T (x* − x) should define the gradient with respect to the image and state that the second-order term is omitted, so the approximation is transparent.","section":"§3.1, Eq. (1)"},{"comment":"In §4.1, 'All images were normalized to dimensions of 32 × 32 pixels' should specify the resizing or padding operation and any value normalization (e.g., mean and standard deviation), since these directly affect both the reconstruction-error target and the classification accuracy.","section":"§4.1"},{"comment":"Section 4.1 reports only the chosen values α = 0.2 and β = 0.1 for Eq. (8); the optimizer, learning rate, batch size, number of epochs, and any sensitivity analysis for α and β are missing.","section":"§4.1"},{"comment":"In Eq. (6), it is unclear whether the SaliencyMap is normalized (e.g., by a sigmoid) before the MSE with ReconErrMap is computed; a sentence on the scaling of the two maps would rule out trivial scale artifacts in FixationLoss.","section":"§3.3, Eq. (6)"},{"comment":"In Table 2, the rows for MCP thresholds 0.993 and 0.995 show '–' for FPG1; the paper should state explicitly why FPG1 does not reach these thresholds (e.g., saturation of its maximum classification probability).","section":"§4.4, Table 2"},{"comment":"The in-text citation 'Yulin Xie et al., 2013' should be 'Xie et al., 2013', and the DeepGaze II reference entry contains a typo ('arXiv.1610.01563' should be 'arXiv:1610.01563').","section":"References"},{"comment":"The statement in §2 that the fixed patch schema of ViT is 'impractical' for pixel-level segmentation is stronger than needed for the paper's argument and could be softened.","section":"§2"},{"comment":"The Conclusion states that the method 'is not the optimal solution' and is tailored to TDFN; this honest limitation should be reflected in the abstract, which currently claims a general ability to achieve 'highly accurate fixation points' without qualification.","section":"§5/Abstract"}],"recommendation":"major_revision","confidential_remarks":"To the editor: the conditional verdict is, in my view, correctly calibrated. The manuscript's core claim depends on an unproven identification of reconstruction error with task-relevant saliency; this is a fixable gap if the authors supply target-map ablations and multi-seed comparisons against trivial baselines, so I recommend major revision rather than rejection. The paper builds on the authors' own predecessor (arXiv:2501.01548), so there is no novelty-disclosure concern, but the revision should make the FPG1 description self-contained. The scope is narrow (MNIST only), and acceptance should likely depend on the suggested additional experiments."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the paper replaces reinforcement learning for fixation point selection with a much simpler regression target—the absolute difference between TDFN's internally reconstructed image and the input. That is a neat trick, and on MNIST it works: FPG2 beats their own RL-based FPG1 by a wide margin at every fixation count. The writing is clear, and the method is straightforward enough to reproduce from the description.\n\nWhat's genuinely new is the specific target: no prior work I know of trains a fixation generator to predict the reconstruction error of the network's own reconstructor. The authors are honest in the conclusion that the approach is a practical hack rather than an optimal solution, which is fair.\n\nThe soft spots are real. The Taylor expansion in Sec. 3.1 is doing a lot of work it can't support. The loss L(x) is defined on the full high-resolution image, but TDFN never sees the full image—it sees a 4x4 low-res view plus 8x8 ROIs. And the reconstructor is trained with MSE (Eq. 10), not a task loss, so nothing ties its errors to classification loss. The step from (x* - x) in Eq. (1) to |ReconImage - InputImage| in Eq. (7) is a heuristic. The stress-test criticism holds up: on MNIST, reconstruction error concentrates on digit strokes, so the saliency map works; on natural images it might just be highlighting edges and texture.\n\nThe experimental evidence is also thin. Single runs, no error bars. No comparison to a random or center-fixation baseline, which is essential to show the learned fixations actually help. No dataset beyond MNIST and no comparison to any external method. The claimed 'highly accurate fixation points' is evaluated only by downstream classification accuracy, not against any ground-truth fixation data. All of this is fixable, but it means the paper currently supports an incremental claim: a better training signal for their own architecture on one digit dataset.\n\nI would send this to peer review—it's a clear, self-contained contribution with a well-defined method and a credible result on its home turf. But a responsible referee should ask for more baselines, error bars, and at least one more dataset before taking the general claim seriously. For my own work, I wouldn't cite it as evidence of a general principle; I'd treat it as a useful trick in the fixation-based toolbox.","headline":"Simple self-supervised fixation target from reconstruction error beats their RL baseline on MNIST, but the evidence is too thin to support the general claim.","tokens_in":7252,"tokens_out":2760,"would_cite":false,"duration_ms":25504,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T07","68T45"],"pacs":[],"model":"deepseek-v4-flash","headline":"TDFN can train its fixation-point generator against the reconstruction error map |ReconImage − InputImage|, and doing so yields pixel-accurate fixations that improve classification accuracy and cut the number of fixations needed.","keywords":["fixation point generation","Task-Driven Fixation Network","reconstruction difference","saliency map","dynamic patch selection","Transformer image classification","MNIST"],"falsifier":"A straightforward test: on a variant of MNIST where a high-contrast irrelevant mark is added to the background, the reconstruction error map will concentrate on the mark. If FPG2 fixates there and accuracy drops relative to FPG1, the reconstruction-error proxy is not tracking classification-relevant content.","tokens_in":6327,"feed_emoji":"🎯","tokens_out":6370,"duration_ms":51513,"temperature":0.7,"pith_summary":"The paper replaces the reinforcement-learning-based fixation point selector inside the Task-Driven Fixation Network (TDFN), a Transformer that classifies an image from a low-resolution global view plus a few high-resolution local crops centered on 'fixation points,' with a supervised regression. The new generator is trained to output a saliency map that matches the absolute pixel-wise difference between the network's internally reconstructed image and the actual input, and the next fixation is placed where that difference is largest. On MNIST this yields fixation points at stroke endpoints, corners, and intersections, and it improves classification accuracy at every fixation budget tested—two fixations rise from 84.20% to 94.76%, eight from 96.11% to 99.19%. The method also lets TDFN stop early: with an average of 1.57 fixations it reaches 99.29% accuracy while covering under 10% of the image. If it holds beyond MNIST, this gives a cheap, differentiable way to do dynamic, content-adaptive patching in vision Transformers.","feed_headline":"Fixation points from reconstruction error beat reinforcement learning","feed_subtitle":"On MNIST, two fixations reach 94.76% accuracy; 1.57 average fixations reach 99.29% with under 10% coverage.","key_machinery":"The load-bearing object is the reconstruction error map |ReconImage − InputImage| (Eq. 7). The reconstructor predicts a full-resolution image from the available low-resolution plus ROI inputs; the fixation generator is trained to predict this error map; and the next fixation is the saliency map's argmax. The paper motivates this with a Taylor expansion identity (Eq. 1) that ties quick loss reduction to large differences between the input and a template, with the reconstructed image standing in for the template. The machinery consists of three jointly trained modules—classifier, reconstructor, and fixation generator—with total loss ClassLoss + α·ReconLoss + β·FixationLoss, using α = 0.2 and β = 0.1.","core_discovery":"Starting from a Taylor expansion of the loss around a 'template' input x*, the paper argues that the fastest way to reduce loss is to attend where the derivative L'(x) is large and where (x* − x) is large. Since TDFN cannot compute the derivative at inference time, the paper substitutes a network-internal reconstruction of the input for x*, turning (x* − x) into the computable reconstruction error map |ReconImage − InputImage|. The fixation generator is trained with mean squared error against this error map (Eq. 6), so the argmax of its saliency map is the next fixation point. The paper reports that on 32×32 MNIST this 'FPG2' generator produces fixations at informative locations and beats the prior reinforcement-learning 'FPG1' generator in both fixed-budget and early-termination comparisons.","pith_inferences":["Extension beyond the paper: the reconstruction-error proxy may not align with classification-relevant content. A direct test would compare fixation targets from reconstruction error against class-saliency maps from occluding pixels; if the overlap is no better than chance, the accuracy gain on MNIST may come from digits exactly occupying high-error regions.","Extension: on natural images, background clutter will produce large reconstruction errors irrelevant to the class, so the method may require a reconstruction trained with a classification-aware term or a saliency gating mechanism.","Extension: the same target could be used for other tasks TDFN might support, such as regression or detection, since the reconstruction error map does not depend on the task head.","Testable: increasing the reconstruction-loss weight α at the expense of classification loss should produce sharper saliency maps but could degrade accuracy if reconstruction and classification goals are not aligned."],"forward_implications":["Pixel-level fixation generation becomes a supervised regression problem instead of a reinforcement-learning problem, eliminating the need to scale RL to the full image grid.","On MNIST, FPG2 improves classification accuracy at every fixation count tested, e.g., 84.20% to 94.76% with two fixations and 96.11% to 99.19% with eight.","With dynamic termination, FPG2 averages 1.57 fixations to reach 99.29% accuracy at an MCP threshold of 0.995, covering only 9.84% of the image.","The improved fixations let TDFN act as a dynamic patching mechanism: fine-grained attention where differences are large, coarse elsewhere, at lower computational cost than fixed fine-grained ViT patches."],"supporting_citations":[{"why":"Defines the TDFN architecture and the reinforcement-learning fixation generator FPG1 that this paper replaces; all experiments compare against it.","marker":"[Wang and Wang, 2025]"},{"why":"Provides the Transformer architecture underlying TDFN and its O(N²D + ND²) complexity, which the adaptive-patching argument builds on.","marker":"[Vaswani et al., 2017]"},{"why":"Establishes the fixed-patch ViT baseline whose patch-size/compute trade-off TDFN is contrasted with.","marker":"[Dosovitskiy et al., 2021]"},{"why":"Supplies the derivative-based class saliency map approach that the paper argues is unsuitable for TDFN because the loss is undefined and the full image is inaccessible at inference.","marker":"[Simonyan et al., 2014]"}],"fun_headline_variants":["Reconstruction error beat RL for fixation points","Fixation points from reconstruction error, not RL","Training fixation generator on reconstruction difference","Fewer fixations, better accuracy: reconstruction error wins","Reconstruction difference guides precise fixations"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper assumes that the pixel-wise difference between the reconstructed image and the input marks the places where an extra high-resolution look will most help classification, even though the reconstruction is trained to match pixel intensities rather than to improve classification.","fun_headline_variants_meta":{"raw":{"variants":["Reconstruction error beat RL for fixation points","Fixation points from reconstruction error, not RL","Training fixation generator on reconstruction difference","Fewer fixations, better accuracy: reconstruction error wins","Reconstruction difference guides precise fixations"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000417,"raw_usage":{"total_tokens":2109,"prompt_tokens":861,"completion_tokens":1248,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":477,"completion_tokens_details":{"reasoning_tokens":1181}},"tokens_in":477,"tokens_out":1248,"duration_ms":8690,"temperature":1.0,"reasoning_tokens":1181,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T14:07:02.633551+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A straightforward test: on a variant of MNIST where a high-contrast irrelevant mark is added to the background, the reconstruction error map will concentrate on the mark. If FPG2 fixates there and accuracy drops relative to FPG1, the reconstruction-error proxy is not tracking classification-relevant content.","supporting_citations":[{"cited_title":"Task-Driven Fixation Network: An Efficient Architecture with Fixation Selection","cited_arxiv_id":"2501.01548","evidence_quote":"Defines the TDFN architecture and the reinforcement-learning fixation generator FPG1 that this paper replaces; all experiments compare against it."}],"review_version":1}