{"id":"3e21adea-08bc-4887-af4a-81f166dd8167","arxiv_id":"2508.14508","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A CNN-Transformer sequence model detects injected moving planet signals in synthetic and semi-synthetic JWST high-contrast imaging data, reaching 100% accuracy in one reported run.","lead":"Researchers built a hybrid CNN-Transformer model that watches a sequence of images to detect a faint moving planet against bright speckle noise. In tests with injected signals, including JWST frames of the TW Hya disk, it classified and located the simulated planets with high accuracy, though the data were simplified.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Semi-synthetic test uses a single static real frame as background, so it does not demonstrate performance on real sequences with evolving speckles.","rationale":"The reader's weakest assumption identifies exactly the same load-bearing point: the semi-synthetic dataset is built from a single static real frame, so the background does not evolve over time. I find no additional objection that would change the verdict. The paper is explicitly a proof-of-concept and its limitation section acknowledges simplified orbits and high SNR, but it does not acknowledge that the background is a single frame reused across all frames, making the temporal background perfectly static. The proposed check with a real time-sequence, or even perturbed static frames, would determine whether the model's success is due to detecting coherent motion against a frozen background or to genuinely separating planet motion from evolving speckles. If the check shows a large drop, the correct verdict remains conditional: the architecture is promising but not yet validated on realistic data. If it shows no drop, the concern is resolved and the claim could be strengthened. Thus the reader's CONDITIONAL verdict is appropriate and unchanged.","tokens_in":7093,"tokens_out":4319,"duration_ms":57555,"concrete_test":"Build a second semi-synthetic dataset using a real time-series of TW Hya frames (for example, 10 consecutive NIRCam exposures from program 1179, preserving the natural speckle evolution and dithering). Inject the same 3-pixel Gaussian planet at SNR 5 with a known circular orbit, train the identical architecture, and evaluate on held-out sequences. Compare F1 and position error against the static-background results. If performance drops materially (for example, F1 < 95% or a significant false-positive rate from drifting speckles), the static-background assumption is load-bearing and the 'realistic data' claim must be weakened. A cheaper variant: add per-frame correlated noise (for example, Gaussian-smoothing with σ = 2 pixels, amplitude 10% of background rms) to the current static backgrounds and measure the drop.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The semi-synthetic evaluation in §2.2 is built by taking a single real JWST frame as the background and injecting a synthetic planet into 'each frame of the sequence.' The background is therefore perfectly static across the 10-frame input. On such data, the unique temporally varying signal is the planet, so the Transformer can succeed by learning 'anything that moves' rather than by separating a planet from quasi-static speckle evolution. Real high-contrast imaging sequences, especially with angular differential imaging, have time-varying speckles, field rotation, and PSF changes. The reported high accuracy and confidence on this semi-synthetic set therefore do not establish the abstract's claim that the model can 'function amidst complex, correlated noise and bright disk features'; they only show detection against a frozen background. The paper's limitations mention more complex speckle patterns as future work, but the 'viability on realistic data' claim depends on this static-background assumption.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a hybrid CNN-Transformer architecture for detecting moving point-source signals in sequences of high-contrast imaging frames. A per-frame CNN extracts features, and a Transformer encoder models temporal dependencies, with two output heads for binary planet presence and (x,y) position regression. The model is trained and evaluated on (1) a purely synthetic dataset of 64×64 Gaussian-noise frames with static speckle-like spots and an injected 3-pixel moving dot, and (2) a semi-synthetic dataset built by injecting a synthetic planet signal into real JWST TW Hya frames. The reported trial achieves 100% accuracy/F1 and 0.72-pixel position error on the synthetic test set, and high-confidence classification (with position error of a few pixels) on the semi-synthetic test set. The paper compares against median subtraction, optimized median subtraction, and PCA-KLIP, and presents the work as a proof of concept for automated exoplanet detection.","tokens_in":7343,"tokens_out":3842,"duration_ms":45808,"significance":"If the claims were established on realistic data, the architecture would be a useful addition to the direct-imaging toolset, offering fast automated candidate identification. The paper is commendably concrete: it provides an open demo notebook, clearly describes the architecture and training setup, and includes an honest limitations paragraph that acknowledges the simplified orbits, high SNR, and need for retraining per instrument. However, the evaluation currently supports only a narrow proof-of-concept claim on generated datasets. The semi-synthetic experiment does not demonstrate viability on real time-varying sequences because the background is static across the sequence, and the key quantitative claim rests on a single training run. With additional experiments and repeated-trial statistics, the central idea could become a solid methods contribution; as it stands, the evidence is not yet commensurate with the abstract's 'viability on realistic data' claim.","major_comments":[{"comment":"The paper reports 100.0% accuracy and F1 for a single 'representative trial' while also stating that 'final accuracies typically reached near-perfect levels (≈99%)' and that metrics 'varied slightly between training runs.' With only one reported run, the 100.0% value is not a statistically meaningful claim; the reader cannot tell whether the result is a lucky seed or a typical outcome. Please report the mean and standard deviation over at least 5–10 independent training runs, and provide confidence intervals or bootstrap uncertainties for the metrics in Table 1. This is particularly important because the headline comparison against PCA-KLIP depends on these numbers.","section":"§3.1, Fig. 3, Table 1"},{"comment":"The semi-synthetic dataset uses 'a real astronomical frame of TW Hya from JWST as the background' and injects the planet into each frame of the sequence. This makes the background perfectly static across the 10-frame input, so the only temporally varying component is the injected planet. A model can succeed by learning 'anything that moves' rather than by separating a planet from quasi-static speckle evolution, which is the central challenge in high-contrast imaging. The claim that the model 'can function amidst complex, correlated noise and bright disk features' and that the semi-synthetic result confirms 'viability on realistic data' is therefore not supported. Please test on data with time-varying speckles, field rotation, or PSF changes—for example, real ADI sequences with injected planets, or at minimum synthetic sequences with evolving speckle backgrounds—and report performance as","section":"§2.2, §3.2, Fig. 5–7"},{"comment":"The semi-synthetic training and test sets are generated with the same injection procedure: same circular orbit, same Gaussian dot model, same SNR-5 prescription. The test therefore measures the model's ability to recognize the exact generative template on which it was trained, not its ability to detect an unknown planet on real data. This is a form of train/test overlap that limits the generalization claim. To make the proof-of-concept convincing, hold out different orbit parameters, brightness levels, or PSF shapes during testing, or apply the model to archival observations with independently confirmed planets. The limitations section already acknowledges the need for more diverse data, but the current abstract and Section 3.2 go beyond what the experiment actually establishes.","section":"§2.2, §3.2"}],"minor_comments":[{"comment":"As presented, Table 1 appears to show only the 'Optimized Median Sub' and 'PCA-KLIP' rows; the CNN-Transformer row is missing or cut off. Please ensure the table includes the proposed model's metrics and clearly labels all columns, including the position-error column.","section":"Table 1"},{"comment":"The position-regression head is trained with a Euclidean distance loss, but the manuscript does not specify the regression target for sequences without a planet. If the loss is computed only for positive samples, state this; otherwise the 10:1 loss weighting is undefined for negative sequences.","section":"§2.3"},{"comment":"The synthetic SNR is quoted as 0.5/0.3 ≈ 1.67 for a single image, but the model sees a 10-frame sequence and the injected dot is coherent across frames. Clarify that the per-image SNR is not the effective detection SNR for the sequence.","section":"§2.1"},{"comment":"The text says the model was trained for '50 or 100 epochs' for the two datasets, but does not specify which number is used for which dataset. Please state the exact training schedule for each experiment.","section":"§2.4, §3"},{"comment":"Reference [5] is listed as 'in preparation.' If the data are from a specific JWST program and public archive, provide a persistent data identifier or MAST DOI so the background frames can be identified and the experiment reproduced.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"This is a well-scoped proof-of-concept manuscript with reproducible code and honest limitations, but the evaluation methodology needs strengthening before it meets the standards of a journal paper. The static-background semi-synthetic test is the main issue: it does not support the abstract's claim of viability on realistic data. I would not reject because the architecture and experimental setup are a defensible starting point, but the authors should either substantially expand the experiments or considerably soften the claims. The paper would also benefit from a clearer statement of what is new relative to existing CNN-based direct-imaging detection tools."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The novel thing here is applying a Transformer encoder to sequences of high-contrast images for exoplanet detection. The architecture is simple—CNN per frame, two self-attention blocks, classification and position regression heads—and the experiments are described clearly. On the purely synthetic dataset, the model hits ~100% accuracy, F1, and sub-pixel position error, and beats median subtraction and PCA-KLIP on that simple test. The limitations section is appropriately modest: it says the datasets use circular orbits, high-SNR planets, and that the model needs retraining for different instruments. That honesty is welcome.\n\nThe problem is that the abstract overreaches. The semi-synthetic dataset in §2.2 uses one real JWST frame of TW Hya as the background and injects the planet into each frame of the sequence. So the background is perfectly static across the 10 frames; the only temporally varying feature is the injected planet. The model can succeed by learning 'anything that moves' rather than by separating a planet from evolving quasi-static speckles. Real high-contrast sequences, especially with angular differential imaging, have time-varying speckles, field rotation, and PSF changes. The claim that this confirms the model can 'function amidst complex, correlated noise and bright disk features' is therefore not supported.\n\nThere are also some methodological gaps. Only one training run is reported, though the text says final metrics varied slightly between runs; no error bars are given. There is no CNN-only baseline, so you cannot tell whether the Transformer is actually adding value over the CNN feature extractor. And the example Jupyter notebook link in §2 is broken—there is no working URL in the text—so the 'demo code' is not accessible.\n\nThese are fixable, and none of them sink the narrow proof-of-concept claim. On the synthetic data with a moving dot on Gaussian noise, the model works. What it does not yet show is viability on real high-contrast sequences with evolving speckle noise. The fix is straightforward: build the semi-synthetic set by injecting the planet into a sequence of distinct real frames (or use real ADI sequences), and add a CNN-only comparison plus repeated-trial statistics.\n\nThis paper is for someone working on ML-based direct imaging who wants a quick look at whether transformer attention over frames is worth pursuing. It's a legitimate first step, stated honestly in the limitations, but it's not a definitive result. I'd send it to peer review because the architecture is worth checking, and the weaknesses are addressable in revision, but the path to acceptance requires the additional experiments above.","headline":"A clean but narrow proof-of-concept for a CNN-Transformer on image sequences; the novelty is real, the semi-synthetic evaluation doesn't test what the abstract claims, and the missing error bars and baseline leave the performance claims soft.","tokens_in":7766,"tokens_out":3631,"would_cite":false,"duration_ms":38449,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A hybrid CNN-Transformer detects moving planets in image sequences with near-perfect accuracy.","keywords":["exoplanet direct imaging","high-contrast imaging","transformer networks","convolutional neural networks","angular differential imaging","planet detection","JWST TW Hya","quasi-static speckles"],"falsifier":"Take an archival ADI sequence of a star with a known or injected faint companion, train the same architecture on realistic moving-speckle simulations, and test on the real sequence; if detection confidence collapses or position error grows well beyond a pixel when the background is allowed to evolve, the claim of practical viability fails.","tokens_in":7023,"feed_emoji":"🪐","tokens_out":4697,"duration_ms":51698,"temperature":0.7,"pith_summary":"The paper argues that exoplanet direct imaging can be recast as sequence classification: instead of subtracting starlight, a hybrid CNN-Transformer network learns the temporal signature of a planet moving coherently across frames. On a simplified synthetic test set, the reported trial achieves 100% accuracy, 100% F1-score, and 0.72-pixel position error, outperforming median subtraction and PCA-KLIP. When retrained on semi-synthetic JWST images of TW Hya with injected planet signals, the model detects the signals with high confidence amid bright disk features and correlated noise. The paper frames this as a proof-of-concept for a fast, automated detection tool that could help vet candidates as JWST and future observatories produce large direct-imaging datasets.","feed_headline":"AI model finds moving exoplanets in high-contrast image sequences","feed_subtitle":"A CNN-Transformer network learns coherent motion, beating median subtraction and PCA-KLIP on test data.","key_machinery":"The key mechanism is the self-attention layer of a Transformer encoder applied across CNN feature vectors extracted from each frame. Because a planet moves along a coherent trajectory while speckles are quasi-static or stochastic, attention assigns higher weight to frames containing the planet's motion, letting the model separate signal from background. The two-headed output, a sigmoid classification head and a coordinate-regression head, turns the temporal model into a direct detection-and-localization tool.","core_discovery":"The central claim is that the coherent motion of a planet across a sequence of high-contrast images is a learnable temporal pattern that a Transformer's self-attention mechanism can isolate from static or stochastic background. A CNN compresses each frame into a 128-element feature vector, a Transformer encoder weighs the frames by relevance, and two output heads classify whether a planet is present and regress its coordinates. On purely synthetic data with moving dots, the model achieves perfect classification and sub-pixel localization in the reported trial. On a semi-synthetic JWST dataset built from a real TW Hya frame with injected 3-pixel Gaussian planets at SNR 5, the model again reac","pith_inferences":["Because the semi-synthetic dataset repeats a single real frame, the hard problem of evolving quasi-static speckles is not yet tested; a natural next experiment is to inject planets into real ADI sequences spanning a range of parallactic angles and measure how accuracy degrades as speckles decorrelate.","The model requires retraining for each noise and instrument setting, so its practical value hinges on developing a generalizable or meta-learning version, a direction the paper itself lists as future work.","The same temporal-attention principle could extend to other moving-signal detection problems in astronomy, such as distinguishing moving solar-system objects or spacecraft streaks from a static background.","The reported 100% figures come from one representative trial and vary between training runs, so a multi-seed distribution of metrics would give a more reliable comparison against classical methods."],"forward_implications":["Detection becomes a sequence-level task: a model can flag a planet and localize it from raw image sequences without constructing a PSF model or subtracting starlight.","The same architecture applies to angular differential imaging sequences, since the image sequence can represent either time-series data or telescope-rotation data.","On simplified synthetic data, the CNN-Transformer beats median subtraction, optimized median subtraction, and PCA-KLIP in recall and F1-score, while PCA-KLIP retains good localization on its own detections.","Semi-synthetic JWST results show the model can work amid bright disk features and correlated noise, at least when the background is static and the model is retrained for the dataset.","Automated sequence-based detection could accelerate candidate vetting in the large datasets expected from JWST and future instruments."],"supporting_citations":[{"why":"Supplies the Transformer self-attention mechanism that the architecture uses to model long-range temporal dependencies across frames.","marker":"4"},{"why":"Prior CNN-based exoplanet detection approach that this work extends by adding a Transformer encoder for sequence modeling.","marker":"3"},{"why":"PCA-KLIP baseline algorithm that the hybrid model is benchmarked against on the synthetic test set.","marker":"2"},{"why":"Source of the real JWST TW Hya frames used to construct the semi-synthetic on-sky dataset.","marker":"5"}],"fun_headline_variants":["Transformer AI spots moving exoplanets in noisy images","CNN-Transformer model detects exoplanet motion in JWST data","AI learns coherent motion to find hidden exoplanets","New hybrid AI beats classic methods for exoplanet imaging","Proof-of-concept: Transformer finds exoplanets in high-contrast data"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The semi-synthetic on-sky test repeats a single real frame, so the background is perfectly static; the model's success there does not show it can handle real sequences where quasi-static speckles evolve over time.","fun_headline_variants_meta":{"raw":{"variants":["Transformer AI spots moving exoplanets in noisy images","CNN-Transformer model detects exoplanet motion in JWST data","AI learns coherent motion to find hidden exoplanets","New hybrid AI beats classic methods for exoplanet imaging","Proof-of-concept: Transformer finds exoplanets in high-contrast data"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000251,"raw_usage":{"total_tokens":1422,"prompt_tokens":798,"completion_tokens":624,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":542,"completion_tokens_details":{"reasoning_tokens":539}},"tokens_in":542,"tokens_out":624,"duration_ms":6403,"temperature":1.0,"reasoning_tokens":539,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T18:28:44.448657+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take an archival ADI sequence of a star with a known or injected faint companion, train the same architecture on realistic moving-speckle simulations, and test on the real sequence; if detection confidence collapses or position error grows well beyond a pixel when the background is allowed to evolve, the claim of practical viability fails.","supporting_citations":[{"cited_title":"Exoplanet detection from starshade images using convolutional neural networks,","cited_arxiv_id":null,"evidence_quote":"Source of the real JWST TW Hya frames used to construct the semi-synthetic on-sky dataset."},{"cited_title":"pyKLIP: PSF Subtraction for Exoplanets and Disks","cited_arxiv_id":null,"evidence_quote":"Supplies the Transformer self-attention mechanism that the architecture uses to model long-range temporal dependencies across frames."},{"cited_title":"Direct Imaging and Spectroscopy of Extrasolar Planets,","cited_arxiv_id":null,"evidence_quote":"Prior CNN-based exoplanet detection approach that this work extends by adding a Transformer encoder for sequence modeling."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"PCA-KLIP baseline algorithm that the hybrid model is benchmarked against on the synthetic test set."}],"review_version":1}