{"id":"05ad8d81-cd9e-4f99-b161-ea254a386961","arxiv_id":"2411.11978","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"STIPS is an open-source Python code that builds realistic Roman Space Telescope WFI images from catalogs, PSFs, backgrounds, and noise, and it is validated to a few percent against Pandeia.","lead":"STIPS is an open-source Python package that simulates realistic images of the Roman Space Telescope's Wide-Field Instrument, including stars, galaxies, backgrounds, and detector noise. It gives the Roman community a fast way to plan observations and test analysis pipelines before the telescope launches.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"PSF interpolation error is unmeasured: validation uses only the SCA center, and the 0.06 mag recovery is self-referential.","rationale":"The reader's weakest assumption was that the pre-launch WebbPSF/IPC models may not represent the real on-orbit Roman ePSF. That is a genuine and unavoidable limitation, but it is external to the paper's internal logic. My concern is internal: even granting the WebbPSF optical models, the paper does not validate the spatial interpolation that is central to its claim of position-dependent PSFs. The validation metrics quoted in the strongest claim do not actually test this. The 1-3% flux agreement with Pandeia is measured at the SCA center, a grid point where interpolation error is zero by construction. The 0.06 mag photometric recovery is obtained by fitting with the same ePSF model used to inject the sources, so it only verifies that the implementation can invert its own forward model. The independent tests (Source Extractor, Astropy Gaussian) show systematic offsets, but these are attributed to the incorrect assumed PSF shapes rather than used to quantify STIPS's absolute accuracy. Thus, the accuracy of off-grid PSFs—a key advertised feature—is unmeasured. This weakness is testable immediately with WebbPSF alone, without waiting for on-orbit data. It does not invalidate the tool's usefulness for many planning purposes, and the paper is clear about excluded effects, so the conditional verdict remains appropriate. The condition should be expanded to require an off-grid PSF interpolation error analysis and an independent PSF recovery test to substantiate the 0.06 mag claim.","tokens_in":9691,"tokens_out":5302,"duration_ms":55246,"concrete_test":"Generate a WebbPSF ePSF at an off-grid position in SCA1, e.g., (1000, 3000), using the same optical model and 4x oversampling. Run the STIPS pipeline to obtain the interpolated ePSF at that position from the nine grid PSFs. Compare the two: report the maximum pixel residual, the integrated flux difference, and the difference in centroids. Then inject a simulated star at that position using the directly computed WebbPSF ePSF, and recover its magnitude and position using the STIPS interpolated ePSF model (as in Fig. 9). If the resulting photometric error exceeds 0.06 mag or the astrometric error exceeds 0.004 pixels, the interpolation is a significant, currently unquantified error source. Repeat the same comparison near the SCA edge and near a corner.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that STIPS can accurately render Roman ePSFs at any detector location rests on the bilinear interpolation of nine PSFs described in §2.1. The validation in §3.1, however, exercises only the position (2048, 2048) in SCA1, which is the SCA center and therefore one of the nine grid points; interpolation error vanishes there by construction. The headline photometric precision (≈0.06 mag, Fig. 9) is obtained by fitting with the same STIPS ePSF model used to inject the sources, so it demonstrates internal consistency of the forward/inverse pair, not agreement with an independent PSF. The Pandeia comparison (Fig. 8) also uses the same center position and only compares total flux, not PSF shape. Consequently, the off-grid PSF error—the difference between the interpolated ePSF and a directly computed WebbPSF ePSF at a non-grid pixel—is never quantified. This matters because the injection algorithm itself uses bilinear interpolation for most pixels, with core deviations up to ~3% (Fig. 4); when the same model is used for recovery, such errors are invisible. The paper's advertised capability to estimate the Roman PSF shape 'at any location within the WFI' therefore lacks any direct evidence at off-center locations.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents STIPS, an open-source Python simulator for generating post-pipeline images of the Nancy Grace Roman Space Telescope Wide-Field Instrument. The simulation pipeline is described in detail: effective PSFs are produced with WebbPSF on a 4x-oversampled nine-point grid per SCA and interpolated bilinearly; point sources are injected using a hybrid bicubic/bilinear sub-pixel interpolation; extended sources are generated with an Astropy Sersic2D model or Pandeia's SersicDistribution; background and noise sources include zodiacal/internal background, Poisson noise, readout noise, dark and flat residuals, and cosmic rays. Validation in Section 3 compares STIPS to Pandeia for total flux in all eight filters at the SCA center and performs source-recovery tests for photometry and astrometry with three fitting methods. The paper emphasizes speed (~20x faster than the full romanisim simulator) and the ability to simulate any number of SCAs up to the full 18-SCA array.","tokens_in":9911,"tokens_out":6910,"duration_ms":60779,"significance":"The tool fills a practical niche: it is faster than full up-the-ramp simulators and more flexible in field size than Pandeia, and the open-source code is valuable for planning Roman observations. The comparison to Pandeia provides a useful independent check of total flux at the ~1-3% level for bright sources, and the paper is honest about known omissions (saturation, non-linearity, distortion, diffraction-spike truncation). However, the validation is incomplete in two specific ways: the Pandeia comparison and the recovery tests are performed only at the SCA center, which is a grid point where interpolation error vanishes, and the most precise recovery results are obtained by fitting with the same ePSF model used for injection. These issues do not undermine the core usefulness of the simulator, but they need to be addressed before the claimed capabilities are fully established.","major_comments":[{"comment":"All validation against Pandeia is performed at the SCA center (2048, 2048), which by construction is one of the nine ePSF grid points used in the bilinear interpolation of Section 2.1; at this location the interpolation error vanishes, so the presented flux comparison does not test the position-dependent ePSF interpolation that the paper advertises in Section 1 (\"estimate the shape of the Roman PSF at any location within the WFI\"). I request an additional test that compares the interpolated ePSF at one or more non-grid positions (e.g., near a corner or between grid points) against a directly computed WebbPSF ePSF at the same positions, quantifying both flux and shape residuals as a function of location.","section":"Section 3.1, Figures 7-8"},{"comment":"The recovery tests labelled \"STIPS ePSF Model\" use the same ePSF model to inject and to fit the sources; the resulting photometric accuracy of ~0.06 mag and astrometric accuracy of ~0.4 mas therefore reflect internal consistency of the code rather than agreement with an independent PSF. This should be stated explicitly in the text. To make the test more informative, the authors should also recover sources with an independently generated PSF (e.g., a WebbPSF ePSF computed directly at the source position, bypassing the STIPS interpolation) and report the resulting residuals.","section":"Section 3.2, Figures 9-10"},{"comment":"The text states that the STIPS fluxes deviate by less than about 1% for bright non-saturated sources, while the caption states that the difference is about 3%. These numbers are inconsistent, and because the Pandeia comparison is the main independent validation in the paper, the discrepancy must be resolved.","section":"Section 3.1, text versus Figure 8 caption"}],"minor_comments":[{"comment":"The IPC kernel matrix has inconsistent spacing in several entries (e.g., \"1 .62\" and \"0 .20\"), which appears to be a LaTeX typesetting error; please fix.","section":"Equation (1)"},{"comment":"\"near the code of the ePSF\" should be \"near the core of the ePSF\".","section":"Figure 3 caption"},{"comment":"\"less then\" should be \"less than\".","section":"Section 3.1"},{"comment":"\"eSPF\" is a typo for \"ePSF\".","section":"Figures 9 and 10 captions"},{"comment":"The cosmic ray rate is given as \"5 hits /cm2/s\"; use standard units of hits cm^-2 s^-1.","section":"Section 2.4"},{"comment":"The GitHub repository is cited by URL without a version or DOI; to support reproducibility, please add a software version (e.g., Zenodo DOI or a specific release tag).","section":"Section 4"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is within the scope of the journal and the tool is likely to be widely used. The main revision should focus on adding off-grid validation and clearly labelling the recovery tests as internal-consistency checks. The inconsistency between the text and Figure 8 caption should be fixed before resubmission."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things. First, STIPS is a real, working tool: it generates Roman WFI scenes fast, ships as open source, and the paper is unusually honest about what it does and does not include. Second, the paper's advertised ability to estimate the Roman PSF 'at any location within the WFI' is never actually validated. The stress-test note is right, and it lands on a real gap.\n\nWhat's new and what works: STIPS synthesizes WebbPSF, Pandeia, Astropy, and Source Extractor into a single package that can simulate the full 18-SCA mosaic, inject point and extended sources, and add noise, darks, flats, and cosmic rays. The speed advantage over romanisim/GalSim is clearly stated and plausibly real. The paper also does the right thing by listing excluded effects (saturation, non-linearity, distortion) and by giving concrete error numbers for the quick versus precise extended-source methods. That is the kind of calibration users need. The Pandeia flux comparison, while limited to one field point, is a sensible external check and shows agreement around 1-3% for bright sources.\n\nWhere it goes soft: the validation in Section 3.2 recovers injected sources with the same STIPS ePSF model that created the image. That demonstrates internal consistency of the forward/inverse pair, not accuracy of the PSF model. And because the Pandeia comparison is pinned to the SCA center (2048,2048), which is one of the nine grid points, the bilinear interpolation between ePSFs is never tested against an independent PSF. So the claim that STIPS can render the Roman ePSF at arbitrary off-grid positions is completely unsupported by the measurements in the paper. This is not fatal for the tool's intended use—rapid scene generation for planning—but it is a real gap in the advertised capability, and the 0.06 mag headline number should be read as a self-consistency check, not an accuracy statement. The cosmic ray model is also crude, but that is minor and clearly described.\n\nWho this is for: anyone planning Roman observations or testing pipelines before launch. The code is public, the documentation is linked, and the paper gives enough detail to use the tool without digging through source. A serious referee should engage with it; a revision that adds an off-grid PSF comparison (e.g., interpolated versus directly computed WebbPSF ePSF at a non-grid pixel) would materially strengthen the claims. I'd bring it to a reading group if the group cares about Roman preparation, and I'd cite it if I were doing simulation-based Roman work.","headline":"A genuinely useful Roman simulator with an honest write-up, but the headline PSF accuracy is not actually demonstrated off the interpolation grid.","tokens_in":10508,"tokens_out":1738,"would_cite":true,"duration_ms":20384,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper presents STIPS, a fast simulator that generates post-pipeline Nancy Grace Roman WFI images of any number of detectors (up to all 18) with point-source flux within about 1–3 percent of Pandeia and photometric recovery at the 0.06…","keywords":["Roman Space Telescope","Wide-Field Instrument","image simulation","effective point spread function","Pandeia","survey planning","detector noise","astronomical software"],"falsifier":"Once Roman is on orbit, compare STIPS predictions to real WFI images of a well-measured star field: fit the STIPS ePSF model to actual sources and check whether recovered magnitudes and positions agree with independent photometry/astrometry, and whether the observed PSF shape across a detector matches the nine-point interpolation. A mismatch in the core shape, focus, or interpixel capacitance beyond the claimed few-percent level would invalidate the simulator's accuracy for real data.","tokens_in":9456,"feed_emoji":"🔭","tokens_out":8899,"duration_ms":74262,"temperature":0.7,"pith_summary":"STIPS is a Python package designed to quickly produce realistic, post-pipeline images of the Nancy Grace Roman Space Telescope's Wide-Field Instrument, covering any subset of the 18 detector chips. The paper argues that the simulator is accurate enough for survey planning: point-source fluxes agree with the reference tool Pandeia at the 1–3 percent level for bright non-saturated stars, and a user who fits STIPS's own effective PSF model recovers injected magnitudes to about 0.06 mag and positions to about 0.004 pixels. The package is meant to complement, not replace, high-accuracy exposure time calculators: it trades some physical detail (no saturation, non-linearity, or distortion) for roughly twentyfold speed, enabling rapid exploration of pointings, filters, and source parameters. If the claim holds, it gives the Roman community a practical way to test observing strategies and PSF-dependent science before launch.","feed_headline":"Fast Roman image simulator matches reference fluxes to ~1 percent","feed_subtitle":"Survey-planning tool builds full 18-detector Roman scenes with realistic noise and point-spread variation before launch.","key_machinery":"The central object is the effective point spread function (ePSF): the convolution of the instrumental PSF with the detector's pixel-response function, integrated over pixels. STIPS builds a grid of nine WebbPSF PSFs per detector at 4x oversampling, convolves them with the interpixel-capacitance kernel, and uses bilinear interpolation to estimate the ePSF at any requested position; bicubic interpolation is used for the four core pixels when injecting a source. This position-dependent ePSF, injected at subpixel coordinates and scaled by filter zeropoints and background, is what carries the fidelity of both photometry and astrometry in the simulated scenes.","core_discovery":"The central claim is that STIPS can synthesize post-pipeline Wide-Field Instrument images of Roman that reproduce the key features a planner needs—point spread function shape, extended-source morphology, background, and noise—at speeds that make large survey simulations practical. Point sources are injected as WebbPSF-generated PSFs that have been convolved with the detector pixel response including interpixel capacitance, then interpolated from a nine-point grid per detector to account for the PSF's position dependence; extended sources are drawn from Sersic profiles generated either by Astropy or by the Pandeia engine. Validation against Pandeia shows total flux agreement better than about 1 percent for bright unsaturated sources, degrading above ~10 percent only for very faint sources where background dominates; and in end-to-end tests, fitting the same ePSF model used to create the scenes recovers injected magnitudes with a mean offset of 0.0027 ± 0.0602 mag and positions to 0.004 pixels. The paper is explicit about the trade-off: STIPS omits saturation, non-linearity, and distortion, so it is a rapid scene simulator and PSF-shape explorer rather than a substitute for a high-fidelity exposure time calculator.","pith_inferences":["If the on-orbit Roman PSF differs from the pre-launch model, the validation numbers reported here bound only the internal consistency of the simulator, not its accuracy against flight data; a natural extension is to recalibrate the ePSF grid once commissioning images are available.","The nine-point bilinear PSF interpolation could be tested against denser PSF grids or against in-flight PSF maps to quantify the residual error introduced by spatial interpolation across a detector.","The same scene-generation pipeline could be adapted to other wide-field instruments by swapping the PSF and pixel-response inputs, so the architecture is a template for post-pipeline simulators generally.","Because STIPS omits distortion, saturation, and non-linearity, its images should be used to develop and test algorithms that are robust to those effects rather than to validate calibration pipelines that depend on them."],"forward_implications":["Survey planning can be done at scale: full 18-SCA scenes with thousands of sources can be simulated in about one-twentieth the time of more exact simulators, making parameter-space exploration feasible.","Users can study PSF variation across the WFI: STIPS interpolates the ePSF across each detector, enabling tests of how PSF shape differences affect photometry and astrometry between SCAs.","The ~0.06 mag recovery when fitting the STIPS ePSF model demonstrates that, under ideal simulated conditions, the package's own model is self-consistent as a photometric reference.","Comparisons against Pandeia show that for bright unsaturated sources STIPS flux is within ~1–3%, making it suitable for exposure time estimates at the planning level, though not as a high-accuracy ETC.","The inclusion of Poisson noise, readout noise, flat/dark residuals, cosmic rays, and zodiacal/internal background means the output images carry realistic noise statistics for feasibility studies."],"supporting_citations":[{"why":"Supplies the WebbPSF package that generates the instrumental PSFs from which STIPS constructs each ePSF grid.","marker":"Perrin et al. 2012"},{"why":"Provides Pandeia, the reference benchmark for flux comparisons and the source of Sersic profile and background estimates used in extended-source injection.","marker":"Pontoppidan et al. 2016"},{"why":"Defines the effective PSF concept that STIPS models by convolving the instrumental PSF with the pixel-response function.","marker":"Anderson & King 2000"},{"why":"Offers the JWST-based zodiacal background model used as the default constant background in STIPS scenes.","marker":"Rigby et al. 2023"},{"why":"Supplies the internal telescope background spectra used by STIPS in addition to zodiacal light.","marker":"Ryan & MacKenty 2023"}],"fun_headline_variants":["Open-source Roman image simulator hits 1% flux accuracy","Roman simulator builds full 18-detector scenes with realistic noise","Python tool simulates Roman WFI scenes for survey planning","Fast Roman scene simulator validated to ~1% flux precision","Simulate Roman's 18 detectors with noise and PSF variation"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reliability of every simulated scene rests on the assumption that the WebbPSF point spread functions, computed from GSFC optical models at five field points per detector and combined with the 2019 interpixel-capacitance kernel, faithfully represent the true on-orbit Roman ePSF; no real Roman images yet exist to test this.","fun_headline_variants_meta":{"raw":{"variants":["Open-source Roman image simulator hits 1% flux accuracy","Roman simulator builds full 18-detector scenes with realistic noise","Python tool simulates Roman WFI scenes for survey planning","Fast Roman scene simulator validated to ~1% flux precision","Simulate Roman's 18 detectors with noise and PSF variation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000537,"raw_usage":{"total_tokens":2581,"prompt_tokens":948,"completion_tokens":1633,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":564,"completion_tokens_details":{"reasoning_tokens":1549}},"tokens_in":564,"tokens_out":1633,"duration_ms":11902,"temperature":1.0,"reasoning_tokens":1549,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T18:02:23.295599+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Once Roman is on orbit, compare STIPS predictions to real WFI images of a well-measured star field: fit the STIPS ePSF model to actual sources and check whether recovered magnitudes and positions agree with independent photometry/astrometry, and whether the observed PSF shape across a detector matches the nine-point interpolation. A mismatch in the core shape, focus, or interpixel capacitance beyond the claimed few-percent level would invalidate the simulator's accuracy for real data.","supporting_citations":[{"cited_title":"M., Pickering, T","cited_arxiv_id":null,"evidence_quote":"Provides Pandeia, the reference benchmark for flux comparisons and the source of Sersic profile and background estimates used in extended-source injection."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the effective PSF concept that STIPS models by convolving the instrumental PSF with the pixel-response function."},{"cited_title":"R., Lightsey, P","cited_arxiv_id":null,"evidence_quote":"Offers the JWST-based zodiacal background model used as the default constant background in STIPS scenes."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the internal telescope background spectra used by STIPS in addition to zodiacal light."}],"review_version":1}