{"id":"10d006c6-d57e-4ff5-8ce7-7f3f7938d601","arxiv_id":"2502.00556","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"PyLMT is a real-time pipeline for the ILMT that finds transients using image subtraction and CNN-based classifiers, achieving a precision of 0.91 and discovering 12 extragalactic transients in the 2023-2024 cycle.","lead":"Astronomers built a pipeline called PyLMT that automatically finds new or changing objects in images from the 4-meter International Liquid Mirror Telescope. It uses image subtraction plus two neural networks to separate real transients from artifacts, and it has already detected 12 extragalactic transients, two of which were previously unknown.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The headline precision of 0.91 is an in-sample estimate from the same 2022 season used to train the CNN classifiers; no operational precision is reported for the 2023-24 survey, so the central metric may not generalize.","rationale":"The reader's weakest assumption correctly identifies that the precision and test accuracies are computed on data from the same observing cycle used to train the CNNs. This is the central load-bearing concern because the paper's headline claim is a quantitative precision of 0.91; if that number is an in-sample artifact, the pipeline's operational utility for follow-up triage is not established. The external TNS detections in the 2023-24 cycle are real evidence of detection capability, but they do not measure precision or false-positive rate, so they cannot substitute for an out-of-sample precision measurement. The additional limitation of single non-coadded reference images (Appendix A) is real but secondary, since the operational detections show the subtraction is adequate for finding bright transients. I also note the NovaNet confusion matrix (Table 8) implies an accuracy of about 97.7 percent rather than the stated 98.6 percent, but this discrepancy affects a supporting classifier, not the central detection-precision claim. The concrete test proposed would settle whether the 0.91 precision generalizes, and the reader's conditional verdict is the appropriate stance until such evidence is provided.","tokens_in":19722,"tokens_out":4012,"duration_ms":40241,"concrete_test":"Select a random sample of 50 science images from the November 2023-May 2024 cycle for which reference images exist, run the full PyLMT pipeline, and have at least two independent human vetters label every generated alert as real or bogus using the same criteria as Section 4.2. Compute the achieved precision and compare it to 0.91. Also report the total number of candidates and false positives over the full 761-frame cycle. If the held-out precision is below approximately 0.85 or the operational false-positive rate is not reported, the headline precision should be treated as unvalidated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The PyLMT precision claim rests entirely on Section 4.2, where the pipeline was run on 30 i-band images from 28-30 October 2022, and every real/bogus CNN (Appendix B) was trained on cutouts from the same October-November 2022 cycle. The validation images therefore share the same PSF, background, artifact morphology, and even specific field locations as the training data; the precision of 0.91 (290 true positives out of 320 detections) is an in-sample performance estimate. The paper reports 12 extragalactic transients in 761 frames from the November 2023-May 2024 cycle, but gives no count of total candidates or false positives in that operational period, so we cannot know whether precision remained 0.91, degraded, or improved. The TNS discoveries demonstrate detection capability but not false-positive rate, which is the quantity that determines follow-up feasibility. Compounding this, Appendix A states that only single non-coadded reference images were used; these have lower signal-to-noise than co-adds and are expected to increase subtraction artifacts. Thus the most load-bearing assumption, that the 2022 test set represents the survey, is unsupported by out-of-sample data.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents PyLMT, a transient-detection pipeline for the 4-m International Liquid Mirror Telescope. The pipeline performs image subtraction (ILMTDiff), source detection and real/bogus classification with CNNs (TransiSearch), and host-morphology classification of candidates (NovaNet), followed by catalogue cross-matching against SIMBAD, Gaia, and IMCCE/SkyBot. The authors report a pipeline precision of 0.91 measured on 30 i-band images from October 2022, real/bogus classifier test accuracies of 93.97% and 98.07%, a NovaNet candidate-classifier accuracy of 98.6%, and 12 extragalactic transients detected in the November 2023–May 2024 cycle, including two previously unknown transients reported to the TNS. The paper also describes asteroid recovery with MPC cross-matching and gives a median execution time of about 29 minutes per frame.","tokens_in":20000,"tokens_out":3301,"duration_ms":30694,"significance":"If the reported performance holds, PyLMT provides a complete, operational transient-detection capability for a new survey telescope and has already produced externally validated discoveries (AT 2023yjc, AT 2024fxn, plus confirmed SNe). The pipeline's use of TDI imaging, segmented subtraction, adaptive real/bogus classification, and candidate classification is a useful contribution to the time-domain literature. The public availability of the underlying ILMT images is a strength. However, the two central performance figures—the 0.91 precision and the 98.6% NovaNet accuracy—are less well supported than the text implies: the precision is an in-sample estimate from the same season used for training, and the NovaNet accuracy is inconsistent with the published confusion matrix.","major_comments":[{"comment":"The reported pipeline precision of 0.91 is computed on 30 i-band images from 28–30 October 2022, and the real/bogus classifiers were trained on cutouts from the same October–November 2022 cycle (Appendix B). This is an in-sample estimate: the validation frames share the PSF, background, artifact morphology, and specific field locations with the training data. The operational 2023–24 cycle is not evaluated for false positives: §4.4 reports 12 extragalactic transients in 761 frames but gives no total candidate count or false-positive rate. Please add an out-of-sample precision measurement—even a small sample of vetted candidates from the 2023–24 cycle—or explicitly state that 0.91 is an in-sample figure and not a demonstrated survey precision.","section":"§4.2"},{"comment":"The claimed classification accuracy of 98.6% does not match the confusion matrix in Table 8. Summing the diagonal gives 14+185+1284 = 1483 correct classifications out of 1518 total, which is 97.7%, not 98.6%. Please correct the stated accuracy or the confusion matrix, and update the abstract, Section 4.3, and Section 5 accordingly.","section":"§4.3, Table 8"},{"comment":"The text states that the TransiSearch module 'extracts all the sources above 4σ significance' (Section 3.2), but Table C1 lists the default detection_threshold as 5 standard deviations above noise. This discrepancy directly affects the reported sensitivity, the precision measurement in Section 4.2, and the interpretation of detections. Please specify which threshold was used for the 30-image validation and for the 2023–24 search.","section":"§3.2 vs Table C1"},{"comment":"The adaptive detection scheme relies on a CNN-based preemptive classification of all sources into 'point-host' and 'non-point-host' before applying the high-precision or high-recall real/bogus classifier. The paper does not specify which CNN performs this split, how it was trained, or what its accuracy is. This is load-bearing because the claimed reduction in false positives (from about 35% to about 10%) depends on this split, yet no evaluation of the split itself is presented. Please identify the classifier and report its training data and performance.","section":"§3.5"}],"minor_comments":[{"comment":"Appendix A states that only non-coadded single reference images were used because limited imaging data was available. This is an acknowledged limitation, but its impact on subtraction artifacts and hence on precision should be quantified or at least discussed explicitly in the validation section.","section":"Appendix A"},{"comment":"The caption of Table 7 refers to 'true positive class distribution', but the table only includes classifications of visually confirmed real sources; it is unclear whether false negatives from TransiSearch are included. Please clarify whether the entries are conditional on the source having been correctly detected.","section":"Table 7"},{"comment":"The reference entry 'Makhlouf, K. Turpin, D. Corre, D. Karpov, S. Kann, D. A. Klotz, A. 2022' has an inconsistent author-name formatting that should be brought into line with the journal style.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper describes a useful and operational pipeline, and the TNS/MPC cross-checks give external grounding to the detection claims. The main issues are the in-sample precision estimate, the accuracy inconsistency in Table 8, and the unsupported description of the adaptive-detection split. These are all fixable within the scope of a revision, so I recommend major revision rather than rejection. Please ensure the authors correct the accuracy calculation and either provide out-of-sample precision or soften the claim accordingly."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know about this paper. First, it describes a working transient pipeline for the ILMT, a 4-m zenith telescope running a one-day-cadence survey, and the pipeline has already produced two TNS-reported discoveries (AT 2023yjc, 2024fxn) plus ten other extragalactic transients in the November 2023–May 2024 cycle. That is real external validation. Second, the headline precision of 0.91 is computed on 30 i-band images from October 2022, the same season used to train the CNNs, so it is an in-sample number. The paper gives no false-positive count for the operational 2023–24 run, so we do not know whether the precision holds up in practice. That is the main soft spot, and it is significant but not fatal.\n\nWhat is actually new here is the integrated pipeline tailored to ILMT's TDI images, particularly the adaptive two-CNN strategy that splits candidates into point-host and non-point-host and applies a high-precision classifier to the former. That reduced false positives from 35% to 10% in the validation set, and it is a sensible design choice. The ILMTDiff subtraction module is a straightforward Bramich implementation, but the comparison with HOTPANTS and other codes is a useful sanity check. The asteroid recovery across magnitude bins, anchored by MPC cross-matches, is a nice concrete validation. The median 29-minute processing time for a 17-minute exposure is also an important operational result.\n\nThe soft spots are real but contained. The NovaNet confusion matrix in Table 8 actually gives about 97.7% accuracy, not the stated 98.6%. The detection threshold is described as 4 sigma in Section 3.2 but the default is 5 sigma in Table C1. Neither error changes the conclusions, but both need fixing. The bigger issue is the missing out-of-sample precision. The authors should report the total number of candidates and false positives from the 2023–24 cycle, even a rough count, to show that 0.91 carries over. Appendix A admits that only single non-coadded reference images were used; that is an honest limitation, and co-added templates should improve subtraction.\n\nWho is this for? Anyone building a transient pipeline for a survey telescope, especially ones with elongated TDI images. It deserves a serious referee: the external discoveries make it publishable, and the in-sample precision issue is addressable in revision. I would send it to review with a request for operational precision metrics and cleanup of the threshold inconsistency.","headline":"A solid, externally validated transient pipeline for ILMT, but the headline precision figure is in-sample and needs out-of-sample confirmation.","tokens_in":20543,"tokens_out":2312,"would_cite":true,"duration_ms":22436,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper presents PyLMT, an automated pipeline that detects transient and variable sources in the nightly images of the 4-m International Liquid Mirror Telescope, using image subtraction and convolutional-neural-network classifiers to…","keywords":["transient detection","image subtraction","convolutional neural networks","real/bogus classification","time-domain astronomy","liquid mirror telescope","supernova discovery","asteroid detection"],"falsifier":"Take a set of ILMT frames from a different observing season or a different filter, run the pipeline exactly as described, and manually vet every alert; if precision on the fresh cycle falls materially below 0.91 or the false-positive fraction rises above the reported 10%, the same-season training assumption is falsified. A narrower check is to repeat the asteroid-recovery test with a coadded reference image and count whether the recovery fraction of 231 out of 293 improves.","tokens_in":19547,"feed_emoji":"🔭","tokens_out":8529,"duration_ms":78779,"temperature":0.7,"pith_summary":"This paper presents PyLMT, an automated pipeline that detects transient and variable sources in the nightly images of the 4-m International Liquid Mirror Telescope. The pipeline subtracts a reference image from each science frame, uses convolutional-neural-network classifiers to separate real astrophysical sources from subtraction artifacts, and then sorts the survivors by whether the underlying host looks like a galaxy, a point source, or nothing. On real ILMT frames the authors report a detection precision of 0.91, test accuracies of 94-98% for the real/bogus classifiers, and 98.6% accuracy for the host-morphology classifier. The pipeline has been running operationally and produced 12 extragalactic transients in one observing cycle, including two previously unknown objects. The broader aim is to show that a small, dedicated survey telescope with limited training data can run real-time transient discovery with off-the-shelf machine-learning tools.","feed_headline":"A 4-m liquid-mirror telescope finds transients at 0.91 precision","feed_subtitle":"PyLMT caught 12 extragalactic transients in 761 frames, including two new discoveries.","key_machinery":"The load-bearing component is the adaptive two-classifier detection strategy. Every source in the subtracted image is first pre-classified as point-host or non-point-host; point-host sources go to a high-precision CNN trained with a deliberately artifact-heavy dataset, while all other sources go to a high-recall CNN. This is what reduces false positives from about 35% to about 10% while keeping faint, extended-host transients visible. Around it sit the image-subtraction module, which fits an optimal convolution kernel to match the point-spread function of the reference frame to the science frame, and the host-morphology ensemble, which combines one 31x31-pixel CNN and two 102x102-pixel CNNs to label candidates as extended-host, point-host, or hostless.","core_discovery":"The central claim is that a three-module pipeline can find and classify transients in ILMT images in near real-time with a precision high enough for follow-up. Image subtraction is handled by a module built on delta-basis kernel optimisation, which matches the point-spread functions of the science and reference frames; transient detection then routes sources through two CNN classifiers, a high-recall one for non-point-host candidates and a high-precision one for point-host candidates, because improperly subtracted stars are the dominant false-positive class. This adaptive split raised precision on 30 i-band frames from 0.65 to 0.91. A second CNN ensemble classifies candidates as extended-host, point-host, or hostless, and catalogue cross-matching rejects known solar-system bodies and bright variable stars. Running on 761 frames from the November 2023-May 2024 cycle, the pipeline detected 12 extragalactic transients, including two new discoveries, AT 2023yjc and 2024fxn, reported to the public transient registry.","pith_inferences":["The reported performance numbers are computed on ILMT data from the same observing cycle used to train the CNNs, so the paper's key assumption is that those same-season samples represent the operational survey; a fresh cycle in a different filter would test this directly.","Because the current reference images are single non-coadded frames, building the planned library of coadded references should improve subtraction residuals and push recall for faint transients higher than the values reported here.","The point-host versus non-point-host routing strategy is tied to ILMT's PSF and its dominant false-positive source; it would be a testable transfer question whether the same two-model scheme improves precision on other survey telescopes without retraining.","If the 12-transient yield from 761 frames persists, ILMT will become a regular supplier of supernova candidates that need only external spectroscopic confirmation, which would expand the sample of early-time transients available for follow-up."],"forward_implications":["A full night of up to 35 ILMT frames can be processed in roughly 17 hours, making the pipeline fast enough to run continuously between nights.","The adaptive point-host/non-point-host split cuts the false-positive share in final alerts from about 35% to about 10% without sacrificing faint non-point-host transients.","Extended-host candidates are prioritized for follow-up, which is where supernovae are most likely to be found; four of the twelve operational detections were confirmed supernovae.","The same pipeline recovers 231 of 293 visually confirmed catalogued asteroids, including magnitudes fainter than 21, so it serves as a solar-system survey as well.","Operational use on 761 frames yielded 12 extragalactic transients, including two previously unreported objects, demonstrating that the pipeline is ready for routine use."],"supporting_citations":[{"why":"Supplies the delta-basis convolution-kernel optimization that ILMTDiff uses to match PSFs before subtraction.","marker":"Bramich (2008)"},{"why":"Provides the find_peaks source detection and median background estimation used by TransiSearch and ILMTDiff.","marker":"Bradley et al. (2016)"},{"why":"Provides the SkyBot service that rejects catalogued solar-system objects within 10 arcsec of each candidate.","marker":"Berthier et al. (2006)"},{"why":"Supplies the magnitudes used to reject point-host candidates brighter than 19 as probable variable stars.","marker":"Gaia Collaboration et al. (2016)"},{"why":"Provides the SIMBAD cross-match that attaches a host name and object type to each candidate alert.","marker":"Egret et al. (1991)"},{"why":"Provides HOTPANTS, the comparison subtraction code used to show ILMTDiff's output quality is comparable.","marker":"Becker (2015)"},{"why":"Reports AT 2023yjc, one of the two new transients discovered operationally by the pipeline.","marker":"Pranshu et al. (2023)"},{"why":"Reports 2024fxn, the other new transient discovered operationally by the pipeline.","marker":"Pranshu et al. (2024b)"}],"fun_headline_variants":["Liquid-mirror telescope pipeline spots transients at 0.91 precision","Automated pipeline finds 12 extragalactic transients in ILMT data","CNN-powered pipeline boosts transient precision from 0.65 to 0.91","PyLMT: Real-time transient detection for 4-m liquid mirror telescope","Two new transients discovered by ILMT pipeline in latest cycle"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the same-season ILMT frames used to train the real/bogus CNNs are representative of the frames the pipeline will see in later observing cycles; if the telescope's seeing, artifacts, or sky background change enough, the 0.91 precision will not transfer.","fun_headline_variants_meta":{"raw":{"variants":["Liquid-mirror telescope pipeline spots transients at 0.91 precision","Automated pipeline finds 12 extragalactic transients in ILMT data","CNN-powered pipeline boosts transient precision from 0.65 to 0.91","PyLMT: Real-time transient detection for 4-m liquid mirror telescope","Two new transients discovered by ILMT pipeline in latest cycle"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000398,"raw_usage":{"total_tokens":2459,"prompt_tokens":1072,"completion_tokens":1387,"prompt_tokens_details":{"cached_tokens":1024},"prompt_cache_hit_tokens":1024,"prompt_cache_miss_tokens":48,"completion_tokens_details":{"reasoning_tokens":1286}},"tokens_in":48,"tokens_out":1387,"duration_ms":30759,"temperature":1.0,"reasoning_tokens":1286,"cache_read_input_tokens":1024,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T18:33:26.719107+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a set of ILMT frames from a different observing season or a different filter, run the pipeline exactly as described, and manually vet every alert; if precision on the fresh cycle falls materially below 0.91 or the false-positive fraction rises above the reported 10%, the same-season training assumption is falsified. A narrower check is to repeat the asteroid-recovery test with a coadded reference image and count whether the recovery fraction of 231 out of 293 improves.","supporting_citations":[{"cited_title":"M., 2008, @doi [Monthly Notices of the Royal Astronomical Society: Letters] 10.1111/j.1745-3933.2008.00464.x , 386, L77","cited_arxiv_id":null,"evidence_quote":"Supplies the delta-basis convolution-kernel optimization that ILMTDiff uses to match PSFs before subtraction."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the find_peaks source detection and median background estimation used by TransiSearch and ILMTDiff."},{"cited_title":"351, Astronomical Data Analysis Software and Systems XV","cited_arxiv_id":null,"evidence_quote":"Provides the SkyBot service that rejects catalogued solar-system objects within 10 arcsec of each candidate."},{"cited_title":"A., Egret D., eds, Astrophysics and Space Science Library Vol","cited_arxiv_id":null,"evidence_quote":"Provides the SIMBAD cross-match that attaches a host name and object type to each candidate alert."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides HOTPANTS, the comparison subtraction code used to show ILMTDiff's output quality is comparable."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Reports AT 2023yjc, one of the two new transients discovered operationally by the pipeline."}],"review_version":1}