{"id":"a4a83244-4e13-49de-8070-d149c192c4b4","arxiv_id":"2507.02153","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"low","formal_verification":"none","parameter_count":0,"one_line_summary":"Image Marker is a lightweight Qt-based tool for marking up to nine classes of features in FITS, TIFF, PNG, and JPEG images, logging coordinates to CSV with WCS support.","lead":"This paper describes Image Marker, a free desktop application for quickly marking and categorizing features in scientific images. It is aimed at researchers who must inspect thousands of survey images by eye without the overhead of a crowdsourcing platform.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Central utility claim rests on unverified availability and performance: no repository URL, version identifier, or benchmark appears in the manuscript, so the paper as submitted does not establish that Image Marker actually supports thousands-image workflows.","rationale":"I read the paper as a software announcement whose central claim is that Image Marker exists, is publicly available, and is efficient enough for thousands-image human-inspection workflows. The most load-bearing condition for this claim is that the artifact can actually be installed and that the performance assertion holds. The manuscript provides no URL, no version, no benchmark, and no memory measurement, so neither condition is demonstrated. This is not a mathematical or logical flaw; it is an unverified empirical and availability claim. The reader's weakest_assumption focused on the missing benchmark for efficiency. I agree with that concern, but I think the even more fundamental issue is that the public availability claim itself cannot be checked from the manuscript: without a repository URL or pip version, the existence of the tool is unverifiable. Because the paper is a software note, this is a solvable and concrete problem, not a reason to reject outright. The appropriate verdict remains CONDITIONAL, contingent on the authors providing a stable repository reference and documenting at least one basic performance measurement. I found no internal inconsistency in the feature descriptions, and the bit-depth limitation is explicitly disclosed, so I do not raise it as a separate concern. My disagreement with the reader is only partial because I weight the missing artifact link as equally load-bearing as the missing benchmark, whereas the reader emphasized the performance evidence alone. The proposed concrete test would settle both concerns: attempt installation, then measure wall time and memory on a representative 1,000-image workflow and verify the CSV coordinate output.","tokens_in":5354,"tokens_out":2673,"duration_ms":33852,"concrete_test":"In a clean Python 3.10+ environment, run `pip install imgmarker`; if this fails, verify the GitHub repository named in a revised manuscript and install from the reported release. Then load a directory of 1,000 representative FITS cutouts (e.g., 256x256 pixels), place marks in each of the 9 groups using number keys, and measure end-to-end wall time and peak memory; repeat with 5,000 files to check for scaling or leaks. Finally, compare a subset of saved `*_marks.csv` entries against known pixel and WCS coordinates to confirm the logging pipeline is correct. If the package installs and the 1,000-image marking workflow completes with stable memory and times consistent with 'hundreds of thumbnails in minutes,' the central utility claim is supported; otherwise, the paper should report actual limits.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that Image Marker is a working, lightweight tool enabling efficient marking of thousands of images on personal-scale computers. The support offered is qualitative: screenshots and descriptions of features. The manuscript contains no repository URL, no version or commit identifier, no installation check, and no timing or memory measurement. Section 1.1.1 asserts that 'hundreds of small thumbnail cutouts' can be scanned 'in a matter of minutes,' and the abstract promises 'efficient marking of thousands of images,' but no benchmark, test dataset, or measurement protocol is described. Because this efficiency is the primary differentiator from DS9 and FitsMap, an unmeasured performance failure—for example, Qt rendering slowdown when many files are loaded, memory growth across thousands of images, or GUI blocking during CSV saves—would undercut the central reason to adopt the tool. This is not an internal inconsistency; it is an unsupported empirical assertion. Likewise, the claimed public availability ('pip install imgmarker', 'available on GitHub') is not checkable from the manuscript, so a reviewer cannot confirm the artifact exists at all. The absence of both a runnable artifact and a performance measurement is the load-bearing gap.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents Image Marker, a Python/Qt desktop application for visually inspecting and annotating images in FITS, TIFF, PNG, and JPEG formats. Users can place marks in up to nine groups via numeric keys, edit labels, read pixel and WCS coordinates, save marks to a CSV file, and import external mark files. The application is motivated by two astronomical use cases: identifying brightest cluster galaxies in SPT-3G data and cleaning thumbnail cutouts in Euclid Deep Field South analyses. The central stated value is that Image Marker is a lightweight tool that enables efficient marking of thousands of images on personal-scale computers; the manuscript describes the features and includes screenshots, but it provides no quantitative measurement of performance, no repository URL or version identifier, and no reproducible installation verification.","tokens_in":5431,"tokens_out":4263,"duration_ms":57977,"significance":"If its performance and availability claims were substantiated, Image Marker would fill a useful niche between web-based crowdsourcing platforms and large-image viewers such as DS9 and FitsMap. The feature set is sensible and clearly motivated: multiple mark classes, CSV export, WCS-aware coordinate logging, multi-frame FITS/TIFF support, and import of external mark files are all valuable capabilities. The paper also gives appropriate credit by citing the astronomy software ecosystem on which it builds. However, the contribution rests on an unverified efficiency assertion, and the manuscript as submitted does not allow a reviewer or user to test the tool. The absence of a benchmark, a machine-checked artifact link, or a minimal installation check means the core utility claim is not currently evidence-backed. This is not an internal inconsistency, but it is a load-bearing gap in an otherwise clearly written software description.","major_comments":[{"comment":"The central claim that Image Marker enables 'efficient marking of thousands of images' and that 'hundreds of small thumbnail cutouts' can be scanned 'in a matter of minutes' is unsupported by any measurement. There is no benchmark, timing test, memory usage report, or description of the test environment, and no comparison against DS9 or FitsMap. Because this efficiency is the primary differentiator, the paper should include a minimal benchmark (e.g., load time, render time, mark-saving time, and memory footprint for 100, 1,000, and 10,000 representative images) and should define what is meant by 'personal-scale computers.' Without such evidence, the central utility claim is an assertion rather than a demonstrated result.","section":"Abstract and §1.1.1"},{"comment":"The paper states that Image Marker is 'available on GitHub' and installable via 'pip install imgmarker', but it does not provide a repository URL, version number, commit identifier, or DOI. A reviewer or potential user cannot verify that the artifact exists, install it, or reproduce the described behavior. Add a persistent link and a minimal installation/test section, including the supported Python versions and operating system specifics.","section":"§3"},{"comment":"The paper concedes that images exceeding PyQt bit-depth limits 'will have their bit depth lowered,' but it does not state whether the user is warned, whether the scaling is performed per-image or globally, or what range mapping is applied. For 32-bit FITS images, silent downsampling can affect the visibility of faint sources and therefore the marking task. This should be documented explicitly and, if possible, made user-controllable, since it directly affects the scientific use case described in §1.1.","section":"§2.1"}],"minor_comments":[{"comment":"There is a typo: '9groups' and '1 of 9groups' should read '9 groups' and '1 of 9 groups.'","section":"§2.2"},{"comment":"The abstract limits executables to 'Mac and Windows 11,' while §3 says installation is supported on 'Mac, Linux, and Windows.' Please reconcile these statements.","section":"Abstract and §3"},{"comment":"The description of the CSV output says 'pixel coordinates, WCS coordinates (if applicable), group, label, the name of the image where the mark was placed, and the current date' are saved, but it does not specify column names or the WCS coordinate units. Including an example CSV row would remove ambiguity.","section":"§2.2"},{"comment":"The settings section does not state whether the marks' colors, sizes, and symbol styles are configurable per group; if they are, this should be stated, and if not, it is a limitation worth noting for users who need to distinguish many classes.","section":"§2.4"},{"comment":"The Gaussian blur tool is described only as a 'slider'; specifying the sigma range or showing a sample blurred image would help readers understand its utility for the stated inspection tasks.","section":"§2.3"},{"comment":"The author name 'Ryan W alker' appears with an unintended space; please correct it to 'Ryan Walker.'","section":"Author block"}],"recommendation":"major_revision","confidential_remarks":"The manuscript reads as a software announcement rather than a fully evidenced refereed paper. The core issues—lack of a verifiable code artifact and lack of any performance measurement—are fixable within the scope of a revision. I would not recommend rejection because the described feature set is coherent and the use cases are real, but the revision must add a repository URL/DOI, a benchmark section, and explicit documentation of the bit-depth handling. If the journal's policy allows software notes without quantitative evaluation, this could be resubmitted in a shorter format, but under the current framing the efficiency claim is the central sales point and it remains unsubstantiated."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things about this one. First, it is a well-written software announcement for a genuinely sensible tool: a Qt-based image marker with 9-class marking, WCS-aware CSV logging, mark-file import, and multi-frame FITS/TIFF support. Second, the paper as submitted does not give a reader any way to check that the tool actually exists or that it is efficient at the claimed scale. No repository URL, no version identifier, no commit hash, no install test, no benchmark. That is the whole ballgame for a software paper.\n\nWhat is new and done well: the combination of fast seeking, quick number-key marking, coordinate logging, and imported mark overlays is a reasonable niche that neither FitsMap nor DS9 fills, and the two use cases—SPT-3G BCG identification and Euclid Deep Field South thumbnail cleaning—are concrete and plausible. The text is clear, the UI description is understandable, and the qualitative comparison to existing tools is fair. The authors also responsibly note the bit-depth limitation of PyQt, even if they don't discuss its scientific consequences.\n\nThe soft spots are real but fixable. The central claim—'efficient marking of thousands of images'—is asserted without a single timing or memory measurement. The 'hundreds of thumbnails in minutes' remark in Section 1.1.1 is likewise unsupported. If the tool is slow or unstable in practice, the main reason to prefer it over DS9 or FitsMap disappears. Also, the claimed availability is not checkable: a referee cannot run pip install or visit GitHub without a URL. Finally, the silent bit-depth downsampling for high-bit-depth images could affect measurements in a science workflow; that deserves at least a caveat or a configurable opt-out.\n\nNone of this is fatal. The paper makes no physics claim and fits no circularity concern; it is a utility. For the intended audience—collaborations that need lightweight human inspection without crowdsourcing infrastructure—this could be a useful tool. But the paper should not be accepted as is. It needs the repository link and version, and ideally a small benchmark or a demonstration on a public dataset so that the efficiency claim is grounded.\n\nBottom line: worth engaging with, but only after the authors provide the artifact and some measurement. I would send it to peer review with a conditional verdict, and require those additions before acceptance.","headline":"A clear, honest software note whose central efficiency claim and even the tool's existence are unverifiable from the manuscript as submitted.","tokens_in":6060,"tokens_out":1754,"would_cite":false,"duration_ms":22660,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Image Marker turns image review into a nine-key tagging loop that records pixel and sky coordinates to a CSV.","keywords":["Astronomy software","Classification systems","Galaxy classification","Image inspection","Qt framework","FITS","WCS","Human-in-the-loop"],"falsifier":"Time a complete pass of loading, displaying, and marking 1,000 thumbnail FITS cutouts with one key press each on a standard laptop; if the process takes dramatically longer than the paper's stated rate of 'hundreds ... in a matter of minutes' (for example, more than a few seconds per image), the central efficiency claim is not supported.","tokens_in":5050,"feed_emoji":"📌","tokens_out":6432,"duration_ms":68026,"temperature":0.7,"pith_summary":"Image Marker is a lightweight desktop program for human review and classification of scientific images. With the image on screen, a user presses number keys 1 through 9 to place a mark in one of nine user-defined categories; each mark's pixel coordinates, sky coordinates when a WCS solution is available, label, image file name, and date are written to a CSV file. The tool reads FITS, TIFF, PNG, and JPEG images, supports multi-frame files, and can import mark files from other users. The authors present it as a middle ground between crowdsourcing systems, which require internet access and heavy infrastructure, and general-purpose viewers that are not built for rapid batch marking, and they claim it enables efficient marking of thousands of images on a personal computer.","feed_headline":"Nine keys to tag a thousand survey images","feed_subtitle":"Press 1-9 to place a categorized mark; pixel and sky coordinates log automatically to a CSV.","key_machinery":"The central mechanism is the Qt GUI main window, which maps the number keys 1 through 9 to nine distinct, user-renamable mark groups. Pressing a key drops a mark at the cursor and immediately appends a row to a per-user CSV file containing pixel coordinates, WCS coordinates (if the FITS header or TIFF metadata provides them), the group name, an editable label, the image file name, and the date. Around this core sit supporting tools: multi-frame FITS and TIFF handling, brightness stretch and interval controls with Gaussian blur, and import of external mark files defined by label plus either x/y or RA/Dec columns.","core_discovery":"The paper's contribution is a working, publicly released software tool whose core workflow is a fast keyboard loop: navigate to an image, press a number key to tag a feature, and move on, with all metadata recorded without breaking flow. The central claim is that this Qt-based application is light enough to handle 'thousands of images' on 'personal-scale computers' while still providing the format flexibility (FITS, TIFF, PNG, JPEG), WCS awareness, and shareable mark files that scientific inspection tasks need. Concretely, the paper argues that Image Marker fills the gap between citizen-science platforms and heavyweight viewers, enabling a small collaboration to validate automated selections such as brightest cluster galaxies with human inspection.","pith_inferences":["The paper makes no timing or resource measurements, so a prospective user should benchmark the mark-and-advance loop on their own hardware before trusting the 'hundreds of thumbnails in minutes' claim.","A natural extension the authors do not explore is exporting marks in a format directly usable by the very algorithms being validated, which would turn the tool into a ground-truth generator for centering or classification methods.","The point-mark-only design may limit applicability to tasks needing regions or contours, such as contamination masks or morphological annotation, but adding such shapes appears consistent with the same CSV-logging architecture."],"forward_implications":["A single investigator can label a survey-sized set of cutouts on a laptop, with no internet connection or crowdsourcing platform, if the claimed throughput is genuine.","The CSV output with both pixel and WCS coordinates plugs directly into catalog-based analyses, letting human labels be cross-matched against algorithmic selections like redMaPPer or MCMF.","Because mark files are importable and shareable, several users can contribute to one dataset and their coordinate lists can be compared for consistency.","The customizable groups and image categories transfer the tool beyond astronomy to general image-quality inspection and algorithm validation tasks."],"supporting_citations":[{"why":"Provides the DES Exposure Checker example of crowdsourced inspection, which Image Marker positions itself against as a lighter-weight alternative.","marker":"Melchior et al. 2016"},{"why":"Presents Space Warps as a crowdsourcing example that motivates the paper's argument that such infrastructure is not needed for small collaborations.","marker":"Marshall et al. 2016"},{"why":"Describes Galaxy Zoo as a crowdsourcing case used to set up the contrast with a lightweight local tool.","marker":"Fortson et al. 2012"},{"why":"Documents the Zooniverse framework that the paper notes requires an internet connection, a limitation Image Marker avoids.","marker":"Simpson et al. 2014"},{"why":"Introduces FitsMap, the web client that the paper says lacks fast scanning of many images and coordinate saving, the gap Image Marker fills.","marker":"Hausen & Robertson 2022"},{"why":"Provides SAOImageDS9, the widely used viewer that the paper states handles smaller datasets best.","marker":"Joye 2017"},{"why":"Documents the automatic BCG selection failure rate that motivates human inspection in cluster centering.","marker":"Rozo & Rykoff 2014"}],"fun_headline_variants":["Press 1-9 to tag features across thousands of survey images","Image Marker: keyboard-driven annotation for FITS and more","Tag FITS, PNG, TIFF, and JPEG with nine keyboard shortcuts","One shortcut per feature class: mark thousands of images quickly","Nine-key tagging loop for survey image inspection"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claim that the Qt-based loop can mark thousands of images efficiently on an ordinary computer rests on an unstated performance assumption; the paper gives no benchmarks, so if the tool is slow or unstable in practice the primary reason to adopt it disappears.","fun_headline_variants_meta":{"raw":{"variants":["Press 1-9 to tag features across thousands of survey images","Image Marker: keyboard-driven annotation for FITS and more","Tag FITS, PNG, TIFF, and JPEG with nine keyboard shortcuts","One shortcut per feature class: mark thousands of images quickly","Nine-key tagging loop for survey image inspection"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001534,"raw_usage":{"total_tokens":6107,"prompt_tokens":879,"completion_tokens":5228,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":495,"completion_tokens_details":{"reasoning_tokens":5144}},"tokens_in":495,"tokens_out":5228,"duration_ms":44135,"temperature":1.0,"reasoning_tokens":5144,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T20:36:27.459973+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Time a complete pass of loading, displaying, and marking 1,000 thumbnail FITS cutouts with one key press each on a standard laptop; if the process takes dramatically longer than the paper's stated rate of 'hundreds ... in a matter of minutes' (for example, more than a few seconds per image), the central efficiency claim is not supported.","supporting_citations":[{"cited_title":"2016, Astronomy and Computing, 16, 99, doi: 10.1016/j.ascom.2016.04.003","cited_arxiv_id":null,"evidence_quote":"Provides the DES Exposure Checker example of crowdsourced inspection, which Image Marker positions itself against as a lighter-weight alternative."},{"cited_title":"Galaxy Zoo: Morphological Classification and Citizen Science","cited_arxiv_id":"1104.5513","evidence_quote":"Describes Galaxy Zoo as a crowdsourcing case used to set up the contrast with a lightweight local tool."},{"cited_title":"R., & De Roure, D","cited_arxiv_id":null,"evidence_quote":"Documents the Zooniverse framework that the paper notes requires an internet connection, a limitation Image Marker avoids."}],"review_version":1}