{"id":"d37bca5c-b2c7-43d6-a9d0-8035e59b6ae5","arxiv_id":"2608.03728","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"An edge-agent system on a COTS ARM SoC repeatedly completed two fixed FAIR1M detection workflows 20/20 times, with detector time only about 2.5% to 2.9% of full-agent latency.","lead":"This paper reports a ground-based, hardware-in-the-loop test of an AI orchestration stack for satellite edge intelligence, combining a local language service with an oriented object detector on a COTS ARM edge SoC. It is a carefully scoped systems and reproducibility study that explicitly does not claim detector accuracy, geolocation, flight readiness, or calibrated energy efficiency.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported latency metrics rest on timing records from a pipeline that also produced a systematic factor-of-two defect; independent verification of the sanitized CSVs is needed before relying on the detector-share conclusion.","rationale":"The reader's verdict is CONDITIONAL with moderate confidence, based on the lack of end-to-end reproducibility (private board, weights, LLM) and thin workload coverage. My review focuses on a more specific, previously unhighlighted risk: the manuscript itself documents a systematic factor-of-two timing aggregation defect in the component CSV, which is excluded from the reported numbers. This demonstrates that the measurement pipeline was not flawless. The reported YOLO-tool and Full-Agent latency metrics appear to come from different sources (tool response perf.total_ms and SSE done event) and are cross-validated by a separate profiler, which is good. However, because the sanitized CSVs cannot be independently re-run, the central claim about detector share could be compromised if a similar bug affected the CSV generation. This does not change the overall verdict: CONDITIONAL remains appropriate, but the condition should explicitly include independent verification of the timing records (e.g., raw log cross-check) in addition to broader reproducibility. The paper is otherwise scrupulous about claim boundaries, explicitly disclaiming detector accuracy, geolocation, energy efficiency, and flight readiness, and providing public artifacts. No internal inconsistency or overclaiming was found beyond the timing-pipeline concern. Therefore I do not recommend rejecting or altering the verdict; I recommend keeping CONDITIONAL with the added instrumentation check.","tokens_in":12316,"tokens_out":10625,"duration_ms":112721,"concrete_test":"Request the authors to release the raw unredacted request-level timing log for one single-image run and one serial two-image run, including timestamps for the agent receiving /api/v1/chat/stream, the detector HTTP request send/receive, and the SSE done event. Independently compute Full-Agent latency and detector duration from those raw logs and compare with the corresponding entries in the public fixed_workload_runs.csv and visible_response_timing.csv. If the public CSV entries match the raw logs to within, say, 1% for all checked fields, the sanitization is faithful and the reported detector shares are not artifacts of the aggregation defect. If they diverge by more than that, the reported latency statistics need re-estimation.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central numerical claim is that detector execution is only 2.93% and 2.48% of mean Full-Agent latency. This rests on trusting the timing values in the public CSVs and the tool-reported perf.total_ms. The authors themselves report in Section V: 'Audit against the raw serial-batch captures found a systematic factor-of-two aggregation defect, so those fields are excluded from the manuscript.' This is an explicit admission that the measurement pipeline contained a systematic timing bug. Although the reported YOLO-tool means (861.386 ms, 1510.920 ms) are cross-validated by the profiler-enabled detector-total means (0.857 s, 1.506 s), the profiler run set is separate and also not independently rerunnable because the exact board, private detector weights, and local LLM are withheld. The most load-bearing assumption is that the sanitized request-level CSV records faithfully preserve the timing values without similar aggregation or rounding defects. If the CSV timing values contain a systematic error, the headline detector-share percentages and the P95 values would be wrong. Since the exact hardware and private weights are not public, no reader can currently verify the timing records by re-running the experiment; recalculating from the supplied CSVs would reproduce any embedded errors. The factor-of-two defect found in the same measurement campaign makes this more than a hypothetical concern.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"SAT-Edge-Agent presents a hardware-in-the-loop (HIL) edge-agent architecture for onboard satellite intelligence: a FastAPI agent coordinates a local OpenAI-compatible language service and a project-internal YOLO-style oriented-object-detection endpoint running on a COTS ARM-based heterogeneous edge SoC. The paper reports two fixed FAIR1M workloads (single image and serial two images) repeated 20 times each, with 20/20 completions; mean Full-Agent latencies of 29.353 s and 60.937 s; mean detector times of 861.386 ms and 1510.920 ms, corresponding to 2.93% and 2.48% of the Full-Agent means; plus CPU/NPU telemetry and a profiler-enabled visible-response timeline. A public evidence package contains sanitized CSVs, normalized SSE examples, and recalculation scripts. The authors carefully restrict the claims: no detector accuracy, no new geolocation method, no calibrated energy efficiency, and no flight readiness. The separate plug-meter pilot is explicitly relegated to Appendix A as board-level context.","tokens_in":12470,"tokens_out":9969,"duration_ms":112134,"significance":"If the measurement is accepted, the paper offers a useful systems-level reference point: it demonstrates that a mission-facing agent workflow combining a local LLM and a vision tool can run end-to-end on a low-power COTS board, and that detector execution is a small fraction of end-to-end latency. The paper's care is a genuine strength: statistical definitions are explicit (nearest-rank P95, n-1 sample SD, P99 omitted), the NPU 100% field is correctly labeled as a shared-devfreq software field, the timing-defect disclosure is transparent, and the reproducibility matrix separates public, replaceable, and private layers. These practices make the result assessable even though the exact board, private weights, and raw logs are withheld.","major_comments":[{"comment":"The headline detector-share result (2.93% and 2.48%) rests on `perf.total_ms` and the sanitized request-level CSV. The manuscript discloses a 'systematic factor-of-two aggregation defect' in the component CSV and excludes those fields, but it does not state whether the defect is confined to the excluded substage fields or could also affect `perf.total_ms` / the public `fixed_workload_runs.csv`. Because the exact board and private weights are withheld, no reader can re-run the experiment; therefore the retained timing values need an explicit audit chain. The Table 7 profiler cross-check is a useful sanity check, but it is a separate run set and still relies on the same tool-reported timing. Please provide a precise statement of (i) which fields were affected, (ii) the root cause, and (iii) how `perf.total_ms` and the request-level CSV were validated against raw SSE/network captures, and i","section":"Sec. V, 'Validated Visible-Response Timeline' and Table 7"},{"comment":"The phrase 'reproducible HIL boundary' is stronger than what the shipped evidence supports. The public package allows recalculation of summary statistics from sanitized CSVs, but because timestamps and request identifiers are removed (Appendix A) and the exact board, model identity, and weights are withheld, the experimental measurements themselves are not independently reproducible or re-runnable. This is a legitimate and often necessary boundary, and the paper partially acknowledges it, but the wording should be tightened to 'statistics-recalculation reproducibility' or 'contract-based reproducibility' in the abstract and conclusion. The experimental claims can stand; this is a claim-boundary clarification.","section":"Abstract / Section VIII / Appendix A"}],"minor_comments":[{"comment":"State explicitly that the profiler run set is distinct from the repeated-workload set in Table 6, and explain why the Full-Agent means differ slightly (29.713 s vs 29.353 s; 62.362 s vs 60.937 s). The current text says 'separate' but could be clearer about the relationship.","section":"Table 7"},{"comment":"Clarify the relationship between the serial workload's 20/20 completion and the 19 all-images-successful + 1 partial-result composition of the profiler set. A reader may momentarily interpret 20/20 as 20 fully successful two-image runs.","section":"Tables 6 and 7"},{"comment":"The panels use independent axes, which is appropriate, but the captions should more prominently state that the detector-time panels are nested and in milliseconds, not additive segments of the Full-Agent bars.","section":"Figures 3 and 4"},{"comment":"The paper already clarifies that YOLO26 is a project-internal service label; consider adding a footnote at first mention to prevent misreading as a public detector family.","section":"Sec. III, first mention of YOLO26"}],"recommendation":"major_revision","confidential_remarks":"The paper is unusually honest about its limitations and measurement boundary. The main issue is the timing audit: the factor-of-two defect is disclosed and the excluded fields are not the headline metric, so this is not a fatal flaw, but the current text does not fully rule out contamination of the retained timing values. Once the authors provide a precise audit statement and tighten the reproducibility wording, the paper would be acceptable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a carefully scoped hardware-in-the-loop engineering report, not a scientific breakthrough. The genuinely new thing is the measured end-to-end latency decomposition for an agent-orchestrated detector-plus-local-LLM workflow on a COTS ARM SoC, with a public sanitized evidence package that lets anyone recalculate the reported statistics. The paper is unusually honest about what it does and does not claim: detector accuracy, geolocation, energy calibration, and flight readiness are all explicitly disclaimed, and the NPU 100% field is correctly labeled as a shared-accelerator software field. The n=20 sample size and the omission of P99 are acknowledged. That honesty is real credit.\n\nThe central result—detector time is roughly 2.5–3% of full-agent latency, so most latency lives in orchestration and response formation—is believable. The detector totals are cross-validated by a separate profiler run set (0.857 s and 1.506 s), and the main full-agent means come straight from the CSVs. So the conclusion is about as well-supported as a two-workload, n=20 study can be.\n\nWhere are the soft spots? First, the evidence is not independently re-runnable: the exact board, private detector weights, and local LLM identity are withheld. The public package lets you recalculate, not reproduce. Second, the deployment is a ground test on one board with two fixed images; any transfer to an RF or flight path is speculative. Third, the paper discloses a systematic factor-of-two aggregation defect in its own serial-batch timing fields. That is good transparency, but it also means the reader has to trust that the surviving CSV timing fields were audited correctly. The profiler cross-check helps, but a reader cannot verify the CSV records without the raw captures. This is a real caution, though not a fatal flaw—the paper does not over-rely on the suspect fields, and the defect is disclosed rather than hidden.\n\nThe paper is not ground-shaking, but it is a solid, reproducible reference point for a niche that is getting more attention. It deserves a serious referee, and an editor should send it out. I would want the referee to push for more detail on the audit of the CSV pipeline and maybe one additional workload, but as is, it is a reasonable conditional accept.","headline":"A narrow but honest HIL engineering report; the latency decomposition is plausible, the evidence package is a plus, and the withheld components make it non-reproducible end-to-end.","tokens_in":13089,"tokens_out":2344,"would_cite":true,"duration_ms":25173,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"An observable edge-agent workflow on a COTS ARM SoC turns FAIR1M images into structured mission artifacts, with detector time just 2-3 percent of end-to-end latency.","keywords":["satellite edge computing","hardware-in-the-loop","edge agent orchestration","onboard intelligence","COTS edge SoC","local LLM service","oriented object detection","FAIR1M"],"falsifier":"Run the released recalculation script on the sanitized request-level CSVs and visible-response CSV: if the 20/20 completion counts, the 29.353 s and 60.937 s means, or the 2.93% and 2.48% detector shares do not reproduce, the reproducibility claim fails. Separately, substitute any compatible OBB detector and OpenAI-compatible local language service on a comparable COTS ARM SoC: if the detector share becomes a large fraction of end-to-end latency, or the start→tool→token→done ordering breaks, the orchestration-boundary claim does not transfer.","tokens_in":12060,"feed_emoji":"🛰️","tokens_out":9244,"duration_ms":89539,"temperature":0.7,"pith_summary":"SAT-Edge-Agent asks whether a satellite's onboard computer can do more than run a detector: can it translate a mission request into local tool calls, stream its execution state, and return a structured result a machine can consume, all on a low-power edge processor? The paper's affirmative answer is deliberately bounded to a hardware-in-the-loop demonstration, meaning the software ran on a real commercial ARM-based edge chip rather than in simulation. Two fixed FAIR1M remote-sensing workloads—one single-image request and one serial two-image request—completed 20 of 20 attempts each. Mean end-to-end latency was 29.353 seconds and 60.937 seconds, while detector execution took only 2.93% and 2.48% of those totals, so the visible bottleneck is orchestration and response formation rather than detection. If true, onboard satellite computers can produce structured mission products before downlink, and future engineering should target the agent and response path rather than detector speed.","feed_headline":"Detector takes under 3% of satellite edge-agent latency","feed_subtitle":"Forty hardware-in-the-loop trials on a COTS ARM board show orchestration, not detection, dominates the 29-61 s times.","key_machinery":"The load-bearing mechanism is the event contract start→tool→token→done over Server-Sent Events, paired with the strict separation between the machine-facing detector artifact and the optional operator-facing natural-language summary. The FastAPI agent backend invokes a project-internal YOLO-style oriented-object-detection endpoint—detection of rotated objects in aerial imagery—plus a local OpenAI-compatible language service on the same COTS ARM SoC, and the repeated fixed-workload protocol samples CPU and NPU telemetry every 200 ms. This contract is what makes the orchestration observable, and it is why the 2-3% detector share is meaningful: detector time is nested inside the full-agent wind","core_discovery":"The central claim is that a reproducible hardware-in-the-loop boundary for satellite edge-agent orchestration can be established on a COTS ARM-based heterogeneous edge SoC, with the local LLM-driven agent as an engineering control layer rather than a flight-ready mission planner. In 20 repeated runs of each of two fixed FAIR1M workloads, all 40 attempts completed, and the detector stage averaged 861 ms and 1511 ms—2.93% and 2.48% of the 29.353 s and 60.937 s Full-Agent means. Profiler-validated timelines show most user-visible latency lies outside detector execution, both before and after the first visible token. The paper repeatedly cautions that the results do not validate detector accurac","pith_inferences":["Editorial inference: if the 2-3% detector share holds under a schema-only mode that skips narrative generation, end-to-end latency could drop by most of the 16-29 seconds spent before the first visible token; the paper does not measure this ablation but its architecture makes it directly testable.","Editorial inference: since the NPU is shared between the detector and the local language model, a multi-request or concurrent workload could shift the latency composition substantially; the fixed serial two-image design does not exercise this contention.","Editorial inference: the real stress test for the edge claim is the transition from a localhost FastAPI/SSE path to an RF or thin-client path; if a future radio link becomes the bottleneck, the orchestration-layer latency decomposition measured here would change."],"forward_implications":["For the two measured workloads, speeding up the detector alone cannot meaningfully cut end-to-end latency; the optimization target is the orchestration and response-formation path.","A machine consumer can take the structured tool event before the natural-language summary is generated, so the LLM narrative can be shortened, deferred, or replaced without changing the detection contract.","The system's structured partial-failure behavior—one missing image yields success_count=1, failure_count=1 and still emits done—gives a concrete contract for degraded mission results.","The public sanitized CSVs, SSE examples, and recalculation script allow an independent reader to verify the 20/20 completion counts, means, and P95 values without access to the private weights or exact board.","Because the local LLM and detector share the accelerator, the 100% NPU-load field should be read as shared-accelerator occupancy, not detector-only utilization; the paper treats these as different quantities."],"supporting_citations":[{"why":"Supplies the FAIR1M images, oriented boxes, and metadata-backed geographic fields that define the two fixed workloads and the structured output schema.","marker":"[19]"},{"why":"Establishes the onboard-deep-learning precedent that motivates running interpretation near the sensor.","marker":"[1]"},{"why":"Adds a cloud-detection onboard-inference precedent against which the workflow-level contribution is positioned.","marker":"[4]"},{"why":"Frames orbital nodes as edge computers, the systems perspective the agent orchestration layer extends.","marker":"[7]"},{"why":"Supplies the interleaved reasoning-and-acting agent pattern the FastAPI workflow uses to decide when to invoke the detector.","marker":"[14]"},{"why":"Presents a neighboring onboard multi-agent routing architecture whose approach is contrasted with the local-tool orchestration boundary.","marker":"[17]"},{"why":"Supplies processor-level benchmarking discipline for edge AI chips, used to explain why the results should not be read as a cross-hardware speedup.","marker":"[12]"}],"fun_headline_variants":["Detector: <3% of edge-agent latency in 40 HIL runs","Satellite edge agent: detector idle, orchestration dominates","40 HIL trials: detector barely slows satellite agent","COTS ARM board: detector ~3% of 29-61 s agent time"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The claim collapses if a localhost FastAPI/SSE run on this particular COTS board, with its private detector weights and local language model, is not a faithful stand-in for a real satellite edge-agent path—or if the fixed two-image pair is unrepresentative of the mission workload.","fun_headline_variants_meta":{"raw":{"variants":["Detector: <3% of edge-agent latency in 40 HIL runs","Satellite edge agent: detector idle, orchestration dominates","40 HIL trials: detector barely slows satellite agent","COTS ARM board: detector ~3% of 29-61 s agent time"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000239,"raw_usage":{"total_tokens":1413,"prompt_tokens":871,"completion_tokens":542,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":615,"completion_tokens_details":{"reasoning_tokens":474}},"tokens_in":615,"tokens_out":542,"duration_ms":6564,"temperature":1.0,"reasoning_tokens":474,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T13:55:36.755112+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the released recalculation script on the sanitized request-level CSVs and visible-response CSV: if the 20/20 completion counts, the 29.353 s and 60.937 s means, or the 2.93% and 2.48% detector shares do not reproduce, the reproducibility claim fails. Separately, substitute any compatible OBB detector and OpenAI-compatible local language service on a comparable COTS ARM SoC: if the detector share becomes a large fraction of end-to-end latency, or the start→tool→token→done ordering breaks, the orchestration-boundary claim does not transfer.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Establishes the onboard-deep-learning precedent that motivates running interpretation near the sensor."},{"cited_title":"Giuffridaet al., CloudScout: A Deep Neural Network for On-Board Cloud Detection on Hyperspectral Images,Remote Sensing, vol","cited_arxiv_id":null,"evidence_quote":"Adds a cloud-detection onboard-inference precedent against which the workflow-level contribution is positioned."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Presents a neighboring onboard multi-agent routing architecture whose approach is contrasted with the local-tool orchestration boundary."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies processor-level benchmarking discipline for edge AI chips, used to explain why the results should not be read as a cross-hardware speedup."}],"review_version":1}