{"id":"c3874633-3cc7-46f2-9a7e-713be52909b1","arxiv_id":"2604.04806","paper_version":4,"verdict":"CONDITIONAL","confidence":"LOW","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Online LLM simulation of microservice dependencies achieves 99% status-code and response-shape fidelity versus 62%/16% for record-replay across 110 scenarios.","lead":"MIRAGE answers each microservice dependency call at test time with an LLM that reads source code and traces, instead of replaying fixed stubs. It reports near-perfect status and shape fidelity on scenarios static record-replay misses, at a few seconds and under a dollar per dependency.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"Manuscript mismatch: full text is POSS-I vanishing sources (2604.04810), not MIRAGE (2604.04806); 99% fidelity claim cannot be audited.","rationale":"The reader correctly identified that only the abstract is available for MIRAGE and that the full-text block belongs to another paper, and therefore issued CONDITIONAL with LOW confidence. That is the right diagnosis of the evidence state. Because the full manuscript of the paper under review is simply not present, no technical audit of the 99% fidelity claim, the signal ablation, or the 8/8 integration tests is possible. The appropriate adjustment is therefore from CONDITIONAL to UNVERDICTED until the correct manuscript is supplied. The reader’s weakest_assumption (sufficiency of code+traces for rare paths) remains the right scientific soft spot once the paper appears, but it cannot be stress-tested today. No other load-bearing concern can be raised without inventing content that is not in the provided text.","tokens_in":9103,"tokens_out":538,"duration_ms":5765,"concrete_test":"Replace the CACHEABLE prefix with the actual MIRAGE manuscript (or arXiv PDF 2604.04806). Re-run the fidelity evaluation on the 110 scenarios with the published scenario definitions and the three systems; if the white-box status-code / response-shape numbers remain ≥99% and the 8/8 integration outcomes hold under the same protocol, the claim stands; otherwise the abstract numbers are unsupported.","verdict_should_be":"UNVERDICTED","load_bearing_attack":"The central claim (99% status-code and response-shape fidelity vs 62%/16% record-replay on 110 scenarios; 8/8 integration-test agreement) is stated only in the abstract. The CACHEABLE full-manuscript block is a completely different paper (Hayes, Independent Recovery of Vanishing Sources on POSS-I Photographic Plates, arXiv:2604.04810). No MIRAGE methods, scenario definitions, fidelity metrics, signal-ablation tables, or integration-test protocol appear in the provided full text. Therefore the load-bearing condition for the claim—that production traces + dependency/caller source suffice for rare error-handling and code-reasoning paths, and that the 110 scenarios actually stress those paths—cannot be checked. The reader already flagged this; the mismatch is not a minor artifact gap but a total absence of the paper under review.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"The abstract of MIRAGE claims an online LLM-based approach to microservice dependency simulation that answers each request at runtime while maintaining cross-request state, using dependency source, caller code, and production traces. On 110 scenarios across three systems it reports 99% status-code and 99% response-shape fidelity (vs 62%/16% for record-replay), signal ablations, stability across three LLM families, and 8/8 integration-test parity with real dependencies. The supplied full manuscript text, however, is an unrelated astronomy paper (Hayes, Independent Recovery of Vanishing Sources on POSS-I Photographic Plates, arXiv:2604.04810) describing a POSS-I source-detection pipeline, Solano-catalog cross-match, and a null temporal association with nuclear tests. No MIRAGE methods, metrics, scenarios, or results appear in the body.","tokens_in":9347,"tokens_out":904,"duration_ms":7321,"significance":"If the abstract claims were supported by a complete, auditable manuscript, the work would be a useful contribution to microservice testing: online LLM simulation could address the known coverage gap of static record-replay and stubs on error-handling and code-reasoning paths, with concrete cost/latency trade-offs and multi-system evaluation. The astronomy manuscript that was actually supplied is a competent independent-replication study of VASCO vanishing-source catalogs, but it is not the paper under review and does not advance the MIRAGE claims.","major_comments":[{"comment":"Manuscript identity failure: the paper under review is arXiv:2604.04806 (MIRAGE, cs.SE). The full text provided is arXiv:2604.04810 (Hayes, POSS-I vanishing sources, astro-ph.IM). Every load-bearing claim in the MIRAGE abstract—white-box 99%/99% fidelity on 110 scenarios, record-replay baselines of 62%/16%, signal ablation (source alone 100%; without source 94%/75%), three LLM families within 3%, deterministic repeats, and 8/8 integration parity—has no corresponding methods, tables, figures, or protocols in the body. The central claim cannot be audited.","section":null},{"comment":"Because the body is a different paper, the weakest assumption flagged for MIRAGE (that production traces + dependency/caller source suffice for rare error-handling and code-reasoning paths underrepresented in traces) is untestable. No scenario definitions, fidelity metric formalization (status-code exactness; response-shape), multi-hop state protocol, or ablation tables exist in the supplied text. A referee cannot verify soundness, circularity risk, or generalization.","section":null},{"comment":"Even treating the supplied astronomy text on its own terms, its temporal section (3.4) already acknowledges that the calendar-day RR=1.35 is schedule-dominated (every study-window night has ≥1 candidate) and that the negative-binomial count model is null (IRR=1.03, p=0.71). That paper’s own conclusions correctly limit the nuclear-test claim; it does not rescue or substitute for the missing MIRAGE evaluation.","section":null}],"minor_comments":[{"comment":"Abstract vs body mismatch is total; no amount of local editing of the astronomy manuscript can produce a reviewable MIRAGE paper.","section":null},{"comment":"If a correct MIRAGE full text is later supplied, standard SE-review items would include: precise definition of response-shape fidelity, scenario selection criteria (especially error/code-reasoning coverage), prompt and state-maintenance protocol, cost/latency measurement method, and statistical reporting for the three-LLM stability claim.","section":null}],"recommendation":"reject","confidential_remarks":"The cacheable full-manuscript block is unambiguously a different arXiv paper (2604.04810) from the one named in the review request (2604.04806). This is not an incomplete draft or missing appendix; it is a complete substitution. I recommend the editor reject or withdraw this submission until the correct MIRAGE manuscript is provided. I have not attempted to invent methods for MIRAGE from the abstract alone."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"Punchline: MIRAGE’s abstract sells a clean practical idea—stateful, per-request LLM simulation of microservice dependencies, using source + caller + traces—and reports very high fidelity (99%/99% status/shape on 110 scenarios vs 62%/16% record-replay, 8/8 integration parity). The problem is we do not have that paper. The full manuscript in the packet is Hayes on POSS-I vanishing sources (2604.04810), not MIRAGE (2604.04806). So every load-bearing number is abstract-only.\n\nWhat looks new, if the real paper matches the abstract: moving from pre-generated static stubs (record-replay, pattern mining, specs) to an online oracle that keeps cross-request state and can answer rare error and code-reasoning paths that traces underrepresent. That is a real SE testing angle, not a rebrand of mock generation. The abstract also does the right evaluation moves on paper—three systems (Boutique, Sock Shop, custom), signal ablation (source alone 100%; without source structure drops to 75%), three LLM families within 3%, determinism, and caller pass/fail parity. Cost/latency are stated up front (~3 s, $0.16–$0.82), which is honest about the trade.\n\nSoft spots, in proportion: without methods we cannot check scenario construction, what “response-shape” means, whether error paths are truly out-of-corpus or still code-derivable, multi-hop state stress, or prompt leakage. The weakest assumption is that code + traces let the model invent correct rare behavior rather than happy-path patterns; the ablation hints source is doing heavy lifting, which is good if true and circular if the tests are mostly “read the if.” Circularity risk looks ordinary for LLM evals, not definitional. Citation pattern and math are not assessable here.\n\nWho it is for: practitioners and SE researchers who already pay for integration tests and care about dependency isolation. Not a theory paper.\n\nRecommendation: treat the abstract as a promising conditional result. Do not cite or redesign work on these numbers until the real MIRAGE manuscript, scenarios, and code appear. If that paper exists and matches the abstract’s design, it deserves a serious referee; with the current packet, an editor would desk-reject for wrong/missing full text. Bring the idea to reading group only as a methods discussion, not as established evidence.","headline":"Strong abstract claim for online LLM dependency stubs, but the supplied full text is a different astronomy paper—so the 99% fidelity numbers cannot be audited.","tokens_in":9966,"tokens_out":600,"would_cite":false,"duration_ms":12843,"reading_group":"maybe","serious_thinker":"unclear","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"Runtime LLM simulation of microservice dependencies hits 99% fidelity where static record-replay falls to near zero on error and reasoning paths.","keywords":["microservice testing","dependency simulation","online LLM simulation","record-replay","integration testing","service stubs","white-box testing","response fidelity"],"falsifier":"Re-run the same 110 scenarios (and the eight integration tests) with white-box MIRAGE after stripping dependency source from the prompt and after replacing the evaluation suite with error and multi-hop cases that cannot be derived from the provided code; if status-code or response-shape fidelity collapses or pass/fail outcomes diverge from the real services, the central claim fails.","tokens_in":9936,"feed_emoji":"🧩","tokens_out":907,"duration_ms":15599,"temperature":0.7,"pith_summary":"Microservice tests often need stand-ins for other services. Today those stand-ins are built ahead of time—recorded traces, mined patterns, or hand-written stubs—so they only replay what was already seen or specified. On error-handling and code-reasoning cases that almost never appear in production traces, record-replay in this evaluation drops to 0% and 12% fidelity. The paper argues for answering each dependency call live with a large language model that keeps state across the scenario and is given the dependency’s source, the caller’s source, and production traces. Instantiated as MIRAGE and run on 110 scenarios from three systems, white-box mode reaches 99% status-code and 99% response-shape fidelity versus 62% and 16% for record-replay, and eight caller integration tests produce the same pass/fail results as the real services. The trade is roughly three seconds and a fraction of a dollar per dependency, paid for coverage that static artifacts miss.","feed_headline":"LLM stubs hit 99% fidelity where record-replay hits 0%","feed_subtitle":"Live dependency simulation from source and traces matches real services on error paths static fixtures miss.","key_machinery":"Online LLM simulation (MIRAGE): at test time the model is invoked per dependency request with source and traces as context, keeping scenario state so it can invent correct behavior for paths never encoded in a pre-built artifact.","core_discovery":"An online LLM that answers each dependency request at runtime, maintains cross-request state, and is conditioned on dependency source, caller source, and production traces can reproduce status codes and response shapes at 99% fidelity on 110 scenarios across three microservice systems—far above static record-replay—while producing the same pass/fail outcomes as real dependencies on eight caller integration tests.","pith_inferences":["If source-conditioned online simulation is reliable, stub maintenance for internal services could shrink to “point the simulator at the repo” rather than hand-written contracts.","The same runtime pattern may transfer to other I/O boundaries (databases, message queues, third-party APIs) where static fixtures under-represent failure modes.","Signal ablation already shows structure fidelity drops without source; multi-hop or concurrent scenarios would be the natural next stress test of state-keeping claims."],"forward_implications":["Teams can test caller services against error-handling and code-reasoning paths without recording those rare paths first.","White-box access to dependency source alone can be enough for high-fidelity simulation when traces are thin.","Integration-test pass/fail can match real dependencies, so MIRAGE can stand in during CI without changing test outcomes.","Latency of about three seconds per request and cost of roughly $0.16–$0.82 per dependency become the explicit budget for that coverage.","Results holding within 3% across three LLM families and staying deterministic across runs make the approach portable across model vendors."],"fun_headline_variants":["Online LLM dependency stubs hit 99% fidelity where record-replay hits 0%","MIRAGE: runtime LLM sim reaches 99% status and shape fidelity on 110 scenarios","Live LLM answers each dep request, matching real services on error paths","Source-conditioned online LLM stubs: 99% fidelity vs static record-replay 0-12%","Runtime LLM microservice stubs pass same tests as real deps at 99% fidelity"],"cache_read_input_tokens":128,"weakest_assumption_plain":"That giving the model the dependency’s source, the caller’s code, and production traces at request time is enough for it to produce correct rare error and reasoning behavior rather than only common happy-path patterns.","fun_headline_variants_meta":{"raw":{"variants":["Online LLM dependency stubs hit 99% fidelity where record-replay hits 0%","MIRAGE: runtime LLM sim reaches 99% status and shape fidelity on 110 scenarios","Live LLM answers each dep request, matching real services on error paths","Source-conditioned online LLM stubs: 99% fidelity vs static record-replay 0-12%","Runtime LLM microservice stubs pass same tests as real deps at 99% fidelity"]},"model":"grok-4.5","effort":"low","cost_usd":0.003906,"raw_usage":{"total_tokens":1205,"prompt_tokens":831,"num_sources_used":0,"completion_tokens":110,"cost_in_usd_ticks":39060000,"prompt_tokens_details":{"text_tokens":831,"audio_tokens":0,"image_tokens":0,"cached_tokens":0},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":264,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":831,"tokens_out":110,"duration_ms":3671,"temperature":1.0,"reasoning_tokens":264,"cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-13T09:38:37.523184+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"Re-run the same 110 scenarios (and the eight integration tests) with white-box MIRAGE after stripping dependency source from the prompt and after replacing the evaluation suite with error and multi-hop cases that cannot be derived from the provided code; if status-code or response-shape fidelity collapses or pass/fail outcomes diverge from the real services, the central claim fails.","supporting_citations":[],"review_version":2}