{"id":"6d1d0f80-e005-44e7-b6db-0ceb1f69b484","arxiv_id":"2505.21599","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":0,"one_line_summary":"A design proposal for a phase-aware JIT compiler IR visualization tool, with a working backend but an unimplemented front-end.","lead":"This paper outlines JITScope, a visualization framework that aims to show how a just-in-time compiler's intermediate representation changes across optimization phases. The system is still being built, so the paper focuses on architecture and plans rather than a working tool.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The phase-mapping step in Section III-B is load-bearing and unverified: if function-ID ranges do not correspond to actual optimization-phase order, every phase-aware view JITScope produces is mislabeled.","rationale":"I read this paper as a design-and-prototype report rather than a completed system. The authors are transparent about the front-end still being implemented and evaluation being future work, so a conditional acceptance is fair. The single most load-bearing technical assumption is not the D3 layout or the SQLite schema; it is the mapping from function IDs to optimization phases, because every novel feature (phase filtering, animation, transformation counts, node details) depends on those labels being correct. The reader's weakest-assumption identification matches this concern, and the proposed check directly supplies the missing ground-truth comparison. If the mapping fails, the paper's main contribution collapses into a static IR visualizer plus an incorrect phase axis; if it passes, the remaining issues are implementation maturity and usability, which the authors already acknowledge.","tokens_in":6896,"tokens_out":3604,"duration_ms":41898,"concrete_test":"Using the same IR JSON generator and a real JIT engine, instrument engine compilation to log each optimization phase's start and end events together with the compiler function IDs executed during that phase. For a benchmark input, compare the logged phase for each instruction against JITScope's inferred phase. If any instruction is assigned to a different phase, the phase-aware visualization is invalidated; agreement on at least 100 instructions spanning all phases would provisionally validate the heuristic.","verdict_should_be":"UNCHANGED","load_bearing_attack":"JITScope's central claim is that it can show how IR nodes and instructions evolve across compiler phases. That claim rests entirely on the backend's phase-label assignment, described only as 'traversing function ID ranges and mapping them to their corresponding optimization phases' (Section III-B). This requires two unstated assumptions: function IDs are assigned in execution order, and phase boundaries coincide with ID-range boundaries. Neither is proven, and Section V-A notes the JSON data was built for bug localization, not for phase-boundary recovery. If the ID order is registration order, or a compiler function spans multiple phases, the dropdown, animation, and phase-diff views will all be wrong. The paper's own Section V-B admits the front-end output has not been correctness-verified, and the evaluation plan in Section IV is entirely prospective, so no evidence currently supports the phase mapping.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes JITScope, a visualization framework intended to show how JIT compiler intermediate representations (IRs) evolve across optimization phases. The described pipeline consists of a Python backend that converts JSON-formatted IR data from the authors' earlier bug-localization tool into a normalized SQLite database, a controller layer that serves CSV data, and a D3.js-based frontend that is planned to provide interactive phase selection, node tracking, and instruction-level exploration. The manuscript presents the system architecture, database schema, a conceptual interface, an evaluation plan, and related work. It also explicitly states that the frontend is not yet finalized and that the correctness of visualization outputs has not been verified.","tokens_in":7134,"tokens_out":3826,"duration_ms":46564,"significance":"If the system were fully implemented and validated, JITScope would address a real gap: existing JIT IR visualizations, including the authors' own metro-map approach, are largely static and do not support phase-by-phase analysis. The paper's modular MVC design and its use of an existing IR dataset are reasonable starting points. The authors are also transparent about the system's incompleteness and about the provenance of their input data. However, as submitted, the central claim of an interactive visualization framework is not supported by any realized artifact or evaluation; the paper is best described as a design and roadmap document rather than a systems paper.","major_comments":[{"comment":"The abstract and introduction describe JITScope as an interactive visualization framework with a full-stack architecture and a D3.js-powered frontend that renders phase-aware graphs. Yet Section I states that 'we have not finalized the full front-end deployment,' and Section V-B adds that 'we also faced limited time to verify the correctness of visualization outputs.' The central contribution as claimed is therefore not currently realized. The paper must either present a completed and validated system or be reframed as a design/position paper with the claims adjusted accordingly.","section":"Abstract and Section I"},{"comment":"The phase-mapping step is load-bearing but unvalidated. Section III-B states that phase names are assigned 'by traversing function ID ranges and mapping them to their corresponding optimization phases.' This assumes that function ID ranges correspond monotonically to optimization-phase execution order and that phase boundaries coincide with ID-range boundaries. Section V-A notes that the input JSON was originally designed for bug localization, not for phase-boundary recovery. If the ID-to-phase mapping is incorrect, every phase-aware view—the dropdown selector, the animation, and the phase-diff summaries—will assign nodes and instructions to the wrong phases. The authors need to justify this mapping empirically, for example by showing known phases from the JIT compiler's source code or by validating against a ground-truth trace.","section":"Section III-B and Section V-A"},{"comment":"The evaluation plan is entirely prospective. No user study, task-based evaluation, or quantitative result is reported in the paper. For a claimed interactive visualization framework, at least a preliminary evaluation—even on a small dataset or with a few expert users—would be needed to substantiate the usability and interpretability claims. As it stands, the manuscript contains no evidence that the proposed visualizations will help developers track IR node evolution.","section":"Section IV"},{"comment":"The input data is described as 'an abstract model of the JIT compiler IR' generated by the authors' prior tool. This raises a fidelity concern: the visualization may not reflect the actual IR of a real JIT compiler such as V8 or HotSpot. The paper should state explicitly how the abstract model relates to real JIT IRs and why conclusions drawn from this model would transfer to realistic settings.","section":"Section II-B"}],"minor_comments":[{"comment":"There is a stray space in 'CSV ,' on the first paragraph of Section III-A.","section":"Section III-A"},{"comment":"The text mentions a 'na¨ıve visualization' but the spelling should be 'naïve' or 'naive.'","section":"Section III-E"},{"comment":"Figure 4 is described as a 'concept interface'; the caption should explicitly state that this is a mockup or wireframe, not a rendered output from the implemented system.","section":"Section III-E/Figure 4"},{"comment":"The phrase 'function ID ranges' is used without a precise definition. It would help to explain how function IDs are generated and why their ranges are meaningful for phase assignment.","section":"Section III-B"},{"comment":"The sentence 'the current prototype has not yet been adjusted fully for scalability' is awkwardly worded; consider rephrasing and providing concrete details on dataset sizes or node counts that were tested.","section":"Section V-B"},{"comment":"The related-work discussion is brief and focuses on the authors' own prior metro-map visualization. It would benefit from a more systematic comparison with other JIT or compiler IR visualization tools, including those not based on the same IR model.","section":"Section VI"}],"recommendation":"reject","confidential_remarks":"This manuscript is essentially an extended abstract for an unfinished project: the frontend is not implemented, the phase-mapping logic is unvalidated, and the evaluation plan is entirely future work. The paper's own admissions in Sections I and V-B contradict the abstract's claim of an interactive framework. While the topic is relevant and the authors' prior work is interesting, the current submission does not meet the standard for a journal publication. It might be suitable for a workshop or as a position paper if the claims are appropriately scaled back."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Chun,\n\nRead JITScope. Short version: an honest design paper for phase-aware JIT IR visualization, with a genuinely new interaction model, but the shipped artifact today is a backend pipeline and mockups. The front-end is a concept, and the load-bearing phase-mapping step is unverified.\n\nThe problem is real: JIT developers lack a way to see how IR nodes change across optimization phases. The existing tools, including Lim and Kobourov's metro map, give a static snapshot. JITScope's proposed phase dropdown, animation, node detail, and search are sensible. The backend is described concretely: a Python script normalizes JSON into a SQLite schema, a controller emits CSV, and the paper includes an ER diagram. The authors are candid: Section V-B says the front-end is not finalized and correctness of outputs was not verified, and the evaluation plan is explicitly prospective.\n\nSoft spots. The phase-mapping heuristic in Section III-B is the weak point. Phase names come from 'traversing function ID ranges,' which assumes function IDs follow execution order and align with phase boundaries. Neither assumption is shown, and Section V-A says the JSON was built for bug localization, not phase-boundary recovery. If that mapping is wrong, every phase-aware view is mislabeled. Second, no evaluation exists — no task-based study, no baseline comparison, no scalability test. Third, the abstract overclaims: calling JITScope an 'interactive visualization framework' is generous when the current prototype is a backend plus concept drawings. The related-work coverage is reasonable, and the reliance on the authors' own earlier tool for data is worth noting but not disqualifying.\n\nI would not accept this for a top venue yet. It deserves a workshop or vision track, and a constructive referee could help the authors see what validation is needed. If they implement the front-end, verify the phase mapping against real JIT compiler behavior, and run a small task-based comparison, the idea could turn into a solid systems paper. As it stands, the plan is clear but the central mechanism is untested.","headline":"An honest design paper for a phase-aware JIT IR visualization, but the shipped artifact is a backend plus mockups, and the phase-mapping heuristic that carries the whole idea is unverified.","tokens_in":7535,"tokens_out":3638,"would_cite":false,"duration_ms":39045,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper proposes JITScope, an interactive visualization framework that shows how JIT compiler IR nodes and instructions evolve across optimization phases, and describes its JSON-to-SQLite-to-D3 pipeline.","keywords":["Just-in-Time compiler","intermediate representation","compiler visualization","optimization phases","IR graph","SQLite","D3.js"],"falsifier":"Instrument a real JIT compiler to record the actual optimization phase of each instruction that touches an IR node, then compare that ground truth with JITScope's phase labels derived from function ID ranges; any mismatch in the phase attribution reveals that the central phase-aware view is unreliable without validation.","tokens_in":6678,"feed_emoji":"📊","tokens_out":6100,"duration_ms":54310,"temperature":0.7,"pith_summary":"JITScope is a proposed interactive visualization framework for Just-In-Time (JIT) compiler intermediate representations (IRs). The paper argues that a phase-aware, graph-based view of IR transformations—showing how nodes, values, and edges change across optimization phases—can help developers understand and debug JIT compilation. The system extracts raw JSON IR data into a normalized SQLite database, serves processed CSV to a D3.js front end, and renders an interactive node-link graph with phase selection, node tracking, and instruction-level detail. The paper presents the architecture and a working backend prototype, with the front end still in active implementation, and outlines a task-based evaluation plan for usability with compiler engineers. A fair reading treats JITScope as establishing a foundation and design direction for interactive IR-evolution tooling, not as a completed and evaluated system.","feed_headline":"JITScope visualizes IR evolution across JIT compiler phases","feed_subtitle":"Phase-by-phase IR views could help developers debug missed and incorrect JIT optimizations","key_machinery":"The load-bearing mechanism is the JSON-to-SQLite normalization pipeline inside a Model-View-Controller architecture. Raw IR JSON—nodes with addresses, opcodes, opcode updates, edges, values, an alive flag, and a log of accessing instructions—is parsed into ten relational tables connected by foreign keys; the Instructions table ties node modifications to function IDs, and phase names are assigned by traversing function ID ranges and mapping them to optimization phases. The controller serves query results as CSV, and the D3.js front end renders an interactive node-link graph with a phase dropdown, playback controls, search, an optimization overview panel, and a node details panel. This pipeline is what moves the IR from a static, deeply nested model into a phase-aware, queryable, and visually explorable form.","core_discovery":"On the paper's own terms, the central claim is that a JIT compiler's IR evolution can be made queryable and visually explorable by normalizing deeply nested JSON IR records into relational tables keyed to instructions and function IDs. The Instructions table is the core: each instruction that accessed or modified an IR node carries an ID that maps to a compiler source function, and by assigning phase names to ranges of function IDs, the pipeline reconstructs which optimization phase created or changed each node. This enables the paper's main visualization claim: instead of a static snapshot of the entire IR, developers can select a phase, see which nodes were generated, removed, or altered in that phase, and track an individual node's opcode, value, and edge history across the whole compilation.","pith_inferences":["The phase-mapping step is the main risk: replacing the function-ID-range heuristic with compiler-confirmed phase boundaries, or validating it against phase logs, would be a natural next test beyond the paper's stated roadmap.","The relational schema suggests analyses the paper does not spell out, such as detecting phases that churn many nodes or recognizing nodes that survive many phases with unchanged opcodes as candidates for missed optimizations.","If task-based evaluation shows that phase filtering helps users answer value-change and function-access questions faster than the prior static metro-map view, that would confirm the paper's core value proposition.","The pipeline could generalize to other JIT IRs—sea-of-nodes, CFG, or DFG—if their dump formats were normalized into the same schema, turning JITScope into a general debugging lens for JavaScript engines and JVMs."],"forward_implications":["Developers could select an optimization phase and immediately see which IR nodes were created, removed, or had their opcode, value, or edges changed in that phase, without reading compiler source code.","A node's full value and edge history across phases would let engineers trace exactly when an optimization transformed, kept, or eliminated a value, supporting miss-optimization debugging.","The SQLite model makes the IR queryable, so questions like 'which compiler function last accessed this node' or 'which phases touched this node' become direct database queries.","Because the view layer is decoupled from data extraction, the same backend pipeline could be adapted to other front ends or extended to additional IR formats without rewriting the normalization logic."],"supporting_citations":[{"why":"Supplies the IR JSON data model and generation tool that JITScope parses; the node and instruction records come from this earlier work.","marker":"[8]"},{"why":"Establishes the function-ID and instruction-log structure used for phase mapping and bug localization in JIT compilers.","marker":"[9]"},{"why":"Extends the same modeling approach to JIT compiler back ends, providing the data family JITScope adapts.","marker":"[10]"},{"why":"Prior static visualization of JIT compiler IRs, the closest related work whose lack of phase awareness JITScope directly addresses.","marker":"[15]"},{"why":"Describes the sea-of-nodes IR representation used by major JIT compilers, motivating the need for IR graph visualization.","marker":"[19]"}],"fun_headline_variants":["See JIT IR evolve phase by phase","JITScope: track IR nodes across compiler phases","Interactive map of JIT IR transformations","JITScope: visualize IR changes per optimization phase"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The central assumption is that ranges of function IDs correspond to optimization phases in a monotonic, reliable way, and that the JSON IR data is a faithful picture of the real JIT compiler's IR; if either is wrong, the phase-aware view will assign transformations to the wrong phase and mislead users.","fun_headline_variants_meta":{"raw":{"variants":["See JIT IR evolve phase by phase","JITScope: track IR nodes across compiler phases","Interactive map of JIT IR transformations","JITScope: visualize IR changes per optimization phase"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00037,"raw_usage":{"total_tokens":1942,"prompt_tokens":862,"completion_tokens":1080,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":478,"completion_tokens_details":{"reasoning_tokens":1022}},"tokens_in":478,"tokens_out":1080,"duration_ms":9222,"temperature":1.0,"reasoning_tokens":1022,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T13:27:47.458544+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Instrument a real JIT compiler to record the actual optimization phase of each instruction that touches an IR node, then compare that ground truth with JITScope's phase labels derived from function ID ranges; any mismatch in the phase attribution reveals that the central phase-aware view is unreliable without validation.","supporting_citations":[{"cited_title":"Modeling code manipulation in jit compilers,","cited_arxiv_id":null,"evidence_quote":"Supplies the IR JSON data model and generation tool that JITScope parses; the node and instruction records come from this earlier work."},{"cited_title":"Automated bug localization in jit compilers,","cited_arxiv_id":null,"evidence_quote":"Establishes the function-ID and instruction-log structure used for phase mapping and bug localization in JIT compilers."},{"cited_title":"Automatically localizing dynamic code generation bugs in jit compiler back-end,","cited_arxiv_id":null,"evidence_quote":"Extends the same modeling approach to JIT compiler back ends, providing the data family JITScope adapts."},{"cited_title":"Semantic reasoning about the sea of nodes,","cited_arxiv_id":null,"evidence_quote":"Describes the sea-of-nodes IR representation used by major JIT compilers, motivating the need for IR graph visualization."}],"review_version":1}