{"id":"0bf5ccc8-1412-4e64-a4d3-21df65856d14","arxiv_id":"1908.08375","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"An integrated tool visualizes how conditional compilation flags affect C program structure, letting developers toggle features and inspect resulting entities and source code.","lead":"This paper presents a visual analytics tool that combines the TypeChef parser, the jQAssistant graph database, and the Getaviz visualization framework to let developers explore how C preprocessor feature flags change program structure. It is a short SPLC tool paper aimed at easing maintenance of C codebases with conditional compilation.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claim that Q2 can be answered completely rests on unvalidated TypeChef extraction; a ground-truth preprocessing check is needed.","rationale":"The reader's weakest assumption is that TypeChef's variability-aware AST accurately and completely captures all relevant preprocessor constructs for arbitrary C code, with no validation provided on the BusyBox example. My stress-test refines that assumption into the specific overclaim 'Q2 can already be answered completely' and ties it to a concrete, falsifiable check. The concern is load-bearing because the central value proposition—correctly showing which entities exist under each feature configuration—collapses if the extraction is incomplete. However, the paper is clearly a tool demonstration, the limitation is partially acknowledged in the Discussion, and TypeChef is a reasonable and widely used basis for variability-aware C parsing. The lack of validation justifies a conditional rather than a reject verdict, which matches the reader's original CONDITIONAL verdict. I therefore recommend no change to the verdict. Other secondary concerns, such as the absence of an actual feature-model input despite the 'bridge' language, are real but less central; the Q2 completeness claim is the sharpest point on which the paper's contribution stands or falls.","tokens_in":6232,"tokens_out":4408,"duration_ms":131776,"concrete_test":"Use the C preprocessor to generate ground truth for BusyBox 1.18.5: choose 10 configurations over the CONFIG_* flags used in find.c and 20 other translation units; for each configuration run `cpp -D... -E` or `gcc -E` and parse the preprocessed output to extract functions, global variables, and typedefs; compare that set with the entities the tool renders as non-transparent under the same flags. Any mismatch—a function that disappears or an entity that remains visible—refutes the 'completely' claim. In addition, run one hand-crafted C file containing function-like macros that define entities and `#if` conditions over undefined identifiers to check whether TypeChef's presence conditions survive; if they do not, the extraction pipeline has a concrete completeness gap.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's strongest surviving claim, after its own concession that Q1 is only partially answerable, is the Discussion statement that 'Q2 can already be answered completely.' This claim is load-bearing: if the visualization omits or misattributes a code entity under a feature configuration, the tool's central value—tracing the effect of enabling or disabling feature flags—fails. The completeness of Q2 depends entirely on the unvalidated assumption in Section 3 that TypeChef's variability-aware AST, serialized to XML, captures every relevant preprocessor directive, macro expansion, nested condition, and feature reference. TypeChef is known to be incomplete for arbitrary C: it handles a defined subset of preprocessor constructs and can reject or approximate configurations outside that subset. The paper's own requirement (3) acknowledges that macro expansion can affect feature detection and location, yet no check is reported that expanded macros preserve the presence conditions needed for the visualization. No comparison with brute-force preprocessing, no ground-truth entity sets, and no independently exercisable artifact or demo verification are provided for the BusyBox scenario. Thus the assertion that Q2 is answered 'completely' is not established; it may be true for the demonstrated example, but the paper provides no evidence that it holds generally.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents an interactive visual analytics tool, built as an extension of Getaviz, for investigating conditional compilation in C source code. The tool uses TypeChef to parse C code without manual preprocessing, serializes the variability-aware abstract syntax tree to XML, and stores the result in a graph database via jQAssistant. The user interface visualizes all potentially compiled code entities using a Recursive Disk metaphor, lets the user toggle extracted feature flags, and updates the visualization by rendering excluded entities transparently. The authors illustrate the tool on BusyBox 1.18.5, showing the effect of the CONFIG_DESKTOP feature flag on several translation units. They claim the tool bridges the gap between feature models and concrete CPP-based implementations, that Q2 ('which elements are contained in the source code given a certain feature configuration') can already be answered completely, and that Q1 ('what effect does the activation of a feature have on the structure of a program') can be answered only partially. The paper is framed as a demonstration and reports no user study or quantitative evaluation.","tokens_in":6456,"tokens_out":3937,"duration_ms":136676,"significance":"If the central claims were established, the tool would fill a real gap: most existing variability visualizations operate at the feature-model level or require manual preprocessing, whereas this tool attempts to connect feature configurations directly to source-level entities in an integrated environment. The adaptation of the Recursive Disk metaphor to C translation units is a sensible design choice, and the integration of TypeChef, jQAssistant, and Getaviz is a plausible pipeline that avoids manual preparation steps. The paper includes an online demo and a screencast, which are valuable for a tool-demo venue. However, the significance is limited by two issues: the completeness of Q2 depends on an unvalidated assumption about TypeChef's extraction accuracy, and the usefulness claims rest solely on the authors' own BusyBox demonstration. The contribution is best viewed as a prototype and a design proposal that needs stronger evidence before the stronger claims are acceptable.","major_comments":[{"comment":"The paper states in §6 that Q2 'can already be answered completely', but this claim is not supported by the evaluation in §5. The completeness of the answer depends on the TypeChef variability-aware AST, serialized to XML, containing every relevant conditional compilation directive, macro expansion, nested condition, and feature reference. No ground-truth check is reported: there is no comparison with brute-force preprocessing for a set of configurations, no enumeration of expected entity sets, and no error analysis of TypeChef's known limitations for arbitrary preprocessor constructs. In addition, requirement (3) in §3 acknowledges that macro expansions can influence feature detection and location, yet no evidence is given that expanded macros preserve the presence conditions needed for the visualization. I ask the authors to add a validation on the BusyBox example, for instance by preprocessing several concrete configurations and comparing the entities shown by the tool with those actually compiled, and to state the limitations of TypeChef that remain.","section":"§3, §6"},{"comment":"The abstract and conclusion claim that the tool 'simplifies tracing and understanding' the effect of enabling or disabling feature flags, and §5 states that it 'improves the developers understanding' of the resulting structure and behavior. The only evidence is the authors' own BusyBox demonstration, consisting of screenshots and a screencast. There is no user study, no baseline comparison with existing tools or manual inspection, and no task-based or time-based measurement. The claim as stated is therefore not supported. The authors should either add a small comparative evaluation or carefully rephrase the claims as demonstrating feasibility of the visualization approach rather than established improvement in developer understanding.","section":"§1, §5, §7"},{"comment":"The extraction pipeline is described only at a high level: TypeChef is modified to serialize the AST to XML, and a plugin is implemented to include C code and feature flags. It is not explained how feature flags are identified and extracted, how the mapping from preprocessor expressions to named flags is derived, or how macros such as ENABLE_DESKTOP, IF_DESKTOP, and IF_NOT_DESKTOP are resolved into the presence conditions shown in the visualization. Without this explanation, the reader cannot assess whether the flag list shown in the FeatureExplorer is complete or whether flags with complex expression dependencies are handled correctly. A concrete description of the extraction steps and a sample of the produced graph would strengthen the paper.","section":"§3"}],"minor_comments":[{"comment":"There are typos in the abstract and keywords, e.g., 'variablity' and 'prodect line' should be corrected to 'variability' and 'product line'.","section":"Abstract, Keywords"},{"comment":"The caption of Figure 2 uses inconsistent capitalization in 'fEATURE_find_EXEC' and 'fEATURE_find_XdEV'; these should be normalized to the actual feature flag names, e.g., FEATURE_FIND_EXEC and FEATURE_FIND_XDEV.","section":"Figure 2"},{"comment":"The sentence 'Our tool improves the developers understanding of the resulting structur and behavior' contains grammar and spelling errors ('developers' should be 'developer's', 'structur' should be 'structure').","section":"§5"},{"comment":"The scalability claim 'We can visualize systems with up to four million lines of code without any problems' is unsupported: no measurements of rendering time, interaction latency, or memory usage are given, and 'a conventional notebook' is not a precise specification. The authors should either provide concrete measurements or qualify this statement as an observed limit rather than a general capability.","section":"§6"},{"comment":"The demo and screencast are only given as URLs. Since these are central for verifying the described behavior, the authors should provide a stable artifact, e.g., a DOI link or an archived repository, so that the demo remains accessible.","section":"§1, §5"},{"comment":"The text refers to regions I, II, III, and IV in Figure 1, but the figure itself appears to use labels within the screenshot; a legend or a more explicit labeling would make the relation between text and figure clearer.","section":"Figure 1"}],"recommendation":"major_revision","confidential_remarks":"The paper is a short tool-demo contribution and the presentation is generally clear. The main risk is overclaiming: the Q2 completeness statement and the usability claim go beyond what the current evidence supports. If the authors add a ground-truth check for extraction and rephrase the usability claims as feasibility claims, the paper could be acceptable for a demonstration-oriented venue. I do not see a citation or novelty problem: the use of Getaviz and TypeChef is transparent, and the authors' reliance on their own prior work is appropriate for a prototype paper."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague — quick take: this is a short SPLC tool paper, not a research breakthrough. What's new is the integration: TypeChef's variability-aware AST serialized to XML, jQAssistant as the graph layer, and Getaviz's Recursive Disk visualization with interactive feature toggles for C conditional compilation. I'm not aware of another tool that gives this system-wide feature-configuration view of C code, and the design rationale is sensible. The BusyBox demo is a reasonable choice, and the paper is honest that Q1 is only partially answerable and that comparisons with existing tools are future work.\n\nThe thing to scrutinize is the Discussion claim that \"Q2 can already be answered completely.\" That claim is load-bearing: if the extracted entity set under a feature configuration omits or misattributes anything, the tool's core value—tracing what enabling a flag affects—collapses. The completeness depends entirely on TypeChef capturing every relevant directive, macro expansion, and nested condition, and the paper gives no ground-truth validation of that extraction on BusyBox. TypeChef is known to handle only a subset of arbitrary C preprocessor constructs, and requirement (3) acknowledges that macros can affect feature detection. A simple brute-force preprocessing comparison would settle it, or at least a check on a few files with known presence conditions. Without that, \"completely\" is an overstatement.\n\nOther soft spots: no user study, no baseline comparison, and no performance measurement behind \"up to four million lines of code.\" Those are minor for a 4-page tool demo, and the authors explicitly list them as future work. The lack of an independent artifact is partly offset by the online demo and screencast, though the demo wasn't exercisable from the paper text. One caveat: the tool and Getaviz come from the same group, so the demo is self-referential; that's normal for tool papers, but it raises the bar for independent validation, which isn't met.\n\nIf I were refereeing, I'd accept with revisions asking for the Q2 completeness claim to be either downgraded to \"demonstrated on this example\" or backed by a ground-truth extraction check. The integration is real, the write-up is clear, and the topic is useful for people maintaining configurable C systems. It deserves a serious referee but not publication as is.","headline":"A competent SPLC short-paper tool demo with a real prototype but a load-bearing 'complete' claim about Q2 that needs a ground-truth check before it can be taken literally.","tokens_in":6959,"tokens_out":1887,"would_cite":false,"duration_ms":26036,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A C-preprocessor-aware analytics tool lets developers toggle feature flags and watch which code entities disappear.","keywords":["conditional compilation","C preprocessor","feature flags","software product lines","variability","software visualization","visual analytics","TypeChef"],"falsifier":"Take a C system such as BusyBox 1.18.5, run the TypeChef-to-graph pipeline, then for a set of flag configurations compile the system and count which functions, global variables, and complex types actually appear in the preprocessed translation units; any entity the tool marks as present but the preprocessed output omits, or vice versa, refutes the tool's claim that Q2 is answered completely.","tokens_in":6059,"feed_emoji":"🧩","tokens_out":6928,"duration_ms":238763,"temperature":0.7,"pith_summary":"The paper is trying to establish that the variability hidden in C conditional compilation can be made directly inspectable: a developer should be able to pick a set of feature flags and see exactly which functions, global variables, and complex types will exist in that variant. To that end it builds a tool pipeline that parses C with preprocessor awareness, stores the result as a graph, and renders it interactively so excluded entities become transparent. The concrete demonstration is BusyBox; toggling CONFIG_DESKTOP immediately shows how macro chains spread the flag's effect across 75 of 354 translation units. The paper's own standard of success is two questions: 'what does activating a feature do to program structure' and 'which elements exist under a given configuration'; it claims the second is already answered completely, and the first only partially because not every feature location is visually exposed yet.","feed_headline":"See which C functions each feature flag compiles or excludes","feed_subtitle":"The paper's visualization pipeline turns C preprocessor directives into a queryable graph and renders excluded code transparent.","key_machinery":"The load-bearing mechanism is a four-stage pipeline: TypeChef modified to serialize a variability-aware AST to XML, an jQAssistant plugin that ingests that XML into a Neo4j graph, the Getaviz visualization toolkit, and the Recursive Disk metaphor adapted to C. The Recursive Disk layout maps each translation unit to a gray disk, functions to blue segments with area proportional to lines of code, global variables to fixed-size yellow segments, and structs, enums, and unions to purple disks. This mapping is what lets a flag toggle be rendered as transparency rather than a textual search, and the graph is what lets entity-presence questions be answered by structure rather than by hand-tracing macros.","core_discovery":"The central claim is that integrating variability information with source-code structure in one graph model makes preprocessor-based product lines comprehensible. The tool serializes TypeChef's variability-aware AST to XML, loads it through an jQAssistant plugin into a graph containing code entities, method calls, read/write accesses, features, and dependencies, and visualizes the graph with Getaviz's Recursive Disk metaphor. In the visualization, translation units are gray disks, functions are blue segments sized by lines of code, global variables are yellow segments, and structs, unions, and enums are purple disks. The user toggles feature flags in the FeatureExplorer UI; entities excluded by the C preprocessor become transparent, turning a configuration query into a visible structural comparison. The application to BusyBox 1.18.5 shows that this answers Q2 completely, while Q1 remains partially supported until feature-to-method highlighting is exposed in the interface.","pith_inferences":["The same pipeline should transfer to any preprocessor-conditional system, not just product-line feature flags: platform-specific #ifdefs, compile-time debugging switches, and version guards would all become queryable graph structure rather than scattered directives.","Because the graph retains nested conditions and branch alternatives, it could be extended to search for dead-feature configurations, flag sets for which no entity remains, or to detect feature flags that affect no entity, without new parsing.","A testable extension would be to color translation units by variability density, the number of distinct flags guarding their entities, so feature-heavy modules stand out in the overview, a design choice the paper does not itself explore."],"forward_implications":["Developers can see every function, global variable, and complex type that could be part of any variant before choosing a configuration, giving a system-wide overview.","Selecting or deselecting feature flags updates the structure, so excluded entities become transparent and the effect of the flag is visible without jumping between source files.","A single feature flag such as CONFIG_DESKTOP, which reaches 75 of 354 BusyBox translation units through macros like ENABLE_DESKTOP, IF_DESKTOP, and IF_NOT_DESKTOP, is handled automatically.","The analysis runs without manual preprocessing and, on the reported setup, handles systems up to roughly four million lines of code, with the BusyBox graph generation taking about one day on a conventional notebook.","Q2 is claimed to be fully answered: for a given feature configuration, the tool can list which elements are present. Q1 is only partially answered because not all feature locations are yet visually detectable."],"supporting_citations":[{"why":"Variability-aware parser used to build the AST that contains conditional-compilation information and macro-expanded code.","marker":"[10]"},{"why":"Graph-store stack that the modified TypeChef XML is loaded into, making entities and feature dependencies queryable.","marker":"[18]"},{"why":"Getaviz visual analytics toolkit used as the base for the interactive user interface.","marker":"[3]"},{"why":"Recursive Disk layout that was adapted to show translation units, functions, variables, and types as nested disks and segments.","marker":"[19]"},{"why":"Source of the BusyBox example's configurability, showing the complex feature-flag dependencies the tool must handle.","marker":"[11]"}],"fun_headline_variants":["Toggle feature flags to see excluded C code go transparent","Graph tool maps C preprocessor directives to feature toggles","Interactive map shows what each C feature flag compiles","See hidden C code when you flip feature flags"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The tool's correctness rests on TypeChef's variability-aware parse: if it misreads or omits a preprocessor directive, macro-expanded branch, or feature reference, the visualization shows a different variant than the compiler would build.","fun_headline_variants_meta":{"raw":{"variants":["Toggle feature flags to see excluded C code go transparent","Graph tool maps C preprocessor directives to feature toggles","Interactive map shows what each C feature flag compiles","See hidden C code when you flip feature flags"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000399,"raw_usage":{"total_tokens":2029,"prompt_tokens":834,"completion_tokens":1195,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":450,"completion_tokens_details":{"reasoning_tokens":1131}},"tokens_in":450,"tokens_out":1195,"duration_ms":8509,"temperature":1.0,"reasoning_tokens":1131,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T12:00:42.577644+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a C system such as BusyBox 1.18.5, run the TypeChef-to-graph pipeline, then for a set of flag configurations compile the system and count which functions, global variables, and complex types actually appear in the preprocessed translation units; any entity the tool marks as present but the preprocessed output omits, or vice versa, refutes the tool's claim that Q2 is answered completely.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Graph-store stack that the modified TypeChef XML is loaded into, making entities and feature dependencies queryable."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Getaviz visual analytics toolkit used as the base for the interactive user interface."}],"review_version":1}