{"id":"3bcae1a9-fd55-47c4-8b22-206a20bcd61f","arxiv_id":"2511.23213","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A static path-reconstruction and guided UI-exploration tool reaches 57.44% of randomly chosen Android methods on AndroTest, outperforming APE, GoalExplorer, and Guardian.","lead":"GAPS is a new tool that reads an Android app's code backwards from a chosen method, works out which buttons and entry points lead there, and then clicks through the app to actually run that method. It reports reaching 57% of target methods on a standard benchmark, far above existing GUI testers, which makes it a promising building block for security testing.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Head-to-head numbers may be unfair: GUI baselines were not target-aware and 65% of target methods lie outside Activities.","rationale":"The reader's formal weakest_assumption was the completeness of EdgeMiner/Soot/FlowDroid callback mappings. I do not dispute that this matters for static path recall, but it is not the most load-bearing issue for the headline claim: a missing framework edge produces false negatives, lowering the static 88.24% rate, whereas the dynamic 57.44% is measured by actual execution and therefore already accounts for many runtime failures. The more serious risk is that the comparison to APE, GoalExplorer, and Guardian is not a same-task comparison: those tools were never told which methods to reach, and a large share of the target methods are not GUI-reachable by design. If a target-aware baseline were run, GAPS's margin could shrink substantially. This does not refute the architecture—GAPS's static guidance is a real contribution—but it does mean the paper's quantitative 'outperforms state-of-the-art' claim is not yet supported as stated. The reader's CONDITIONAL verdict remains appropriate: the paper should be accepted only after the comparison is made fair (e.g., target-aware Guardian/GoalExplorer runs or a stratified analysis), and after the abstract/body number discrepancies are reconciled.","tokens_in":18371,"tokens_out":5890,"duration_ms":62276,"concrete_test":"Use the released evaluation logs to recompute Table 1 dynamic reachability separately for the 34.39% of target methods located inside Activities versus all others; then rerun Guardian once with a target-aware prompt (method signature plus containing activity/class) under the same 5-minute, 3-run protocol. If Guardian's Activity-local reachability approaches GAPS's and the GAPS margin is concentrated in non-GUI methods, the superiority claim is an artifact of target selection; if the margin persists in both strata, the claim survives.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central quantitative claim—GAPS reaches 57.44% of target methods vs 12.82%/9.69%/17.12% for APE/GoalExplorer/Guardian—rests on a comparison in which the GUI baselines were not given the target. §4.2 says Guardian was tasked to 'interact with the application to maximize exploration'; APE and GoalExplorer were run as coverage/exploration tools with no target specification, and GoalExplorer's target-flagging mode was unusable due to broken Maven dependencies. GAPS, by contrast, receives each target method and its statically reconstructed path. Moreover, the 50 targets per AndroTest app are randomly drawn from all methods, with only 34.39% located inside an Activity (§4.1); the remaining 65% live in services, receivers, libraries, or background code that GUI testers are not designed to reach. The authors themselves note that GUI testers are 'unable to reach methods that are not embedded in a GUI component.' Thus the large gap may reflect task setup rather than technical superiority. The 'first to bridge method-level reachability' claim is load-bearing on this comparison, and the limitation admitted in §7.3 (unsound implicit-flow mappings) is less threatening to the headline because missing edges would lower static recall, not inflate the dynamic success rate.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"GAPS is a hybrid Android analysis system that, given a target method, performs a lightweight context-sensitive backward traversal of a statically constructed call graph enriched with data-flow analysis (points-to, constant propagation) to reconstruct feasible execution paths from entry points to the target, resolve conditional statements, and detect GUI elements. These paths are compiled into high-level interaction plans executed dynamically via adb, AndroidViewClient, and an optional Guardian-based fallback, with Frida hooks or AndroLogs used to confirm method execution. The paper reports evaluations on the 56-app AndroTest benchmark and 50 real-world Play Store apps, claiming large improvements over APE, GoalExplorer, Guardian (dynamic) and FlowDroid, DroidReach (static).","tokens_in":18695,"tokens_out":6936,"duration_ms":67572,"significance":"If the results are valid, GAPS would be a valuable first step toward practical method-level reachability, with clear downstream applications in vulnerability triage and targeted dynamic analysis. Strengths of the paper include the open-source release of source code and evaluation data, use of the standard AndroTest benchmark, instrumentation-based measurement rather than simulation, and a candid discussion of limitations. However, the central empirical claims are currently undermined by internal numerical inconsistencies and by a baseline setup that may be fundamentally unfair. These issues must be resolved before the contribution can be considered established.","major_comments":[{"comment":"The headline numbers disagree between the abstract and the full text. The abstract reports Guardian 34%, GoalExplorer 4.75%, APE 11.12%, GAPS AndroTest dynamic 56.93%, real-world dynamic 54.80%, and a static analysis time of 12.67 s; the full text reports Guardian 17.12%, GoalExplorer 9.69%, APE 12.82%, GAPS dynamic 57.44%, real-world dynamic 59.86%, and 4.27 s. Since these values are the paper's central quantitative claims, the authors must identify which set is correct and ensure all versions (abstract, introduction, conclusions) are consistent.","section":"Abstract vs. §4.2, §7.1, §7.2"},{"comment":"The head-to-head comparison is not apples-to-apples. APE and GoalExplorer were run as coverage/exploration tools without target specification; GoalExplorer's target-flagging mode was unusable due to broken Maven dependencies. Guardian was prompted to 'interact with the application to maximize exploration' rather than to reach a specific method. GAPS receives each target method plus a statically reconstructed path. Moreover, only 34.39% of the targets are inside an Activity (§4.1); the other ~65% live in components that GUI testers are not designed to reach, as the authors themselves note in §2.1. The large gap (57.44% vs ≤17.12%) may therefore reflect task setup rather than technical superiority. Please provide target-aware baselines (e.g., a Guardian prompt that includes the target method's class/signature) or explicitly frame the comparison as 'unrestricted exploration vs. targeted gui","section":"§4.2, §7.1"},{"comment":"GAPS's dynamic module incorporates Guardian as an optional fallback when a widget/activity cannot be found, and §7.1 states Guardian was invoked on average four times per app, 'contributing to reaching 9.3% of the methods.' If the 57.44% figure includes these contributions, GAPS's reported success partly depends on the same LLM-based tool it is compared against. Please ablate the Guardian fallback and report GAPS's reachability with and without it. If the fallback is essential, the claim of outperforming Guardian must be reinterpreted.","section":"§6.3, §7.1"},{"comment":"Despite 'three runs', no measure of dispersion is reported: no standard deviations, confidence intervals, per-app results, or random seeds for target selection. The reader cannot assess whether the 57.44% vs 17.12% difference is stable or driven by a few outlier apps. Please report per-tool and per-app summary statistics, and if appropriate a paired significance test (e.g., Wilcoxon). Also state the target-selection seed and whether the three runs used identical or different target sets.","section":"§7.1"},{"comment":"Dynamic reachability is measured by logging every method execution via AndroLogs (AndroTest) or Frida hooks (real-world apps). This counts any execution during the run, including executions that may happen at app startup or via background components independently of GAPS's guidance. The paper should demonstrate that the target methods are executed after executing the statically reconstructed plan, not merely at any point during the dynamic run. This concern is acute in the real-world set, where reaching 59.86% of targets when static paths exist for only 62.03% (i.e., ~96.5% of statically reachable targets) suggests that some executions may be incidental rather than caused by the GAPS-directed plan.","section":"§7.1, §7.2"}],"minor_comments":[{"comment":"The code contains artifacts such as 'Ap pCo mp at tiv it y' and the line numbers are misaligned. Please proofread the listings.","section":"Listing 1"},{"comment":"The average GAPS dynamic time is reported as '00:03:15', inconsistent with other time formats. Include standard deviation or per-app variability in the table.","section":"Table 1"},{"comment":"State clearly whether the three runs use the same set of 50 target methods or a different random sample per run; this is needed to interpret the reported averages.","section":"§4.2"},{"comment":"The phrase 'we improved DroidReach by restricting entry points to only exported components' modifies the baseline. This should be labeled as a modified DroidReach variant in Table 1 and the text, since results may differ from the original tool.","section":"§4.3"},{"comment":"The predicate 'verify_condition' is used without a detailed definition. Please describe how non-constant operands (e.g., method return values) are evaluated and how satisfiability is decided.","section":"§5.5, Algorithm 3"},{"comment":"Reference [47] appears to duplicate [24] in intent (Stoat); check for duplicate or unintended entries and ensure all citations in the text match the bibliography.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The core idea and the open-source release are strengths, but the evaluation's fairness and internal consistency are currently the main blockers. If the authors can correct the numerical inconsistencies, ablate the Guardian fallback, and re-frame or re-run the baseline comparison in a target-aware manner, the paper may become publishable. I would not reject outright because the static path reconstruction contribution is clearly described and the limitations section is candid."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea is solid: instead of building a whole-program call graph, GAPS walks backward from a target method, uses lightweight data-flow to satisfy conditions and find GUI elements, then replays that plan dynamically. That is a real and useful combination, and the release of code and evaluation data makes it reproducible. Static path reconstruction on 88.24% of randomly selected methods in AndroTest, in a few seconds per app, is a meaningful improvement over FlowDroid's 58.81% and DroidReach's 9.48% in the same setup. The dynamic reachability of 57.44% is also impressive as an absolute number, and the real-world study on 50 Play Store apps shows scalability, even if the 62.03% static rate there is lower.\n\nThe biggest problem is the comparison against the GUI baselines. APE, GoalExplorer, and Guardian were not given the target; they were asked to explore. GAPS receives each target and a statically derived path. The authors even note that only 34.39% of targets are inside Activities, so the other ~65% are in services, receivers, or libraries that GUI testers are not designed to reach. That explains much of the gap: APE at 12.82% and Guardian at 17.12% are coverage tools, not oracle-powered reachability tools. The paper presents this as GAPS outperforming the state of the art, but the task setup is not comparable. This is not fatal to the technical contribution, but it means the headline '57% vs 17%' overstates GAPS's edge over a properly targeted baseline.\n\nThere is also a concrete reporting problem: the arXiv abstract lists Guardian as 34% and GoalExplorer as 4.75%, while the full text says 17.12% and 9.69%. The real-world dynamic numbers also differ (54.80% vs 59.86%). The authors need to reconcile these versions. I would also want seed/variance disclosure for the three runs; without it, the 57.44% is a point estimate with unknown noise.\n\nThe implicit-callback concern raised by the stress test is real but minor: GAPS relies on EdgeMiner, Soot, and FlowDroid mappings, which is standard practice, and the paper explicitly admits unsoundness for untracked implicit flows in §7.3. Missing edges would lower static recall, not inflate dynamic success, so it doesn't undermine the main result.\n\nOverall: this is a solid systems paper with a credible architecture and honest limitations. It deserves serious refereeing, but the empirical claims need either retargeted baselines or clearly scoped language ('GUI testers without target information'). I'd bring it to a reading group for the design discussion, and I'd cite it for the backward-slicing approach, though not for the comparative numbers.\n\nRecommendation: send to peer review with a request for major revision on the evaluation framing and version consistency.","headline":"GAPS is a genuinely useful hybrid for method-level Android reachability, but the head-to-head numbers are not apples-to-apples and the abstract/body mismatch needs fixing before the quantitative claims can be trusted.","tokens_in":121,"tokens_out":868,"would_cite":true,"duration_ms":22585,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Static backward call-graph traversal can steer an Android app to execute a target method, and GAPS reaches 57.44% of targets where prior GUI testers stayed under 18%.","keywords":["targeted execution","Android app testing","static path reconstruction","backward call graph traversal","data-flow analysis","GUI-guided dynamic analysis","method reachability","Android security"],"falsifier":"Construct a small app whose target method is guarded by a boolean set inside a dynamically registered broadcast receiver with an action absent from GAPS's callback mappings, then run GAPS: if it fails to produce a static path for that target while the method is genuinely reachable at runtime, that app is a counterexample to the completeness of the path-reconstruction claim. More directly, re-instrument the same AndroTest apps with exhaustive runtime method logging and recompute the 57.44% figure; a large drop would show the result depends on the specific sample or instrumentation rather than o","tokens_in":18292,"feed_emoji":"🎯","tokens_out":5699,"duration_ms":55893,"temperature":0.7,"pith_summary":"The paper claims that targeted method-level execution in Android apps—long stuck because GUI testers explore breadth-first and whole-program call graphs are expensive—becomes practical if you reverse the direction of analysis. Instead of building a full call graph, GAPS starts from the target method and walks backward through callers, enriching the partial graph with data-flow facts about conditions, inter-component intents, and GUI element bindings. Those facts are compiled into concrete interaction plans—send a broadcast, click these buttons—that the dynamic phase executes, with an LLM-driven fallback when the UI is unpredictable. On 56 AndroTest apps this reaches 57.44% of randomly selected targets at runtime, versus 9.69–17.12% for existing GUI testers; on the 50 most-downloaded Play Store apps it still reaches about 55–60% within a five-minute budget. The claim matters because vulnerability and malware analyses routinely identify methods that are believed to be runnable but that no dynamic tool can actually be forced to execute.","feed_headline":"Backward call-graph search reaches 57% of Android target methods","feed_subtitle":"Static path plans steer GUI interaction, beating prior tools that stayed below 18 percent.","key_machinery":"The central mechanism is a context-sensitive backward traversal that builds a partial call graph slice ending at the target method, using Androguard's method and basic-block representation for smali-level inspection and networkx for path enumeration. Points-to analysis and constant propagation resolve conditional statements by linking guard operands to the constant assignments—including assignments made through broadcast receivers—that make the condition true; ICC mapping resolves intent senders and receivers into entry points; and GUI event retrieval maps listener registrations back to resource IDs via findViewById traces. The output is a JSON execution plan: entry-point intents plus ordere","core_discovery":"The core discovery is that a demand-driven, backward partial call graph—built only from the target method upward to entry points—is not just faster than whole-program call graph construction but also provides the exact routing information that a GUI driver lacks: which intents trigger which components, how to satisfy a guard condition by setting a static field through a broadcast receiver, and which view IDs to click. GAPS demonstrates this on AndroTest: static paths for 88.24% of target methods in 4.27 seconds average, with 57.44% dynamically executed, compared with 58.81% static reachability for FlowDroid and 12.82% dynamic reachability for APE, 9.69% for GoalExplorer, and 17.12% for Guard","pith_inferences":["A natural extension is to transport the backward-slice recipe to other event-driven platforms (web, iOS, desktop) where the bottleneck is the framework's implicit callback edges; the paper itself flags Flutter and React Native as needing a different bytecode front-end.","The drop from 88.24% static path coverage to 57.44% dynamic execution can be decomposed into missing framework edges versus difficulty in reconstructing runtime state; if missing edges dominate, investing in richer callback maps (e.g., Jetpack Compose support) would directly raise dynamic reach.","Because GAPS emits explicit high-level plans, those plans are themselves artifacts that could seed regression tests, replay scripts, or mutation targets without requiring the dynamic module at all."],"forward_implications":["If GAPS's numbers hold, vulnerability triage can move from 'is the method present?' to 'can we actually reach it dynamically?'—for 62% of flagged suspicious methods in popular apps, a concrete execution plan exists.","Dynamic reachability of about 57% within a five-minute cap makes targeted execution a feasible oracle for confirming whether a reported bug or leak is triggerable, not just statically plausible.","The static phase's average of 4.27 seconds on benchmark apps suggests path reconstruction can be embedded in continuous-integration pipelines without blowing up the analysis budget.","GAPS's conditional-path linking turns inter-component dependencies (e.g., broadcast must arrive before click) into enforced orderings in the interaction plan, enabling test generation that respects Android's event-driven semantics."],"fun_headline_variants":["Static call-graph paths steer Android GUI to 57% target hits","Backward path synthesis beats LLM and GUI tools in Android targeting","GAPS: fast static routing for Android apps reaches targets dynamically","Demand-driven call graph guides Android testing to 57% of targets","Android targeted execution boosted by static path reconstruction"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"GAPS's backward search finds a real path only if the static graph already contains the framework edge—lifecycle callback, broadcast, or listener—that actually fires at runtime; if those callback mappings miss an implicit flow, the reconstructed path does not correspond to any feasible execution and the reachability numbers overstate what a real run will do.","fun_headline_variants_meta":{"raw":{"variants":["Static call-graph paths steer Android GUI to 57% target hits","Backward path synthesis beats LLM and GUI tools in Android targeting","GAPS: fast static routing for Android apps reaches targets dynamically","Demand-driven call graph guides Android testing to 57% of targets","Android targeted execution boosted by static path reconstruction"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000201,"raw_usage":{"total_tokens":1279,"prompt_tokens":869,"completion_tokens":410,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":613,"completion_tokens_details":{"reasoning_tokens":323}},"tokens_in":613,"tokens_out":410,"duration_ms":4932,"temperature":1.0,"reasoning_tokens":323,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T19:34:12.080097+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Construct a small app whose target method is guarded by a boolean set inside a dynamically registered broadcast receiver with an action absent from GAPS's callback mappings, then run GAPS: if it fails to produce a static path for that target while the method is genuinely reachable at runtime, that app is a counterexample to the completeness of the path-reconstruction claim. More directly, re-instrument the same AndroTest apps with exhaustive runtime method logging and recompute the 57.44% figure; a large drop would show the result depends on the specific sample or instrumentation rather than o","supporting_citations":[],"review_version":1}