{"id":"317e0cde-64ac-412f-9425-b7eee2851df1","arxiv_id":"2507.07246","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":10,"one_line_summary":"Disa uses self-attention over superset instructions to identify function entries, true instructions, and memory block boundaries, improving disassembly robustness and CFG recovery on obfuscated binaries.","lead":"The paper presents Disa, a deep-learning tool that finds function and instruction boundaries in compiled binaries, and can also locate the boundaries of data structures in memory. It combines a transformer model over candidate instructions with static value-tracking analysis to build more accurate control-flow graphs for stripped and obfuscated programs.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"DWARF-derived memory block boundary ground truth is unverified; if prologue analysis or DWARF locations are wrong, T3 labels, memory-block precision, and the AICT reduction claims are all biased.","rationale":"The paper's central novelty is task T3 (memory block boundary identification) and its integration into BPA. The T3 pipeline depends entirely on DWARF as ground truth. The reader's weakest assumption identifies exactly this dependency. My stress-test agrees: this is the single most load-bearing concern because it affects training, evaluation, and the reference baseline simultaneously. A concrete check can settle it: compare DWARF-derived boundaries against an independent oracle on a sample. If the check reveals no mismatches, the concern is resolved and the T3 claims stand; if it reveals mismatches, the reported improvements are suspect and the paper would need re-evaluation. I do not think the concern alone justifies rejection before the check, but it justifies the CONDITIONAL verdict the reader already assigned; no other concern (DeepDi baseline, incomplete Pin ground truths, uneven AICT) is as central because those affect only parts of the evaluation, whereas a DWARF flaw contaminates the entire T3 contribution and the headline CFG improvement.","tokens_in":62962,"tokens_out":9852,"duration_ms":106300,"concrete_test":"Select a stratified sample of about 20 functions from SPx86test across O0-O3, including frame-pointer-omitted and stack-realignment cases. Independently reconstruct memory block boundaries from source (SPEC benchmark C code is available) or via a second oracle (e.g., Dyninst variable/type recovery, or a debugger with optimized -g). Compare these independent boundaries against the DWARF extraction pipeline of Section 3.3.1. If any mismatch is found, retrain the T3 model and rerun bVTR on the same test binaries using corrected ground truth, then recompute memory-block precision/recall/F1 and the BPA-Disa AICT comparison of Table 3. If the 18.5% precision gain and the 4.4% AICT reduction are not preserved, the central CFG claim does not survive.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.3.1 assumes that DWARF debugging information provides correct memory block boundaries, stating: 'Assuming that pointer arithmetic should not cross the boundary of source-level data structures.' This assumption is load-bearing for the entire T3 contribution: it generates the BRel training labels (iVTR), the memory-block precision evaluation in Section 5.5.1, and the BPADWARF upper-bound baseline in Table 3. The paper itself notes that DWARF location information is sometimes missing (e.g., when DW_AT_location contains pointer dereferences that cannot be resolved statically) and that a custom prologue analysis converts DWARF offsets to initial-esp offsets. If the prologue analysis mis-handles frame-pointer omission, stack realignment, or unusual prologues, every stack boundary for that function is shifted. Since the same DWARF oracle is used to train the model and to score it, systematic DWARF errors (missing, shifted, or spurious boundaries) are baked into the BRel labels; the model could be learning the DWARF extraction heuristics rather than true block boundaries. The comparison against BPA is then judged by the same potentially flawed oracle, so the reported 18.5% precision gain and 4.4% AICT reduction could shrink or disappear if the oracle were corrected. This is the weakest link because the T1/T2 claims are grounded in binary symbols or Pin-instrumented instructions, which are more direct, whereas T3 is the novel contribution and the basis for the CFG claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents Disa, a learning-based static disassembly framework built on a transformer encoder over superset instructions. Disa addresses three tasks: function entry-point identification (T1), true instruction identification (T2), and a newly introduced memory block boundary identification (T3), where a BRel-instruction classifier is combined with an intra-procedural value-tracking analysis (bVTR) to recover memory block boundaries. These boundaries are then fed into the block-memory-model points-to analysis BPA to refine indirect call targets and improve CFG recovery. The evaluation compares Disa with XDA, biRNN, DeepDi, IDA Pro, Ghidra, and Binary Ninja on non-obfuscated, real-world, and obfuscated binaries (desync-cc, OLLVM, Tigress, VMProtect, Themida), and compares BPADisa with BPA and a DWARF-oracle version of BPA. The headline results are 9.1% and 13.2% F1 improvements over prior deep-learning approaches on desync-cc- and Tigress-obfuscated binaries, 18.5% memory-block precision improvement over BPA, and a 4.4% AICT reduction.","tokens_in":63287,"tokens_out":5308,"duration_ms":63910,"significance":"The T3 contribution is potentially significant: using deep learning to predict memory block boundaries, rather than instruction/function boundaries alone, is a new direction that directly targets a known bottleneck in binary-level points-to analysis. The paper is also strong in evaluation breadth: multiple compilers, optimization levels, file formats, obfuscators, and real-world corpora, with held-out splits and a publicly available implementation. The ablation study with BPA variants and the comparison against a DWARF-based upper bound are useful. However, the strength of the central claims is currently limited by three load-bearing concerns: the T3 labels and evaluation share the same DWARF-derived oracle, the obfuscated instruction ground truths are incomplete and use a permissive one-byte overlap rule, and the comparison with DeepDi is against a frozen reference model not trained on the same data. These issues are addressable, so the work is worth revising rather than rejecting.","major_comments":[{"comment":"The DWARF-derived memory block boundary ground truth is used both to generate BRel training labels (via iVTR in Algorithm 2) and to score Disa's memory block predictions and the BPADWARF upper bound. Section 3.3.1 explicitly assumes that pointer arithmetic does not cross source-level data structure boundaries, and also reports that DWARF location information is sometimes missing and requires a custom prologue analysis. If the prologue analysis mis-handles frame-pointer omission, stack realignment, or unusual prologues, all stack boundaries for a function shift, and the same shift is baked into training, evaluation, and the BPADWARF reference. The model could therefore be learning the DWARF-extraction heuristics rather than true block boundaries, and the reported 18.5% precision gain and 4.4% AICT reduction could partly reflect this shared-oracle bias. Please provide a sensitivity analysis: e.g., perturb the prologue offset by one word per function and report the resulting change in BRel label noise, memory-block precision, and AICT; additionally, manually audit a random sample of SPx86_test blocks against source code and report the fraction of DWARF boundaries that are missing or displaced. Without such validation, the T3 claim should be framed as aligned with DWARF-based boundaries rather than as ground-truth memory block recovery.","section":"§3.3.1"},{"comment":"The instruction ground truths for obfuscated binaries are sound-but-incomplete Pin traces, and a predicted instruction is treated as positive if it overlaps at least one byte with a runtime-reached true instruction; predictions in unknown slots are excluded from the denominator. This can inflate T2 precision/F1: a model that emits many incorrect instructions in unreached code, or whose boundaries are shifted but overlapping, is not penalized. Since the paper makes robustness claims on obfuscated binaries and uses T2 in the integrated BPA ablation, please report additional conservative metrics: (a) treating unknown regions as negative, (b) requiring exact instruction-boundary matches, and (c) segment-level overlap rather than one-byte overlap. Also report the fraction of test-set bytes that fall into unknown slots for each obfuscator and optimization level, and show how F1 changes when those slots are excluded versus included.","section":"§5.1.1"},{"comment":"DeepDi is compared only as a reference model obtained through its released APIs, not retrained or finetuned on the same 90% training split used for Disa-SN and the other baselines. The claimed superiority over DeepDi (including the 9.1% F1 gain on desync-cc binaries) may therefore be attributable to differences in training data distribution, compiler coverage, or hyperparameters rather than to the proposed architecture. To support the 'outperforms prior deep-learning approaches' claim, please either train DeepDi on the identical SN training split (or a clearly documented equivalent), or explicitly narrow all such claims to 'Disa-SN vs. the released DeepDi reference model' and add a discussion of how training-data differences might confound the comparison.","section":"§5.2"}],"minor_comments":[{"comment":"The claim that Disa is 'the first disassembly approach that can identify memory block boundaries at the binary level' should be positioned against prior binary type and data-structure recovery work such as TypeMiner and OSPREY; the current text asserts novelty without describing how those approaches differ in task formulation or evaluation.","section":"Contribution 2"},{"comment":"The paper states that a reevaluation of BPA found an implementation issue that changes the original results, but it does not specify the fix. Please document the corrected BPA behavior, the affected code, or release a patch, since the baseline numbers in Table 3 and the ablation depend on this correction.","section":"§5.5.2"},{"comment":"Several heatmaps (Figs. 6, 7, 10, 12, 15, 17) are difficult to read at the available resolution because the underlying cell values are not printed numerically. Please provide the corresponding numeric tables in an appendix so that readers can verify the reported F1, precision, recall, and AICT values.","section":"§5.3 / §5.4 / §5.5"},{"comment":"The training procedure reports only that XDA finetuning takes several days per epoch; please report the number of epochs, early-stopping criterion, learning rate, and random seed for all deep models to support reproducibility.","section":"Appendix A"}],"recommendation":"major_revision","confidential_remarks":"The T3 novelty is the least externally validated part of the paper. I would encourage the editor to ask the authors for source-level or otherwise independent validation of a subset of SPx86_test memory blocks, because without it the central 'memory block boundary identification' claim rests on a single DWARF-based oracle used for both training and evaluation. This is fixable within the scope of a revision, but it is currently the main risk to the paper's significance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know this paper is worth taking seriously but not uncritically. It is a substantial experimental effort: a transformer-based disassembler with three tasks, and the genuinely new one is T3, learning to identify memory-block-boundary-related instructions to feed a block-memory points-to analysis. The evaluation is wide—multiple compilers, ISAs, and obfuscators, plus an ablation linking each task to CFG quality. The code and data are public.\n\nWhat is actually new: T3. No prior learning-based disassembler attempts to recover memory block boundaries at the binary level, and integrating that with BPA to reduce indirect call targets is a useful direction. The T1/T2 architecture, superset instructions plus transformer self-attention, is a reasonable incremental step over XDA and DeepDi; the gains on desync-cc and tigress are believable, though the one-byte-overlap positive criterion in the Pin-based ground truth makes those F1 numbers less sharp than they look.\n\nThe biggest soft spot is the DWARF-derived ground truth for T3. The paper assumes pointer arithmetic does not cross source-level data structure boundaries, then extracts boundaries from DWARF with a hand-written prologue analysis, and uses the same oracle for training and for scoring memory-block precision and the BPADWARF upper bound. If that oracle is biased—and the paper admits DWARF locations are sometimes missing or unresolvable—the T3 labels and the precision gains inherit the bias. That does not kill the AICT comparison with BPA, which is grounded in Pin traces, but it does mean the 18.5% precision claim is only as good as the DWARF extraction.\n\nTwo other things to fix. DeepDi is evaluated as a fixed reference model, not retrained, which puts Disa at an advantage. And the authors' note that their BPA re-evaluation deviates from the original results in [32] due to an implementation issue needs a clear explanation; readers need to know whether the baseline is the original BPA or a modified variant.\n\nThe AICT results are uneven; the headline 4.4% average masks some negative results (h264ref-O2, gcc-O0/O3), and the ablation shows recall drops in some configurations (milc-O0). These are minor but worth discussing.\n\nWho this is for: binary analysis and reverse engineering researchers, especially those working on CFG recovery and points-to analysis. It deserves a serious referee—the novel task is well-motivated, the experiments are extensive, and the soft spots are addressable. I'd send it out with a request that the authors validate the DWARF extraction quality, retrain or properly constrain DeepDi, and clarify the BPA baseline discrepancy. Conditional accept.","headline":"Worth a serious referee, but push on the DWARF ground truth and the DeepDi/BPA baselines.","tokens_in":63865,"tokens_out":4335,"would_cite":true,"duration_ms":46115,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper argues that all three hard boundary decisions in static disassembly—function entries, true instructions, and memory-block boundaries—can be learned as binary classifications over superset-decoded instructions, and that the…","keywords":["static disassembly","function entry-point identification","memory block boundary","self-attention","superset disassembly","indirect call target","control flow graph recovery","binary obfuscation"],"falsifier":"Compile a corpus of stripped binaries from code that intentionally performs pointer arithmetic across adjacent source-level objects (e.g., container_of macros, flexible array members, or manual memory pooling), then compare Disa's memory-block predictions against manually verified boundaries; if precision drops to roughly the heuristic baseline's level on those binaries, the DWARF-ground-truth assumption is the reason.","tokens_in":62730,"feed_emoji":"🛡️","tokens_out":7258,"duration_ms":77292,"temperature":0.7,"pith_summary":"Disa sets out to show that the three hardest boundary decisions in static disassembly—where functions begin, which decoded instructions are real, and where memory blocks corresponding to source-level data structures start and end—can all be learned as binary classifications over a superset of decoded instructions. The paper's core move is to feed each superset instruction's field encoding through a transformer encoder, letting multi-head self-attention capture long-range correlations that byte-level models miss, and then to feed the learned memory-block boundaries into a block-memory points-to analysis. If the approach is right, obfuscated binaries that desynchronize linear disassembly become tractable, and static CFG recovery resolves indirect calls more precisely: the paper reports 9.1% and 13.2% F1 gains in function-entry recovery on desynchronized and source-level-obfuscated binaries, 18.5% higher memory-block precision, and 4.4% fewer average indirect call targets.","feed_headline":"Disassembler learns memory-block boundaries to tighten CFGs","feed_subtitle":"Function-entry F1 gains 9.1-13.2% on obfuscated binaries; memory-block precision rises 18.5% and CFG edges tighten.","key_machinery":"The carrying mechanism is the superset instruction sequence $SI$ with learned per-field and positional embeddings, passed through a 6-layer transformer encoder with 8 attention heads and embedding dimension 384. For T3, the encoding adds the memory region $\\mathrm{Rgn}$ and relative displacement $\\mathrm{Disp}$, so attention learns which memory-access patterns touch compound-data-structure boundaries. The second piece is boundary-targeted value tracking (bVTR): an intra-procedural value-set analysis that converts each predicted BRel instruction's memory operand into offsets from the initial $\\mathtt{esp}$ or from a global section, producing the actual memory-block boundaries consumed by the block-memory points-to analysis.","core_discovery":"The central claim is that disassembly reduces to classifying instructions in a superset, and that the same transformer encoder can perform all three classifications. For tasks T1 and T2, instructions are encoded as integer triples (opcode, ModRM, SIB); for T3, memory-access and branching instructions are encoded as quintuples that add a memory region and a relative displacement. The model labels function-entry instructions, true instructions, and block-boundary-related (BRel) instructions, and a lightweight intra-procedural value-tracking pass converts BRel labels into concrete block boundaries as offsets from a function's initial stack pointer or from global section starts. Fed into block-memory points-to analysis, these boundaries replace heuristic blocks and reduce the ambiguity of indirect-call targets, yielding assembly-level CFGs, which Disa claims no previous deep-learning disassembler did.","pith_inferences":["The paper's instruction-level BRel prediction cannot distinguish multiple block boundaries touched by one instruction, so array and loop accesses can smear several candidate boundaries into one label; an operand-level or element-level supervision signal is the natural next experiment.","The DWARF-ground-truth dependency suggests a boundary curriculum: train on debug-rich binaries, then distill onto stripped binaries using the model's own high-precision BRel predictions as pseudo-labels.","The same BRel idea should transfer to object-boundary consumers beyond CFG recovery, such as memory sanitization for binary fuzzing or type-assisted control-flow integrity.","A cleaner isolation test of T3's contribution would hold function and instruction boundaries fixed and vary only the memory-block source; the paper's ablation varies T1 and T2 together, so the marginal value of learned boundaries alone is not fully separated."],"forward_implications":["If Disa's classification of superset instructions is correct, function and instruction boundary recovery no longer needs per-architecture disassembly heuristics or a hand-built instruction-flow graph for the deep model.","If T3's memory-block boundaries are as accurate as reported, block-memory points-to analysis can drop its conservative heuristic blocks, sharpening points-to sets and removing spurious indirect-call edges from binary CFGs.","On binaries obfuscated by disassembly desynchronization and source-level transforms, function-entry F1 gains of 9.1% and 13.2% imply that instruction-level context, not raw byte statistics, is what lets a disassembler resist those obfuscations.","Because all three tasks share one encoder architecture and differ only in input fields and labels, the same pipeline can be retrained for new boundary-like classification tasks without redesign."],"supporting_citations":[{"why":"Supplies the block-memory-model points-to analysis that Disa feeds with learned boundaries, and the baseline for memory-block and AICT comparisons.","marker":"[32]"},{"why":"Byte-level BERT disassembly baseline that Disa must beat on function-entry and instruction identification.","marker":"[46]"},{"why":"Superset-instruction R-GCN disassembler that gives the superset classification idea and serves as a main comparison baseline.","marker":"[69]"},{"why":"Neural function-boundary recognizer that provides an earlier learning-based baseline for function entry-point identification.","marker":"[53]"},{"why":"Establishes the superset disassembly principle that underlies Disa's decode-all-instructions step.","marker":"[11]"},{"why":"desync-cc obfuscation technique whose disassembly-desynchronized binaries stress function-entry recovery.","marker":"[30]"},{"why":"Tigress source-level obfuscator used to build the obfuscated test set for robustness evaluation.","marker":"[20]"},{"why":"Transformer encoder architecture whose multi-head self-attention is the core of Disa's model.","marker":"[62]"}],"fun_headline_variants":["Transformer disassembler sharpens function and instruction boundaries","Disa uses self-attention to improve disassembly on obfuscated binaries","Learning-based disassembler with attention cuts indirect-call targets","Attention-based disassembler tightens CFGs by learning block boundaries","Disa transformer improves function-entry F1 by 9.1-13.2% on obfuscated code"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise, stated in Section 3.3.1, is that memory-block boundaries extracted from DWARF debugging metadata are true boundaries, meaning pointer arithmetic never deliberately crosses source-level data structure boundaries; if a real binary violates that, the T3 training labels and measured precision gains inherit the error.","fun_headline_variants_meta":{"raw":{"variants":["Transformer disassembler sharpens function and instruction boundaries","Disa uses self-attention to improve disassembly on obfuscated binaries","Learning-based disassembler with attention cuts indirect-call targets","Attention-based disassembler tightens CFGs by learning block boundaries","Disa transformer improves function-entry F1 by 9.1-13.2% on obfuscated code"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001273,"raw_usage":{"total_tokens":5231,"prompt_tokens":994,"completion_tokens":4237,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":610,"completion_tokens_details":{"reasoning_tokens":4136}},"tokens_in":610,"tokens_out":4237,"duration_ms":31148,"temperature":1.0,"reasoning_tokens":4136,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T18:45:31.132248+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compile a corpus of stripped binaries from code that intentionally performs pointer arithmetic across adjacent source-level objects (e.g., container_of macros, flexible array members, or manual memory pooling), then compare Disa's memory-block predictions against manually verified boundaries; if precision drops to roughly the heuristic baseline's level on those binaries, the DWARF-ground-truth assumption is the reason.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the block-memory-model points-to analysis that Disa feeds with learned boundaries, and the baseline for memory-block and AICT comparisons."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Byte-level BERT disassembly baseline that Disa must beat on function-entry and instruction identification."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Superset-instruction R-GCN disassembler that gives the superset classification idea and serves as a main comparison baseline."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Neural function-boundary recognizer that provides an earlier learning-based baseline for function entry-point identification."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Establishes the superset disassembly principle that underlies Disa's decode-all-instructions step."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"desync-cc obfuscation technique whose disassembly-desynchronized binaries stress function-entry recovery."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Tigress source-level obfuscator used to build the obfuscated test set for robustness evaluation."},{"cited_title":"Gomez, Lukasz Kaiser, and Illia Polosukhin","cited_arxiv_id":null,"evidence_quote":"Transformer encoder architecture whose multi-head self-attention is the core of Disa's model."}],"review_version":1}