Pith. sign in

REVIEW 4 major objections 5 minor 26 references

Practical Source Code Recovery from Binary Functions Using Anchor-Based Retrieval and LLM Reasoning

T0 review · 4 major / 5 minor · reviewed 2026-07-13 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A pipeline of reverse-engineering anchors, database search, and LLM re-ranking recovers the true source for most functions in a stripped binary when that source is in a clean corpus.

desk verdict Clean systems pipeline with real 95% coverage on a curated index and an honest collapse on GitHub; retrieval—not the LLM—is the load-bearing limit. read the letter →

arxiv 2607.09452 v1 pith:LRPDDUO5 submitted 2026-07-10 cs.SE cs.AI

Charles Edward Gagnon , Steven H. H. Ding , Philippe Charland , Benjamin C. M. Fung This is my paper
classification cs.SEcs.AI
keywords binary-to-sourcematchingreverseengineeringsourcecoderecoveryanchor-basedretrievallargelanguagemodelsbinaryanalysisdecompilation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Compilers erase names, structure, and comments, so reverse engineers are left with opaque assembly. This paper argues that much of that lost source can be recovered by searching rather than inventing: extract distinctive anchors (strings, constants, external calls, names) from a stripped binary function, retrieve candidate source files from a large code database, narrow them to function snippets, and let a language model re-rank the candidates against the disassembly and decompilation. On a stripped, optimized tcpdump binary against a curated open-source database the method maps source functions that cover 95.2% of the assembly instructions. Against public GitHub search the same pipeline still recovers roughly a third of the instructions on average, the main failures being retrieval misses. The practical claim is that high-quality source corpora plus lightweight LLM verification can turn reverse engineering of reused open-source code into a largely automatic lookup problem.

What carries the argument

Anchor-based binary-to-source retrieval: compact anchors (strings, constants, external calls, names) drive inverted-index file retrieval; dense features (disassembly, decompilation, metadata) feed an LLM that scores, reasons about, and re-ranks the top candidate snippets, with high-confidence matches later reused as anchors for callers.

What would settle it

Re-run the identical pipeline on the same stripped tcpdump binary but force every inverted-index query to omit the true source file (or replace the curated corpus with a deliberately incomplete index); if instruction coverage collapses toward the zero-anchor baseline, the central claim is falsified.

Watch

Extended reading notes

Core claim

When the correct source function is present in a clean, well-indexed database, anchor-based retrieval followed by LLM re-ranking of decompiled code and disassembly recovers source-level matches covering 95.2% of assembly instructions in a stripped, O2-optimized tcpdump binary (Hit@1 0.856, Hit@3 0.876). Performance drops sharply under noisy external search because many correct files never reach the re-ranker.

Load-bearing premise

The correct or sufficiently similar source function must already sit inside the backing database and be returned among the top candidates; if retrieval misses, the language model cannot recover the match.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a practical pipeline that recovers source-level function implementations for stripped binary functions by combining Ghidra-based anchor extraction (strings, constants, external calls, names), inverted-index retrieval over a source corpus, snippet localization with macro/inlining heuristics, and LLM re-ranking of candidates using disassembly, decompiled code, and metadata. Confident matches can be propagated as call-graph anchors in later passes. On a stripped O2 x86_64 tcpdump binary against a curated database of 8,061 projects, the method reports Hit@1 0.856, Hit@3 0.876, MRR 0.866, and 95.2% assembly-instruction coverage; against GitHub code search on BusyBox, sqlite3, curl, and dropbear it reports average Hits@1 0.283 and 35.5% coverage, with database miss as the dominant failure mode.

Significance. If the results hold under stronger evaluation, the work is a useful systems contribution for reverse engineering: it targets exact source recovery rather than pseudocode, is database-agnostic, and produces interpretable LLM rationales. The controlled tcpdump result (95.2% instruction coverage with a clean index) and the honest GitHub degradation analysis (retrieval miss ~53%, Hits@1 among retrieved ~0.59, re-ranker lift over non-LLM baseline) are practically informative. Strengths include a clear ground-truth construction path (compile with symbols, nm map, strip, evaluate), explicit failure-mode reporting, and a secondary-pass idea that could compound recovery. Novelty is primarily integrative rather than algorithmic; impact depends on evaluation rigor and retrieval quality.

major comments (4)
  1. [§4.1, Table 1] §4.1 defines a correct match solely as name+file agreement with the nm-derived address map, then defines Coverage as the fraction of assembly instructions in such matched functions (Table 1: 95.2%). That identity check does not verify that the retrieved snippet is the implementation that produced the binary body (version drift, near-duplicates, wrappers, or post-O2 inlining/shims can share name+file while differing in body). The LLM is asked for semantic reasoning (Fig. 2; §3.4), yet evaluation never audits reasoning correctness or body-level equivalence against the binary. Please strengthen the headline metric with at least one of: (i) line-range / AST / token-level agreement with the debug map, (ii) a manually audited sample of matched functions, or (iii) an explicit bound on how often name+file can succeed without implementation match.
  2. [§3.4; §4] §3.4 and the abstract claim that confident matches become anchors for callers in later passes and that this recovers functions with few local anchors. The experiments (§4.1–4.2, Tables 1–2) report only end-to-end metrics; there is no ablation of single-pass vs multi-pass recovery, no count of functions recovered only via propagated anchors, and no measurement of error propagation from false-positive seeds. Without that, the iterative mechanism remains an untested design claim relative to the central recovery numbers.
  3. [§2; Table 2] Related work (§2) cites RESource, BinPro, B2SFinder, and SBOM/library-detection systems, but experiments compare only against an internal “Hits @ 1 no re-ranking” baseline (Table 2). There is no head-to-head on the same binaries/databases against prior binary-to-source matchers or against strong binary-similarity + source lookup baselines. For a systems claim of practical recovery, at least one external baseline (or a clear impossibility argument) is needed to locate the contribution.
  4. [§4.1; §5] The controlled high-coverage claim rests on a single stripped O2 x86_64 tcpdump binary (§4.1). Architecture, optimization level, language dialect, and project structure are fixed; zero-anchor failures (101/815) are noted but not stress-tested on crypto/DSP-style kernels beyond the limitations discussion (§5). Either expand the controlled corpus (multiple projects/opt levels) or narrow the claim language so 95.2% is not read as general binary recovery performance.
minor comments (5)
  1. [§3.1–§4.2] Several section headings appear with internal spaces (“F eature Extraction”, “F unction similarity reasoning”, “Real-W orld Use Case”), likely PDF extraction or typesetting artifacts; clean for camera-ready.
  2. [Abstract] Abstract and early prose have run-together tokens in the provided text (e.g., “candidatestolikelyfunctionsnippets”, “binary-to-sourcecodematching”); verify spacing in the source.
  3. [Table 2] Table 2 reports False Positives and Database Miss but does not define the decision threshold or how “false positive” is labeled when the correct file is absent; a short formal definition would help.
  4. [§4.1–4.2] LLM choice and decoding settings (GPT-5.5 vs GPT-5 mini, temperature, schema enforcement) are only lightly specified; non-determinism of re-ranking is not quantified with repeated runs.
  5. [§5, Fig. 3] Fig. 3 usefully shows decompiler ambiguity for field accessors; consider quantifying how often such near-ties occur in the tcpdump/GitHub candidate sets.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: evaluation metrics rest on independent nm ground-truth maps, not on quantities fitted or defined by the pipeline itself.

full rationale

This is an empirical systems paper, not a first-principles derivation. The pipeline (Ghidra anchor extraction, inverted-index retrieval, snippet localization, LLM re-ranking, optional secondary-pass anchors) is a constructive engineering method whose success is measured against an external address-to-function map obtained by compiling with debug symbols, running nm, then stripping. Correctness is defined as name+file agreement with that map; coverage is the fraction of assembly instructions belonging to such matched functions. Those metrics are not fitted parameters of the method, nor are they defined in terms of the LLM scores or anchor densities. Related-work citations to prior binary-to-source or SBOM work (including some with author overlap) are background and do not force the reported Hit@1/MRR/coverage numbers. The paper itself reports failure modes (zero-anchor functions, GitHub database misses) rather than hiding them inside a self-referential definition. Any concern that name+file matching is a weaker semantic check than the claim implies is an evaluation-validity issue, not circularity. Score 0 is therefore the honest finding.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The central empirical claims rest on standard reverse-engineering tooling assumptions, the presence of the target source in the backing corpus, and the reliability of commercial LLM judgment. No free parameters are fitted to produce the coverage numbers; the main load-bearing premises are domain assumptions about open-source reuse and retrieval completeness.

free parameters (2)
  • top-k candidates for LLM re-ranking = 12
    k is set to 12 in the experiments; the paper does not justify or ablate this choice, yet it bounds what the re-ranker can recover.
  • LLM model choice and temperature / decoding settings
    GPT-5.5 used for the curated experiment and GPT-5 mini for the GitHub experiment; exact prompts, temperature, and sampling parameters are not reported and affect match scores.
assumptions (5)
  • domain assumption Compilers are irreversible; perfect recovery of all original identifiers and structure is impossible, so the goal is identification of an existing source function rather than synthesis.
    Stated in the introduction and used to motivate retrieval over pure decompilation.
  • domain assumption A large fraction of modern binaries reuses open-source code that can be found in a searchable corpus.
    Cited via Census II and related studies; required for the method to be useful on proprietary or malware samples.
  • domain assumption Ghidra-extracted strings, constants, external calls, and names are sufficiently distinctive anchors for inverted-index retrieval.
    Core of §3.1–3.2; the paper later shows that zero-anchor functions fail completely.
  • ad hoc to paper An LLM can reliably compare decompiled pseudocode / disassembly against candidate source snippets and produce calibrated match scores and rationales.
    Assumed in §3.4 and validated only by the reported Hit rates; no independent calibration study is provided.
  • ad hoc to paper Function-boundary detection, macro use-site expansion, and simple inlining heuristics correctly localize the relevant source snippet.
    Described in §3.3; correctness of these heuristics is not separately measured.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Practical Source Code Recovery from Binary Functions Using Anchor-Based Retrieval and LLM Reasoning." pith.science (2026). https://pith.science/paper/LRPDDUO5

@misc{pith2026260709452,
  author       = {Pith},
  title        = {Pith review of: Practical Source Code Recovery from Binary Functions Using Anchor-Based Retrieval and LLM Reasoning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LRPDDUO5}},
  note         = {Machine review of arXiv:2607.09452}
}
read the original abstract

We present a practical pipeline for recovering source code from stripped binary functions by combining reverse engineering, anchor-based source code retrieval, and large language model reasoning. Our binary-to-source-code retrieval method attempts to identify the source function from a source code database, rather than generating approximate decompiled pseudocode. It extracts anchors such as strings, constants, external calls, and available function names using Ghidra, retrieves candidate files via an inverted-index search database, narrows candidates to likely function snippets, and re-ranks them with a large language model (LLM) based on disassembly, decompiled code, and source metadata. Confident matches can also serve as anchors in later passes. In an evaluation backed by our high-fidelity source code database on a stripped, optimized tcpdump binary, our proposed binary-to-source matching method achieves 95.2% assembly instruction coverage. Experiments on a GitHub-based retrieval database showed lower performance with 35.5% instruction coverage on average, mainly due to retrieval misses. These results show that source-level binary recovery excels with high-quality databases and remains a useful tool in noisy environments.

Figures

Figures reproduced from arXiv: 2607.09452 by the authors.

Figure 1
Figure 1. Overview of our method. An unknown function is first disassembled and its features are extracted. The anchors are used to perform the initial database query. The disassembled code, along with information-dense features, is used to re-order the top-k candidates. In our experiments, k is set to 12 [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Example of successful retrieval from decompiled code to source code. The lan￾guage model is able to precisely explain the reasoning behind the match [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Comparison of two structure-field accessors and their corresponding Ghidra decompilations. The decompiled functions are nearly identical, differing only in the accessed field offset (0x08 versus 0x16). Second, our method is limited by the ambiguity of decompiled code. The lan￾guage model re-ranker makes extensive use of the decompiled code to compare query results and correctly find the most relevant match. An issue… view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

26 extracted references · 4 canonical work pages

  1. [1]

    Addison-Wesley, 2 edn

    Aho, A.V., Lam, M.S., Sethi, R., Ullman, J.D.: Compilers: Principles, Techniques, and Tools. Addison-Wesley, 2 edn. (2006)

    2006

  2. [2]

    Barr, M.: Embedded C Coding Standard.https://barrgroup.com/ embedded-systems/books/embedded-c-coding-standard1(2018), bARR-C:2018, Barr Group

    2018

  3. [3]

    Beninger, M., Charland, P., Ding, S., Fung, B.: ERS0: Enhancing Military Cy- bersecurity with AI-Driven SBOM for Firmware Vulnerability Detection and As- set Management. pp. 141–160 (2024).https://doi.org/10.23919/CyCon62501. 2024.10685598

    work page doi:10.23919/cycon62501 2024

  4. [4]

    Cifuentes, C., Gough, K.J.: Decompilation of binary programs. Softw. Pract. Exp. 25(7), 811–829 (1995).https://doi.org/10.1002/spe.4380250706

    work page doi:10.1002/spe.4380250706 1995

  5. [5]

    In: Proc

    Ding, S.H.H., Fung, B.C.M., Charland, P.: Asm2Vec: Boosting Static Representa- tion Robustness for Binary Clone Search against Code Obfuscation and Compiler Optimization. In: Proc. IEEE S&P. pp. 472–489 (2019).https://doi.org/10. 1109/SP.2019.00003

    arXiv 2019

  6. [6]

    In: Proc

    Dong,C.,Guo,J.,Yang,S.,Li,Y.,Fang,D.,Xiao,Y.,Chen,Y.,Sun,L.:Advancing Binary Code Similarity Detection via Context-Content Fusion and LLM Verifica- tion. In: Proc. ASE. pp. 304–316 (2025).https://doi.org/10.1109/ASE63991. 2025.00033

    doi:10.1109/ase63991 2025

  7. [7]

    In: Proc

    Dong, C., Li, S., Yang, S., Xiao, Y., Wang, Y., Li, H., Li, Z., Sun, L.: LibvDiff: Library Version Difference Guided OSS Version Identification in Binaries. In: Proc. ICSE (2024).https://doi.org/10.1145/3597503.3623336

    doi:10.1145/3597503.3623336 2024

  8. [8]

    Gagnon, C.E., Ding, S.H.H., Charland, P., Fung, B.C.M.: Beyond Embed- dings: Interpretable Feature Extraction for Binary Code Similarity (2025), arXiv:2509.23449

    arXiv 2025

Show all 26 references
  1. [9]

    org/binutils/docs/binutils/nm.html, part of GNU Binutils

    GNU Project: nm: List symbols from object files (2026),https://sourceware. org/binutils/docs/binutils/nm.html, part of GNU Binutils

    2026

  2. [10]

    Hex-Rays: IDA Pro.https://hex-rays.com/ida-pro(2025-02-28), version 9.1

    2025

  3. [11]

    Gagnon et al

    Li, B.: Incompressible Knowledge Probes: Estimating Black-Box LLM Parameter Counts via Factual Capacity (2026), arXiv:2604.24827 12 C. Gagnon et al

    2026 arXiv

  4. [12]

    In: Proc

    Li, X., Qu, Y., Yin, H.: PalmTree: Learning an Assembly Language Model for Instruction Embedding. In: Proc. ACM CCS. pp. 3236–3251 (2021).https://doi. org/10.1145/3460120.3484587

    2021 doi

  5. [13]

    Massarelli, L., Luna, G.A.D., Petroni, F., Querzoni, L., Baldoni, R.: SAFE: Self- Attentive Function Embeddings for Binary Similarity (2019), arXiv:1811.05296

    2019 arXiv

  6. [14]

    Miyani, D., Huang, Z., Lie, D.: BinPro: A Tool for Binary Source Code Provenance (2017), arXiv:1711.00830

    2017 arXiv

  7. [15]

    In: Proc

    Musseau, J., Meyers, J.S., Sieniawski, G.P., Thompson, C.A., German, D.: Is Open Source Eating the World’s Software? Measuring the Proportion of Open Source in Proprietary Software Using Java Binaries. In: Proc. MSR. pp. 561–565 (2022). https://doi.org/10.1145/3524842.3528473

    2022 doi

  8. [16]

    Nagle, F., Dana, J., Hoffman, J., Randazzo, T., Zhou, K.: Census II of Free and Open Source Software – Application Libraries. Tech. rep., Linux Foundation Re- search (2022).https://doi.org/10.70828/KHEH5209

    2022 doi

  9. [17]

    National Security Agency: Ghidra Software Reverse Engineering Frame- work.https://github.com/NationalSecurityAgency/ghidra(2025-07-31), ver- sion 11.4.1

    2025

  10. [18]

    Pereira, D., Molloy, C., Acharya, S., Ding, S.H.H.: Automating SBOM Generation with Zero-Shot Semantic Similarity (2024), arXiv:2403.08799

    2024 arXiv

  11. [19]

    In: Foundations and Practice of Security

    Rahimian, A., Charland, P., Preda, S., Debbabi, M.: RESource: A Framework for Online Matching of Assembly with Open Source Code. In: Foundations and Practice of Security. pp. 211–226 (2013)

    2013

  12. [20]

    In: Proc

    Tan, H., Luo, Q., Li, J., Zhang, Y.: LLM4Decompile: Decompiling Binary Code with Large Language Models. In: Proc. EMNLP. pp. 3473–3487 (2024).https: //doi.org/10.18653/v1/2024.emnlp-main.203

    2024 doi

  13. [21]

    IEEE Internet Things J.12(24), 51842– 51853 (2025).https://doi.org/10.1109/JIOT.2025.3579231

    Wan, B., Wang, S., Wei, Z., Huang, J., Hu, C.: Binary Code Similarity Detection via LLM-Based Source Code Conversion. IEEE Internet Things J.12(24), 51842– 51853 (2025).https://doi.org/10.1109/JIOT.2025.3579231

    2025 doi

  14. [22]

    In: Proc

    Wang, H., Gao, Z., Zhang, C., Sha, Z., Sun, M., Zhou, Y., Zhu, W., Sun, W., Qiu, H., Xiao, X.: CLAP: Learning Transferable Binary Code Representations with Natural Language Supervision. In: Proc. ISSTA. pp. 503–515 (2024).https: //doi.org/10.1145/3650212.3652145

    2024 doi

  15. [23]

    Xu, H., Wang, S., Xing, Y., Feng, P., Wang, H., Li, Q., Chen, S., Sun, K.: BinProv: Binary Code Provenance Identification without Disassembly. pp. 350–363 (2022). https://doi.org/10.1145/3545948.3545956

    2022 doi

  16. [24]

    Yu, S.: Accurate and Efficient SBOM Generation for Software Supply Chain Secu- rity. Ph.D. thesis, University of California, Riverside (2024)

    2024

  17. [25]

    Yu, Z., Cao, R., Tang, Q., Nie, S., Huang, J., Wu, S.: Order Matters: Semantic- Aware Neural Networks for Binary Code Similarity Detection. Proc. AAAI Conf. Artif. Intell.34(01), 1145–1152 (2020).https://doi.org/10.1609/aaai.v34i01. 5466

    2020 doi

  18. [26]

    In: Proc

    Yuan, Z., Feng, M., Li, F., Ban, G., Xiao, Y., Wang, S., Tang, Q., Su, H., Yu, C., Xu, J., Piao, A., Xuey, J., Huo, W.: B2SFinder: Detecting Open-Source Software Reuse in COTS Software. In: Proc. ASE. pp. 1038–1049 (2019).https://doi.org/ 10.1109/ASE.2019.00100

    2019 doi

Pith tools

Reviewed July 13, 2026 · model on record in the stance chip above.