{"id":"ed1d764b-263d-4254-aaf0-e3a8b8b81b19","arxiv_id":"2606.20436","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":5.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"Multi-decompiler prompting improves LLM malware classification F1 by supplying complementary views of the same binary.","lead":"The paper finds that feeding LLMs decompiled pseudo-C from two different tools (Ghidra and RetDec) raises malicious-class F1 mainly via higher recall. A smart generalist might read it to see whether a zero-training-cost trick can make current AI tools more reliable for real malware triage.","discovery_kind":"new_application","skeptic_critique":{"model":"grok-4.3","headline":"Benchmark representativeness is the load-bearing assumption for generalizing the F1 gain to practical settings.","rationale":"The reader's weakest_assumption directly identifies the same point. No stronger internal inconsistency (e.g., in the matched decompilation or agreement analysis) is visible from the given text; the empirical claim is straightforward but its scope depends on benchmark coverage, which remains unspecified.","tokens_in":1646,"tokens_out":319,"duration_ms":12132,"concrete_test":"Release the exact list of malicious sample sources/hashes, per-family counts, and threat-behavior taxonomy used; recompute malicious F1 on a fresh hold-out set of 50+ recent samples from a disjoint family (e.g., post-2023 ransomware) using the same prompting protocol—if the dual-view recall gain drops below 5 points, the generalization claim does not hold.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim requires that the F1 improvement (driven by higher malicious recall) from dual Ghidra+RetDec views will hold for other binaries and threat behaviors. This rests on the curated benchmark being representative. The abstract states only that samples 'span a range of threat behaviors' with no reported counts, sources (e.g., specific malware corpora or families), compilation flags, binary sizes, or diversity metrics. If the malicious set is small, skewed toward particular families, or overlaps with LLM pretraining data, the observed complementarity (different errors between decompilers) may not replicate, making the practical triage recommendation unsupported.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper claims that providing matched pseudo-C views from both Ghidra and RetDec decompilers to LLMs improves malicious-class F1 (primarily via higher recall) over single-decompiler baselines on a curated benchmark of benign utilities and malicious programs; it further claims that agreement analyses show the decompilers make partially different errors, supporting the use of multi-view prompting as a training-free improvement for LLM-based malware triage.","tokens_in":1771,"tokens_out":361,"duration_ms":17473,"significance":"If the empirical results hold under proper controls and on representative data, the work would demonstrate a low-cost way to exploit decompiler complementarity for better LLM malware classification, which could be directly useful in practical reverse-engineering settings.","major_comments":[{"comment":"Abstract: the central claim of improved malicious F1 and complementary errors rests on an empirical measurement, yet the abstract (and by extension the reported evaluation) supplies no dataset size, sample sources, compilation details, model names/sizes, statistical significance tests, or controls for prompt length/ordering; without these the observed directional improvement cannot be verified or reproduced.","section":"Abstract"},{"comment":"Abstract: the benchmark is described only as spanning 'a range of threat behaviors' with no counts, families, binary sizes, or diversity metrics reported; this directly undermines the load-bearing assumption that the F1 gain (driven by recall) will generalize beyond the specific curated set.","section":"Abstract"}],"minor_comments":[{"comment":"The manuscript would benefit from an explicit experimental-setup subsection detailing the exact prompting templates, LLM inference parameters, and how agreement between decompiler views was quantified.","section":null}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the constructive feedback on the abstract. We agree that greater specificity in the abstract will improve clarity, reproducibility, and support for the claims. We address each comment below and will revise the abstract in the next version.","responses":[{"response":"The full manuscript reports dataset size, sample sources, compilation settings, model names/sizes, and evaluation procedures in Sections 3 and 4. We acknowledge that the abstract itself is insufficiently self-contained. We will revise the abstract to include these elements (dataset cardinality, model families, and note on significance testing). We will also add an explicit ablation on prompt length and ordering effects in the experiments section of the revised manuscript to confirm robustness.","revision_made":"yes","referee_comment":"[Abstract] Abstract: the central claim of improved malicious F1 and complementary errors rests on an empirical measurement, yet the abstract (and by extension the reported evaluation) supplies no dataset size, sample sources, compilation details, model names/sizes, statistical significance tests, or controls for prompt length/ordering; without these the observed directional improvement cannot be verified or reproduced."},{"response":"Section 3 of the manuscript already supplies the requested counts, families, size distributions, and diversity metrics for the curated benchmark. The abstract uses a concise phrasing for brevity, but we agree this limits immediate assessment of scope. We will revise the abstract to report these key statistics so that the generalization argument is better grounded.","revision_made":"yes","referee_comment":"[Abstract] Abstract: the benchmark is described only as spanning 'a range of threat behaviors' with no counts, families, binary sizes, or diversity metrics reported; this directly undermines the load-bearing assumption that the F1 gain (driven by recall) will generalize beyond the specific curated set."}],"tokens_in":1268,"tokens_out":398,"duration_ms":33054,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The main point is that prompting LLMs with both Ghidra and RetDec decompilations of the same binary raises malicious-class F1 over single-view baselines, driven mostly by better recall, and the two decompilers produce partially different errors on the test cases.\n\nThe work applies a straightforward multi-view idea to an existing LLM decompilation pipeline. The agreement analysis is a useful addition because it gives a concrete reason the combination helps instead of just showing an accuracy bump. Keeping everything training-free is also a practical choice for malware triage where you often only have the binary.\n\nThe soft spots are in the experimental reporting. The abstract supplies no sample counts, no exact F1 deltas, no statistical tests, and no checks for prompt-length or ordering effects. The benchmark is described only as spanning threat behaviors with no details on sources, families, sizes, or compilation settings. That makes the representativeness concern real: if the malicious set is small or narrow, the observed complementarity may not carry over to other binaries or threat types. Without those numbers the directional claim cannot be verified from the text provided.\n\nThis is aimed at people already working on LLM-assisted binary analysis or malware detection who want simple prompting tweaks. A reader looking for quick, no-training ideas might extract the multi-view angle, but only after seeing the full methods and results sections.\n\nI would send it to peer review if the complete manuscript supplies the dataset documentation, controls, and quantitative results that are currently absent; otherwise the gaps are large enough that it would need substantial revision first.","headline":"Dual decompiler views lift malicious F1 on the authors' benchmark via higher recall and some error complementarity, but missing dataset size, stats, and controls leave the practical claim hard to assess.","tokens_in":2219,"tokens_out":399,"would_cite":false,"duration_ms":27928,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"Feeding LLMs decompiled views from both Ghidra and RetDec raises malicious-class F1 over single-view baselines.","keywords":["malware classification","decompilation","large language models","Ghidra","RetDec","binary analysis","multi-view prompting","pseudo-C"],"falsifier":"Applying the identical single-view versus dual-view comparison to a new collection of binaries drawn from different malware families and observing no improvement or a decline in malicious-class F1.","tokens_in":2550,"feed_emoji":"🛡️","tokens_out":642,"duration_ms":19660,"temperature":0.7,"pith_summary":"The paper tests whether an LLM receives more useful signals for malware classification when it sees pseudo-C from two different decompilers instead of one. Because each decompiler applies its own heuristics and loses different details from the original binary, the two outputs can flag different indicators of malicious behavior. Experiments on a curated set of benign and malicious samples show consistent gains in F1 for the malicious class, driven mainly by higher recall. The result holds across multiple model families and requires no training or fine-tuning. A sympathetic reader would care because malware analysts routinely work from decompiled code, and a change limited to the prompt could reduce missed detections in existing pipelines.","feed_headline":"Two decompiler views lift LLM malware recall","feed_subtitle":"Ghidra and RetDec outputs supply complementary signals that single-view pipelines miss on the same binaries.","key_machinery":"Paired pseudo-C outputs from Ghidra and RetDec, supplied together in the LLM prompt as complementary evidence for the benign/malicious decision.","core_discovery":"The paper establishes that supplying matched pseudo-C from both Ghidra and RetDec to an LLM classifier produces higher malicious-class F1 scores than either decompiler alone, chiefly by lifting recall on malicious samples, and that the two decompilers commit partially non-overlapping errors on the same binaries.","pith_inferences":["The same pairing of decompiler outputs could be tested on related binary-analysis tasks such as vulnerability discovery or packer identification.","Adding a third decompiler might produce further recall gains or show diminishing returns once the main sources of disagreement are covered.","If decompiler diversity helps LLMs overcome individual tool limitations, comparable multi-tool prompting may help on other lossy reverse-engineering problems outside malware."],"forward_implications":["Malicious recall increases when both decompiler views are provided to the LLM.","Ghidra and RetDec generate classification errors that only partially overlap on the benchmark samples.","The F1 improvement appears across a range of LLMs without any model changes or additional training.","Multi-decompiler prompting supplies a training-free route to better malware triage in settings that already use decompiled code."],"fun_headline_variants":["Ghidra RetDec pair raises LLM malware recall","Dual decompiler views increase malicious recall","Complementary decompiler errors aid LLM triage","Multi-view input raises LLM malware F1"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"The curated collection of benign utilities and malicious programs is representative enough that the observed F1 gain will appear on other binaries and threat behaviors.","fun_headline_variants_meta":{"raw":{"variants":["Ghidra RetDec pair raises LLM malware recall","Dual decompiler views increase malicious recall","Complementary decompiler errors aid LLM triage","Multi-view input raises LLM malware F1"]},"model":"grok-4.3","cost_usd":0.005765,"raw_usage":{"total_tokens":2638,"prompt_tokens":610,"num_sources_used":0,"completion_tokens":54,"cost_in_usd_ticks":57653000,"prompt_tokens_details":{"text_tokens":610,"audio_tokens":0,"image_tokens":0,"cached_tokens":64},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":1974,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":610,"tokens_out":54,"duration_ms":23300,"temperature":1.0,"reasoning_tokens":1974,"cache_read_input_tokens":64,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-26T17:08:18.214884+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"Applying the identical single-view versus dual-view comparison to a new collection of binaries drawn from different malware families and observing no improvement or a decline in malicious-class F1.","supporting_citations":[],"review_version":1}