{"id":"922fa017-52e8-40db-81aa-401f62e55cc1","arxiv_id":"2509.04887","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"RINSER uses a masked-language model trained on API codeprints (assembly context around API calls) to predict Windows API names, reporting 85.77% accuracy on normal and 82.88% on stripped binaries, and finding 65 APIs that IDA misses.","lead":"This paper presents RINSER, a system that trains a BERT-style language model on snippets of assembly code around Windows API calls, then uses it to guess the API name when the name is hidden. If the approach holds up, malware analysts could auto-fill obfuscated API names in suspicious Windows binaries instead of tracing each call by hand.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported accuracy may partly measure agreement with IDA FLIRT labels; independent ground-truth validation is needed before accepting the 85.77% / 82.88% headline numbers.","rationale":"The reader's weakest assumption correctly identifies that both training and test labels are produced by the same FLIRT-based extraction. My independent review finds this to be the single most load-bearing concern: the central claim 'RINSER accurately predicts Windows API names at 85.77%/82.88%' is only as strong as the ground truth labels. If FLIRT is systematically wrong in ways the model learns to mimic, the reported accuracy overstates real-world API identification. The 65 manually validated APIs are a commendable independent check but do not repair the main evaluation. Secondary issues (Table 3 arithmetic errors, the dummy-trace Trex comparison, the abstract's robustness claim contradicting Table 6) are real but do not undermine the central claim as directly. Since the reader already returned CONDITIONAL, and my concern matches theirs, no verdict change is needed; however, the concrete test should be a condition for accepting the headline numbers.","tokens_in":27303,"tokens_out":5229,"duration_ms":51723,"concrete_test":"Randomly sample 1,000 API codeprints from the 10% test split. For each codeprint, establish independent ground truth by: (1) inspecting the import table of the originating binary to identify the call target; (2) using dynamic tracing (e.g., API Monitor or an IDA debugger) to log the actual API invoked at that call site; and (3) comparing the FLIRT-derived API name and parameter list against Microsoft's official documentation. Replace FLIRT labels wherever they disagree with the independent ground truth. Re-run RINSER's masked prediction on the original codeprints and compute accuracy against the corrected labels. If the accuracy drops by more than 3–5 percentage points relative to the reported 85.77%, the FLIRT circularity is material and the headline figures require qualification.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central accuracy claims rest on ground truth extracted by Algorithm 1, which obtains API names and parameter annotations directly from IDA FLIRT (Steps 8–15). Section 4.2 says annotations were validated against Microsoft documentation, but the actual training and test labels are the FLIRT annotations, so the model can learn to reproduce IDA's annotation decisions rather than independently identify APIs. The 91% extraction accuracy in §5.2 does not break this circularity: it only measures how often the heuristic recovers FLIRT's annotations. The in-the-wild evaluation (§E.2) also uses FLIRT as ground truth, so it is subject to the same concern. Although the manual validation of 65 obfuscated APIs in §5.5 is a genuinely independent check, it covers only a tiny subset of the 69,921 predictions and does not validate the main 85.77%/82.88% figures. Consequently, a systematic FLIRT error—for example, a misidentified call or a wrong parameter annotation—would be counted as a correct prediction if the model agrees with FLIRT, inflating the reported accuracy.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents RINSER, a framework for predicting Windows API function names from x86 PE binaries. RINSER introduces 'API codeprints', which combine an API's name, its parameter names/values, and semantically related assembly instructions obtained by register-based backtracking from IDA Pro disassembly (Algorithm 1 and 2). The codeprints are tokenized and normalized, then used to pre-train a BERT-style masked language model; the model is fine-tuned for stripped binaries. The authors report 85.77% accuracy on a 10% test split of 4.7M codeprints, 82.88% on simulated stripped codeprints, 88.80% on 1,148 unseen binaries, and 69,921 predicted API names in cases where IDA did not resolve the API. They also compare with three baselines and evaluate robustness against instruction randomization and code displacement.","tokens_in":27540,"tokens_out":6426,"duration_ms":59120,"significance":"If the central claims hold, the paper makes a useful contribution: it provides a large malware-derived WinAPI dataset, a concrete codeprint construction method, and evidence that masked language modeling can exploit API calling context. The ablation study (Appendix A) convincingly shows that full codeprints matter, and the manual validation of 65 obfuscated API names (Appendix E.3) is a genuinely independent check. However, the headline accuracy numbers rest on FLIRT-derived labels, and several reported numbers are internally inconsistent. These issues are load-bearing for the main claims and need to be fixed before the results can be accepted at face value.","major_comments":[{"comment":"Table 3 reports per-row accuracies that do not match the counts in the same table. Row 1: 337,283/449,472 = 75.0%, not 56.94%. Row 2: 164,763/187,084 = 88.1%, not 73.34%. Row ≥6: 74,437/77,092 = 96.6%, not 91.84%. Only the overall 821,895/991,561 = 82.88% is consistent. The surrounding paragraph explains the parameter-count trend using these rows, so the corrected numbers are needed; as printed, the claim that accuracy increases monotonically with parameter count is not supported.","section":"§5.4, Table 3"},{"comment":"Algorithm 1 (Steps 8–15) identifies API names and parameter names from IDA FLIRT annotations, and §4.2 constructs the labeled dataset from those same annotations. The validation against Microsoft documentation in §4.2 checks API/parameter-name lists, not the per-callsite labels used for training and testing. Both train and test labels therefore originate from the same FLIRT/IDA pipeline, so the reported accuracy may partly measure the model's agreement with IDA's annotation decisions rather than its ability to infer API identity independently. The §5.2 extraction accuracy (91.4%/90.5%) also derives from FLIRT-based extraction. Please provide an independent ground truth for a random sample of test codeprints (e.g., compiled known source or manual reverse engineering) and report accuracy on that sample, along with an analysis of cases where FLIRT annotations are wrong.","section":"§3.2.1, §4.2, §5.2"},{"comment":"Table 2 reports a 10% test split of 4,744,969 codeprints, but the paper does not state that the split is at the binary level. If codeprints from the same binary appear in both training and test, the 85.77% accuracy in §5.3 can be inflated by binary-specific leakage. The in-the-wild evaluation in §E.2 on 1,148 unseen binaries is an encouraging complementary result, but the main headline number should either be recomputed with a binary-disjoint split or explicitly accompanied by a statement of the split unit and the corresponding accuracy.","section":"§4.2, Table 2, §5.3"},{"comment":"Section 5.5 reports that RINSER 'correctly predicted' 69,921 API names among 148,685 IDA-unresolved codeprints. Since IDA did not resolve these names, correctness is not defined unless there is an independent ground truth, such as the corresponding non-obfuscated binaries. The manual validation of 65 unique APIs in Appendix E.3 covers only the unique-name level, not the 69,921 predicted instances. Please specify how each of the 69,921 predictions was verified, or report the deobfuscation evaluation only on the subset with known ground truth. The '66% deobfuscation' result on the manually crafted dataset also lacks the denominator and per-API details.","section":"§5.5, §E.3"},{"comment":"Table 9 is internally inconsistent with the main results. The 'Actual predictions' column gives 89.21% for Normal §5.3 and 82.08% for Stripped §5.4, but §§5.3–5.4 report 85.77% and 82.88%. The Obfuscated row's 47.03% matches 69,921/148,685 only if that is the intended denominator, which is not stated in §5.5. Please reconcile these values and clarify whether different test sets were used. Also, the improvement column reports percentage-point differences (e.g., 0.41 pp) but labels them as percentages; this should be corrected.","section":"Appendix D, Table 9"}],"minor_comments":[{"comment":"The paper calls the dataset 'the largest publicly available dataset of this type' but no repository link or availability statement appears in this version. Please add a URL or DOI, and clarify the exact release conditions.","section":"Abstract, §1"},{"comment":"The algorithm pseudocode contains typesetting artifacts such as '/u1D439 instrs' and repeated 'instrs' placeholders, and the Unicode variables are difficult to read. Please clean the pseudocode so that the extraction steps are unambiguous.","section":"Algorithm 1, Algorithm 2"},{"comment":"Reference [25] is labeled 'Hugging Face. Windows apis to intents by malware' and points to malapi.io, but the text in §5.1 cites it for the Transformers fill-mask pipeline. The correct reference appears to be [33]. Please verify and fix this citation.","section":"References"},{"comment":"Table 10 lists 'WriteProcessMemory' twice in the 5-parameter row. Remove the duplicate entry.","section":"Table 10"},{"comment":"The notation for predicted and ground-truth API names uses the same symbol for both quantities ('predicted API name (/u1D464)' and 'ground truth API name (/u1D464)'). Use distinct symbols, e.g., ŷ and y, to define the accuracy metric.","section":"§5.1"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: RINSER is a solid idea with real contributions—the API codeprint representation and the 4.7M-codeprint dataset—but the reporting is sloppier than it should be, and the headline numbers need one careful correction before I'd trust them. The core approach is probably sound; the flaws are fixable.\n\nWhat's actually new: RINSER applies masked language modeling to Windows API name prediction, which is a new application. The API codeprint—register-backtracked assembly context around each API call—is a sensible way to give a model more than just parameter values. The dataset of 4.7M codeprints from 11k malware binaries is large, and if released, genuinely useful. The in-the-wild evaluation on 1,148 unseen binaries is a good idea, and the manual validation of 65 obfuscated APIs that IDA missed is real evidence the model finds things a commercial tool doesn't.\n\nWhere the soft spots are: Table 3 has arithmetic that doesn't add up—337,283/449,472 is 75%, not 56.94%, and 164,763/187,084 is 88%, not 73.34%. That looks like a copy-paste error, but it makes you want to re-check every number in the paper. More substantively, the main test split isn't shown to be binary-disjoint from training; the 10% split on codeprints may leak binaries across sets. The comparison to Trex uses a reimplementation with dummy traces, which isn't a fair test of Trex. And the abstract's 'no more than 3% drop' claim is undercut by Table 6, where code displacement drops accuracy from 89.03% to 85.43%—a 3.6% drop.\n\nOn the FLIRT circularity: the stress-test note is right that the labels are IDA FLIRT annotations, so the model could partially be learning to reproduce IDA's decisions. The 91% extraction accuracy only measures how well the heuristic recovers FLIRT's annotations, not independent correctness. However, the manual validation of 65 obfuscated APIs is a genuine external check, and it supports the overall direction. Still, the 85.77%/82.88% numbers need to be backed by an evaluation on a set where labels are independently verified, not just FLIRT-derived.\n\nBottom line: This is a paper for malware analysts and anyone working on binary understanding with ML. The core idea is good and the dataset is a contribution on its own. It deserves a serious referee, but the revision needs to fix the arithmetic, report a binary-disjoint split or explain why it's not needed, and either run the real Trex or clearly label the comparison as approximate.","headline":"RINSER is a genuinely useful new application with a large dataset, but the headline numbers are undercut by arithmetic errors, a binary-split leak risk, and a FLIRT-label circularity that needs an independent validation set.","tokens_in":28124,"tokens_out":2844,"would_cite":true,"duration_ms":26946,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"RINSER predicts Windows API names in obfuscated malware with 85.77% accuracy by training BERT on 'API codeprints' extracted from disassembly.","keywords":["Windows API prediction","API codeprints","masked language model","malware deobfuscation","static binary analysis","stripped binaries","BERT","PE binaries"],"falsifier":"Compile a set of Windows C programs with symbols and known API names, strip them, and obfuscate names with a method other than dynamic GetProcAddress resolution (e.g., direct IAT rewriting with hash-based lookup), then compare RINSER's predictions to the true names; if codeprint extraction drops below the claimed ~91% parameter-detection accuracy or accuracy on such binaries falls far below 85%, the push-and-annotation assumption is the breaking point. Alternatively, inspect mispredictions: RINSER outputs parameters like 'hwnd' or symbols like 'maddr' for non-API tokens, indicating the model i","tokens_in":27140,"feed_emoji":"🛡️","tokens_out":5525,"duration_ms":49197,"temperature":0.7,"pith_summary":"The paper claims that Windows API calls can be identified even when their names are stripped or obfuscated, by training a masked language model on 'API codeprints' — small windows of assembly instructions that set up each API call's parameters. On a test split of 4.7M codeprints from malware binaries, the model names the hidden API correctly 85.77% of the time on normal binaries and 82.88% on stripped binaries. RINSER also resolves 69,921 API calls that a commercial disassembler leaves unnamed, including 65 capabilities confirmed by manual analysis. The practical payoff: an analyst can be pointed to likely API identities quickly, and the approach beats three prior tools by more than 20 points.","feed_headline":"Masked language model names hidden Windows APIs at 85.77 percent","feed_subtitle":"API codeprints let BERT fill in API names stripped or obfuscated by malware, reaching 88.8 percent on unseen binaries.","key_machinery":"The key object is the API codeprint, built in three steps: find API call sites and parameter pushes from IDA-disassembled code; semantically backtrack from each register-based parameter value through preceding instructions to collect the instructions that touch the same registers; and symbolically map memory and address operands to tokens like 'mem', 'complex', and 'saddr' to control vocabulary. The codeprint is fed to a masked language model (BERT), which is trained to predict the masked API token from its surrounding context; this is what lets the model transfer to stripped and obfuscated binaries.","core_discovery":"The central claim is that the assembly context around a call is enough to name the API, and that a masked-language-model objective can learn that mapping without labels. The paper introduces API codeprints: the API name, its parameter names and values, and the instructions semantically tied to those parameter values. BERT-style pretraining on 4.7M codeprints teaches the model to fill a mask over the API name; fine-tuning adapts it to stripped binaries. On 1,148 unseen binaries accuracy is 88.80%, and on binaries whose API names IDA leaves unresolved, RINSER recovers 69,921 names, including 65 APIs tied to C2, spying, and evasion.","pith_inferences":["The evaluation's ceiling is set by IDA FLIRT: reported accuracy partly measures agreement with FLIRT's annotations, so gains on fully unknown APIs may be lower or higher than 85.77%; a test against independently compiled, symbol-known Windows programs would separate the two.","The semantic-backtracking design suggests the method could extend from x86 stdcall to x64 and fastcall conventions, where parameters pass in registers, provided codeprint construction is adapted.","A natural stress test is malware that hides calls by push/ret sequences or statically linked libraries — the paper notes these as limitations; future work could use a classifier trained on surrounding instructions rather than only the call operand.","If the codeprint dataset is public as claimed, it enables transfer learning for low-resource malware families by fine-tuning on a handful of labeled samples."],"forward_implications":["An analyst who sees an obfuscated 'call dword_4031A5' can be shown the most likely API name in seconds, cutting manual reverse-engineering time.","The approach should transfer to other API-name prediction tasks: parameter-value prediction, API call sequence prediction, and deobfuscation of dynamically resolved imports using GetProcAddress.","Releasing the 4.7M-codeprint dataset gives other researchers a common benchmark to compare API prediction work on Windows, where source-independent large datasets were missing.","Because the model learns context rather than exact import entries, it stays accurate when control flow is displaced by jump-based transformations, with accuracy dropping only from 89.03% to 85.43%."],"supporting_citations":[{"why":"Supplies the disassembler used to generate assembly and FLIRT annotations from which codeprints are extracted.","marker":"[2]"},{"why":"Provides the unpacked malware hashes used to source the 11,098 binaries in the dataset.","marker":"[5]"},{"why":"Defines the masked language model architecture and pretraining objective RINSER adapts.","marker":"[16]"},{"why":"Provides the FLIRT signature-based annotations that name APIs and parameters in ground truth.","marker":"[24]"},{"why":"The closest prior method, predicting Windows API names from parameter values with HMMs; the 25-API baseline RINSER beats.","marker":"[27]"},{"why":"A BERT-style assembly language model baseline; RINSER compares against its embedding-plus-classifier approach.","marker":"[29]"},{"why":"A BERT-style model using microtraced assembly; used as a second baseline adapted to RINSER's codeprints.","marker":"[42]"},{"why":"Supplies the IPR and code displacement transformations used to test adversarial robustness.","marker":"[32]"},{"why":"Documents the dynamic API resolution obfuscation technique used to craft the obfuscated ground-truth binaries.","marker":"[46]"}],"fun_headline_variants":["BERT fills in stripped Windows API names at 88.8%","API codeprints let BERT reveal hidden Windows APIs","RINSER recovers 65 obfuscated Windows APIs IDA missed","BERT trained on 4.7M codeprints names Windows APIs"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The whole pipeline assumes IDA FLIRT reliably annotates which API is called and the names of its parameters; if that annotation is wrong, both the training label and the test label are wrong, so the measured accuracy reflects fidelity to IDA as much as true API identification.","fun_headline_variants_meta":{"raw":{"variants":["BERT fills in stripped Windows API names at 88.8%","API codeprints let BERT reveal hidden Windows APIs","RINSER recovers 65 obfuscated Windows APIs IDA missed","BERT trained on 4.7M codeprints names Windows APIs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000749,"raw_usage":{"total_tokens":3217,"prompt_tokens":835,"completion_tokens":2382,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":579,"completion_tokens_details":{"reasoning_tokens":2303}},"tokens_in":579,"tokens_out":2382,"duration_ms":14183,"temperature":1.0,"reasoning_tokens":2303,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T05:49:12.259919+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compile a set of Windows C programs with symbols and known API names, strip them, and obfuscate names with a method other than dynamic GetProcAddress resolution (e.g., direct IAT rewriting with hash-based lookup), then compare RINSER's predictions to the true names; if codeprint extraction drops below the claimed ~91% parameter-detection accuracy or accuracy on such binaries falls far below 85%, the push-and-annotation assumption is the breaking point. Alternatively, inspect mispredictions: RINSER outputs parameters like 'hwnd' or symbols like 'maddr' for non-API tokens, indicating the model i","supporting_citations":[{"cited_title":"https://hex-rays.com/ida-pro/","cited_arxiv_id":null,"evidence_quote":"Supplies the disassembler used to generate assembly and FLIRT annotations from which codeprints are extracted."},{"cited_title":"https://virusshare.com/hashfiles/unpacked_hashes.md5","cited_arxiv_id":null,"evidence_quote":"Provides the unpacked malware hashes used to source the 11,098 binaries in the dataset."},{"cited_title":"IDA FLIRT Tec","cited_arxiv_id":null,"evidence_quote":"Provides the FLIRT signature-based annotations that name APIs and parameters in ground truth."},{"cited_title":"Towards generic deo bfuscation of win- dows api calls","cited_arxiv_id":null,"evidence_quote":"The closest prior method, predicting Windows API names from parameter values with HMMs; the 25-API baseline RINSER beats."},{"cited_title":"Palmtree: learning a n assembly language model for instruction embedding","cited_arxiv_id":null,"evidence_quote":"A BERT-style assembly language model baseline; RINSER compares against its embedding-plus-classifier approach."},{"cited_title":"Malware makeover: Breaking ml-based static analysis b y modifying exe- cutable bytes","cited_arxiv_id":null,"evidence_quote":"Supplies the IPR and code displacement transformations used to test adversarial robustness."},{"cited_title":"A museum of api obfuscation on win32","cited_arxiv_id":null,"evidence_quote":"Documents the dynamic API resolution obfuscation technique used to craft the obfuscated ground-truth binaries."}],"review_version":1}