{"id":"6d7eece1-34e1-4015-86eb-c477ef30d4b9","arxiv_id":"2502.04636","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"From 2016 to 2023, code obfuscation in Google Play apps increased by about 13 percentage points, reaching roughly 66% of apps by 2023, with ProGuard and Allatori as the most common tools.","lead":"Researchers analyzed over 500,000 Android apps from Google Play and found that code obfuscation, which hides app code from reverse engineering, rose by about 13 percentage points between 2016 and 2023. The study is the largest look at which obfuscation tools and techniques developers actually use, and it shows gaming and casino apps are the most heavily protected.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 13% obfuscation increase may be a cohort artifact: year labels are 'last update date' pooled across two unbalanced crawls, so 2016 apps are largely abandoned while 2023 apps are actively maintained.","rationale":"The reader identified classifier transferability as the weakest assumption, and that concern is valid: real-world Play apps may use unknown tools, partial/library-level obfuscation, or build pipelines that produce feature distributions absent from the F-droid-derived training set. The 50-app manual validation (D4) is small and its labeling method (examining identifier names) is partly circular with the classifier's main features. However, the paper does provide held-out evaluations (D2, D3, D6, D8, D9) suggesting the classifiers generalize to at least some unseen tools and techniques. The temporal cohort confound is less mitigated: the paper never controls for the fact that '2016' apps are those whose last update occurred in 2016, while '2023' apps were updated in 2023. Comparing these groups is not a clean before/after measurement of industry practice; it conflates the year with app maintenance status and crawl coverage. This affects the central claim (the 13% increase) directly, even under a perfect classifier. The conditional verdict remains appropriate, but the condition should explicitly require a within-crawl, last-update-stratified analysis before the longitudinal claim is accepted. If the proposed test shows a flat within-2023-crawl trend, the headline 13% increase should be revised or re-framed as an abandoned-vs-active difference. If the test reproduces the trend, the reader's original classifier-transfer concern remains the main residual risk.","tokens_in":19757,"tokens_out":8193,"duration_ms":95643,"concrete_test":"Recompute obfuscation percentages separately for APKs collected in the 2018 crawl and for APKs collected in the 2023 crawl, and within the 2023 crawl stratify by last-update year (e.g., 2016-2018 vs 2021-2023). If the within-2023-crawl comparison shows little or no rise, the reported 13% trend is a pooling artifact; if it reproduces the rise, the trend is real.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing weakness is the construction of the longitudinal comparison, not classifier error alone. Year labels in Figure 2 and Table IX come from the metadata field 'last update date', and the yearly APK pools merge two crawls with very different coverage (1,023,521 APKs in 2018 vs 395,396 in 2023). An app last updated in 2016 is, by definition, an app untouched for seven or more years and likely abandoned; an app last updated in 2023 is actively maintained. The paper excludes 2019-2020 because those apps are 'more likely to be abandoned by app developers', yet applies no such control to 2016-2018. The claimed rise from ~53% in 2016 to ~66% in 2023 may therefore reflect the difference between abandoned and active apps, or between the two crawl cohorts, rather than a real evolution of developer obfuscation practice. Even a perfectly accurate classifier would produce this artifact if maintenance status correlates with obfuscation. The paper's own Section VII-B acknowledges the crawl limitation only for 2019-2020, asserting without evidence that overall trends are unlikely to change. This threatens the central longitudinal claim independently of the classifier-transfer issue identified by the reader.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents a large-scale empirical study of code obfuscation adoption in Google Play, analyzing 548,967 APKs from two crawls (2018 and 2023) spanning last-update years 2016-2023. The authors build a bank of machine-learning classifiers to detect (i) whether an APK is obfuscated, (ii) which obfuscation tool was used (ProGuard, Allatori, DashO, or other), and (iii) which obfuscation techniques are present (identifier renaming, control-flow modification, string encryption). They report an overall 13 percentage-point increase in obfuscation between 2016 and 2023, with 56.25% of all analyzed APKs obfuscated, ProGuard and Allatori as the dominant tools, and the Casino genre the most obfuscated at 80%. They additionally report genre-wise, developer-wise, and top-k analyses showing higher obfuscation among top-ranked apps and top developers.","tokens_in":19967,"tokens_out":3042,"duration_ms":30788,"significance":"If the results are reliable, this would be the first large-scale longitudinal account of obfuscation adoption in Google Play, with direct implications for malware analysis, app-store policy, and developer practice. A credible measurement that obfuscation now covers a majority of Play apps and is growing would be a useful reference point for the security community. The paper has strengths: it uses a large sample, it validates classifiers on multiple held-out sets, it releases source code and models, and it draws on externally generated ground truth (F-droid projects and AndroOBFS). These strengths make the core measurement apparatus worth scrutiny rather than dismissal. However, the central longitudinal claim rests on a year-labeling and cohort-construction choice that may induce the observed trend, and the real-world validation of the classifiers is extremely thin. The headline percentages are point estimates without uncertainty quantification, and the authors themselves acknowledge the limitation that no accurate ground truth exists for real-world APKs.","major_comments":[{"comment":"The '13% increase between 2016 and 2023' may be an artifact of how years are assigned. Year labels come from the metadata field 'last update date', and the APK pool is formed by merging two crawls with very different coverage: 1,023,521 APKs from 2018 and 395,396 from 2023, as shown in Table IX. Apps last updated in 2016 are, by construction, apps untouched for at least two years at the time of the 2018 crawl and therefore likely abandoned, whereas apps last updated in 2023 are actively maintained. The paper excludes 2019-2020 because those apps are 'more likely to be abandoned by app developers' (Section IV-A), but applies no such control to 2016-2018. The apparent increase from about 53% (2016-2018) to about 62% (2021-2023) could reflect the difference between abandoned and active apps, or between the two crawl cohorts, rather than an evolution of obfuscation practice. The claim in Section VII-B that 'overall trends we observed are unlikely to change even including those data' is asserted without evidence and does not address the cohort confound. To make the longitudinal claim load-bearing, the authors should compare apps under conditions that control for maintenance status, for example by restricting to apps updated within one year before each crawl, by tracking the same apps across both crawls, or by showing that the trend persists within each crawl when stratifying by time since last update.","section":"Section IV-A, Table IX, Figure 2, Section V-A"},{"comment":"The external validity of the headline percentages is not established. The classifiers are trained on F-droid apps manually obfuscated with ProGuard, Allatori, and DashO, plus the AndroOBFS malware set, but the only real-world validation is D4, a set of 50 manually labeled Google Play APKs (Section III-C1, Table VI). The authors themselves note that 'with real-world APKs, there is no method for accurate ground-truth establishment' (Section VII-B). With only 50 Play apps, the 92% accuracy on D4 carries a wide confidence interval, and the D4 labels were based solely on identifier-name anomalies, which may not capture other obfuscation forms. The tool and technique detectors are validated only on D6 and D8/D9, which are drawn from the same manually obfuscated F-droid and AndroOBFS distributions; no real-world validation of tool or technique identification is performed. The reported 40.92% ProGuard and 36.64% Allatori shares, and the technique breakdown in Section V-A, can therefore be substantially biased if real-world obfuscators produce feature distributions different from the training distributions. The paper should either provide a larger and more diverse real-world validation set, report bounds on the headline estimates under plausible label-error rates, or temper the claims accordingly.","section":"Section III-C, Section III-D, Table VI, Section VII-B"},{"comment":"All headline percentages are reported as point estimates without any uncertainty quantification. For example, Figure 2 plots yearly obfuscation percentages, and Table XI reports top-k percentages to two decimal places, yet no confidence intervals, standard errors, or sensitivity analyses are given. Because each percentage is the output of a classifier with known misclassification rates on held-out data, and because the samples are large but not exhaustive, the absence of uncertainty intervals makes it difficult to judge whether differences such as the 13-point trend, or the differences between genres in Figure 5, are statistically meaningful. The authors should at least report binomial confidence intervals for the large-sample percentages and, where possible, propagate classifier error rates into the final estimates.","section":"Section V, Tables VI-VIII and XI"}],"minor_comments":[{"comment":"The table would benefit from a column showing the number of analyzed APKs as a percentage of available APKs for each year, since the sampling rate differs widely (e.g., 2016 has 74,817 of 174,136 analyzed, while 2023 has 65,543 of 65,697 analyzed).","section":"Section IV-A, Table IX"},{"comment":"The limitation paragraph says the 2019-2020 exclusion is unlikely to change overall trends, but this is not supported by any sensitivity analysis; either provide such an analysis or soften the claim to acknowledge that the trend is measured only on the 2016-2018 and 2021-2023 cohorts.","section":"Section VII-B"},{"comment":"In the D9 rows, precision values of 1.00 for IR, CF, and SE are suspiciously perfect and suggest the AndroOBFS labels may align very closely with the feature thresholds; a short explanation or discussion of why precision is exactly 1.00 would help readers interpret these results.","section":"Section III-D, Table VIII"},{"comment":"The category labels in Figure 4 (e.g., 'Only IR', 'IR & CF') are embedded in the figure body, while the legend below contains only 'Category' and numeric row labels; this makes the figure hard to read, and the numeric rows should be clearly associated with their category names.","section":"Section V-A, Figure 4"},{"comment":"The paper repeatedly calls the study 'first of its kind' (Abstract, Section VI-B), but OBFUSCAN is cited as a prior large-scale investigation of obfuscation use in Google Play; the novelty claim should be phrased more precisely, e.g., as the first to cover multiple tools and techniques over an eight-year span.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The paper is a solid empirical contribution in terms of scope and reproducibility artifacts, but the central longitudinal claim needs substantially more work to separate a true trend from cohort and classifier-transfer effects. The authors' own acknowledgment that 2019-2020 apps are 'likely abandoned' while applying no such control to 2016-2018 is a telling inconsistency that should be pressed in revision. I do not see this as an outright reject; the data and classifiers could plausibly support the claim if re-analyzed with proper controls, but as presented the 13% figure is not load-bearing."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague, here is my take. The paper is a serious large-scale measurement of obfuscation in Google Play: 548,967 APKs, two crawls from 2018 and 2023, and a classifier bank for obfuscation presence, tool, and technique. The genuinely new thing is the combined scope. Prior work either looked at identifier renaming over 1.7M apps, or techniques over tens of thousands, or control flow over 13k. This one covers tools and techniques together across eight year-labels and releases code, models, and ground-truth data. The classifier validation is honest in an important way: it uses unseen data from AndroOBFS and 50 manually labelled Play apps, and accuracy drops from 97% on the test split to 87-92% on the unseen sets. I believe the numbers are not fabricated; they are plausible.\n\nThe weak spot is the longitudinal claim, and it is load-bearing. Year labels come from \"last update date\", pooled across two unbalanced crawls. A 2016 label means the app was not updated for seven years; a 2023 label means actively maintained. Comparing ~53% in 2016 to ~66% in 2023 may be comparing abandoned apps with active apps, not measuring the evolution of developer practice. The paper excludes 2019 and 2020 exactly because those apps are likely abandoned, but it applies the same reasoning inconsistently to 2016-2018. That is a real cohort issue, not a nitpick. The stress-test note is right. On top of that, there are no confidence intervals on any headline percentage, the real-world validation set is only 50 apps, and classifier error propagates into every reported figure. The authors acknowledge some of this in Section VII-B, but the claim that overall trends are \"unlikely to change\" is assertion, not evidence.\n\nWhat holds up: the snapshot measurements. The overall 56% obfuscation rate, ProGuard and Allatori dominance, Casino apps at 80%, and top apps being more heavily obfuscated are useful reference points even if the time trend is shaky. The released artifacts mean someone can reanalyze the data with a better cohort design. The paper also gives a fair account of related work and does not oversell the classifier contribution.\n\nWho is this for? Android security researchers and app-store mining people. It is a useful reference measurement and a good case study in how crawl design affects longitudinal conclusions. It deserves peer review because the dataset and framing are valuable and the flaws are fixable. I would send it to review with a request for major revision: add confidence intervals, redo the longitudinal analysis controlling for app age or crawl year, expand the real-world validation, and soften the causal language.","headline":"A valuable large-scale snapshot of Android obfuscation whose headline time trend is likely a cohort artifact from pooling two crawls by last-update date.","tokens_in":20543,"tokens_out":2985,"would_cite":true,"duration_ms":30084,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that code obfuscation in Google Play rose by about 13 percentage points from 2016 to 2023, so that a majority of the store's apps are now obfuscated.","keywords":["code obfuscation","Android","Google Play","ProGuard","Allatori","longitudinal study","machine learning classification","app store mining"],"falsifier":"Manually reverse-engineer a few hundred randomly selected 2023 Play APKs: if the independently measured obfuscation fraction and the ProGuard/Allatori split differ systematically from the paper's 66% and 40.92%/36.64%, then the trend is an artifact of classifier transfer.","tokens_in":19531,"feed_emoji":"📱","tokens_out":9596,"duration_ms":81505,"temperature":0.7,"pith_summary":"Code obfuscation has quietly become standard practice on Google Play. This paper builds machine-learning classifiers that read static features of an Android APK and decide whether the code is obfuscated, which obfuscation tool produced it, and which techniques (identifier renaming, control-flow modification, string encryption) are present. Applying the classifiers to 548,967 APKs from two Play snapshots spanning 2016 to 2023, the authors find that 56.25% of apps are obfuscated and that the obfuscated share rose by about 13 percentage points, reaching roughly 66% in 2023. If this is right, any security analysis of the Play ecosystem must treat obfuscation as the norm rather than an exception.","feed_headline":"Google Play app code obfuscation rose 13 percent by 2023","feed_subtitle":"Analysis of 548,967 Android APKs finds 56% obfuscated overall, with casino games leading at 80%.","key_machinery":"The machinery is a bank of supervised classifiers over 37 static APK features: the percentage of class, method, field, and string names by length (1, 2, 3, 4, >4) and by presence of special or numeric characters, plus the share of nop, goto, invoke, if, and move instructions in the DEX bytecode. A binary multilayer perceptron decides whether an app is obfuscated; three one-vs-rest random-forest classifiers decide between ProGuard, Allatori, or DashO against an 'other' class; and three more random forests decide whether identifier renaming, control-flow modification, or string encryption is present. The classifiers are trained on apps built from open-source projects and obfuscated in the lab with the three tools, then validated on unseen in-lab apps, on an obfuscated-malware set produced by a different tool, and on 50 manually labelled Play apps. This feature-based transfer is what lets the authors scale the detection to half a million real-world APKs.","core_discovery":"The central discovery is longitudinal: between 2016 and 2023 the fraction of obfuscated Google Play apps increased by about 13 percentage points, from a stable 50-55% range in 2016-2018 to about 66% in 2023, and roughly 56% of all 548,967 analyzed APKs are obfuscated. The paper attributes most obfuscation to two tools — ProGuard at 40.92% of obfuscated apps and Allatori at 36.64% — with DashO at 1.01% and a substantial 21.43% bucket of unknown tools. Identifier renaming is nearly universal among obfuscated apps (99.62%), while control-flow modification appears in 81.04% and string encryption in 62.76%, and 58.7% of obfuscated apps combine all three techniques. Obfuscation is not uniform: casino games lead at 80%, game genres generally sit above other categories, over 90% of top-1,000 apps are obfuscated, and even single-app developers moved from 45.5% to 57.2% obfuscation between the two snapshots.","pith_inferences":["If the trend continues past 2023, app-store security screening may need to switch from asking whether code is obfuscated to asking what the obfuscation is hiding, since benign and malicious apps both obfuscate.","The paper does not separate library-level obfuscation from app-level obfuscation; a natural next test is to check whether the 13-point rise is driven by new default build tools rather than by deliberate developer choice.","The 21.43% unknown-tool share is likely an undercount of commercial hardening tools that resemble ProGuard; a testable extension is to add a DexGuard-trained class and see how much of the ProGuard and 'other' buckets move.","Because the features are name-length and instruction-frequency statistics, apps obfuscated only through string encryption with normal-looking identifiers could evade the detectors; building a validation set from manually labelled Play apps with diverse real-world tools would test whether 56.25% holds."],"forward_implications":["Static analysis that ignores obfuscation will misread a majority of current Play apps, since 56.25% of analyzed APKs are obfuscated and the share was still rising in 2023.","Deobfuscation research must handle combinations: 58.7% of obfuscated apps use all three main techniques, so single-technique deobfuscators cover a minority of real apps.","Identifier renaming is the near-universal first step (99.62%), so symbol-recovery heuristics will be a prerequisite for almost any code-level analysis of Play apps.","The large unknown-tool bucket (21.43%) means the market is not captured by the three studied tools, and detectors designed for a closed tool set will silently push new tools into the 'other' category.","Obfuscation is no longer a signal of sophistication: single-app developers' usage rose from 45.5% to 57.2%, so small and amateur apps are now obfuscated too."],"supporting_citations":[{"why":"Provides the prior large-scale Google Play obfuscation measurement that this work extends to eight years and over half a million APKs.","marker":"[26]"},{"why":"The free tool built into Android Studio; its identifier-renaming-only output is a core class the tool detector must recognize.","marker":"[27]"},{"why":"The commercial obfuscator whose usage share of 36.64% is one of the paper's headline results.","marker":"[28]"},{"why":"The open-source obfuscator used to create validation APKs that the detectors never saw during training.","marker":"[36]"},{"why":"The obfuscated-malware dataset used as an unseen validation set for all three detector types.","marker":"[37]"},{"why":"A prior large-scale study of obfuscation techniques in the wild whose technique categories the paper adopts.","marker":"[22]"},{"why":"A prior obfuscator-identification method used as the baseline for the tool-detection results.","marker":"[25]"},{"why":"The open-source app source collection from which the paper built its manually obfuscated and non-obfuscated training APKs.","marker":"[40]"}],"fun_headline_variants":["Obfuscation on Google Play jumps 13 points since 2016","56% of Google Play apps now obfuscated, study finds","Casino apps lead obfuscation: 80% on Google Play","Study: Obfuscated code on Google Play hits 66% in 2023"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The classifiers were trained largely on APKs the authors obfuscated themselves with ProGuard, Allatori, and DashO, and the paper assumes those feature patterns represent the whole Google Play population, including apps obfuscated with unknown tools, for which the authors write there is no accurate ground-truth method.","fun_headline_variants_meta":{"raw":{"variants":["Obfuscation on Google Play jumps 13 points since 2016","56% of Google Play apps now obfuscated, study finds","Casino apps lead obfuscation: 80% on Google Play","Study: Obfuscated code on Google Play hits 66% in 2023"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000256,"raw_usage":{"total_tokens":1613,"prompt_tokens":1024,"completion_tokens":589,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":640,"completion_tokens_details":{"reasoning_tokens":504}},"tokens_in":640,"tokens_out":589,"duration_ms":5236,"temperature":1.0,"reasoning_tokens":504,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T22:00:14.954740+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Manually reverse-engineer a few hundred randomly selected 2023 Play APKs: if the independently measured obfuscation fraction and the ProGuard/Allatori split differ systematically from the paper's 66% and 40.92%/36.64%, then the trend is an artifact of classifier transfer.","supporting_citations":[{"cited_title":"A large scale investigation of obfuscation use in Google Play,","cited_arxiv_id":null,"evidence_quote":"Provides the prior large-scale Google Play obfuscation measurement that this work extends to eight years and over half a million APKs."},{"cited_title":"Java obfuscator and Android app optimizer — ProGuard,","cited_arxiv_id":null,"evidence_quote":"The free tool built into Android Studio; its identifier-renaming-only output is a core class the tool detector must recognize."},{"cited_title":"Allatori java obfuscator - professional java obfuscation,","cited_arxiv_id":null,"evidence_quote":"The commercial obfuscator whose usage share of 36.64% is one of the paper's headline results."},{"cited_title":"Obfuscapk: An open-source black-box obfuscation tool for Android apps,","cited_arxiv_id":null,"evidence_quote":"The open-source obfuscator used to create validation APKs that the detectors never saw during training."},{"cited_title":"AndroOBFS: Time- tagged obfuscated Android malware dataset with family information,","cited_arxiv_id":null,"evidence_quote":"The obfuscated-malware dataset used as an unseen validation set for all three detector types."},{"cited_title":"Understanding Android obfuscation techniques: A large- scale investigation in the wild,","cited_arxiv_id":null,"evidence_quote":"A prior large-scale study of obfuscation techniques in the wild whose technique categories the paper adopts."},{"cited_title":"Who changed you? obfuscator identification for Android,","cited_arxiv_id":null,"evidence_quote":"A prior obfuscator-identification method used as the baseline for the tool-detection results."},{"cited_title":"F-droid - free and open source Android app repository,","cited_arxiv_id":null,"evidence_quote":"The open-source app source collection from which the paper built its manually obfuscated and non-obfuscated training APKs."}],"review_version":1}