{"id":"0b93bb64-c8af-4878-842f-8f3df1f56c38","arxiv_id":"1908.06412","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"About 19% of F-Droid apps updated after October 2017 adopted Kotlin, often quickly, and Kotlin-using projects had more GitHub stars on average, a correlation the authors caution is not causal.","lead":"This study mined 1,232 open-source Android apps on F-Droid and matched them to Play Store and GitHub to measure how many adopted Kotlin and whether Kotlin apps are more popular. It found that about 19% of recently updated apps used Kotlin, and apps with Kotlin tend to have more GitHub stars, though the link may not be causal.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'influence on GitHub stars' claim is an unadjusted observational correlation; confounders such as project age and activity are uncontrolled, so the paper's causal-sounding wording and 'no cost' conclusion overstate what the data support.","rationale":"The reader's weakest_assumption identified confounding between Kotlin presence and GitHub stars, which is indeed the same load-bearing concern. The paper's descriptive diffusion and evolution results are credible and transparently reported, but the RQ3 popularity analysis is a single unadjusted correlation with acknowledged confounders, and the conclusion overstates it as 'no cost' and 'influence.' This justifies a conditional verdict: the descriptive contribution stands, but the causal-sounding popularity claim needs controls or softened wording. I agree with the reader's assessment and recommend no change from the CONDITIONAL verdict.","tokens_in":10193,"tokens_out":2096,"duration_ms":22984,"concrete_test":"Re-analyze the 475 non-abandoned projects used for RQ3 with a regression of log(GitHub stars + 1) on a Kotlin-presence indicator, controlling for project age, number of releases, and pre-October-2017 star counts (or use matched Java-only control projects with similar age and activity). If the Kotlin coefficient becomes non-significant or reverses sign, the paper's 'influence' and 'no cost' claims are unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The only statistically significant popularity result is the Wilcoxon test for H_sk0 (p=0.0002), comparing the presence of Kotlin with the number of GitHub stars, reported in Table 5 and Section 3. This test is purely observational: projects with Kotlin may differ from Java-only projects in age, release count, developer reputation, or general activity, and any of those differences could explain the star difference. The paper itself acknowledges this in Section 4, stating that 'the detected correlation... does not imply causality' and that active high-star projects may have moved to Kotlin rather than Kotlin attracting stars. Yet the RQ3 answer says Kotlin has a 'statistically significant influence' on stars, and the abstract/conclusion say adoption 'seems to come at no cost in terms of popularity' — a causal or equivalence reading that the design cannot support. Multiple-testing correction is not the main issue: p=0.0002 would survive Bonferroni (0.0083). A further technical concern is that the tests are labeled 'Wilkoxon Paired Signed Rank' even though Kotlin and non-Kotlin projects appear to be independent groups; if the tests were truly paired, the pairing variable is unspecified, and if they were unpaired, the named test is inappropriate. These issues do not undermine the descriptive RQ1/RQ2 findings, but they do weaken the central inferential popularity claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This mining study characterizes the adoption of Kotlin in open-source Android applications by combining data from F-Droid, Google Play, and GitHub. The authors define eight metrics capturing the presence and relative amount of Kotlin code, compute them on 1,232 F-Droid projects updated after October 2017, and follow the release history of 145 projects to describe the Java-to-Kotlin transition. They then test, via Wilcoxon tests, whether Kotlin presence or Kotlin majority relates to Play Store ratings, downloads, and GitHub stars. The descriptive findings are that about 19% of the updated projects feature Kotlin, adopters transition quickly, and the only statistically significant popularity result is an association between Kotlin presence and GitHub star count (p=0.0002). The paper concludes that migration to Kotlin is seamless and 'seems to come at no cost' in terms of popularity.","tokens_in":10418,"tokens_out":2713,"duration_ms":28114,"significance":"If taken at face value, the descriptive contribution is a useful, reproducible snapshot of Kotlin diffusion in open-source Android apps, with clearly defined metrics and a published dataset that other researchers can reuse. The evolution analysis over tagged releases is a reasonable way to quantify the speed of language transition. The inferential claim about popularity, however, is substantially weaker than the abstract and conclusions suggest: the star-count result is an unadjusted cross-sectional association, one of six tests, with an inconsistent test nomenclature, and the paper itself acknowledges that it does not imply causality. The study is therefore valuable mainly as an exploratory characterization, not as evidence that Kotlin adoption causes popularity gains.","major_comments":[{"comment":"The tests are labeled 'Wilkoxon Paired Signed Rank', but the comparison is between independent groups of projects (those with and without Kotlin). If the actual procedure was the rank-sum test described in §2.3 as 'Wilkoxon Rank Sum tests', the table and text should be corrected; if a paired test was truly used, the pairing variable must be specified. As written, the reported p=0.0002 cannot be interpreted faithfully.","section":"§3, Table 5"},{"comment":"The wording 'statistically significant influence' and the conclusion that adoption 'seems to come at no cost in terms of popularity' overstate what the design supports. The comparison does not control for project age, release count, general activity, or developer reputation, and Section 4 explicitly acknowledges that the detected correlation does not imply causality. The manuscript should be revised to describe a statistically significant association, with a discussion of confounders, or should add an adjusted analysis such as matching or regression with control variables.","section":"§3, RQ3 answer; §6; Abstract"},{"comment":"The presence of Kotlin is associated with GitHub stars (p=0.0002) but the majority-of-Kotlin comparison is not (p=0.560). This inconsistency is not discussed, and it weakens the robustness of the popularity claim; the paper should either explain why a larger Kotlin share has no detectable association while mere presence does, or temper the claim accordingly.","section":"§3, Table 5, H_sk0 vs H_sm0"}],"minor_comments":[{"comment":"There are several typographical errors: 'Wilkoxon' should be 'Wilcoxon'; 'provididing' in §2.3; 'one fo' and 'in orderd' in §6; and the p-value reported as '0.06.6' in §3 should be a valid decimal.","section":"Throughout"},{"comment":"The manual verification of the oldest-repository heuristic is described as confirming correctness 'in all cases', but no number of manually checked repositories is reported; please provide the count to support this claim.","section":"§2.2.4"},{"comment":"The bars would be easier to interpret if sample sizes were shown in the plot or caption; currently the reader must cross-reference Table 2 to understand the groups.","section":"Figure 1"},{"comment":"The abbreviation 'Ab.' in the table header and row labels is not expanded in the caption; define it for readability.","section":"Table 3"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a solid descriptive mining study, and the RQ1/RQ2 diffusion and evolution results are worth having. The RQ3 popularity finding is an unadjusted observational correlation dressed in causal vocabulary—the paper itself acknowledges the key caveat in the threats section, but the abstract and conclusion still say Kotlin adoption \"seems to come at no cost in terms of popularity\" and has a \"statistically significant influence\" on stars. Those statements overstate what the design can support.\n\nWhat's actually new: a release-level analysis of 145 F-Droid apps on GitHub showing quick, mostly unidirectional transition from Java to Kotlin, plus a snapshot of ~19% Kotlin-featuring projects among 1,232 apps updated after October 2017. The cross-platform matching to Play Store and GitHub includes manual verification of package-name matches, which is more careful than a lot of mining studies. The metrics are simple but transparent, and they publish the data.\n\nSoft spots: the main one is causal language on RQ3. A Wilcoxon rank-sum test comparing Kotlin vs. non-Kotlin projects cannot separate the effect of Kotlin from project age, activity, developer reputation, or any other confounder. The authors do say in Section 4 that the correlation does not imply causality and that active high-star projects may have moved to Kotlin—that caveat should be in the abstract and conclusion, not buried in threats. Also, the test is labeled \"Wilkoxon Paired Signed Rank\" but the groups being compared are independent (Java-only vs. Kotlin-featuring projects); either the pairing is unspecified or the test name is wrong. That's a real blemish. On multiple testing: p=0.0002 would survive a Bonferroni correction for six tests, so the significance itself isn't the issue. The issue is confounding.\n\nMinor stuff: the 45-day abandonment threshold is arbitrary and not sensitivity-checked; \"0.06.6\" is a typo; the sample for RQ2 (145 projects) is small but the paper flags that.\n\nBottom line: this is a reasonable workshop-level contribution. The descriptive findings are trustworthy, and the limitations are mostly acknowledged. The inferential claim needs to be toned down or controlled for confounders. I'd send it to peer review with a request for revision, not desk-reject. It deserves a serious referee; it's a useful baseline for future language-adoption studies, and the release-level evolution data is a genuine addition.","headline":"Solid descriptive mining study of Kotlin adoption in F-Droid apps; the popularity claim outruns the evidence.","tokens_in":10953,"tokens_out":1802,"would_cite":false,"duration_ms":19100,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"About 19% of F-Droid Android apps updated since October 2017 now use Kotlin code, and their GitHub popularity tends to be higher than Java-only apps.","keywords":["Android apps","Kotlin","Java","F-Droid","GitHub","Play Store","language adoption","empirical software engineering"],"falsifier":"A direct falsifier would be a matching study that pairs each Kotlin-using F-Droid project with a Java-only project of comparable age, activity, and developer reputation, and shows that the star-count difference disappears once these confounders are controlled. If the significance vanishes, the paper's central popularity claim would be unsupported. Alternatively, examining repositories that switched from Java to Kotlin mid-history and finding no change in star acquisition rate before versus after the switch would contradict the claimed influence.","tokens_in":9960,"feed_emoji":"📱","tokens_out":1335,"duration_ms":14832,"temperature":0.7,"pith_summary":"This empirical study mines the F-Droid open-source Android repository, matches projects to their Google Play and GitHub counterparts, and measures how much Kotlin code appears in these apps over time. The central finding is that roughly one in five of 1,232 projects updated after October 2017 features Kotlin, and when Kotlin is adopted the transition is typically fast: most projects quickly move to a majority of Kotlin code. The paper also tests whether Kotlin presence correlates with user and developer popularity, finding a statistically significant link with the number of GitHub stars. For Android developers, the results suggest Kotlin migration is rapid and appears to carry no popularity penalty. A careful reading notes that the authors themselves caution that the star correlation does not imply causality.","feed_headline":"Kotlin reaches 1 in 5 F-Droid Android apps","feed_subtitle":"Apps that switch to Kotlin gain GitHub stars, but causality isn't proven.","key_machinery":"The central objects are a set of diffusion metrics (KRL: Kotlin relative lines of code; KRF: Kotlin relative files; KFPR: Kotlin-featuring projects ratio; KMPR: Kotlin-majority projects ratio) and evolution metrics (KNR, KAR, KMR, KOR over tagged releases), computed by static analysis with the cloc tool and git release histories. These metrics quantify how much Kotlin code exists in the latest snapshot and how it evolved over release history. The statistical engine is the Wilcoxon paired signed-rank test, used to compare popularity distributions between Kotlin and non-Kotlin projects.","core_discovery":"The paper reports that of 1,232 F-Droid projects updated after October 2017, 19% contain Kotlin code, and about 12% have more Kotlin code than Java. Among projects that adopt Kotlin, the migration is quick: on average, 70% of tagged releases contain Kotlin, and in 30% of releases Kotlin completely replaces Java. Using Wilcoxon rank-sum tests on popularity metrics, the study finds that Kotlin presence is statistically associated with more GitHub stars (p = 0.0002), while differences in Play Store ratings and download counts do not reach significance. The authors conclude that Kotlin seems to enable a seamless migration from Java at no cost to app popularity.","pith_inferences":["The paper's own caution that the GitHub-star correlation does not imply causality suggests a probable confounding direction: popular or actively maintained projects may adopt Kotlin earlier, making the star association an effect of project vitality rather than of the language itself. ","The reported pattern of 'diffusion first on GitHub, then on app stores' may indicate that developer-facing signals matter more than user-facing ones for technology adoption in the Android ecosystem. ","A natural testable extension would be a controlled longitudinal study that tracks star counts before and after Kotlin adoption within the same repositories, holding project age and activity fixed, to separate language effect from popularity-driven adoption. ","Given that the study sampled only projects active after October 2017, the results may underrepresent early Kotlin adopters and should be validated on a cohort that includes older, active projects from 2016–2017."],"forward_implications":["Android developers considering Kotlin can expect a rapid, largely unidirectional transition from Java, since projects that adopt Kotlin quickly reach majority-Kotlin code. ","Open-source maintainers can anticipate no observed penalty in user ratings or downloads when switching to Kotlin, based on the study's metrics. ","The statistically significant link between Kotlin presence and GitHub stars may attract researchers to study why developers are drawn to Kotlin-using repositories. ","The dataset of 1,232 projects and their release histories could serve as a baseline for future empirical studies of language adoption in Android development. ","The eight-metric framework (diffusion and evolution ratios) can be reused to track the introduction of other programming languages or major libraries in mobile codebases."],"supporting_citations":[{"why":"Establishes the existence of large Android app datasets from market mining (AndroZoo), providing context for why F-Droid and GitHub counterparts are considered representative.","marker":"[1]"},{"why":"Discusses cross-platform mobile development frameworks, framing the context of language choice for Android developers.","marker":"[2]"},{"why":"Provides an empirical study of code obfuscation in Kotlin apps, citing prior work that motivates investigating Kotlin-specific properties.","marker":"[3]"},{"why":"Supplies the Goal Question Metric paradigm used to structure the study design and metric definitions.","marker":"[4]"},{"why":"Compares Java and Kotlin code quality (code smells), supporting the premise that Kotlin is a viable alternative to Java.","marker":"[14]"},{"why":"Concludes that Kotlin code is more concise and safer than Java, echoing the selling points the paper's results corroborate.","marker":"[15]"}],"fun_headline_variants":["1 in 5 F-Droid Android apps have adopted Kotlin","Rapid Kotlin adoption linked to GitHub stars","12% of Android apps now more Kotlin than Java","Kotlin migration is fast in open-source Android apps"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claim that Kotlin presence is associated with higher GitHub stars assumes that the observed correlation is not driven by other factors such as project age, developer reputation, or overall activity, a caveat the authors explicitly acknowledge. ","fun_headline_variants_meta":{"raw":{"variants":["1 in 5 F-Droid Android apps have adopted Kotlin","Rapid Kotlin adoption linked to GitHub stars","12% of Android apps now more Kotlin than Java","Kotlin migration is fast in open-source Android apps"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001029,"raw_usage":{"total_tokens":4373,"prompt_tokens":1021,"completion_tokens":3352,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":637,"completion_tokens_details":{"reasoning_tokens":3285}},"tokens_in":637,"tokens_out":3352,"duration_ms":22750,"temperature":1.0,"reasoning_tokens":3285,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T12:45:39.002997+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A direct falsifier would be a matching study that pairs each Kotlin-using F-Droid project with a Java-only project of comparable age, activity, and developer reputation, and shows that the star-count difference disappears once these confounders are controlled. If the significance vanishes, the paper's central popularity claim would be unsupported. Alternatively, examining repositories that switched from Java to Kotlin mid-history and finding no change in star acquisition rate before versus after the switch would contradict the claimed influence.","supporting_citations":[{"cited_title":"Allix, T","cited_arxiv_id":null,"evidence_quote":"Establishes the existence of large Android app datasets from market mining (AndroZoo), providing context for why F-Droid and GitHub counterparts are considered representative."},{"cited_title":"Towards the quality improvement of cross- platform mobile applications","cited_arxiv_id":null,"evidence_quote":"Discusses cross-platform mobile development frameworks, framing the context of language choice for Android developers."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides an empirical study of code obfuscation in Kotlin apps, citing prior work that motivates investigating Kotlin-specific properties."},{"cited_title":"Goal question metric paradigm","cited_arxiv_id":null,"evidence_quote":"Supplies the Goal Question Metric paradigm used to structure the study design and metric definitions."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Compares Java and Kotlin code quality (code smells), supporting the premise that Kotlin is a viable alternative to Java."},{"cited_title":"A comparative study: Java vs kotlin programming in android application devel- opment","cited_arxiv_id":null,"evidence_quote":"Concludes that Kotlin code is more concise and safer than Java, echoing the selling points the paper's results corroborate."}],"review_version":1}