{"id":"dccb5fce-984d-4ab5-90da-7a85311b50f1","arxiv_id":"1908.07883","paper_version":3,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Across 7,280 GitHub Scala projects, 98.2% use implicits, 78.2% define them, and about 27% of call sites involve compiler-injected implicit resolution.","lead":"This paper analyzed 7,280 Scala projects on GitHub and measured how often the language's implicit parameters and conversions are used in practice. It found that nearly every project uses them, about one in four call sites involves them, and the analysis pipeline is publicly released.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 27.2% call-site ratio rests on unvalidated SemanticDB extraction; the paper never reports precision/recall, and the denominator may include compiler-inserted synthetic trees.","rationale":"The reader identified corpus representativeness as the weakest assumption. I agree that is a real threat, but I think the more load-bearing link is the extraction itself: even a perfect corpus gives wrong percentages if the extractor systematically misses or fabricates implicit call sites. The paper is unusually transparent, with a released artifact, a described pipeline, and listed limitations such as Scalameta version bounds, macro support, and 3% unresolved symbols. Transparency, however, does not substitute for a precision/recall measurement. The headline percentages are plausible and broadly consistent with the ecosystem; I do not see an internal contradiction that would justify rejection. The concern is that 27.2% is reported with more precision than the toolchain has demonstrated. The right resolution is to keep the verdict CONDITIONAL and require a validation experiment or a re-scoped claim. Since the paper already received CONDITIONAL, my read does not move the verdict.","tokens_in":26908,"tokens_out":12267,"duration_ms":127805,"concrete_test":"Stratified random sample of 20 corpus projects; for 10 call sites per project (200 total), run scalac with -Xprint:typer and manually classify inserted implicit applications, then compare with the artifact's extracted call-site records to compute precision and recall. Also recompute the 27.2% ratio using only call sites whose position maps to an original source tree, excluding purely synthetic application nodes; if the ratio differs by more than 5% relative, the headline overstates source-level usage. The artifact at zenodo.3369436 makes both checks runnable.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The strongest claim is the 27.2% call-site ratio and the 98.2% project-level use figure. Both are outputs of a custom extractor over SemanticDB synthetics described in Sections 3.1 and 5. The paper reports no validation of this extractor against a ground truth. This matters because SemanticDB explicitly includes 'synthetic call sites and parameters injected by the compiler' (Section 3.1), and Section 5 counts 29.6M call sites as 'explicit and implicit combined' without stating whether compiler-inserted synthetic applications are in the denominator. If synthetic nodes are in the denominator, the ratio is a property of the typed tree, not of source-level call sites, so the 'one out of every four call sites' reading is not established. Scalameta's version restrictions and unsupported white-box macros (Sections 3 and 5.6), together with the disclosed 3% unresolved symbols, add further risk of systematic undercounting in exactly the implicit-heavy projects that matter most. This is an internal-validity concern distinct from the external-validity worry about the corpus: even on the compiled Sbt-only corpus, the headline numbers could shift if extraction precision or recall is not high.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper reports a large-scale empirical study of implicit parameters and implicit conversions in Scala. The authors build an automated pipeline that clones GitHub projects, filters to Sbt-based projects with compatible Scala versions, deduplicates them, compiles them with a Scalameta/SemanticDB plugin, and extracts implicit declarations and call sites. On a final corpus of 7,280 projects (18.7M lines of Scala code), the paper reports that 98.2% of projects use implicits, 78.2% define implicit declarations, and about 27.2% of call sites involve implicit resolution. It also classifies implicit idioms (conversions, type classes, contexts, type proofs, etc.), measures the complexity of injected implicit arguments, and analyzes the relationship between implicit density and compilation speed. The artifact is publicly available, and the paper explicitly documents corpus construction steps and failure rates.","tokens_in":27068,"tokens_out":6392,"duration_ms":62307,"significance":"If the headline figures are correct, this is the first large-scale quantitative demonstration of how pervasive Scala implicits are, with direct relevance to language design (e.g., Scala 3's redesign of implicits) and tooling. The pipeline is a reusable, open-source artifact, and the paper is transparent about corpus construction, compilation failures, and the limitations of Scalameta. The reported statistics are derived from a defined pipeline rather than from fitted constants or tuned targets, so the circularity risk is low. The main gaps are the absence of validation of the SemanticDB-based extractor and the lack of rigorous statistical support for the compile-time claims; both are addressable within the scope of the manuscript.","major_comments":[{"comment":"The headline figures—98.2% of projects using implicits and 27.2% of call sites involving implicit resolution—are produced by a custom extractor over SemanticDB synthetic trees, but the paper reports no precision/recall validation of this extractor and never defines the denominator '29.6M call sites (explicit and implicit combined)' operationally. SemanticDB includes synthetic call sites and injected parameters that do not appear in source (Section 3.1), and Section 5.6 discloses that 3% of implicit uses have unresolvable symbols. Without a ground-truth sample (e.g., a manually inspected set of projects or a comparison with scalac's -Yshow-implicit output), the central claim that 'one out of every four call sites' uses implicits could be an artifact of what SemanticDB counts as a call site. This is load-bearing and should be addressed with a validation study and a precise definition of the denominator.","section":"§3.1, §5"},{"comment":"The compile-time claim in Section 5.5 and Figure 11 is supported only by loess curves with no confidence intervals, significance tests, or confounder control. The two groups compared in Figure 11 differ in size (1,969 vs. 488 projects) and in total lines of code (8.4M vs. 2.8M), and the metric 'lines per second' can be affected by project size, dependency count, compiler version, and compilation settings. The statement that 'the cost of compilation increases with the density of implicits' is therefore not established by the presented analysis. The authors should either add a regression or sensitivity analysis with appropriate controls and uncertainty quantification, or weaken the claim to a description of the observed trend.","section":"§5.5, Figure 11"},{"comment":"The corpus is restricted to Sbt-based projects on specific Scala versions that compile, and the pipeline shrinks the initial 65,177 projects to 11,057 and then to 7,326 projects that build successfully. Section 5.6 acknowledges that findings may not generalize to industrial code, but the title and conclusions speak to implicits 'in the wild.' Because the central prevalence figures could shift if excluded projects (other build systems, older Scala versions, or failing builds) use implicits differently, the paper should quantitatively compare included and excluded projects on observable attributes (e.g., project size, star counts, dependency structure) or restrict the conclusions to the analyzed corpus. This is an external-validity concern that is disclosed but not yet quantified.","section":"§4, §5.6"}],"minor_comments":[{"comment":"The abstract reports '370.7K implicit declarations,' while Section 5.2 reports 'over 370.7K implicit parameter declarations'; the terminology should be made consistent.","section":"Abstract, §5.2"},{"comment":"The paper reports both an overall ratio of 27.2% of call sites and a per-project median of 23.4% (Figure 7); the text should clarify which quantity is used in each summary statement.","section":"§5, Figure 7"},{"comment":"The sentence 'Calls sites with implicit parameters are frequent, they account for 46.2% (3.7M) of all Scala call sites' is ambiguous about whether 'all Scala call sites' includes implicit conversion call sites and synthetic trees; this should be defined precisely.","section":"§5.2"},{"comment":"There is a typo: 'Graddle' should be 'Gradle.'","section":"§4"},{"comment":"The relationship between the 1,969 projects with compiler statistics and the 488 projects using shapeless is not stated; the potential overlap should be reported to help interpret the comparison.","section":"§5.5"}],"recommendation":"major_revision","confidential_remarks":"The paper fits OOPSLA's scope as an empirical study of a language feature. The central uncertainty is the unvalidated extractor on which the headline percentages rely; a focused validation study, plus a re-analysis of the compile-time claims, would make the contribution solid. The external-validity limitation is acknowledged but should be quantified or the claims carefully scoped."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. This is the first large-scale quantitative measurement of Scala implicits, and the central claim—that implicits are everywhere—is credible. The exact 27.2% call-site figure is weaker than it looks, because the extractor is never validated against ground truth and the denominator may include compiler-inserted synthetic trees.\n\nWhat is genuinely new is the corpus itself: 7,280 projects, 18.7M LOC, 8.1M implicit call sites, with a released artifact. The paper is transparent about how the corpus was built, including the 65K-to-7.3K attrition, duplicate filtering, build failures, and the 3% unresolved symbols. That is real evidence, not a black box. The pattern taxonomy (type classes, extension methods, contexts, anti-patterns) is useful and the heuristics are concrete enough to audit. The related work is fair and includes prior empirical Scala studies; no citation red flags.\n\nThe stress-test note is on target. SemanticDB explicitly contains synthetic call sites and compiler-injected parameters, and the paper counts 29.6M call sites as explicit-plus-implicit without stating whether synthetic desugarings are in the denominator. No precision/recall numbers are reported for the implicit extractor. That means the 'one out of every four call sites' reading is not fully established; it could be a property of the typed tree, not of source-level calls. This is an internal-validity issue, not just a corpus-representativeness issue, and it deserves a fix. That said, the 98.2% project-level figure is less sensitive to extractor details, and the broad picture of pervasiveness would almost certainly survive a careful validation.\n\nThe soft spots beyond that are minor-to-moderate. The compile-time slowdown section uses loess fits without significance tests or confounder controls, yet the text says the data 'confirms our hypothesis'; that wording should be softened or the analysis strengthened. Calling implicits 'the most used feature of Scala' is an overclaim since no comparison against other features is made. The small/large app LOC cutoff and duplicate thresholds are hand-set, but the paper discloses this and they were not tuned to produce the headline percentages, so I do not treat that as a real problem.\n\nWho this is for: Scala language designers, tool builders, and empiricists studying language feature adoption. I would bring it to a reading group and would cite the corpus-level results. It deserves a serious referee. The right outcome is probably major revision: add extractor validation or at least bound the error, and tone down the causal claims.","headline":"A credible first large-scale measurement of Scala implicits; the pervasiveness result holds, but the headline call-site ratio needs validation and the compile-time claims need tempering.","tokens_in":27609,"tokens_out":3176,"would_cite":true,"duration_ms":32963,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Implicit parameters and implicit conversions are not a niche Scala feature: a 7,280-project corpus shows they appear at 27.2% of all call sites and in 98.2% of projects.","keywords":["Scala","implicit parameters","implicit conversions","call-site analysis","corpus study","type classes","compile-time overhead","language design"],"falsifier":"Re-run the same analysis on the projects the pipeline discarded—those that failed to compile or used other build tools—and check whether the 27.2% call-site ratio and the 98.2% project-level usage rate survive; if a large share of those projects show near-zero implicit use, the universality result is an artifact of the buildable subset.","tokens_in":26664,"feed_emoji":"📊","tokens_out":8099,"duration_ms":72686,"temperature":0.7,"pith_summary":"The paper sets out to show that Scala's implicits—the compiler filling in missing arguments and silently applying type conversions—are a near-universal, load-bearing part of real Scala programs, not an advanced corner of the language. Using a corpus of 7,280 buildable, deduplicated Scala projects, it reports that 98.2% of projects contain at least one implicit-involving call site, that 78.2% declare at least one implicit definition, and that about 27.2% of all call sites involve implicit resolution. It also documents which idioms dominate (type classes and extension methods above all) and reports that the density of implicit call sites correlates with slower compilation. The stakes are concrete: proposed changes to Scala's implicit mechanisms would touch nearly every codebase in the corpus, including 16.2% of projects that would break if unrelated implicit conversions were removed.","feed_headline":"98% of Scala projects rely on implicits, study finds","feed_subtitle":"In 7,280 projects, 27% of call sites involve implicits; resolution-rule changes would touch nearly every Scala codebase.","key_machinery":"The argument rides on a semantic compiler plugin that records 'synthetic trees'—trees the compiler inserts during type checking, such as the hidden executor argument in Future(1) and the injected conversion in 'just like magic!'.enEspanol. Because implicit calls leave little trace in the original source text, syntactic scanning cannot see them; the plugin's record of injected code is what lets each call site be matched to its defining declaration. The pipeline then classifies declarations into idioms—conversions, parameters, type classes, extension methods, contexts, type proofs—and aggregates them across the deduplicated corpus, separating application code, library code, and tests.","core_discovery":"In the paper's own terms: implicits are the most used feature of Scala. Across 7,280 projects with 18.7M lines of code, 98.2% of projects have at least one call site resolved through implicits, 78.2% define at least one implicit value, and 27.2% of all call sites involve implicit resolution, which amounts to 8.1M of 29.6M call sites and 370.7K implicit declarations. Implicit parameters account for 60.3% of these call sites and conversions for the rest, with tests using implicits at roughly twice the rate of application code. The paper classifies the dominant idioms—type classes are the most widely declared and used pattern, followed by extension methods and contexts—and measures a compile-time penalty that grows with implicit density. It concludes that any future design of implicits should limit expressivity to improve comprehensibility and that tooling must agree with the compiler on how resolution works.","pith_inferences":["If Scala's next compiler ships with a new implicit design, this corpus suggests the migration tools should prioritize type classes, extension methods, and context parameters, since those three idioms account for most declarations and call sites.","The compile-time correlation suggests a concrete experiment the paper does not run: instrument a set of these projects with a resolution cache or a restricted search scope and measure whether compilation speed improves by the amount the density curve would predict.","The finding that tests use implicits roughly twice as often suggests that usability studies of implicit-heavy APIs should recruit test-code authors, whose experience may differ from library implementors'.","A tool that visualizes injected implicit calls at each call site could be evaluated against this corpus by checking whether it reduces the time programmers take to explain the injected-code cases such as the 5,695-argument type-level example."],"forward_implications":["Any language change that alters implicit resolution—restricting conversions, changing scope priority, or renaming resolution mechanics—has a wide blast radius: 98.2% of projects use implicits and 78.2% define them.","Deprecating unrelated implicit conversions, a change already discussed in the Scala community, would break 1.2K projects (16.2%) in this corpus.","Because 27.2% of call sites depend on implicit resolution, compiler and IDE implementations that disagree about resolution will mislead a large share of users; the corpus gives a baseline for testing such divergence.","Compile-time optimization of implicit resolution deserves priority: the data show compilation speed falling as implicit call-site density rises, with type-class use compounding the effect.","Testing code is the densest implicit user, so test frameworks and test code are the first place to expect readability and tooling problems."],"supporting_citations":[{"why":"Provided the GitHub project database that defined the initial 65,177-project sample.","marker":"Gousios 2013"},{"why":"Duplicate-detection tool used to strip unofficial copies from the corpus before counting.","marker":"Lopes et al. 2017"},{"why":"Underlies the semantic compiler-plugin infrastructure that records compiler-injected calls, making detection of implicit call sites possible.","marker":"Burmako 2017"},{"why":"Introduced implicit parameters in Haskell, the lineage the paper traces for Scala's feature.","marker":"Lewis et al. 2000"},{"why":"Established the type-class idiom that the paper measures as the most widely used implicit pattern.","marker":"Oliveira et al. 2010"}],"fun_headline_variants":["98% of Scala projects rely on implicits, study finds","Implicits in 98% of Scala projects, 27% of call sites","Scala implicits: 98% of projects, 27% of calls","Study: Implicits most used Scala feature, 98% projects"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The entire study depends on the sample being representative: the 7,280 projects that could be compiled with one build tool and a narrow range of Scala versions stand in for all Scala code in the wild, including projects that failed to build or used other build setups.","fun_headline_variants_meta":{"raw":{"variants":["98% of Scala projects rely on implicits, study finds","Implicits in 98% of Scala projects, 27% of call sites","Scala implicits: 98% of projects, 27% of calls","Study: Implicits most used Scala feature, 98% projects"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000204,"raw_usage":{"total_tokens":1356,"prompt_tokens":877,"completion_tokens":479,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":493,"completion_tokens_details":{"reasoning_tokens":397}},"tokens_in":493,"tokens_out":479,"duration_ms":4791,"temperature":1.0,"reasoning_tokens":397,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T11:54:13.341732+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the same analysis on the projects the pipeline discarded—those that failed to compile or used other build tools—and check whether the 27.2% call-site ratio and the 98.2% project-level usage rate survive; if a large share of those projects show near-zero implicit use, the universality result is an artifact of the buildable subset.","supporting_citations":[],"review_version":1}