{"id":"8a821fa0-c3fd-48b3-95ad-a68cd9607a97","arxiv_id":"2501.11581","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":9,"one_line_summary":"Firms with a moderate, not extreme, share of compatible applications are most likely to open source their LLMs, a relationship shown numerically in a dynamic model and supported by empirical patterns.","lead":"This paper studies why profit-making tech companies release advanced AI models as open source, using patent, benchmark, and GitHub data plus a new economic model. It argues firms open source when they hold a moderate share of compatible applications, and finds evidence that releasing LLaMA increased researchers' activity.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Inverted-U prediction is shown only at one parameter vector (Table 6/Figure 10); absent sensitivity analysis or proof, the paper's central claim is conditional.","rationale":"Reading the paper in good faith, the empirical components are genuinely informative: the LLaMA event study is robust across multiple specifications and falsification checks, and the quality-lead regression has a plausible negative coefficient. The theoretical framework is a reasonable stylization of the open-sourcing trade-off, and the model's qualitative predictions about quality leads and firm size are intuitive. The reader's CONDITIONAL verdict is appropriate, and my stress-test reinforces the condition rather than overturning the paper. The single most load-bearing concern is that the paper's signature prediction—the inverted-U—is only a numerical example under one calibration. This is not an allegation of error; it is a request for evidence. The paper itself flags the parameter values as illustrative, and the absence of a proof for the shape in m is not remedied by the existence of Proposition 1, which only characterizes the threshold in quality. A parameter sweep is cheap for a two-state VFI model and would determine whether the proposed qualitative prediction is a robust property of the model or an artifact of the chosen numbers. Until that is done, the central claim should be stated as conditionally supported, exactly as the reader's verdict says.","tokens_in":29925,"tokens_out":4985,"duration_ms":54864,"concrete_test":"Re-implement the VFI model from Appendix C (or obtain the author's code) and compute the open source window size w(m) for m in [0,1] on a fine grid, sweeping parameters over α in {0.2,0.3,...,0.8}, γ in {0.5,1,2}, φ in {0.1,0.25,0.5,0.75,1}, ψ in {0.25,0.5,0.75,1}, and β in {0.8,0.9,0.99}. For each parameter vector, record whether w(m) is single-peaked with w(0)=w(1)=0. Also refine the state grid, e.g., from 101 to 1001 quality points at the baseline, to rule out discretization artifacts. If the inverted-U fails in a substantial share of plausible parameter space, the central prediction should be downgraded to a calibrated illustration.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central theoretical claim is the inverted-U relationship between firm size m and open-source propensity (Section 6.3). The only evidence is Figure 10, computed by Value Function Iteration at the baseline parameters of Table 6 (β=0.9, γ=1.0, α=0.45, ψ=0.5, ϕ=0.5, λ=5.0, cD=0.4). The paper states these values 'should not be taken literally' (Section 6.2) and provides no sensitivity analysis, no analytical characterization of the shape in m, and no code for reproducibility. The model's economics give an intuitive mechanism: external contributions ϕK_−A shrink and internal profits grow with m, so window size is zero at the extremes. But whether the trade-off is single-peaked in between depends on parameters such as ϕ, γ, and α. Appendix Figure C.2 already shows the open source window is zero for sufficiently small ϕ; if ϕ is low, the inverted-U may vanish or degenerate. Since the summary 'moderate market concentration may be beneficial to open source ecosystems' is the headline contribution (abstract and Section 7), this unvalidated numerical shape is load-bearing.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper studies why for-profit firms open-source advanced LLMs. It constructs a latent technology space from USPTO patent CPC codes and a fine-tuned LLM classifier to show that LLMs are compatible with many technologically diverse firms. A regression on CRFM ecosystem data finds that a larger quality lead over the best open-source model reduces the probability of open-sourcing, that for-profit developers are less likely to open-source, and that Big Tech firms are more likely to do so. An event-study and difference-in-differences analysis around the release of LLaMA finds increased GitHub contributions among LLM researchers. The paper then develops a dynamic discrete-choice model in which a firm owning a share m of LLM-compatible applications trades off closed API revenue against the accelerated quality growth made possible by open-source contributions. The model produces threshold open-source windows, and a numerical value-function iteration predicts an inverted-U relationship between firm size m and open-sourcing propensity. The paper concludes that moderate market concentration may benefit open-source ecosystems of multi-purpose software technologies.","tokens_in":30178,"tokens_out":5367,"duration_ms":59054,"significance":"If the results hold, the paper offers several contributions: a new latent-space method based on hierarchical patent classifications, plausibly causal evidence that open-sourcing LLaMA stimulated research-related activity, and a theoretical framework connecting GPT-like applicability to open-sourcing decisions. The empirical DiD analysis is carefully checked with synthetic DiD, narrow event windows, and alternative treatment definitions. However, the central theoretical prediction is currently supported only by a single numerical experiment at parameters the paper says should not be taken literally, with no sensitivity analysis or reproducible code. Moreover, the model's 'predictions aligned with empirical findings' are not independent confirmations because the model was explicitly motivated by those same findings. The significance of the paper is therefore conditional on strengthening the theoretical robustness and on clarifying the status of the empirical-theoretical alignment.","major_comments":[{"comment":"The inverted-U relationship between firm size and open-sourcing propensity is computed by value-function iteration at the single parameter vector in Table 6, and Section 6.2 states that these values 'should not be taken literally.' The paper provides no analytical characterization of the shape in m, no sensitivity analysis over β, γ, α, ψ, ϕ, λ, and cD, and no code (Appendix C.2 only refers to 'accompanying code'). Because Figure C.2 already shows that the open-source window is zero for sufficiently small ϕ, the inverted-U could be an artifact of the baseline calibration. This is the paper's headline prediction, so it needs either a proof of single-peakedness under stated conditions or a systematic sensitivity analysis with reproducible code and data.","section":"Section 6.3, Figure 10 and Table 6"},{"comment":"The quality-lead regression is estimated on 86 models with available MMLU scores, and score availability is likely correlated with model prominence and open/closed status. The Big-Tech dummy is defined by only three firms, and the specifications include no time fixed effects or developer-level clustering. Since the theoretical framework in Section 6 is explicitly motivated by these empirical regularities, sample selection in this table weakens the empirical grounding. Please report robustness to selection/score imputation, time fixed effects, and alternative clustering, or temper the interpretation accordingly.","section":"Section 5.1, Table 2"},{"comment":"The model is introduced as 'motivated by these findings' and its output is then described as providing 'predictions aligned with empirical findings.' This is not an independent validation: the model is not calibrated to reduced-form moments, and the inverted-U is not tested against data. The theoretical results should be presented as consistency checks, or the paper should provide an out-of-sample empirical test of the inverted-U using the constructed LLM-compatibility measure.","section":"Section 6.3 with Section 1"}],"minor_comments":[{"comment":"The phrase 'Open Soruce' appears in the Figure 10 title and in the surrounding text; this should read 'Open Source.'","section":"Figure 10 and Section 6.3"},{"comment":"Both tables are captioned 'VFI Parameters- Opens Model,' but Table C.5 describes the closed-model solution; the caption of Table C.5 should be corrected.","section":"Appendix C.2, Tables C.4 and C.5"},{"comment":"The profit function is written with inconsistent subscripts as π_{i,τ,t} in the text and π_{i,t,τ} in the equation, and the licensing price appears as both P_{τ,t} and P_t. Please standardize the notation.","section":"Equation (3)"},{"comment":"The table notes refer to a 'Transformer Sim.' column, but the actual column is labeled 'LLM Compatibility'; the note and column header should be aligned.","section":"Table 1"},{"comment":"The reference for Meta (2023b) contains the placeholder '[insert date you accessed the site]', and the text contains typos such as 'daset' in Appendix A.1 and 'Huggingace' in Section 5.1; these should be cleaned up.","section":"References and general text"}],"recommendation":"major_revision","confidential_remarks":"The paper fits the scope of the journal and has interesting ingredients, but the absence of code and data availability despite the 'accompanying code' reference is a reproducibility concern. I would encourage the editor to require a supplementary package as part of the revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThis paper is worth reading. It does three empirical things and one theoretical thing, and the empirical parts are mostly careful. The patent-classification latent space method (CPC n-grams plus SVD) is a real methodological contribution, and the sanity check with known firm pairs is good. The LLaMA event study is the strongest piece: a plausible control group, several robustness checks (synthetic DiD, a narrow window around GPT-4's release, alternative treatment definitions), and the 40–140% range is honestly reported. The quality-lead regression is suggestive but fragile; N=86 with MMLU available only for a selected subset, and the paper itself flags the selection concern.\n\nThe theory is the soft spot. The mechanism is intuitive: small firms want API revenue, large firms don't need the community, and intermediate firms benefit most from open-sourcing. But the inverted-U headline is demonstrated only in one numerical example (Table 6, Figure 10), and the paper explicitly says those parameters should not be taken literally. There is no analytical characterization and no sensitivity analysis. The appendix even shows the open-source window is zero for sufficiently small phi, so the inverted-U could degenerate under plausible parameters. That is the load-bearing claim of Section 6, and it needs more than one hand-picked calibration.\n\nThere is also a mild circularity: the model is 'motivated by' the empirical findings, then generates predictions 'aligned' with the same findings. That is not fatal—the model does produce a genuinely new prediction about firm size—but the paper should be more careful about claiming confirmation.\n\nThe paper is honest about its limitations, which I credit. But the central new prediction is currently conditional, not established. A serious referee should push for (a) sensitivity analysis across the parameter space, (b) at least a partial analytical argument for single-peakedness, and (c) the code, which is referenced but not actually included.\n\nBottom line: this deserves peer review and would be a useful contribution to the economics of open-source AI even after revision. I would bring it to a reading group, and I would cite the patent-space method and the LLaMA results. Just do not let the inverted-U pass without more support.\n\nRecommendation: send it out for review, expecting a major revision.","headline":"A careful empirical package with a genuinely new latent-space method and a solid LLaMA event study, but the headline inverted-U rests on a single numerical calibration with no sensitivity analysis.","tokens_in":30718,"tokens_out":1604,"would_cite":true,"duration_ms":17233,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Owners are most likely to open source an LLM at moderate size, the paper argues.","keywords":["open source","large language models","general-purpose technology","inverted-U relationship","technology compatibility","research spillovers","firm size","LLM ecosystem"],"falsifier":"One check would be to sweep the model's parameters, especially the open-source ecosystem efficiency $\\phi$ and discount factor $\\beta$, and ask whether the open-source window is hump-shaped in the firm-size parameter $m$ on a broad grid; the current paper reports the shape only at its baseline parameter values. A complementary empirical check would estimate the open-sourcing decision against a continuous measure of the owner's share of compatible applications across a larger sample of model releases, rather than the categorical Big-Tech indicator used in the regression.","tokens_in":29676,"feed_emoji":"🤖","tokens_out":6894,"duration_ms":72058,"temperature":0.7,"pith_summary":"This paper argues that a profit-maximizing firm's decision to open source an advanced large language model is a trade-off between accelerating the technology's growth through community contributions and securing immediate revenue by keeping the model closed. The empirical parts show that LLMs are compatible with the research-and-development portfolios of many technologically diverse firms, that a larger quality lead over the best open source alternative makes open sourcing less likely, and that opening a frontier model can stimulate outside research activity. The paper's central prediction is an inverted-U relationship: open sourcing is most likely for owners with an intermediate share of LLM-compatible applications, while very small and very large owners keep their models closed. If correct, this means moderate market concentration could be good for the open source ecosystem of multi-purpose software.","feed_headline":"Owners are most likely to open source an LLM at moderate size","feed_subtitle":"Openness peaks at a mid-sized share of compatible apps, not for tiny or dominant owners.","key_machinery":"The carrying object is a two-state dynamic programming problem with Bellman equations for the value of an open model, $V^O(q)$, and a closed model, $V^C(q, q_B)$, where $q$ is the model's quality and $q_B$ is the quality of the open-source alternative. Open-sourcing is irreversible and lets external compute contribute to next-period quality; a closed model is priced through an API, and the price influences how fast the rival model improves. Proposition 1 establishes a unique quality threshold $q^*$: the firm open sources when $q < q^*$ and stays closed when $q > q^*$. Proposition 2 shows the firm sets its API price below the static revenue-maximizing level to slow the rival's growth. Varying $m$, the mass of applications the firm owns, produces the inverted-U relationship between firm size and the width of the open-source window.","core_discovery":"The author models a tech firm that owns all software producers in a segment of a downstream application sector and can irreversibly open source its higher-quality LLM or license it through an API. Open sourcing sacrifices current API revenue but lets internal and external compute improve the model's quality over time; a closed strategy earns immediate license fees while allowing the rival open model to catch up. Solving the dynamic discrete-choice model shows that the firm opens the model when its quality lead over the open alternative is modest and keeps it closed when the lead is large. The same mechanism delivers the paper's central prediction: as the owner's share of compatible applications grows, openness first rises and then falls, because small firms need API revenue, dominant firms already internalize most applications, and intermediate firms gain the most from community-accelerated growth.","pith_inferences":["The framework implies a testable prediction beyond the paper's sample: as the best open-source alternative improves, the absolute quality lead at which a firm switches from open to closed should also rise, so the open-source window shifts but does not disappear.","The same logic should apply to other general-purpose software whose owners also sell compatible applications, suggesting that ownership share of complements, not firm revenue, is the right predictor of open-sourcing.","The paper stops short of welfare analysis; if open-sourcing accelerates research activity, a social planner might favor openness even for models with large quality leads, because the spillover compounds over time."],"forward_implications":["A model with a large quality lead over the best open source alternative is unlikely to be open sourced, which matches the persistence of closed frontier models.","Large tech firms should open source more often than other for-profit firms, since their broad application portfolios internalize more of the community's quality improvements.","Open sourcing a frontier model can be an R&D catalyst, with the paper's estimates implying a 40 to 140 percent increase in contributions by LLM researchers after the release studied.","Moderate concentration in the application market, not low or high concentration, is what should sustain a vibrant open source ecosystem for general-purpose models."],"supporting_citations":[{"why":"Supplies the Transformer architecture; patents citing it define LLM-related technology in the latent space.","marker":"Vaswani et al. (2017)"},{"why":"Provides the economics-of-open-source framing and the working definition of open source software used throughout.","marker":"Lerner and Tirole (2002)"},{"why":"Establishes scaling laws that motivate why frontier LLM development is costly and why the quality lead matters.","marker":"Kaplan et al. (2020)"},{"why":"Provides the synthetic difference-in-differences estimator used to test robustness of the research-activity result.","marker":"Arkhangelsky et al. (2021)"},{"why":"Gives the latent semantic analysis technique underlying the paper's latent technology space.","marker":"Deerwester et al. (1990)"},{"why":"Supplies the patent-class positioning approach that the paper extends with hierarchical code interactions and dimensionality reduction.","marker":"Jaffe (1986)"},{"why":"Provides the dynamic mixed-duopoly comparison of proprietary versus open platforms on which the theoretical model draws.","marker":"Casadesus-Masanell and Ghemawat (2006)"}],"fun_headline_variants":["Open sourcing peaks at moderate market share","LLM open sourcing peaks for mid-sized owners","For LLMs, openness peaks at moderate size","Mid-sized owners most likely to open source LLMs","Open source likelihood peaks at mid-sized share"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The inverted-U shape is demonstrated by numerical simulation at one illustrative parameter vector, so the claim rests on the assumption that the qualitative shape survives across plausible values of the discount factor, ecosystem efficiency, and firm size rather than being an artifact of that particular calibration.","fun_headline_variants_meta":{"raw":{"variants":["Open sourcing peaks at moderate market share","LLM open sourcing peaks for mid-sized owners","For LLMs, openness peaks at moderate size","Mid-sized owners most likely to open source LLMs","Open source likelihood peaks at mid-sized share"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000536,"raw_usage":{"total_tokens":2539,"prompt_tokens":873,"completion_tokens":1666,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":489,"completion_tokens_details":{"reasoning_tokens":1597}},"tokens_in":489,"tokens_out":1666,"duration_ms":14134,"temperature":1.0,"reasoning_tokens":1597,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T18:05:06.840247+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"One check would be to sweep the model's parameters, especially the open-source ecosystem efficiency $\\phi$ and discount factor $\\beta$, and ask whether the open-source window is hump-shaped in the firm-size parameter $m$ on a broad grid; the current paper reports the shape only at its baseline parameter values. A complementary empirical check would estimate the open-sourcing decision against a continuous measure of the owner's share of compatible applications across a larger sample of model releases, rather than the categorical Big-Tech indicator used in the regression.","supporting_citations":[{"cited_title":"Shazeer, N","cited_arxiv_id":null,"evidence_quote":"Supplies the Transformer architecture; patents citing it define LLM-related technology in the latent space."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the economics-of-open-source framing and the working definition of open source software used throughout."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Gives the latent semantic analysis technique underlying the paper's latent technology space."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the patent-class positioning approach that the paper extends with hierarchical code interactions and dimensionality reduction."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the dynamic mixed-duopoly comparison of proprietary versus open platforms on which the theoretical model draws."}],"review_version":1}