ShapleyLaw: A Game-Theoretic Approach to Multilingual Scaling Laws

Chuan Xiao; Daisuke Kawahara; Jiayi Wang; Makoto Onizuka; Pontus Stenetorp; Qianying Liu; Sadao Kurohashi; Shuyuan Zheng; Xuyang Cao; Yusuke Oda

ShapleyLaw: A Game-Theoretic Approach to Multilingual Scaling Laws

T0 review · 4 major / 2 minor · reviewed 2026-07-13 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Multilingual pretraining is a cooperative game among languages, and Shapley values of each language's contribution yield a scaling law that better predicts test loss and optimizes mixture ratios.

desk verdict Abstract-only: plausible game-theoretic framing for multilingual mixture ratios, but no equations or evidence to check the claim. read the letter →

arxiv 2603.17945 v2 pith:B4NXQR4H submitted 2026-03-18 cs.CL

Xuyang Cao , Qianying Liu , Chuan Xiao , Yusuke Oda , Jiayi Wang , Pontus Stenetorp , Daisuke Kawahara , Makoto Onizuka

show 2 more authors

Sadao Kurohashi Shuyuan Zheng

This is my paper · ORCID

classification cs.CL

keywords multilingualpretrainingscalinglawslanguagemixtureratioscross-lingualtransferShapleyvaluescooperativegametheorytest-lossprediction

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that existing multilingual scaling laws fail because they ignore cross-lingual transfer: how training data in one language improves performance on others. By treating languages as players in a cooperative game whose payoff is the reduction in test loss from any coalition of languages, the authors compute each language's Shapley value—the average marginal contribution of that language across all possible mixtures. Those values become the coefficients of a new scaling law, ShapleyLaw, that predicts test loss under any mixture and therefore can be optimized to choose better training ratios. Experiments reported in the abstract show that this game-theoretic law outperforms prior multilingual scaling laws both at predicting held-out loss and at finding mixture ratios that improve final model quality.

What carries the argument

Shapley values of languages in a cooperative game whose payoff is test-loss reduction: each language's average marginal contribution across all coalitions becomes the weight that captures both its direct utility and its transfer to other languages, and those weights enter the scaling formula used for prediction and mixture optimization.

What would settle it

Train models on a fixed set of languages under several mixture ratios, measure actual test losses, then check whether the losses predicted by ShapleyLaw (using Shapley values estimated from a smaller set of coalitions) match the observed losses more closely than baseline multilingual scaling laws; a clear failure of that match would falsify the claim.

Watch

Extended reading notes

Core claim

Multilingual pretraining can be cast as a cooperative game in which languages are players and the characteristic function is the reduction in test loss achieved by any coalition of languages; the resulting Shapley values quantify each language's true contribution, including cross-lingual transfer, and plug directly into a scaling law (ShapleyLaw) that predicts test loss more accurately and yields better language-mixture ratios than previous methods.

Load-bearing premise

That multilingual pretraining is well-modeled as a cooperative game whose characteristic function is simply the reduction in test loss over coalitions of languages, so that the resulting Shapley values are the right quantities to insert into a scaling law.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

We only have the abstract for 2603.17945, so this is a thin read. The punchline is simple: they cast multilingual pretraining as a cooperative game with languages as players, take Shapley values of test-loss reduction as the measure of each language’s contribution (including transfer), and plug those into a scaling law they call ShapleyLaw. They claim better loss prediction and better mixture ratios than prior multilingual scaling laws that ignore transfer.

What is actually new is the framing. Treating mixture optimization as a cooperative game and using Shapley values for cross-lingual contribution is a legitimate move relative to the usual power-law fits that treat languages more independently. If the full paper delivers a clean characteristic function, a workable approximation for the 2^n coalitions, and a scaling form that reduces to the monolingual case when transfer is zero, that would be a useful tool for people who actually set language ratios in pretraining. Credit where due: the problem is real, and the abstract states a clear, falsifiable claim (outperformance on prediction and optimization).

The soft spots are exactly what you expect from abstract-only. We have no equations, no definition of v(S), no approximation scheme, no baselines, no error bars, no ablations. The stress-test concern is fair: it is not obvious that average marginal contributions over discrete coalitions are the right continuous scalars for mixture ratios, or that v(S) is independent of the complement and of training dynamics. That mapping needs formal justification and consistency checks; without them the outperformance claim is just an assertion. Circularity risk is moderate rather than fatal—Shapley values will depend on how you estimate coalition losses—but we cannot score it from the abstract.

This is for people who care about multilingual data mixture and scaling laws. A serious referee should see the full paper if the methods section is real; I would not desk-reject on the abstract alone. I would not cite it yet and I would not bring it to reading group until we have the math and the numbers. Send it to review if the full text exists and is coherent; otherwise it stays a title and a promise.

Referee Report

4 major / 2 minor

Summary. The manuscript proposes ShapleyLaw, a multilingual scaling law obtained by casting multilingual pretraining as a cooperative game whose players are languages and whose characteristic function is the reduction in test loss realized by coalitions of languages. Shapley values of this game are taken to quantify each language’s contribution, including cross-lingual transfer, and are then inserted into a scaling law that predicts test loss under arbitrary language-mixture ratios and that is used to optimize those ratios. The abstract reports that ShapleyLaw outperforms existing multilingual scaling-law baselines on both loss prediction and mixture optimization.

Significance. If the formal construction is sound and the empirical gains hold, the work would supply a principled, game-theoretic account of cross-lingual transfer inside multilingual scaling laws—an important and currently under-modeled factor in data-mixture design. Explicit transfer quantification via Shapley values, together with claimed improvements in both prediction and optimization, would be of practical interest for multilingual pretraining and of conceptual interest at the intersection of cooperative game theory and scaling laws. Because only the abstract is available, however, neither the derivation nor the experimental evidence can be assessed, so the significance remains conditional.

major comments (4)

[Abstract] Abstract: the central claim rests on treating multilingual pretraining as a cooperative game with characteristic function v(S) equal to test-loss reduction for coalition S. For the resulting Shapley values to isolate cross-lingual transfer usable in continuous mixture optimization, v(S) must be well-defined and essentially independent of languages outside S and of training dynamics beyond the coalition. The abstract supplies neither a formal definition of v nor any consistency check of this independence; without them the mapping from discrete coalitions to continuous ratios is unmotivated and the claimed superiority over baselines cannot be evaluated.
[Abstract] Abstract: embedding discrete Shapley values into a predictive scaling law for continuous mixture ratios requires a functional form that recovers ordinary monolingual scaling when transfer vanishes and that improves prediction when it does not. No such form, limiting-case argument, or recovery check is stated. This is load-bearing for both the prediction and the optimization claims.
[Abstract] Abstract: exact Shapley values require 2^n characteristic-function evaluations. For any realistic number of languages this is intractable, so an approximation scheme (and its error analysis) is indispensable. The abstract does not mention any approximation, sampling method, or complexity bound; without one the method is not practically usable and the reported experiments cannot be interpreted.
[Abstract] Abstract: the experimental claims of outperformance on “model performance prediction and language mixture optimization” are unsupported by any baseline definitions, metrics, error bars, data splits, ablations of the game-theoretic terms, or statistical tests. With only the abstract available these claims are not checkable and therefore cannot underwrite acceptance.

minor comments (2)

[Abstract] The abstract is dense and introduces several technical notions (cooperative game, characteristic function, Shapley contribution, mixture-ratio scaling law) without even brief parenthetical definitions; a short clarifying sentence for each would improve accessibility.
[Abstract] The term “ShapleyLaw” is introduced without indicating whether it denotes a specific closed-form expression, a family of laws, or an estimation procedure; consistent terminology would help.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity detectable from abstract alone; ShapleyLaw is presented as an application of cooperative-game Shapley values to multilingual mixture scaling, with claimed experimental outperformance and no quotable self-referential reduction.

full rationale

Only the abstract is available; it contains no equations, no fitted-parameter definitions, no uniqueness theorems, and no self-citations. The claimed derivation is the framing of multilingual pretraining as a cooperative game (languages as players, test-loss reduction as payoff) whose Shapley values quantify cross-lingual transfer and are then inserted into a scaling law for mixture prediction/optimization. Experiments are asserted to show outperformance over baselines. Because no concrete derivation steps, characteristic-function estimation procedure, or scaling-law functional form appear in the provided text, it is impossible to exhibit any reduction of a 'prediction' to its inputs by construction, any self-definitional loop, any fitted quantity renamed as a prediction, or any load-bearing self-citation chain. Per the analyzer rules, absence of quotable circular steps yields score 0 and an empty steps list; the abstract is treated as self-contained against the limited evidence it supplies. Concerns about whether the cooperative-game mapping is well-motivated or whether the characteristic function re-uses evaluation mixtures are correctness/motivation issues, not circularity.

Assumptions & free parameters 2 free parameters · 3 assumptions · 1 invented entities

Abstract-only: free parameters of any fitted scaling law, exact characteristic function, and coalition sampling scheme are not specified. The ledger records the modeling axioms visible in the abstract and notes that invented entities and fitted constants cannot be enumerated without the full text.

free parameters (2)

language mixture ratios / coalition loss estimates
Any empirical Shapley estimate over languages requires measured losses for many coalitions; those measurements and any subsequent scaling-law coefficients are free parameters fitted to training runs. Values not given in the abstract.
ShapleyLaw scaling coefficients
A scaling law that maps mixture contributions to test loss almost certainly has fitted coefficients; the abstract does not state a parameter-free form.

assumptions (3)

ad hoc to paper Multilingual pretraining can be treated as a cooperative game with languages as players and test-loss reduction as payoff.
Core modeling choice stated in the abstract; not a standard theorem of pretraining.
ad hoc to paper Shapley values of languages under that game correctly quantify cross-lingual transfer for mixture optimization.
Abstract equates contribution in the game with cross-lingual transfer; this identification is assumed, not derived from first principles in the available text.
standard math Standard cooperative-game / Shapley-value definitions apply to finite sets of languages and estimated characteristic functions.
Uses classical Shapley value machinery from cooperative game theory.

invented entities (1)

ShapleyLaw
purpose: Named multilingual scaling law that folds Shapley contributions into loss prediction and mixture optimization.
Introduced in this paper as the proposed method; independent evidence would be external replications or released code/data, neither visible in the abstract.

how reviews work

0 comments

Cite this review

Pith. "Pith review of ShapleyLaw: A Game-Theoretic Approach to Multilingual Scaling Laws." pith.science (2026). https://pith.science/paper/B4NXQR4H

@misc{pith2026260317945,
  author       = {Pith},
  title        = {Pith review of: ShapleyLaw: A Game-Theoretic Approach to Multilingual Scaling Laws},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/B4NXQR4H}},
  note         = {Machine review of arXiv:2603.17945}
}

read the original abstract

In multilingual pretraining, the test loss of a pretrained model is heavily influenced by the proportion of each language in the pretraining data, namely the \textit{language mixture ratios}. Multilingual scaling laws can predict the test loss under different language mixture ratios and can therefore be used to estimate the optimal ratios. However, the current approaches to multilingual scaling laws do not measure the \textit{cross-lingual transfer} effect, resulting in suboptimal mixture ratios. In this paper, we consider multilingual pretraining as a cooperative game in which each language acts as a player that jointly contributes to pretraining, gaining the resulting reduction in test loss as the payoff. Consequently, from the perspective of cooperative game theory, we quantify the cross-lingual transfer from each language by its contribution in the game, and propose a game-theoretic multilingual scaling law called \textit{ShapleyLaw}. Our experiments show that ShapleyLaw outperforms baseline methods in model performance prediction and language mixture optimization.

Discussion (0). Continue with ORCID to comment.

Pith tools

Reviewed July 13, 2026 · model on record in the stance chip above.