Pith. sign in

REVIEW 3 major objections 7 minor 44 references

Pipeline-ordered Shapley is the unique fair credit rule for federated foundation models

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · glm-5.2

2026-07-09 07:50 UTC pith:CK4WYEOT

load-bearing objection Private letter on FedMark-FM (arXiv:2607.07529) the 3 major comments →

arxiv 2607.07529 v1 pith:CK4WYEOT submitted 2026-07-08 cs.GT

FedMark-FM: Auditable, Risk-Adjusted Data Markets for Federated Foundation-Model Adaptation

Phat T. Tran-Truong , Xuan-Bach Le , Minh Nhat Nguyen This is my paper
classification cs.GT
keywords federated learningfoundation modelsdata valuationShapley valuemechanism designdata marketspipeline-ordered valuationretrieval-augmented generation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that when multiple organizations contribute heterogeneous artifacts—retrieval corpora, LoRA adapters, prompts, safety data—to a shared foundation-model pipeline, standard Shapley-based reward allocation is both unfair and strategically exploitable because it treats all contributors as interchangeable and ignores the fact that retrieval precedes prompting, which precedes adaptation, which precedes safety. The authors propose FedMark-FM, a market framework that prices each artifact type separately, estimates marginal contribution under privacy constraints using a stratified surrogate Shapley estimator called S3Val, and converts lower-confidence-bound value estimates into budget-feasible payments that penalize duplication, sybil splitting, poisoned adapters, privacy-budget gaming, and cost inflation. The central theoretical result is Theorem 1: restricting Shapley permutations to respect the serving order (retrieval before prompting before adaptation before safety) is the unique credit rule satisfying linearity, null player, efficiency, within-layer symmetry, and a downstream-realization axiom (A5) that routes each coalition's complementarity dividend to its most downstream layer. Empirically, on FEVER retrieval and generator-backed RAG tracks, FedMark-FM improves held-out downstream accuracy by 7.5–8.1 percentage points over volume, leave-one-out, and FL-Shapley baselines under a prompt-injection poisoner, while selecting zero strategic clients. Split-conformal calibration achieves full lower-bound coverage at mean interval width 0.0141, versus 0.33 for naive intervals. The framework operates at 200–1000-client scale and maintains audit-ready ledgers that can be replayed when models or policies change.

Core claim

The paper's core discovery is that pipeline-ordered valuation—restricting Shapley marginal-credit permutations to respect the serving order of foundation-model pipelines—is the unique credit assignment satisfying standard fairness axioms plus a serving-causal axiom (A5), and that coupling this valuation with lower-confidence-bound, risk-adjusted payments yields a market that rejects strategic clients (sybils, duplicates, poisoners, cost inflaters) while preserving rare specialists and improving downstream task accuracy. The uniqueness proof works by showing that any value satisfying Axioms A1–A5 is pinned on the unanimity basis of cooperative games: upstream members of a coalition receive no

What carries the argument

Pipeline-ordered Shapley value (Eq. 3); S3Val (Secure Surrogate Shapley Valuation) combining contribution sketches, redundancy clustering, stratified coalition sampling, and a learned utility surrogate; lower-confidence-bound payment rule (Eq. 5) with penalties for duplication, manipulation, privacy consumption, cost, and a scarcity bonus; split-conformal calibration for uncertainty intervals; commit-reveal contract cards and signed evidence ledgers for dispute resolution.

Load-bearing premise

The contract utility requires that task quality, safety, robustness, latency, privacy budget, and cost can all be reliably measured for sampled coalitions inside a secure evaluation sandbox. In practice, the prototype uses a linear surrogate model and a cheap coverage utility for scale experiments, and the paper acknowledges that full multi-adapter PEFT valuation is a scale-up experiment rather than the default. If these utility components cannot be measured accurately under

What would settle it

If pipeline-ordered valuation does not satisfy A5 (downstream realization) in some pipeline game, Theorem 1's uniqueness claim fails. If the risk-adjusted payment rule selects strategic clients at a rate comparable to baselines under a held-out poisoner, the practical advantage of FedMark-FM collapses. If split-conformal intervals do not achieve coverage significantly better than naive intervals at comparable widths, the calibration claim is vacuous.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If pipeline-ordered valuation is adopted, organizations contributing upstream artifacts (retrieval corpora) would be paid only for their standalone and same-layer marginal value, not for cross-layer synergies realized downstream—fundamentally changing how revenue is split in federated RAG and adapter markets.
  • The Corollary 1 result (no upstream synergy capture) means identity-splitting or collusion aimed at harvesting downstream complementarity is structurally unprofitable under ordered credit, even before duplicate penalties apply—this could reduce the attack surface for sybil and collusion attacks in federated markets.
  • The commit-reveal contract card protocol and immutable per-round ledgers provide a template for auditable ML data markets where model or policy changes trigger new valuation rounds rather than silent recomputation, which could become a standard for regulatory compliance in federated AI.
  • The split-conformal calibration achieving full coverage at width 0.0141 versus 0.33 for naive intervals suggests that calibrated uncertainty estimates are essential for payment fairness—markets using uncalibrated Shapley approximations may systematically overpay high-variance contributors.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The A5 downstream-realization axiom embodies a specific normative stance—that complementarity value accrues at the serving frontier, not at upstream sources. An operator who believes upstream contributors are entitled to a share of downstream synergy they enable would reject A5 and recover the symmetric Shapley value. The paper acknowledges this is a contract-level design choice, but the market im
  • The contract utility U(C) = w_q Q(C) + w_s S(C) + w_r R(C) - w_l L(C) - w_p Φ(C) - w_k K(C) requires measurable quality, safety, robustness, latency, privacy, and cost signals for sampled coalitions. The prototype uses a linear surrogate and a cheap coverage utility for scale experiments, which means the full multi-component utility has not been stress-tested at 1000-client scale with real RAG eva
  • The serving-neutrality result (Table S15: Δaccuracy ≈ 0 ± 0.014) means pipeline-ordered valuation redistributes credit without changing task quality. This raises the question of whether the redistribution is worth the added complexity. The paper frames it as a payment-fairness guarantee, but if downstream accuracy is the primary objective, unordered S3Val achieves the same result with simpler impl
  • The framework's approximate manipulation resistance (Propositions 2–5) provides bounds conditioned on audit detection probability q, duplicate recall r_dup, and maximum apparent-value gain g_i. These bounds are only as tight as the assumptions on adversary capabilities. An adaptive adversary who can evade semantic clustering or probe hidden safety tests could potentially exceed the assumed g_i bou

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The paper proposes FedMark-FM, a data-market framework for federated foundation-model adaptation that treats heterogeneous artifacts (retrieval corpora, LoRA adapters, prompts, safety data, etc.) as typed, separately priced products. The framework includes S3Val (a stratified, uncertainty-aware Shapley estimator), pipeline-ordered valuation that respects serving causality, risk-adjusted payments with penalties for duplication and manipulation, and an auditable ledger system. A uniqueness theorem (Theorem 1) characterizes the pipeline-ordered value as the unique rule satisfying linearity, null player, efficiency, within-layer symmetry, and a downstream-realization axiom. Experiments on FEVER retrieval, generator-backed RAG, and LoRA tracks show FedMark-FM selecting zero strategic clients while improving downstream accuracy by 7.5-8.1 points over baselines under a held-out prompt-injection poisoner. The paper ships reproducible code, a benchmark harness, and a commit-reveal protocol for auditability.

Significance. The paper addresses a genuine gap: existing FL incentive mechanisms price clients as homogeneous data providers, while foundation-model pipelines involve heterogeneous, pipeline-dependent, privacy-constrained artifacts. The uniqueness theorem for pipeline-ordered valuation (Theorem 1) is cleanly proved via a unanimity-basis argument and provides a principled foundation for the serving-causal credit rule. The de-circularized serving experiment (Table 4) properly separates validation-card selection from held-out test-card scoring, and the split-conformal calibration result (full coverage at width 0.0141) is a concrete falsifiable result. The reproducible code, benchmark harness with JSON schemas, and commit-reveal protocol are notable strengths. However, the central empirical claim bundles two logically separate contributions—pipeline-ordered valuation and risk-adjusted payments—whose individual roles are not cleanly isolated in the headline comparison.

major comments (3)
  1. Table 4 (Section 8.2): The headline 7.5-8.1 point accuracy improvement over Volume, leave-one-out, and FL-Shapley is driven by FedMark-FM selecting zero strategic (poison) clients while baselines select 1-3.6. However, FedMark-FM avoids poisoners through its risk-adjustment layer (Eq. 5: penalties for duplicate risk, manipulation risk, uncertainty discount, hidden probes), not through its valuation mechanism or pipeline ordering. The baselines lack these penalties entirely. The ablation (Table S7) confirms this: removing risk penalties drops strategic selections to 3.0 and R-utility to -0.41. No experiment adds the same risk penalties to a baseline (e.g., FL-Shapley + duplicate/manipulation penalties + hidden probes). Without this control, the accuracy gap cannot be attributed to the S3Val valuation or pipeline-ordered credit that the paper emphasizes as novel contributions. The abstract
  2. Table S15 (Section S8.1): Pipeline-ordered valuation is shown to be serving-neutral (Δ accuracy = +0.004±0.014, all CIs covering zero). The paper is transparent about this in Section 3.6 and Section 9.2, but the abstract still claims pipeline-ordered valuation as a central contribution alongside the empirical accuracy improvement. Since pipeline ordering does not contribute to the accuracy improvement and is a payment-fairness property only, the abstract's framing conflates two results of different significance. The paper should explicitly state in the abstract that pipeline ordering is a payment-fairness guarantee, not an accuracy driver, and separate it from the empirical accuracy claims.
  3. Section 5.4 and Table S3: The prototype uses a linear surrogate g_θ for utility prediction and a cheap coverage utility for scale experiments (200-1000 clients). Table S3 shows FedMark-FM loses to FL-Shapley on raw coverage quality at all three scales (quality gap 0.0326 vs 0.0256 at 204 clients, 0.0851 vs 0.0794 at 504, 0.2257 vs 0.1748 at 1004). The paper acknowledges this but the abstract claims the market 'preserves rare specialists with audit-ready ledgers at 200-1000-client scale' without mentioning that raw selection quality degrades at scale. The contract utility U(C) in Eq. 1 requires six measurable components (Q, S, R, L, Φ, K), but the scale experiments use only a coverage proxy. The gap between the full contract utility and the proxy used at scale should be discussed more prominently as a threat to external validity.
minor comments (7)
  1. Section 3.6, Theorem 1: The proof is correct but the paper should cite the specific result in Faigle and Kern [41] that pipeline-ordered valuation corresponds to, since the characterization is essentially the Shapley value for games under precedence constraints. Making this connection explicit would help readers understand the novelty boundary.
  2. Table 4: The confidence intervals for baselines are wide (e.g., FL-Shapley accuracy 0.3125±0.0624). The paired CIs for the differences are reported in text but should be added to the table for clarity.
  3. Section 6.1, Eq. 5: The default parameter values (λ=0.75, β=0.28, γ=0.20, η=0.75, ρ=0.25) are stated as 'operator policy rather than data-fitted parameters.' It would help to state how sensitive the zero-strategic-selection result is to these defaults, since Table S6 shows some sweeps but the specific combination is not justified.
  4. Table S1: FL-Shapley exceeds FedMark-FM on raw utility (0.1869 vs 0.1792) in the 28-client FEVER market. This is mentioned in Section 9.2 but should be cross-referenced in the main results section (Section 8) for completeness.
  5. Section 8.4, Table 6: At 50 clients, FedMark-FM's CI (0.465±0.091) overlaps substantially with Equal (0.475±0.060) and Shapley-UCB (0.455±0.059). The claim 'best or statistically tied-for-best' is accurate but the text should note that Equal-weighting is competitive, which raises questions about the marginal value of the valuation mechanism at small scale.
  6. Figure S4: The rank sensitivity heatmap shows FedMark-FM's rank degrades to 4.33 when α_rare=0. This is important context for the operating-point dependence and should be mentioned in the main text, not only in the supplement.
  7. The paper uses 'S3Val' and 'FedMark-FM-Bench' as named entities. These are fine as introduced terms, but the paper should clarify whether FedMark-FM-Bench is intended as a community benchmark with a maintained leaderboard or a reproduction harness for this paper's experiments.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for a careful and substantive report. The referee raises three major comments, all of which we find legitimate. (1) The headline accuracy improvement conflates the roles of pipeline-ordered valuation and risk-adjusted payments; we agree this needs an additional ablation (baseline + risk penalties) and revised framing. (2) The abstract conflates pipeline ordering (a payment-fairness property) with the empirical accuracy claim; we agree the abstract should separate these. (3) The scale experiments use a coverage proxy rather than the full contract utility, and the abstract omits the quality degradation at scale; we agree this gap should be discussed more prominently. We address each below.

read point-by-point responses

  1. Referee: Table 4 headline accuracy improvement bundles pipeline-ordered valuation and risk-adjusted payments; no experiment adds risk penalties to a baseline (e.g., FL-Shapley + penalties). Without this control, the accuracy gap cannot be attributed to S3Val or pipeline ordering.

    Authors: The referee is correct that the headline 7.5-8.1 point improvement in Table 4 is driven primarily by the risk-adjustment layer (Eq. 5), not by S3Val's valuation mechanism or pipeline ordering per se. We acknowledge this conflation in the current framing. The ablation in Table S7 confirms that removing risk penalties drops strategic selections to 3.0 and R-utility to -0.41, which is consistent with the referee's reading. We will add the requested control experiment: FL-Shapley (and leave-one-out) augmented with the same duplicate, manipulation, uncertainty, and hidden-probe penalties from Eq. 5. We expect this baseline+penalties variant to close most of the strategic-selection gap, demonstrating that the risk-adjustment layer is the primary accuracy driver. What FedMark-FM contributes beyond the penalty layer is (a) the typed-artifact market structure, (b) the auditable ledger and commit-reveal protocol, (c) the S3Val estimator with calibrated uncertainty that feeds the LCB payment rule, and (d) pipeline-ordered valuation as a payment-fairness guarantee. We will revise the abstract, Table 4 discussion, and Section 8.2 to attribute the accuracy improvement to the risk-adjustment layer specifically, and to position S3Val and pipeline ordering as contributions on different axes (scalable estimation and payment fairness, respectively) rather than as accuracy drivers. We agree this is a necessary revision and will implement it. revision: yes

  2. Referee: Table S15 shows pipeline-ordered valuation is serving-neutral (Δ accuracy = +0.004±0.014, all CIs covering zero). The abstract still claims pipeline-ordered valuation as a central contribution alongside the empirical accuracy improvement, conflating two results of different significance. The abstract should explicitly state that pipeline ordering is a payment-fairness guarantee, not an accuracy driver.

    Authors: We agree. The body of the paper is already transparent about this: Section 3.6 states that ordered valuation 'is therefore a principled payment-fairness guarantee rather than an accuracy heuristic,' and Section 9.2 reiterates this. However, the abstract does not make this distinction clear. The current abstract lists pipeline-ordered valuation as a contribution and then immediately reports the 7.5-8.1 point accuracy improvement, which creates the impression that pipeline ordering contributes to that improvement. It does not, and we will revise the abstract to separate these claims explicitly. Specifically, we will (a) state that pipeline-ordered valuation is a payment-fairness guarantee that changes credit assignment (Spearman 0.76, overlap 0.67) without affecting held-out task accuracy, and (b) attribute the accuracy improvement to the risk-adjusted payment mechanism. We will also add a sentence in the abstract noting that pipeline ordering is serving-neutral on task quality (Δ = +0.004±0.014), so readers understand the two results are independent. The uniqueness theorem (Theorem 1) remains a standalone theoretical contribution characterizing the unique serving-causal credit rule; we will frame it as such rather than tying it to the empirical accuracy numbers. revision: yes

  3. Referee: Section 5.4 and Table S3: prototype uses linear surrogate and cheap coverage utility for scale experiments (200-1000 clients). FedMark-FM loses to FL-Shapley on raw coverage quality at all three scales. Abstract claims market 'preserves rare specialists at 200-1000-client scale' without mentioning quality degradation. Gap between full contract utility (Eq. 1) and proxy should be discussed as threat to external validity.

    Authors: The referee is correct on both points. First, Table S3 shows FedMark-FM loses to FL-Shapley on raw coverage quality at all three scales (quality gap 0.0326 vs 0.0256 at 204 clients, widening to 0.2257 vs 0.1748 at 1004). The paper acknowledges this in Section S1.3 ('FL-Shapley is the strongest raw selector at scale, while FedMark-FM remains above Volume but gives up raw coverage at 1000 clients because its risk/duplication discounts are not tuned for the proxy'), but the abstract's claim that the market 'preserves rare specialists with audit-ready ledgers at 200-1000-client scale' omits this degradation. We will revise the abstract to note that raw selection quality degrades at scale under the coverage proxy. Second, the full contract utility U(C) in Eq. 1 requires six components (Q, S, R, L, Φ, K), but the scale experiments use only a coverage proxy. This is a genuine threat to external validity: the scale experiments validate wall-clock feasibility and the selection pipeline, but not the full utility contract. We will add a prominent discussion in Section 8.4 (and in the limitations of Section 9.2) stating that the scale results use a coverage proxy rather than the full contract utility, that the full utility requires RAG/adapter evaluation calls that are infeasible at 1000-client scale in the prototype, and that the quality degradation at scale may differ under the full utility. We will also note that the risk/duplication discounts were not retuned for the proxy, which partially explains the widening gap. These are honest limitations of the current prototype and should be stated as such. revision: yes

Circularity Check

1 steps flagged

No significant circularity; R-utility metric is partly self-fulfilling but paper separates it from headline accuracy claims

specific steps
  1. fitted input called prediction [Section S4, U_risk definition; Table 5 R-utility column]
    "U_risk(S) = U(S) − α_strat|S ∩ S_strategic| + α_rare 1{S ∩ S_rare ≠ ∅}. The coefficients are fixed by the experiment card."

    FedMark-FM's payment rule (Eq. 5) is explicitly designed to penalize manipulation risk r_i and duplicate risk d_i, driving strategic clients' payments to zero. The R-utility metric then subtracts α_strat times the number of selected strategic clients. So FedMark-FM's strong R-utility score (0.1621 vs −0.19 to −0.25 for baselines in Table 5) is partly by construction: the mechanism that excludes strategic clients is then scored by a metric that penalizes selecting strategic clients. However, the paper is transparent about this: it calls R-utility 'an operator composite rather than an independent accuracy metric' and the headline accuracy claim (Table 4) uses raw held-out accuracy and macro-F1, not R-utility. This is a minor self-reinforcement in a secondary metric, not in the central claim.

full rationale

The paper's two main claims are not circular. (1) Theorem 1 is a standard axiomatic characterization: Axiom A5 (downstream realization) encodes the property that upstream members receive no cross-layer complementarity, and the proof shows uniqueness via the unanimity basis. This is how all Shapley-style characterization theorems work; the axiom is a design choice, not a restatement of the conclusion. The proof is self-contained and cites external work (Faigle-Kern [41], Owen [42]) for context, not for load-bearing logic. (2) The headline empirical claim (Table 4: 7.5-8.1 point accuracy improvement) uses a de-circularized protocol: coalitions are selected on a validation card and served on a disjoint test card, with metrics (accuracy, macro-F1) that do not use attack labels or the payment formula. The paper acknowledges that the advantage comes from the risk-adjustment layer excluding poisoners, not from the valuation mechanism alone (Section 9.2: 'FL-Shapley and Shapley-UCB can exceed it on raw utility'). The ablation (Table S7) confirms risk penalties drive strategic exclusion. The skeptic's concern that no baseline receives the same risk penalties is a confounding/experimental-design issue, not circularity: the test-card accuracy is a genuine downstream measurement, not a restatement of the payment formula. No self-citation chain is load-bearing. Score 2 reflects the minor self-reinforcement in the R-utility composite metric, which the paper itself flags as non-independent.

Axiom & Free-Parameter Ledger

8 free parameters · 10 axioms · 4 invented entities

The axiom ledger reveals that the framework's theoretical guarantees (Propositions 1-5, Theorem 2) depend on several domain assumptions (bounded utilities, surrogate error bounds, sketch distortion bounds, duplicate recall, exchangeability) that are stated but not independently verified at production scale. The payment coefficients are operator policy rather than fitted, which is appropriate. The pipeline-ordered value's uniqueness theorem rests on the non-standard A5 axiom, which the paper honestly frames as a design choice rather than a universal truth.

free parameters (8)
  • λ (uncertainty discount) = 0.75
    Stated as operator policy in Table S11; controls how much uncertainty reduces payment. Sensitivity sweep in Table S6 shows ordering is stable.
  • β (cost penalty) = 0.28
    Stated as operator policy; converts verified cost to payment penalty.
  • γ (privacy-consumption penalty) = 0.20
    Stated as operator policy; converts privacy spend to payment penalty. Swept over {0, 0.05, 0.10, 0.20, 0.35}.
  • η (manipulation-risk penalty) = 0.75
    Stated as operator policy; penalizes manipulation risk score. Ablation (Table S7) shows removing it causes largest degradation.
  • ρ (scarcity bonus weight) = 0.25
    Stated as operator policy; rewards underrepresented domains/artifact types.
  • Utility weights w_q, w_s, w_r, w_l, w_p, w_k = Not specified numerically
    Contract parameters in Eq. 1; stated as 'contract parameters and must be reported as part of the benchmark card' but default values not given in the main text.
  • Duplicate threshold sdup = 0.65 (token) / 0.50 (embedding) / 0.65 (hard paraphrase)
    Thresholds for duplicate detection in Section S8.3; chosen to trade precision vs recall.
  • Audit predicate thresholds τ_w, τ_r, τ_b, h_min = Not specified numerically
    Control when direct audits are triggered in Algorithm 1; stated as configurable but defaults not given.
axioms (10)
  • standard math A1 Linearity: ψ(αU+βV) = αψ(U) + βψ(V)
    Standard Shapley axiom; invoked in Section 3.6 for Theorem 1.
  • standard math A2 Null player: if U(S∪{i}) = U(S) for all S, then ψ_i(U) = 0
    Standard Shapley axiom; invoked in Section 3.6.
  • standard math A3 Efficiency: Σ ψ_i(U) = U(N)
    Standard Shapley axiom; invoked in Section 3.6.
  • standard math A4 Within-layer symmetry: same-layer clients with identical marginal contributions receive equal value
    Relaxation of full symmetry to within-layer; invoked in Section 3.6.
  • domain assumption A5 Downstream realization: upstream members of a coalition receive none of its pure complementarity dividend
    The serving-causal content axiom; stated as a 'deliberate contract-level design choice about serving-time attribution' in Section 3.6. This is the key non-standard axiom.
  • domain assumption Utilities are bounded in [a, b]
    Required for Hoeffding bound in Proposition 1 (Supplementary S1.1).
  • domain assumption Surrogate marginal prediction error is at most ε_sur in expectation
    Required for Proposition 1 and Theorem 2; not independently verified beyond FEVER MAE measurements.
  • domain assumption Privacy-preserving sketches introduce at most ε_priv utility distortion
    Required for Proposition 1; assumed but not independently bounded.
  • domain assumption Duplicate clustering identifies same-source splits with recall at least r_dup
    Required for Proposition 3 (sybil unprofitability); measured at 0.9575 for hard paraphrases but not for adaptive attacks.
  • domain assumption Exchangeability of calibration and test residuals within contract stratum
    Required for split-conformal calibration coverage; paper notes this is 'only approximate under non-IID coalitions' (Supplementary S1.1).
invented entities (4)
  • S3Val (Secure Surrogate Shapley Valuation) independent evidence
    purpose: Estimates Shapley-style marginal contribution under privacy constraints using sketches, clustering, stratified sampling, surrogate modeling, and uncertainty-triggered audits
    The algorithm is fully specified (Algorithm 1) and evaluated on real FEVER and AG News data with measured MAE, rank correlation, and coverage. Code is shipped.
  • FedMark-FM-Bench independent evidence
    purpose: Benchmark harness with four tracks (retrieval, adapter, prompt, preference/safety) and stress-test clients for sybil, duplicate, poison, privacy-gaming, and cost-inflation attacks
    Shipped as runnable code with JSON schemas, toy clients, and a one-command harness. Frozen contract-card hash enables comparable submissions.
  • Pipeline-ordered value ϕ_ord independent evidence
    purpose: Credit rule that restricts Shapley permutations to respect serving precedence (retrieval ≺ prompt ≺ adapter ≺ preference/safety)
    Proven unique under axioms A1-A5 (Theorem 1). The mathematical object is an instance of precedence-constrained Shapley (Faigle & Kern 1992), but the serving-pipeline application and downstream-realization axiom are new.
  • Contract utility U(C) with six components (Q, S, R, L, Φ, K) no independent evidence
    purpose: Multi-objective utility for evaluating coalition value spanning task quality, safety, robustness, latency, privacy, and cost
    The utility function is defined (Eq. 1) but its components are not independently validated as measurable under privacy constraints at scale. The prototype uses a coverage utility proxy for scale experiments (Table S3).

pith-pipeline@v1.1.0-glm · 36255 in / 4300 out tokens · 479107 ms · 2026-07-09T07:50:47.102736+00:00 · methodology

0 comments
read the original abstract

Federated foundation-model adaptation increasingly relies on heterogeneous private artifacts (retrieval corpora, prompts and demonstrations, LoRA adapters, preference and safety data, and update sketches), yet existing federated-learning incentive mechanisms price clients as homogeneous data or update providers. This assumption poorly matches foundation-model pipelines, where contribution value is heterogeneous, non-IID, pipeline-dependent, privacy-constrained, and vulnerable to strategic behavior. We propose FedMark-FM, an auditable, risk-adjusted data-market framework that models clients as sellers of typed artifacts, estimates marginal contribution with S3Val, a stratified, uncertainty-aware Shapley estimator supporting pipeline-ordered valuation, and converts lower-confidence-bound values into budget-feasible payments penalizing duplication, sybil splitting, poisoned adapters, privacy-budget gaming, and cost inflation. We evaluate FedMark-FM-Bench across FEVER retrieval, held-out generator-backed RAG, and trained PEFT/LoRA tracks. Under a held-out prompt-injection poisoner, FedMark-FM improves downstream accuracy by 7.5-8.1 points over volume, leave-one-out, and FL-Shapley while selecting zero strategic clients. Split-conformal calibration reaches full lower-bound coverage at mean width 0.0141, versus 0.33 for naive intervals. We prove pipeline-ordered valuation is the unique credit rule respecting serving causality, and show it materially changes credit assignment (Spearman 0.76, selected-set overlap 0.67) while leaving held-out task quality unchanged; the market preserves rare specialists with audit-ready ledgers at 200-1000-client scale. FedMark-FM shows incentives for federated foundation models can be engineered as auditable data infrastructure coupling valuation, mechanism design, privacy interfaces, and pipeline-order semantics.

Figures

Figures reproduced from arXiv: 2607.07529 by Minh Nhat Nguyen, Phat T. Tran-Truong, Xuan-Bach Le.

Figure 1
Figure 1. Figure 1: Heterogeneous contribution taxonomy: RAG corpora, prompts, [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Architecture of FedMark-FM. Solid arrows are data/control flow; dashed arrows are audit/dispute flow. [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Single market-round dataflow, from contract-card commitment [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: FedMark-FM-Bench submission flow. TABLE 5 Real-data market validation. R-utility is risk-adjusted utility. Spearman is rank correlation with leave-one-out values. AUC measures harmful-client ranking. Track Method Utility R-utility Spearman AUC Strategic FEVER RAG Equal 0.0219±0.0308 −0.1881±0.0308 -0.5874 0.5000 2.0 FEVER RAG Volume −0.0386±0.0308 −0.2485±0.0308 -0.0489 0.7500 2.0 FEVER RAG Leave-one-out 0… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

44 extracted references · 44 canonical work pages · 12 internal anchors

  1. [1]

    Advances and open problems in federated learning,

    P . Kairouz, H. B. McMahan, B. Avent, A. Bellet, M. Bennis, A. N. Bhagoji, K. Bonawitz, Z. Charles, G. Cormode, R. Cummingset al., “Advances and open problems in federated learning,”Foundations and Trends in Machine Learning, vol. 14, no. 1–2, pp. 1–210, 2021

    work page 2021

  2. [2]

    Federated learning incentive mechanism design via shapley value and pareto optimality,

    X. Yang, S. Xiang, C. Penget al., “Federated learning incentive mechanism design via shapley value and pareto optimality,”Ax- ioms, vol. 12, no. 7, p. 636, 2023

    work page 2023

  3. [3]

    Data shapley: Equitable valuation of data for machine learning,

    A. Ghorbani and J. Zou, “Data shapley: Equitable valuation of data for machine learning,” inProceedings of the 36th International Conference on Machine Learning, 2019, pp. 2242–2251

    work page 2019

  4. [4]

    Efficient task-specific data valuation for nearest neighbor algorithms,

    R. Jia, D. Dao, B. Wang, F. A. Hubis, N. Hynes, N. M. G ¨urel, B. Li, C. Zhang, D. Song, and C. J. Spanos, “Efficient task-specific data valuation for nearest neighbor algorithms,”Proceedings of the VLDB Endowment, vol. 12, no. 11, pp. 1610–1623, 2019

    work page 2019

  5. [5]

    Ten challenging problems in federated foundation models,

    T. Fan, H. Gu, X. Caoet al., “Ten challenging problems in federated foundation models,”IEEE Transactions on Knowledge and Data Engineering, vol. 37, no. 7, pp. 4314–4337, 2025

    work page 2025

  6. [6]

    A value for n-person games,

    L. S. Shapley, “A value for n-person games,” inContributions to the Theory of Games II. Princeton University Press, 1953, pp. 307–317

    work page 1953

  7. [7]

    Federated Learning for Data Market: Shapley-UCB for Seller Selection and Incentives

    K. Chen and Z. Xu, “Federated learning for data market: Shapley-ucb for seller selection and incentives,”arXiv preprint arXiv:2410.09107, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  8. [8]

    Fedave: Adaptive data value evaluation frame- work for collaborative fairness in federated learning,

    Z. Wanget al., “Fedave: Adaptive data value evaluation frame- work for collaborative fairness in federated learning,”Neurocom- puting, vol. 574, p. 127227, 2024

    work page 2024

  9. [9]

    Incentive mecha- nism of foundation model enabled cross-silo federated learning,

    N. Zhang, X. Xu, X. Liu, J. Wu, and H. Tang, “Incentive mecha- nism of foundation model enabled cross-silo federated learning,” Scientific Reports, vol. 15, p. 24181, 2025

    work page 2025

  10. [10]

    Counterspeculation, auctions, and competitive sealed tenders,

    W. Vickrey, “Counterspeculation, auctions, and competitive sealed tenders,”Journal of Finance, vol. 16, no. 1, pp. 8–37, 1961

    work page 1961

  11. [11]

    Multipart pricing of public goods,

    E. H. Clarke, “Multipart pricing of public goods,”Public Choice, vol. 11, pp. 17–33, 1971

    work page 1971

  12. [12]

    Incentives in teams,

    T. Groves, “Incentives in teams,”Econometrica, vol. 41, no. 4, pp. 617–631, 1973

    work page 1973

  13. [13]

    Eliciting informative feedback: The peer-prediction method,

    N. Miller, P . Resnick, and R. Zeckhauser, “Eliciting informative feedback: The peer-prediction method,” inManagement Science, vol. 51, no. 9, 2005, pp. 1359–1373

    work page 2005

  14. [14]

    FL-Market: Trading Private Models in Federated Learning

    Z. Jiang, Y. Cao, Y. Wang, H. Chen, and C. Xu, “FL-Market: Trading private models in federated learning,”arXiv preprint arXiv:2106.04384, 2021

    work page internal anchor Pith review Pith/arXiv arXiv 2021

  15. [15]

    Mitigating Sybils in Federated Learning Poisoning

    C. Fung, C. J. M. Yoon, and I. Beschastnikh, “Mitigating sybils in federated learning poisoning,”arXiv preprint arXiv:1808.04866, 2018

    work page internal anchor Pith review Pith/arXiv arXiv 2018

  16. [16]

    Understanding black-box predictions via influence functions,

    P . W. Koh and P . Liang, “Understanding black-box predictions via influence functions,” inProceedings of the 34th International Conference on Machine Learning, 2017, pp. 1885–1894

    work page 2017

  17. [17]

    Data valuation using reinforce- ment learning,

    J. Yoon, S. O. Arik, and T. Pfister, “Data valuation using reinforce- ment learning,” inProceedings of the 37th International Conference on Machine Learning (ICML), ser. Proceedings of Machine Learning Research, vol. 119, 2020, pp. 10 842–10 851

    work page 2020

  18. [18]

    Beta shapley: A unified and noise-reduced data valuation framework for machine learning,

    Y. Kwon and J. Zou, “Beta shapley: A unified and noise-reduced data valuation framework for machine learning,” inInternational Conference on Artificial Intelligence and Statistics, 2022, pp. 8780– 8802

    work page 2022

  19. [19]

    Fair document valuation in llm summaries via shapley values,

    Z. Ye and H. Yoganarasimhan, “Fair document valuation in llm summaries via shapley values,”arXiv preprint arXiv:2505.23842, 2025. IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING 12

    work page internal anchor Pith review arXiv 2025

  20. [20]

    Data auctions for retrieval augmented generation,

    M. Han, S. A. Esmaeili, M. Albert, and H. Xu, “Data auctions for retrieval augmented generation,”arXiv preprint arXiv:2508.16007, 2025

    work page arXiv 2025

  21. [21]

    Retrieval- augmented generation for knowledge-intensive nlp tasks,

    P . Lewis, E. Perez, A. Piktus, F. Petroni, V . Karpukhin, N. Goyal, H. K ¨uttler, M. Lewis, W.-t. Yih, T. Rockt ¨aschelet al., “Retrieval- augmented generation for knowledge-intensive nlp tasks,” in Advances in Neural Information Processing Systems, 2020

    work page 2020

  22. [22]

    Lora: Low-rank adaptation of large language mod- els,

    E. J. Hu, Y. Shen, P . Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen, “Lora: Low-rank adaptation of large language mod- els,” inInternational Conference on Learning Representations, 2022

    work page 2022

  23. [23]

    Training language models to follow instructions with human feedback,

    L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. L. Wainwright, P . Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Rayet al., “Training language models to follow instructions with human feedback,” in Advances in Neural Information Processing Systems, 2022

    work page 2022

  24. [24]

    Heterogeneous LoRA for Federated Fine-tuning of On-Device Foundation Models

    Y. J. Cho, L. Liu, Z. Xu, A. Fahrezi, and G. Joshi, “Heterogeneous lora for federated fine-tuning of on-device foundation models,” arXiv preprint arXiv:2401.06432, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  25. [25]

    Robust Federated Finetuning of Foundation Models via Alternating Minimization of LoRA

    S. Chen, Y. Ju, H. Dalal, Z. Zhu, and A. Khisti, “Robust federated finetuning of foundation models via alternating minimization of lora,”arXiv preprint arXiv:2409.02346, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  26. [26]

    FedEx-LoRA: Exact Aggregation for Federated and Efficient Fine-Tuning of Foundation Models

    R. Singhal, K. Ponkshe, and P . Vepakomma, “FedEx-LoRA: Exact aggregation for federated and efficient fine-tuning of foundation models,”arXiv preprint arXiv:2410.09432, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  27. [27]

    LoRA-FAIR: Federated LoRA fine-tuning with aggregation and initialization refinement,

    J. Bian, L. Wang, L. Zhang, and J. Xu, “LoRA-FAIR: Federated LoRA fine-tuning with aggregation and initialization refinement,” arXiv preprint arXiv:2411.14961, 2024

    work page arXiv 2024

  28. [28]

    FLoRA: Federated fine-tuning large language models with heterogeneous low-rank adaptations,

    Y. Wanget al., “FLoRA: Federated fine-tuning large language models with heterogeneous low-rank adaptations,” inAdvances in Neural Information Processing Systems (NeurIPS), 2024

    work page 2024

  29. [29]

    Stabilized fine-tuning with lora in federated learning: Mitigating the side effect of client size and rank via the scaling factor,

    J. Huang, X. Wu, T. He, and Q. Lao, “Stabilized fine-tuning with lora in federated learning: Mitigating the side effect of client size and rank via the scaling factor,”arXiv preprint arXiv:2603.08058, 2026

    work page arXiv 2026

  30. [30]

    WinFLoRA: Incentivizing client-adaptive aggregation in feder- ated LoRA under privacy heterogeneity,

    M. Kou, X. Xia, Z. Wang, I. Khalil, R. Luo, J. Zhou, and M. Xue, “WinFLoRA: Incentivizing client-adaptive aggregation in feder- ated LoRA under privacy heterogeneity,” inProceedings of the ACM Web Conference 2026, 2026, pp. 5241–5252

    work page 2026

  31. [31]

    CoRAG: Collaborative retrieval-augmented generation,

    A. Muhamed, M. Diab, and V . Smith, “CoRAG: Collaborative retrieval-augmented generation,” inProceedings of the 2025 Con- ference of the Nations of the Americas Chapter of the Association for Computational Linguistics (NAACL): Short Papers, 2025, pp. 265–276

    work page 2025

  32. [32]

    Privacy-Preserving Federated Embedding Learning for Localized Retrieval-Augmented Generation

    Q. Maoet al., “FedE4RAG: Privacy-preserving federated embed- ding learning for retrieval-augmented generation,”arXiv preprint arXiv:2504.19101, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  33. [33]

    Data Measurements for Decentralized Data Markets

    C. Lu, M. M. Amiri, and R. Raskar, “Data measurements for decentralized data markets,”arXiv preprint arXiv:2406.04257, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  34. [34]

    Towards data valuation via asymmetric data shapley,

    X. Zheng, X. Chang, R. Jia, and Y. Tan, “Towards data valuation via asymmetric data shapley,”arXiv preprint arXiv:2411.00388, 2024

    work page arXiv 2024

  35. [35]

    Rethinking data value: Asymmetric data shapley for structure-aware valuation in data markets and machine learning pipelines,

    X. Zheng, Y. Huang, X. Chang, R. Jia, and Y. Tan, “Rethinking data value: Asymmetric data shapley for structure-aware valuation in data markets and machine learning pipelines,”arXiv preprint arXiv:2511.12863, 2025

    work page arXiv 2025

  36. [36]

    Precedence-Constrained Winter Value for Effective Graph Data Valuation

    H. Chi, Z. Yang, L. Zeng, W. Fan, and Y. Ma, “Precedence- constrained winter value for effective graph data valuation,”arXiv preprint arXiv:2402.01943, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  37. [37]

    Owen sam- pling accelerates contribution estimation in federated learning,

    H. KhademSohi, H. Hemmati, J. Zhou, and S. Drew, “Owen sam- pling accelerates contribution estimation in federated learning,” arXiv preprint arXiv:2508.21261, 2025

    work page arXiv 2025

  38. [38]

    Data-Free Contribution Estimation in Federated Learning using Gradient von Neumann Entropy

    A. Ukaye, M. Abdu-Aguye, N. Tastan, and K. Nandakumar, “Data-free contribution estimation in federated learning using gradient von neumann entropy,”arXiv preprint arXiv:2604.22562, 2026

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  39. [39]

    C-FedRAG: A Confidential Federated Retrieval-Augmented Generation System

    P . Addison, M.-T. H. Nguyen, T. Medan, J. Shah, M. T. Manzari, B. McElrone, L. Lalwani, A. More, S. Sharma, H. R. Roth, I. Yang, C. Chen, D. Xu, Y. Cheng, A. Feng, and Z. Xu, “C-FedRAG: A confidential federated retrieval-augmented generation system,” arXiv preprint arXiv:2412.13163, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  40. [40]

    Fweb3: A practical incentive-aware federated learning framework,

    P . Yan, S. Liang, Y. Hua, L. Jiang, K. Yu, Y. Sun, Y. Zhang, T. Song, N. Hu, X. Liang, B. He, and H. Guan, “Fweb3: A practical incentive-aware federated learning framework,”arXiv preprint arXiv:2603.00666, 2026

    work page arXiv 2026

  41. [41]

    The Shapley value for cooperative games under precedence constraints,

    U. Faigle and W. Kern, “The Shapley value for cooperative games under precedence constraints,”International Journal of Game Theory, vol. 21, no. 3, pp. 249–266, 1992

    work page 1992

  42. [42]

    Values of games with a priori unions,

    G. Owen, “Values of games with a priori unions,” inMathematical Economics and Game Theory. Springer, 1977, pp. 76–88

    work page 1977

  43. [43]

    Concentrated differential privacy: Simpli- fications, extensions, and lower bounds,

    M. Bun and T. Steinke, “Concentrated differential privacy: Simpli- fications, extensions, and lower bounds,” inTheory of Cryptography Conference, 2016, pp. 635–658

    work page 2016

  44. [44]

    R ´enyi differential privacy,

    I. Mironov, “R ´enyi differential privacy,” in2017 IEEE 30th Com- puter Security Foundations Symposium (CSF). IEEE, 2017, pp. 263– 275. IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING 13 SUPPLEMENTARYMATERIAL This supplement contains the appendices referenced from the main paper; its sections, tables, figures, and equations are numbered with an “S” pr...

    work page 2017