{"id":"94f69a70-00b4-40f7-9f2f-76eed17149b2","arxiv_id":"2412.16969","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"MRFF, a federated sequential recommender with a group gating network and private user FFNs, improves CTR prediction over FedSASRec, FedHSTU, and FedLLaMA on three Kuai datasets.","lead":"This paper adds a user-group routing mechanism to federated sequential recommenders, so each user gets a private personal model plus shared group-level models. On three short-video datasets, the approach improves click prediction, especially log-loss, over three transformer backbones.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reported gains over FedSASRec, FedHSTU, and FedLLaMA are confounded by added parameters: Eq. 5 doubles the FFN branch and adds a gating MLP, yet no equal-capacity baseline or component ablation is reported, so the central performance claim is not yet isolated.","rationale":"I am not rejecting the paper: the direction of the results is plausible, the code is released, and the authors report five repetitions with t-tests. But the experimental design does not yet rule out the simplest alternative explanation—that the increase in parameters, not the proposed user-grouping mechanism, produces the gains. The reader's weakest assumption about privacy is also real: the paper does not specify how the server aggregates group-level FFNs without learning group assignments, and Eq. 7 requires the global group proportion f_i^l, which local clients cannot compute unless the server broadcasts it. That privacy and protocol gap should remain a condition for acceptance. However, for the strongest claim as stated (consistent AUC and LogLoss improvements), the capacity/ablation confound is the more direct threat. I therefore keep the verdict conditional rather than moving it; the requested additions are an equal-capacity baseline, component ablations, and a precise description of the group-aggregation protocol with or without formal differential privacy guarantees.","tokens_in":16601,"tokens_out":8396,"duration_ms":80552,"concrete_test":"Run a matched-capacity control: give FedSASRec two standard FFNs per transformer layer (or an ungated mixture-of-experts layer) with total parameters equalized to MRFF's 59.78k count, then compare AUC and LogLoss on KuaiRand-Pure and KuaiSAR-S. If the control reproduces Table 1's gains, the group-gating mechanism is not the active ingredient. Also run an MRFF ablation with group gating replaced by random uniform assignment to test whether the gating network itself contributes beyond added capacity.","verdict_should_be":"UNCHANGED","load_bearing_attack":"MRFF's central performance claim is that the proposed mechanism—hierarchical group gating plus a private user-specific FFN—is what improves AUC and LogLoss. The evidence in Table 1, however, compares MRFF-augmented backbones only to the untouched backbones. Eq. 5 replaces a single FFN per transformer block with two parallel FFNs (FFN_u and FFN_g) and adds an MLP-based gating network, so MRFF has strictly more parameters and capacity than the baselines. Table 3 reports MRFF parameter counts around 52–68k but gives no baseline counts, making the capacity gap unmeasured. The large LogLoss drops with small AUC changes (e.g., FedSASRec on KuaiSAR-S: 0.7034 to 0.5259) are exactly the signature of a calibration shift or extra model capacity, not necessarily of better user-group modeling. Without an equal-parameter control and without ablations that remove either the private FFN or the group gating network, the experiments do not isolate the mechanism that drives the headline numbers. This is load-bearing because the table is the paper's only direct support for the strongest claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MRFF (Multifaceted user modeling in Recommendations with Federated Foundation models), a federated sequential recommendation framework in which each client trains a lightweight transformer from scratch. Within each transformer block, MRFF adds a group gating network that routes the user to one of several group-level FFNs, while a private, user-specific FFN is kept locally. The server aggregates the public and group parameters, and a balance loss regularizes the group distribution toward uniformity. Experiments on three Kuaishou datasets (KuaiRand-Pure, KuaiSAR-R, KuaiSAR-S) report consistent AUC/LogLoss improvements over FedSASRec, FedHSTU, and FedLLaMA backbones, plus two non-sequential baselines. The paper also reports efficiency and a noise-based privacy analysis.","tokens_in":16858,"tokens_out":5619,"duration_ms":46966,"significance":"If the reported gains are attributable to the proposed mechanism, MRFF would be a practically useful way to combine user-specific and group-level personalization in federated recommendation without relying on large pretrained models. The work has clear strengths: it is architecture-agnostic (shown across three backbones), the code is released, experiments are repeated five times with significance tests, and the balance-loss idea is simple and easy to replicate. However, the current evidence does not isolate the mechanism that drives the headline numbers, the federated privacy protocol is underspecified, and the balance-loss analysis is partially circular. These issues must be addressed before the central claims can be accepted.","major_comments":[{"comment":"The headline improvement of MRFF over its backbones is not isolated from added model capacity. Eq. (5) replaces the single FFN in each transformer block with two parallel FFNs (FFN_u and FFN_g) plus an MLP gating network, so the MRFF variants have strictly more parameters than the untouched backbones. Table 3 reports parameter counts only for MRFF (51.75k–67.89k) and does not give the corresponding backbone counts, so the capacity gap is unmeasured. The large LogLoss reductions with small AUC changes (e.g., FedSASRec on KuaiSAR-S: LogLoss 0.7034 to 0.5259 with only 0.84% AUC gain) are consistent with a calibration shift from extra capacity rather than with better user-group modeling. To support the central claim, please add (i) an equal-capacity control (e.g., a wider single-FFN backbone matched to the MRFF parameter count) and (ii) ablations that remove the group gating network or the private user-specific FFN, reporting standard deviations for all configurations.","section":"Methodology, Eq. (5); Table 1"},{"comment":"The federated protocol is underspecified at the point where group-level aggregation happens. The server averages group FFNs by group (Figure 1), but the paper never states how the server learns a client's group assignment without that assignment acting as a user-behavior signal. If clients upload their group ID, the server can infer coarse behavioral clusters; if they do not, the group averaging in Figure 1 cannot be executed as described. Additionally, the privacy experiments in Table 2 add noise to uploaded parameters but give no mechanism, no privacy budget ϵ, and no formal guarantee, so the 'privacy-preserving' claim is not yet supported. Please specify the exact messages exchanged per round and provide a differential-privacy analysis (or explicitly weaken the privacy claim to 'parameter separation').","section":"Overall Framework / Figure 1; Discussions about Practical Viability; Table 2"},{"comment":"Figure 2 does not validate the balance-loss design because Eq. (7) explicitly penalizes non-uniform group proportions; the figure merely shows the regularizer achieving its target. The claim that a uniform (balanced) group distribution is beneficial for group-level personalization is an unverified assumption. Figure 3 sweeps α over {0.0001, 0.001, 0.01, 0.1} but does not include α = 0, so it cannot show that the balance loss improves—or is even needed for—accuracy. Please report performance for α = 0 and, ideally, for a version with a fixed random balanced assignment to separate the effect of balance from the effect of learned routing.","section":"Efficacy Analysis of Balance Loss; Eq. (7); Figures 2–3"},{"comment":"The empirical evidence is confined to three Kuaishou short-video datasets (KuaiRand-Pure, KuaiSAR-R, KuaiSAR-S), which share platform and domain characteristics. This limits the generality of the 'multifaceted user modeling' claim; adding a non-Kuaishou dataset (e.g., Amazon, MovieLens, or a news dataset) would strengthen the cross-domain relevance. Also, Table 1 reports only AUC and LogLoss; for sequential recommendation, a ranking metric such as NDCG@k or Recall@k would show whether the improvements affect ranking quality and not only calibration.","section":"Experiment, Experimental Setup; Table 1"}],"minor_comments":[{"comment":"Equation (4) has unbalanced parentheses: the expression `softmax(MLP(Concat(Attentionout, uemb))` is missing one closing parenthesis.","section":"Methodology, Eq. (4)"},{"comment":"The caption of Figure 3 contains a typo: 'performace' should be 'performance'; in the same subsection, 'maintains a and uniform user distribution' should read 'maintains a uniform user distribution.'","section":"Figure 3 caption; Efficacy Analysis text"},{"comment":"The paper states that experiments were repeated five times and average results are reported, but Table 1 does not show standard deviations or confidence intervals; please add them to support the significance tests.","section":"Experiment, Implementation Details; Table 1"},{"comment":"The term 'foundation model' is used for a transformer trained from scratch with ~60k parameters and two blocks, which conflicts with common usage (large-scale pretrained models). Please rename to 'lightweight transformer' in the title and abstract, or justify the terminology explicitly.","section":"Introduction / Methodology"},{"comment":"The related work cites FedPA (Zhang et al., IJCAI 2024b) as a leading foundation-model-based federated recommendation method, but no personalized federated learning baseline (e.g., Per-FedAvg, pFedMe, FedPer) is compared; adding one such baseline would better position MRFF in the federated personalization literature.","section":"Related Work / Baselines"},{"comment":"The batch sizes for the three transformer-based backbones are not reported (only the non-sequential baselines are given as 1,024), which hampers reproducibility; please report these values.","section":"Experiment, Implementation Details"}],"recommendation":"major_revision","confidential_remarks":"The manuscript's central claim is plausible and the code is released, but the experimental isolation is currently insufficient: the main table compares MRFF-augmented backbones only against the untouched backbones, with no capacity-matched control or component ablation. The paper also leans on the phrase 'foundation model' for a model with ~60k parameters, which may attract criticism; I do not see this as a scientific error, but it may affect fit with the venue's expectations. The use of a same-group baseline (FedPA) is not improper, but the absence of any personalized-FL baseline weakens the comparative positioning."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"One thing to know: MRFF is a sensible combination of known parts—group-gating/MoE routing, a balance loss, and private per-user FFNs—dropped into federated sequential recommendation. That combination appears to be new, and it is evaluated on three backbones with code released. But the main performance claim is not yet isolated: the proposed layer has more parameters than the backbone it replaces, and no equal-capacity baseline or component ablation is reported.\n\nThe new part is Eq. 5: each transformer block runs both a private user FFN and a group FFN selected by a gating MLP, with a balance loss to keep group sizes uniform. Sticking this into FedSASRec, FedHSTU, and FedLLaMA gives consistent improvements on KuaiRand-Pure, KuaiSAR-R, and KuaiSAR-S, with significance stars. That consistency and the released code are real strengths. The efficiency numbers and the noise-robustness experiment are useful practical checks.\n\nThe soft spots are proportionally large. First and most important, the stress-test critique holds: you are comparing a two-FFN-plus-gating block to a one-FFN block, and Table 3 gives no baseline parameter counts. The big LogLoss drops with small AUC changes are exactly what a calibration change or added capacity would produce. The paper needs an equal-parameter control (e.g., widen the single FFN) and ablations dropping the group gating or the private FFN. Without those, the table cannot support the causal claim that group-level routing is doing the work.\n\nSecond, the federated protocol is underspecified: the server aggregates group FFNs, but the paper never explains how group membership is communicated without leaking which users are in which group, and no formal privacy analysis is given. The DP noise experiment is suggestive, not a guarantee. Third, there are no personalized federated learning baselines (FedPer, Per-FedAvg, etc.), and standard deviations are not reported despite five runs. Finally, calling a 60k-parameter model trained from scratch a 'foundation model' is a stretch; it is a small transformer.\n\nI would send this to peer review rather than desk reject. A good referee should ask for equal-capacity baselines, ablations, and a privacy clarification before the empirical claim can be accepted. It is not a paradigm shift, but it is a useful extension worth engaging.","headline":"A competent extension of MoE-style personalization to federated sequential recommendation, but the headline gain is confounded by added capacity and needs ablations.","tokens_in":17372,"tokens_out":3229,"would_cite":true,"duration_ms":31238,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A federated recommender improves click-through prediction by adding a group gating network to each Transformer layer, routing users to shared group-level experts while keeping a user-private feed-forward network local.","keywords":["federated learning","recommender systems","user modeling","personalization","Transformer","group gating network","click-through rate prediction","privacy-preserving machine learning"],"falsifier":"An experiment that would settle the privacy claim: train MRFF on KuaiSAR-S, record the server-received group FFN updates, and attempt to infer each user's group assignment from those updates; if the inference accuracy is far above chance, the claim that group assignment stays private is falsified.","tokens_in":16406,"feed_emoji":"🤝","tokens_out":6841,"duration_ms":53818,"temperature":0.7,"pith_summary":"This paper claims that a federated recommender system can get more accurate click-through predictions by modeling users at two levels at once: a private, user-specific feed-forward network that never leaves the device, and a shared group-level feed-forward network chosen by a gating network trained under federation. The proposed framework, MRFF, inserts this split into every Transformer layer, so the layer output is the sum of the user-private and group contributions. On the KuaiRand-Pure, KuaiSAR-R, and KuaiSAR-S datasets, adding MRFF to federated SASRec, HSTU, and LLaMA backbones improves AUC and LogLoss in nearly every comparison, with the largest LogLoss reductions exceeding 30 percent. The paper further argues that training a roughly 60k-parameter model from scratch is more practical for edge devices than fine-tuning large pre-trained foundation models in a federated setting.","feed_headline":"Adding user-group routing lifts federated recommendations","feed_subtitle":"MRFF splits each Transformer layer into private and group models, improving AUC and LogLoss on three Kuai datasets.","key_machinery":"The load-bearing mechanism is the group gating network: a small MLP followed by softmax that consumes the attention output and the user embedding, producing a probability distribution over user groups at each Transformer layer. The highest-probability group selects a shared group-level FFN, and the layer's output adds that group contribution to a user-private FFN (Eq. 5). The balance loss, weighted by a hyper-parameter $\\alpha$, regularizes the gate toward uniform user allocation, and the number of groups is a second hyper-parameter. Together these components let the model express both shared behavioral segments and individual idiosyncrasies in a single forward pass.","core_discovery":"The central claim is that the tension between personalization and privacy in federated recommendation can be resolved by splitting each Transformer feed-forward block into two branches: a private FFN kept on the user's device and a group-level FFN whose parameters are aggregated on the server. A gating network, fed by the attention output and the user embedding, assigns the user to one of several groups per layer, and the layer computes $x_{out} = FFN_u(Attention(x_{in})) + FFN_g(Attention(x_{in}))$. A balance loss pushes the gate toward uniform group occupancy so that no group dominates. The authors report that this design consistently beats federated SASRec, HSTU, and LLaMA baselines on three Kuai datasets, while keeping user embeddings and user-specific FFNs private and adding noise to uploaded parameters for extra privacy.","pith_inferences":["Structurally, MRFF is a federated mixture-of-experts layer: the gating network plus group FFNs mirror sparse MoE routing. Training techniques and failure modes from the MoE literature, such as expert collapse and routing instability, likely transfer and could be tested directly.","The privacy claim is weaker than the framing suggests. The paper adds noise to uploaded parameters and keeps some modules local, but it provides no formal differential-privacy accounting and does not analyze what the group-level aggregated updates reveal about individual group membership; a membership-inference test would clarify the actual protection.","The group gating network depends on the user embedding, which is kept private. In a cross-silo setting—where each client is an institution rather than an individual—the same gating could route institutions to shared expert networks, which would be a natural deployment test of the architecture.","A direct check of the 'multifaceted' interpretation would be to compare learned group assignments against known user segments, such as demographic or intent labels; if the gate does not align with any stable user distinction, the gains may come from the extra capacity rather than from genuine facet discovery."],"forward_implications":["Federated recommenders can obtain personalized parameters without uploading user embeddings or user-specific FFNs, so per-user data stays on-device under the protocol as described.","The mechanism is architecture-agnostic: the same gating-plus-two-FFN block can be dropped into any Transformer-based sequential recommender, not just the three backbones tested.","Training a compact model from scratch avoids the communication and computation overhead of fine-tuning large pre-trained foundation models, which the paper argues is the main obstacle to real-world deployment.","The balance loss gives operators a direct control for group granularity; experiments with two, four, six, and eight groups show that four groups balances performance and model size.","The group-level FFN lets users share statistical strength with similar users, which the paper identifies as the benefit of group-level personalization."],"supporting_citations":[{"why":"Provides the SASRec backbone that MRFF is tested on as FedSASRec.","marker":"(Kang and McAuley 2018)"},{"why":"Provides the HSTU architecture used as the FedHSTU backbone.","marker":"(Zhai et al. 2024a)"},{"why":"Provides the LLaMA model adapted as FedLLaMA.","marker":"(Touvron et al. 2023)"},{"why":"Supplies the federated averaging protocol that coordinates server-client optimization.","marker":"(McMahan et al. 2017)"},{"why":"Inspires the balance-loss regularization for even user routing across groups.","marker":"(Shazeer et al. 2016)"},{"why":"Supplies the KuaiRand-Pure evaluation dataset.","marker":"(Gao et al. 2022)"},{"why":"Supplies the KuaiSAR-R and KuaiSAR-S evaluation datasets.","marker":"(Sun et al. 2023)"},{"why":"Provides the local differential privacy mechanism the paper uses to add noise to uploaded parameters.","marker":"(Choi et al. 2018)"},{"why":"Defines the FedPA federated foundation-model baseline that MRFF compares against.","marker":"(Zhang et al. 2024b)"}],"fun_headline_variants":["Federated recs with gated group routing","Private group FFNs lift federated recommendations","Group-gated Transformer for private federated recs"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the server can aggregate group-level FFN parameters without learning which users belong to which group, yet the paper provides no formal mechanism or proof for that separation.","fun_headline_variants_meta":{"raw":{"variants":["Federated recs with gated group routing","Private group FFNs lift federated recommendations","Group-gated Transformer for private federated recs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000272,"raw_usage":{"total_tokens":1598,"prompt_tokens":876,"completion_tokens":722,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":492,"completion_tokens_details":{"reasoning_tokens":674}},"tokens_in":492,"tokens_out":722,"duration_ms":6283,"temperature":1.0,"reasoning_tokens":674,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T05:54:59.408539+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"An experiment that would settle the privacy claim: train MRFF on KuaiSAR-S, record the server-received group FFN updates, and attempt to infer each user's group assignment from those updates; if the inference accuracy is far above chance, the claim that group assignment stays private is falsified.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Inspires the balance-loss regularization for even user routing across groups."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the KuaiRand-Pure evaluation dataset."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the KuaiSAR-R and KuaiSAR-S evaluation datasets."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the local differential privacy mechanism the paper uses to add noise to uploaded parameters."}],"review_version":1}