{"id":"776a2dcf-da3d-4c78-b45c-f7ad43f68234","arxiv_id":"2506.17342","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"ASMS combines federated averaging, local differential privacy, and multi-agent PPO to select streaming bitrates for multi-user XR, reporting consistent QoE gains over seven baselines in simulated network conditions.","lead":"ASMS is a streaming system for social metaverse apps that lets many VR/AR headsets coordinate video bitrate choices using federated multi-agent reinforcement learning, keeping raw user data on devices. In a testbed with six network scenarios, it reports higher model-based QoE scores than seven baselines.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. (1) as written rewards packet loss and compares bitrate to a frame-rate target, so the reward/evaluation metric contradicts its own description; the 14% claim is not reproducible from the paper.","rationale":"The paper's central claim is that F-MAPPO improves user experience by at least 14% relative to existing streaming methods. That claim stands or falls with the QoE model in Eq. (1), which is simultaneously the reward for training and the metric for evaluation. The reader's weakest assumption was external validity: whether the MOS fit from 8 users makes the QoE score a good proxy for perceived quality. My stress-test goes one step earlier: the equation as printed is internally inconsistent with the text that defines it. The plus sign before δ2·P(pt) turns a packet-loss penalty into a packet-loss bonus, and the choppiness term uses a bitrate where the text specifies a frame rate. If the authors implemented the printed equation, the RL agents are rewarded for exceeding the packet-loss threshold and the evaluation scores are inflated in the disruption scenarios where F-MAPPO is claimed to be most robust. If they implemented a corrected equation, then the manuscript does not state the actual objective, so the reported scores and the 14% figure are not reproducible. Either way, the evidence for the headline claim is missing. The MOS fitting does not resolve this, because fitting coefficients to one version of the equation says nothing about whether the experiments used that version. No code or artifact is provided. I therefore recommend REJECT for the current manuscript, with the path to acceptance being a corrected Eq. (1) and a re-run of the evaluation under the corrected metric.","tokens_in":16564,"tokens_out":9652,"duration_ms":103940,"concrete_test":"Implement Eq. (1) exactly as printed with α=1, β=0.4, γ=0.2, δ1=0.6, δ2=0.5. In scenario S5 (network congestion), vary packet loss p_t from p_threshold upward while holding bitrate, latency, jitter, and user count fixed. If QoE increases when p_t increases, the printed metric rewards packet loss, confirming the inconsistency. Then correct the sign to −δ2·P(pt) and the choppiness term to β|f_t−ftarget|, and rerun the Section IV-E comparison; a material change in the QoE ordering or in the computed improvement vs. baselines would demonstrate that the reported 14% claim depends on the uncorrected formula.","verdict_should_be":"REJECT","load_bearing_attack":"Section III-A defines the QoE function used both as the RL reward and as the evaluation metric. Equation (1) ends with '− δ1|q(yt+1) − q(yt)| + δ2·P(pt)', where P(pt)=max(0, pt−pthreshold). The adjacent text says δ2P(pt) 'penalizes packet losses' and that P(pt) 'addresses sudden network disruptions.' With the printed plus sign, any packet loss above threshold increases QoE, so the objective rewards the disruptions the paper claims to suppress. The same equation writes the choppiness penalty as β|yt − ftarget|, although the description defines it as β|ft − ftarget| with ft the frame rate; yt is the received bitrate, not a frame rate. These are not typographical side-notes: Eq. (1) is the only reward and the only outcome metric. The MOS fitting in Section IV-B cannot validate an ambiguous expression, and no code is provided to show which formula was actually implemented. Consequently, every QoE score in Section IV-E, and the abstract's 'at least 14%' improvement, is computed from a metric that contradicts its own semantics.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes ASMS, an adaptive streaming system for social metaverse applications, built on Federated Multi-Agent Proximal Policy Optimization (F-MAPPO). The authors model the problem as a Dec-POMDP, define a time-step QoE metric that incorporates bitrate, frame rate, latency, jitter, packet loss, and user density, and train multiple headset agents with federated averaging and local differential privacy. They report offline training curves and online comparisons against seven baselines across six network scenarios, claiming that ASMS improves user experience by at least 14% and consistently achieves the highest QoE scores. The paper also includes a small MOS study used to fit the QoE model coefficients and a feasibility analysis of communication and computational overhead.","tokens_in":16855,"tokens_out":5933,"duration_ms":68782,"significance":"The combination of federated learning and multi-agent deep RL for multi-user adaptive bitrate streaming in an XR/metaverse setting is a timely and relevant direction, and the paper provides a concrete MEC-based architecture and a hardware testbed with Unreal Engine and multiple headsets. If the reported results were supported by a validated QoE metric and statistically grounded comparisons, the work would be of interest to the computational social systems and multimedia networking communities. However, the central quantitative claims currently rest on a single fitted QoE model used both as the RL reward and as the outcome metric, and the paper ships no code or data. The paper's most distinctive contribution, privacy-preserving federated multi-agent coordination, is also under-specified: no privacy budget, sensitivity analysis, or details of the federated averaging weights are given. The direction is promising, but the evidence as presented does not yet support the stated user-experience claims.","major_comments":[{"comment":"Equation (1) is internally inconsistent with its own description, and this is load-bearing because the same expression is used as the RL reward and as the evaluation metric throughout Section IV-E. The printed last term is '+ δ2·P(pt)', where P(pt)=max(0, pt−pthreshold), while the adjacent text states that δ2P(pt) 'penalizes packet losses' and 'addresses sudden network disruptions.' With a plus sign, packet loss above the threshold increases QoE, so the objective rewards the very disruptions the paper claims to suppress. In addition, the choppiness penalty is printed as β|yt − ftarget|, but the text defines it as β|ft − ftarget| where ft is the frame rate; since yt is defined in Section III-A as the received bitrate, the printed equation compares a bitrate with a frame-rate target. The MOS fitting in Section IV-B cannot resolve this ambiguity, and no code is provided to show which formula was actually implemented. The authors must correct Eq. (1), clarify the exact implemented reward, and either release the implementation or provide a version of the evaluation computed from the corrected expression.","section":"Section III-A, Eq. (1)"},{"comment":"The validation of the QoE model is not sufficient to support the 'user experience' claims. The coefficients α, β, γ, δ1, δ2 are fit by grid search to 192 MOS ratings from 8 participants across 6 scenarios, with no held-out split, no cross-validation, and no confidence intervals; the reported R²=0.92 is a training goodness-of-fit, not a measure of predictive validity. The sensitivity analysis (±20% coefficient perturbations, 5.2% average RMSE change) only shows local stability of the fit. Moreover, because this model is both the reward function and the evaluation metric, the reported improvements in Section IV-E compare policies in terms of the fitted objective rather than measuring independent user-perceived quality. The abstract's 'improves user experience by at least 14%' therefore overstates what the experiment actually demonstrates; the authors should reframe the claims or add an independent validation of the learned policy against MOS ratings.","section":"Section IV-B"},{"comment":"The online testing results are reported as single point QoE values with no number of runs, no variance or confidence intervals, and no statistical significance test. The text states that F-MAPPO 'consistently achieved the highest QoE scores across all network conditions,' but the data shown in Fig. 5 do not support a claim of consistency in a statistical sense. The abstract's 'at least 14% improvement' is not derived anywhere in Section IV-E: no per-scenario percentage improvements or averaging method are given, and it is unclear against which baseline or over which scenarios this number is computed. The authors should add run-to-run variability, explicit error bars, and a clear derivation of the reported improvement percentage.","section":"Section IV-E, Fig. 5"},{"comment":"The privacy-preservation claim is not quantifiable as presented. Equation (14) applies a Laplacian mechanism for local differential privacy, but the paper gives no privacy budget ε, no sensitivity bound Δθ, and no analysis of the privacy-utility tradeoff. Equation (15) says the federated averaging weights w_i are 'dynamically adjusted,' but the adjustment rule is not specified. Without these details, the reader cannot assess whether the privacy protection is meaningful or how the added noise affects the QoE results reported in Section IV-E. This is a central contribution of the paper and should be specified and, ideally, empirically characterized.","section":"Section III-C"}],"minor_comments":[{"comment":"The state vector includes the received bitrate yt but the choppiness penalty in the text refers to frame rate ft; if ft is a derived quantity from yt or another source, its definition should be given explicitly.","section":"Section III-A, state definition"},{"comment":"The description of Ruyi as a user-preference-aware ABR method does not match the cited paper, which is about a secure multi-party learning framework; please verify and correct the citation.","section":"Section IV-C, reference [36]"},{"comment":"The communication overhead comparison is not apples-to-apples: F-MAPPO transmits roughly 0.5 MB per device per aggregation round, while centralized MAPPO is said to transmit about 1 KB per episode, but the number of episodes per aggregation round is not stated, so the total cost comparison is ambiguous.","section":"Section IV-F"},{"comment":"There are minor typographical and formatting issues, including 'Adjucnt Professor' in the author biography, inconsistent spacing in the index terms, and a few malformed reference DOIs; these should be cleaned up in the final version.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. The authors build a real testbed for multi-user adaptive streaming in XR using federated multi-agent PPO, which is a reasonable extension of Pensieve-style DRL. But the paper's central claim—'improves user experience by at least 14%'—rests on a QoE model that is both the RL reward and the evaluation metric, and that model is mis-specified as printed.\n\nWhat is genuinely new and useful: the Dec-POMDP formulation for social metaverse streaming, the time-step QoE model that includes MTP latency, user density, and disruption penalties, and the use of FL plus LDP to keep raw user data off the server. The feasibility analysis with GPU utilization and update sizes is practical. They also ran a real user study with headsets and a network emulator, which is more than many RL papers do.\n\nThe biggest problem is Eq. (1). It ends with '+ δ2·P(pt)', but the text says δ2P(pt) 'penalizes packet losses.' A plus sign rewards packet loss above threshold. Also, the choppiness penalty is printed as β|yt − ftarget|, but the description defines it as β|ft − ftarget|, where f_t is frame rate and y_t is received bitrate. These are not minor typos: this equation is the only reward and the only outcome metric. The MOS fitting in Section IV-B (8 participants, 192 ratings, grid search, R²=0.92, no held-out validation) can't disambiguate the formula, and no code is included. So the QoE scores in Section IV-E and the abstract's 14% claim are computed from a metric that contradicts its own semantics. The circularity goes deeper: the model is fitted to MOS and then used to compare agents, so the comparison is against the fitted objective, not against independent user experience. There are also no error bars or repeated runs, and the Ruyi baseline description does not match the cited paper (which is about secure multi-party learning, not ABR).\n\nThe flaws are addressable. Fix the equation, release code, add held-out MOS validation, and correct the baseline citation. The system itself is credible and the direction is worth pursuing.\n\nI would not desk-reject this. It deserves serious refereeing because the contribution is substantial and the problems are fixable. But a referee should reject the current version and demand a corrected, reproducible evaluation before the 14% claim can be taken seriously.","headline":"A plausible systems paper whose central QoE metric—used as both reward and evaluation—contradicts its own text, so the headline 14% claim is not currently reproducible.","tokens_in":17388,"tokens_out":3958,"would_cite":false,"duration_ms":41098,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A federated multi-agent reinforcement learning system, F-MAPPO, coordinates bitrate choices across XR headsets, and the paper reports it improves user experience by at least 14% over seven streaming baselines.","keywords":["adaptive bitrate streaming","social metaverse","federated learning","multi-agent reinforcement learning","proximal policy optimization","quality of experience","extended reality","local differential privacy"],"falsifier":"Run a blind user study with a larger and more diverse participant pool comparing F-MAPPO against GCC or BBR in the congestion scenario S5: if the mean user-rating difference is below the roughly 14% improvement the paper reports in QoE-score terms, or if users show no preference for F-MAPPO's streams, the claim that optimizing Equation (1) improves user experience is falsified.","tokens_in":16373,"feed_emoji":"🥽","tokens_out":14222,"duration_ms":133576,"temperature":0.7,"pith_summary":"This paper proposes ASMS, a system for streaming rendered metaverse scenes to XR headsets over wireless networks, and claims it improves user experience by at least 14% relative to existing streaming methods. The central idea is F-MAPPO, a training scheme that lets each headset run its own reinforcement-learning agent while sharing model updates with an edge server through federated averaging, so raw user data never leaves the device. Agents choose small bitrate adjustments each second using a reward built from a time-step quality-of-experience model that penalizes latency, choppiness, quality fluctuation, and packet-loss disruptions. The authors report that F-MAPPO reaches the highest QoE score in all six tested network scenarios, including congestion and recovery, while adding only modest communication overhead. If the QoE model is a faithful proxy for perceived quality, this shows that privacy-preserving multi-agent learning is a viable adaptive-bitrate strategy for interactive XR.","feed_headline":"Federated multi-agent learning lifts metaverse streaming quality by 14%","feed_subtitle":"Headsets share model updates, not raw data, and beat seven streaming baselines on the paper's QoE metric.","key_machinery":"The machine doing the work is F-MAPPO, a federated multi-agent proximal policy optimization scheme. A global agent on the edge server periodically broadcasts actor-critic parameters, each headset runs a local PPO agent that collects trajectories and updates its policy, and the local updates are aggregated back at the server with FedAvg after Laplacian noise is added, so only perturbed parameters, never raw observations, are shared. The argument is carried by the reward signal: a time-step QoE model, Equation (1), whose five terms discount scene quality by user density, penalize choppiness against the target frame rate, penalize motion-to-photon latency scaled by throughput, penalize quality swings between steps, and penalize packet loss above a threshold. Its coefficients were fit by grid search to 192 Mean Opinion Score ratings from 8 users, and the same equation is used both as the RL reward and as the evaluation metric, so the reported quality improvements flow through it.","core_discovery":"The paper's central claim is that social metaverse streaming should be treated as a multi-user, privacy-constrained decision problem rather than a single-user bitrate adaptation task, and that F-MAPPO solves it better than the alternatives. F-MAPPO models each headset as an agent in a decentralized partially observable Markov decision process whose state is six network features (last chosen bitrate, received bitrate, motion-to-photon latency, jitter, packet loss, and negative acknowledgments) and whose action is an increment, decrement, or hold on the bitrate. The reward is the average over agents of a time-step QoE score, Equation (1), that combines scene quality with a user-density discount, a choppiness penalty, a latency penalty scaled by throughput, a stability penalty, and a packet-loss disruption penalty. Training alternates local PPO updates with federated averaging of perturbed model updates, adding local differential privacy. In online tests across six scenarios, F-MAPPO reports the highest QoE in every scenario, with the largest margins under congestion and recovery.","pith_inferences":["An implication the authors leave implicit is that the same F-MAPPO machinery would transfer to cloud gaming, telepresence, or remote guidance, wherever multiple clients share a bottleneck and quality is latency-dominated; that transfer is plausible but untested here.","Because the QoE coefficients came from eight participants, a natural next step is to fit Equation (1) per user or per population segment and repeat the ±20% sensitivity analysis; larger RMSE changes across groups would indicate that personalized coefficients are needed.","The privacy guarantee rests on Laplacian noise and FedAvg, but the paper does not quantify how the 14% margin degrades as the privacy budget is tightened over many aggregation rounds, which would be a direct test of the privacy-performance trade-off."],"forward_implications":["If F-MAPPO works as claimed, multi-user XR streaming can coordinate bitrate choices at the edge without centralizing personal data, removing the main privacy objection to learning-based adaptive bitrate control in the metaverse.","The reported QoE gains are largest in the difficult cases of congestion and recovery, so the method's value lies in robustness to network shocks rather than only in steady-state quality.","The measured overhead (about 0.5 MB per device per aggregation round and 74.5% GPU utilization for training, versus 68.2% for centralized MAPPO) suggests the federated loop is light enough to run on existing edge infrastructure.","The framework is built so that additional headsets join as new local agents sharing the same global model, which is the stated path toward large-scale deployment."],"supporting_citations":[{"why":"Supplies the PPO clipped-surrogate objective that each local F-MAPPO agent uses for policy updates.","marker":"[14]"},{"why":"Supplies the FedAvg rule used to aggregate perturbed local model updates into the global model.","marker":"[21]"},{"why":"Supplies the generalized advantage estimation used to compute advantages and returns in local training.","marker":"[33]"},{"why":"Soft Actor-Critic serves as a decentralized RL baseline whose QoE F-MAPPO is compared against.","marker":"[34]"},{"why":"GreenABR is a DRL-based adaptive bitrate streaming baseline included in the comparison.","marker":"[35]"},{"why":"Ruyi is a user-preference-aware streaming baseline included in the comparison.","marker":"[36]"},{"why":"ARTEMIS is a bitrate-ladder optimization baseline included in the comparison.","marker":"[37]"},{"why":"GCC is the WebRTC congestion-control baseline whose QoE scores drop in the tested scenarios.","marker":"[38]"},{"why":"BBR is the congestion-control baseline that F-MAPPO beats by the largest reported margins.","marker":"[39]"}],"fun_headline_variants":["Privacy-preserving federated RL boosts metaverse streaming by 14%","Federated multi-agent RL improves adaptive metaverse bitrate choice","F-MAPPO: federated RL for private, higher-quality metaverse streaming","Multi-agent federated RL lifts metaverse streaming QoE 14%","Adaptive streaming with federated multi-agent RL: +14% QoE"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the time-step QoE model in Equation (1), whose coefficients were fitted by grid search to 192 Mean Opinion Score ratings from 8 users, captures how users actually perceive streaming quality in the social metaverse.","fun_headline_variants_meta":{"raw":{"variants":["Privacy-preserving federated RL boosts metaverse streaming by 14%","Federated multi-agent RL improves adaptive metaverse bitrate choice","F-MAPPO: federated RL for private, higher-quality metaverse streaming","Multi-agent federated RL lifts metaverse streaming QoE 14%","Adaptive streaming with federated multi-agent RL: +14% QoE"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000294,"raw_usage":{"total_tokens":1719,"prompt_tokens":961,"completion_tokens":758,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":577,"completion_tokens_details":{"reasoning_tokens":658}},"tokens_in":577,"tokens_out":758,"duration_ms":7847,"temperature":1.0,"reasoning_tokens":658,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T23:43:48.184862+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a blind user study with a larger and more diverse participant pool comparing F-MAPPO against GCC or BBR in the congestion scenario S5: if the mean user-rating difference is below the roughly 14% improvement the paper reports in QoE-score terms, or if users show no preference for F-MAPPO's streams, the claim that optimizing Equation (1) improves user experience is falsified.","supporting_citations":[{"cited_title":"Communication-efficient learning of deep networks from decentralized data,","cited_arxiv_id":null,"evidence_quote":"Supplies the FedAvg rule used to aggregate perturbed local model updates into the global model."},{"cited_title":"GreenABR: Energy-aware adaptive bitrate streaming with deep reinforcement learning,","cited_arxiv_id":null,"evidence_quote":"GreenABR is a DRL-based adaptive bitrate streaming baseline included in the comparison."},{"cited_title":"ARTEMIS: Adaptive bitrate ladder optimization for live video streaming,","cited_arxiv_id":null,"evidence_quote":"ARTEMIS is a bitrate-ladder optimization baseline included in the comparison."}],"review_version":1}