Pith. sign in

REVIEW 2 major objections 1 minor 62 references

A dynamic weighted aggregation mechanism elicits truthful feedback from strategic workers and achieves sublinear regret of order sqrt(T).

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.3

2026-06-30 16:05 UTC pith:RTMZPS2A

load-bearing objection The paper puts forward a dynamic Bayesian game and weighted aggregation rule to enforce truthful reports from strategic workers while getting O(sqrt(T)) regret for online LLM preference aggregation, but the accuracy proxy without ground truth is the unshown step that carries both the truthfulness and regret claims. the 2 major comments →

arxiv 2605.24052 v1 pith:RTMZPS2A submitted 2026-05-22 cs.LG cs.AI

Truthful Online Preference Aggregation for LLM Fine-Tuning in Mobile Crowdsourcing

Shugang Hao , Lingjie Duan This is my paper
classification cs.LG cs.AI
keywords truthful mechanismonline aggregationLLM fine-tuningmobile crowdsourcingsublinear regretBayesian gamepreference feedback
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper models interactions between a crowdsourcing platform and strategic mobile workers as a dynamic Bayesian game for collecting preferences to fine-tune LLMs. It proposes an online mechanism that updates each worker's weight based on the accuracy of their reported preferences over successive time slots. This design makes misreporting unprofitable for workers and limits the platform's total regret to grow as the square root of the number of rounds. The approach extends to cases where only a subset of workers can provide feedback each round while preserving the sublinear regret bound. Real-world experiments confirm improved alignment of LLM outputs compared to standard methods.

Core claim

The proposed online weighted aggregation mechanism, formulated within a dynamic Bayesian game, dynamically adjusts worker weights according to feedback accuracy, thereby ensuring truthful reporting by strategic workers and delivering a sublinear regret bound of O(sqrt(T)) over T time slots, with an extension to limited per-slot feedback maintaining the same bound.

What carries the argument

Online weighted aggregation mechanism that dynamically adjusts each worker's weight according to their feedback accuracy.

Load-bearing premise

The dynamic Bayesian game model accurately captures strategic worker behavior and that feedback accuracy can be assessed online without access to ground-truth preferences each round.

What would settle it

A simulation or deployment where workers can profit by misreporting preferences and the observed regret grows linearly with T instead of as sqrt(T).

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • The mechanism guarantees that strategic workers report their true preferences.
  • The cumulative regret remains sublinear at O(sqrt(T)) rather than linear in T.
  • The same sublinear regret holds when feedback is limited per time slot.
  • Experiments on real datasets show significant performance gains over benchmark schemes.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • This approach may extend to other online aggregation tasks involving strategic agents beyond mobile crowdsourcing.
  • Platforms could apply similar weighting to reduce reliance on ground-truth labels in iterative LLM training.
  • Testing the mechanism in non-mobile settings like web-based crowdsourcing would check its broader applicability.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 1 minor

Summary. The paper claims to formulate a dynamic Bayesian game to model strategic mobile workers providing online preference feedback for LLM fine-tuning in mobile crowdsourcing. It proposes a novel online weighted aggregation mechanism that dynamically adjusts each worker's weight according to their feedback accuracy, proving that the mechanism ensures truthful feedback from strategic workers and achieves sublinear regret O(sqrt(T)) over T time slots. An extension to the challenging case of limited worker feedback per time slot is also claimed to retain the O(sqrt(T)) regret guarantee. Experiments on real-world datasets are said to demonstrate significant performance gains over benchmarks.

Significance. If the central claims hold, the work would advance mechanism design for online crowdsourced preference aggregation with strategic agents by delivering both dominant-strategy truthfulness and sublinear regret in a dynamic Bayesian setting. The extension to limited feedback and the experimental results on LLM fine-tuning would add practical value for mobile applications where ground-truth preferences are unavailable each round.

major comments (2)
  1. [Abstract and mechanism definition] Abstract and mechanism section: The accuracy assessment without ground truth is load-bearing for both the truthfulness proof and the O(sqrt(T)) regret bound. No explicit observable signal or update rule is described that computes feedback accuracy online when true preferences are unavailable each round; any proxy (e.g., consistency across workers or Bayesian posterior) must be shown to remain incentive-compatible and uncorrelated with strategic misreporting, otherwise both the dominant-strategy truthfulness argument and the standard online-learning regret analysis are invalidated.
  2. [Proof sections] Proof of truthfulness and regret analysis (likely the analysis section following the mechanism): The abstract asserts proofs of truthfulness and O(sqrt(T)) regret but supplies no derivation steps, assumptions, or proof sketches. The full manuscript must exhibit the strategy space, payment/weight rules, and how the dynamic Bayesian game yields dominant-strategy truthfulness together with the regret bound, without circular dependence on the accuracy proxy.
minor comments (1)
  1. [Notation and preliminaries] Notation: Consistently use mathcal{O} for asymptotic bounds throughout the text and ensure all variables in the dynamic Bayesian game are defined before first use.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the thorough review and valuable feedback on our manuscript. The comments highlight important aspects of the mechanism description and proofs that require clarification. We address each point below and will incorporate revisions to strengthen the presentation.

read point-by-point responses

  1. Referee: [Abstract and mechanism definition] Abstract and mechanism section: The accuracy assessment without ground truth is load-bearing for both the truthfulness proof and the O(sqrt(T)) regret bound. No explicit observable signal or update rule is described that computes feedback accuracy online when true preferences are unavailable each round; any proxy (e.g., consistency across workers or Bayesian posterior) must be shown to remain incentive-compatible and uncorrelated with strategic misreporting, otherwise both the dominant-strategy truthfulness argument and the standard online-learning regret analysis are invalidated.

    Authors: We agree that the accuracy assessment mechanism requires a more explicit description to support the claims. In the revised manuscript, we will add a dedicated subsection in the mechanism definition that specifies the observable signal as the consistency between individual feedback and the Bayesian-updated aggregate preference (computed from historical reports). The update rule uses a posterior over worker reliability parameters, and we will include a proof that this proxy is incentive-compatible under the dynamic Bayesian game (i.e., misreporting does not improve a worker's long-term weight or payment). This addition will also confirm that the proxy is uncorrelated with strategic deviations, preserving both truthfulness and the regret bound. revision: yes

  2. Referee: [Proof sections] Proof of truthfulness and regret analysis (likely the analysis section following the mechanism): The abstract asserts proofs of truthfulness and O(sqrt(T)) regret but supplies no derivation steps, assumptions, or proof sketches. The full manuscript must exhibit the strategy space, payment/weight rules, and how the dynamic Bayesian game yields dominant-strategy truthfulness together with the regret bound, without circular dependence on the accuracy proxy.

    Authors: The full proofs, including the strategy space definition, payment/weight update rules, and the derivation of dominant-strategy truthfulness in the dynamic Bayesian game, are contained in the appendix. However, we acknowledge that the main text lacks sufficient sketches and explicit assumptions. In the revision, we will insert a proof outline in Section 4 (following the mechanism) that details the key steps: (i) characterization of the strategy space, (ii) showing that truthful reporting is dominant by demonstrating that any deviation reduces expected weight without increasing payment, and (iii) the regret analysis via online learning techniques that avoids circularity by bounding the accuracy proxy's deviation independently. Assumptions (e.g., bounded rationality and i.i.d. preference noise) will be stated upfront. revision: yes

Circularity Check

0 steps flagged

No significant circularity; derivation self-contained against external benchmarks

full rationale

The abstract formulates a dynamic Bayesian game and an online weighted aggregation mechanism that adjusts weights by feedback accuracy, then claims to prove dominant-strategy truthfulness plus O(sqrt(T)) regret (and the limited-feedback extension). No equation, parameter fit, or self-citation is shown reducing either the truthfulness argument or the regret bound to the target result by construction. The accuracy-assessment step is presented as an input modeling choice whose incentive compatibility is asserted to hold inside the game, not derived from the regret expression itself. Standard online-learning analysis supplies the O(sqrt(T)) bound once the mechanism is incentive-compatible; that analysis is independent of the specific Bayesian-game parameters. No enumerated circularity pattern is exhibited.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 1 invented entities

Abstract-only review; ledger reflects high-level modeling choices stated in the abstract with no access to full derivations or parameter lists.

axioms (1)
  • domain assumption The multi-agent interaction is accurately modeled as a dynamic Bayesian game between platform and strategic workers.
    Invoked to formulate the online learning process.
invented entities (1)
  • Online weighted aggregation mechanism no independent evidence
    purpose: Dynamically adjusts worker weights according to feedback accuracy to enforce truthfulness and sublinear regret.
    New mechanism introduced to solve the identified linear-regret problem.

pith-pipeline@v0.9.1-grok · 5771 in / 1250 out tokens · 49183 ms · 2026-06-30T16:05:16.087633+00:00 · methodology

0 comments
read the original abstract

To better serve users' demands in mobile applications (e.g., navigation), mobile crowdsourcing platforms can iteratively align large language model (LLM)-generated content (e.g., AI-generated traffic condition predictions) with human feedback collected from crowdsourcing workers (e.g., mobile users). However, workers may strategically misreport their online preference feedback to maximize their influence or payment. Existing pipelines in mobile crowdsourcing (e.g., EM-based weight estimation) fail to identify the most accurate worker in this online setting, resulting in a linear regret $\mathcal{O}(T)$ over $T$ time slots. In this paper, we study truthful online preference aggregation for LLM fine-tuning in mobile crowdsourcing. We formulate a new dynamic Bayesian game to model the multi-agent online learning process between the platform and strategic mobile workers. We propose a novel online weighted aggregation mechanism that dynamically adjusts each worker's weight in the preference aggregation according to their feedback accuracy. We prove that our mechanism ensures truthful feedback from strategic workers and achieves a sublinear regret $\mathcal{O}(\sqrt{T})$ over $T$ time slots. We further extend our mechanism to a challenging scenario with limited worker feedback per time slot, still guaranteeing a sublinear regret $\mathcal{O}(\sqrt{T})$. Experiments on LLM fine-tuning with real-world datasets further demonstrate significant performance gains of our mechanisms over benchmark schemes.

Figures

Figures reproduced from arXiv: 2605.24052 by Lingjie Duan, Shugang Hao.

Figure 1
Figure 1. Figure 1: System model of LLM fine-tuning for mobile crowdsourcing. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Each worker i’s weight wt i of our truthful online weighted aggregation versus time slot t under full feedback. Here we fix worker number N = 5, total time slot number T = 500, and prompt number mt = 20. keeps decreasing from 1 and becomes smaller than the others. Therefore, the probability that he will be chosen in future time slots is lower than that of the unchosen workers, especially when his feedback … view at source ↗
Figure 3
Figure 3. Figure 3: Each worker i’s chosen probability wt ∑ i N i ′=1 wt i ′ of our truthful online weighted aggregation versus time slot t under full feedback. Here we fix worker number N = 5, total time slot T = 500, and prompt number mt = 20. most accurate worker with more time slots and samples. Thus, it chooses a smaller exploitation parameter β and a smaller step-size α to punish inaccurate (chosen) workers less in the … view at source ↗
Figure 5
Figure 5. Figure 5: Each worker i’s chosen probability wt ∑ i N i ′=1 wt i ′ of our truthful online mixed selection versus time slot t under limited worker feedback. Here we fix worker number N = 5, total time slot number T = 2500, and prompt number mt = 20. largest probability to the most accurate worker 1 (over 0.9) and assign near-zero probabilities to the remaining ones especially for the most inaccurate worker 5, which f… view at source ↗
Figure 6
Figure 6. Figure 6: Time-average regrets of our mechanism under full worker [PITH_FULL_IMAGE:figures/full_fig_p012_6.png] view at source ↗
Figure 8
Figure 8. Figure 8: Time-average regrets of our mechanism under limited worker [PITH_FULL_IMAGE:figures/full_fig_p012_8.png] view at source ↗
Figure 10
Figure 10. Figure 10: A prompt example for LLM fine-tuning training and testing [PITH_FULL_IMAGE:figures/full_fig_p013_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Win-rates after fine-tuning the GPT-2 (124M) with bench [PITH_FULL_IMAGE:figures/full_fig_p014_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Win-rates after fine-tuning the GPT-2 with benchmarks [PITH_FULL_IMAGE:figures/full_fig_p014_12.png] view at source ↗
Figure 15
Figure 15. Figure 15: Win-rates after fine-tuning the GPT-2 with our mechanism [PITH_FULL_IMAGE:figures/full_fig_p015_15.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

62 extracted references · 62 canonical work pages · 4 internal anchors

  1. [1]

    Online learning from strategic human feedback in llm fine-tuning,

    S. Hao and L. Duan, “Online learning from strategic human feedback in llm fine-tuning,” in ICASSP 2025-2025 IEEE Inter- national Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2025, pp. 1–5

    work page 2025

  2. [2]

    https://doi.org/10.48550/arXiv

    H. Touvron, T. Lavril, G. Izacard, X. Martinet, M. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar et al., “Open and efficient foundation language models,” Preprint at arXiv. https://doi. org/10.48550/arXiv, vol. 2302, 2023

    work page internal anchor Pith review doi:10.48550/arxiv 2023

  3. [3]

    Cached model-as-a-resource: Provisioning large language model agents for edge intelligence in space-air-ground integrated networks,

    M. Xu, D. Niyato, H. Zhang, J. Kang, Z. Xiong, S. Mao, and Z. Han, “Cached model-as-a-resource: Provisioning large language model agents for edge intelligence in space-air-ground integrated networks,” IEEE Transactions on Networking, 2025

    work page 2025

  4. [4]

    “Waze,” https://www.waze.com, accessed 2025

    work page 2025

  5. [5]

    Crowdsensing-based urban traffic monitoring using mobile social media,

    Z. He, Y. Liu, and F. R. Yu, “Crowdsensing-based urban traffic monitoring using mobile social media,” IEEE Communications Magazine, 2016. 124M 355M 774M 1.5B GPT-2 Model Size 40 60 80Win-rate at T=20 (%) Fig. 16. Win-rates after fine-tuning the GPT-2 with our mechanism in Definition 5 and benchmark 3 (median) at time round T = 20 versus GPT-2 model size un...

    work page 2016

  6. [6]

    Training language models to follow instructions with human feedback,

    L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray et al., “Training language models to follow instructions with human feedback,” Advances in neural information processing systems, vol. 35, pp. 27 730–27 744, 2022

    work page 2022

  7. [7]

    Gemini apps privacy hub,

    “Gemini apps privacy hub,” https://support.google.com/ gemini/answer/13594961, accessed 2025

    work page arXiv 2025

  8. [8]

    Mechanism design for llm fine-tuning with multiple reward models,

    H. Sun, Y. Chen, S. Wang, W. Chen, and X. Deng, “Mechanism design for llm fine-tuning with multiple reward models,” arXiv preprint arXiv:2405.16276, 2024

    work page arXiv 2024

  9. [9]

    Truthful aggregation of llms with an application to online advertising,

    E. Soumalias, M. J. Curry, and S. Seuken, “Truthful aggregation of llms with an application to online advertising,” arXiv preprint arXiv:2405.05905, 2024

    work page arXiv 2024

  10. [10]

    Rlhf from heterogeneous feedback via personalization and preference aggregation,

    C. Park, M. Liu, D. Kong, K. Zhang, and A. E. Ozdaglar, “Rlhf from heterogeneous feedback via personalization and preference aggregation,” in ICML 2024 Workshop on Theoretical Founda- tions of Foundation Models, 2024

    work page 2024

  11. [11]

    arXiv preprint arXiv:2404.10271 (2024)

    V. Conitzer, R. Freedman, J. Heitzig, W. H. Holliday, B. M. Ja- cobs, N. Lambert, M. Mossé, E. Pacuit, S. Russell, H. Schoelkopf et al., “Social choice for ai alignment: Dealing with diverse human feedback,” arXiv preprint arXiv:2404.10271, 2024

    work page arXiv 2024

  12. [12]

    Online prediction with selfish experts,

    T. Roughgarden and O. Schrijvers, “Online prediction with selfish experts,” Advances in Neural Information Processing Systems, vol. 30, 2017

    work page 2017

  13. [13]

    Lie for a dime: When most prescreening responses are honest but most study participants are impostors,

    J. J. Chandler and G. Paolacci, “Lie for a dime: When most prescreening responses are honest but most study participants are impostors,” Social Psychological and Personality Science, vol. 8, no. 5, pp. 500–508, 2017

    work page 2017

  14. [14]

    The shape of and solutions to the mturk quality crisis,

    R. Kennedy, S. Clifford, T. Burleigh, P. D. Waggoner, R. Jewell, and N. J. Winter, “The shape of and solutions to the mturk quality crisis,” Political Science Research and Methods, vol. 8, no. 4, pp. 614–629, 2020

    work page 2020

  15. [15]

    Online detection and snr estimation in cooperative spectrum sensing,

    J. Perez, J. Via, L. Vielva, and D. Ramirez, “Online detection and snr estimation in cooperative spectrum sensing,” IEEE Transactions on Wireless Communications, vol. 21, no. 4, pp. 2521–2533, 2021

    work page 2021

  16. [16]

    Adaptive em-based al- gorithm for cooperative spectrum sensing in mobile environ- ments,

    J. Perez, I. Santamaria, and J. Via, “Adaptive em-based al- gorithm for cooperative spectrum sensing in mobile environ- ments,” in 2018 IEEE Statistical Signal Processing Workshop (SSP). IEEE, 2018, pp. 732–736

    work page 2018

  17. [17]

    On the universal near optimality of hedge in combinatorial settings,

    Z. Fan, A. Maiti, L. J. Ratliff, K. Jamieson, and G. Farina, “On the universal near optimality of hedge in combinatorial settings,” in The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025

    work page 2025

  18. [18]

    Efficient online learning with memory via frank-wolfe optimization: Algorithms with bounded dynamic regret and applications to control,

    H. Zhou, Z. Xu, and V. Tzoumas, “Efficient online learning with memory via frank-wolfe optimization: Algorithms with bounded dynamic regret and applications to control,” in 2023 62nd IEEE Conference on Decision and Control (CDC). IEEE, 2023, pp. 8266–8273

    work page 2023

  19. [19]

    Auctions with llm summaries,

    K. A. Dubey, Z. Feng, R. Kidambi, A. Mehta, and D. Wang, “Auctions with llm summaries,” arXiv preprint arXiv:2404.08126, 2024

    work page arXiv 2024

  20. [20]

    Epvisa: Efficient auction design for real-time physical-virtual synchronization in the human-centric 16 metaverse,

    M. Xu, D. Niyato, B. Wright, H. Zhang, J. Kang, Z. Xiong, S. Mao, and Z. Han, “Epvisa: Efficient auction design for real-time physical-virtual synchronization in the human-centric 16 metaverse,” IEEE Journal on Selected Areas in Communica- tions, vol. 42, no. 3, pp. 694–709, 2023

    work page 2023

  21. [21]

    Rlhf workflow: From reward modeling to online rlhf,

    H. Dong, W. Xiong, B. Pang, H. Wang, H. Zhao, Y. Zhou, N. Jiang, D. Sahoo, C. Xiong, and T. Zhang, “Rlhf workflow: From reward modeling to online rlhf,” Transactions on Machine Learning Research

    work page

  22. [22]

    Online iterative reinforcement learning from human feedback with general preference model,

    C. Ye, W. Xiong, Y. Zhang, H. Dong, N. Jiang, and T. Zhang, “Online iterative reinforcement learning from human feedback with general preference model,” Advances in Neural Information Processing Systems, vol. 37, pp. 81 773–81 807, 2024

    work page 2024

  23. [23]

    Iterative preference learning from human feed- back: Bridging theory and practice for rlhf under kl-constraint,

    W. Xiong, H. Dong, C. Ye, Z. Wang, H. Zhong, H. Ji, N. Jiang, and T. Zhang, “Iterative preference learning from human feed- back: Bridging theory and practice for rlhf under kl-constraint,” in Forty-first International Conference on Machine Learning, 2024

    work page 2024

  24. [24]

    Col- laborative algorithms for online personalized mean estimation,

    M. Asadi, A. Bellet, O.-A. Maillard, and M. Tommasi, “Col- laborative algorithms for online personalized mean estimation,” Transactions on Machine Learning Research Journal, 2022

    work page 2022

  25. [25]

    Mechanism design for collaborative normal mean estimation,

    Y. Chen, J. Zhu, and K. Kandasamy, “Mechanism design for collaborative normal mean estimation,” Advances in Neural Information Processing Systems, vol. 36, 2024

    work page 2024

  26. [26]

    Strategyproof mechanisms for group- fair obnoxious facility location problems,

    J. Li, M. Li, and H. Chan, “Strategyproof mechanisms for group- fair obnoxious facility location problems,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, no. 9, 2024, pp. 9832–9839

    work page 2024

  27. [27]

    Positive intra-group externalities in facility location,

    Y. Wang, H. Zhou, and M. Li, “Positive intra-group externalities in facility location,” in Proceedings of the 23rd International Conference on Autonomous Agents and Multiagent Systems, 2024, pp. 1883–1891

    work page 2024

  28. [28]

    Reinforcement Learning from Human Feedback

    N. Lambert, “Reinforcement learning from human feedback,” arXiv preprint arXiv:2504.12501, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  29. [29]

    On strategy-proofness and single peakedness,

    H. Moulin, “On strategy-proofness and single peakedness,” Public Choice, vol. 35, no. 4, pp. 437–455, 1980

    work page 1980

  30. [30]

    A Bayesian truth serum for subjective data,

    D. Prelec, “A Bayesian truth serum for subjective data,” Science, vol. 306, no. 5695, pp. 462–466, 2004

    work page 2004

  31. [31]

    Eliciting informative feedback: The peer-prediction method,

    N. Miller, P. Resnick, and R. Zeckhauser, “Eliciting informative feedback: The peer-prediction method,” Management Science, vol. 51, no. 9, pp. 1359–1373, 2005

    work page 2005

  32. [32]

    Machine-learning aided peer prediction,

    Y. Liu and Y. Chen, “Machine-learning aided peer prediction,” in Proceedings of the 2017 ACM Conference on Economics and Computation, 2017, pp. 63–80

    work page 2017

  33. [33]

    Incentive-compatible forecasting competitions,

    J. Witkowski, R. Freeman, J. W. Vaughan, D. M. Pennock, and A. Krause, “Incentive-compatible forecasting competitions,” Management Science, vol. 69, no. 3, pp. 1354–1374, 2023

    work page 2023

  34. [34]

    Maximum likelihood estimation of observer error-rates using the EM algorithm,

    A. P. Dawid and A. M. Skene, “Maximum likelihood estimation of observer error-rates using the EM algorithm,” Journal of the Royal Statistical Society: Series C (Applied Statistics), vol. 28, no. 1, pp. 20–28, 1979

    work page 1979

  35. [35]

    Slowly changing adversarial bandit algorithms are efficient for discounted mdps,

    I. A. Kash, L. Reyzin, and Z. Yu, “Slowly changing adversarial bandit algorithms are efficient for discounted mdps,” in Inter- national Conference on Algorithmic Learning Theory. PMLR, 2024, pp. 683–718

    work page 2024

  36. [36]

    Meta-learning adversarial bandit algo- rithms,

    M. Khodak, I. Osadchiy, K. Harris, M.-F. F. Balcan, K. Y. Levy, R. Meir, and S. Z. Wu, “Meta-learning adversarial bandit algo- rithms,” Advances in Neural Information Processing Systems, vol. 36, pp. 35 441–35 471, 2023

    work page 2023

  37. [37]

    Double or nothing: Multiplicative incentive mechanisms for crowdsourcing,

    N. B. Shah and D. Zhou, “Double or nothing: Multiplicative incentive mechanisms for crowdsourcing,” Journal of Machine Learning Research, vol. 17, no. 165, pp. 1–52, 2016

    work page 2016

  38. [38]

    No-regret and incentive-compatible online learning,

    R. Freeman, D. Pennock, C. Podimata, and J. W. Vaughan, “No-regret and incentive-compatible online learning,” in Inter- national Conference on Machine Learning. PMLR, 2020, pp. 3270–3279

    work page 2020

  39. [39]

    Incentivized truthful communication for federated bandits,

    Z. Wei, C. Li, T. Ren, H. Xu, and H. Wang, “Incentivized truthful communication for federated bandits,” in Proceedings of the 12th International Conference on Learning Representa- tions (ICLR 2024)

    work page 2024

  40. [40]

    Large language models and their applications in roadway safety and mobility enhancement: A comprehensive review,

    M. M. Karim, Y. Shi, S. Zhang, B. Wang, M. Nasri, and Y. Wang, “Large language models and their applications in roadway safety and mobility enhancement: A comprehensive review,” arXiv preprint arXiv:2506.06301, 2025

    work page arXiv 2025

  41. [41]

    Towards explainable road nav- igation systems,

    K. Alsheeb and M. Brandão, “Towards explainable road nav- igation systems,” in 2023 IEEE 26th International Conference on Intelligent Transportation Systems (ITSC). IEEE, 2023, pp. 16–22

    work page 2023

  42. [42]

    Machine learning- based spectrum occupancy prediction: A comprehensive sur- vey,

    M. A. Aygül, H. A. Çırpan, and H. Arslan, “Machine learning- based spectrum occupancy prediction: A comprehensive sur- vey,” Frontiers in Communications and Networks, vol. 6, p. 1482698, 2025

    work page 2025

  43. [43]

    Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback

    Y. Bai, A. Jones, K. Ndousse, A. Askell, A. Chen, N. DasSarma, D. Drain, S. Fort, D. Ganguli, T. Henighan et al., “Training a helpful and harmless assistant with reinforcement learning from human feedback. corr, abs/2204.05862, 2022a. doi: 10.48550,” arXiv preprint arXiv.2204.05862, 2022

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  44. [44]

    Llama 2: Open Foundation and Fine-Tuned Chat Models

    H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y. Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale et al., “Llama 2: Open foundation and fine-tuned chat models,” arXiv preprint arXiv:2307.09288, 2023

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  45. [45]

    Direct preference optimization: Your language model is secretly a reward model,

    R. Rafailov, A. Sharma, E. Mitchell, C. D. Manning, S. Ermon, and C. Finn, “Direct preference optimization: Your language model is secretly a reward model,” Advances in Neural Infor- mation Processing Systems, vol. 36, 2024

    work page 2024

  46. [46]

    Deep reinforcement learning from human pref- erences,

    P. F. Christiano, J. Leike, T. Brown, M. Martic, S. Legg, and D. Amodei, “Deep reinforcement learning from human pref- erences,” Advances in neural information processing systems, vol. 30, 2017

    work page 2017

  47. [47]

    Lora: Low-rank adaptation of large language models,

    E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen, “Lora: Low-rank adaptation of large language models,” in International Conference on Learning Representations (ICLR), 2022

    work page 2022

  48. [48]

    Freeway performance measurement system: Mining loop de- tector data,

    C. Chen, K. Petty, A. Skabardonis, P. Varaiya, and Z. Jia, “Freeway performance measurement system: Mining loop de- tector data,” Transportation Research Record, vol. 1748, no. 1, pp. 96–102, 2001

    work page 2001

  49. [49]

    SHARP: Spectrum harvesting with ARQ retransmission and probing in cognitive radio,

    S.-H. Li, Y. Zhang, A. Nosratinia, and J. Yuan, “SHARP: Spectrum harvesting with ARQ retransmission and probing in cognitive radio,” IEEE Transactions on Communications, vol. 61, no. 3, pp. 886–897, March 2013

    work page 2013

  50. [50]

    Indexability of restless bandit problems and optimality of whittle index for dynamic multichannel access,

    K. Liu and Q. Zhao, “Indexability of restless bandit problems and optimality of whittle index for dynamic multichannel access,” IEEE Transactions on Information Theory, vol. 56, no. 11, pp. 5547–5567, 2010

    work page 2010

  51. [51]

    Decentralized cognitive mac for dynamic spectrum access,

    Q. Zhao, L. Tong, and A. Swami, “Decentralized cognitive mac for dynamic spectrum access,” in First IEEE International Symposium on New Frontiers in Dynamic Spectrum Access Networks, 2005. DySPAN 2005. IEEE, 2005, pp. 224–232

    work page 2005

  52. [52]

    Amendment of the Commission’s Rules with Regard to Commercial Operations in the 3550-3650 MHz Band,

    Federal Communications Commission, “Amendment of the Commission’s Rules with Regard to Commercial Operations in the 3550-3650 MHz Band,” FCC 15-47, Report and Order and Second Further Notice of Proposed Rulemaking, Apr. 2015, gN Docket No. 12-354

    work page 2015

  53. [53]

    Marshall, Three-Tier Shared Spectrum, Shared Infrastruc- ture, and a Path to 5G

    P. Marshall, Three-Tier Shared Spectrum, Shared Infrastruc- ture, and a Path to 5G. Cambridge, UK: Cambridge University Press, 2017

    work page 2017

  54. [54]

    Wireless Innovation Forum, “Signaling Protocols and Proce- dures for Citizens Broadband Radio Service (CBRS): Spectrum Access System (SAS) - Citizens Broadband Radio Service Device (CBSD) Interface Technical Specification,” Wireless Innovation Forum, Tech. Rep. WINNF-TS-0016-V1.2.1, 2017

    work page 2017

  55. [55]

    Reputation-based incentive protocols in crowdsourcing applications,

    Y. Zhang and M. Van der Schaar, “Reputation-based incentive protocols in crowdsourcing applications,” in 2012 Proceedings IEEE INFOCOM. IEEE, 2012, pp. 2140–2148

    work page 2012

  56. [56]

    Design and analysis of incentive and reputation mechanisms for online crowdsourcing systems,

    H. Xie, J. C. Lui, and D. Towsley, “Design and analysis of incentive and reputation mechanisms for online crowdsourcing systems,” ACM Transactions on Modeling and Performance Evaluation of Computing Systems (TOMPECS), vol. 1, no. 3, pp. 1–27, 2016

    work page 2016

  57. [57]

    FedAB: Truthful federated learning with auction-based combi- natorial multi-armed bandit,

    C. Wu, Y. Zhu, R. Zhang, Y. Chen, F. Wang, and S. Cui, “FedAB: Truthful federated learning with auction-based combi- natorial multi-armed bandit,” vol. 10, no. 17, pp. 15 159–15 170

    work page

  58. [58]

    Truthful incentive mechanism for federated learning with crowdsourced data labeling,

    Y. Zhao, X. Gong, and S. Mao, “Truthful incentive mechanism for federated learning with crowdsourced data labeling,” in Proceedings of IEEE INFOCOM 2023, pp. 1–10

    work page 2023

  59. [59]

    Deepsense: Fast wideband spectrum sensing through real-time in-the-loop deep learning,

    D. Uvaydov, S. D’Oro, F. Restuccia, and T. Melodia, “Deepsense: Fast wideband spectrum sensing through real-time in-the-loop deep learning,” in IEEE INFOCOM 2021-IEEE Conference on Computer Communications. IEEE, 2021, pp. 1–10

    work page 2021

  60. [60]

    Online mixture of experts: No-regret learning for optimal collective decision-making,

    L. Liu and J. Etesami, “Online mixture of experts: No-regret learning for optimal collective decision-making,” in Advances in Neural Information Processing Systems (NeurIPS), 2025. 17 Shugang Hao (M’22) received the Ph.D. degree from Singapore University of Technology and Design (SUTD) in 2022. He is a postdoctoral research fellow at SUTD from Sep. 2022. ...

    work page 2025

  61. [61]

    If k lies (i.e., ˆy1 k = 1), then 1{ˆy1 k = 1} = 1, so E[1{ˆy1 k = p1} | ˆy1, w(1)] = γ1, and hence w2 k,lie = α + γ1 α + β + 1

    work page

  62. [62]

    Therefore, E[w2 k,lie] − E[w2 k,truth] =(1 − q) · γ1 + γ0 − 1 α + β + 1

    If k is truthful (i.e., ˆy1 k ∼ Bernoulli(q)), then conditioning on ˆy1 k: w2 k,truth = 8 < : α+γ1 α+β+1 , if ˆy1 k = 1, α+(1−γ0) α+β+1 , if ˆy1 k = 0, so taking expectation over ˆy1 k gives E[w2 k,truth] = q · α + γ1 α + β + 1 + (1 − q) · α + 1 − γ0 α + β + 1 . Therefore, E[w2 k,lie] − E[w2 k,truth] =(1 − q) · γ1 + γ0 − 1 α + β + 1 . Since γ0 +γ1 > 1, th...

    work page