Pith. sign in

REVIEW 5 cited by

Neural Dueling Bandits: Preference-Based Optimization with Human Feedback

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2407.17112 v2 pith:GEINMZKM submitted 2024-07-24 cs.LG cs.AIstat.ML

Arun Verma , Zhongxiang Dai , Xiaoqiang Lin , Patrick Jaillet , Bryan Kian Hsiang Low This is my paper · ORCID
classification cs.LGcs.AIstat.ML
keywords feedbackresultsarmsbanditalgorithmscontextualduelingfunction
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Contextual dueling bandit is used to model the bandit problems, where a learner's goal is to find the best arm for a given context using observed noisy human preference feedback over the selected arms for the past contexts. However, existing algorithms assume the reward function is linear, which can be complex and non-linear in many real-life applications like online recommendations or ranking web search results. To overcome this challenge, we use a neural network to estimate the reward function using preference feedback for the previously selected arms. We propose upper confidence bound- and Thompson sampling-based algorithms with sub-linear regret guarantees that efficiently select arms in each round. We also extend our theoretical results to contextual bandit problems with binary feedback, which is in itself a non-trivial contribution. Experimental results on the problem instances derived from synthetic datasets corroborate our theoretical results.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Neural Variance-aware Dueling Bandits with Deep Representation and Shallow Exploration

    cs.LG 2025-06 unverdicted novelty 6.0 of 10

    Variance-aware neural dueling bandit algorithms achieve sublinear regret of order O(d sqrt(sum sigma_t^2) + sqrt(d T)) for wide networks on nonlinear utilities.

  2. Online Clustering of Dueling Bandits

    cs.LG 2025-02 conditional novelty 6.0 of 10

    COLDB and CONDB are the first algorithms to combine online user clustering with dueling (preference) bandits, with regret bounds that improve as users are grouped into fewer clusters.

  3. Federated Linear Dueling Bandits

    cs.LG 2025-02 reject novelty 6.0 of 10

    A new federated linear dueling bandit algorithm with claimed sublinear regret, but the key proof step is invalid.

  4. Large Language Model-Enhanced Multi-Armed Bandits

    cs.LG 2025-02 conditional novelty 5.0 of 10

    Using an LLM as a reward predictor inside Thompson sampling and regression-oracle bandits outperforms LLM direct arm selection in the tested tasks.

  5. T-POP: Test-Time Personalization with Online Preference Feedback

    cs.LG 2025-09 conditional novelty 4.0 of 10

    T-POP uses dueling-bandit token selection to learn a reward function online from pairwise user feedback, enabling test-time personalization of a frozen LLM without fine-tuning.

Pith tools