Pith. sign in

REVIEW 3 major objections 5 minor 29 references

A large language model, guided by one natural-language priority statement, can pick the preferred item from thousands of multi-attribute candidates.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 07:34 UTC pith:MS6EBGAT

load-bearing objection Useful, clearly written LLM selection framework; the headline claims rest on a coarse ranking metric, and the appendix's own utility metric reverses some conclusions, so treat the method ranking as provisional. the 3 major comments →

arxiv 2510.25799 v3 pith:MS6EBGAT submitted 2025-10-29 cs.CL

LISTEN to Your Preferences: An LLM Framework for Multi-Objective Selection

Adam S. Jovine , Tinghan Ye , Francis Bahk , Jingjing Wang , Matthew Ford , David B. Shmoys , Peter I. Frazier This is my paper
classification cs.CL
keywords large language modelsmulti-objective selectionpreference elicitationutility refinementtournament selectionconcordance metriczero-shot preference oracledecision support
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to show that a large language model, given only a natural-language statement of a decision maker's priorities, can pick a strongly preferred item from a large set of multi-attribute candidates without pairwise preference queries or fine-tuning. It proposes two iterative algorithms: LISTEN-U, which has the LLM repeatedly refine a linear utility function, and LISTEN-T, which has the LLM choose winners from small random batches in a tournament. Experiments on flight booking, headphone shopping, and exam scheduling suggest LISTEN-U performs best when the human's preferences align with a linear utility model—a property measured by a new concordance metric—while LISTEN-T is more robust across all tasks. A sympathetic reader would care because, if true, this offers a cheap way to replace tedious preference elicitation with a single sentence.

Core claim

The paper's central claim is that an LLM can act as a zero-shot preference oracle for multi-objective selection: given a set of items described by numerical, categorical, and textual attributes, plus a natural-language utterance describing the decision maker's goals, the framework can identify the most preferred item using only a small number of LLM calls. The claim is supported by two mechanisms. LISTEN-U assumes a linear utility function over normalized numerical attributes and iteratively asks the LLM to critique the current best solution and propose refined weights; this lets it improve dramatically even on the lowest-concordance dataset, exam scheduling. LISTEN-T avoids parametric assum

What carries the argument

The key machinery is the LLM used as a zero-shot preference oracle inside two iterative algorithms. LISTEN-U maintains a weight vector for a linear utility function u(s)=w^T s_num; each iteration it shows the LLM the current best solution unnormalized and asks for revised weights, then rescales scores across the whole set using normalized attributes. LISTEN-T samples batches of size B uniformly, asks the LLM to pick the best in each batch, and then runs a final LLM playoff over the batch champions. A supporting object is the concordance metric: the proportion of 1,000 random linear utility functions for which the argmax equals the human's top-ranked item; low concordance flags datasets where

Load-bearing premise

The evaluation treats the rankings of one self-selected expert per domain—the same person who wrote the preference utterance—as ground truth; if that expert's preferences or phrasing do not represent the target user population, the relative ranking of the algorithms could change.

What would settle it

Have several independent human experts rank the same candidate items from the same written preference utterance, then rerun the comparison. If LISTEN-T no longer consistently beats the z-score baseline across those experts, or if LISTEN-U fails to outperform baselines on the high-concordance headphones dataset, the central claim would be refuted. A cheaper check: re-estimate the concordance metric using each expert's top-ranked item and see whether the predicted 'hard' datasets stay hard.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If the claims hold, a decision maker can replace pairwise comparisons or utility queries with one natural-language sentence, and the LLM will select from thousands of options within a few dozen calls.
  • LISTEN-U's iterative refinement can overcome a bad initial linear model on low-concordance problems, so parametric utility methods need not be abandoned when preferences are non-linear.
  • LISTEN-T provides a robust fallback that never underperforms the z-score baseline across the tested domains, so a non-parametric tournament is a safe default when preference structure is unknown.
  • Including the natural-language preference utterance was never harmful and often helped, particularly for subjective preferences like headphone shopping, so richer preference text is a safe investment.
  • The concordance metric gives a dataset-inherent difficulty score, independent of any algorithm, which can predict when a linear-utility method will pay off.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper's single-expert ground truth leaves open whether these methods generalize to a population; a direct extension would be to test LISTEN with multiple experts who independently rank the same items and see if LISTEN-T remains robust and LISTEN-U still wins on high-concordance subsets.
  • The concordance metric could be repurposed as an online diagnostic: before running LISTEN-U, sample a few random linear utilities, compare their top items, and decide which algorithm to deploy—this is a testable extension the paper does not run.
  • If natural-language preference oracles become reliable, the bottleneck shifts to writing good utterances; the paper's ablations suggest the marginal value of utterance detail is domain-dependent, so future work might learn to rewrite user statements for maximal selectivity.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes LISTEN, an agentic LLM framework for selecting a single preferred item from a large multi-attribute candidate set using only a natural-language preference utterance. Two algorithms are presented: LISTEN-U, which iteratively asks an LLM to refine the weights of a linear utility function, and LISTEN-T, which samples small batches and runs an LLM tournament. Using one expert-authored preference utterance and partial expert rankings per domain, the authors compare against random and z-score baselines on flight, shopping, and exam-scheduling datasets, reporting Normalized Average Rank (NAR) over 25 iterations, plus a concordance metric meant to predict when linear-utility methods succeed. They report that LISTEN-U excels on exam scheduling and high-concordance domains while failing on one flight dataset, and that LISTEN-T is robust across all.

Significance. The potential contribution is real: if LLMs can translate a short preference statement into a good selection after a bounded number of calls, it would reduce the elicitation burden in multi-objective choice. Strengths include the clear algorithmic framework, two LLMs, 50 replications with error bars, public code/data, and an honest appendix that introduces a second metric. However, the empirical case is not yet conclusive: the primary metric collapses all unranked items into one score, and the authors’ own auxiliary metric reverses several method rankings. The single-expert ground truth also limits generalization. The work is a promising proof of concept, but the central claims are metric-dependent and need strengthening.

major comments (3)
  1. [§4.4, Table 1, Appendix B.3] The primary metric makes the central ranking fragile. NAR assigns every unranked item the same value (m+1+N)/2, and Table 1 shows m/N is 0.020–0.195. Consequently NAR is essentially a top-m hit rate: every miss is scored ~0.5 and only hits inside a tiny ranked set receive differentiated credit. The paper itself concedes this in Appendix B, yet the alternative AUS metric reverses the conclusions on low-concordance datasets: B.3 reports that on Exam, LISTEN-U’s NAR improves while its AUS shows little improvement, and on Flights00/01 LISTEN-U is best by AUS but worst by NAR. The statement in §4.5 that LISTEN-U “progressively refines its utility function to find dramatically better solutions” on Exam is therefore not supported independently of a metric that cannot distinguish among the vast majority of items. Please report a metric that gives partial credit to unranked items or top-k hit rat
  2. [§4.1, §5] The ground truth for every dataset is a single self-selected expert who also wrote the natural-language utterance used by the algorithms. There is no inter-rater reliability, no held-out expert, and no multi-user validation. The paper acknowledges this as a limitation, but it is load-bearing because the method ranking—especially the claim that LISTEN-U “excels” on a given dataset—could change with a different expert’s preferences or phrasing. At minimum, add a second expert for a subset of datasets or an utterance-sensitivity analysis; otherwise the conclusions should be framed as a single-user proof of concept.
  3. [§1.1, §4.5] The paper uses “significantly outperforms” and “best performance” without formal significance tests. The 50-replication error bars are helpful, but overlapping intervals and multiple comparisons across 25 iterations and five datasets make informal comparisons unreliable. Report paired tests (e.g., bootstrap or Wilcoxon over seeds) for the final iteration and for the learning trajectory, and correct for multiple comparisons.
minor comments (5)
  1. [§4.2] Please clarify what randomness is used for LISTEN-U, since the algorithm is deterministic given the LLM outputs. Is it sampling temperature, and how is it controlled across replications?
  2. [Appendix B.1] The ranked-vs-unranked comparisons always label the ranked item as preferred. This can bias the fitted utility function, especially when ranked items are a tiny subset. Please justify this choice or test its sensitivity.
  3. [Figures 5 and 9] The “Hidden ground truth max” line and the scaling of AUS are not clearly defined. The AUS ranges differ wildly across datasets (e.g., ~0.04 for Exam vs ~80 for Flights01), making the plots hard to interpret. Add definitions and axis explanations.
  4. [Appendix E.1] The text contains informal phrasing (“chalked up to differences in LLM”) and a typo (“performing”). Also, the Gemini 2.5 reference has a placeholder author list “(and many authors)” that should be completed.
  5. [§4.7, Appendix D] The claim that including the preference utterance “never degraded performance” is asserted from plots without numerical support. Report the relevant aggregated numbers for the final iteration.

Circularity Check

0 steps flagged

No significant circularity: the central evaluation chain is empirical and the algorithm outputs are not derived from the human rankings or from the auxiliary fitted utility.

full rationale

The paper's central claim is an empirical comparison of LISTEN-U, LISTEN-T, and baselines against human rankings. The algorithm outputs are produced by LLM calls from the natural-language utterance and candidate sets; the human rankings are not used to construct or refine those outputs. Normalized Average Rank is coarse (all unranked items share one average rank), but that is a metric-resolution limitation, not a circular reduction: the metric does not define the algorithms' outputs. The concordance metric is an independent dataset property (fraction of random linear utilities matching the human top item), and the validation in Section 4.6 is a separate experimental test, not a fitted-input-as-prediction step. The Average Utility Score in Appendix B fits a linear utility to the same human rankings and then scores algorithm outputs with it; although this is a self-referential evaluation form and can flatter a linear-utility method like LISTEN-U, the paper does not use it to generate the algorithm's choices, and it explicitly reports divergence between NAR and AUS and frames AUS as complementary, not as a prediction. The only self-citation, Ye et al. (2024), is used to generate the exam-scheduling candidate set and is not load-bearing for the framework's claims. The acknowledged single-expert ground-truth limitation is a benchmarking-validity concern, not a circular-derivation concern. No equation or metric in the paper reduces the claimed predictions to their own inputs by construction, so no circular step is present.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 0 invented entities

The paper contributes algorithms, not a mathematical derivation. Its central claims rest on domain assumptions about LLM preference fidelity, single-expert ground truth, linear utility structure, and the validity of the concordance metric. There are no invented physical entities. The main hand-chosen degrees of freedom are prompt templates, iteration budgets, batch size, and concordance sampling details.

free parameters (5)
  • LISTEN-T batch size B = not reported in the paper (only in code)
    Algorithm 2 requires B, and experiments plot over 25 iterations but never state B; results depend on this choice.
  • Number of refinement iterations T = 25 in all plots
    Performance curves are shown to 25 iterations; the stopping rule for LISTEN-U is a fixed budget chosen by the authors, not determined by the data.
  • Concordance sampling distribution = U[-1,1] with 1,000 samples
    The concordance metric samples weights uniformly from [-1,1]; the choice of distribution and number of samples is a modeling choice that defines the metric's behavior.
  • AUS logistic-regression hyperparameters = q=0.95, N=10,000, 5,000/5,000 split
    Hyperparameters in Appendix B's utility-fitting procedure; not varied or justified. They affect only the secondary metric.
  • Domain prompt templates and preference utterance wording = hand-authored per dataset
    LISTEN's outputs are entirely determined by these prompts; the paper does not vary prompts systematically except the Headphones-Strict test. This is a hand-chosen input, not a fitted number, listed for completeness.
axioms (5)
  • domain assumption A pretrained LLM can act as a zero-shot preference oracle whose comparative judgments align with the target human's preferences (Section 3.1).
    The entire framework assumes LLM choices and weight proposals are meaningful proxies for the human's implicit utility.
  • domain assumption Linear utility over normalized numerical attributes is an adequate preference model for LISTEN-U's intended settings (Algorithm 1).
    LISTEN-U scores items as w^T s_num; the paper acknowledges non-linear preferences break this and uses concordance to measure the mismatch.
  • domain assumption Single-expert rankings of a small subset of items constitute valid ground truth (Section 4.1).
    All headline comparisons are made against rankings from one expert per domain; the paper's own limitation section says this may not generalize.
  • domain assumption The concordance metric—fraction of random linear utilities matching the human's top pick—measures the difficulty a parametric approach will face (Section 4.4).
    The paper uses this metric to explain when LISTEN-U succeeds or fails; if the metric is not predictive, that explanation weakens.
  • domain assumption The natural-language utterance U faithfully and completely expresses the same preferences used to produce the ground-truth ranking (Section 4.1).
    The same expert wrote both the utterance and the ranking; if the utterance omits or distorts key trade-offs, the LLM cannot align with the ranking.

pith-pipeline@v1.3.0-alltime-deepseek · 19112 in / 12394 out tokens · 114846 ms · 2026-08-04T07:34:25.439974+00:00 · methodology

0 comments
read the original abstract

Human experts often struggle to select the best option from a large set of items with multiple competing objectives, a process bottlenecked by the difficulty of formalizing complex, implicit preferences. To address this, we introduce LISTEN (LLM-based Iterative Selection with Trade-off Evaluation from Natural-language), an agentic LLM-based framework that treats the LLM as a decision-making agent capable of iteratively refining its internal preference model and taking actions (e.g., proposing utilities or selecting candidates) to maximize alignment with a user's implicit goals. To operate within LLM constraints like context windows and inference costs, we propose two iterative algorithms: LISTEN-U, which uses the LLM to refine a parametric utility function, and LISTEN-T, a non-parametric method that performs tournament-style selections over small batches of solutions. Evaluated on diverse tasks including flight booking, shopping, and exam scheduling, our results show LISTEN-U excels when preferences are parametrically aligned (a property we measure with a novel concordance metric), while LISTEN-T offers more robust performance overall. This work explores a promising direction for steering complex multi-objective decisions directly with natural language, reducing the cognitive burden of traditional preference elicitation. Code is available at https://github.com/AdamJovine/LISTEN; data is available at https://huggingface.co/datasets/AdamJovine/LISTEN-benchmark.

Figures

Figures reproduced from arXiv: 2510.25799 by Adam S. Jovine, David B. Shmoys, Francis Bahk, Jingjing Wang, Matthew Ford, Peter I. Frazier, Tinghan Ye.

Figure 1
Figure 1. Figure 1: A schematic overview of the LISTEN framework. A human decision maker provides preferences in [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Performance of LISTEN algorithms and baselines on four datasets, showing the Normalized Average [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Performance of LISTEN-U and LISTEN-T on the Headphones-General prompt and the modified Headphones-Strict prompt, which has lower concor￾dance. 4.7 Ablation Study: Impact of the Preference Utterance To isolate the value of the user’s natural language pref￾erence utterance (U), we conducted an ablation study comparing performance with a full, preference-guided prompt against a base prompt containing only the… view at source ↗
Figure 4
Figure 4. Figure 4: Ablation study evaluating the impact of pref [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Performance of LISTEN algorithms and baselines on five datasets, showing the Average Utility Score [PITH_FULL_IMAGE:figures/full_fig_p014_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Performance of LISTEN algorithms and baselines on Flights01 dataset, showing the Normalized [PITH_FULL_IMAGE:figures/full_fig_p015_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Ablation study evaluating the impact of preference utterance. Performance using the preference-guided [PITH_FULL_IMAGE:figures/full_fig_p016_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Performance of LISTEN algorithms and baselines on five datasets, showing the Normalized Average [PITH_FULL_IMAGE:figures/full_fig_p017_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Performance of LISTEN algorithms and baselines on five datasets, showing the Average Utility Score [PITH_FULL_IMAGE:figures/full_fig_p018_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Ablation study evaluating the impact of preference utterance. Performance using the preference [PITH_FULL_IMAGE:figures/full_fig_p019_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: An example prompt for the exam scheduling dataset. [PITH_FULL_IMAGE:figures/full_fig_p020_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: An example prompt for a flight itinerary preference, which utilizes the natural language description [PITH_FULL_IMAGE:figures/full_fig_p021_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: An example prompt for headphone preferences. [PITH_FULL_IMAGE:figures/full_fig_p022_13.png] view at source ↗

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

29 extracted references · 8 linked inside Pith

  1. [1]

    AhmadiTeshnizi, A., Gao, W., and Udell, M. (2023). Optimus: Optimization modeling using mip solvers and large language models. arXiv preprint arXiv:2310.06116

    Pith/arXiv arXiv 2023

  2. [2]

    E., Korikov, A., Toroghi, A., and Sanner, S

    Austin, D. E., Korikov, A., Toroghi, A., and Sanner, S. (2024). Bayesian optimization with llm-based acquisition functions for natural language preference elicitation. arXiv preprint arXiv:2405.00981

    Pith/arXiv arXiv 2024

  3. [3]

    and Song, H

    Bang, S. and Song, H. (2025). Llm-based user profile management for recommender system. arXiv preprint arXiv:2502.14541

    Pith/arXiv arXiv 2025

  4. [4]

    Branke, J. (2008). Multiobjective optimization: Interactive and evolutionary approaches , volume 5252. Springer Science & Business Media

    2008

  5. [5]

    C., Jacobsson, H., Szpektor, I., Jiang, N.-J., Haridasan, K., and (and many authors),

    Comanici, G., Bieber, E., Schaekermann, M., Pasupat, I., Sachdeva, N., Dhillon, I., Blistein, M., Ram, O., Zhang, D., Rosen, E., Marris, L., Petulla, S., Gaffney, C., Aharoni, A., Lintz, N., Pais, T. C., Jacobsson, H., Szpektor, I., Jiang, N.-J., Haridasan, K., and (and many authors), . (2025). Gemini 2.5: Pushing the frontier with advanced reasoning, mul...

    2025

  6. [6]

    and Alireza B, D

    Farzane, K. and Alireza B, D. (2022). A review and evaluation of multi and many-objective optimization: Methods and algorithms. Global Journal of Ecology , 7(2):104–119

    2022

  7. [7]

    Gao, T., Xu, R., Wang, W., and Chen, D. (2025). Take caution in using llms as human surrogates: Scylla ex machina. arXiv preprint arXiv:2410.19599

    Pith/arXiv arXiv 2025

  8. [8]

    Gunantara, N. (2018). A review of multi-objective optimization: Methods and its applications. Cogent Engineering , 5(1):1502242

    2018

  9. [9]

    Huang, C., Tang, Z., Hu, S., Jiang, R., Zheng, X., Ge, D., Wang, B., and Wang, Z. (2025). Orlm: A customizable framework in training large models for automated optimization modeling. Operations Research

    2025

  10. [10]

    R., and Astudillo, R

    Huber, F., Gonzalez, S. R., and Astudillo, R. (2025). Bayesian preference elicitation for decision support in multiobjective optimization. arXiv preprint arXiv:2507.16999

    arXiv 2025

  11. [11]

    Klamkin, M., Deza, A., Cheng, S., Zhao, H., and Van Hentenryck, P. (2025). Dualschool: How reliable are llms for optimization education? arXiv preprint arXiv:2505.21775

    Pith/arXiv arXiv 2025

  12. [12]

    Knowles, J. D. (2006). Parego: A hybrid algorithm with on-line landscape approximation for expensive multiobjective optimization problems. IEEE Transactions on Evolutionary Computation , 10(1):50--66

    2006

  13. [13]

    Lawless, C., Li, Y., Wikum, A., Udell, M., and Vitercik, E. (2025). Llms for cold-start cutting plane separator configuration. In International Conference on the Integration of Constraint Programming, Artificial Intelligence, and Operations Research , pages 51--69. Springer

    2025

  14. [14]

    I Want It That Way

    Lawless, C., Schoeffer, J., Le, L., Rowan, K., Sen, S., Hill, C. S., Suh, J., and Sarrafzadeh, B. (2023). “I Want It That Way”: Enabling Interactive Decision Support Using Large Language Models and Constraint Programming . arXiv preprint arXiv:2312.06908 . Submitted December 12, 2023; revised October 1, 2024

    Pith/arXiv arXiv 2023

  15. [15]

    Li, F. et al. (2025). Aligning with logic: Measuring, evaluating and improving logical preference consistency. In Proceedings of the 42nd International Conference on Machine Learning (ICML) . Poster

    2025

  16. [16]

    The llama 3 herd of models

    Meta AI (2024). The llama 3 herd of models

    2024

  17. [17]

    Obayashi, S., Jeong, S., Chiba, K., and Morino, H. (2007). Multi-objective design exploration and its application to regional-jet wing design. Transactions of the Japan Society for Aeronautical and Space Sciences , 50(167):1--8

    2007

  18. [18]

    Okeukwu-Ogbonnaya, A., Amatapu, R., Bergtold, J., and Amariucai, G. (2025). Llm-based community surveys for operational decision making in interconnected utility infrastructures. arXiv preprint arXiv:2507.13577

    Pith/arXiv arXiv 2025

  19. [19]

    Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022). Training language models to follow instructions with human feedback. Advances in neural information processing systems , 35:27730--27744

    2022

  20. [20]

    Ozaki, R., Ishikawa, K., Kanzaki, Y., Suzuki, S., Takeno, S., Takeuchi, I., and Karasuyama, M. (2024). Multi-objective bayesian optimization with active preference learning. In Proceedings of the Thirty-Eighth AAAI Conference on Artificial Intelligence , pages 14490--14498, Vancouver, Canada. AAAI Press

    2024

  21. [21]

    Ramamonjison, R., Yu, T., Li, R., Li, H., Carenini, G., Ghaddar, B., He, S., Mostajabdaveh, M., Banitalebi-Dehkordi, A., Zhou, Z., et al. (2023). Nl4opt competition: Formulating optimization problems based on their natural language descriptions. In NeurIPS 2022 competition track , pages 189--203. PMLR

    2023

  22. [22]

    Schwartz, B. (2015). The paradox of choice. Positive psychology in practice: Promoting human flourishing in work, health, education, and everyday life , pages 121--138

    2015

  23. [23]

    and Kumar, V

    Sharma, S. and Kumar, V. (2022). A comprehensive review on multi-objective optimization techniques: Past, present and future. Archives of Computational Methods in Engineering , 29(7):5605--5633

    2022

  24. [24]

    Wang, X., Jin, Y., Schmitt, S., and Olhofer, M. (2022). Recent advances in bayesian optimization. arXiv preprint arXiv:2206.03301

    Pith/arXiv arXiv 2022

  25. [25]

    J., Han, X., Fu, X., Zhong, T., Zeng, J., Song, M., et al

    Xiao, Z., Zhang, D., Wu, Y., Xu, L., Wang, Y. J., Han, X., Fu, X., Zhong, T., Zeng, J., Song, M., et al. (2023). Chain-of-experts: When llms meet complex operations research problems. In The twelfth international conference on learning representations

    2023

  26. [26]

    V., Zhou, D., and Chen, X

    Yang, C., Wang, X., Lu, Y., Liu, H., Le, Q. V., Zhou, D., and Chen, X. (2023). Large language models as optimizers. In The Twelfth International Conference on Learning Representations

    2023

  27. [27]

    Ye, T., Jovine, A., van Osselaer, W., Zhu, Q., and Shmoys, D. B. (2024). Cornell university uses integer programming to optimize final exam scheduling. arXiv preprint arXiv:2409.04959

    arXiv 2024

  28. [28]

    Zhang, H., Zhu, Q., and Dou, Z. (2025). Enhancing reranking for recommendation with llms through user preference retrieval. Proceedings of the 31st International Conference on Computational Linguistics , pages 658--671

    2025

  29. [29]

    Zheng, L., Chiang, W.-L., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., Li, D., Xing, E., et al. (2023). Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in neural information processing systems , 36:46595--46623

    2023