Pith. sign in

REVIEW 2 major objections 6 minor 2 cited by

Division-of-Thoughts: Harnessing Hybrid Language Model Synergy for Efficient On-Device Agents

T0 review · 2 major / 6 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that routing sub-tasks between a small on-device model and a cloud LLM, with a lightweight adapter trained without labels, keeps accuracy close to the best baselines while cutting reasoning time by 66% and API cost by 84%.

desk verdict Division-of-Thoughts has a genuinely clever self-reinforced label search, but the evaluation as written trains the adapter on the test set, so the headline cost-accuracy numbers are not trustworthy. read the letter →

arxiv 2502.04392 v1 pith:EW5OQL3K submitted 2025-02-06 cs.CL cs.AI

Chenyang Shao , Xinyuan Hu , Yutang Lin , Fengli Xu This is my paper · ORCID
classification cs.CLcs.AI
keywords LargeLanguageModelsSmallOn-deviceAIagentsEdge-cloudcollaborationTaskdecompositionModelallocationSelf-reinforcedtrainingDifficultyestimation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper is trying to establish that a 'division of labour' at the sub-task level can make on-device AI agents both accurate and affordable: decompose a user query into sub-tasks, build a dependency graph for parallel execution, and assign each sub-task to either the local small model or the cloud LLM based on difficulty. The difficulty scores come from a detachable adapter that reads sentence embeddings from the small model and outputs a routing decision, without modifying the small model's weights. The adapter's training labels are generated automatically by an alpha-tree search that uses the small model's per-token probabilities to rank sub-task difficulty and then swaps sub-tasks between models until the final answer is correct. Across seven benchmarks, the paper reports average reasoning time and API cost reductions of 66.12% and 83.57%, respectively, while keeping accuracy comparable to the strongest baselines.

What carries the argument

The load-bearing mechanism is the $\alpha$-quantile difficulty proxy: for each sub-task, the SLM answers the question and records the per-token sampling probabilities; the $\alpha$-quantile of those probabilities (Eq. 4) is taken as a difficulty score, exploiting the assumption that harder sub-tasks yield lower token probabilities. These scores order sub-tasks by difficulty, and the $\alpha$-tree search (Algorithm 1) converts that ordering into allocation labels by repeatedly moving sub-tasks between models based only on whether the final answer is correct. The resulting labels train a plug-and-play MLP adapter that maps the SLM's prompt-based sentence embeddings to a difficulty score, routing each sub-task to the SLM or LLM without altering the SLM's parameters.

What would settle it

Run the $\alpha$-tree search on a held-out set and measure the rank correlation between the SLM's $\alpha$-quantile token probabilities and per-subtask error rates; if the correlation is weak (e.g., Spearman $\rho < 0.3$) on MATH or DROP, the difficulty ordering is not load-bearing and the adapter would route at near-chance.

Watch

Extended reading notes

Core claim

The central claim is that sub-task-level routing between a locally deployed small language model and a cloud-based LLM, guided by a lightweight adapter trained through self-reinforced feedback, achieves accuracy comparable to the best baseline methods while cutting average reasoning time by 66.12% and API costs by 83.57% (Table 1). The paper further claims that the alpha-tree search produces high-quality allocation labels without human annotation, reaching an SLM usage ratio of 85.53% with a 99.44% success rate on MATH (Table 2), and that the resulting adapter generalizes across benchmarks better than no training at all (Appendix A.1).

Load-bearing premise

The method rests on the premise that a small model's per-token sampling probabilities order sub-tasks by true difficulty (Eq. 4), so that the alpha-tree search's final-answer feedback produces correct allocation labels; if hard sub-tasks can have high token probabilities, the adapter learns a faulty routing policy and the cost savings come with a hidden accuracy penalty.

Editorial extensions

If this is right

  • An on-device agent using DoT would run the majority of reasoning steps locally, reducing latency, privacy exposure, and per-query monetary cost; the paper reports average time and API cost reductions of 66.12% and 83.57%.
  • The adapter can be retrained or extended to new tasks by re-running the alpha-tree search, since its training requires no human annotation and leaves the SLM's general capabilities intact.
  • Dependency-graph scheduling allows independent sub-tasks to be reasoned about in parallel, with time savings that grow as the number of sub-tasks increases (Table 6).
  • The adapter generalizes across benchmarks: training on one set of tasks still outperforms no training on held-out tasks (Appendix A.1), and the cost-accuracy trade-off curve stays above the DataShunt baseline across budgets (Figure 6).

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: if the $\alpha$-quantile probability reliably orders difficulty, the same signal could be used for direct routing without the adapter, or to decide when to defer an entire query rather than individual sub-tasks.
  • Editorial inference: the alpha-tree search attributes changes in final-answer correctness to the swapped sub-task, which is reasonable when sub-tasks are nearly independent; on benchmarks with tightly coupled sub-tasks, the search may misattribute errors and produce noisier labels than reported.
  • Editorial inference: a stress test that deliberately miscalibrates the SLM's token probabilities (e.g., with adversarial prompts) would reveal how much of the reported cost savings depends on the probability-difficulty correlation holding.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. The paper introduces Division-of-Thoughts (DoT), a hybrid edge-cloud reasoning framework in which a local SLM decomposes a user query into subtasks, a dependency graph schedules the subtasks, and a lightweight MLP adapter attached to the SLM routes each subtask to either the local SLM or a cloud LLM. The adapter is trained on allocation labels generated by an alpha-tree search that ranks subtask difficulty using alpha-quantiles of token sampling probabilities and then reassigns subtasks based on final-answer correctness. On seven benchmarks, the authors report that DoT reduces average reasoning time and API cost by 66.12% and 83.57%, respectively, while maintaining accuracy close to the best baselines. The paper also reports ablations, cost-accuracy trade-off curves, and generalization experiments for the adapter.

Significance. The contribution is potentially significant: subtask-level routing between an on-device SLM and a cloud LLM, with a detachable adapter that does not modify the SLM, is a practical and timely idea for cost-efficient agents, and the self-reinforced label-generation loop addresses a real annotation bottleneck. The paper is generally well structured, provides code, includes careful ablations, and offers a useful decomposition-quality analysis. If the train/test-overlap concern below is resolved, the large and consistent cost savings would make this a valuable addition to the edge-cloud agent literature. However, as written, the headline numbers are not interpretable until the relationship between the adapter's training data and the 200-question test set is clarified.

major comments (2)
  1. [§4.4, §5.1, Appendix A.1, Table 4] The manuscript never states that the tasks used to generate alpha-tree labels for adapter training are disjoint from the 200-question test set described in §5.1. Section 4.4 and Algorithm 1 describe label generation over 'tasks' without mentioning a separate training set, and Appendix A.1 provides rows such as 'MATH MATH 59%' as 'task-specific training,' which only makes sense if the adapter is trained on the same benchmark from which the test set is drawn. If the adapter is trained on the same 200 test questions, then Table 1's accuracy figures are in-sample estimates and the claimed 'comparable accuracy' may not hold for never-seen tasks. The Appendix's own disjoint-training numbers are lower on every benchmark: MATH drops from 59% to 53%, CHAMP from 58% to 50%, CSQA from 82% to 76%, and P3 from 41% to 33%, and in several cases those disjoint numbers fall below the best baseline (MATH 53% vs. ToT 63%, CHAMP 50% vs. ToT 57%, P3 33% vs. CoT 42%). Please state explicitly whether the alpha-tree labels are generated from the same 200 questions used for evaluation, and if they are, rerun the main experiments with a genuinely disjoint training set for adapter training.
  2. [§4.4, Eq. (4), Algorithm 1, Table 2] The alpha-quantile difficulty ordering is a load-bearing assumption of the label-generation process, but the paper provides no direct evidence that the quantile ranking correlates with sub-task-level difficulty across task types. The paper asserts that higher difficulty leads to lower token sampling probabilities and then reverses the ranking, but this correlation could vary across math, logic, commonsense, and web-browsing subtasks, and Section 5.3 validates the method only on MATH. Please report, for at least MATH and one non-math benchmark, the relationship between the alpha-quantile values and per-subtask correctness, or otherwise justify that the ordering is reliable enough to support the adapter's routing decisions outside the training distribution.
minor comments (6)
  1. [§4.4] The sentence 'How cam we construct a dataset to train our adapter?' contains a typo; it should be 'How can we construct...'.
  2. [Algorithm 1] The notation in Algorithm 1 is not fully consistent with Eq. (4): the alpha-quantile values are written as V_i without a definition in the algorithm, and the per-token probabilities p are introduced in Eq. (4) but not explicitly tied to the algorithm's Prob_token variable. Please align the notation.
  3. [Table 4] The table contains multiple typos and formatting issues: 'CHANP' should be 'CHAMP' in panels (b) and (f), and the cross-training row 'DROP,CHAMP,MATH.P3,SCAN,Webshop,CSQA' should use commas consistently. These should be corrected.
  4. [§5.1] The paper uses 'SLM' and 'on-device LLM' interchangeably for Llama 3-8B; please standardize the terminology to avoid confusion.
  5. [§5.5] The procedure for generating the trade-off curves in Figure 6 is described only vaguely as 'proportionally assigning more sub-tasks to either the cloud-based LLM or the local deployed SLM.' Please specify how the proportion is chosen at each data point so the curves are reproducible.
  6. [§5.1] The main results are point estimates on 200 sampled questions per benchmark with no confidence intervals or significance tests. Given that some accuracy differences are small (e.g., P3 41% vs. 42% for CoT), please report variance or a paired test, at least for the central comparison.

Circularity Check

1 steps flagged · score 6.0 of 10

Headline accuracy figures are produced by an adapter whose training labels come from the same benchmark/test set, with no stated disjoint split.

  1. fitted input called prediction [Section 4.4 (alpha-tree training data), Section 5.1 (200-question test set), Appendix A.1 (Table 4 train/test rows)]
    ""Additionally, we randomly extract 200 questions from each benchmark to constitute our test set." (Section 5.1); "Intra-category generalization: Within three mathematical benchmarks, train on two and test on the remaining one." (Appendix A.1); Table 4(a): "Train MATH Test MATH Acc 59%" vs. "/ MATH 32%"."

    Algorithm 1's alpha-tree generates the adapter's allocation labels using final-answer correctness on the task set, and the adapter is then evaluated on the same benchmarks. The paper never describes a training split for the adapter that is disjoint from the 200-question test set described in Section 5.1. Appendix A.1's 'task-specific training' configuration (Train MATH, Test MATH, Acc 59%) is identical to the main MATH result in Table 1, showing that the default protocol trains on the benchmark being evaluated. In the absence of any stated separation, the 200 test questions can themselves supply the correctness feedback used by the alpha-tree search, which would make the headline accuracy and cost figures in-sample fits rather than predictions.

full rationale

The central efficiency claim of the paper is that DoT reduces time and API cost by 66.12% and 83.57% while maintaining accuracy comparable to the best baselines (Abstract, Table 1). That claim depends on the Plug-and-Play Adapter trained on labels produced by the alpha-tree search (Algorithm 1), which uses task correctness to reassign subtasks between the SLM and LLM. The paper specifies a 200-question test set per benchmark (Section 5.1) but never specifies that adapter-training labels come from a different, held-out set. Appendix A.1's generalization tables treat 'task-specific training' as training and testing on the same benchmark (e.g., Train MATH / Test MATH = 59%, matching the main table), which is strong evidence that the main results are computed on the same benchmark distribution used for label generation. Unless a disjoint split is supplied, the reported accuracy is not an independent prediction but a fit to the questions used to create the routing labels. No other load-bearing circularity was found: the alpha-quantile difficulty-ordering assumption is an empirical premise rather than a tautology, and the paper's self-citations are not used to justify the central result. The issue is therefore concentrated in the missing train/test separation for the adapter, making the headline figures partially circular.

Assumptions & free parameters 5 free parameters · 5 assumptions · 1 invented entities

DoT relies on a small number of unverified domain assumptions: a probability-difficulty correlation, binary correctness as a search signal, and an A100 stand-in for edge hardware. The key free parameters (alpha, theta, adapter threshold) are unreported, and the manual decomposition examples are a per-benchmark human input.

free parameters (5)
  • alpha (quantile level for uncertainty)
    Used in Eq. 4 to compute alpha-quantile token probabilities for difficulty ranking. Value not reported in the paper; affects which subtasks are judged easy or hard.
  • threshold theta (initial allocation threshold)
    In Algorithm 1, subtasks with alpha-quantile above theta are assigned to SLM, below to LLM. Value not reported; directly determines initial allocation and thus the search start.
  • N (number of subtasks reassigned per search step) = 1 or 2
    Algorithm 1 moves N subtasks between models at each iteration. Tested with n=1 and n=2; the choice affects search depth and dataset quality.
  • Adapter decision threshold
    The MLP outputs a difficulty score in [0,1]; the paper says scores are set as 0 or 1 but does not specify the inference-time cutoff for sending a subtask to the LLM.
  • Manual decomposition examples per benchmark = 8 per benchmark
    For each benchmark, 8 randomly selected tasks are manually decomposed and included in the ICL prompt (Section 4.2). These hand-crafted examples are chosen by the authors and affect decomposition quality.
assumptions (5)
  • domain assumption Higher subtask difficulty leads to lower token sampling probabilities, so alpha-quantile ranks difficulty.
    Section 4.4: 'higher task difficulty leads to greater uncertainty in the model's answers, resulting in lower token sampling probabilities'. This is the foundation of the alpha-tree difficulty ranking and is not empirically validated in the paper.
  • domain assumption Final task correctness (True/False) is a sufficient signal to identify which subtasks to reassign between SLM and LLM.
    Algorithm 1 uses only the binary result of the full reasoning pass to move subtasks; this assumes the error is attributable to the hardest subtasks currently on the SLM (or the easiest on the LLM when correct).
  • domain assumption The LLM-generated dependency graph is accurate enough that depth-based batching preserves correctness.
    Section 4.3 relies on prompt-generated pairwise dependencies; if the graph misses or adds edges, parallel reasoning may use incomplete or invalid context.
  • domain assumption An A100 GPU hosting Llama 3-8B represents an on-device deployment.
    Section 5.1: 'The Llama 3-8B model is deployed on a local A100 GPU'. Real phones have much lower memory bandwidth and compute, so measured time savings may not hold on-device.
  • domain assumption The 8 hand-crafted decomposition examples per benchmark elicit sufficiently good decompositions for unseen tasks.
    Section 4.2: decomposition quality is evaluated only via downstream accuracy and an independence score, not proven to generalize.
invented entities (1)
  • Plug-and-Play Adapter (MLP task head)
    purpose: Maps SLM sentence embeddings to a difficulty score used to route each subtask to SLM or LLM.
    The adapter is a new component introduced by DoT. Its only evidence comes from the benchmarks in this paper; there is no external falsifiable prediction or independent evaluation outside the reported datasets.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Division-of-Thoughts: Harnessing Hybrid Language Model Synergy for Efficient On-Device Agents." pith.science (2026). https://pith.science/paper/EW5OQL3K

@misc{pith2026250204392,
  author       = {Pith},
  title        = {Pith review of: Division-of-Thoughts: Harnessing Hybrid Language Model Synergy for Efficient On-Device Agents},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EW5OQL3K}},
  note         = {Machine review of arXiv:2502.04392}
}
read the original abstract

The rapid expansion of web content has made on-device AI assistants indispensable for helping users manage the increasing complexity of online tasks. The emergent reasoning ability in large language models offer a promising path for next-generation on-device AI agents. However, deploying full-scale Large Language Models (LLMs) on resource-limited local devices is challenging. In this paper, we propose Division-of-Thoughts (DoT), a collaborative reasoning framework leveraging the synergy between locally deployed Smaller-scale Language Models (SLMs) and cloud-based LLMs. DoT leverages a Task Decomposer to elicit the inherent planning abilities in language models to decompose user queries into smaller sub-tasks, which allows hybrid language models to fully exploit their respective strengths. Besides, DoT employs a Task Scheduler to analyze the pair-wise dependency of sub-tasks and create a dependency graph, facilitating parallel reasoning of sub-tasks and the identification of key steps. To allocate the appropriate model based on the difficulty of sub-tasks, DoT leverages a Plug-and-Play Adapter, which is an additional task head attached to the SLM that does not alter the SLM's parameters. To boost adapter's task allocation capability, we propose a self-reinforced training method that relies solely on task execution feedback. Extensive experiments on various benchmarks demonstrate that our DoT significantly reduces LLM costs while maintaining competitive reasoning accuracy. Specifically, DoT reduces the average reasoning time and API costs by 66.12% and 83.57%, while achieving comparable reasoning accuracy with the best baseline methods.

Figures

Figures reproduced from arXiv: 2502.04392 by the authors.

Figure 1
Figure 1. Overview of Our Proposed DoT Framework. Octopus—focused on efficiency and adaptability. Both approaches highlight the trend of edge-cloud collaboration to ensure powerful, low-latency AI experiences, though Apple leans more on cloud support while Octo-planner emphasizes on-device optimization. 3 Preliminaries Problem Definition Denote the local deployed SLM as MD, and the cloud-based LLM as MC. The user’s original q… view at source ↗
Figure 2
Figure 2. Advantage of “Division-and-Allocate” Strategy [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 4
Figure 4. Tree Search-Based Dataset Construction Process [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figures from the paper (2 more)
Figure 5
Figure 5. Figure 5: Proportion of SLMs in time cost and # sub-tasks. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Acc − Cost trade-off curves on 4 benchmarks. Moreover, We are surprised to discover that, without considering costs, using the cloud-based LLM exclusively can greatly improve the reasoning accuracy on the challenging MATH and CHAMP benchmarks, where accuracy increased …

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Federated Learning-Enabled Hybrid Language Models for Communication-Efficient Token Transmission

    cs.LG 2025-06 reject novelty 4.0 of 10

    FedHLM uses federated learning to learn token-level uncertainty thresholds that decide when to offload tokens from a small edge LM to a large cloud LM, claiming a 95 percent reduction in LLM transmissions.

  2. Get Experience from Practice: LLM Agents with Record & Replay

    cs.LG 2025-05 reject novelty 4.0 of 10

    AgentRR is a proposed paradigm that records agent traces, generalizes them into multi-level experiences, and replays them under safety checks to make LLM agents cheaper, faster, and more reliable.

Reference graph

Works this paper leans on

50 extracted references · 10 canonical work pages · cited by 2 Pith papers

  1. [1]

    Marah Abdin, Sam Ade Jacobs, Ammar Ahmad Awan, Jyoti Aneja, Ahmed Awadallah, Hany Awadalla, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Harki- rat Behl, et al. 2024. Phi-3 technical report: A highly capable language model locally on your phone. arXiv preprint arXiv:2404.14219 (2024)

    arXiv 2024

  2. [2]

    Maciej Besta, Nils Blach, Ales Kubicek, Robert Gerstenberger, Michal Podstawski, Lukas Gianinazzi, Joanna Gajda, Tomasz Lehmann, Hubert Niewiadomski, Piotr Nyczyk, et al. 2024. Graph of thoughts: Solving elaborate problems with large language models. In Proceedings of the AAAI Conference on Artificial Intelligence , Vol. 38. 17682–17690

    2024

  3. [3]

    Dong Chen, Yueting Zhuang, Shuo Zhang, Jinfeng Liu, Su Dong, and Siliang Tang. 2024. Data Shunt: Collaboration of Small and Large Models for Lower Costs and Better Performance. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 38. 11249–11257

    work page 2024

  4. [4]

    Lin Chen, Fengli Xu, Nian Li, Zhenyu Han, Meng Wang, Yong Li, and Pan Hui

  5. [5]

    Lingjiao Chen, Matei Zaharia, and James Zou. 2023. Frugalgpt: How to use large language models while reducing cost and improving performance. arXiv preprint arXiv:2305.05176 (2023)

    arXiv 2023

  6. [6]

    Wei Chen, Zhiyuan Li, Zhen Guo, and Yikang Shen. 2024. Octo-planner: On-device Language Model for Planner-Action Agents. arXiv preprint arXiv:2406.18082 (2024)

    work page Pith review arXiv 2024

  7. [7]

    Dickson KW Chiu, Yves TF Yueh, Ho-fung Leung, and Patrick CK Hung. 2009. Towards ubiquitous tourist service coordination and process integration: A collab- orative travel agent system architecture with semantic web services. Information Systems Frontiers 11 (2009), 241–256

    work page 2009

  8. [8]

    Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Zhiyong Wu, Baobao Chang, Xu Sun, Jingjing Xu, and Zhifang Sui. 2022. A survey on in-context learning. arXiv preprint arXiv:2301.00234 (2022)

    arXiv 2022

Show all 50 references
  1. [9]

    Dheeru Dua, Yizhong Wang, Pradeep Dasigi, Gabriel Stanovsky, Sameer Singh, and Matt Gardner. 2019. DROP: A reading comprehension benchmark requiring discrete reasoning over paragraphs. arXiv preprint arXiv:1903.00161 (2019)

    2019 arXiv

  2. [10]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783 (2024)

    2024 arXiv

  3. [11]

    Jiahui Gong, Jingtao Ding, Fanjin Meng, Guilong Chen, Hong Chen, Shen Zhao, Haisheng Lu, and Yong Li. 2024. A Population-to-individual Tuning Framework for Adapting Pretrained LM to On-device User Intent Prediction. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge...

    2024

  4. [12]

    Tom Gunter, Zirui Wang, Chong Wang, Ruoming Pang, Andy Narayanan, Aonan Zhang, Bowen Zhang, Chen Chen, Chung-Cheng Chiu, David Qiu, et al. 2024. Apple intelligence foundation language models. arXiv preprint arXiv:2407.21075 (2024)

    2024

  5. [13]

    Neha Gupta, Harikrishna Narasimhan, Wittawat Jitkrittum, Ankit Singh Rawat, Aditya Krishna Menon, and Sanjiv Kumar. 2024. Language Model Cascades: Token-level uncertainty and beyond. arXiv preprint arXiv:2404.10136 (2024)

    2024 arXiv

  6. [14]

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.03874 (2021)

    2021 arXiv

  7. [15]

    Ting Jiang, Shaohan Huang, Zhongzhi Luan, Deqing Wang, and Fuzhen Zhuang

  8. [16]

    Veton Kepuska and Gamal Bohouta. 2018. Next-generation of virtual personal assistants (microsoft cortana, apple siri, amazon alexa and google home). In 2018 IEEE 8th annual computing and communication workshop and conference (CCWC) . IEEE, 99–103

    2018

  9. [17]

    Tushar Khot, Harsh Trivedi, Matthew Finlayson, Yao Fu, Kyle Richardson, Peter Clark, and Ashish Sabharwal. 2022. Decomposed prompting: A modular approach for solving complex tasks. arXiv preprint arXiv:2210.02406 (2022)

    2022 arXiv

  10. [18]

    Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. Large language models are zero-shot reasoners. Advances in neural information processing systems 35 (2022), 22199–22213

    2022

  11. [19]

    Hanyu Lai, Xiao Liu, Iat Long Iong, Shuntian Yao, Yuxuan Chen, Pengbo Shen, Hao Yu, Hanchen Zhang, Xiaohan Zhang, Yuxiao Dong, et al. 2024. AutoWebGLM: A Large Language Model-based Web Navigating Agent. InProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and...

    2024

  12. [20]

    Brenden Lake and Marco Baroni. 2018. Generalization without systematicity: On the compositional skills of sequence-to-sequence recurrent networks. In International conference on machine learning . PMLR, 2873–2882

    2018

  13. [21]

    Songwei Li, Jie Feng, Jiawei Chi, Xinyuan Hu, Xiaomeng Zhao, and Fengli Xu

  14. [22]

    Yuanchun Li, Hao Wen, Weijun Wang, Xiangyu Li, Yizhen Yuan, Guohong Liu, Jiacheng Liu, Wenxing Xu, Xiang Wang, Yi Sun, et al. 2024. Personal llm agents: Insights and survey about the capability, efficiency and security. arXiv preprint arXiv:2401.05459 (2024)

    2024 arXiv

  15. [23]

    Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Wei-Ming Chen, Wei-Chen Wang, Guangxuan Xiao, Xingyu Dang, Chuang Gan, and Song Han. 2024. AWQ: Activation-aware Weight Quantization for On-Device LLM Compression and Acceleration. Proceedings of Machine Learning and Systems 6 (2...

    2024

  16. [24]

    arXiv preprint arXiv:2408.12832 (2024)

    Limp: Large language model enhanced intent-aware mobility prediction. arXiv preprint arXiv:2408.12832 (2024)

    2024 arXiv

  17. [25]

    Yujun Mao, Yoon Kim, and Yilun Zhou. 2024. CHAMP: A Competition-level Dataset for Fine-Grained Analyses of LLMs’ Mathematical Reasoning Capabilities. arXiv preprint arXiv:2401.06961 (2024)

    2024 arXiv

  18. [26]

    Alex Mari. 2019. Voice Commerce: Understanding shopping-related voice assis- tants and their effect on brands. (2019)

    2019

  19. [27]

    Zhenyan Lu, Xiang Li, Dongqi Cai, Rongjie Yi, Fangming Liu, Xiwen Zhang, Nicholas D Lane, and Mengwei Xu. 2024. Small Language Models: Survey, Measurements, and Insights. arXiv preprint arXiv:2409.15790 (2024)

    2024 arXiv

  20. [28]

    Guangtao Nie, Rong Zhi, Xiaofan Yan, Yufan Du, Xiangyang Zhang, Jianwei Chen, Mi Zhou, Hongshen Chen, Tianhao Li, Ziguang Cheng, et al . 2024. A Hybrid Multi-Agent Conversational Recommender System with LLM and Search Engine in E-commerce. In Proceedings of the 18th ACM Confer...

    2024

  21. [29]

    OpenAI. 2024. Hello GPT-4o. https://openai.com/index/hello-gpt-4o/

    2024

  22. [30]

    Meta. 2024. Introducing Meta Llama 3: The most capable openly available LLM to date. https://ai.meta.com/blog/meta-llama-3/

    2024

  23. [31]

    Yu Shang, Yu Li, Fengli Xu, and Yong Li. 2024. DefInt: A Default-interventionist Framework for Efficient Reasoning with Hybrid Large Language Models. arXiv preprint arXiv:2402.02563 (2024)

    2024 arXiv

  24. [32]

    Yu Shang, Yu Li, Keyu Zhao, Likai Ma, Jiahe Liu, Fengli Xu, and Yong Li. 2024. Agentsquare: Automatic llm agent search in modular design space.arXiv preprint arXiv:2410.06153 (2024)

    2024 arXiv

  25. [33]

    Tal Schuster, Ashwin Kalyan, Oleksandr Polozov, and Adam Tauman Kalai. 2021. Programming puzzles. arXiv preprint arXiv:2106.05784 (2021)

    2021 arXiv

  26. [34]

    Nikhil Sharma, Q Vera Liao, and Ziang Xiao. 2024. Generative Echo Chamber? Effect of LLM-Powered Search Systems on Diverse Information Seeking. In Proceedings of the CHI Conference on Human Factors in Computing Systems . 1–17

    2024

  27. [35]

    Chan Hee Song, Jiaman Wu, Clayton Washington, Brian M Sadler, Wei-Lun Chao, and Yu Su. 2023. Llm-planner: Few-shot grounded planning for embodied agents with large language models. In Proceedings of the IEEE/CVF International Conference on Computer Vision . 2998–3009

    2023

  28. [36]

    Chenyang Shao, Fengli Xu, Bingbing Fan, Jingtao Ding, Yuan Yuan, Meng Wang, and Yong Li. 2024. Beyond imitation: Generating human mobility from context- aware reasoning with large language models. arXiv preprint arXiv:2402.09836 (2024)

    2024 arXiv

  29. [37]

    Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, et al

  30. [38]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems 35 (2022), 24824–24837

    2022

  31. [39]

    Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. 2018. Com- monsenseqa: A question answering challenge targeting commonsense knowledge. arXiv preprint arXiv:1811.00937 (2018)

    2018 arXiv

  32. [40]

    Jiajun Xu, Zhiyuan Li, Wei Chen, Qun Wang, Xin Gao, Qi Cai, and Ziyuan Ling

  33. [41]

    Mengwei Xu, Wangsong Yin, Dongqi Cai, Rongjie Yi, Daliang Xu, Qipeng Wang, Bingyang Wu, Yihao Zhao, Chen Yang, Shihe Wang, et al . 2024. A survey of resource-efficient llm and multimodal foundation models. arXiv preprint arXiv:2401.08092 (2024)

    2024 arXiv

  34. [42]

    Shunyu Yao, Howard Chen, John Yang, and Karthik Narasimhan. 2022. Webshop: Towards scalable real-world web interaction with grounded language agents. Advances in Neural Information Processing Systems 35 (2022), 20744–20757

    2022

  35. [43]

    Fengli Xu, Qianyue Hao, Zefang Zong, Jingwei Wang, Yunke Zhang, Jingyi Wang, Xiaochong Lan, Jiahui Gong, Tianjian Ouyang, Fanjin Meng, et al. 2025. Towards Large Reasoning Models: A Survey of Reinforced Reasoning with Large Language Models. arXiv preprint arXiv:2501.09686 (2025)

    2025 arXiv

  36. [44]

    Denny Zhou, Nathanael Schärli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Claire Cui, Olivier Bousquet, Quoc Le, et al. 2022. Least-to- most prompting enables complex reasoning in large language models. arXiv preprint arXiv:2205.10625 (2022)

    2022 arXiv

  37. [45]

    arXiv preprint arXiv:2409.00088 (2024)

    On-device language models: A comprehensive review. arXiv preprint arXiv:2409.00088 (2024)

    2024 arXiv

  38. [48]

    Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L Griffiths, Yuan Cao, and Karthik Narasimhan. 2023. Tree of thoughts: Deliberate problem solving with large language models. arXiv preprint arXiv:2305.10601 (2023)

    2023 arXiv

  39. [50]

    jump" or

    Zhilun Zhou, Jingyang Fan, Yu Liu, Fengli Xu, Depeng Jin, and Yong Li. 2024. Synergizing llm agents and knowledge graph for socioeconomic prediction in lbsn. arXiv preprint arXiv:2411.00028 (2024). WWW ’25, April 28-May 2, 2025, Sydney, NSW, Australia Chenyang Shao, Xinyuan Hu...

    2024 arXiv

  40. [2022]

    Emergent abilities of large language models.arXiv preprint arXiv:2206.07682 (2022)

    2022 arXiv

  41. [2023]

    arXiv preprint arXiv:2307.16645 (2023)

    Scaling sentence embeddings with large language models. arXiv preprint arXiv:2307.16645 (2023)

    2023 arXiv

  42. [2024]

    In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining

    Large language model-driven meta-structure discovery in heterogeneous information network. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining . 307–318

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.