Pith. sign in

REVIEW 22 cited by

Chameleon: Plug-and-Play Compositional Reasoning with Large Language Models

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2304.09842 v3 pith:6T57ZI3K submitted 2023-04-19 cs.CL cs.AIcs.CVcs.LG

Pan Lu , Baolin Peng , Hao Cheng , Michel Galley , Kai-Wei Chang , Ying Nian Wu , Song-Chun Zhu , Jianfeng Gao This is my paper
classification cs.CLcs.AIcs.CVcs.LG
keywords chameleonreasoningllmslanguagemodelsplannertaskstools
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Large language models (LLMs) have achieved remarkable progress in solving various natural language processing tasks due to emergent reasoning abilities. However, LLMs have inherent limitations as they are incapable of accessing up-to-date information (stored on the Web or in task-specific knowledge bases), using external tools, and performing precise mathematical and logical reasoning. In this paper, we present Chameleon, an AI system that mitigates these limitations by augmenting LLMs with plug-and-play modules for compositional reasoning. Chameleon synthesizes programs by composing various tools (e.g., LLMs, off-the-shelf vision models, web search engines, Python functions, and heuristic-based modules) for accomplishing complex reasoning tasks. At the heart of Chameleon is an LLM-based planner that assembles a sequence of tools to execute to generate the final response. We showcase the effectiveness of Chameleon on two multi-modal knowledge-intensive reasoning tasks: ScienceQA and TabMWP. Chameleon, powered by GPT-4, achieves an 86.54% overall accuracy on ScienceQA, improving the best published few-shot result by 11.37%. On TabMWP, GPT-4-powered Chameleon improves the accuracy by 17.0%, lifting the state of the art to 98.78%. Our analysis also shows that the GPT-4-powered planner exhibits more consistent and rational tool selection via inferring potential constraints from instructions, compared to a ChatGPT-powered planner. The project is available at https://chameleon-llm.github.io.

Discussion (0). Sign in to comment.

Forward citations

Cited by 22 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Harnessing Embodied Agents: Runtime Governance for Policy-Constrained Execution

    cs.RO 2026-04 unverdicted novelty 7.0 of 10

    A runtime governance framework for embodied agents achieves 96.2% interception of unauthorized actions and 91.4% recovery success in 1000 simulation trials by externalizing policy enforcement.

  2. Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation

    cs.AI 2025-03 conditional novelty 7.0 of 10

    Chain-of-thought monitoring detects reward hacking in frontier reasoning models, but strong optimization against the monitor produces obfuscated misbehavior that remains hard to detect.

  3. Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads

    cs.LG 2024-01 conditional novelty 7.0 of 10

    Medusa augments LLMs with multiple decoding heads and tree-based attention to predict and verify several tokens in parallel, yielding 2.2-3.6x inference speedup via two fine-tuning regimes.

  4. BVS: Bayesian Visual Search with Multimodal Large Language Model for Fine-grained Perception

    cs.CV 2026-07 conditional novelty 6.0 of 10

    BVS combines early-stop attention rollout priors with a scale-aware non-stationary kernel and GP-UCB to locate tiny objects in UHR images more accurately and with fewer MLLM queries than prior visual-search methods.

  5. E-VLA: Event-Augmented Vision-Language-Action Model for Dark and Blurred Scenes

    cs.CV 2026-04 conditional novelty 6.0 of 10

    E-VLA integrates event streams directly into VLA models via lightweight fusion, raising Pick-Place success from 0% to 60-90% at 20 lux and from 0% to 20-25% under severe motion blur.

  6. F1: A Vision-Language-Action Model Bridging Understanding and Generation to Actions

    cs.RO 2025-09 unverdicted novelty 6.0 of 10

    F1 integrates next-scale visual foresight prediction into a Mixture-of-Transformer VLA architecture to reformulate action generation as foresight-guided inverse dynamics, achieving higher success rates on 136 tasks.

  7. What to Say and When to Say it: Live Fitness Coaching as a Testbed for Situated Interaction

    cs.CV 2024-07 unverdicted novelty 6.0 of 10

    Introduces the QEVD benchmark for asynchronous situated interaction in fitness coaching and proposes a streaming baseline to address limitations of existing vision-language models.

  8. LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code

    cs.SE 2024-03 unverdicted novelty 6.0 of 10

    LiveCodeBench collects 400 recent contest problems to create a contamination-free benchmark evaluating LLMs on code generation and related capabilities like self-repair and execution.

  9. ProLaViT: Learning Progressive Latent Visual Thoughts in Structured Latent Space

    cs.CV 2026-07 conditional novelty 5.5 of 10

    Progressive multi-step latent visual thoughts, endogenously distilled from a model's own encoder on synthetic trajectories and regularized by distance-weighted diversity, improve MLLM visual reasoning accuracy and efficiency.

  10. Harnessing Embodied Agents: Runtime Governance for Policy-Constrained Execution

    cs.RO 2026-04 conditional novelty 5.5 of 10

    An external runtime governance layer for embodied agents intercepts unauthorized actions at ~96% and recovers from runtime drift at ~91% under policy constraints in simulation, outperforming pre-execution-only baselin...

  11. Cut Your Losses! Learning to Prune Paths Early for Efficient Parallel Reasoning

    cs.CL 2026-04 unverdicted novelty 5.0 of 10

    STOP, a learnable internal super-token pruner, is claimed to raise parallel-reasoning accuracy and efficiency on LRMs from 1.5B to 20B, e.g. AIME25 from 84% to ~90% under fixed compute.

  12. Cut Your Losses! Learning to Prune Paths Early for Efficient Parallel Reasoning

    cs.CL 2026-04 unverdicted novelty 5.0 of 10

    STOP is a new learnable internal path-pruning technique that improves efficiency and accuracy of parallel reasoning in LRMs under fixed compute budgets.

  13. Harnessing Embodied Agents: Runtime Governance for Policy-Constrained Execution

    cs.RO 2026-04 unverdicted novelty 5.0 of 10

    A runtime governance framework for embodied agents intercepts 96.2% of unauthorized actions and achieves 91.4% recovery success in 1000 simulation trials while outperforming baselines.

  14. Less Detail, Better Answers: Degradation-Driven Prompting for VQA

    cs.CV 2026-04 unverdicted novelty 5.0 of 10

    Degradation-Driven Prompting improves VQA by intentionally reducing image detail and using masks, lines, and examples to guide models toward essential structures.

  15. E-VLA: Event-Augmented Vision-Language-Action Model for Dark and Blurred Scenes

    cs.CV 2026-04 conditional novelty 5.0 of 10

    Degradation-Driven Prompting improves VLM accuracy on deceptive visual tasks by downsampling inputs and routing them through classification, external visual tools, and a critic.

  16. LLaMA-Adapter V2: Parameter-Efficient Visual Instruction Model

    cs.CV 2023-04 conditional novelty 5.0 of 10

    LLaMA-Adapter V2 achieves open-ended visual instruction following in LLMs by unlocking more parameters, early fusion of visual tokens, and joint training on disjoint parameter groups with only 14M added parameters.

  17. Microskill Architecture: A Modular Skill-Driven Framework for AI-Native Code Generation

    cs.SE 2026-06 unverdicted novelty 4.0 of 10

    MicroSkill Architecture partitions knowledge into atomic skill capsules selected via constrained optimization to cut token use over 90% and improve code generation metrics in one enterprise case study.

  18. Interfaze: The Future of AI is built on Task-Specific Small Models

    cs.AI 2026-02 reject novelty 4.0 of 10

    Interfaze-Beta uses small specialist models and tools to build a compact context that a general-purpose LLM answers from, reporting competitive benchmark scores without reproducible evidence.

  19. The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)

    cs.CV 2023-09 conditional novelty 4.0 of 10

    GPT-4V processes interleaved image-text inputs generically and supports visual referring prompting for new human-AI interaction.

  20. The Rise and Potential of Large Language Model Based Agents: A Survey

    cs.AI 2023-09 accept novelty 4.0 of 10

    The paper surveys the origins, frameworks, applications, and open challenges of AI agents built on large language models.

  21. A Survey on Multimodal Large Language Models

    cs.CV 2023-06 accept novelty 3.0 of 10

    This survey organizes the architectures, training strategies, data, evaluation methods, extensions, and challenges of Multimodal Large Language Models.

  22. A Comprehensive Overview of Large Language Models

    cs.CL 2023-07 unverdicted novelty 2.0 of 10

    A survey paper providing an overview of Large Language Models, their background, and recent advances in the field.

Pith tools