Pith. sign in

REVIEW 3 cited by

Cascade-Aware Training of Language Models

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2406.00060 v1 pith:7A7JGKLH submitted 2024-05-29 cs.CL cs.LG

Congchao Wang , Sean Augenstein , Keith Rush , Wittawat Jitkrittum , Harikrishna Narasimhan , Ankit Singh Rawat , Aditya Krishna Menon , Alec Go This is my paper · ORCID
classification cs.CLcs.LG
keywords modelstrainingcascadecascade-awarecascadedinference-timelanguageachieve
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Reducing serving cost and latency is a fundamental concern for the deployment of language models (LMs) in business applications. To address this, cascades of LMs offer an effective solution that conditionally employ smaller models for simpler queries. Cascaded systems are typically built with independently trained models, neglecting the advantages of considering inference-time interactions of the cascaded LMs during training. In this paper, we present cascade-aware training(CAT), an approach to optimizing the overall quality-cost performance tradeoff of a cascade of LMs. We achieve inference-time benefits by training the small LM with awareness of its place in a cascade and downstream capabilities. We demonstrate the value of the proposed method with over 60 LM tasks of the SuperGLUE, WMT22, and FLAN2021 datasets.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Stable Curves, Unstable Items: Item-Level Scaling Heterogeneity in Video LLMs

    cs.CV 2026-08 conditional novelty 7.0 of 10

    Across five video LLMs, 12.5-25.5% of multiple-choice items are correct at a lower visual budget but wrong at a higher one, even when aggregate scaling curves look safe.

  2. Active Data Curation Effectively Distills Large-Scale Multimodal Models

    cs.CV 2024-11 conditional novelty 7.0 of 10

    Selecting training data by a reference model's loss acts as an implicit distillation, and combining it with explicit distillation yields more FLOP-efficient vision-language models that beat prior SoTA on 27 benchmarks.

  3. Universal Model Routing for Efficient LLM Inference

    cs.CL 2025-02 conditional novelty 5.0 of 10

    UniRoute represents each language model by its error rates on a few prompt clusters, letting a router choose among models it has never seen during training.

Pith tools