Pith. sign in

REVIEW 4 major objections 4 minor 47 references

Prompt optimization alone can lift compact language models to within 0.3 points of a 27B router on guarded query routing.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 08:30 UTC pith:TJTYS477

load-bearing objection Useful empirical paper with one solid qualitative finding—few-shot prompts repair ID-commitment failures—but the single-run design can't support the 0.27-point near-parity claim or the precise strategy rankings. the 4 major comments →

arxiv 2607.24801 v1 pith:TJTYS477 submitted 2026-07-06 cs.IR cs.CL

Influence of Prompt Engineering on Small Language Models for Guarded Query Routing

Richard \v{S}l\'eher , William Brach , Kristi\'an Ko\v{s}\v{t}\'al , Lukas Galke Poech This is my paper
classification cs.IR cs.CL
keywords guarded query routingsmall language modelsprompt optimizationfew-shot promptingout-of-distribution detectionselective classificationDSPyGQR-Bench
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Guarded query routing asks a model to do two things at once: send in-scope queries to the right expert (law, finance, or healthcare) and reject everything else. The paper tries to establish that compact open-weight language models can handle this joint decision well enough for practical use, provided the prompt teaches them the output format. Across 22 models on GQR-Bench, most small models reject out-of-scope input reliably but hesitate on valid in-scope queries, a format-following failure rather than missing knowledge. Prompt-only optimization, especially few-shot exemplars, repairs much of that failure: Mistral 7B rises from 81.79 to 90.87 GQR-Score, and a 9B model reaches 95.74, within 0.3 points of the best 27B model. The practical payoff is that a router's accuracy can be improved or adapted without retraining or replacing weights.

Core claim

The paper's central claim is that the main obstacle for small language models as guarded routers is not detecting out-of-distribution queries but committing to one of the allowed in-distribution labels under a strict output schema. Granite 4 Tiny rejects OOD input 99.75% of the time yet labels only 37.29% of valid queries correctly under a standard prompt; adding a bare structured prompt signature raises its GQR-Score from 54.29 to 83.05, and few-shot exemplars recover ID accuracy to 94.88% at some OOD cost. The best optimized result is Qwen3.5 9B at 95.74 under few-shot prompting, 0.27 points below the strongest unoptimized model, Gemma 3 27B at 96.01, while running at lower latency. Prompt

What carries the argument

The load-bearing mechanism is the GQR-Score, the harmonic mean of ID accuracy (macro-averaged over Law, Finance, Healthcare) and OOD accuracy (size-weighted over seven rejection splits), which forces a router to do both jobs. The other central object is the prompt-optimization pipeline built on the DSPy signature: a structured four-label output field (law/finance/healthcare/ood) plus, in the few-shot variant, automatically selected in-context exemplars from 30 training samples. The exemplars demonstrate the exact expected output format, and that demonstration, not new domain knowledge, is what repairs the partial-commit failure. Evolutionary refinement of the instruction is also tested but n

Load-bearing premise

The conclusions rest on GQR-Bench's schema—three in-distribution domains and one catch-all 'ood' label—faithfully capturing real guarded-routing deployments, and on the harmonic GQR-Score being the right objective; with finer label granularity, hierarchical abstention, or different cost asymmetries, the prompt-only gains may not transfer.

What would settle it

Take a model with the documented high-OOD/low-ID profile (e.g., Mistral 7B or Granite 4 Tiny) and evaluate it on a new, held-out domain with the same four-label schema but different surface vocabulary. If few-shot exemplars from the original 30 training samples fail to restore ID accuracy, or if a model with high OOD and low ID never recovers under any prompt variant, the claim that the failure is prompt-repairable format-following would be falsified. A simpler check: re-run the reported numbers over multiple random seeds; the paper reports single runs, so if run-to-run variance exceeds the 0.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Deployment teams can treat prompt optimization as a first repair step for compact routers: several 2B–9B models reach 90+ GQR-Score without weight updates.
  • Optimized prompts increase latency roughly 2–6x over standard prompts, so the accuracy gain must be weighed against per-request latency.
  • Models around 0.8B and below appear too fragile for reliable four-way routing even with exemplars, so they likely need lightweight weight-level adaptation rather than more prompting.
  • Adding exemplars to an already-balanced router is not safe by default: Qwen3.5 2B drops from 90.58 to 70.43 GQR-Score under few-shot prompting because OOD rejection collapses.
  • The near-saturation of top scores suggests GQR-Bench may not separate the strongest routers; harder or finer-grained splits would be needed to rank them.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A cheap screening test for any compact router follows from this work: measure ID accuracy under a strict one-token-output schema separately from OOD accuracy; a high-OOD/low-ID profile is a format-following problem that prompting may fix, while low OOD accuracy is a different failure.
  • The exemplar-based format demonstration likely transfers to other low-cardinality structured-output tasks, such as guardrail classification or intent tagging with a reject class, because the bottleneck is generic instruction following rather than routing-specific knowledge.
  • A testable extension is to fine-tune the smallest models with lightweight adapters on the optimized few-shot prompts; if the format-following bottleneck is real, weight-level binding should recover ID accuracy without the OOD suppression seen under prompting alone.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper studies whether small open-weight language models can serve as guarded query routers that both dispatch in-distribution queries (Law, Finance, Healthcare) and reject out-of-distribution inputs under a four-label schema. Using GQR-Bench and the harmonic GQR-Score from the authors' prior work [38], the authors benchmark 22 open-weight models under standard prompting and then apply three DSPy-based prompt-optimization strategies (bare DSPy signature, Few-Shot BootstrapFewShotWithRandomSearch, and GEPA with gpt-5.4 as teacher) to seven selected models. The central finding is that many compact models over-reject: they have high OOD accuracy but low ID accuracy, and prompt optimization, especially Few-Shot, repairs much of this ID-commitment failure without weight updates. Headline numbers include Mistral 7B rising from 81.79 to 90.87 GQR-Score, Granite 4 Tiny rising from 54.29 to 83.05 under the DSPy baseline, and Qwen3.5 9B reaching 95.74 under Few-Shot, which the paper describes as 'within 0.3 points' of the best unoptimized model, Gemma 3 27B at 96.01. The paper concludes that prompt-only optimization is a useful first step for guarded routing but notes that weaker models may still need weight-level adaptation.

Significance. If the headline quantitative claims were supported by appropriate uncertainty quantification, the paper would make a useful empirical contribution: it identifies a specific, actionable failure mode—format-constrained ID under-commitment—and shows that in-context exemplars can partially repair it in several model families. The strength of the study is its breadth (22 open-weight models, multiple families) and its decomposition of performance into ID and OOD components, which goes beyond a single aggregate score. The qualitative ID-repair mechanism is plausible and is supported by large gains (e.g., +24.4 GQR for Granite 4 Tiny, +9.1 for Mistral 7B), so the core insight is likely robust. However, the paper's most prominent quantitative claims—near-parity with Gemma 3 27B and the 'best optimized score' ranking—rest on single-run point estimates with no variance characterization and on candidate selection over a small validation slice. The GEPA comparison is also confounded by the use of a frontier teacher model and a different sample budget. These issues do not invalidate the qualitative mechanism but they do undermine the precision of the headline claims as currently worded.

major comments (4)
  1. [Section 3.2 / Section 7 / Abstract] The manuscript explicitly states in Section 3.2 that 'All results come from a single run' and in Section 7 that 'no run-to-run variance is characterized,' yet the abstract and Section 5 assert that Qwen3.5 9B Few-Shot (95.74) is 'within 0.3 points' of Gemma 3 27B (96.01) and is the 'best optimized score.' BootstrapFewShotWithRandomSearch (Section 3.1) selects the single best of num_candidate_programs=4 on a small held-out validation slice, so the 0.27-point gap and the rankings among strategies separated by less than 1 GQR point (e.g., Qwen3.5 4B: standard 92.97 vs Few-Shot 92.51 vs GEPA 91.72) are not established at the reported precision. The ID-repair finding is large enough to be robust, but the near-parity and 'best optimized score' claims need variance estimates (e.g., bootstrap confidence intervals or multiple optimization runs) or must be reworded as point estimates.
  2. [Section 3.1 / Section 4.2 (GEPA)] The GEPA comparison is not apples-to-apples with the other strategies. GEPA uses gpt-5.4 as a teacher model, is initialized from the Few-Shot prompt, and is run with 100 training and 30 validation samples, whereas Few-Shot uses no teacher and 30 training samples. Therefore the conclusion that 'GEPA does not yield the best score for any model' and the claim that the instruction-optimization landscape is brittle conflate the optimizer itself with the teacher model, the initialization, and the sample budget. The negative result should be presented as 'GEPA with a gpt-5.4 teacher under this budget,' or the comparison should be controlled by using the same teacher/budget across strategies.
  3. [Section 3.1 / Section 5] The seven models selected for prompt optimization are chosen from the range 'where over-rejection under standard prompting is most pronounced.' This selection maximizes the apparent benefit of prompt optimization and prevents the paper from estimating how often prompt-only optimization helps or hurts across the broader pool of 22 models. The general conclusion that 'prompt optimization is a useful first step for guarded query routing' should be explicitly scoped to models exhibiting the partial-commit failure mode; the Qwen3.5 2B result (standard 90.58 vs Few-Shot 70.43) shows the effect is not uniformly positive. As written, the title and abstract over-generalize from a deliberately biased subset.
  4. [Section 3 / Section 7] The evaluation relies entirely on GQR-Bench and the GQR-Score from the authors' prior paper [38]. While the underlying datasets are public and the central mechanism (format-following failures rather than missing domain knowledge) is unlikely to be a pure artifact of the metric definition, the absence of any external benchmark or sensitivity analysis means the practical recommendations remain conditional on the four-label GQR-Bench schema. Section 7 acknowledges this ('tied to the GQR-Bench schema'), but the abstract and conclusion state general conclusions about guarded query routing. Please either add a secondary analysis with a different label granularity or cost asymmetry, or consistently scope the claims to the tested schema.
minor comments (4)
  1. [Section 3.2] The text says models are served with 'vLLM [27]', but reference [27] is the Ollama citation. Either the citation or the inference server name is wrong; the vLLM reference should be corrected.
  2. [Table 1] Table 1's header is garbled in places ('W eb Q ML QUnsafe A vg.'), and the column alignment makes the table difficult to read. Please regenerate the table with clean column separators and consistent model names.
  3. [Throughout] The term 'SLM' is used loosely: the model pool includes Llama 3.3 70B, Gemma 3 27B, and GPT-OSS 20B, which are not 'small' by most definitions. The paper distinguishes 'mid-scale' in places, but the abstract and introduction should clarify the parameter range covered and avoid conflating small and mid-scale models.
  4. [Section 4.2] The sentence about Qwen3.5 0.8B under GEPA is hard to parse: 'GEPA actually underperforms both Few-Shot (67.79) and the standard-prompting baseline (60.35), reaching only 48.92.' Since standard is 60.35, the phrase 'both Few-Shot and the standard-prompting baseline' is correct but reads as if standard were below 48.92; consider 'underperforms the Few-Shot result (67.79) and even the standard-prompting baseline (60.35), reaching only 48.92.'

Circularity Check

0 steps flagged

No significant circularity: prompt-optimization claims rest on held-out test evaluation; the only self-citation is to the authors' own GQR-Bench, which is a public-dataset artifact rather than a fitted target.

full rationale

The paper's derivation chain is not circular. The benchmark and headline metric come from the authors' prior ECAI paper [38], cited in Section 3 ('We run all evaluations on GQR-Bench [38]' and 'GQR-Score, the harmonic mean of ID and OOD accuracy, is our headline metric'), but this is a self-citation to an external artifact assembled from ten public datasets, not to the target result. The central empirical claim—that few-shot prompt optimization repairs ID-commitment failures—is established by comparing standard prompting to optimized prompts on the same test evaluation protocol; no fitted quantity is renamed as a prediction. The optimizer does select one of four candidate programs on a 30-sample validation slice, and all results are single-run point estimates, which is a statistical robustness limitation (acknowledged in Section 7: 'All results are single-run point estimates on a fixed two-GPU configuration; no run-to-run variance is characterized') but not a definitional reduction. The harmonic GQR-Score is more sensitive to the lower of ID/OOD accuracy, but the paper separately reports both components, so the ID-repair mechanism is measured rather than assumed by construction. The Section 7 caveat that the analysis is 'tied to the GQR-Bench schema' is a scope limitation, not evidence of circularity. The self-citation to [38] is therefore not load-bearing in the sense that would raise the circularity score beyond the minor-self-citation range.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 0 invented entities

The paper's contribution is purely empirical. Its load-bearing assumptions are the benchmark's validity, the metric choice, and the stability of single-run measurements; the only hand-tuned values are DSPy hyperparameters. These are disclosed in the text.

free parameters (4)
  • max_labeled_demos = 6
    Hand-chosen in Section 3.1, below DSPy default of 16; controls exemplar count and hence both accuracy and latency. Different values could change which strategy wins.
  • num_candidate_programs = 4
    Search budget for BootstrapFewShotWithRandomSearch; a larger budget could yield a different best prompt.
  • Few-Shot training sample count = 30
    Number of examples used to bootstrap demonstrations; small sample may limit generalizability of the optimized prompt.
  • GEPA training/validation counts = 100/30
    GEPA light preset on 100 training and 30 validation samples; these choices affect evolutionary search outcome.
axioms (5)
  • domain assumption GQR-Bench is a valid proxy for guarded query routing
    All conclusions are measured on this benchmark; Section 3 defines the task. The paper itself notes in Section 7 that other schemas could change the balance.
  • domain assumption The harmonic GQR-Score is the appropriate objective
    Headline metric; a router that optimizes harmonic mean may not match deployment cost asymmetries.
  • domain assumption Single-run greedy-decoding measurements are stable enough to rank models
    Section 3.2 states results come from a single run; Section 7 concedes no run-to-run variance is characterized.
  • domain assumption Public datasets in GQR-Bench represent OOD and unsafe content
    Seven OOD splits from Jigsaw, OLID, etc. are assumed to cover relevant out-of-distribution queries.
  • domain assumption vLLM default quantization does not alter relative model rankings
    Serving setup in Section 3.2 uses default quantization for each model tag; this is not controlled across models.

pith-pipeline@v1.3.0-alltime-deepseek · 12954 in / 10644 out tokens · 107065 ms · 2026-08-02T08:30:25.491818+00:00 · methodology

0 comments
read the original abstract

We study the problem of guarded query routing, where we assume that a user query first meets a router that either determines the ideal endpoint for in-distribution queries or rejects out-of-distribution queries that are potentially unsafe or out of the system's scope. We investigate whether compact open-weight Small Language Models (SLMs) can jointly handle both tasks under latency constraints. We evaluate 22 models on GQR-Bench and score them with the harmonic mean of in-distribution and out-of-distribution accuracy. We find that mid-scale SLMs come close to frontier model routing quality at much lower latency. Still, many compact models fail because they do not reliably follow the required output format. However, our results show that prompt optimization techniques enable SLMs to handle such cases gracefully, without changing the models' weights. Moreover, few-shot prompt optimization raises Mistral 7B from 81.79 to 90.87 GQR-Score and lifts Qwen3.5 9B to 95.74, the best optimized score in our study and within 0.3 points of the strongest unoptimized larger model: Gemma 3 27B at 96.01. The bare DSPy signature, without in-context exemplars, is the most effective strategy for Granite 4 Tiny, raising its score from 54.29 to 83.05. These results show that prompt optimization is a useful first step for guarded query routing, while weaker models may still need weight-level adaptation or schema-aware training

Figures

Figures reproduced from arXiv: 2607.24801 by Kristi\'an Ko\v{s}\v{t}\'al, Lukas Galke Poech, Richard \v{S}l\'eher, William Brach.

Figure 1
Figure 1. Figure 1: Inference latency vs. GQR-Score trade-off of evaluated open-weight mod [PITH_FULL_IMAGE:figures/full_fig_p007_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Decomposition of GQR-Score gains from prompt optimization. The left [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Average router inference latency by prompting strategy. Few-Shot in [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

47 extracted references · 2 canonical work pages

  1. [1]

    Abdin, M., et al.: Phi-4 technical report (2024), https://arxiv.org/pdf/2412.08905.pdf

    Pith/arXiv arXiv 2024

  2. [2]

    Agrawal, A., et al.: GEPA: Reflective prompt evolution can outperform reinforce- ment learning (2025), https://arxiv.org/pdf/2507.19457.pdf

    Pith/arXiv arXiv 2025

  3. [3]

    Transactions of the association for computational linguistics 5, 135–146 (2017)

    Bojanowski, P., Grave, E., Joulin, A., Mikolov, T.: Enriching word vectors with subword information. Transactions of the association for computational linguistics 5, 135–146 (2017)

    2017

  4. [4]

    In: Compan- ion Proceedings of The 2019 World Wide Web Conference (WWW)

    Borkan, D., Dixon, L., Sorensen, J., Thain, N., Vasserman, L.: Nuanced metrics for measuring unintended bias with real data for text classification. In: Compan- ion Proceedings of The 2019 World Wide Web Conference (WWW). pp. 491–500 (2019). https://doi.org/10.1145/3308560.3317593

    arXiv 2019

  5. [5]

    In: Advances in Neural Information Processing Systems (NeurIPS) (2020)

    Brown, T.B., et al.: Language models are few-shot learners. In: Advances in Neural Information Processing Systems (NeurIPS) (2020)

    2020

  6. [6]

    Transactions on Machine Learning Re- search (2023)

    Chen, L., Zaharia, M., Zou, J.: Frugalgpt: How to use large language models while reducing cost and improving performance. Transactions on Machine Learning Re- search (2023)

    2023

  7. [7]

    arXiv preprint arXiv:2410.10347 (2024)

    Dekoninck, J., Baader, M., Vechev, M.: A unified approach to routing and cascad- ing for llms. arXiv preprint arXiv:2410.10347 (2024)

    Pith/arXiv arXiv 2024

  8. [8]

    In: ICML

    Dekoninck, J., Baader, M., Vechev, M.: A unified approach to routing and cascad- ing for LLMs. In: ICML. pp. 12987–13010 (2025)

    2025

  9. [9]

    arXiv preprint arXiv:2204.03954 (2022)

    Galke, L., Diera, A., Lin, B.X., Khera, B., Meuser, T., Singhal, T., Karl, F., Scherp, A.:Arewereallymakingmuchprogressintextclassification?acomparativereview. arXiv preprint arXiv:2204.03954 (2022)

    Pith/arXiv arXiv 2022

  10. [10]

    graph vs

    Galke, L., Scherp, A.: Bag-of-words vs. graph vs. sequence in text classification: Questioningthenecessityoftext-graphsandthesurprisingstrengthofawideMLP. In: ACL. pp. 4038–4051 (2022). https://doi.org/10.18653/v1/2022.acl-long.279

    work page doi:10.18653/v1/2022.acl-long.279 2022

  11. [11]

    Gemma Team: Gemma 3 technical report (2025), https://arxiv.org/pdf/2503.19786.pdf 14 Šléher et al

    Pith/arXiv arXiv 2025

  12. [12]

    Grattafiori, A., et al.: The llama 3 herd of models (2024), https://arxiv.org/pdf/2407.21783.pdf

    Pith/arXiv arXiv 2024

  13. [13]

    In: ICLR (2017)

    Hendrycks, D., Gimpel, K.: A baseline for detecting misclassified and out-of- distribution examples in neural networks. In: ICLR (2017)

    2017

  14. [14]

    In: Proceedings of the International Conference on Emerging eLearning Technologies and Applications (ICETA)

    Hládek, D., Staš, J., Pleva, M., Bobrýek, Y.: Slovak dataset for hate speech de- tection. In: Proceedings of the International Conference on Emerging eLearning Technologies and Applications (ICETA). IEEE (2023)

    2023

  15. [15]

    IBM Granite Team: Granite 3.3 language models (2025), https://huggingface.co/ibm-granite/granite-3.3-8b-instruct

    2025

  16. [16]

    Inan, H., et al.: Llama guard: Llm-based input-output safeguard for human-ai conversations (2023), https://arxiv.org/abs/2312.06674

    Pith/arXiv arXiv 2023

  17. [17]

    Jiang, A.Q., et al.: Mistral 7b (2023), https://arxiv.org/abs/2310.06825

    Pith/arXiv arXiv 2023

  18. [18]

    In: EACL: Short papers

    Joulin, A., Grave, E., Bojanowski, P., Mikolov, T.: Bag of tricks for efficient text classification. In: EACL: Short papers. pp. 427–431 (2017)

    2017

  19. [19]

    Jung, M.C., Zhao, H., Dipnall, J., Gabbe, B., Du, L.: Enhancing near out-of- distributiondetectioninpromptlearning:Maximumgains,minimalcosts.In:ICLR (2025)

    2025

  20. [20]

    In: ICLR (2024)

    Khattab, O., et al.: DSPy: Compiling declarative language model calls into state- of-the-art pipelines. In: ICLR (2024)

    2024

  21. [21]

    In: ICLR (2025)

    Lakha, J., Yu, M., Shahout, R.: Faster, cheaper, just as good: Cost- and latency- constrained routing for llms. In: ICLR (2025)

    2025

  22. [22]

    In: EMNLP

    Lester, B., Al-Rfou, R., Constant, N.: The power of scale for parameter-efficient prompt tuning. In: EMNLP. pp. 3045–3059 (2021)

    2021

  23. [23]

    In: ICLR (2018)

    Liang, S., Li, Y., Srikant, R.: Enhancing the reliability of out-of-distribution image detection in neural networks. In: ICLR (2018)

    2018

  24. [24]

    In: Advances in Neural Information Processing Systems (NeurIPS) (2020)

    Liu, W., Wang, X., Owens, J.D., Li, Y.: Energy-based out-of-distribution detection. In: Advances in Neural Information Processing Systems (NeurIPS) (2020)

    2020

  25. [25]

    In: Advances in Neural Information Processing Systems (NeurIPS) (2023)

    Madaan, A., et al.: Self-refine: Iterative refinement with self-feedback. In: Advances in Neural Information Processing Systems (NeurIPS) (2023)

    2023

  26. [26]

    In: Proceed- ings of the AAAI Conference on Artificial Intelligence

    Mathew, B., Saha, P., Yimam, S.M., Biemann, C., Goyal, P., Mukherjee, A.: Hat- eXplain: A benchmark dataset for explainable hate speech detection. In: Proceed- ings of the AAAI Conference on Artificial Intelligence. vol. 35, pp. 14867–14875 (2021)

    2021

  27. [27]

    Ollama: Ollama (2025), https://github.com/ollama/ollama

    2025

  28. [28]

    Ong, I., Almahairi, A., Wu, V., Chiang, W.L., Wu, T., Gonzalez, J.E., Kadous, M.W., Stoica, I.: RouteLLM: Learning to route LLMs with preference data (2024), https://arxiv.org/abs/2406.18665

    Pith/arXiv arXiv 2024

  29. [29]

    Agarwal and others: gpt-oss-120b & gpt-oss-20b model card (2025), https://arxiv.org/abs/2508.10925

    OpenAI and S. Agarwal and others: gpt-oss-120b & gpt-oss-20b model card (2025), https://arxiv.org/abs/2508.10925

    Pith/arXiv arXiv 2025

  30. [30]

    In: EMNLP (2024)

    Opsahl-Ong, K., Ryan, M.J., Purtell, J., Broman, D., Potts, C., Zaharia, M., Khattab, O.: Optimizing instructions and demonstrations for multi-stage language model programs. In: EMNLP (2024)

    2024

  31. [31]

    arXiv preprint arXiv:2510.08439 (2025)

    Qian, et al.: xrouter: Training cost-aware llms orchestration system via reinforce- ment learning. arXiv preprint arXiv:2510.08439 (2025)

    arXiv 2025

  32. [32]

    In: EMNLP: System Demonstrations (2023)

    Rebedea, T., Dinu, R., Sreedhar, M., Parisien, C., Cohen, J.: NeMo Guardrails: A toolkit for controllable and safe LLM applications with programmable rails. In: EMNLP: System Demonstrations (2023)

    2023

  33. [33]

    In: EACL (2021)

    Schick, T., Schütze, H.: Exploiting cloze questions for few-shot text classification and natural language inference. In: EACL (2021)

    2021

  34. [34]

    Schulhoff, S., et al.: The prompt report: A systematic survey of prompting tech- niques (2024), https://arxiv.org/pdf/2406.06608.pdf Small Language Models for Guarded Query Routing 15

    Pith/arXiv arXiv 2024

  35. [35]

    In: EMNLP (2020)

    Shin, T., Razeghi, Y., IV, R.L.L., Wallace, E., Singh, S.: Autoprompt: Elicit- ing knowledge from language models with automatically generated prompts. In: EMNLP (2020)

    2020

  36. [36]

    In: CoLM (2024)

    Shnitzer, T., et al.: Large language model routing with benchmark datasets. In: CoLM (2024)

    2024

  37. [37]

    In: Proceedings of the 12th Language Resources and Evaluation Con- ference (LREC)

    Sigurbergsson, G.I., Derczynski, L.: Offensive language and hate speech detection for Danish. In: Proceedings of the 12th Language Resources and Evaluation Con- ference (LREC). pp. 3498–3508 (2020)

    2020

  38. [38]

    In: ECAI (2025)

    Šléher, R., Brach, W., Sloboda, T., Košťáľ, K., Galke, L.: Guarded query routing for large language models. In: ECAI (2025)

    2025

  39. [39]

    In: Asian Conference on Machine Learning (ACML) (2025)

    Tayebati, S., Kumar, D., Darabi, N., Jayasuriya, D., Krishnan, R., Trivedi, A.R.: CAP: Conformalized abstention policies for context-adaptive risk management for LLMs and VLMs. In: Asian Conference on Machine Learning (ACML) (2025)

    2025

  40. [40]

    In: ICLR (2025)

    Tomani, C., Chaudhuri, K., Evtimov, I., Cremers, D., Ibrahim, M.: Uncertainty- based abstention in large language models improves safety and reduces hallucina- tions. In: ICLR (2025)

    2025

  41. [41]

    ACM Transactions on Intelligent Systems and Technol- ogy (2025)

    Wang, F., et al.: A comprehensive survey of small language models in the era of large language models: Techniques, enhancements, applications, collaboration with llms, and trustworthiness. ACM Transactions on Intelligent Systems and Technol- ogy (2025)

    2025

  42. [42]

    Willard, B.T., Louf, R.: Efficient guided generation for large language models (2023), https://arxiv.org/abs/2307.09702

    Pith/arXiv arXiv 2023

  43. [43]

    In: Findings of the Association for Computational Linguistics: NAACL (2025)

    Xu, R., Ding, K.: Large language models for anomaly and out-of-distribution de- tection: A survey. In: Findings of the Association for Computational Linguistics: NAACL (2025)

    2025

  44. [44]

    Yang, A., et al.: Qwen3 technical report (2025), https://arxiv.org/pdf/2505.09388.pdf

    Pith/arXiv arXiv 2025

  45. [45]

    In: Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT)

    Zampieri, M., Malmasi, S., Nakov, P., Rosenthal, S., Farra, N., Kumar, R.: Pre- dicting the type and target of offensive posts in social media. In: Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT). pp. 1415–1420 (2019)

    2019

  46. [46]

    In: Globerson, A., et al

    Zhao, E., Awasthi, P., Chen, Z., Gollapudi, S., Delling, D.: Semantic routing via autoregressive modeling. In: Globerson, A., et al. (eds.) Advances in Neural In- formation Processing Systems. vol. 37, pp. 10060–10087. Curran Associates, Inc. (2024). https://doi.org/10.52202/079017-0323

    work page doi:10.52202/079017-0323 2024

  47. [47]

    Zhou, J., Lu, T., Mishra, S., Brahma, S., Basu, S., Luan, Y., Zhou, D., Hou, L.: Instruction-following evaluation for large language models (2023), https://arxiv.org/abs/2311.07911

    Pith/arXiv arXiv 2023