Linear probes trained on pre-solution hidden states, supervised by post-solution correctness probe outputs, recover 32–66% of the calibration gap between pre- and post-solution confidence across five open-source LLMs.
Self Knowledge Re-expression: A Fully Local Method for Adapting LLMs to Tasks Using Intrinsic Knowledge
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
While the next-token prediction (NTP) paradigm enables large language models (LLMs) to express their intrinsic knowledge, its sequential nature constrains performance on specialized, non-generative tasks. We attribute this performance bottleneck to the LLMs' knowledge expression mechanism, rather than to deficiencies in knowledge acquisition. To address this, we propose Self-Knowledge Re-expression (SKR), a novel, task-agnostic adaptation method. SKR transforms the LLM's output from generic token generation to highly efficient, task-specific expression. SKR is a fully local method that uses only unannotated data, requiring neither human supervision nor model distillation. Experiments on a large financial document dataset demonstrate substantial improvements: over 40% in Recall@1 for information retrieval tasks, over 76% reduction in object detection latency, and over 33% increase in anomaly detection AUPRC. Our results on the MMDocRAG dataset surpass those of leading retrieval models by at least 12.6%.
fields
cs.CL 1years
2026 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Future Confidence Distillation in Large Language Models
Linear probes trained on pre-solution hidden states, supervised by post-solution correctness probe outputs, recover 32–66% of the calibration gap between pre- and post-solution confidence across five open-source LLMs.