Compressing intent labels for LLM fine-tuning and using self-consistency-filtered LLM pseudo-labeling improve multi-turn intent classification accuracy and enable small, low-latency production models.
Towards Objective and Unbiased Decision Assessments with LLM-Enhanced Hierarchical Attention Networks
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
How objective and unbiased are we while making decisions? This work investigates cognitive bias identification in high-stake decision making process by human experts, questioning its effectiveness in real-world settings, such as candidates assessments for university admission. We begin with a statistical analysis assessing correlations among different decision points among in the current process, which discovers discrepancies that imply cognitive bias and inconsistency in decisions. This motivates our exploration of bias-aware AI-augmented workflow that surpass human judgment. We propose BGM-HAN, an enhanced Hierarchical Attention Network with Byte-Pair Encoding, Gated Residual Connections and Multi-Head Attention. Using it as a backbone model, we further propose a Shortlist-Analyse-Recommend (SAR) agentic workflow, which simulate real-world decision-making. In our experiments, both the proposed model and the agentic workflow significantly improves on both human judgment and alternative models, validated with real-world data.
fields
cs.CL 1years
2024 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Balancing Accuracy and Efficiency in Multi-Turn Intent Classification for LLM-Powered Dialog Systems in Production
Compressing intent labels for LLM fine-tuning and using self-consistency-filtered LLM pseudo-labeling improve multi-turn intent classification accuracy and enable small, low-latency production models.