TSpec-LLM: An Open-source Dataset for LLM Understanding of 3GPP Specifications
read the original abstract
Understanding telecom standards involves sorting through numerous technical documents, such as those produced by the 3rd Generation Partnership Project (3GPP), which is time-consuming and labor-intensive. While large language models (LLMs) can assist with the extensive 3GPP knowledge base, an inclusive dataset is crucial for their effective pre-training and fine-tuning. In this paper, we introduce \textit{TSpec-LLM}, an open-source comprehensive dataset covering all 3GPP documents from Release 8 to Release 19 (1999--2023). To evaluate its efficacy, we first select a representative sample of 3GPP documents, create corresponding technical questions, and assess the baseline performance of various LLMs. We then incorporate a retrieval-augmented generation (RAG) framework to enhance LLM capabilities by retrieving relevant context from the \textit{TSpec-LLM} dataset. Our evaluation shows that using a naive-RAG framework on \textit{TSpec-LLM} improves the accuracy of GPT-3.5, Gemini 1.0 Pro, and GPT-4 from 44\%, 46\%, and 51\% to 71\%, 75\%, and 72\%, respectively.
This paper has not been read by Pith yet.
Forward citations
Cited by 6 Pith papers
-
TeleCom-Bench: How Far Are Large Language Models from Industrial Telecommunication Applications?
TeleCom-Bench reveals LLMs reach 90% on telecom intent and entity tasks but drop to 30% on solution generation and root cause analysis in live network scenarios.
-
Multimodal Large Language Model Enabled Robust Beamforming for HAP Downlink Communications
A vision-language LLM forecasts HAP attitudes from telemetry for proactive beamforming, achieving 22.1% higher user service ratio and 12.5% higher sum-rate than baselines in simulations with mean latency of 36 ms.
-
MM-Telco: Benchmarks and Multimodal Large Language Models for Telecom Applications
MM-Telco creates multimodal benchmarks for telecom and demonstrates that fine-tuned LLMs and VLMs achieve significant performance gains on domain-specific tasks.
-
Against the Monolithic Wireless World Model: Why NextG Needs Composable and Agentic Intelligence
Wireless data lacks the self-contained tokenized substrate of text, so monolithic wireless world models are unsuitable for 6G; composable agentic systems using specialized components and explicit interfaces are the re...
-
Against the Monolithic Wireless World Model: Why NextG Needs Composable and Agentic Intelligence
Argues that wireless data's configuration dependence and lack of self-containment make monolithic foundation models unsuitable for AI-native 6G, favoring instead composable agentic architectures.
-
Customized Generative AI Agent for Transportation Engineering Practice: A Development and Continued Pre-training Guideline
A framework is described for adapting six LLMs to transportation engineering via LoRA-based continued pretraining on domain documents, with two models showing strongest results on BLEU-4 and ROUGE metrics.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.