REVIEW 5 cited by
Generative Pre-trained Transformer: A Comprehensive Review on Enabling Technologies, Potential Applications, Emerging Challenges, and Future Directions
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
The Generative Pre-trained Transformer (GPT) represents a notable breakthrough in the domain of natural language processing, which is propelling us toward the development of machines that can understand and communicate using language in a manner that closely resembles that of humans. GPT is based on the transformer architecture, a deep neural network designed for natural language processing tasks. Due to their impressive performance on natural language processing tasks and ability to effectively converse, GPT have gained significant popularity among researchers and industrial communities, making them one of the most widely used and effective models in natural language processing and related fields, which motivated to conduct this review. This review provides a detailed overview of the GPT, including its architecture, working process, training procedures, enabling technologies, and its impact on various applications. In this review, we also explored the potential challenges and limitations of a GPT. Furthermore, we discuss potential solutions and future directions. Overall, this paper aims to provide a comprehensive understanding of GPT, enabling technologies, their impact on various applications, emerging challenges, and potential solutions.
Forward citations
Cited by 5 Pith papers
-
PolyMicros: Bootstrapping a Foundation Model for Polycrystalline Material Structure
PolyMicros bootstraps a generative foundation model for polycrystalline microstructures from five experimental volumes and applies it zero-shot to microscopy super-resolution and 2D-to-3D dimensionality expansion.
-
Revisiting Autoregressive Models for Generative Image Classification
Order-marginalized any-order AR models (RandAR) outperform diffusion generative classifiers on ImageNet and OOD sets and match strong SSL models at far lower cost.
-
GEM: Empowering LLM for both Embedding Generation and Language Understanding
GEM fine-tunes decoder-only LLMs with bottleneck special tokens and a mixed next-token prediction and contrastive objective, giving them text-embedding ability with only a modest MMLU drop.
-
Energy- and Memory-Efficient PEFT Methods for Personalized On-Device SLMs on Consumer GPUs
On consumer GPUs, LoRA+ gives the best energy-focused fine-tuning score in 19 of 24 small-model task configurations, while QLoRA wins the memory-focused score when peak VRAM is the binding constraint.
-
Krul: Efficient State Restoration for Multi-turn Conversations with Dynamic Cross-layer KV Sharing
Krul dynamically selects per-conversation cross-layer KV cache compression from attention similarity patterns, cutting TTFT by 1.28x to 2.68x and KV storage by 1.33x to 2.35x with less than 1% average accuracy loss on...
Discussion (0). Sign in to comment.