REVIEW 5 cited by
FT-Transformer: Resilient and Reliable Transformer with End-to-End Fault Tolerant Attention
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
FT-Transformer: Resilient and Reliable Transformer with End-to-End Fault Tolerant Attention
read the original abstract
Transformer models rely on High-Performance Computing (HPC) resources for inference, where soft errors are inevitable in large-scale systems, making the reliability of the model particularly critical. Existing fault tolerance frameworks for Transformers are designed at the operation level without architectural optimization, leading to significant computational and memory overhead, which in turn reduces protection efficiency and limits scalability to larger models. In this paper, we implement module-level protection for Transformers by treating the operations within the attention module as a single kernel and applying end-to-end fault tolerance. This method provides unified protection across multi-step computations, while achieving comprehensive coverage of potential errors in the nonlinear computations. For linear modules, we design a strided algorithm-based fault tolerance (ABFT) that avoids inter-thread communication. Experimental results show that our end-to-end fault tolerance achieves up to 7.56x speedup over traditional methods with an average fault tolerance overhead of 13.9%.
Forward citations
Cited by 5 Pith papers
-
Tight Lower Bounds on The Single-Error Detection Threshold for Analog Error-Correcting Codes
Every (n-2)-dimensional subspace of R^n (n even) contains a nonzero vector with largest-to-second-largest absolute entry ratio at least n/2 - 1, and the general lower bound on single-error detection height is tight wh...
-
Self-Verifying Measurement Records: Hash-Linked Evidence Graphs for Hardware Benchmarking
The paper constructs hash-linked evidence graphs that bind hardware measurement quantities to their verification records, enabling offline auditing with probabilistic matrix checks and security measures against probe ...
-
Hierarchical Fault Detection and Diagnosis for Transformer Architectures
DEFault++ delivers automated hierarchical fault detection, categorization into 12 transformer-specific types, and root-cause diagnosis among 45 mechanisms on a new benchmark of 3,739 mutated instances, with AUROC >0.9...
-
Hierarchical Fault Detection and Diagnosis for Transformer Architectures
DEFault++ applies hierarchical learning with a Fault Propagation Graph to detect, localize, and diagnose faults in transformers, improving F1 to 0.826-0.909 and developer repair accuracy from 57.1% to 83.3% on a new b...
-
A Foundation Model for Multimodal Event Sequences in Financial Applications
Early-fusion next-event pretraining on multimodal bank event sequences yields reusable user embeddings that, combined with engineered features, improve multi-task financial predictions and production NPV.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.