REVIEW 3 cited by
Positional Encodings for Light Curve Transformers: Playing with Positions and Attention
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
We conducted empirical experiments to assess the transferability of a light curve transformer to datasets with different cadences and magnitude distributions using various positional encodings (PEs). We proposed a new approach to incorporate the temporal information directly to the output of the last attention layer. Our results indicated that using trainable PEs lead to significant improvements in the transformer performances and training times. Our proposed PE on attention can be trained faster than the traditional non-trainable PE transformer while achieving competitive results when transfered to other datasets.
Forward citations
Cited by 3 Pith papers
-
How Low Can We Go? Minimum Spectroscopic Requirements For Supernova Subtype Classification
ABC-SN classifies ten supernova subtypes with no performance loss down to R_λ=50 and SNR=5, and only minimal loss at R_λ=25.
-
StarCLR: Contrastive Learning Representation for Astronomical Light Curves
StarCLR pretrains on TESS light curves via contrastive learning on overlapping subsequences and improves variable star classification F1 scores over scratch-trained models when fine-tuned on TESS, ZTF, and Gaia.
-
ABC-SN: Attention Based Classifier for Supernova Spectra
ABC-SN, a transformer-based classifier, reaches 82.5% macro F1 on ten supernova subtypes, outperforming a retrained DASH at 58.9% on the same test set.
Discussion (0). Sign in to comment.