{"work":{"id":"b2114616-92b2-4b9a-80db-89bda6422f4e","openalex_id":null,"doi":null,"arxiv_id":"2511.08544","raw_key":null,"title":"LeJEPA: Provable and Scalable Self-Supervised Learning Without the Heuristics","authors":null,"authors_text":"Randall Balestriero, Yann LeCun","year":2025,"venue":"cs.LG","abstract":"Learning manipulable representations of the world and its dynamics is central to AI. Joint-Embedding Predictive Architectures (JEPAs) offer a promising blueprint, but lack of practical guidance and theory has led to ad-hoc R&D. We present a comprehensive theory of JEPAs and instantiate it in {\\bf LeJEPA}, a lean, scalable, and theoretically grounded training objective. First, we identify the isotropic Gaussian as the optimal distribution that JEPAs' embeddings should follow to minimize downstream prediction risk. Second, we introduce a novel objective--{\\bf Sketched Isotropic Gaussian Regularization} (SIGReg)--to constrain embeddings to reach that ideal distribution. Combining the JEPA predictive loss with SIGReg yields LeJEPA with numerous theoretical and practical benefits: (i) single trade-off hyperparameter, (ii) linear time and memory complexity, (iii) stability across hyper-parameters, architectures (ResNets, ViTs, ConvNets) and domains, (iv) heuristics-free, e.g., no stop-gradient, no teacher-student, no hyper-parameter schedulers, and (v) distributed training-friendly implementation requiring only $\\approx$50 lines of code. Our empirical validation covers 10+ datasets, 60+ architectures, all with varying scales and domains. As an example, using imagenet-1k for pretraining and linear evaluation with frozen backbone, LeJEPA reaches 79\\% with a ViT-H/14. We hope that the simplicity and theory-friendly ecosystem offered by LeJEPA will reestablish self-supervised pre-training as a core pillar of AI research (\\href{https://github.com/rbalestr-lab/lejepa}{GitHub repo}).","external_url":"https://arxiv.org/abs/2511.08544","cited_by_count":null,"metadata_source":"pith","metadata_fetched_at":"2026-05-25T05:36:39.290450+00:00","pith_arxiv_id":"2511.08544","created_at":"2026-05-10T00:19:46.810339+00:00","updated_at":"2026-05-25T05:36:39.290450+00:00","title_quality_ok":true,"display_title":"LeJEPA: Provable and Scalable Self-Supervised Learning Without the Heuristics","render_title":"LeJEPA: Provable and Scalable Self-Supervised Learning Without the Heuristics"},"hub":{"state":{"work_id":"b2114616-92b2-4b9a-80db-89bda6422f4e","tier":"hub","tier_reason":"10+ Pith inbound or 1,000+ external citations","pith_inbound_count":31,"external_cited_by_count":null,"distinct_field_count":7,"first_pith_cited_at":"2026-02-04T14:10:36+00:00","last_pith_cited_at":"2026-05-21T20:58:42+00:00","author_build_status":"not_needed","summary_status":"needed","contexts_status":"needed","graph_status":"needed","ask_index_status":"not_needed","reader_status":"not_needed","recognition_status":"not_needed","updated_at":"2026-06-03T22:36:32.347282+00:00","tier_text":"hub"},"tier":"hub","role_counts":[{"context_role":"background","n":2},{"context_role":"method","n":2}],"polarity_counts":[{"context_polarity":"background","n":2},{"context_polarity":"use_method","n":2}],"runs":{},"summary":{},"graph":{},"authors":[]}}