Pith. sign in

REVIEW 4 major objections 4 minor 1 cited by

Memory Storyboard: Leveraging Temporal Segmentation for Streaming Self-Supervised Learning from Egocentric Videos

T0 review · 4 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Memory Storyboard claims that grouping egocentric video frames into short temporal segments and training with a temporal contrastive loss produces state-of-the-art streaming self-supervised representations, nearly matching offline IID…

desk verdict Solid streaming SSL framework with thorough experiments, but the key claim that temporal segmentation drives the gains is untested because the controlled ablation is missing. read the letter →

arxiv 2501.12254 v3 pith:HRWK4MKS submitted 2025-01-21 cs.CV cs.LG

Yanlai Yang , Mengye Ren This is my paper · ORCID
classification cs.CVcs.LG
keywords streamingself-supervisedlearningegocentricvideotemporalsegmentationmemoryreplaycontrastivecontinualSAYCamKrishnaCam
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Streaming self-supervised learning promises to let a model learn visual representations from a continuous, uncurated video stream without replaying the data, but standard contrastive methods collapse under the temporal correlation of egocentric video. This paper tries to establish that a model can avoid that collapse by treating time as structure: segmenting recent frames into coherent storyboard events, using those events as pseudo-classes for a contrastive loss, and storing the segments in a replay buffer. The authors report that this approach, trained from scratch on the SAYCam and KrishnaCam egocentric datasets, outperforms existing streaming and unsupervised continual learning baselines on downstream classification and object detection, and in the best configuration reaches or exceeds the same-cost IID-trained baseline. The practical stake is that embodied agents could learn useful vision from their own raw experience, with memory buffers storing only about 0.3% to 2% of the total stream.

What carries the argument

The load-bearing mechanism is the temporal segmentation module paired with a two-tier memory hierarchy. A first-in-first-out short-term buffer holds recently seen, subsampled frames, and a greedy change-point algorithm maximizes the average within-segment cosine similarity of the encoder's embeddings to split the buffer into a fixed number of segments, roughly $n = L/T$ segments of average length $T$. Each segment receives a pseudo-class label, the labels are propagated to matching frames in the reservoir-sampled long-term buffer, and a supervised contrastive loss pulls augmented views of the same segment together while pushing different segments apart, added to a base self-supervised loss. This design converts the unlabeled stream into a moving set of pseudo-labels without requiring a classifier whose size grows with the number of events, and it is what lets the method avoid treating temporally close frames as negatives.

What would settle it

Take a labeled egocentric stream with known annotated event boundaries, run Memory Storyboard from scratch, and compare the greedy segmentation's change points to ground truth at several early checkpoints; if the pseudo-labels are near-random whenever performance gains appear, then the reported gains are not caused by accurate event segmentation.

Watch

Extended reading notes

Core claim

The paper's central claim is that a streaming self-supervised learner can learn good visual representations from a single pass over long, uncurated egocentric video if it organizes what it sees into temporally coherent storyboard segments rather than isolated frames. Memory Storyboard divides the recent past in a short-term buffer into segments via a greedy change-point algorithm, assigns each segment a pseudo-class label, stores those labels in a long-term replay buffer, and trains a supervised contrastive loss on top of a standard self-supervised loss. On SAYCam and KrishnaCam, models trained this way from scratch outperform prior streaming and unsupervised continual baselines on classification readouts (mini-ImageNet, ImageNet-1K, iNaturalist, Labeled-S) and on OAK object detection; with SimSiam as the base self-supervised loss, Memory Storyboard also exceeds IID SimSiam training on every reported readout.

Load-bearing premise

Everything rests on the temporal segmentation module's pseudo-labels being informative: the greedy algorithm assumes the encoder's cosine similarities mark real event boundaries and that the fixed segment count matches the stream's event structure, even though the encoder starts from random weights and only becomes reliable as training proceeds.

Editorial extensions

If this is right

  • With a 50K-frame buffer on SAYCam, Memory Storyboard narrows the gap to IID training to a few accuracy points on mini-ImageNet and ImageNet-1K, and with SimSiam it matches or passes IID on every readout.
  • With a small 10K buffer, sampling more of each training batch from short-term memory prevents overfitting and is preferable, while with a large 100K buffer the best use of a fixed data budget is to sample mostly from long-term memory.
  • Under a fixed number of gradient updates, a balanced batch (roughly half short-term, half long-term) is the most effective composition for real-time learning.
  • Subsampling the stream to roughly 3 frames per second and using segments of 1 to 5 minutes are safe design defaults; 10-minute segments and higher subsampling rates hurt downstream transfer.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct test of the mechanism's generality would be to run the same storyboard pipeline on non-egocentric streams, such as robot navigation footage or broadcast video, and measure whether early segment boundaries predict final accuracy.
  • The label-merging step, which groups segments whose average embeddings are similar, effectively builds a growing semantic taxonomy; one could test whether the merged label structure aligns with annotated scene or object categories rather than only improving out-of-domain linear readouts.
  • The fixed segment count could be replaced by a learned boundary-prediction head after an initial stabilization phase; if boundary quality is the bottleneck, this should further improve the method.
  • The IID experiments suggest the temporal contrastive loss is useful beyond streaming: it could be applied as an auxiliary objective in any video self-supervised setup where temporal order provides pseudo-labels.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes Memory Storyboard, a streaming self-supervised learning method for long-form egocentric video. The method maintains a two-tier memory (a short-term FIFO buffer and a long-term reservoir buffer), runs a greedy temporal segmentation algorithm on the short-term buffer to group frames into pseudo-labeled temporal segments, and trains with a supervised contrastive loss on those pseudo-labels together with a standard SSL loss (SimCLR or SimSiam). Experiments on SAYCam and KrishnaCam report downstream classification (mini-ImageNet, ImageNet-1K, iNaturalist, Labeled-S) and object detection (OAK) results, claiming that Memory Storyboard outperforms prior streaming SSL baselines and in some cases approaches or exceeds IID training. The paper also includes ablations on label merging, subsampling rate, average segment length, batch composition, and several additional design choices in the appendix.

Significance. If the claims hold, the work is a useful step toward practical streaming SSL from real-world egocentric video: it addresses a realistic setting, uses datasets with naturally occurring temporal structure, and proposes a biologically motivated memory architecture. The strengths are the scope of the experiments (two real-world datasets, multiple downstream tasks, many ablations), the clear separation of short- and long-term memory, and the explicit study of batch-composition trade-offs under memory constraints. The main weaknesses are that the contribution of the temporal segmentation module itself is never isolated from the two-tier memory and temporal contrastive loss, that part of the evaluation uses best-over-time checkpoints rather than the final model, and that several ablation conclusions rest on differences of about one accuracy point without reported variance. These issues are significant but appear addressable with additional experiments and clarification; they do not, in my reading, invalidate the overall direction of the work.

major comments (4)
  1. [Appendix A] Appendix A states that for mini-ImageNet and Labeled-S evaluations, the best result among 20 checkpoints is reported, while the IID references and the INet/iNat/OAK metrics use the final model checkpoint. This asymmetric protocol inflates the streaming results relative to the IID upper bound and makes the contribution claim in Section 1 that Memory Storyboard is 'the only one that is competitive with or even outperforms IID training' difficult to assess. Please report final-checkpoint numbers for all streaming methods, or apply the same best-checkpoint selection to every method uniformly.
  2. [Section 5.3 / Tables 1–2] The contribution of the temporal segmentation module is never isolated. The TC baseline differs in two ways: it uses fixed intervals rather than Algorithm 1, and it uses a classification loss rather than a temporal SupCon loss; it also does not use the two-tier hierarchy. Thus the comparison to TC cannot attribute any gain to temporal segmentation. Add an ablation that keeps the two-tier memory and the temporal SupCon loss exactly the same but replaces Algorithm 1 with fixed, equal-length segments; without this control, the central claim that temporal segmentation is the key mechanism is not supported.
  3. [Section 4 / Appendix C.8] The temporal class labels are generated by the same feature extractor that is being optimized, and Appendix C.8 shows that randomly initialized encoders produce poor segments. The method therefore relies on co-adaptation of the encoder and the pseudo-labels, but the paper provides no quantitative evidence that segmentation quality improves over training or that the temporal contrastive loss is not amplifying early noisy labels. Please report pseudo-label quality (e.g., agreement with human event boundaries, or consistency over time) or compare with a control that uses a frozen random encoder for segmentation, to establish that the temporal objective is learning from signal rather than reinforcing initial biases.
  4. [Section 5.3 / Tables 3–6] Several ablation conclusions are drawn from differences of about one accuracy point or less without any reported variance. For example, in Table 5 the 4x and 8x subsampling ratios both give 36.36 on KrishnaCam mini-ImageNet, and in Table 3 label merging reduces SimCLR 10k mini-ImageNet from 35.02 to 34.18 while the text says it 'consistently improves' performance. Without repeated seeds or error bars, the claimed trends (for example, that an effective frame rate near 3 fps is preferable) are not statistically supported. Please add variances or at least temper the claims accordingly.
minor comments (4)
  1. [Algorithm 2] The line 'n = len(M_s) / T' is not an integer for the default settings (e.g., 5000/4500); specify whether floor or rounding is used, since the subsequent loop in Algorithm 1 assumes an integer number of segments.
  2. [Equation (1)] Equation (1) defines segments with indices t_0 and t_n but the objective sums from i=2; clarify the indexing convention and the role of the first segment in the objective.
  3. [Section 5.3 / Table 3] The text claims that label merging 'consistently improves' performance, but the SimCLR 10k mini-ImageNet row shows a decrease (35.02 without merging vs 34.18 with merging); please revise the wording to reflect the mixed results in the tables.
  4. [Appendix C.4] Appendix C.4 shows that the cross-entropy temporal objective outperforms SupCon on several benchmarks; the main text should mention this caveat when motivating the choice of SupCon for its flexibility in a never-ending stream.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the temporal pseudo-labels are model-generated, but downstream readouts are external and the loss is not equation-for-equation equivalent to the segmentation objective.

full rationale

Memory Storyboard's temporal segmentation uses the current encoder's cosine similarity to define pseudo-labels (Eq. 1 and Algorithm 1), and the temporal contrastive loss (Eq. 2) trains that same encoder with those labels. This is a self-training loop, but it does not reduce to a derivation from its own inputs by construction: Eq. 2 is not Eq. 1, the pseudo-labels are not fitted to the downstream evaluation targets, the overall loss includes SimCLR/SimSiam augmentation-based SSL terms, and all headline results are measured on held-out tasks (mini-ImageNet, ImageNet-1K, iNaturalist, Labeled-S, OAK). The paper explicitly acknowledges that randomly initialized encoders produce poor segments (Appendix C.8), which is a limitation of pseudo-label quality under co-adaptation rather than a circular proof step. The self-citations are not load-bearing: Osiris (Zhang et al., 2024) is used as an architectural template and is also evaluated as an external baseline, and Ren et al. (2021) is cited alongside an independent work (Purushwalkam et al., 2022) for the known observation that streaming SSL without replay performs poorly. The absence of a controlled ablation that replaces Algorithm 1 with fixed, equal-length segments while keeping the same two-tier memory and temporal SupCon loss is an experimental gap, not circularity. The central claims are therefore supported by external benchmarks rather than being equivalent to the method's inputs.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim rests on several free hyperparameters (segment length, subsampling rate, label merging threshold) and on the assumption that the model's own embeddings produce useful temporal pseudo-labels. No new physical or mathematical entities are introduced. The method is a self-training loop over the model's own segmentation.

free parameters (5)
  • Average segment length T = 4.5K frames for SAYCam, 1.8K frames for KrishnaCam
    Determines number of segments n = L/T. Chosen by tuning on downstream performance (Section 5.3); results degrade for T=10 min.
  • Subsampling rate r = 8 for SAYCam, 4 for KrishnaCam
    Controls temporal correlation in short-term memory. Tuned per dataset to achieve about 3 fps effective rate (Section 5.3).
  • Label merging quantile threshold tau = 0.998
    Used to set the similarity threshold delta for merging temporal segments. Fixed for main experiments (Section 5.1).
  • Warm-start length = Equal to |M_long|
    Model makes no updates for the first M_long frames to fill memory. This is a hyperparameter affecting early training.
  • Batch composition (b from short-term) = b=64, B=512 (12.5% from short-term) in main experiments
    The ratio of short-term to long-term samples in the training batch is studied in Section 5.4 and affects performance.
assumptions (5)
  • ad hoc to paper Greedy temporal segmentation approximates the optimum of Eq. (1)
    Algorithm 1 uses a greedy change-point search; there is no guarantee it finds the true maximum within-class similarity. The paper relies on this approximation for the labels.
  • ad hoc to paper Number of temporal segments is fixed as n = L/T
    The model assumes uniform segment length a priori. Real video events vary in length, so this may not match actual event boundaries.
  • domain assumption Cosine similarity of embeddings reflects semantic event boundaries
    The segmentation uses f_theta(x) features from the model being trained. At initialization these are random, and the paper shows random models produce poor segments (Appendix C.8).
  • domain assumption Temporal segments are semantically coherent and useful as positive pairs
    The temporal contrastive loss treats frames within a segment as positives. This assumes that event segmentation provides a good learning signal for downstream visual tasks.
  • standard math Standard self-supervised loss (SimCLR or SimSiam) prevents representation collapse
    The overall loss is L_TCL + L_SSL; the paper relies on the SSL loss to avoid trivial solutions. This is a known property of these methods.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Memory Storyboard: Leveraging Temporal Segmentation for Streaming Self-Supervised Learning from Egocentric Videos." pith.science (2026). https://pith.science/paper/HRWK4MKS

@misc{pith2026250112254,
  author       = {Pith},
  title        = {Pith review of: Memory Storyboard: Leveraging Temporal Segmentation for Streaming Self-Supervised Learning from Egocentric Videos},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HRWK4MKS}},
  note         = {Machine review of arXiv:2501.12254}
}
read the original abstract

Self-supervised learning holds the promise of learning good representations from real-world continuous uncurated data streams. However, most existing works in visual self-supervised learning focus on static images or artificial data streams. Towards exploring a more realistic learning substrate, we investigate streaming self-supervised learning from long-form real-world egocentric video streams. Inspired by the event segmentation mechanism in human perception and memory, we propose "Memory Storyboard" that groups recent past frames into temporal segments for more effective summarization of the past visual streams for memory replay. To accommodate efficient temporal segmentation, we propose a two-tier memory hierarchy: the recent past is stored in a short-term memory, and the storyboard temporal segments are then transferred to a long-term memory. Experiments on real-world egocentric video datasets including SAYCam and KrishnaCam show that contrastive learning objectives on top of storyboard frames result in semantically meaningful representations that outperform those produced by state-of-the-art unsupervised continual learning methods.

Figures

Figures reproduced from arXiv: 2501.12254 by the authors.

Figure 1
Figure 1. Memory Storyboard framework for streaming self-supervised learning (SSL) from egocentric videos. Given a continuous stream of images from an egocentric video, Memory Storyboard effectively learns visual represen￾tations by clustering similar frames into temporal segments and updating their labels (text information for illustration purposes only) in the long-term memory buffer for replay. SSL involves contrastive lea… view at source ↗
Figure 2
Figure 2. Details of our two-tier memory in Memory Storyboard. Long-term memory is updated with reser￾voir sampling and short-term memory with first-in-first￾out (FIFO). Temporal segmentation is applied on the short￾term memory, which then updates the labels of correspond￾ing images in the long-term memory. Two-tier Memory Hierarchy. Inspired by the Com￾plementary Learning Systems (CLS) theory (McClel￾land et al., 1995; O’Rei… view at source ↗
Figure 3
Figure 3. Visualization of the temporal segments produced by Memory Storyboard on (a) SAYCam (b)(c) KrishnaCam at the end of training. The images are sampled at 10 seconds per frame. Each color bar corresponds to a temporal class (the first and the last class might be incomplete). Temporal segments produced at the beginning of training are provided in the appendix for comparison. CLR (Chen et al., 2020) in our experiments bec… view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Memory Storyboard model performance on SAYCam with different long-term memory sizes (5k, 10k, 50k, and 100k) and varying training batch compositions (12.5% – 75.0% from Mshort) using SVM readout. Each colored line represents the performance of different training batch …
Figure 5
Figure 5. Figure 5: Visualization of object detection results on the OAK validation set. The Memory Storyboard model is trained on KrishnaCam and fine-tuned on the OAK training set. Red boxes show the predictions and the green boxes are ground truth bounding boxes. E MORE COMPREHENSIVE CO…
Figure 6
Figure 6. Figure 6: Visualization of the temporal segments produced by randomly initialized models on (a) SAYCam (b)(c) KrishnaCam. The images are the same as the ones in [PITH_FULL_IMAGE:figures/full_fig_p021_6.png]
Figure 7
Figure 7. Figure 7: Visualization of label merging by Memory Storyboard on SAYCam. Each image represents a temporal segment; segments sharing the same color bar have been merged. Memory Storyboard successfully groups semanti￾cally similar scenes—e.g., segments marked with the light blue b…
Figure 8
Figure 8. Figure 8: SimCLR model performance on SAYCam with different long-term memory sizes (5k, 10k, 50k, and 100k) and varying training batch compositions (12.5% – 75.0% from Mshort) using SVM readout. Each colored line represents the performance of different training batch composition…
Figure 9
Figure 9. Figure 9: Comparison of Memory Storyboard (solid lines) and SimCLR (dashed lines) model performance on SAYCam using SVM readout, controlling the amount of data the model has seen from the stream. 22 [PITH_FULL_IMAGE:figures/full_fig_p022_9.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. EgoEverything: A Benchmark for Human Behavior Inspired Long Context Egocentric Video Understanding in AR Environment

    cs.LG 2026-04 unverdicted novelty 6.0 of 10

    EgoEverything is a new benchmark for long-context egocentric video understanding that uses human gaze-based attention signals to generate questions reflecting natural behavior.

Reference graph

Works this paper leans on

35 extracted references · 26 canonical work pages · cited by 1 Pith paper

  1. [3]

    Figure 7:Visualization of label merging by Memory Storyboard on SAYCam.Each image represents a temporal segment; segments sharing the same color bar have been merged

    We observe that Memory Storyboard training enables to model to capture more intricate transitions between scenes. Figure 7:Visualization of label merging by Memory Storyboard on SAYCam.Each image represents a temporal segment; segments sharing the same color bar have been merged. Memory Storyboard successfully groups semanti- cally similar scenes—e.g., se...

    work page 2025

  2. [4]

    produces a sizable improvement on the ImageNet readout evaluation on KrishnaCam but not on the other benchmarks. We also observed that the improvement of multiple gradient steps is a lot smaller on SAYCam (sometimes even harming the performance), presumably due to the fact that SAYCam is a much larger training dataset than KrishnaCam and streaming learnin...

    work page 2025

  3. [7]

    Lassiter and David Slaw

    12 Published at 4th Conference on Lifelong Learning Agents (CoLLAs), 2025 G. Lassiter and David Slaw. The unitization and memory of events.Journal of Experimental Psychology: General, 120:80–82, 03

    work page 2025

  4. [8]

    The results here demonstrate that applying the temporal contrastive loss only on data from long-term memory or on the entire training batch achieves the best performance

    For experiments in the main paper, we apply the temporal contrastive loss only on data from long-term memory. The results here demonstrate that applying the temporal contrastive loss only on data from long-term memory or on the entire training batch achieves the best performance. Applying the temporal contrastive loss only on data from short-term memory a...

    work page 2025

  5. [9]

    In particular, we note that Memory Storyboard significantly outperforms SimCLR when we sample more data fromM short (towards the right side of thex-axis)

    We observe that Memory Storyboard outperforms SimCLR under the same amount of seen data, across a wide range of memory sizes and batch compositions. In particular, we note that Memory Storyboard significantly outperforms SimCLR when we sample more data fromM short (towards the right side of thex-axis). This results in higher optimal performance when the m...

    work page 2025

  6. [10]

    Category-specific video summarization

    Danila Potapov, Matthijs Douze, Zaid Harchaoui, and Cordelia Schmid. Category-specific video summarization. In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceed- ings, Part VI 13, pp. 540–555. Springer,

    work page 2014

  7. [11]

    We observe that the performance of only using the SupCon loss is also inferior to the full memory storyboard method, demonstrating the necessity of joint training on both losses for best performance. Method SAYCam 10K SAYCam 50K KrishnaCam 10K KrishnaCam 50K mini-INet INet iNat mini-INet INet iNat mini-INet INet iNat mini-INet INet iNat SimCLR MemStoryboa...

    work page 2022

  8. [13]

    Megan M Saylor, Dare A Baldwin, Jodie A Baird, and Jennifer LaBounty

    doi: 10.3758/s13428-022-01832-5. Megan M Saylor, Dare A Baldwin, Jodie A Baird, and Jennifer LaBounty. Infants’ on-line segmentation of dynamic human action.Journal of Cognition and Development, 8(1):113–128,

    work page doi:10.3758/s13428-022-01832-5

Show all 35 references
  1. [15]

    Is imagenet worth 1 video? learning strong image encoders from 1 long unlabelled video.arXiv preprint arXiv:2310.08584,

    Shashanka Venkataramanan, Mamshad Nayeem Rizve, João Carreira, Yuki M Asano, and Yannis Avrithis. Is imagenet worth 1 video? learning strong image encoders from 1 long unlabelled video.arXiv preprint arXiv:2310.08584,

    arXiv

  2. [17]

    Neural event segmentation of continuous experience in human infants.Proceedings of the National Academy of Sciences, 119(43):e2200257119,

    14 Published at 4th Conference on Lifelong Learning Agents (CoLLAs), 2025 Tristan S Yates, Lena J Skalaban, Cameron T Ellis, Angelika J Bracher, Christopher Baldassano, and Nicholas B Turk-Browne. Neural event segmentation of continuous experience in human infants.Proceedings ...

    2025

  3. [18]

    Large batch training of convolutional networks.arXiv preprint arXiv:1708.03888,

    Yang You, Igor Gitman, and Boris Ginsburg. Large batch training of convolutional networks.arXiv preprint arXiv:1708.03888,

    arXiv

  4. [19]

    Video summarization with long short-term memory

    Ke Zhang, Wei-Lun Chao, Fei Sha, and Kristen Grauman. Video summarization with long short-term memory. In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part VII 14, pp. 766–782. Springer,

    2016

  5. [20]

    Integrating present and past in unsupervised continual learning.arXiv preprint arXiv:2404.19132,

    Yipeng Zhang, Laurent Charlin, Richard Zemel, and Mengye Ren. Integrating present and past in unsupervised continual learning.arXiv preprint arXiv:2404.19132,

    arXiv

  6. [21]

    In Memory Storyboard, we create two separate projectors for LT CLandL SSL

    15 Published at 4th Conference on Lifelong Learning Agents (CoLLAs), 2025 A EXPERIMENTDETAILS Model Architecture.On top of the ResNet backbone, we use a two-layer MLP with 2048 hidden units, 128 output units, and ReLU activation function as the projector. In Memory Storyboard,...

    2025

  7. [23]

    For the SimSiam (Chen & He,

    optimizer with a constant learning rate of 0.001, and a projector with 2 MLP lay- ers of size 2048 and 128 respectively. For the SimSiam (Chen & He,

  8. [26]

    For OAK evaluations, we use Faster R-CNN (Ren et al., 2015), a popular two-stage object detector

    optimizer with learning rate 12.0, momentum 0.9, and cosine learning rate schedule for 20 epochs. For OAK evaluations, we use Faster R-CNN (Ren et al., 2015), a popular two-stage object detector. We initialize the ResNet-50 (He et al.,

    2015

  9. [27]

    backbone with the backbone of the final checkpoint of the streaming SSL model, and fine-tune the entire model on OAK with IID training for 10 epochs, following the training configurations of (Wu et al., 2023). B ADDITIONALRELATEDWORK Self-Supervised Learning.A large number of ...

    2023

  10. [28]

    uses the labels as an extra supervision signal to get multiple positive crops for each anchor image. Other recent self-supervised learning works include pretext tasks (Doersch et al., 2015; Noroozi & Favaro, 2016; Gidaris et al., 2018; Pathak et al., 2016), feature space clust...

    2020

  11. [29]

    that BatchNorm is less compatible with unsupervised continual learning, and extends the conclusion to streaming SSL. SimCLR Osiris TC MemStoryboard Batch Norm 33.62 33.32 33.16 33.68 Group Norm37.96 36.90 36.68 39.58 Table 7: Group norm is better at dealing with temporal non-s...

    2024

  12. [33]

    We observe that the analysis and the conclusions of section 5.4 still hold: when we have a large memory, we either prefer balanced training batches (with a fixed amount of computation) or a bigger batch from long-term memory (with a fixed amount of data); when we can only affo...

    2025

  13. [112]

    We apply a standard data augmentation pipeline for SSL methods following Zhuang et al. (2022), which include random resized crop, random horizontal flip, random color jitter, random grayscale, random Gaussian filter, and color-normalization with ImageNet (Deng et al., 2009). F...

    2022

  14. [1991]

    Divyam Madaan, Jaehong Yoon, Yuanchun Li, Yunxin Liu, and Sung Ju Hwang

    doi: 10.1037/0096-3445.120.1.80. Divyam Madaan, Jaehong Yoon, Yuanchun Li, Yunxin Liu, and Sung Ju Hwang. Representational continuity for unsupervised continual learning. InThe Tenth International Conference on Learning Representations,

    doi

  15. [2001]

    Class incremental online streaming learning.arXiv preprint arXiv:2110.10741,

    Soumya Banerjee, Vinay Kumar Verma, Toufiq Parag, Maneesh Singh, and Vinay P Namboodiri. Class incremental online streaming learning.arXiv preprint arXiv:2110.10741,

    arXiv

  16. [2011]

    Enrico Fini, Victor G Turrisi Da Costa, Xavier Alameda-Pineda, Elisa Ricci, Karteek Alahari, and Julien Mairal

    doi: 10.1177/0956797610393742. Enrico Fini, Victor G Turrisi Da Costa, Xavier Alameda-Pineda, Elisa Ricci, Karteek Alahari, and Julien Mairal. Self- supervised models are continual learners. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p...

    doi

  17. [2013]

    Youssef Ezzyat and Lila Davachi

    doi: 10.1037/a0034024. Youssef Ezzyat and Lila Davachi. What constitutes an episode in episodic memory?Psychological science, 22: 243–52, 02

    doi

  18. [2015]

    The influence of context boundaries on memory for the sequential order of events

    11 Published at 4th Conference on Lifelong Learning Agents (CoLLAs), 2025 Sarah DuBrow and Lila Davachi. The influence of context boundaries on memory for the sequential order of events. Journal of Experimental Psychology: General, 142:1277–1286, 08

    2025

  19. [2016]

    Representation learning with contrastive predictive coding.arXiv preprint arXiv:1807.03748,

    Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding.arXiv preprint arXiv:1807.03748,

    arXiv

  20. [2017]

    For iNaturalist-2018, we used the LARS (You et al.,

    optimizer with learning rate 3.0, momentum 0.9, and cosine learning rate schedule for 10 epochs. For iNaturalist-2018, we used the LARS (You et al.,

    2018

  21. [2018]

    Measuring event segmentation: An investigation into the stability of event bound- ary agreement across groups.Behavior Research Methods, 55, 04

    13 Published at 4th Conference on Lifelong Learning Agents (CoLLAs), 2025 Karen Sasmita and Khena Swallow. Measuring event segmentation: An investigation into the stability of event bound- ary agreement across groups.Behavior Research Methods, 55, 04

    2025

  22. [2019]

    Scott, Michael L

    Mengye Ren, Tyler R. Scott, Michael L. Iuzzolino, Michael C. Mozer, and Richard S. Zemel. Online unsupervised learning of visual representations and categories.arXiv preprint arXiv:2109.05675,

    arXiv

  23. [2020]

    Kingma and Jimmy Ba

    Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings,

    2015

  24. [2021]

    Contrastive multiview coding

    Yonglong Tian, Dilip Krishnan, and Phillip Isola. Contrastive multiview coding. InComputer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XI 16, pp. 776–794. Springer,

    2020

  25. [2022]

    Forgetting order of continual learning: Examples that are learned first are forgot- ten last.arXiv preprint arXiv:2406.09935,

    Guy Hacohen and Tinne Tuytelaars. Forgetting order of continual learning: Examples that are learned first are forgot- ten last.arXiv preprint arXiv:2406.09935,

    arXiv

  26. [2024]

    Poodle: Pooled and dense self- supervised learning from naturalistic videos.arXiv preprint arXiv:2408.11208,

    Alex N Wang, Christopher Hoang, Yuwen Xiong, Yann LeCun, and Mengye Ren. Poodle: Pooled and dense self- supervised learning from naturalistic videos.arXiv preprint arXiv:2408.11208,

    arXiv

  27. [2048]

    That is, we store 20 model checkpoints throughout the streaming training and evaluate them on mini-ImageNet and Labeled-S with SVM readout

    Evaluation.Formini-ImageNet and Labeled-S evaluations, the streaming SSL models are evaluated every 5% of the entire dataset. That is, we store 20 model checkpoints throughout the streaming training and evaluate them on mini-ImageNet and Labeled-S with SVM readout. Thebestresu...

    2022

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.