pith. sign in

arxiv: 2604.15814 · v1 · submitted 2026-04-17 · 💻 cs.CV · cs.RO

Continual Hand-Eye Calibration for Open-world Robotic Manipulation

Pith reviewed 2026-05-10 08:50 UTC · model grok-4.3

classification 💻 cs.CV cs.RO
keywords scenecalibrationreplaycontinualmanipulationopen-worldscenesdistillation
0
0 comments X

The pith

A continual learning framework with spatial replay and dual distillation enables hand-eye calibration models to retain accuracy on past scenes while adapting to new open-world robotic manipulation scenes.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Hand-eye calibration helps robots know exactly where their hand is relative to what they see with cameras. In open-world robotic manipulation, the environment changes over time, and deep learning models often forget how to calibrate for old scenes when learning new ones. This is known as catastrophic forgetting. To fix this, the authors developed a continual learning framework. It uses a Spatial-Aware Replay Strategy called SARS. This strategy builds a replay buffer with geometrically uniform samples from each past scene. It replaces similar nearby frames with more informative viewpoints to cover the pose space better. They also created Structure-Preserving Dual Distillation or SPDD. This splits the localization knowledge into coarse scene layout and fine pose precision. These are distilled separately to the model to prevent forgetting both aspects. When a new scene appears, the system uses replay samples from old scenes and applies the distillation to keep old knowledge. After learning the new scene, it adds selected new samples to the buffer. This way, the model accumulates calibration skills across multiple scenes. The approach was tested on several public datasets and showed good performance in keeping accuracy on previous scenes while adapting to new ones.

Core claim

Experiments on multiple public datasets show significant anti scene forgetting performance, maintaining accuracy on past scenes while preserving adaptation to new scenes, confirming the effectiveness of the framework.

Load-bearing premise

That the spatially uniform replay buffer from SARS and the coarse/fine decomposition in SPDD will effectively mitigate both types of forgetting across diverse open-world scene changes without post-hoc adjustments or unstated limitations in coverage.

Figures

Figures reproduced from arXiv: 2604.15814 by Chenxi Liu, Fazeng Li, Gan Sun, Wei Cong, Yang Cong, Yao He.

Figure 1
Figure 1. Figure 1: Motivation of continual hand-eye calibration problem, where a single calibration model is sequentially trained on several open-world scenes ( [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of the proposed continual hand-eye calibration framework. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Per-scene localization accuracy comparison across all three datasets. Each group shows the final accuracy of all continual learning methods on [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Task forgetting rate comparisons on the i12S dataset. [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Task forgetting rate comparisons on the i7S dataset. [PITH_FULL_IMAGE:figures/full_fig_p009_5.png] view at source ↗
Figure 7
Figure 7. Figure 7: Qualitative localization results on the i7S dataset. For each scene, the left image shows the 3D model rendered using the estimated camera pose [PITH_FULL_IMAGE:figures/full_fig_p010_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Qualitative localization results on the i12S dataset. Each pair shows the rendered 3D model (left, grayscale) overlaid with the query image (right) [PITH_FULL_IMAGE:figures/full_fig_p011_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Visualization of localization results on the Simulation dataset. The [PITH_FULL_IMAGE:figures/full_fig_p011_9.png] view at source ↗
read the original abstract

Hand-eye calibration through visual localization is a critical capability for robotic manipulation in open-world environments. However, most deep learning-based calibration models suffer from catastrophic forgetting when adapting into unseen data amongst open-world scene changes, while simple rehearsal-based continual learning strategy cannot well mitigate this issue. To overcome this challenge, we propose a continual hand-eye calibration framework, enabling robots to adapt to sequentially encountered open-world manipulation scenes through spatially replay strategy and structure-preserving distillation. Specifically, a Spatial-Aware Replay Strategy (SARS) constructs a geometrically uniform replay buffer that ensures comprehensive coverage of each scene pose space, replacing redundant adjacent frames with maximally informative viewpoints. Meanwhile, a Structure-Preserving Dual Distillation (SPDD) is proposed to decompose localization knowledge into coarse scene layout and fine pose precision, and distills them separately to alleviate both types of forgetting during continual adaptation. As a new manipulation scene arrives, SARS provides geometrically representative replay samples from all prior scenes, and SPDD applies structured distillation on these samples to retain previously learned knowledge. After training on the new scene, SARS incorporates selected samples from the new scene into the replay buffer for future rehearsal, allowing the model to continuously accumulate multi-scene calibration capability. Experiments on multiple public datasets show significant anti scene forgetting performance, maintaining accuracy on past scenes while preserving adaptation to new scenes, confirming the effectiveness of the framework.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 3 minor

Summary. The paper proposes a continual hand-eye calibration framework for open-world robotic manipulation to address catastrophic forgetting in deep learning models when adapting to sequential scene changes. It introduces a Spatial-Aware Replay Strategy (SARS) that constructs a geometrically uniform replay buffer by replacing redundant frames with maximally informative viewpoints, and a Structure-Preserving Dual Distillation (SPDD) method that decomposes localization knowledge into coarse scene layout and fine pose precision components for separate distillation. As new scenes arrive, the framework replays representative samples from prior scenes and applies structured distillation to retain past knowledge while adapting to the new scene; selected new samples are then added to the buffer. Experiments on multiple public datasets are claimed to demonstrate maintained accuracy on past scenes alongside successful adaptation to new ones.

Significance. If the experimental claims hold under detailed scrutiny, this work would offer a targeted contribution to continual learning for geometric vision tasks in robotics. The geometry-aware replay buffer and dual-level (layout + precision) distillation address domain-specific aspects of hand-eye calibration forgetting that generic rehearsal or distillation methods may not handle as effectively. This could support more reliable long-term operation of manipulation robots in unstructured, changing environments. The approach builds on established replay and distillation ideas but specializes them to spatial pose spaces and hierarchical scene structure.

major comments (1)
  1. [§4] §4 (Experiments): The central claim of 'significant anti scene forgetting performance' and 'maintaining accuracy on past scenes while preserving adaptation to new scenes' lacks supporting quantitative details such as specific pose estimation errors (e.g., translation/rotation RMSE), forgetting rates, or direct comparisons against standard continual learning baselines (e.g., plain rehearsal, EWC, or LwF). Without these metrics, ablation results isolating SARS and SPDD contributions, or analysis of confounding factors like scene similarity, the experimental validation of the framework's effectiveness remains insufficient to substantiate the main result.
minor comments (3)
  1. [§3.2] §3.2 (SPDD description): The decomposition into 'coarse scene layout' and 'fine pose precision' is conceptually clear but would benefit from an explicit equation or pseudocode showing how the two distillation losses are formulated and combined (e.g., weighting between layout and precision terms).
  2. [§3.1] Abstract and §3.1 (SARS): The phrase 'maximally informative viewpoints' is used without a precise definition or selection criterion (e.g., based on pose entropy or coverage of the SE(3) manifold); adding this would improve reproducibility.
  3. Throughout: Notation for replay buffer size, scene indexing, and the continual training schedule (e.g., number of scenes, epochs per scene) should be introduced consistently with symbols defined in a table or early in §3.

Circularity Check

0 steps flagged

No significant circularity; derivation chain is self-contained

full rationale

The paper introduces SARS (Spatial-Aware Replay Strategy) and SPDD (Structure-Preserving Dual Distillation) as distinct, explicitly described mechanisms to mitigate layout-level and precision-level forgetting in sequential scene adaptation. No equation or claim reduces a result to its own inputs by construction, nor does any prediction rename a fitted parameter or rely on a self-citation chain for its core justification. The abstract and described framework treat the replay buffer construction and dual-distillation decomposition as independent engineering choices whose effectiveness is asserted via external experiments on public datasets. This matches the default expectation of a non-circular continual-learning proposal.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract provides no explicit free parameters, mathematical axioms, or invented physical entities; SARS and SPDD are methodological inventions whose effectiveness is asserted but not derived from prior equations or data in the provided text.

pith-pipeline@v0.9.0 · 5547 in / 1182 out tokens · 46464 ms · 2026-05-10T08:50:37.884713+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

47 extracted references · 47 canonical work pages

  1. [1]

    Active camera relocalization from a single reference image without hand-eye calibration,

    F.-P. Tian, W. Feng, Q. Zhang, X. Wang, J. Sun, V . Loia, and Z.-Q. Liu, “Active camera relocalization from a single reference image without hand-eye calibration,”IEEE Trans. Pattern Anal. Mach. Intell., vol. 41, no. 12, pp. 2791–2806, 2018

  2. [2]

    Automatic robot hand-eye calibration enabled by learning-based 3D vision,

    S. Li, X. Wang, H. Liang, and Y . Zheng, “Automatic robot hand-eye calibration enabled by learning-based 3D vision,”J. Intell. Robot. Syst., vol. 110, no. 2, p. 60, 2024

  3. [3]

    Nonsingular hand-eye and robot-world calibration for scara-type robots: A comparative study,

    G. Jin, X. Yu, Y . Chen, and J. Li, “Nonsingular hand-eye and robot-world calibration for scara-type robots: A comparative study,”IEEE Trans. Ind. Informat., vol. 21, no. 4, pp. 3057–3066, 2025

  4. [4]

    A calibrator fuzzy ensemble for highly-accurate robot arm calibration,

    X. Luo, Z. Li, W. Yue, and S. Li, “A calibrator fuzzy ensemble for highly-accurate robot arm calibration,”IEEE Trans. Neural Netw. & Learn. Syst., vol. 36, no. 2, pp. 2169–2181, 2024

  5. [5]

    From perception to precision: Vision-based mobile robotic ma- nipulation for assembly screwdriving,

    A. Stefanov, M. Zorman, S. ˇSlajpah, J. Podobnik, M. Mihelj, and M. Mu- nih, “From perception to precision: Vision-based mobile robotic ma- nipulation for assembly screwdriving,”Robot. Comput.-Integr. Manuf., vol. 98, p. 103148, 2026

  6. [6]

    Large and small-scale models’ fusion-driven proactive robotic manipulation control for human-robot collaborative assembly in industry 5.0,

    D. Ma, C. Zhang, Q. Xu, and G. Zhou, “Large and small-scale models’ fusion-driven proactive robotic manipulation control for human-robot collaborative assembly in industry 5.0,”Robot. Comput.-Integr. Manuf., vol. 97, p. 103078, 2026

  7. [7]

    Scalable and time-efficient bin- picking for unknown objects in dense clutter,

    P. Raj, L. Behera, and T. Sandhan, “Scalable and time-efficient bin- picking for unknown objects in dense clutter,”IEEE Trans. Autom. Sci. Eng., vol. 21, no. 3, pp. 2289–2301, 2023

  8. [8]

    A review of learning-based dynamics models for robotic manipulation,

    B. Ai, S. Tian, H. Shi, Y . Wang, T. Pfaff, C. Tan, H. I. Christensen, H. Su, J. Wu, and Y . Li, “A review of learning-based dynamics models for robotic manipulation,”Sci. Robot., vol. 10, no. 106, p. eadt1497, 2025

  9. [9]

    You only estimate once: Unified, one-stage, real-time category-level articulated object 6d pose estimation for robotic grasping,

    J. Huang, H. Lin, T. Wang, Y . Fu, Y .-G. Jiang, and X. Xue, “You only estimate once: Unified, one-stage, real-time category-level articulated object 6d pose estimation for robotic grasping,” inProc. IEEE Int. Conf. Robot. Autom.IEEE, 2025, pp. 14 044–14 051

  10. [10]

    Digital twin driven measurement in robotic flexible printed circuit assembly,

    M. Yang, Z. Huang, Y . Sun, Y . Zhao, R. Sun, Q. Sun, J. Chen, B. Qiang, J. Wang, and F. Sun, “Digital twin driven measurement in robotic flexible printed circuit assembly,”IEEE Trans. Instrum. Meas., vol. 72, pp. 1–12, 2023

  11. [11]

    Aligning cyber space with physical world: A comprehensive survey on embodied ai,

    Y . Liu, W. Chen, Y . Bai, X. Liang, G. Li, W. Gao, and L. Lin, “Aligning cyber space with physical world: A comprehensive survey on embodied ai,”IEEE/ASME Trans. Mechatronics, 2025

  12. [12]

    Accelerated coordi- nate encoding: Learning to relocalize in minutes using RGB and poses,

    E. Brachmann, T. Cavallari, and V . A. Prisacariu, “Accelerated coordi- nate encoding: Learning to relocalize in minutes using RGB and poses,” inProc. IEEE/CVF Conf. Comp. Vis. Patt. Recogn., 2023, pp. 5044– 5053

  13. [13]

    Glace: Global local accelerated coordinate encoding,

    F. Wang, X. Jiang, S. Galliani, C. V ogel, and M. Pollefeys, “Glace: Global local accelerated coordinate encoding,” inProc. IEEE/CVF Conf. Comp. Vis. Patt. Recogn., 2024, pp. 21 562–21 571

  14. [14]

    A comprehensive survey of continual learning: Theory, method and application,

    L. Wang, X. Zhang, H. Su, and J. Zhu, “A comprehensive survey of continual learning: Theory, method and application,”IEEE Trans. Pattern Anal. Mach. Intell., vol. 46, no. 8, pp. 5362–5383, 2024

  15. [15]

    Continual learning with knowledge distillation: A survey,

    S. Li, T. Su, X.-Y . Zhang, and Z. Wang, “Continual learning with knowledge distillation: A survey,”IEEE Trans. Neural Netw. & Learn. Syst., vol. 36, no. 6, pp. 9798–9818, 2024

  16. [16]

    Continual learning for image-based camera localization,

    S. Wang, Z. Laskar, I. Melekhov, X. Li, and J. Kannala, “Continual learning for image-based camera localization,” inProc. IEEE/CVF Int. Conf. Comp. Vis., 2021, pp. 3252–3262

  17. [17]

    Hierarchical scene coordinate classification and regression for visual localization,

    X. Li, S. Wang, Y . Zhao, J. Verbeek, and J. Kannala, “Hierarchical scene coordinate classification and regression for visual localization,” inProc. IEEE/CVF Conf. Comp. Vis. Patt. Recogn., 2020, pp. 11 983–11 992

  18. [18]

    Continual learning approaches to hand–eye calibration in robots,

    O. Bahadir, J. P. Siebert, and G. Aragon-Camarasa, “Continual learning approaches to hand–eye calibration in robots,”Mach. Vis. Appl., vol. 35, no. 4, p. 97, 2024

  19. [19]

    A new technique for fully autonomous and efficient 3 d robotics hand/eye calibration,

    R. Y . Tsai, R. K. Lenzet al., “A new technique for fully autonomous and efficient 3 d robotics hand/eye calibration,”IEEE Trans. Robot. Autom., vol. 5, no. 3, pp. 345–358, 1989

  20. [20]

    EasyHeC: Accurate and automatic hand-eye calibration via differentiable rendering and space exploration,

    L. Chen, Y . Shi, A. Yang, L. Zhang, J. Wang, and H. Lu, “EasyHeC: Accurate and automatic hand-eye calibration via differentiable rendering and space exploration,” inIEEE Robot. Autom. Lett., vol. 8, no. 10, 2023, pp. 6301–6308

  21. [21]

    Easyhec++: Fully automatic hand- eye calibration with pretrained image models,

    Z. Hong, K. Zheng, and L. Chen, “Easyhec++: Fully automatic hand- eye calibration with pretrained image models,” inIEEE/RSJ Int. Conf. Intell. Robots Syst.IEEE, 2024, pp. 816–823

  22. [22]

    A certifiably correct algorithm for gener- alized robot-world and hand-eye calibration,

    E. Wise, P. Kaveti, Q. Cheng, W. Wang, H. Singh, J. Kelly, D. M. Rosen, and M. Giamou, “A certifiably correct algorithm for gener- alized robot-world and hand-eye calibration,”Int. J. Robot. Res., p. 02783649261420308, 2026

  23. [23]

    DSAC–differentiable RANSAC for camera localization,

    E. Brachmann, A. Krull, S. Nowozin, J. Shotton, F. Michel, S. Gumhold, and C. Rother, “DSAC–differentiable RANSAC for camera localization,” inProc. IEEE Conf. Comp. Vis. Patt. Recogn., 2017, pp. 6684–6692

  24. [24]

    Learning less is more–6D camera localization via 3D surface regression,

    E. Brachmann and C. Rother, “Learning less is more–6D camera localization via 3D surface regression,” inProc. IEEE Conf. Comp. Vis. Patt. Recogn., 2018, pp. 4654–4662

  25. [25]

    Expert sample consensus applied to camera re-localization,

    E. Brachmann and C. Rother, “Expert sample consensus applied to camera re-localization,” inProc. IEEE/CVF Int. Conf. Comp. Vis., 2019, pp. 7525–7534

  26. [26]

    R-score: Revisiting scene coordinate regression for robust large-scale visual localization,

    X. Jiang, F. Wang, S. Galliani, C. V ogel, and M. Pollefeys, “R-score: Revisiting scene coordinate regression for robust large-scale visual localization,” inProc. IEEE/CVF Conf. Comp. Vis. Patt. Recogn., 2025, pp. 11 536–11 546

  27. [27]

    Reloc3r: Large-scale training of relative camera pose regression for generalizable, fast, and accurate visual localization,

    S. Dong, S. Wang, S. Liu, L. Cai, Q. Fan, J. Kannala, and Y . Yang, “Reloc3r: Large-scale training of relative camera pose regression for generalizable, fast, and accurate visual localization,” inProc. IEEE/CVF Conf. Comp. Vis. Patt. Recogn., 2025, pp. 16 739–16 752

  28. [28]

    Scene-agnostic pose regression for visual localization,

    J. Zheng, R. Liu, Y . Chen, Z. Chen, K. Yang, J. Zhang, and R. Stiefel- hagen, “Scene-agnostic pose regression for visual localization,” inProc. IEEE/CVF Conf. Comp. Vis. Patt. Recogn., 2025, pp. 27 092–27 102

  29. [29]

    Hvlf: A holistic visual localization framework across diverse scenes,

    K. Dai, Z. Jiang, F. Qiu, D. Liu, T. Xie, K. Wang, R. Li, and L. Zhao, “Hvlf: A holistic visual localization framework across diverse scenes,” IEEE Trans. Neural Netw. & Learn. Syst., 2025

  30. [30]

    Efficiently identifying task groupings for multi-task learning,

    C. Fifty, E. Amid, Z. Zhao, T. Yu, R. Anil, and C. Finn, “Efficiently identifying task groupings for multi-task learning,”Advances in Neural Inf. Process. Syst., vol. 34, pp. 27 503–27 516, 2021

  31. [31]

    Data poisoning attacks on federated machine learning,

    G. Sun, Y . Cong, J. Dong, Q. Wang, L. Lyu, and J. Liu, “Data poisoning attacks on federated machine learning,”IEEE Internet Things J., vol. 9, no. 13, pp. 11 365–11 375, 2021

  32. [32]

    A continual learning survey: Defying forgetting in classification tasks,

    M. De Lange, R. Aljundi, M. Masana, S. Parisot, X. Jia, A. Leonardis, G. Slabaugh, and T. Tuytelaars, “A continual learning survey: Defying forgetting in classification tasks,”IEEE Trans. Pattern Anal. Mach. Intell., vol. 44, no. 7, pp. 3366–3385, 2021

  33. [33]

    Create your world: Lifelong text-to-image diffusion,

    G. Sun, W. Liang, J. Dong, J. Li, Z. Ding, and Y . Cong, “Create your world: Lifelong text-to-image diffusion,”IEEE Trans. Pattern Anal. Mach. Intell., vol. 46, no. 9, pp. 6454–6470, 2024

  34. [34]

    arXiv preprint arXiv:2403.00336 , year=

    W. Liang, G. Sun, Y . He, Y . Ren, J. Dong, and Y . Cong, “Never- ending behavior-cloning agent for robotic manipulation,”arXiv preprint arXiv:2403.00336, 2024

  35. [35]

    icarl: Incremental classifier and representation learning,

    S.-A. Rebuffi, A. Kolesnikov, G. Sperl, and C. H. Lampert, “icarl: Incremental classifier and representation learning,” inProc. IEEE Conf. Comp. Vis. Patt. Recogn., 2017, pp. 2001–2010

  36. [36]

    Overcoming catastrophic forgetting in neural networks,

    J. Kirkpatrick, R. Pascanu, N. Rabinowitz, J. Veness, G. Desjardins, A. A. Rusu, K. Milan, J. Quan, T. Ramalho, A. Grabska-Barwinska et al., “Overcoming catastrophic forgetting in neural networks,” inProc. Natl. Acad. Sci. U.S.A., vol. 114, no. 13, 2017, pp. 3521–3526

  37. [37]

    Helpful or harmful: Inter-task association in continual learning,

    H. Jin and E. Kim, “Helpful or harmful: Inter-task association in continual learning,” inProc. Eur. Conf. Comp. Vis.Springer, 2022, pp. 519–535

  38. [38]

    Instruction-grounded visual projectors for continual learning of generative vision-language models,

    H. Jin, H. J. Chang, and E. Kim, “Instruction-grounded visual projectors for continual learning of generative vision-language models,” inProc. IEEE/CVF Int. Conf. Comp. Vis., 2025, pp. 3466–3476

  39. [39]

    Gradient based sample selection for online continual learning,

    R. Aljundi, M. Lin, B. Goujaud, and Y . Bengio, “Gradient based sample selection for online continual learning,” inAdvances in Neural Inf. Process. Syst., vol. 32, 2019

  40. [40]

    Online continual learning with maximal interfered retrieval,

    R. Aljundi, E. Belilovsky, T. Tuytelaars, L. Charlin, M. Caccia, M. Lin, and L. Page-Caccia, “Online continual learning with maximal interfered retrieval,” inAdvances in Neural Inf. Process. Syst., vol. 32, 2019

  41. [41]

    Co 2L: Contrastive continual learning,

    H. Cha, J. Lee, and J. Shin, “Co 2L: Contrastive continual learning,” in Proc. IEEE/CVF Int. Conf. Comp. Vis., 2021, pp. 9516–9525

  42. [42]

    Gradient-guided epsilon constraint method for online continual learn- ing,

    S. Lai, C. Ma, F. Zhu, Z. Zhao, X. Lin, G. Meng, and Q. Zhang, “Gradient-guided epsilon constraint method for online continual learn- ing,” inAdvances in Neural Inf. Process. Syst., 2025

  43. [43]

    Class- wise balancing data replay for federated class-incremental learning,

    Z. Qi, Y .-P. Tang, L. Meng, H. Yu, X. Li, and X. Meng, “Class- wise balancing data replay for federated class-incremental learning,” in Advances in Neural Inf. Process. Syst., 2025. 13

  44. [44]

    Parallel poisson disk sampling,

    L.-Y . Wei, “Parallel poisson disk sampling,”ACM Trans. Graph., vol. 27, no. 3, pp. 1–9, 2008

  45. [45]

    Scene coordinate regression forests for camera relocalization in RGB-D images,

    J. Shotton, B. Glocker, C. Zach, S. Izadi, A. Criminisi, and A. Fitzgib- bon, “Scene coordinate regression forests for camera relocalization in RGB-D images,” inProc. IEEE Conf. Comp. Vis. Patt. Recogn., 2013, pp. 2930–2937

  46. [46]

    Learning to navigate the energy landscape,

    J. Valentin, A. Dai, M. Nießner, P. Kohli, P. Torr, S. Izadi, and C. Keskin, “Learning to navigate the energy landscape,” inProc. Int. Conf. 3D Vis. IEEE, 2016, pp. 323–332

  47. [47]

    Don’t forget, there is more than forgetting: new metrics for continual learning,

    N. D ´ıaz-Rodr´ıguez, V . Lomonaco, D. Filliat, and D. Maltoni, “Don’t forget, there is more than forgetting: new metrics for continual learning,” inProc. NeurIPS Workshop on Continual Learning, 2018