Pith. sign in

REVIEW 4 major objections 4 minor 118 references

LTLZinc: a Benchmarking Framework for Continual Learning and Neuro-Symbolic Temporal Reasoning

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read LTLZinc establishes that an LTLf formula over MiniZinc constraints and arbitrary image domains is enough to generate expressive, fine-grained temporal reasoning and continual learning datasets, and that current neuro-symbolic and…

desk verdict Solid benchmark generator with a fixable semantics bug in Section 2; worth reviewing, conditional accept. read the letter →

arxiv 2507.17482 v1 pith:Q43W7JUK submitted 2025-07-23 cs.AI

Luca Salvatore Lorello , Nikolaos Manginas , Marco Lippi , Stefano Melacci This is my paper · ORCID
classification cs.AI
keywords LTLZinclineartemporallogiconfinitetracesneuro-symbolicAIcontinuallearningdatasetgenerationreasoningconstraintsatisfactionsequenceclassification
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

LTLZinc is a dataset generator that starts from a linear temporal logic formula over finite traces (LTLf), written on top of MiniZinc constraints between symbolic labels, and produces either labeled sequences of images (sequential mode) or sequences of learning episodes (incremental mode). The same specification can be instantiated over any image classification domain, and each timestep carries image labels, constraint truth values, automaton states, and a sequence-level label. The paper reports that six neuro-symbolic sequence classification tasks and four class-continual learning tasks built this way are hard for current methods: purely neural pipelines often fail on the temporal objective, and continual learners without injected temporal knowledge lose rare classes. The point of the framework is to give the community a single, customizable source of temporal benchmarks with controllable relational constraints and multiple supervision levels.

What carries the argument

The load-bearing object is the symbolic finite automaton obtained from the LTLf specification—a finite automaton whose transition guards are expressions over the user's MiniZinc constraints rather than single alphabet symbols. LTLZinc compiles the formula into this automaton once, then generates every dataset by random walks over it, fixing the sequence label by whether the walk ends in an accepting state. For each transition the generator builds and solves a MiniZinc constraint satisfaction problem that makes the symbolic labels satisfy the guard, caches all solutions, and samples from the cache to fill timesteps; this cache is what turns a potentially exponential per-sample cost into a constant-time draw after a one-time per-transition solve. The same automaton also serves as background knowledge and as the annotation source for automaton-state traces, which the experiments use for next-state prediction and for knowledge-unit injection in continual learning.

What would settle it

Generate a small dataset from a short LTLZinc formula, then independently recheck every sequence label with a different LTLf model checker or by exhaustive enumeration of the constraint alphabet; any mismatch would show the automaton conversion or the constraint solving is wrong.

Watch

Extended reading notes

Core claim

On its own terms, the paper's claim is that a user-defined temporal specification—an LTLf formula whose atomic propositions are MiniZinc constraints over symbolic labels—is enough to define a benchmark family with a precise semantics. LTLZinc converts the formula into a symbolic finite automaton, samples random walks of chosen length from the start state, and labels a sequence positive exactly when its walk ends in an accepting state; each transition's guard is turned into a constraint satisfaction problem whose solutions are cached and sampled to populate the timesteps with concrete images and labels. In sequential mode the output is a dataset of positive and negative sequences with per-timestep annotations, and in incremental mode the single positive walk becomes a curriculum of episodes that shares the same temporal property across different random seeds. The experimental sections argue that these generated tasks separate methods: neural-only pipelines have low temporal accuracy, symbolic components help but are harder to optimize, and in class-continual tasks knowledge of the temporal structure (predicate or automaton-state units) is what prevents rare classes from being forgotten. The central discovery is therefore a generative reduction: temporal reasoning and continual learning benchmarks can be produced from a compact declarative specification rather than hand-crafted per-task code.

Load-bearing premise

The whole generator trusts that turning an LTLf formula into an automaton and solving every automaton transition's constraint problem is done correctly, because every sequence label and curriculum is derived from those two steps.

Editorial extensions

If this is right

  • Given one LTLf formula and one image domain, a researcher can generate many datasets with different lengths, positive/negative balance, and—in incremental mode—different curricula that all satisfy the same temporal specification.
  • Because every timestep is annotated with image labels, constraint values, and automaton state, the same generated data supports end-to-end sequence classification, temporally distant supervision with intermediate annotations, and constraint induction over time.
  • If the experimental results hold, purely neural sequence models are not enough for relational-temporal tasks, and replacing or augmenting neural modules with symbolic constraint reasoning and automaton knowledge improves temporal accuracy while being harder to optimize.
  • In class-continual tasks, injecting temporal background knowledge (which predicates or automaton states are active in an episode) is what prevents catastrophic forgetting of rare classes that appear once.
  • The released generator and ten ready-to-use tasks give other researchers a shared testbed for comparing neuro-symbolic and continual learning methods under the same temporal specifications.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: because the perceptual domain is decoupled from the temporal specification, one could hold a formula fixed and swap image difficulty, isolating whether failures come from perception or from temporal reasoning—an experiment the paper reports only partially.
  • Editorial extension: the constraint cache implies generation cost depends on the number of automaton transitions, not the dataset size; the practical ceiling is the automaton blow-up of the LTLf-to-SFA conversion, so very long or heavily nested formulas are where the generator will first slow down.
  • Editorial extension: a testable use of the framework would be to generate negative sequences that violate the formula only in one controlled way (wrong timing rather than wrong constraints), and probe whether neuro-symbolic models learn the temporal rule or a shallower correlation.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper introduces LTLZinc, a framework that generates temporal reasoning and continual learning datasets from LTLf specifications over MiniZinc constraints, grounded in arbitrary image classification datasets. The generation pipeline converts an LTLf formula into a symbolic finite automaton via the flloat compiler, solves transition-guard constraint satisfaction problems with a cache, and produces either labeled sequences (sequential mode) or multi-episode curricula (incremental mode). The authors release the generator and ten tasks, and report experiments with modular neuro-symbolic pipelines on six sequence classification tasks and with continual learning strategies on four class-continual tasks. The central claims are that LTLZinc can produce expressive, correctly annotated temporal benchmarks and that current methods struggle on them.

Significance. If the generation pipeline is correct, LTLZinc fills a real gap: it combines temporal logic, relational constraints, and arbitrary perceptual domains, with per-timestep annotations that support several training regimes, and it can generate curricula automatically from a single formula. The paper's strengths include a clearly described generator architecture, a public codebase, use of off-the-shelf components (flloat, MiniZinc, Scallop, ProbLog), extensive experiments across two benchmark families, and explicit release of the ten tasks. However, the manuscript currently contains a load-bearing inconsistency in the formal semantics of Until, and the experimental claims rest on runs that are discarded without counts and on mean values computed over as few as one surviving run. These issues need to be resolved before the framework and its benchmark results can be fully trusted.

major comments (4)
  1. [§2 and §8.1.1] The printed semantics of Until in §2 are not the standard LTLf semantics and are inconsistent with the generated tasks. The definition requires that for some t′ ≤ |π|, π,t′ |= φ2 and for all t′′ ≤ t′: π,t′′ |= φ1; this makes φ1 mandatory at the same position as φ2. Under this definition, the class-continual Task 1 formula in §8.1.1, F : ¬zero ∧ (¬zero U (zero ∧ e □¬zero)), is unsatisfiable, yet the paper reports generated positive datasets and experimental results for it. The implementation must therefore be following standard LTLf semantics (as flloat does), not the semantics printed in the manuscript. The authors must correct the formal definition to the standard reading (φ1 required for positions strictly before φ2) and state explicitly which semantics the generated datasets obey.
  2. [§4 and §7/§8] The correctness of every generated label and curriculum rests on two unverified links: the flloat LTLf-to-SFA conversion and the MiniZinc transition-guard cache. The paper reports no independent validation of the generated labels—no second LTLf checker, no brute-force audit of transition guards, no re-checking of sampled traces against the original formula. Since a miscompiled transition or an incorrectly reified constraint would invalidate all positive/negative labels and all curricula, the authors should add and document such validation (e.g., randomized trace re-checking with an independent tool, unit tests for the guard cache, and a statement of which compiler version and MiniZinc configuration were used).
  3. [Appendix A.4 and Tables 1–2] The experimental results discard diverging runs without reporting how many runs were discarded, making the reported means and standard deviations unreliable. Appendix A.4 states that runs diverging due to NaN or infinite gradients are discarded, and the text around Figure 3 and Table 1 admits that zero variance is often due to only one experiment converging. For example, Table 1 Task 6 reports 0.60±0.00 and 0.50±0.00 for several configurations, and Task 5 shows very large standard deviations consistent with one or two surviving runs. The authors should report the number of successful runs for every reported cell, analyze whether divergences are systematic across methods, and either include the failed runs in the comparison or justify their exclusion.
  4. [§4, sequence sampling] The description of negative-sequence generation is underspecified with respect to the backtracking and length-lowering procedure. The text says that when no path of the target length satisfies the target label, the generator lowers the sequence length until the minimum is reached; it is not clear whether a negative label is always verified against the automaton by the same acceptance test used for positives, or whether a trace that becomes accepting at a shorter length can be relabeled as positive. The authors should specify the exact acceptance/labeling rule and ensure that negative labels are defined by the same LTLf semantics as positive labels.
minor comments (4)
  1. [Table 2] Table 2 contains two rows both labeled "Task 2(long)"; the second of these rows appears to correspond to Task 3(long). Please correct the labels.
  2. [§8.1.1] The displayed formula for Task 1 appears to have unbalanced parentheses ("¬zero ∧ (¬zero U (zero ∧ e □¬zero)."), and the weak-next operator is written without clarifying parentheses. Please reformat the formula so that its parse is unambiguous.
  3. [§3] There is a typo in the Core50 description: "contidions" should be "conditions".
  4. [Figure 2] The figure legend says "green: symbolic, reddish: neural", but the plots in Figures 3 and 4 use different colors and markers; please align the color/marker conventions between the figure caption and the text.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: generator and benchmarks are self-contained; noted risks are correctness issues, not circularity.

full rationale

LTLZinc's central claim is that it turns an LTLf formula over MiniZinc constraints into labeled sequential or incremental datasets. That transformation is not derived from the paper's own outputs: sequential labels are produced by random walks over an SFA obtained from the formula via an off-the-shelf compiler (flloat), and each transition guard is solved as a MiniZinc constraint satisfaction problem and cached. These are independent computational steps, not fitted parameters renamed as predictions. The experimental sections evaluate methods on held-out validation and test splits, with model selection on validation accuracy; no test-set quantity is reverse-engineered into the generator. The paper's self-citations [41,42,43] motivate the continual-learning setting and provide the KANDY comparison, but they do not carry the LTLf-to-SFA conversion, the constraint solving, or the benchmark generation; those rest on external tools and the paper's own code release. The main risks are correctness risks rather than circularity: the Until clause printed in Section 2 ('for all t'' ≤ t': π,t'' |= φ1') is inconsistent with the class-continual Task 1 formula ¬zero ∧ (¬zero U (zero ∧ e□¬zero)), which would be unsatisfiable under the printed semantics, and the paper does not independently validate generated labels against a second LTLf checker. Section 4 also explicitly limits its complexity analysis to an informal discussion. These are verification gaps and semantic inconsistencies, not self-definitional reductions or fitted-input predictions, so they do not raise the circularity score.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The framework itself introduces no mathematical axioms beyond standard LTLf and constraint programming semantics. The listed free parameters are experimental hyperparameters that affect the reported accuracies, not the correctness of the generator.

free parameters (4)
  • Training loss weights (sequence classification) = λic=0.1, λcc=λnsp=λsc=1.0
    Chosen by hand and not swept; affects convergence and accuracy of every reported model.
  • Pre-training and training epochs = 1 pre-training epoch; 20 or 50 epochs, best by validation
    The number of epochs changes the results (see epochs markers in Figures 3 and 4); selection on validation is standard but a free choice.
  • Replay buffer size and batch = buffer_size=500, buffer_batch=16
    Hand-set for continual learning; directly bounds how much past data is retained.
  • Temperature calibration parameters (CC/NSP) = learned per module
    These are learned parameters that adjust entropy and confidence; they are part of the method and affect all numeric accuracies.
assumptions (4)
  • domain assumption LTLf-to-SFA compilation is correct (via flloat compiler)
    Generation relies on an off-the-shelf compiler to build the automaton used for sampling and labeling; an error would invalidate all sequence labels.
  • domain assumption MiniZinc constraint solver correctly enumerates all solutions for transition guards
    The constraint cache stores solutions per transition; if a solution violates the guard, sample-label coherence breaks.
  • domain assumption Neural and symbolic modules receive coherent, correct annotations
    IC pretraining and CC programs assume ground-truth labels and constraint semantics as defined; independence violations are discussed by the authors.
  • domain assumption Reported random seeds and hardware do not materially change qualitative conclusions
    Only 3 seeds and 9 runs are used; the paper assumes variance is small enough to support its claims.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LTLZinc: a Benchmarking Framework for Continual Learning and Neuro-Symbolic Temporal Reasoning." pith.science (2026). https://pith.science/paper/Q43W7JUK

@misc{pith2026250717482,
  author       = {Pith},
  title        = {Pith review of: LTLZinc: a Benchmarking Framework for Continual Learning and Neuro-Symbolic Temporal Reasoning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Q43W7JUK}},
  note         = {Machine review of arXiv:2507.17482}
}
read the original abstract

Neuro-symbolic artificial intelligence aims to combine neural architectures with symbolic approaches that can represent knowledge in a human-interpretable formalism. Continual learning concerns with agents that expand their knowledge over time, improving their skills while avoiding to forget previously learned concepts. Most of the existing approaches for neuro-symbolic artificial intelligence are applied to static scenarios only, and the challenging setting where reasoning along the temporal dimension is necessary has been seldom explored. In this work we introduce LTLZinc, a benchmarking framework that can be used to generate datasets covering a variety of different problems, against which neuro-symbolic and continual learning methods can be evaluated along the temporal and constraint-driven dimensions. Our framework generates expressive temporal reasoning and continual learning tasks from a linear temporal logic specification over MiniZinc constraints, and arbitrary image classification datasets. Fine-grained annotations allow multiple neural and neuro-symbolic training settings on the same generated datasets. Experiments on six neuro-symbolic sequence classification and four class-continual learning tasks generated by LTLZinc, demonstrate the challenging nature of temporal learning and reasoning, and highlight limitations of current state-of-the-art methods. We release the LTLZinc generator and ten ready-to-use tasks to the neuro-symbolic and continual learning communities, in the hope of fostering research towards unified temporal learning and reasoning frameworks.

Figures

Figures reproduced from arXiv: 2507.17482 by the authors.

Figure 1
Figure 1. A symbolic automaton for the ltlf formula ♢r ∧ ((p ↔ ⃝q) U r). As {p, q, r} can only assume values in {⊤, ⊥}, this is in practice a condensed representation of a deterministic finite state automaton with transitions defined over an alphabet of 23 symbols. 21 [PITH_FULL_IMAGE:figures/full_fig_p021_1.png] view at source ↗
Figure 2
Figure 2. Stages of our architecture for sequence classification experiments. [PITH_FULL_IMAGE:figures/full_fig_p030_2.png] view at source ↗
Figure 3
Figure 3. cc-nsp accuracy trade-off on short-sequence experiments for dif￾ferent categories. 37 [PITH_FULL_IMAGE:figures/full_fig_p037_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: cc-nsp accuracy trade-off on long-sequence experiments for dif￾ferent categories. markers were in the upper right corner for short sequences, while they are in the lower left in this case), however in this task Scallop is mostly unaf￾fected (performance is only subject…
Figure 5
Figure 5. Figure 5: Average accuracy and Average forgetting for class-continual ex [PITH_FULL_IMAGE:figures/full_fig_p048_5.png]
Figure 6
Figure 6. Figure 6: Average accuracy and Average forgetting for class-continual exper [PITH_FULL_IMAGE:figures/full_fig_p049_6.png]
Figure 7
Figure 7. Figure 7: Average accuracy and Average forgetting for class-continual exper [PITH_FULL_IMAGE:figures/full_fig_p051_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

118 extracted references · 74 canonical work pages

  1. [1]

    P., and Singh, S

    Abel, D., Barreto, A., V an Roy, B., Precup, D., van Hasselt, H. P., and Singh, S. A definition of continual reinforcement learning. Advances in Neural Information Processing Systems 36 (2023), 50377– 50407

    2023

  2. [2]

    Task-free continual learning

    Aljundi, R., Kelchtermans, K., and Tuytelaars, T. Task-free continual learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (2019), pp. 11254–11263

    2019

  3. [3]

    Gradient based sample selection for online continual learning

    Aljundi, R., Lin, M., Goujaud, B., and Bengio, Y. Gradient based sample selection for online continual learning. Advances in neural information processing systems 32 (2019)

    2019

  4. [4]

    d., Serafini, L., and Spranger, M

    Badreddine, S., Garcez, A. d., Serafini, L., and Spranger, M. Logic tensor networks. Artificial Intelligence 303 (2022), 103649

    2022

  5. [5]

    Rainbow memory: Continual learning with a memory of diverse samples

    Bang, J., Kim, H., Yoo, Y., Ha, J.-W., and Choi, J. Rainbow memory: Continual learning with a memory of diverse samples. In Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition (2021), pp. 8218–8227

    2021

  6. [6]

    R., d’A vila Garcez, A., Bader, S., Bowman, H., Domingos, P., Hitzler, P., K ¨uhnberger, K.-U., Lamb, L

    Besold, T. R., d’A vila Garcez, A., Bader, S., Bowman, H., Domingos, P., Hitzler, P., K ¨uhnberger, K.-U., Lamb, L. C., 52 Lima, P. M. V., de Penning, L., et al. Neural-symbolic learn- ing and reasoning: A survey and interpretation 1. In Neuro-Symbolic Artificial Intelligence: The State of the Art . IOS press, 2021, pp. 1–51

    2021

  7. [7]

    Where is the Truth? The Risk of Getting Confounded in a Continual World

    Busch, F. P., Kamath, R., Mitchell, R., Stammer, W., Ker- sting, K., and Mundt, M. Where is the truth? the risk of get- ting confounded in a continual world. arXiv preprint arXiv:2402.06434 (2024)

    work page Pith review arXiv 2024

  8. [8]

    Neuro-symbolic artificial intelligence for safety engineering

    Carnevali, L., and Lippi, M. Neuro-symbolic artificial intelligence for safety engineering. In International Conference on Computer Safety, Reliability, and Security (2024), Springer, pp. 438–445

    2024

Show all 118 references
  1. [9]

    Pitfalls in processing infinite-length sequences with pop- ular approaches for sequential data

    Casoni, M., Guidi, T., Tiezzi, M., Betti, A., Gori, M., and Melacci, S. Pitfalls in processing infinite-length sequences with pop- ular approaches for sequential data. In IAPR Workshop on Artificial Neural Networks in Pattern Recognition (2024), Springer, pp. 37–48

    2024

  2. [10]

    On probabilistic inference by weighted model counting

    Chavira, M., and Darwiche, A. On probabilistic inference by weighted model counting. Artificial Intelligence 172 , 6-7 (2008), 772– 799

    2008

  3. [11]

    Towards neuro-symbolic video understanding

    Choi, M., Goel, H., Omama, M., Yang, Y., Shah, S., and Chin- chali, S. Towards neuro-symbolic video understanding. In European Conference on Computer Vision (2024), Springer, pp. 220–236

    2024

  4. [12]

    Online continual learning from imbalanced data

    Chrysakis, A., and Moens, M.-F. Online continual learning from imbalanced data. In International Conference on Machine Learning (2020), PMLR, pp. 1952–1961

    2020

  5. [13]

    Timebench: A comprehensive evaluation of temporal reasoning abilities in large language models

    Chu, Z., Chen, J., Chen, Q., Yu, W., W ang, H., Liu, M., and Qin, B. Timebench: A comprehensive evaluation of temporal reasoning abilities in large language models. arXiv preprint arXiv:2311.17667 (2023)

    2023 arXiv

  6. [14]

    Nusmv 2: An opensource tool for symbolic model checking

    Cimatti, A., Clarke, E., Giunchiglia, E., Giunchiglia, F., Pistore, M., Roveri, M., Sebastiani, R., and Tacchella, A. Nusmv 2: An opensource tool for symbolic model checking. In Computer Aided Verification: 14th International Conference, CA V 2002 Copenhagen, Denmark, July 27–...

    2002

  7. [15]

    A knowledge compilation map

    Darwiche, A., and Marquis, P. A knowledge compilation map. Journal of Artificial Intelligence Research 17 (2002), 229–264. 53

    2002

  8. [16]

    Y.Linear temporal logic and linear dynamic logic on finite traces

    De Giacomo, G., and V ardi, M. Y.Linear temporal logic and linear dynamic logic on finite traces. In Ijcai (2013), vol. 13, pp. 854–860

    2013

  9. [17]

    A contin- ual learning survey: Defying forgetting in classification tasks

    De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., and Tuytelaars, T. A contin- ual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44 , 7 (2021), 3366–3385

    2021

  10. [18]

    From statistical relational to neuro-symbolic artificial intelligence

    De Raedt, L., Dumancic, S., Manhaeve, R., and Marra, G. From statistical relational to neuro-symbolic artificial intelligence. In Twenty-Ninth International Joint Conference on Artificial Intelligence (IJCAI 2020), Yokohama, Japan, Januray 7-15, 2021. (2021), ijcai. org, pp. 4943–4950

    2021

  11. [19]

    Synthesis lectures on artificial intelligence and machine learning 10, 2 (2016), 1–189

    De Raedt, L., Kersting, K., Natarajan, S., and Poole, D.Sta- tistical relational artificial intelligence: Logic, probability, and compu- tation. Synthesis lectures on artificial intelligence and machine learning 10, 2 (2016), 1–189

    2016

  12. [20]

    Problog: A prob- abilistic prolog and its application in link discovery

    De Raedt, L., Kimmig, A., and Toivonen, H. Problog: A prob- abilistic prolog and its application in link discovery. In IJCAI 2007, Proceedings of the 20th international joint conference on artificial intel- ligence (2007), IJCAI-INT JOINT CONF ARTIF INTELL, pp. 2462– 2467

    2007

  13. [21]

    B., A vrunin, G

    Dwyer, M. B., A vrunin, G. S., and Corbett, J. C. Property specification patterns for finite-state verification. In Proceedings of the second workshop on Formal methods in software practice (1998), pp. 7– 15

    1998

  14. [22]

    M., Trzcinski, T., Tuytelaars, T., and Bethge, M

    Dziadzio, S., C ¸ agatay Yildiz, van de Ven, G. M., Trzcinski, T., Tuytelaars, T., and Bethge, M. Infinite dsprites for disen- tangled continual learning: Separating memory edits from generaliza- tion. In Proceedings of The 3rd Conference on Lifelong Learning Agents (2024), vo...

    2024

  15. [23]

    Neural computation 27, 2 (2015), 388–480

    Gnecco, G., Gori, M., Melacci, S., and Sanguineti, M.Founda- tions of support constraint machines. Neural computation 27, 2 (2015), 388–480. 54

    2015

  16. [24]

    Position Paper: Collectionless Artificial Intelligence

    Gori, M., and Melacci, S. Position Paper: Collectionless Artificial Intelligence. arXiv preprint arXiv:2309.06938 (to be presented at the IJCNN 2025 conference) (2023)

    2023 arXiv

  17. [25]

    Task-incremental learning on long text sequences

    Graziuso, N., Zugarini, A., and Melacci, S. Task-incremental learning on long text sequences. In Proceedings of the Tenth Italian Conference on Computational Linguistics (CLiC-it 2024), volume TBA of CEUR Workshop Proceedings (CEUR-WS. org, ISSN 1613-0073), page TBA. CEUR (2024)

    2024

  18. [26]

    J., Karvounarakis, G., and Tannen, V

    Green, T. J., Karvounarakis, G., and Tannen, V. Provenance semirings. In Proceedings of the twenty-sixth ACM SIGMOD-SIGACT- SIGART symposium on Principles of database systems (2007), pp. 31– 40

    2007

  19. [27]

    U., Rabe, M

    Hahn, C., Schmitt, F., Kreber, J. U., Rabe, M. N., and Finkbeiner, B. Teaching temporal logics to neural networks. arXiv preprint arXiv:2003.04218 (2020)

    2020 arXiv

  20. [28]

    Class-incremental learning with repetition

    Hemati, H., Cossu, A., Carta, A., Hurtado, J., Pellegrini, L., Bacciu, D., Lomonaco, V., and Borth, D. Class-incremental learning with repetition. In Proceedings of The 2nd Conference on Lifelong Learning Agents (2023), vol. 232 of Proceedings of Machine Learning Research, PML...

    2023

  21. [29]

    Chain-of-thought textual reasoning for few-shot temporal action localization

    Ji, H., Yun, W., Qi, M., and Ma, H. Chain-of-thought textual reasoning for few-shot temporal action localization. arXiv preprint arXiv:2504.13460 (2025)

    2025

  22. [30]

    Algebraic model counting

    Kimmig, A., V an den Broeck, G., and De Raedt, L. Algebraic model counting. Journal of Applied Logic 22 (2017), 46–62

    2017

  23. [31]

    Learning multiple layers of features from tiny images

    Krizhevsky, A., Hinton, G., et al. Learning multiple layers of features from tiny images

  24. [32]

    Gradient- based learning applied to document recognition

    LeCun, Y., Bottou, L., Bengio, Y., and Haffner, P. Gradient- based learning applied to document recognition. Proceedings of the IEEE 86 , 11 (1998), 2278–2324

    1998

  25. [33]

    R., Charlin, L., and Rish, I

    Lesort, T., Ostapenko, O., Rodr´ıguez, P., Misra, D., Arefin, M. R., Charlin, L., and Rish, I. Challenging common assumptions about catastrophic forgetting and knowledge accumulation. In Pro- ceedings of The 2nd Conference on Lifelong Learning Agents (2023), 55 vol. 232 of Pro...

    2023

  26. [34]

    Interpretable deep learning: Interpretation, inter- pretability, trustworthiness, and beyond

    Li, X., Xiong, H., Li, X., Wu, X., Zhang, X., Liu, J., Bian, J., and Dou, D. Interpretable deep learning: Interpretation, inter- pretability, trustworthiness, and beyond. Knowledge and Information Systems 64 , 12 (2022), 3197–3234

    2022

  27. [35]

    Learning without forgetting

    Li, Z., and Hoiem, D. Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40 , 12 (2017), 2935–2947

    2017

  28. [36]

    Scallop: A language for neu- rosymbolic programming

    Li, Z., Huang, J., and Naik, M. Scallop: A language for neu- rosymbolic programming. Proceedings of the ACM on Programming Languages 7, PLDI (2023), 1463–1487

    2023

  29. [37]

    The clear bench- mark: Continual learning on real-world imagery

    Lin, Z., Shi, J., Pathak, D., and Ramanan, D. The clear bench- mark: Continual learning on real-world imagery. In Thirty-fifth confer- ence on neural information processing systems datasets and benchmarks track (round 2) (2021)

    2021

  30. [38]

    Core50: a new dataset and bench- mark for continuous object recognition

    Lomonaco, V., and Maltoni, D. Core50: a new dataset and bench- mark for continuous object recognition. In Conference on robot learning (2017), PMLR, pp. 17–26

    2017

  31. [39]

    L., De Lange, M., Masana, M., Pomponi, J., V an de Ven, G

    Lomonaco, V., Pellegrini, L., Cossu, A., Carta, A., Graffi- eti, G., Hayes, T. L., De Lange, M., Masana, M., Pomponi, J., V an de Ven, G. M., et al. Avalanche: an end-to-end library for continual learning. In Proceedings of the IEEE/CVF conference on computer vision and patter...

    2021

  32. [40]

    Gradient episodic memory for continual learning

    Lopez-Paz, D., and Ranzato, M. Gradient episodic memory for continual learning. Advances in neural information processing systems 30 (2017)

    2017

  33. [41]

    S., Lippi, M., and Melacci, S

    Lorello, L. S., Lippi, M., and Melacci, S. Continual learning for unsupervised concept bottleneck discovery. In Proceedings of The 3rd Conference on Lifelong Learning Agents (2024), vol. 274 of Proceedings of Machine Learning Research, PMLR, pp. 597–619

    2024

  34. [42]

    S., Lippi, M., and Melacci, S

    Lorello, L. S., Lippi, M., and Melacci, S. The kandy benchmark: Incremental neuro-symbolic learning and reasoning with kandinsky pat- terns. arXiv preprint arXiv:2402.17431 (2024). 56

    2024 arXiv

  35. [43]

    S., Lippi, M., and Melacci, S

    Lorello, L. S., Lippi, M., and Melacci, S. A neuro-symbolic framework for sequence classification with relational and temporal knowledge. arXiv preprint arXiv:2505.05106 (2025)

    2025 arXiv

  36. [44]

    On the hardness of probabilistic neurosymbolic learning

    Maene, J., Derkinderen, V., and De Raedt, L. On the hardness of probabilistic neurosymbolic learning. arXiv preprint arXiv:2406.04472 (2024)

    2024 arXiv

  37. [45]

    Online continual learning in image classification: An empirical survey

    Mai, Z., Li, R., Jeong, J., Quispe, D., Kim, H., and Sanner, S. Online continual learning in image classification: An empirical survey. Neurocomputing 469 (2022), 28–51

    2022

  38. [46]

    Continuous learning in single- incremental-task scenarios

    Maltoni, D., and Lomonaco, V. Continuous learning in single- incremental-task scenarios. Neural Networks 116 (2019), 56–73

    2019

  39. [47]

    Nesya: Neu- rosymbolic automata

    Manginas, N., Paliouras, G., and De Raedt, L. Nesya: Neu- rosymbolic automata. arXiv preprint arXiv:2412.07331 (2024)

    2024 arXiv

  40. [48]

    Deepproblog: Neural probabilistic logic programming

    Manhaeve, R., Dumancic, S., Kimmig, A., Demeester, T., and De Raedt, L. Deepproblog: Neural probabilistic logic programming. Advances in neural information processing systems 31 (2018)

    2018

  41. [49]

    Benchmarking in neuro-symbolic ai

    Manhaeve, R., Giannini, F., Ali, M., Azzolini, D., Bizzarri, A., Borghesi, A., Bortolotti, S., De Raedt, L., Dhami, D., Diligenti, M., et al. Benchmarking in neuro-symbolic ai. In Pro- ceedings of The 4th International Joint Conference on Learning & Rea- soning (2024)

    2024

  42. [50]

    Approximate infer- ence for neural probabilistic logic programming

    Manhaeve, R., Marra, G., and De Raedt, L. Approximate infer- ence for neural probabilistic logic programming. In KR (2021), pp. 475– 486

    2021

  43. [51]

    S., Ivashynka, Y., Saxe, A., and Saglietti, L

    Mannelli, S. S., Ivashynka, Y., Saxe, A., and Saglietti, L. Tilting the odds at the lottery: the interplay of overparameterisation and curricula in neural networks. Journal of Statistical Mechanics: Theory and Experiment 2024 , 11 (2024), 114001

    2024

  44. [52]

    Neuro-symbolic continual learning: Knowledge, reasoning shortcuts and concept rehearsal

    Marconato, E., Bontempo, G., Ficarra, E., Calderara, S., Passerini, A., and Teso, S. Neuro-symbolic continual learning: Knowledge, reasoning shortcuts and concept rehearsal. arXiv preprint arXiv:2302.01242 (2023). 57

    2023 arXiv

  45. [53]

    Not all neuro-symbolic concepts are created equal: Analysis and mitigation of reasoning shortcuts

    Marconato, E., Teso, S., Vergari, A., and Passerini, A. Not all neuro-symbolic concepts are created equal: Analysis and mitigation of reasoning shortcuts. Advances in Neural Information Processing Sys- tems 36 (2023), 72507–72539

    2023

  46. [54]

    From statistical relational to neurosymbolic artificial intelligence: A survey

    Marra, G., Duman ˇci´c, S., Manhaeve, R., and De Raedt, L. From statistical relational to neurosymbolic artificial intelligence: A survey. Artificial Intelligence (2024), 104062

    2024

  47. [55]

    A unified framework for neural computation and learning over time

    Melacci, S., Betti, A., Casoni, M., Guidi, T., Tiezzi, M., and Gori, M. A unified framework for neural computation and learning over time. arXiv preprint arXiv:2409.12038 (2024)

    2024 arXiv

  48. [56]

    Domain knowledge alleviates adver- sarial attacks in multi-label classifiers

    Melacci, S., Ciravegna, G., Sotgiu, A., Demontis, A., Biggio, B., Gori, M., and Roli, F. Domain knowledge alleviates adver- sarial attacks in multi-label classifiers. IEEE Transactions on Pattern Analysis and Machine Intelligence 44 , 12 (2021), 9944–9959

    2021

  49. [57]

    P., Paprocki, M., ˇCert´ık, O., Kir- pichev, S

    Meurer, A., Smith, C. P., Paprocki, M., ˇCert´ık, O., Kir- pichev, S. B., Rocklin, M., Kumar, A., Ivanov, S., Moore, J. K., Singh, S., et al. Sympy: symbolic computing in python. PeerJ Computer Science 3 (2017), e103

    2017

  50. [58]

    J., Becket, R., Brand, S., Duck, G

    Nethercote, N., Stuckey, P. J., Becket, R., Brand, S., Duck, G. J., and Tack, G. Minizinc: Towards a standard cp modelling language. In International Conference on Principles and Practice of Constraint Programming (2007), Springer, pp. 529–543

    2007

  51. [59]

    Y., et al

    Netzer, Y., W ang, T., Coates, A., Bissacco, A., Wu, B., Ng, A. Y., et al. Reading digits in natural images with unsupervised feature learning. In NIPS workshop on deep learning and unsupervised feature learning (2011), vol. 2011, Granada, p. 4

    2011

  52. [60]

    I., Kemker, R., Part, J

    Parisi, G. I., Kemker, R., Part, J. L., Kanan, C., and Wermter, S. Continual lifelong learning with neural networks: A review. Neural Networks 113 (2019), 54–71

    2019

  53. [61]

    The temporal logic of programs

    Pnueli, A. The temporal logic of programs. In 18th annual symposium on foundations of computer science (sfcs 1977) (1977), ieee, pp. 46–57

    1977

  54. [62]

    Rebuffi, S.-A., Kolesnikov, A., Sperl, G., and Lampert, C. H. icarl: Incremental classifier and representation learning. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition (2017), pp. 2001–2010. 58

    2017

  55. [63]

    L., V aidya, H., and Kanan, C

    Roady, R., Hayes, T. L., V aidya, H., and Kanan, C. Stream-51: Streaming classification and novelty detection from videos. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (2020), pp. 228–229

    2020

  56. [64]

    Imagenet large scale visual recognition challenge

    Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. Imagenet large scale visual recognition challenge. International journal of computer vision 115 (2015), 211–252

    2015

  57. [65]

    W., Pascanu, R., and Hadsell, R

    Schwarz, J., Czarnecki, W., Luketina, J., Grabska- Barwinska, A., Teh, Y. W., Pascanu, R., and Hadsell, R. Progress & compress: A scalable framework for continual learning. In International conference on machine learning (2018), PMLR, pp. 4528– 4537

    2018

  58. [66]

    Video question answering: a survey of models and datasets

    Sun, G., Liang, L., Li, T., Yu, B., Wu, M., and Zhang, B. Video question answering: a survey of models and datasets. Mobile Networks and Applications (2021), 1–34

    2021

  59. [67]

    In Proceedings of the AAAI conference on artificial intelligence (2020), vol

    Sun, Y., W ang, S., Li, Y., Feng, S., Tian, H., Wu, H., and W ang, H.Ernie 2.0: A continual pre-training framework for language understanding. In Proceedings of the AAAI conference on artificial intelligence (2020), vol. 34, pp. 8968–8975

    2020

  60. [68]

    Going deeper with convolutions

    Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., V anhoucke, V., and Rabinovich, A. Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition (2015), pp. 1–9

    2015

  61. [69]

    Ltlbench: Towards benchmarks for evalu- ating temporal logic reasoning in large language models

    Tang, W., and Belle, V. Ltlbench: Towards benchmarks for evalu- ating temporal logic reasoning in large language models. arXiv preprint arXiv:2407.05434 (2024)

    2024

  62. [70]

    Back to recurrent processing at the crossroad of trans- formers and state-space models

    Tiezzi, M., Casoni, M., Betti, A., Guidi, T., Gori, M., and Melacci, S. Back to recurrent processing at the crossroad of trans- formers and state-space models. Nature Machine Intelligence (2025), 1–11

    2025

  63. [71]

    Deepdfa: Automata learning through neural probabilistic relaxations

    Umili, E., and Capobianco, R. Deepdfa: Automata learning through neural probabilistic relaxations. In ECAI 2024. Ios Press, 2024, pp. 1051–1058. 59

    2024

  64. [72]

    Grounding ltlf specifications in image sequences

    Umili, E., Capobianco, R., and De Giacomo, G. Grounding ltlf specifications in image sequences. In Proceedings of the International Conference on Principles of Knowledge Representation and Reasoning (2023), vol. 19, pp. 668–678

    2023

  65. [73]

    Analyzing differentiable fuzzy logic operators

    van Krieken, E., Acar, E., and van Harmelen, F. Analyzing differentiable fuzzy logic operators. Artificial Intelligence 302 (2022), 103602

    2022

  66. [74]

    M., and Vergari, A

    van Krieken, E., Minervini, P., Ponti, E. M., and Vergari, A. On the independence assumption in neurosymbolic learning. arXiv preprint arXiv:2404.08458 (2024)

    2024 arXiv

  67. [75]

    Symbolic au- tomata constraint solving

    Veanes, M., Bjørner, N., and De Moura, L. Symbolic au- tomata constraint solving. In Logic for Programming, Artificial In- telligence, and Reasoning: 17th International Conference, LPAR-17, Yogyakarta, Indonesia, October 10-15, 2010. Proceedings 17 (2010), Springer, pp. 640–654

    2010

  68. [76]

    IEEE Transac- tions on Pattern Analysis and Machine Intelligence (2024)

    W ang, L., Zhang, X., Su, H., and Zhu, J.A comprehensive survey of continual learning: Theory, method and application. IEEE Transac- tions on Pattern Analysis and Machine Intelligence (2024)

    2024

  69. [77]

    Deepstochlog: Neural stochastic logic programming

    Winters, T., Marra, G., Manhaeve, R., and De Raedt, L. Deepstochlog: Neural stochastic logic programming. In Proceedings of the AAAI Conference on Artificial Intelligence (2022), vol. 36, pp. 10090–10100

    2022

  70. [78]

    Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms, 2017

    Xiao, H., Rasul, K., and Vollgraf, R. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms, 2017

    2017

  71. [79]

    Large language models can learn temporal reasoning

    Xiong, S., Payani, A., Kompella, R., and Fekri, F. Large language models can learn temporal reasoning. arXiv preprint arXiv:2401.06853 (2024)

    2024 arXiv

  72. [80]

    A semantic loss function for deep learning with symbolic knowledge

    Xu, J., Zhang, Z., Friedman, T., Liang, Y., and Broeck, G. A semantic loss function for deep learning with symbolic knowledge. In International conference on machine learning (2018), PMLR, pp. 5502– 5511

    2018

  73. [81]

    Neuro-symbolic temporal point processes

    Yang, Y., Yang, C., Li, B., Fu, Y., and Li, S. Neuro-symbolic temporal point processes. arXiv preprint arXiv:2406.03914 (2024). 60

    2024 arXiv

  74. [82]

    Continual learning through synaptic intelligence

    Zenke, F., Poole, B., and Ganguli, S. Continual learning through synaptic intelligence. In International conference on machine learning (2017), PMLR, pp. 3987–3995

    2017

  75. [83]

    Slca: Slow learner with classifier alignment for continual learning on a pre- trained model

    Zhang, G., W ang, L., Kang, G., Chen, L., and Wei, Y. Slca: Slow learner with classifier alignment for continual learning on a pre- trained model. In Proceedings of the IEEE/CVF International Confer- ence on Computer Vision (2023), pp. 19148–19158

    2023

  76. [84]

    Continual learning with pre-trained models: A survey

    Zhou, D.-W., Sun, H.-L., Ning, J., Ye, H.-J., and Zhan, D.-C. Continual learning with pre-trained models: A survey. arXiv preprint arXiv:2401.16386 (2024)

    2024 arXiv

  77. [85]

    Continual learning with unknown task boundary

    Zhu, X., Yi, J., and Zhang, L. Continual learning with unknown task boundary. IEEE transactions on neural networks and learning systems (2024). A Experimental details for Sequence Classifica- tion For simplicity, we describe each module of or pipeline ( ic, cc, nsp) with a cod...

    2024

  78. [86]

    All claims investigated in this work are clearly stated. [yes]

  79. [87]

    Clear explanations are given how the work reported substantiates the claims. [yes]

  80. [88]

    Limitations or technical assumptions are stated clearly and explicitly. [yes]

  81. [89]

    Conceptual outlines and/or pseudo-code descriptions of the AI meth- ods introduced in this work are provided, and important implementa- tion details are discussed. [yes]

  82. [90]

    [yes] Articles containing theoretical contributions: Does this paper make theoretical contributions? [no] If yes, please complete the list below

    Motivation is provided for all design choices, including algorithms, implementation choices, parameters, data sets and experimental pro- tocols beyond metrics. [yes] Articles containing theoretical contributions: Does this paper make theoretical contributions? [no] If yes, ple...

  83. [91]

    [yes/partially/no] 29https://wandb.ai

    All assumptions and restrictions are stated clearly and formally. [yes/partially/no] 29https://wandb.ai. 68

  84. [92]

    [yes/partially/no]

    All novel claims are stated formally (e.g., in theorem statements). [yes/partially/no]

  85. [93]

    [yes/partially/no]

    Proofs of all non-trivial claims are provided in sufficient detail to permit verification by readers with a reasonable degree of expertise (e.g., that expected from a PhD candidate in the same area of AI). [yes/partially/no]

  86. [94]

    [yes/partially/no]

    Complex formalism, such as definitions or proofs, is motivated and explained clearly. [yes/partially/no]

  87. [95]

    [yes/partially/no]

    The use of mathematical notation and formalism serves the purpose of enhancing clarity and precision; gratuitous use of mathematical for- malism (i.e., use that does not enhance clarity or precision) is avoided. [yes/partially/no]

  88. [96]

    [yes/partially/no] Articles reporting on computational experiments: Does this paper include computational experiments? [yes] If yes, please complete the list below

    Appropriate citations are given for all non-trivial theoretical tools and techniques. [yes/partially/no] Articles reporting on computational experiments: Does this paper include computational experiments? [yes] If yes, please complete the list below

  89. [97]

    The online appendix follows best practices for source code readability and documentation as well as for long-term accessibility

    All source code required for conducting experiments is included in an online appendix or will be made publicly available upon publication of the paper. The online appendix follows best practices for source code readability and documentation as well as for long-term accessibili...

  90. [98]

    The source code comes with a license that allows free usage for repro- ducibility purposes. [yes]

  91. [99]

    The source code comes with a license that allows free usage for research purposes in general. [yes]

  92. [100]

    The online appendix follows best practices for long-term accessibility

    Raw, unaggregated data from all experiments is included in an on- line appendix or will be made publicly available upon publication of the paper. The online appendix follows best practices for long-term accessibility. [yes]

  93. [101]

    [yes] 69

    The unaggregated data comes with a license that allows free usage for reproducibility purposes. [yes] 69

  94. [102]

    The unaggregated data comes with a license that allows free usage for research purposes in general. [yes]

  95. [103]

    If an algorithm depends on randomness, then the method used for generating random numbers and for setting seeds is described in a way sufficient to allow replication of results. [yes]

  96. [104]

    The execution environment for experiments, the computing infrastruc- ture (hardware and software) used for running them, is described, in- cluding GPU/CPU makes and models; amount of memory (cache and RAM); make and version of operating system; names and versions of relevant s...

  97. [105]

    The evaluation metrics used in experiments are clearly explained and their choice is explicitly motivated. [yes]

  98. [106]

    The number of algorithm runs used to compute each result is reported. [yes]

  99. [107]

    cherry-picked

    Reported results have not been “cherry-picked” by silently ignoring unsuccessful or unsatisfactory experiments. [yes]

  100. [108]

    Analysis of results goes beyond single-dimensional summaries of per- formance (e.g., average, median) to include measures of variation, con- fidence, or other distributional information. [yes]

  101. [109]

    All (hyper-) parameter settings for the algorithms/methods used in experiments have been reported, along with the rationale or method for determining them. [yes]

  102. [110]

    The number and range of (hyper-) parameter settings explored prior to conducting final experiments have been indicated, along with the effort spent on (hyper-) parameter optimisation. [NA]

  103. [111]

    Appropriately chosen statistical hypothesis tests are used to establish statistical significance in the presence of noise effects. [NA] Articles using data sets: Does this work rely on one or more data sets (possibly obtained from a benchmark generator or similar software arti...

  104. [112]

    The online appendix follows best practices for long-term accessibility with a license that allows free usage for research purposes

    All newly introduced data sets are included in an online appendix or will be made publicly available upon publication of the paper. The online appendix follows best practices for long-term accessibility with a license that allows free usage for research purposes. [yes]

  105. [113]

    The newly introduced data set comes with a license that allows free usage for reproducibility purposes. [yes]

  106. [114]

    The newly introduced data set comes with a license that allows free usage for research purposes in general. [yes]

  107. [115]

    All data sets drawn from the literature or other public sources (poten- tially including authors’ own previously published work) are accom- panied by appropriate citations. [yes]

  108. [116]

    All data sets drawn from the existing literature (potentially including authors’ own previously published work) are publicly available. [yes]

  109. [117]

    All new data sets and data sets that are not publicly available are described in detail, including relevant statistics, the data collection process and annotation process if relevant. [NA]

  110. [118]

    [NA] Explanations on any of the answers above (optional): [Text here; please keep this brief.] 71

    All methods used for preprocessing, augmenting, batching or split- ting data sets (e.g., in the context of hold-out or cross-validation) are described in detail. [NA] Explanations on any of the answers above (optional): [Text here; please keep this brief.] 71

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.