{"id":"ab4a248e-a103-4e33-a9ac-b302908cfada","arxiv_id":"2502.07250","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A two-stage framework learns complex event rules from LLM-generated pseudo traces and then maps sensor embeddings into that rule space, matching a stronger baseline with half the labels on a synthetic benchmark.","lead":"This paper tests a two-stage scheme for detecting complex events from sensor streams: first learn the event rules from cheap synthetic activity traces, then adapt to real sensor data with few labels. The authors report matching an end-to-end model's accuracy with less than half the labeled sensor data on a synthetic benchmark.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Stage 1 pretraining uses an engineered, vocabulary-restricted pseudo-trace distribution; the paper never measures its gap to real sensor-derived AE sequences, so the data-efficiency claim rests on an untested proxy.","rationale":"The reader's weakest assumption correctly targets the pseudo-trace proxy: whether LLM-generated pseudo AE traces are a valid stand-in for the atomic event sequences the sensor adapter will encounter. My stress-test sharpens that concern with a concrete, internal observation from Appendix H: the e1 prompt's activity set omits 'eat', 'drink', and 'brush_teeth', even though these are AEs in the benchmark's 9-class AE set and appear as distractors in real e1 traces. Thus the frozen CENAR has never learned to treat those tokens as irrelevant in e1 contexts. Combined with the explicit instruction to guarantee at least one target-pattern group per sequence, the pretraining distribution is clearly engineered rather than measured against real activity statistics, and the paper offers no quantification of the gap. This is the most load-bearing assumption because every downstream result—Hypotheses I and II, the Wilcoxon comparisons, and the headline 'less than half the labeled data'—would be undermined if the adapter is aligning to a latent space whose rule priors do not match the sensor domain. The paper does have independent support that partially mitigates this: the t-SNE, ARI/NMI, and linear-probe results in Appendix B.2 show the adapter output is AE-discriminative even with CE-only supervision, and the ablation in Table 7 shows Focal Loss contributes to generalization. Those results make the concern a verification gap rather than a demonstrated failure. The proposed test would settle it directly: if retraining Stage 1 with the full AE vocabulary and without the forced-pattern instruction leaves the reported F1 within the confidence intervals, the engineered prompts are not load-bearing and the central claim holds. If performance shifts, the paper's conclusions need to be conditioned on the pseudo-trace distribution. This is why the verdict should remain CONDITIONAL: the claimed data-efficiency result is plausible and internally consistent, but its central transfer mechanism depends on an unmeasured distributional assumption that a single controlled experiment can resolve.","tokens_in":23185,"tokens_out":5905,"duration_ms":57326,"concrete_test":"Using ground-truth AE labels from the existing sensor simulator's held-out test traces, extract sensor-derived AE sequences and compare their per-AE frequency and bigram transition matrices to the LLM pseudo traces (e.g., KL divergence per CE rule). Then retrain the Stage 1 CENAR under two perturbations, keeping the Stage 2 adapter and sensor data fixed: (a) generate pseudo traces with the full 9-AE vocabulary for every CE-rule prompt, and (b) remove the 'guarantee at least one group' instruction from the prompt protocol. If the 5-min or 30-min positive F1 moves outside the reported 2-sigma intervals under either perturbation, the engineered pretraining distribution is load-bearing for the data-efficiency claim.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim that NAROCE matches or exceeds Mamba with less than half the labeled sensor data depends on the Stage 1 CENAR having learned the correct CE rules in a latent space that the Stage 2 adapter can align to real sensor embeddings. That alignment is only as good as the match between the LLM-generated pseudo AE traces and the sensor-derived AE sequences. Appendix H shows the pseudo-trace distribution is heavily engineered: prompt step 5 explicitly instructs the LLM to 'guarantee at least one group in the sequence' containing the target pattern, inflating CE frequency and altering AE transition statistics relative to any natural activity stream. Moreover, the example prompt for e1 restricts the user-defined activity set to ['click_mouse', 'flush_toilet', 'sit', 'type', 'walk', 'wash'], omitting three of the nine benchmark AEs ('eat', 'drink', 'brush_teeth'). Since those AEs appear as distractors in real e1 sensor sequences, the frozen CENAR is asked at inference to ignore token types it never saw in e1 pseudo traces; its state updates for those tokens were learned only in other rules' contexts. The paper reports no measurement of the distributional gap between pseudo traces and sensor-derived AE sequences, and no experiment varying the prompt's forced pattern guarantee or vocabulary restriction. If that gap is large, the Stage 2 adapter must compensate for a misaligned frozen reasoner using only sparse CE labels, which would weaken the claimed data-efficiency advantage.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes NAROCE, a two-stage framework for online complex event detection. Stage 1 trains a Mamba-based neural algorithmic reasoner (CENAR) on LLM-generated pseudo atomic-event (AE) traces paired with FSM-based CE labels, thereby learning CE rules without sensor data. Stage 2 freezes the CENAR and trains a smaller sensor adapter to project pretrained sensor embeddings into the CENAR's latent space using only sparse CE-level labels and Focal Loss. Experiments on a synthetic multimodal (IMU+audio) benchmark with 10 CE classes and 9 AEs claim that NAROCE with 4,000 labeled sensor samples matches an end-to-end Mamba trained on 10,000 samples, and generalizes better to longer (15/30-minute) out-of-distribution sequences.","tokens_in":23457,"tokens_out":8341,"duration_ms":67151,"significance":"If the results hold beyond the synthetic testbed, NAROCE offers a practical way to reduce CE annotation cost by decoupling rule learning from sensor data. The empirical methodology is careful in several respects: 10 random seeds, 2-sigma confidence intervals, Wilcoxon signed-rank tests, and ablations on Focal Loss and pseudo-trace quantity. The finding that adapter embeddings trained with only CE labels exhibit AE-like cluster structure (Appendix B.2) is an interesting interpretability result. The main limitation is that the entire evaluation is on a synthetic dataset, and the pseudo-trace distribution is engineered rather than compared against measured AE statistics; the data-efficiency claim therefore rests on an untested proxy.","major_comments":[{"comment":"The claim that 'NAROCE_4k significantly outperforms Mamba_4k' is stated without qualification in Section 6.2, but Appendix I.2 reports p=0.30 for the 15-minute test under the null that NAROCE_4k is worse than Mamba_4k. The claim should be restricted to the 5-min and 30-min test sets, or the main-text wording should be adjusted to reflect the non-significant 15-min result. This overstatement is load-bearing because the abstract and introduction assert that NAROCE 'outperforms the strongest baseline' without noting this caveat.","section":"Section 6.2"},{"comment":"The pseudo AE traces used to train the CENAR are generated by LLM-prompted simulators whose activity vocabularies are restricted per rule and whose transition probabilities are engineered to force target patterns. For the e1 prompt (Appendix H.2), the user-defined activity set is ['click_mouse', 'flush_toilet', 'sit', 'type', 'walk', 'wash'], omitting 'eat', 'drink', and 'brush_teeth' that appear as distractors in the real e1 sensor sequences of the benchmark. The paper does not quantify the distributional gap between LLM pseudo traces and the sensor-derived AE sequences (Section 5.1), nor does it test sensitivity to the prompt vocabulary or the forced-pattern guarantee. Since Stage 2 relies on a frozen CENAR whose latent space was learned on this engineered distribution, the claimed data-efficiency advantage could be an artifact of the proxy if the adapter must compensate for misaligned token coverage. The authors should add a quantitative comparison of AE statistics (e.g., transition matrices, n-gram frequencies, t-SNE alignment) and ablations varying the prompt constraints.","section":"Section 4.3 and Appendix H"},{"comment":"The conclusion that 'there is no significant difference between NAROCE_4k and Mamba_10k' is based on two one-sided Wilcoxon tests that both fail to reject the respective nulls. With only 10 seeds, this is not evidence of equivalence; the correct reading is that the data are insufficient to detect a difference. The phrase 'matches or exceeds' in Section 1 and the abstract's 'comparable performance' are acceptable as descriptive of point estimates, but the paper should avoid implying statistical equivalence. An equivalence test or a report of the minimal detectable effect size would strengthen the claim.","section":"Appendix I.2"},{"comment":"All reported results are on a synthetic benchmark in which sensor data are synthesized by concatenating real IMU/audio clips according to AE sequences produced by the authors' stochastic simulator. The pretrained feature encoder is also trained on these same synthetic AE sensor clips (Appendix E). The conclusion that NAROCE 'reduces reliance on large-scale labeled sensor data' for real-world online CED is therefore not yet supported; the paper's own Discussion (Section 7) identifies real-life CE datasets as future work. At minimum, the abstract and introduction should restrict the claim to the synthetic testbed or add a real-data validation (even a small case study).","section":"Section 5.1 and Appendix C"}],"minor_comments":[{"comment":"The figure label 'NARCE Framework' should read 'NAROCE Framework'.","section":"Figure 2"},{"comment":"In the note under Table 4, 'user-deinfed' is a typo for 'user-defined'.","section":"Table 4"},{"comment":"The hypothesis description uses 'narce_4k' in lowercase while the rest of the text uses 'NAROCE_4k'; please make the notation consistent.","section":"Appendix I.2"},{"comment":"The code and dataset link is given as '/r/naroce-DC82/' without a domain; please provide a complete URL.","section":"Section 1"},{"comment":"The paper says 'we instantiate 10 LLM-based simulators' while Appendix H.2 says 'we use 10 LLM-generated simulators'; unify the terminology and clarify that the LLM produces simulator code that is then executed to generate traces.","section":"Section 4.3"},{"comment":"The 'Neural AE + FSM' baseline uses the same FSMs as those used for ground-truth labeling; this should be stated explicitly in the main text, as it places that baseline as a near-upper reference rather than a fully learned model.","section":"Section 5.2"},{"comment":"Table 7 shows that 'Only Adapter w/o FL' with 4k data reaches F1_pos .86/.75/.68, close to standard NAROCE .89/.77/.71; include a significance test for this ablation to justify the claim that Focal Loss is 'crucial' (Section 6.3).","section":"Appendix B.1"}],"recommendation":"major_revision","confidential_remarks":"The manuscript reports a well-controlled synthetic evaluation and the central idea of decoupling CE rule learning from sensor data is interesting. The main risks are the overstatement of significance on the 15-min test and the unmeasured pseudo-trace proxy. I recommend major revision. The paper would benefit from a quantitative analysis of the pseudo-trace distribution, an ablation varying prompt constraints, and a correction of the statistical overclaims."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this paper has a genuinely new architecture for online complex event detection—pretrain a Mamba-based reasoner on synthetic atomic-event traces, freeze it, and train a small adapter on real sensor data—and the experiments are run carefully. The data-efficiency claim, however, rests on a proxy that is never validated: the LLM-generated pseudo-traces are engineered to make the target patterns appear, and the paper never measures how close they are to what a sensor adapter will actually see.\n\nWhat's new and good: The two-stage decoupling is a real departure from prior end-to-end Mamba work [16]. The synthetic pretraining at scale (40k–80k traces) plus a frozen reasoner plus adapter is a sensible way to attack label scarcity. The evaluation is honest: 10 seeds, Wilcoxon tests, confidence intervals, ablations on focal loss and pretraining corpus size. The claim that NAROCE_4k is statistically indistinguishable from Mamba_10k is supported by the reported numbers. The t-SNE/ARI analysis showing AE-like structure in adapter embeddings is a nice touch, since it suggests the adapter is learning atomic-event semantics without AE labels.\n\nWhere it's soft: Everything is synthetic. The CE rules are authored by the researchers, and the same rules generate the pseudo-traces, the sensor simulator, and the labels, so the benchmark measures how well the model can learn exactly the rules the authors wrote. That's a real limitation, though not a fatal one for a first paper. More specific: the LLM prompt in Appendix H guarantees at least one target pattern per sequence and, for e1, restricts the activity vocabulary to six AEs, leaving out eat, drink, and brush_teeth—which do appear as distractors in real e1 sensor traces. The frozen reasoner has seen those tokens in other rules' contexts, but there's no measurement of the distributional gap between pseudo traces and sensor-derived AE sequences. If that gap is large, the adapter is compensating for a misaligned reasoner, and the data-efficiency advantage could shrink. The paper should at least report AE-transition statistics for both distributions or, better, run a sensitivity analysis where the prompt's forced-pattern and vocabulary constraints are relaxed. Also, the feature encoder is pretrained with AE supervision, shared by all baselines, so 'less than half the labeled data' counts only CE labels—fair, but worth stating. Finally, the code/data link is broken (a relative URL).\n\nWho it's for: researchers in complex event detection, neural algorithmic reasoning, and sensor-based activity recognition. It deserves a serious referee. My recommendation: send it to review; ask the authors to address the distributional gap and provide a working code link.","headline":"A well-argued two-stage framework for data-efficient complex event detection, with careful statistics, but the central data-efficiency claim rests on an unvalidated proxy: engineered LLM pseudo-traces whose distributional gap to real sensor sequences is never measured.","tokens_in":24029,"tokens_out":3554,"would_cite":true,"duration_ms":31489,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"NAROCE claims that a Mamba-based reasoner pretrained on synthetic atomic-event traces, frozen and paired with a small sensor adapter, matches the strongest baseline with less than half the labeled sensor data.","keywords":["complex event detection","neural algorithmic reasoning","online detection","state space models","Mamba","data efficiency","synthetic data pretraining","sensor adapters"],"falsifier":"Train NAROCE identically but replace the LLM-generated pseudo traces with ground-truth atomic-event sequences from the same simulator that generated the CE sensor dataset; if the data-efficiency advantage over end-to-end Mamba disappears, the result is an artifact of the pseudo-trace distribution rather than of rule/sensor decoupling. A complementary calculation is to estimate the distributional distance between pseudo-trace atomic-event statistics and sensor-derived atomic-event classifier outputs, and check whether NAROCE's accuracy drops as that distance grows.","tokens_in":22965,"feed_emoji":"🧠","tokens_out":7838,"duration_ms":66932,"temperature":0.7,"pith_summary":"Complex events—patterns like \"used restroom, skipped handwashing, then ate\"—are rare, long-range, and expensive to label in sensor data. NAROCE's central claim is that the rule-following part of this task can be learned once, for free, from synthetic atomic-event traces, and then reused for real sensors by training only a small adapter. The paper reports that a Mamba-based reasoner pretrained on 40,000 LLM-synthesized traces and adapted with 4,000 labeled sensor sequences matches or beats an end-to-end Mamba trained on 10,000 labeled sequences, and beats a strong online-action-detection baseline on longer out-of-distribution sequences. If true, this decoupling turns sensor annotation cost into a one-time adapter expense and points toward reusable neural reasoners for event rules.","feed_headline":"Complex event detection with half the labeled data","feed_subtitle":"A reasoner pretrained on synthetic traces matches a model trained on 10,000 labels using only 4,000.","key_machinery":"The load-bearing object is the frozen latent reasoning space. Stage 1 constructs it: an AE tokenizer maps each atomic event to a token, a learnable embedding encoder projects tokens into 128 dimensions, and a 12-layer Mamba—a selective state-space sequence model suited to long-range dependencies—is trained on pseudo traces with Focal Loss to predict sparse online CE labels. Stage 2 discards the embedding encoder, freezes the Mamba reasoner, and trains a 6-layer Mamba sensor adapter to output 128-dimensional vectors from pretrained audio/IMU embeddings. The pseudo traces are the enabler: they are cheap, abundant, and generated by ten LLM-based simulators per rule, with labels assigned by finite-state machines rather than by the LLM.","core_discovery":"The central discovery is that online complex event detection separates cleanly into rule learning and sensor projection. The authors train a 12-layer Mamba model as a CE reasoner on 40,000 tokenized pseudo atomic-event traces generated by ten LLM-built stochastic simulators, with labels produced by hand-written finite-state machines for ten CE rules. Once trained, that reasoner is frozen and a smaller 6-layer Mamba adapter is trained on 4,000 labeled IMU+audio windows to embed sensor features into the reasoner's latent space. The resulting system's positive F1 is statistically indistinguishable from an end-to-end Mamba trained on 10,000 labeled windows and significantly better than the same model trained on 4,000, while also generalizing to 15- and 30-minute test sequences with longer temporal gaps.","pith_inferences":["The LLM is not essential: any stochastic simulator that reproduces realistic atomic-event statistics should work, so replacing the LLM synthesizer with a calibrated hand-written simulator is a direct test of whether the pseudo-trace distribution is the active ingredient.","The frozen CE reasoner could be reused across sensors and modalities, amortizing Stage 1 pretraining and turning it into a reusable rule module for new complex-event tasks.","Because the adapter learns atomic-event structure from CE labels alone, NAROCE may bootstrap atomic-event semantics in domains without fine-grained annotations, but only within the atomic-event vocabulary encoded in the pseudo traces.","The fixed 5-second window, closed atomic-event set, and lack of an \"other\" class are deployment limits; relaxing them would stress the tokenizer and the reasoner's state memory."],"forward_implications":["NAROCE with 4,000 labeled sensor windows matches an end-to-end Mamba trained on 10,000 windows, and with 2,000 it shows no significant difference from Mamba_4k or MiniROAD_4k on the 5-minute test set.","Scaling pseudo-trace pretraining from 20,000 to 40,000 and 80,000 traces improves accuracy and generalization to longer out-of-distribution sequences, with the largest gains when sensor labels are scarce.","Focal Loss at both training stages is necessary for out-of-distribution robustness; without it, performance on 30-minute sequences degrades sharply.","The adapter's latent space separates atomic-event classes even though it was trained only with CE-level labels, indicating the frozen reasoner imposes atomic-event structure on the sensor stream."],"supporting_citations":[{"why":"Supplies the Mamba selective state-space model used as the CE reasoner backbone and, in smaller form, as the sensor adapter.","marker":"[14]"},{"why":"Prior work on SSM-based online complex event detection that this paper extends and that motivates the data-efficiency gap.","marker":"[16]"},{"why":"Defines the neural algorithmic reasoning benchmark that supplies the idea of learning algorithms with neural networks.","marker":"[30]"},{"why":"Defines the neural algorithmic reasoning paradigm that NAROCE applies to complex event rules.","marker":"[29]"},{"why":"Focal loss is the training objective used at both stages to handle sparse online CE labels.","marker":"[20]"},{"why":"Provides the accelerometer and gyroscope sensor clips used to synthesize the multimodal CE dataset.","marker":"[34]"},{"why":"Provides the audio clips used to synthesize the sensor streams for each atomic-event class.","marker":"[23]"},{"why":"MiniROAD is the online-action-detection baseline that NAROCE is compared against, especially on long out-of-distribution sequences.","marker":"[1]"}],"fun_headline_variants":["Synthetic traces halve labeled data for complex event detection","Half the data, same accuracy for online complex event reasoning","Reasoner from synthetic traces cuts sensor labels in half","Online event detection: rule learning from sims, sensor adapter"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the LLM-generated pseudo atomic-event traces are a valid stand-in for the atomic-event sequences the sensor adapter will encounter; the prompts in Appendix H explicitly steer the traces toward the target patterns, and the paper does not quantify the gap between this engineered distribution and real activity statistics.","fun_headline_variants_meta":{"raw":{"variants":["Synthetic traces halve labeled data for complex event detection","Half the data, same accuracy for online complex event reasoning","Reasoner from synthetic traces cuts sensor labels in half","Online event detection: rule learning from sims, sensor adapter"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000202,"raw_usage":{"total_tokens":1377,"prompt_tokens":934,"completion_tokens":443,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":550,"completion_tokens_details":{"reasoning_tokens":376}},"tokens_in":550,"tokens_out":443,"duration_ms":4621,"temperature":1.0,"reasoning_tokens":376,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T13:20:09.291470+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train NAROCE identically but replace the LLM-generated pseudo traces with ground-truth atomic-event sequences from the same simulator that generated the CE sensor dataset; if the data-efficiency advantage over end-to-end Mamba disappears, the result is an artifact of the pseudo-trace distribution rather than of rule/sensor decoupling. A complementary calculation is to estimate the distributional distance between pseudo-trace atomic-event statistics and sensor-derived atomic-event classifier outputs, and check whether NAROCE's accuracy drops as that distance grows.","supporting_citations":[{"cited_title":"Toward foundation models for online complex event detection in cps-iot: A case study","cited_arxiv_id":null,"evidence_quote":"Prior work on SSM-based online complex event detection that this paper extends and that motivates the data-efficiency gap."},{"cited_title":"The clrs algorithmic reasoning benchmark, 2022","cited_arxiv_id":null,"evidence_quote":"Defines the neural algorithmic reasoning benchmark that supplies the idea of learning algorithms with neural networks."},{"cited_title":"Benchmark for kitchen20, a daily life dataset for audio-based human action recognition","cited_arxiv_id":null,"evidence_quote":"Provides the audio clips used to synthesize the sensor streams for each atomic-event class."},{"cited_title":"Miniroad: Minimal rnn framework for online action detection","cited_arxiv_id":null,"evidence_quote":"MiniROAD is the online-action-detection baseline that NAROCE is compared against, especially on long out-of-distribution sequences."}],"review_version":1}