{"id":"706688da-a373-44f7-ad13-6a586fb3d714","arxiv_id":"2509.06570","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"RARL is a rehearsal-based incremental open set recognition method that drops inactive prototypes from the classifier loss, adds virtual class interactions, and rectifies old/new boundaries to outperform prior CIL and OSR baselines on CIFAR100 and TinyImageNet.","lead":"This paper introduces a training method that lets image classifiers keep recognizing old classes and new \"unknown\" classes as data arrives over time. It combines fixed prototype geometry with synthetic virtual classes, and reports top results on a new incremental open set benchmark built from CIFAR100 and TinyImageNet.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Unknown representations are never trained, so the claimed alignment with inactive prototypes is an unenforced geometric assumption; the benchmark's future-class unknowns may satisfy it accidentally, leaving the open-set gains unestablished.","rationale":"The reader's weakest assumption identifies the same load-bearing concern: the method never trains on unknown data, so the claim that unknown representations align with inactive prototypes is an unverified geometric property. My reading confirms this is the central soft spot. Excluding inactive prototypes from the softmax denominator is a real architectural choice, but it only prevents known classes from being pushed away from inactive directions; it does not place unknown features there. The paper's benchmark defines the unknown set at task t as the training labels of task t+1, which are drawn from the same dataset and will later be assigned to inactive prototypes. This makes the assumed alignment maximally plausible but does not establish it for genuinely novel classes. The proposed check directly measures whether unknown features occupy the reserved angular regions and whether the open-set advantage persists on disjoint unknowns. Because this concern is empirical and addressable, and because the reader already conditioned acceptance on independent verification, the correct verdict remains CONDITIONAL. I do not see an internal contradiction that would justify rejection; the missing normalization in Eq. (5) and the abstract/method wording inconsistency are additional reproducibility issues, but the conceptual load-bearing risk is the unenforced unknown-alignment assumption.","tokens_in":12921,"tokens_out":10326,"duration_ms":104503,"concrete_test":"At the final task of the CIFAR100/TinyImageNet runs, take the models trained by RARL and LUCIR and compute open-set AUROC/OSCR on two unknown sets: (i) the benchmark's future-class unknowns, and (ii) a disjoint held-out class set from the same dataset (or SVHN for CIFAR100). In parallel, for each unknown sample compute the maximum cosine similarity to inactive prototypes versus active known prototypes. If RARL's AUROC/OSCR margin over LUCIR disappears on the disjoint set, or if unknown samples are not closer to inactive prototypes than to active known prototypes, then the 'retentive angular space' mechanism is not the source of the reported open-set gains.","verdict_should_be":"UNCHANGED","load_bearing_attack":"RARL's open-set gain is attributed to a 'retentive angular space' in which unknown representations are 'encouraged to be distributed in the broader open space opposite known prototypes' (Sec. 4.2, Fig. 2d). However, no term in L_Total (Eq. 12) is evaluated on unknown samples; unknowns never appear in training, so no gradient aligns them with inactive prototypes. The only real effect of excluding inactive prototypes from the softmax denominator in Eq. (3) is that active known classes are not repelled from inactive directions. Whether unseen inputs actually land in the reserved complementary region is an empirical assumption about feature geometry, not an enforced property. The benchmark (Sec. 5.1.4) makes this assumption easy to satisfy: the unknown set at task t is exactly the training labels of task t+1, drawn from the same dataset and later assigned to those inactive prototypes. On semantically novel or disjoint unknowns, the claimed mechanism has no demonstrated support. The abstract's phrasing that unknowns are 'encouraged to align around inactive prototypes' is not backed by any loss operating on unknowns; it is at most an indirect side effect. If that side effect does not occur, the reported AUROC/OSCR improvements over LUCIR may be an artifact of the chosen future-class protocol rather than of the retentive angular representation.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper formalizes incremental open set recognition (IOSR), where each task adds new classes and the model must both classify seen classes and reject samples from classes that will appear in future tasks. It proposes RARL, which fixes ETF classifiers, excludes inactive (future) prototypes from the softmax denominator, synthesizes virtual classes for a virtual-intrinsic interaction loss, and applies PNBR/ONBR to rebalance gradient strengths. The reported experiments on CIFAR-100 and TinyImageNet compare RARL with OSR and CIL baselines and show consistent improvements in accuracy, AUROC, and OSCR, with an ablation study on the main components.","tokens_in":13248,"tokens_out":3956,"duration_ms":38848,"significance":"The paper's strength is its clear problem formulation and broad comparison: it defines an IOSR protocol, reports consistent gains across two datasets and four task splits, and provides an ablation that isolates the contribution of each loss. If the claimed mechanism were validated on genuinely novel unknowns, RARL would be a simple and practical baseline for IOSR. As it stands, the open-set advantage depends on an unenforced geometric assumption and on a benchmark where 'unknown' samples are drawn from the same distribution as future training classes.","major_comments":[{"comment":"The evaluation protocol defines the unknown set at task t as the training labels of task t+1, i.e., Y_t^U = Y_{t+1}^{train}. This means the 'unknown' inputs are not semantically novel categories but future known classes that will later be assigned to the very ETF prototypes that were kept inactive. The reported AUROC/OSCR improvements are therefore partly a measure of future-class placement rather than of rejection of truly unseen inputs. The paper should evaluate at least one configuration with disjoint held-out unknown classes (e.g., from an external dataset) or report open-set metrics for samples that never appear as future training classes, otherwise the open-set claim is not established for genuinely novel inputs.","section":"Sec. 5.1.4"},{"comment":"The claim that unknown representations are 'encouraged to be distributed in the broader open space opposite known prototypes' (Fig. 2d, Sec. 4.2) is not enforced by any term in the total loss. The training objective in Eq. (12) is evaluated only on known and synthetic virtual instances; no gradient is ever computed for unknown samples. Excluding inactive prototypes from the softmax denominator in Eq. (3) removes a repulsive force on active classes, but it does not place unknown inputs into the reserved region. This is an assumed geometric side effect, not a mechanism demonstrated in the paper. The authors should either add an explicit regularizer that scores the complementary region or provide direct feature-space evidence (e.g., angular distance of novel unknowns to inactive versus known prototypes) that the claimed alignment occurs.","section":"Secs. 4.2, 4.5, Eq. (12)"},{"comment":"Several implementation parameters needed for reproduction are missing: the ONBR hyperparameter A is introduced in Sec. 4.4 but never specified, and the less-forget loss weight λ_dis in Eq. (12) is never given. In addition, all accuracy, AUROC, and OSCR numbers appear to come from a single run with no standard deviations or significance tests. Because the reported open-set gains over LUCIR are as small as 0.26% in average AUROC (CIFAR-100, Base 20 with 8 Steps), the reader cannot tell whether these differences are stable. The authors should report the missing hyperparameter values and provide multi-seed means with error bars, or at least a paired comparison across seeds.","section":"Secs. 4.4, 4.5, Tables 1-3"}],"minor_comments":[{"comment":"The symbol K is overloaded: in Eq. (2) it is the number of preallocated ETF prototypes, while in Eq. (3) it is the number of current-task classes; using distinct symbols such as K_total and K_t would clarify the scope of the softmax denominator.","section":"Eqs. (2)-(3)"},{"comment":"It is unclear whether the virtual instance mixing is applied to raw pixels or to batch-normalized features, and the meaning of K_batch ('the number of intrinsic classes in a batch') is ambiguous in a per-instance formula; please specify the exact batch construction and normalization used.","section":"Eq. (5)"},{"comment":"The LUCIR row for the Base 60 with 7 Steps setting contains '66.3966.25' without a separator; this appears to be a formatting typo that should be corrected.","section":"Table 2"},{"comment":"The caption says the figure illustrates binary classification, but the panel shows two known prototypes and one inactive prototype; please clarify whether the 'binary classification' refers to a reduced 2D example or to the known/unknown distinction.","section":"Fig. 2"},{"comment":"The baseline label 'SoftMax (ALL)' in Table 3 is explained only in the ablation text; consider defining it in the table caption, since the name alone does not indicate that all ETF prototypes are included in the softmax.","section":"Sec. 5.1.3 and Table 3"}],"recommendation":"major_revision","confidential_remarks":"The central concern is that the benchmark's unknown set equals the next task's training labels, which makes the open-set evaluation partly a future-class anticipation task. I recommend that the editors require the authors to add a disjoint-unknown evaluation and report variance; without these, the modest AUROC gains over LUCIR cannot be attributed to the proposed retentive angular mechanism rather than to the protocol design."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper has one genuinely new idea worth taking seriously: when you pre-fix an ETF of prototypes, leave the inactive ones out of the softmax denominator. That is a small, clean twist, and the paper shows it helps. The VII loss, PNBR, and ONBR are sensible additions, the ablations are properly done, and the evaluation against both OSR and CIL baselines on CIFAR100 and TinyImageNet is thorough. The IOSR benchmark formulation, where unknown classes become the next task's known classes, is a useful way to think about evolving open sets.\n\nThe soft spot the stress-test flags is real and central. No term in the loss is ever evaluated on unknown samples. Excluding inactive prototypes from the denominator merely stops known features from being pushed away from those directions; it does not encourage unknown features to move into the open space. The paper's claim that unknowns will 'align around inactive prototypes' (abstract) or 'be distributed in the broader open space opposite known prototypes' (Sec. 4.2) is a hypothesis about feature geometry, not a trained property. And the benchmark makes that hypothesis unusually easy to satisfy: at task t, the unknowns are exactly the task t+1 training classes, drawn from the same dataset and later assigned to those inactive prototypes. On semantically novel or disjoint unknowns, the reported AUROC/OSCR gains could shrink or disappear. This is not a fatal flaw in the incremental accuracy story, but it means the open-set contribution is not yet established.\n\nOther issues are more mundane but still need fixing: hyperparameters A and λ_dis are never specified, results are single runs without error bars, and there is no code. Eq. (5) shows a simple input-space mix with no mention of batch normalization or any standardization; if that is intentional, it should be compared against the usual mixup variant. The abstract also overstates what the denominator trick does—it says unknowns are 'encouraged to align around inactive prototypes,' but the method only ignores those prototypes in the loss.\n\nNone of this is load-bearing for the closed-set gains. The core idea is plausible, the experiments are consistent, and the paper is worth a serious referee. I would send it to review, but I would ask the authors to address the unknown-geometry issue head-on, add error bars and hyperparameters, and release code.","headline":"One genuinely new knob—excluding inactive ETF prototypes from the softmax denominator—with consistent gains, but the open-set mechanism is unenforced geometry that the self-defined benchmark may accidentally satisfy.","tokens_in":13721,"tokens_out":1775,"would_cite":false,"duration_ms":19002,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Incremental open-set recognition improves when future class prototypes are excluded from the loss.","keywords":["incremental open set recognition","equiangular tight frame","retentive angular space","virtual-intrinsic interaction","open set recognition","class incremental learning","representation drift","prototype learning"],"falsifier":"Evaluate RARL with unknown classes whose labels never appear in any later task and measure AUROC/OSCR, or inspect the angular distance of those unknowns to inactive prototypes: if open-set detection drops to baseline levels or unknowns do not cluster near inactive prototypes, the central geometric claim is refuted.","tokens_in":12736,"feed_emoji":"🎯","tokens_out":5625,"duration_ms":52574,"temperature":0.7,"pith_summary":"RARL tackles incremental open set recognition, where a model must reject unknown classes at task t and then learn those same classes at task t+1. The paper claims that existing equiangular-tight-frame methods hurt open-set performance because they push known features away from all prototypes, including inactive future ones, squeezing unknown representations into narrow gaps between known classes. RARL instead keeps inactive prototypes out of the softmax denominator, so known classes anchor to their own fixed prototypes while unknown samples are encouraged to reside in the broader open region. A virtual-intrinsic interactive loss plus positive/negative and old/new boundary rectification sharpen inter-class margins and counter rehearsal imbalance. The method reports better accuracy, unknown detection, and open-set classification than ten existing OSR and CIL baselines on CIFAR100 and TinyImageNet.","feed_headline":"Reserving class space for unknowns lifts incremental open-set learning","feed_subtitle":"RARL keeps future prototypes out of the loss and beats OSR and CIL baselines on CIFAR100 and TinyImageNet.","key_machinery":"The retentive angular space is the central object: a set of class prototypes pre-fixed as vertices of an equiangular tight frame in the normalized feature space, with some vertices left inactive for future classes. The pivotal mechanism is that the softmax denominator omits inactive prototypes, so optimization never pushes known features away from future class regions. The virtual-intrinsic interactive loss then synthesizes virtual instances near class boundaries and uses sigmoid-transformed cosine similarities to pull virtual features toward their own learnable prototypes, repel intrinsic features from ambiguous virtual regions, and separate virtual classes from each other; PNBR rebalances the positive/negative gradient asymmetry in this loss, and ONBR reduces cosine similarity for old classes to counteract boundary drift toward new classes.","core_discovery":"The central claim is that an incremental open-set model should keep future classes geometrically available by removing inactive ETF prototypes from the classification loss. In the angular space defined by pre-fixed prototypes, known representations are pulled toward their active prototypes while unknown representations are left to occupy the wide complement around inactive prototypes, so converting an unknown into a known later causes little representation drift. The paper further claims that a virtual-intrinsic interactive loss, which synthesizes boundary-proximal virtual instances and imposes a triple pull/push dynamic, makes known-class features more compact and separated, and that the stratified rectification strategy corrects the resulting positive/negative imbalance and the old/new imbalance from rehearsal data. These components together, the paper argues, maintain both closed-set discrimination and open-set rejection as the recognizable scope grows, and the experiments support this with consistent gains over existing methods.","pith_inferences":["A testable extension is to apply the same loss recipe in static OSR: masking unused ETF prototypes may improve unknown detection even when no incremental tasks follow.","Because the benchmark defines unknowns as the next task's training labels, the open-space assumption is validated against 'future knowns' rather than arbitrary outliers; real-world unknowns that never become known may behave differently.","If the geometric complement hypothesis is correct, unknown-ness could be scored directly by angular distance to the nearest inactive prototype, yielding a simple per-sample confidence threshold.","The success of geometric space reservation suggests that similar pre-allocated prototype gaps could help task-free or few-shot continual learning, where classes arrive unpredictably."],"forward_implications":["Existing OSR losses that penalize proximity to all fixed prototypes can be modified to ignore future prototypes, giving unknown inputs a defined home region and improving separation from known classes.","When an unknown class is later promoted to known, its features should already be near the activated prototype, so the model needs less adaptation and forgets less.","The virtual-intrinsic interaction provides a rehearsal-friendly way to enforce inter-class margins using only current-batch virtual instances, which is valuable when old exemplars are scarce.","The proposed benchmark, where each task's unknowns are the next task's training classes, offers a reusable evaluation protocol for incremental open set recognition."],"supporting_citations":[{"why":"Supplies the neural-collapse result that final-layer features converge to ETF vertices, the geometric premise for fixing prototypes.","marker":"[27]"},{"why":"Supplies the method of pre-allocating fixed ETF classifiers for class-incremental learning that RARL extends.","marker":"[29]"},{"why":"Provides the cosine classifier, less-forget distillation, and the LUCIR baseline that RARL compares against and improves on.","marker":"[14]"},{"why":"Supplies the mix-based virtual instance synthesis that RARL converts into boundary-proximal virtual classes.","marker":"[19]"},{"why":"Provides neural-collapse-inspired feature-classifier alignment for incremental learning, motivating the retention of the angular space.","marker":"[42]"},{"why":"Supplies the claim that closed-set discriminability is directly related to open-set discrimination, used to explain baseline behavior.","marker":"[38]"},{"why":"Provides the iCaRL exemplar rehearsal baseline and the 20-exemplar-per-class protocol used in the experiments.","marker":"[30]"}],"fun_headline_variants":["Keep future classes open in angular space for incremental open-set","RARL: angular prototypes reserve room for unknowns in incremental learning","Reserve angular class space to ease incremental open-set learning","Incremental open-set learning improves by leaving unknowns angular space","Angular retention lets unknowns live to become future knowns"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that inputs the model has never seen will actually fall into the large angular region reserved around unused prototypes, even though the training procedure never shows the model an unknown and the benchmark's unknown set is just the next task's training classes.","fun_headline_variants_meta":{"raw":{"variants":["Keep future classes open in angular space for incremental open-set","RARL: angular prototypes reserve room for unknowns in incremental learning","Reserve angular class space to ease incremental open-set learning","Incremental open-set learning improves by leaving unknowns angular space","Angular retention lets unknowns live to become future knowns"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00136,"raw_usage":{"total_tokens":5525,"prompt_tokens":956,"completion_tokens":4569,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":572,"completion_tokens_details":{"reasoning_tokens":4485}},"tokens_in":572,"tokens_out":4569,"duration_ms":26817,"temperature":1.0,"reasoning_tokens":4485,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T16:15:47.266990+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Evaluate RARL with unknown classes whose labels never appear in any later task and measure AUROC/OSCR, or inspect the angular distance of those unknowns to inactive prototypes: if open-set detection drops to baseline levels or unknowns do not cluster near inactive prototypes, the central geometric claim is refuted.","supporting_citations":[{"cited_title":"Class-incremental learn- ing with pre-allocated fixed classifiers","cited_arxiv_id":null,"evidence_quote":"Supplies the method of pre-allocating fixed ETF classifiers for class-incremental learning that RARL extends."},{"cited_title":"Hard negative mix- ing for contrastive learning.Advances in neural information processing systems, 33:21798–21809, 2020","cited_arxiv_id":null,"evidence_quote":"Supplies the mix-based virtual instance synthesis that RARL converts into boundary-proximal virtual classes."},{"cited_title":"Open-set recognition: A good closed-set classifier is all you need? 2021","cited_arxiv_id":null,"evidence_quote":"Supplies the claim that closed-set discriminability is directly related to open-set discrimination, used to explain baseline behavior."},{"cited_title":"icarl: Incremental classifier and representation learning","cited_arxiv_id":null,"evidence_quote":"Provides the iCaRL exemplar rehearsal baseline and the 20-exemplar-per-class protocol used in the experiments."}],"review_version":1}