{"id":"42e37197-3403-4f6a-9cba-1b614f70b7af","arxiv_id":"2501.09525","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"SCLIFD combines supervised contrastive learning, feature-space distillation, marginal exemplar replay, and a balanced random forest to improve class-incremental fault diagnosis with few fault samples.","lead":"Industrial faults are rare and new fault types keep appearing, so diagnosis models must learn new classes without forgetting old ones. This paper combines contrastive learning, knowledge distillation, boundary-sample replay, and a balanced random forest into one framework and reports high accuracy on two benchmark datasets.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Single-run results with test-set-tuned memory size leave the claimed superiority over all baselines statistically unestablished.","rationale":"The reader's weakest assumption (segment-shuffle augmentation) is a real internal-validity risk, but it is less decisive: the SCL ablation already shows a large gain under that augmentation on the reported datasets, so the assumption seems to hold empirically there; its failure is more a generalizability concern than a direct threat to the headline comparisons. The more load-bearing issue is statistical: the manuscript reports no repeated trials and tunes K on what appears to be the test set. This affects the central 'outperforms all baselines' claim directly. The margins are large, so the claim may survive a multi-seed check; but the paper as written does not provide the evidence needed to rule out seed/order luck or K-selection leakage. My proposed multi-seed rerun with validation-based K would settle the question. I therefore do not change the reader's CONDITIONAL verdict, and I partially agree with the reader: the same methodological concerns appear in the reader's rationale, though the reader's formally named weakest assumption is different.","tokens_in":17696,"tokens_out":8815,"duration_ms":100365,"concrete_test":"Run each of the four TEP/MFF settings with at least 10 random seeds varying initialization, data order, and class order; choose K on a held-out validation split, not the test set; report mean±std for SCLIFD and the top two baselines. If any baseline mean is within one standard deviation of SCLIFD on any setting, or if validation-based K changes the reported accuracy by more than a few points, the claimed universal superiority is not established.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that SCLIFD outperforms six existing class-incremental methods across four imbalanced/long-tailed settings. All reported numbers in Tables III-V come from one run per method, and the memory buffer size K is selected from average-accuracy curves (Fig. 8, Sec. IV-E) that appear to be computed on the same test set used for final evaluation. Selecting K on test accuracy can give SCLIFD an advantage over baselines evaluated with a fixed, possibly suboptimal K, and a single favorable class order or initialization can produce the observed 18-point session-5 gap on TEP. Because every component claim (SCKD, MES, BRF) is validated only by these single-run comparisons, the absence of variance estimates and of a validation-based K selection is the most load-bearing weakness: if the comparisons are not stable under seeds/class orders, the universal superiority claim is unsupported. The large final-session drops (TEP imbalanced session 5: 72.25% vs 54.02% for iCaRL) make the ranking especially sensitive to experimental noise.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SCLIFD, a class-incremental fault diagnosis framework for limited, imbalanced, and long-tailed fault data. It combines supervised contrastive knowledge distillation (SCKD) for representation learning, a marginal exemplar selection (MES) strategy for replay memory, and a balanced random forest (BRF) classifier to counter class imbalance. The method is evaluated on the Tennessee Eastman Process (TEP) and Multiphase Flow Facility (MFF) datasets in one imbalanced and three long-tailed settings, and compared against six baselines. The authors report consistently higher average accuracies, e.g., 90.23% and 84.27% on TEP imbalanced and long-tailed, and 99.73% and 99.44% on MFF long-tailed 1 and 2, and provide ablations for each component.","tokens_in":17897,"tokens_out":4651,"duration_ms":48263,"significance":"If the reported results are reproducible and robust, SCLIFD addresses a practically important gap: most class-incremental fault diagnosis work assumes balanced data, whereas real fault data are typically scarce and imbalanced. The design is plausible, the three components are clearly motivated, and the ablation study gives some support for each contribution. The paper also provides code. However, the central claim of superiority over all baselines is an empirical claim, and the current experimental protocol does not establish it statistically: all numbers are single-run, the memory size K is selected on what appears to be the test set, and one of the two datasets uses only two exemplars per class for classifier training. These issues are load-bearing because the ranking across methods is exactly what the paper promises.","major_comments":[{"comment":"Every accuracy in Tables III and IV comes from a single run, with no standard deviations, no number of seeds, and no repeated class orders. The headline comparison includes large gaps, e.g., TEP imbalanced session 5 (72.25% vs. 54.02% for iCaRL), but some differences are small, e.g., MFF Long-Tailed 1: 97.94% (Mixed) vs. 99.73% (full SCLIFD) in Table IV. Without variance estimates it is impossible to tell whether the claimed ordering is stable under random initialization, data order, or class order. Please report means and standard deviations over at least five seeds (and preferably multiple class orders) for the main comparisons and the key ablation rows.","section":"§IV-B, Tables III–IV"},{"comment":"The memory buffer size K is selected in a sensitivity analysis by looking at average accuracy, and the same average accuracy is the reported evaluation metric in Table III. This amounts to test-set model selection for SCLIFD if the curves in Fig. 8 are computed on the test set, and the paper does not state whether the baselines were given the same per-dataset K tuning or were evaluated with a single fixed K. If baselines were not tuned in the same way, the comparison is biased. Please either select K on a validation split, or report accuracy as a function of K for all methods and adopt an identical selection protocol.","section":"§IV-E, Fig. 8"},{"comment":"For MFF the memory size is K=10 and the number of classes is 5, so Algorithm 1 (line 3) allocates m=2 exemplars per class. Algorithm 3 then trains the balanced random forest on features extracted from these two exemplars per class. This is an extremely small training set, and it is not clear whether the high MFF accuracies reflect the method's components or the triviality of the classifier problem under this construction. Please report how accuracy varies with m for MFF, clarify the exact per-class exemplar count, and discuss the stability of the BRF when trained on two samples per class.","section":"§IV-A, Table I and Algorithm 3"},{"comment":"The supervised contrastive representation learning uses a random segment-shuffle augmentation that reorders a segment of each input and integrates it back into the sequence. For fault signatures that depend on temporal order, such as step changes, ramps, or valve sticking, this augmentation may not preserve the fault class, creating contradictory positive pairs in the SCL loss. The paper does not validate that the augmentation preserves fault identity for the TEP and MFF fault types used. Please add an experiment comparing this augmentation with a label-preserving augmentation or a quantitative test of augmentation-induced invariance, since the entire representation-learning stage relies on this assumption.","section":"§IV-A.3, data augmentation"}],"minor_comments":[{"comment":"The expression for Lencoder contains an outer sum over (xi, yi) followed by sums over y=1..t and y=1..s-1, which appears to multiply the per-batch losses by the batch size and by the number of classes. Please clarify the intended indexing and whether Lscl and Ldis are per-sample or per-batch quantities.","section":"Algorithm 2, line 4"},{"comment":"The text refers to 'MFF's imbalanced case', but Table I only defines MFF as long-tailed 1 and long-tailed 2; please correct this inconsistency.","section":"§IV-B.2"},{"comment":"There are small textual slips: 'distiLlation' in the abstract, 'Dateset' in Table I, and the table column layout for 'Novel-Class Shot' and 'Training Set' is ambiguous and should be reformatted.","section":"Abstract and Table I"},{"comment":"In Eq. (4), P(zi;za) is defined as a softmax over all other samples, so Ldis is a cross-entropy between pairwise similarity distributions. Please state explicitly whether the teacher and student use the same negative sets and whether gradients flow through the student's negative samples; this is important for reproducibility.","section":"§III-B, Eq. (4)"},{"comment":"Reference [19] is an earlier arXiv paper with the same name and likely substantial overlap with this submission. The relationship between the two works should be stated explicitly so that readers can assess the novelty of the journal version.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The core idea is reasonable and the code availability is a plus. My main concern is the experimental protocol: single-run results and test-set-derived K selection are not sufficient to support the paper's universal-superiority claim. I would also ask the editor to verify the disclosure relationship with the authors' earlier preprint [19], since the current manuscript does not discuss the overlap. The manuscript fits the journal's scope, but the empirical evidence needs substantial strengthening before acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core SCLIFD recipe—supervised contrastive distillation, marginal exemplar selection, balanced random forest—is coherent and plausible, but the headline superiority claims rest on single runs and a memory size selected on test accuracy, so the strong \"outperforms all baselines\" framing is not yet established.\n\nWhat is genuinely useful: the paper tackles a real gap, combining class-incremental learning with imbalanced and long-tailed fault data, and the three components fit together sensibly. The MES idea (select boundary samples rather than centroids) is a sensible variant of herding, and the BRF classifier is a pragmatic fix for imbalance. The ablation study shows each component contributes, which is the right way to validate the design. The code is public, and the MFF experiments at least give some weight to the real-world claim. This is a credible engineering contribution to applied fault-diagnosis ML.\n\nThe weak spots are mostly empirical hygiene. All numbers are single-run, no seeds, no standard deviations, so we cannot tell whether the 18-point session-5 gap over iCaRL is robust or lucky. Memory size K is chosen from average-accuracy curves that appear to use the same test set as the final evaluation; that is selection on the test set, and it weakens the comparison against baselines that use a fixed K. The relationship with the authors' own 2023 SCLIFD preprint [19] should be disclosed: the current paper is an extension, not a first introduction of the framework. Less centrally, the segment-shuffle augmentation could break temporal fault signatures, and this is not validated.\n\nWho should read this: researchers and engineers working on industrial incremental fault diagnosis, especially with imbalanced data. It deserves a serious referee and likely acceptance after revision; the method is plausible and the code is available, but the empirical claims need variance reporting and a validation-based K selection to be convincing. I would not cite the superiority claims as established, but I would cite the framework and the public code if I needed a baseline in this niche.","headline":"Solid engineering framework for class-incremental fault diagnosis with limited data, but the empirical claims need variance reporting and a validation-based memory-size selection before they should be taken as established.","tokens_in":18423,"tokens_out":1137,"would_cite":true,"duration_ms":11038,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"SCLIFD claims a three-part framework—contrastive knowledge distillation, boundary-sample replay, and a balanced random forest—keeps class-incremental fault diagnosis accurate when fault data are scarce, imbalanced, and long-tailed.","keywords":["class incremental learning","fault diagnosis","supervised contrastive learning","knowledge distillation","catastrophic forgetting","class imbalance","long-tailed distribution","exemplar replay"],"falsifier":"Train SCLIFD on a fault type whose diagnosis depends on the order of events inside the signal, with and without the segment-shuffle augmentation; if the shuffled model's accuracy on that fault falls clearly below the unshuffled model's, the augmentation is creating wrong positive pairs and the representation stage is unreliable.","tokens_in":1761,"feed_emoji":"🏭","tokens_out":2119,"duration_ms":76137,"temperature":0.7,"pith_summary":"This paper tries to establish that class-incremental fault diagnosis can work even when new fault classes arrive with very few samples and the overall data are imbalanced or long-tailed. It proposes SCLIFD, a framework that combines supervised contrastive learning with feature-space knowledge distillation, plus a boundary-focused exemplar replay method and a balanced random forest classifier. The claimed payoff is that a fault-diagnosis model can keep learning new faults over successive sessions without catastrophic forgetting and without biasing predictions toward the normal class. If true, industrial diagnosis systems could be updated periodically with new fault types instead of being retrained from scratch.","feed_headline":"Fault-diagnosis model learns new faults without forgetting old ones","feed_subtitle":"On imbalanced and long-tailed industrial data it reports 90.23% and 84.27% average accuracy, beating eight baselines.","key_machinery":"The central machinery is the Supervised Contrastive Knowledge Distillation (SCKD) loss, which adds a cross-session feature-space distillation term to a supervised contrastive loss: the frozen previous-session extractor serves as teacher, the current extractor as student, and the student must reproduce the teacher's pairwise similarity structure over old features. This is paired with Marginal Exemplar Selection (MES), which stores samples farthest from the class mean in feature space, and a Balanced Random Forest (BRF) classifier that fits each tree to equal numbers of minority and majority exemplars. The three components are designed to work as a chain: SCKD produces the clean feature space that lets MES find boundary samples, MES fills the replay buffer with informative samples, and BRF makes the final class decision on the embedded, rebalanced exemplars.","core_discovery":"The central claim is that SCLIFD, built from three coordinated components, outperforms eight existing class-incremental learning methods on imbalanced and long-tailed fault diagnosis. The supervised contrastive knowledge distillation loss lets the current session's feature extractor learn discriminative features while emulating the previous session's feature space, reducing forgetting. The Marginal Exemplar Selection method replays the hardest boundary samples rather than centroid samples, keeping the decision boundaries visible to later sessions. The Balanced Random Forest classifier counters normal-class bias by training each tree on balanced bootstraps. The paper reports average accuracies of 90.23% and 84.27% on the imbalanced and long-tailed Tennessee Eastman Process cases, and 99.73% and 99.44% on the two Multiphase Flow Facility long-tailed cases, all above the compared baselines in the reported tables.","pith_inferences":["The paper leaves implicit that the segment-shuffle augmentation used for contrastive learning is the least protected assumption: if a fault signature depends on the temporal order inside the shuffled segment, the augmentation can create contradicting positive pairs and corrupt the learned features.","Because MES selects exemplars using the current feature extractor, the 'marginal' set is a moving target; a testable extension is to re-score exemplars after each session and refresh the buffer so boundary samples remain boundary samples.","The Balanced Random Forest is trained only on the small exemplar buffer, so with very small memories the balanced bootstrap may become degenerate; mixing synthetic oversampling into the classifier stage is a natural extension.","The three-component recipe is not obviously limited to fault diagnosis and could be tested in other time-series class-incremental domains with rare classes and a normal-bias problem, such as predictive maintenance or anomaly detection in process control."],"forward_implications":["If the reported numbers hold, industrial systems can add new fault classes through periodic offline updates using a small constant-sized memory buffer rather than full retraining.","The gap at the final incremental session (e.g., 72.25% versus 54.02% for iCaRL on imbalanced TEP) implies that feature-space distillation plus boundary-sample replay is a workable recipe against catastrophic forgetting under limited fault data.","The per-component ablation results imply that each of the three modules—SCKD, MES, and BRF—contributes distinct value, so the framework does not rest on a single trick.","The generalization experiment across different TEP fault selections, with an average accuracy of 85.86%, suggests the method is not tuned to one particular set of fault signatures.","The sensitivity analysis suggests that small memory buffers (K=100 or lower) can sustain accuracy, which matters for deployment with strict memory constraints."],"supporting_citations":[{"why":"Supplies the supervised contrastive loss that SCLIFD's representation-learning stage builds on.","marker":"[31]"},{"why":"Supplies the iCaRL baseline whose distillation and replay ideas SCLIFD extends, and which SCLIFD outperforms in the comparisons.","marker":"[33]"},{"why":"Supplies the Herding exemplar-selection baseline that MES is designed to improve.","marker":"[34]"},{"why":"Supplies the Balanced Random Forest classifier used in the classification stage.","marker":"[32]"},{"why":"Defines the limited-fault-data and long-tailed training settings that motivate the paper's problem formulation and shot numbers.","marker":"[39]"},{"why":"Supplies the EEIL incremental-learning baseline used in the comparisons.","marker":"[35]"},{"why":"Supplies the BiC baseline used in the comparisons and the memory-buffer design constraint.","marker":"[38]"},{"why":"Supplies the SAVC few-shot class-incremental baseline used in the comparisons.","marker":"[44]"},{"why":"Supplies the WaRP-CIFSL baseline used in the comparisons.","marker":"[45]"},{"why":"Supplies the BiDistFSCIL baseline used in the comparisons.","marker":"[46]"}],"fun_headline_variants":["Fault-diagnosis AI adds new faults without erasing old ones","Incremental fault diagnosis: learn new, keep old","Supervised contrastive distillation stops fault-forgetting","Few-shot fault learning that remembers prior faults"],"cache_read_input_tokens":20608,"weakest_assumption_plain":"The whole method assumes that the data augmentation used during training—cutting a random segment out of each fault signal, shuffling it, and putting it back—does not change the fault's identity, so the contrastive loss never pairs two different faults as if they were the same.","fun_headline_variants_meta":{"raw":{"variants":["Fault-diagnosis AI adds new faults without erasing old ones","Incremental fault diagnosis: learn new, keep old","Supervised contrastive distillation stops fault-forgetting","Few-shot fault learning that remembers prior faults"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000193,"raw_usage":{"total_tokens":1330,"prompt_tokens":904,"completion_tokens":426,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":520,"completion_tokens_details":{"reasoning_tokens":361}},"tokens_in":520,"tokens_out":426,"duration_ms":4772,"temperature":1.0,"reasoning_tokens":361,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T19:55:36.734440+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train SCLIFD on a fault type whose diagnosis depends on the order of events inside the signal, with and without the segment-shuffle augmentation; if the shuffled model's accuracy on that fault falls clearly below the unshuffled model's, the augmentation is creating wrong positive pairs and the representation stage is unreliable.","supporting_citations":[{"cited_title":"Supervised contrastive learn- ing,","cited_arxiv_id":null,"evidence_quote":"Supplies the supervised contrastive loss that SCLIFD's representation-learning stage builds on."},{"cited_title":"iCaRL: Incremental classifier and representation learning,","cited_arxiv_id":null,"evidence_quote":"Supplies the iCaRL baseline whose distillation and replay ideas SCLIFD extends, and which SCLIFD outperforms in the comparisons."},{"cited_title":"Herding dynamical weights to learn,","cited_arxiv_id":null,"evidence_quote":"Supplies the Herding exemplar-selection baseline that MES is designed to improve."},{"cited_title":"Using random forest to learn imbalanced data,","cited_arxiv_id":null,"evidence_quote":"Supplies the Balanced Random Forest classifier used in the classification stage."},{"cited_title":"Progressively balanced supervised contrastive representation learning for long-tailed fault diagnosis,","cited_arxiv_id":null,"evidence_quote":"Defines the limited-fault-data and long-tailed training settings that motivate the paper's problem formulation and shot numbers."},{"cited_title":"End-to-end incremental learning,","cited_arxiv_id":null,"evidence_quote":"Supplies the EEIL incremental-learning baseline used in the comparisons."},{"cited_title":"Large scale incremental learning,","cited_arxiv_id":null,"evidence_quote":"Supplies the BiC baseline used in the comparisons and the memory-buffer design constraint."},{"cited_title":"Few- shot class-incremental learning via class-aware bilateral distillation,","cited_arxiv_id":null,"evidence_quote":"Supplies the BiDistFSCIL baseline used in the comparisons."}],"review_version":1}