{"id":"bd8bd925-61a8-4ea3-b82f-21d38cec14cf","arxiv_id":"2505.11521","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"A proposed mutual-information regularizer for point-cloud segmentation reports big gains on held-out classes, but the evaluation protocol trains on those classes, so it does not demonstrate open-set detection.","lead":"The paper proposes a regularizer called 3CM that is supposed to help point-cloud segmentation models detect object classes they never saw during training. It reports large gains on 'unseen' classes, but its experiments actually fine-tune on those classes, so the open-set claim is not supported.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 80.1 unseen IoU is measured after 30 epochs of training on the actual labels of the supposed unknown classes, so the experiment does not test open-set detection.","rationale":"The reader identified the same load-bearing flaw: the experimental protocol trains on the actual labels of the supposed unknown classes, so the reported gains do not demonstrate open-set detection. This concern is decisive because it targets the central claim as stated in the abstract and introduction. I agree with REJECT: the paper's own Section 5 and Section 7 confirm that 3CM requires unknown-class labels during training, and no experiment in the paper measures performance on classes that are truly never seen during training. The mutual-information derivation is secondary; even if Eq. 7 is a valid regularizer, it does not support the advertised capability. No code or error bars are provided, but the fatal issue is the evaluation protocol itself. Therefore the reader's verdict should remain unchanged.","tokens_in":9306,"tokens_out":4143,"duration_ms":39264,"concrete_test":"Hold out the eight S3DIS unknown classes during both the initial and post-training phases. Train baseline and 3CM models using only known-class labels plus synthetic pseudo-unknowns generated from known-class data (e.g., PointCaM-style cut-and-mix or sampled outlier regions). Evaluate unknown-class IoU at inference without any additional training on actual unknown labels. If the 3CM gain over baseline disappears or shrinks to noise, the 80.1 result in Table 3 is an artifact of label exposure and the central open-set claim is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that 3CM lets a model 'distinguish and segment previously unseen categories.' For that claim to hold, the evaluation must not use labels of the unseen classes during training. Section 5 violates this: after initial training, the authors 'remove the output column corresponding to the unknown class, append 8 new output columns, and train the model for an additional 30 epochs.' The unknown classes are therefore revealed and supervised during post-training. Equation 8 also trains with CE(P_x_unknown, Y_unknown) plus the 3CM term, so the loss itself requires labels for unknown samples. Section 7 admits '3CM assumes the availability of pseudo-unknown labels during training.' Thus the headline result in Table 3 (62.3 to 80.1 unseen IoU) is not evidence of detecting previously unseen categories; it is evidence that a model can fit an expanded output head when given the previously held-out labels. The open-set claim is untested by the reported protocol.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Conditional Channel Capacity Maximization (3CM), a regularizer intended to improve open-set semantic segmentation in 3D point clouds. The authors model segmentation as a conditional Markov chain and claim that maximizing a conditional mutual information term between features and predictions, conditioned on class, preserves richer class-conditional information, thereby improving detection and segmentation of previously unseen classes. The method is evaluated on ShapeNet Part, ModelNet40, and Stanford Indoor3D, with reported gains such as unseen-class IoU rising from 62.3 to 80.1 on Stanford Indoor3D while seen-class accuracy remains flat. The paper also includes ablations over the regularization weight λ and the EMA factor β. The stated central claim is that 3CM enhances the network's ability to distinguish and segment previously unseen categories without modifying the base architecture.","tokens_in":9542,"tokens_out":2156,"duration_ms":23329,"significance":"If the central claim were supported, an information-theoretic plug-in regularizer that improves open-set segmentation in point clouds without architectural change would be a useful contribution to the computer vision community. The paper also provides machine-checked derivations only in a weak sense: the main equations are asserted rather than proven, and the experiments are not reproducible from the manuscript (no code, no dataset splits, no detailed hyperparameters). The proposed regularizer is presented as novel, but it closely follows the conditional mutual information constrained learning framework of Yang et al. (refs. [25–27]) without a reformulation that addresses the open-set setting. The reported results do not measure open-set performance: the protocol reveals the supposed unknown classes during a second training phase, so the claimed 'unseen-class' improvements are in-distribution adaptation results, not evidence of novel-category detection. This is a load-bearing flaw that invalidates the paper's headline claims, and it cannot be fixed by minor changes within the manuscript's current scope.","major_comments":[{"comment":"The evaluation protocol does not test open-set semantic segmentation as defined in the abstract and introduction. After initial training, the authors state they 'remove the output column corresponding to the unknown class, append 8 new output columns, and train the model for an additional 30 epochs' on the actual labels of the previously held-out classes. Equation (8) trains with CE(P_x_unknown, Y_unknown) plus the 3CM term, which requires ground-truth labels for unknown samples. The headline result of 62.3 to 80.1 unseen IoU in Table 3 is therefore a measure of how well the model can fit an expanded output head when given the held-out labels, not of detecting previously unseen categories. For a true open-set protocol, no label or output column for the target unseen classes may appear during training; here the entire second-phase training violates that requirement.","section":"Section 5, Table 3"},{"comment":"The derivation of the empirical CMI in Eq. (5) from the definition in Eqs. (1)–(4) is asserted rather than shown. The joint entropy and conditional distributions are not specified clearly: Eq. (2) contains P(Ŷ=1|x) and P_{Ŷ|y}(Ŷ=i|Y=i), which mix indices and condition on a single class y, yet Eq. (5) sums over all y∈[C] with Q^y_emp defined as the mean of P_{x_j}. The transition from a per-class conditional KL divergence to a sum over all classes with a class-specific mean distribution is nontrivial and requires an explicit derivation, especially because the conditioning variable Y is discrete and the sum over y in Eq. (5) appears to marginalize Y rather than condition on it. As written, Eq. (5) is not justified by Eq. (4).","section":"Section 4.1, Eqs. (4)–(5)"},{"comment":"The paper admits in Section 7 that '3CM assumes the availability of pseudo-unknown labels during training.' This assumption is not merely a limitation but is central to the method: the loss in Eq. (8) uses CE(P_x_unknown, Y_unknown) and the 3CM term is computed on D_unknown. In a genuine open-set deployment, pseudo-unknown labels are not available for novel classes at test time. The paper does not provide any mechanism to obtain such labels during training (e.g., synthetic unknown generation, out-of-distribution detection, or self-training), so the method reduces to a supervised adaptation scheme for classes whose labels are known in advance. This invalidates the claim that 3CM 'enhanc[es] the network's ability to distinguish and segment previously unseen categories' in the standard open-set sense.","section":"Section 4.3, Eq. (8) and Section 7"},{"comment":"The novelty of the regularizer is overstated. The proposed L3CM in Eq. (7) is essentially the same conditional mutual information regularizer introduced in refs. [25]–[27], which the paper cites but does not reformulate for the open-set setting. The authors do not explain how maximizing KL(P_x_unknown || Q_hat_emp) on pseudo-unknown samples differs from the previously proposed conditional mutual information maximization, nor why it would transfer to classes that are not represented in D_unknown. The experimental gains in Tables 1 and 2 are also based on the same flawed protocol: the 'unseen' classes are revealed during training, so these results do not support the claimed generalization to novel categories.","section":"Section 4 and Tables 1–2"}],"minor_comments":[{"comment":"The abstract says 'Experimental results demonstrate effectiveness of proposed method on detecting unseen objects' but the only quantitative evidence involves classes whose labels are used in training; the wording should be revised to avoid claiming open-set detection.","section":"Abstract and Section 1"},{"comment":"Related work discusses several prior methods, but no open-set baseline is included in the experiments. Reporting comparisons to the cited PDF [23] and APF [12] methods would be necessary to support the claim of state-of-the-art performance.","section":"Section 2"},{"comment":"The ShapeNet Part table header is malformed: the column names and shape counts are misaligned, and some cells contain only dashes. The table is hard to read and should be reformatted.","section":"Section 5, Table 1 caption and table body"},{"comment":"There are multiple grammatical errors: 'In this section, we conclude the paper by conduct a through set of experiments' and 'improvement in opem-set segmentation performance.' The paper should be proofread.","section":"Section 5"},{"comment":"Figure 1 is described as showing Stanford-Indoor3D results, but the figure caption references Table 1 for alignment; the caption should be corrected to reference Table 3.","section":"Section 5 and Figure 1"},{"comment":"Figure 2 shows the objective value over the unseen class, but the y-axis label is missing and the caption does not define what 'objective value' means. Please clarify.","section":"Section 6.1, Figure 2"},{"comment":"The discussion correctly lists limitations, but these limitations are not addressed in the experiments. In particular, the CMI estimation sensitivity and the assumption of pseudo-unknown labels should be tested empirically rather than deferred to future work.","section":"Section 7"}],"recommendation":"reject","confidential_remarks":"The manuscript has a fundamental flaw in its evaluation protocol: the 'unseen' classes are trained on in a second phase, so the reported gains do not support the open-set claim. The derivations are also incomplete, and the novelty relative to the authors' own prior work (refs. [25]–[27]) is not established. These issues are load-bearing and cannot be resolved by local revisions; a rejection is appropriate. If the authors wish to resubmit, they would need to (1) evaluate on a protocol that does not use any labels of the target unseen classes during training, (2) provide a rigorous derivation of Eq. (5), and (3) clarify the contribution relative to the existing CMI framework."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the central claim is not supported by the experiments. The reported +17.8 IoU gain on 'unseen' classes comes from a second training phase in which those classes' actual labels are revealed: the authors remove the unknown output column, append eight new output columns, and train for another 30 epochs with a cross-entropy loss on those labels. That is supervised adaptation to the held-out classes, not detection of previously unseen categories. The paper's own conclusion even concedes that 3CM assumes pseudo-unknown labels during training. So Table 3's rise from 62.3 to 80.1 is a fit to the very classes the paper claims to discover.\n\nWhat is actually new is the application of the conditional mutual information regularizer from refs. 26/27 to point-cloud segmentation. The Markov-chain framing is borrowed, and Eq. 7 is essentially the same CMI objective used in those works, but applying it to the 'unknown' class in PointNet and testing on ShapeNet Parts, ModelNet40, and S3DIS is a fresh domain. The plug-in design is simple, the writing is clear, and the ablations on lambda and EMA factor are reasonable additions. Credit where it's due: the authors are transparent about the assumption, at least in the conclusion, and the method is easy to try.\n\nThe soft spots beyond the protocol flaw: Eq. 5 is stated as following from Eq. 4 with no derivation, which may or may not hold; the comparison set is weak, skipping the cited O3S methods (PDF, APF, PointCaM, AutoVoc3D); and there is no code or error bars. These are secondary, but they would matter even after a protocol fix.\n\nWho should read this? Someone interested in whether CMI regularization transfers to point clouds might glance at it, but it should not be cited as evidence for open-set segmentation. I would not bring it to reading group as a positive example. For peer review, I would desk reject: the main experiment does not test the paper's stated problem, and the regularizer is not novel. The authors could rework the evaluation to a true open-set protocol (e.g., train on partial classes, test on completely unseen ones without fine-tuning) and then resubmit; until then, the open-set claims are unsupported.","headline":"The 17.8-point 'unseen' IoU gain is produced by supervised post-training on the unknown classes' labels, so the paper's central open-set claim is untested.","tokens_in":10053,"tokens_out":3434,"would_cite":false,"duration_ms":33462,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a single plug-in regularizer, 3CM, which maximizes conditional mutual information between predictions and inputs for the pseudo-unknown class, lifts reported unseen-class IoU on Stanford Indoor3D from 62.3 to 80.1…","keywords":["open-set semantic segmentation","3D point cloud segmentation","conditional mutual information","channel capacity","regularization","PointNet","pseudo-unknown labels","information theory"],"falsifier":"Run the same Stanford Indoor3D open-set split with 3CM but delete the 30-epoch post-training phase: if unseen-class IoU no longer rises above the 62.3 baseline, then the reported 80.1 result depends on exposure to the held-out classes' labels rather than on the regularizer itself.","tokens_in":9116,"feed_emoji":"","tokens_out":8473,"duration_ms":82592,"temperature":0.7,"pith_summary":"The paper tries to establish that a single information-theoretic regularizer, applied to the loss of an existing point-cloud segmentation network, can make the model substantially better at labeling object classes it never saw during training. It models segmentation as a conditional Markov chain and derives a term called Conditional Channel Capacity Maximization (3CM) that encourages the encoder to keep more class-conditional information in its features. On the Stanford Indoor3D benchmark, the authors report unseen-class IoU rising from 62.3 to 80.1 with the regularizer, with no drop on known classes (85.3 vs 85.4). The same term also improves part segmentation on ShapeNet Part and classification on ModelNet40. A key caveat is that the evaluation includes a post-training phase on the labels of the supposedly unseen classes.","feed_headline":"A plug-in regularizer lifts unseen-class IoU by 17.8 points","feed_subtitle":"One added loss term raises unknown-class IoU from 62.3 to 80.1 on indoor scans.","key_machinery":"The load-bearing object is the empirical conditional mutual information (CMI) of the segmentation model: for each class $y$, the average KL divergence between a point's softmax prediction and the class-conditional mean prediction, $\\mathrm{CMI}_{\\mathrm{emp}}(f) = \\frac{1}{N}\\sum_{y\\in[C]}\\sum_{x_j\\in D_y} \\mathrm{KL}(P_{x_j}\\|Q^y_{\\mathrm{emp}})$, with $Q^y_{\\mathrm{emp}}$ the mean prediction over class $y$. The paper's regularizer, 3CM, maximizes this quantity for the pseudo-unknown cluster only, $\\mathcal{L}_{3CM} = \\frac{1}{N_{\\mathrm{unknown}}}\\sum_{x_j\\in D_{\\mathrm{unknown}}} \\mathrm{KL}(P_{x_j}\\|\\hat{Q}^y_{\\mathrm{emp}})$, updating the reference $\\hat{Q}^y_{\\mathrm{emp}}$ with an exponential moving average (EMA factor 0.995). This single term, added to the cross-entropy loss with weight $\\lambda=0.5$, is what carries the reported open-set gains.","core_discovery":"The paper's central claim is that maximizing the conditional channel capacity—the average KL divergence between per-point prediction distributions and their class-conditional mean distribution—for the unknown class is a usable, generic objective for open-set segmentation. On the author's terms, 3CM is derived from modeling the segmentation pipeline as a conditional Markov chain, and it acts as a plug-in loss term requiring neither new architecture nor generative modules. The headline evidence is a 17.8-point IoU gain on unseen classes in Stanford Indoor3D (80.1 vs 62.3) alongside unchanged known-class accuracy, with similar gains across ShapeNet Part and ModelNet40. The paper is explicit about a limitation: the method 'assumes the availability of pseudo-unknown labels during training,' and its reported protocol fine-tunes the model for 30 additional epochs using the actual labels of the classes later called unseen.","pith_inferences":["The reported protocol trains for 30 extra epochs on the held-out classes' labels; a fair open-set test would measure unseen-class IoU immediately after initial training, before any post-training, and the paper does not report that number.","If the effect reproduces, the same CMI objective could be applied to the known classes as a regularizer for class-feature separation, offering a direct test of whether conditional channel capacity on seen classes predicts robustness on related novel classes.","Injecting controlled noise into the pseudo-unknown labels during training would test how much the gain depends on clean unknown-class supervision, since real pseudo-labeling is noisy.","The single-seed, single-split comparisons in Tables 1–3 leave open how stable the 17.8-point gap is; rerunning the same split with multiple seeds and different held-out class choices would bound the effect."],"forward_implications":["If 3CM works as reported, any point-cloud segmentation pipeline can obtain open-set robustness by adding one loss term, without new architecture or generative models.","Known-class accuracy is not traded away: the S3DIS experiment shows seen-class IoU of 85.3 with 3CM versus 85.4 without.","The method's sensitivity to its two hyperparameters is small in a working range: $\\lambda$ between 0.1 and 0.5 keeps seen-class IoU stable (85.3–85.6), and EMA factors between 0.99 and 0.999 keep unseen-class IoU between 78.1 and 80.1; the paper reports non-convergence at $\\beta=0$.","Because the regularizer is a loss-level term, it can be combined with other open-set techniques such as uncertainty-based pseudo-labeling or synthetic unknown generation.","The information-theoretic formulation gives a convergence guarantee for the alternating update when batch sampling is ignored, which the ablation's loss curves support."],"supporting_citations":[{"why":"Supplies the PointNet backbone on which 3CM is tested and whose without/with-3CM comparison defines the headline gain.","marker":"[13]"},{"why":"Provides the ModelNet40 shape-classification benchmark used to measure seen and unseen class accuracy.","marker":"[22]"},{"why":"Cited by the paper as the Stanford Indoor3D dataset for the headline open-set segmentation results.","marker":"[26]"},{"why":"Source of the conditional-mutual-information modeling of segmentation as a conditional Markov chain from which 3CM is derived.","marker":"[27]"},{"why":"Provides the conditional mutual information estimation used in the 3CM objective.","marker":"[29]"}],"fun_headline_variants":["Plug-in term lifts unseen-class IoU by 17.8 points","One regularizer: +17.8 points for open-set point-cloud IoU","Conditional channel capacity maximization boosts open-set segmentation","Maximize channel capacity to detect novel 3D point classes","Simple loss term raises open-set IoU from 62.3 to 80.1"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that labels for the unknown classes are available during training (as pseudo-labels) and, in the reported evaluation, that the model can be trained on the actual labels of those 'unseen' classes for 30 epochs; without that leakage, the claimed ability to segment truly unknown categories is not established.","fun_headline_variants_meta":{"raw":{"variants":["Plug-in term lifts unseen-class IoU by 17.8 points","One regularizer: +17.8 points for open-set point-cloud IoU","Conditional channel capacity maximization boosts open-set segmentation","Maximize channel capacity to detect novel 3D point classes","Simple loss term raises open-set IoU from 62.3 to 80.1"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000595,"raw_usage":{"total_tokens":2765,"prompt_tokens":901,"completion_tokens":1864,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":517,"completion_tokens_details":{"reasoning_tokens":1768}},"tokens_in":517,"tokens_out":1864,"duration_ms":15066,"temperature":1.0,"reasoning_tokens":1768,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T22:57:06.872214+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same Stanford Indoor3D open-set split with 3CM but delete the 30-epoch post-training phase: if unseen-class IoU no longer rises above the 62.3 baseline, then the reported 80.1 result depends on exposure to the held-out classes' labels rather than on the regularizer itself.","supporting_citations":[{"cited_title":"3d shapenets: A deep representation for volumetric shapes","cited_arxiv_id":null,"evidence_quote":"Provides the ModelNet40 shape-classification benchmark used to measure seen and unseen class accuracy."},{"cited_title":"Conditional mutual information con- strained deep learning: Framework and preliminary results","cited_arxiv_id":null,"evidence_quote":"Cited by the paper as the Stanford Indoor3D dataset for the headline open-set segmentation results."},{"cited_title":"Conditional mutual information constrained deep learning for classification","cited_arxiv_id":null,"evidence_quote":"Source of the conditional-mutual-information modeling of segmentation as a conditional Markov chain from which 3CM is derived."},{"cited_title":"Bayes conditional distribution estimation for knowledge distillation based on conditional mutual informa- tion","cited_arxiv_id":null,"evidence_quote":"Provides the conditional mutual information estimation used in the 3CM objective."}],"review_version":1}