{"id":"defc4052-13a5-48b6-9da9-39d722af5003","arxiv_id":"2411.14744","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"By masking high-attention patches and aligning the features of masked and unmasked views, PointACL improves point cloud classification, part segmentation, and few-shot learning over baseline pretraining methods.","lead":"PointACL is a training recipe for 3D object recognition that deliberately hides the parts a model pays most attention to, forcing it to learn from the rest of the point cloud. It reports better accuracy and much better robustness to noise on standard point cloud benchmarks when added to existing pretraining methods.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. (2)'s significance score is never validated as a measure of true patch reliance; the central 'attention-driven' mechanism is therefore unsupported, and the small margin over random masking in Table 4 makes this the load-bearing risk.","rationale":"The reader's weakest assumption is that the significance score S_j in Eq. (2) reliably identifies patches the model truly relies on. That is exactly the load-bearing risk I identified: the paper's contribution is explicitly 'attention-driven' masking, and the whole narrative about learning from under-attended regions depends on S_j being a faithful measure of patch dependency. The paper provides no direct validation of this. I agree with the reader on this point. The missing comparison to Contrast with Reconstruct (ref [35]) and the absence of error bars are real but secondary; they affect the strength of the 'state-of-the-art' claim, whereas the attention-validity issue affects the causal interpretation and novelty of the method itself. I do not think the concern requires a change of verdict from CONDITIONAL to a harsher one, because even if S_j is imperfect, the empirical gains over Point-MAE and PointGPT are reported consistently and the ablation shows attention-based masking helps somewhat. A concrete causal test would settle whether the mechanism is genuine. If the test fails, the method still might work as a regularizer, but it would not be an 'attention-driven' method as claimed; if the test passes, the central claim is substantially strengthened. I also credit the paper for providing a training-time comparison (Appendix Table 6) and component ablations, which partially mitigate concerns about equal compute and random masking.","tokens_in":18653,"tokens_out":6604,"duration_ms":73935,"concrete_test":"Run a causal patch-importance check on the pre-trained Point-MAE and PointACL models. For each test point cloud, compute S_j for every patch using Eq. (2). Then, for each patch j independently, replace that patch's points with the mask token (or drop and renormalize the remaining patches), run the standard branch, and measure the L2 change in the global feature H_s and the change in the top-1 logit relative to the full input. Define LOPO_j as that combined sensitivity. Compute the Spearman rank correlation between S_j and LOPO_j across patches and across many samples; report it separately for Point-MAE and PointACL. Additionally, as a negative control, train PointACL with S_j randomly permuted across patches inside the same masking pipeline.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that attention-driven dynamic masking of high-attention patches forces the model to learn from under-attended regions and thereby improves representation quality and robustness. The entire mechanism rests on Eq. (2), where S_j = A_{1,j}||V_j|| / sum_i A_{1,i}||V_i|| is interpreted as the significance of patch j to the global feature. The paper never validates that this score tracks the model's true causal reliance on a patch. Softmax attention weights are known to be imperfect explanations of model behavior: A_{1,j} is just one row of one attention matrix, and ||V_j|| scales the value vector, but neither directly measures how the final representation or downstream prediction would change if patch j were removed. If S_j is only weakly correlated with true dependency, then masking the top-S_j patches is not specifically forcing the model to learn from 'under-attended regions'; it is just a complex random or norm-based augmentation. The robustness gains in §4.3 and Figure 3 would then not be causally attributable to the proposed attention mechanism. The paper's own ablation in Table 4 lends support to this worry: Random Mask + Lcontra reaches 92.1/90.9 on OBJ-BG/OBJ-ONLY, while High-Attention Mask + Lcontra reaches 92.3/91.6, so the attention specificity buys only 0.2/0.7 percentage points. This is a load-bearing, testable assumption, not a cosmetic one.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents PointACL, a self-supervised pretraining framework for 3D point cloud Transformers. During pretraining, PointACL computes per-patch significance scores S_j from the first row of the self-attention matrix and the norms of value vectors (Eq. 2), uses a Gumbel-perturbed score (Eq. 3) to mask the highest-scoring patches, and aligns the global features of the masked and unmasked branches with a contrastive loss (Eq. 5) combined with the original reconstruction/generation loss (Eq. 6). The authors evaluate PointACL on top of Point-BERT, Point-MAE, and PointGPT-S on ScanObjectNN, ModelNet40, ShapeNetPart, and few-shot classification, reporting improved accuracy and robustness, and they provide component ablations, hyperparameter studies, and a training-cost analysis.","tokens_in":18977,"tokens_out":9697,"duration_ms":84809,"significance":"If the reported gains are reproducible, PointACL would be a simple, backbone-agnostic improvement to existing point cloud self-supervised pipelines, with a plausible mechanism (reducing over-reliance on a few patches). The manuscript deserves credit for including component ablations (Table 4), an equal-training-time comparison (Table 6), and hyperparameter sensitivity analysis (Table 8). However, the central mechanism is under-validated, and several headline claims rest on small or single-run differences. The omission of a direct comparison with Contrast with Reconstruct (ref [35]) further weakens the state-of-the-art claim. The paper is a plausible candidate for publication after substantial revision, but the load-bearing assumptions need to be tested and the evidence base strengthened.","major_comments":[{"comment":"The validity of S_j as a measure of patch importance is the load-bearing assumption of the paper, but it is never tested. The first row of one attention matrix times the value norm is not an established causal importance measure, and the authors do not specify which transformer block or layer is used. The paper's own ablation (Table 4) shows that the attention-driven choice adds only +0.2 and +0.7 points over Random Mask when both use Lcontra, and the Low-Attention Mask is only slightly worse. To support the mechanism, please (i) report the correlation between S_j and the change in the global feature (or downstream accuracy) when patch j is removed, and (ii) provide repeated-run means and standard deviations for the mask-strategy comparisons in Table 4, so the reader can judge whether the attention-specific component is significant.","section":"§3.1, Eq. (2)"},{"comment":"The robustness claims are based on single runs. Gains under Gaussian noise are large and inconsistent across backbones (e.g., +13.4 for Point-MAE on OBJ-BG at sigma=0.03 but +1.9 for PointGPT-S on OBJ-ONLY), and Figure 3 shows averaged curves without error bands. Moreover, no robustness ablation compares Random Mask + Lcontra against High-Attention Mask + Lcontra under noise, so the specific contribution of the attention-driven mask to robustness is not established. Please add repeated trials and the corresponding mask-strategy ablation under noise.","section":"§4.3, Table 5 and Figure 3"},{"comment":"Contrast with Reconstruct (Qi et al., 2023, ref [35]) is discussed in the related work but never included in the comparisons. CwR is the closest existing method in spirit (combining a generative pretraining loss with a contrastive objective for 3D representation learning), and its omission makes the state-of-the-art claim in the abstract and Section 4.2 difficult to evaluate. Please add CwR to the main comparison tables (where its backbone is compatible) and discuss the differences.","section":"Tables 1-3 and Section 2"},{"comment":"The few-shot gains are within one standard deviation of the corresponding baselines for nearly all settings (e.g., PointGPT-S 5-way 10-shot: 96.8±2.0 vs 97.1±2.3; Point-MAE 5-way 10-shot: 96.3±2.5 vs 96.7±2.7; 10-way 10-shot: 92.6±4.1 vs 92.8±4.0). Therefore the statement in the abstract that PointACL 'sets new benchmarks across all evaluation tasks' is not supported. Please report paired significance tests or modestly rephrase the claim to reflect the observed overlap.","section":"Table 2"},{"comment":"The equal-training-time analysis is not fully clean. The columns labeled 'Finetune Epoch' introduce PointACL during fine-tuning, which is a different protocol from the main results (where PointACL is only applied during pretraining and the backbone is fine-tuned normally). It is unclear whether the 300/600 fine-tuning numbers use the same loss and masking as the pretraining pipeline. Please specify the exact protocol used for these rows and, if the fine-tuning variant differs, report the baseline under the same protocol.","section":"§6.2/Table 6"}],"minor_comments":[{"comment":"Equation (3) uses ε without defining it; the text says a perturbation probability from U[0,1], but the expression -log(-log ε) is a Gumbel noise, and pdy is not a probability. Please define ε and rename the quantity (e.g., 'perturbation score').","section":"Eq. (3)"},{"comment":"The attention matrix A in Eq. (1) is not specified: which transformer block/layer and which head(s) are used? The text says 'for each head separately... sum over all heads', but not the layer(s).","section":"Eq. (1)"},{"comment":"The projection that produces H_m and H_s from F_m and F_s is not described. Please specify the projection head architecture and whether it is discarded after pretraining.","section":"Eq. (5)"},{"comment":"Figure 2 caption says the masking module selects 'less activated patches', while the text and Section 3.1 say it masks high-attention patches; the caption appears to be the opposite of the method.","section":"Figure 2 caption"},{"comment":"Typos and formatting errors: 'decmidrules' in Figure 3/5 captions, 'decmidrule' in Section 6.4, 'Robutness Analysis' and 'Limatation Analysis' headings, 'Unversity' in the affiliation line, and inconsistent use of 'dec midrule' in the appendix.","section":"Throughout"},{"comment":"Table 4 does not state whether the High-Attention and Low-Attention masks use the fixed or dynamic probability; please clarify, since Table 7 shows the distinction matters.","section":"Table 4"}],"recommendation":"major_revision","confidential_remarks":"I recommend major revision. The paper's central idea is interesting and the ablations are useful, but the attention-significance assumption requires direct validation, the robustness results need error bars and a proper ablation, and the missing comparison with Contrast with Reconstruct is a significant gap for a state-of-the-art claim. The manuscript also contains numerous typos and a few undefined symbols, suggesting a lack of careful proofreading."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: PointACL is a plausible, cheap add-on to point-cloud pre-training that shows consistent if modest gains across several benchmarks, plus large-looking robustness improvements. But I don't buy the paper's preferred explanation for why it works, and the authors don't give me enough to check it.\n\nThe load-bearing claim is that masking the top-scoring patches under Eq. (2) specifically forces the model to learn from under-attended regions. Nothing in the paper validates that this score tracks causal reliance. Softmax attention plus value norms is not a proven proxy for dependency. The authors' own ablation shows the attention-specific edge over random masking with the same contrastive loss is 0.2 points on OBJ-BG and 0.7 on OBJ-ONLY. That's a small and possibly noise-level difference, so the 'attention-driven' part of the story is doing less work than the framing suggests. The contrastive loss and the masking in general might be what helps, and the paper would be more honest if it explored that alternative explanation.\n\nWhat is genuinely new: the particular combination of dynamic attention-weighted masking and contrastive alignment for point cloud pretraining is new, and the paper shows it can be bolted onto Point-MAE, PointGPT, and Point-BERT with consistent gains. The robustness experiments (Gaussian noise, rotation, etc.) are a real contribution even if they are single-run and surprisingly large; the t-SNE and ablations are decent. Table 6 at least makes an effort to control for training time, though the comparison to Contrast with Reconstruct (which does a similar joint contrastive-reconstructive objective) is absent. That is the closest prior art and should be a headline comparison.\n\nOther soft spots: no error bars on main classification tables; the few-shot table has them but the key comparisons are within noise. No code released. Hyperparameters tuned on ScanObjectNN and then reported on the same dataset is a mild circularity, but the gains are consistent across other datasets.\n\nBottom line: this is a useful paper for anyone working on point cloud self-supervised learning; it deserves a serious referee. I'd ask for a validation of Eq. (2) as a real significance measure (e.g., causal intervention or comparison with random masking at the same mask distribution), a head-to-head with Contrast with Reconstruct, and error bars on the main tables. With those, it could be a solid contribution. Without the validation, the attention mechanism remains an unsupported narrative, though the empirical recipe still works.","headline":"Useful training recipe; the 'attention-driven' narrative is under-supported by the evidence.","tokens_in":19512,"tokens_out":1945,"would_cite":false,"duration_ms":19673,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that deliberately hiding the patches a point-cloud Transformer currently attends to most, and then training the masked and original views to agree, improves 3D representation learning and robustness.","keywords":["point cloud understanding","contrastive learning","self-supervised learning","attention-driven masking","Transformer","3D classification","robustness","representation learning"],"falsifier":"Remove the top-scoring patches and measure the drop in the model's reconstruction or classification accuracy: if the significance score $S_j$ identifies true dependency, removing top-$S$ patches must degrade performance far more than removing randomly chosen or low-$S$ patches, and this counterfactual is not run in the paper. A second direct check is to train a PointACL variant with random masking under the same epochs and budget; if its accuracy and robustness match the attention-driven version, the attention signal is not doing the claimed work.","tokens_in":1483,"feed_emoji":"🎯","tokens_out":1904,"duration_ms":105495,"temperature":0.7,"pith_summary":"The paper sets out to fix a blind spot in Transformer-based point-cloud models: they concentrate on a few high-attention patches, which the authors argue makes them fragile to noise and weak at capturing global structure. PointACL attacks this by dynamically masking the patches the model currently attends to most and training the model to keep the global features of the masked and original point clouds aligned through a contrastive loss, while retaining the original pre-training objective. The authors report that this recipe, dropped into Point-BERT, Point-MAE, or PointGPT-S, improves accuracy on ScanObjectNN, ModelNet40, and ShapeNetPart, sets new few-shot benchmarks, and sharply boosts robustness to Gaussian noise, rotation, scaling, and point dropout. If true, this is a cheap, plug-in improvement to existing 3D pre-training pipelines that adds no parameters.","feed_headline":"Attention-driven masking lifts 3D point-cloud accuracy and robustness","feed_subtitle":"PointACL hides the regions a Transformer relies on most, then aligns views to boost 3D accuracy and noise resistance.","key_machinery":"The load-bearing object is the per-patch significance score $S_j = A_{1,j}\\|V_j\\| \\,/\\, \\sum_i A_{1,i}\\|V_i\\|$, which combines the global-token attention weight with the value norm to estimate how much each patch contributes to the global feature. From this score the method builds a dynamic masking probability $p_{\\mathrm{dy}} = \\log(\\mathrm{Softmax}(S/\\tau_{\\mathrm{pro}})) - \\log(-\\log \\varepsilon)$, applies a Top-K selection to mask the currently most-attended patches, and then runs a symmetric contrastive loss aligning the global features of the masked and unmasked views. This machinery forces the encoder to keep representing the whole object even when its preferred patches are removed, while the original reconstruction or generation loss preserves task-specific learning.","core_discovery":"PointACL claims that the over-concentration of self-attention is itself a trainable weakness. The method reads the first row of the attention matrix, weights it by the norm of the value vectors to form a per-patch significance score, and uses a Gumbel-style dynamic masking probability to replace the most significant patches with mask tokens. The masked point cloud and the original are passed through the same encoder, a symmetric contrastive loss aligns their global features, and the total loss adds this to the backbone's original reconstruction or generation loss. The paper reports consistent gains over the same backbones on object classification, part segmentation, and few-shot classification, with the largest margins under heavy Gaussian noise.","pith_inferences":["Beyond the paper, the same 'mask what the model is confident in, then align views' recipe could transfer to other sparse 3D inputs such as LiDAR scans or RGB-D scenes, where the most salient regions are also often the most sensor-fragile.","A reader could test whether the significance score truly tracks dependency by removing top-scoring patches and measuring the drop in reconstruction or classification accuracy; the paper does not run this counterfactual, so the mechanism behind the gains remains open to a direct check.","The dynamic masking probability uses fixed temperature and mask ratio, so an annealed schedule that gradually broadens or sharpens masking over training is a natural untested extension.","The contrastive view here is a masked and an unmasked version of the same object; adding a second independently masked view or a cross-modal view would test whether the alignment benefit comes from view diversity rather than masking alone."],"forward_implications":["PointACL can be dropped into existing Transformer-based point-cloud pre-training, such as Point-MAE and PointGPT, adding no parameters and improving downstream accuracy on classification, part segmentation, and few-shot tasks.","Models trained with PointACL are more robust to Gaussian noise, rotation, scaling, and point dropout than the same backbone trained alone, with the largest gains under strong noise.","Attention maps become more evenly distributed across patches, supporting the claimed shift from relying on a few salient regions to understanding global structure.","The reported training-cost analysis indicates the gains are not merely the result of longer training, since improvement persists when the same number of epochs is used.","The method is presented as architecture-agnostic, with consistent gains shown on Point-BERT, Point-MAE, and PointGPT-S backbones."],"supporting_citations":[{"why":"Supplies the Point-MAE backbone and its masked reconstruction pre-training loss, which PointACL augments with the contrastive objective.","marker":"[32]"},{"why":"Supplies the PointGPT-S autoregressive backbone and generation loss, the second integration testbed for PointACL.","marker":"[5]"},{"why":"Supplies Point-BERT, the third Transformer backbone used to show PointACL generalizes across architectures.","marker":"[57]"},{"why":"Proposes attention-guided contrastive learning for action representation learning, the closest methodological precursor that PointACL adapts to point clouds.","marker":"[1]"},{"why":"Proposes the Contrast with Reconstruct baseline that already combines contrastive learning with generative pretraining; it is the most relevant alternative objective and is cited but not benchmarked in the paper.","marker":"[35]"}],"fun_headline_variants":["PointACL: Contrastive learning with attention-driven masking for 3D","Contrastive learning with attention masking improves 3D understanding","Attention-driven masking + contrastive loss boost 3D point-cloud learning","Masking attention to guide contrastive learning for point clouds"],"cache_read_input_tokens":21632,"weakest_assumption_plain":"The method hinges on the assumption that the significance score computed from attention weights and value norms really pinpoints the patches the model depends on, so masking those patches forces the model to learn from neglected regions; if that score is unreliable, the whole procedure is just a stochastic masking plus contrastive loss whose robustness gains would need another explanation.","fun_headline_variants_meta":{"raw":{"variants":["PointACL: Contrastive learning with attention-driven masking for 3D","Contrastive learning with attention masking improves 3D understanding","Attention-driven masking + contrastive loss boost 3D point-cloud learning","Masking attention to guide contrastive learning for point clouds"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000836,"raw_usage":{"total_tokens":3611,"prompt_tokens":872,"completion_tokens":2739,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":488,"completion_tokens_details":{"reasoning_tokens":2664}},"tokens_in":488,"tokens_out":2739,"duration_ms":19414,"temperature":1.0,"reasoning_tokens":2664,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T14:56:59.346762+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Remove the top-scoring patches and measure the drop in the model's reconstruction or classification accuracy: if the significance score $S_j$ identifies true dependency, removing top-$S$ patches must degrade performance far more than removing randomly chosen or low-$S$ patches, and this counterfactual is not run in the paper. A second direct check is to train a PointACL variant with random masking under the same epochs and budget; if its accuracy and robustness match the attention-driven version, the attention signal is not doing the claimed work.","supporting_citations":[{"cited_title":"Masked autoencoders for point cloud self-supervised learning","cited_arxiv_id":null,"evidence_quote":"Supplies the Point-MAE backbone and its masked reconstruction pre-training loss, which PointACL augments with the contrastive objective."},{"cited_title":"Pointgpt: Auto-regressively generative pre- training from point clouds","cited_arxiv_id":null,"evidence_quote":"Supplies the PointGPT-S autoregressive backbone and generation loss, the second integration testbed for PointACL."},{"cited_title":"Point-bert: Pre-training 3d point cloud transformers with masked point modeling","cited_arxiv_id":null,"evidence_quote":"Supplies Point-BERT, the third Transformer backbone used to show PointACL generalizes across architectures."},{"cited_title":"Maskclr: Attention-guided contrastive learning for robust action representation learning","cited_arxiv_id":null,"evidence_quote":"Proposes attention-guided contrastive learning for action representation learning, the closest methodological precursor that PointACL adapts to point clouds."},{"cited_title":"Contrast with reconstruct: Contrastive 3d representation learning guided by generative pretraining","cited_arxiv_id":null,"evidence_quote":"Proposes the Contrast with Reconstruct baseline that already combines contrastive learning with generative pretraining; it is the most relevant alternative objective and is cited but not benchmarked in the paper."}],"review_version":1}