{"id":"54d2188c-1818-42e0-ba1c-f1e8e84b4c0a","arxiv_id":"2501.09278","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Adding Point-E generated, consistency-filtered synthetic point clouds to ShapeNet improves MixCon3D zero-shot 3D classification on Objaverse-LVIS, ScanObjectNN, and ModelNet40.","lead":"This paper tests whether 3D shapes generated from text by Point-E can expand a small real 3D dataset and improve zero-shot 3D classification. The authors report gains of 3.0 to 8.7 points on three benchmarks by adding filtered synthetic data to ShapeNet before training MixCon3D.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 3 contradicts the paper's own scaling ablation (Table 5, ×1) by up to 4.0 points on ScanObjectNN, nearly the entire claimed gain; the central result is not reproducible from the reported experiments.","rationale":"The reader's weakest_assumption focuses on the consistency filter's indirect text-image to point-cloud transfer. While that is a valid concern, the more load-bearing issue is that the main empirical result is internally contradicted by the paper's own scaling ablation. If the numbers in Table 3 cannot be reproduced from the described setup, then the central claim—that TeGA improves zero-shot 3D classification—is not established, regardless of whether the filter's assumption holds. My recommended verdict remains CONDITIONAL because the issue is testable: with seeds, error bars, and clarified protocol, the claim could be confirmed or refuted. I agree partially with the reader: both concerns undermine the paper, but the experimental inconsistency is the primary obstacle.","tokens_in":15510,"tokens_out":13684,"duration_ms":170800,"concrete_test":"Run the Table 3 'ShapeNet + TeGA' condition and the Table 5 ×1 condition using identical code, the same filtered synthetic dataset, and at least 3 random seeds. If the standard deviation across seeds on ScanObjectNN exceeds ~2 points, or if the mean of the ×1 condition differs from Table 3 by more than 0.5 points, the headline improvements are within noise/unreported settings and the central claim fails. Also run a control doubling ShapeNet by duplicating real samples to separate quantity from synthetic-data value.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"Table 3 is the only direct evidence for the headline claim, reporting ShapeNet+TeGA at 12.4/51.1/73.3 on Objaverse-LVIS/ScanObjectNN/ModelNet40 versus 9.4/46.5/64.6 for ShapeNet alone (§5.3). The scaling ablation in Table 5 (§5.4) purports to run the same nominal configuration—ShapeNet plus Point-E data at ×1, i.e., equal to ShapeNet's size of 53,470 samples—yet reports 11.2/47.1/73.4. The 4.0-point gap on ScanObjectNN is 87% of the claimed gain, and the 1.2-point gap on Objaverse-LVIS is 40% of its gain. Section 5.4 never states whether Table 5 uses the consistency filter; if it does, the numbers are inconsistent; if it does not, the paper fails to identify which component produced Table 3. Adding the paper's own exploratory results, Table 1 shows the filter hurts ModelNet40 (71.3 vs. 73.4) and Table 5 shows ScanObjectNN declines monotonically with scaling (48.2→47.1→45.1). No seeds, error bars, or code are provided. The claimed uniform gains are therefore not supported by a stable, internally consistent set of measurements.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes TeGA, a method for expanding a 3D training dataset by generating synthetic point clouds with the text-to-3D model Point-E, rendering multi-view images from the generated point clouds, and applying a consistency filter that uses BLIP captions and GPT-4 scoring to remove samples whose text and rendered image appear misaligned. The augmented dataset is then used to train MixCon3D, a language-image-3D contrastive model, and the central claim is that training on ShapeNet plus the TeGA-filtered synthetic data improves zero-shot 3D classification on Objaverse-LVIS, ScanObjectNN, and ModelNet40 relative to training on ShapeNet alone. The main evidence is reported in Table 3, with supporting ablations on filtering, guidance scale, mixing ratio, and scaling in Tables 1, 2, 4, and 5.","tokens_in":15823,"tokens_out":3967,"duration_ms":37307,"significance":"If the reported improvements are robust, the paper would provide a practical and timely demonstration that off-the-shelf text-to-3D generators can mitigate the data-scarcity bottleneck in zero-shot 3D classification, and the proposed consistency filtering is a sensible safeguard against misaligned synthetic samples. The paper is also useful in that it reports ablations of several important design choices. However, the central quantitative claim is not reproducible from the experiments as reported: the scaling ablation in Table 5 contradicts the headline results in Table 3, the filtering ablation in Table 1 shows a large negative effect on ModelNet40, and no error bars, seeds, or code are provided. These issues prevent the reader from assessing whether the claimed gains are reliable or are artifacts of a particular run or an unreported configuration.","major_comments":[{"comment":"The headline result in Table 3 (ShapeNet + TeGA at 12.4/51.1/73.3 on Objaverse-LVIS, ScanObjectNN, ModelNet40) is not consistent with the scaling ablation in Table 5. The ×1 row of Table 5 is the same nominal configuration (ShapeNet plus Point-E data equal to ShapeNet's size), yet it reports 11.2/47.1/73.4. The gap is 1.2 points on Objaverse-LVIS (40% of the claimed 3.0-point gain) and 4.0 points on ScanObjectNN (87% of the claimed 4.6-point gain). Section 5.4 does not state whether the Table 5 runs use the consistency filter; if they do, the two tables are irreconcilable, and if they do not, the paper fails to identify which component produced the Table 3 results. The authors need to clarify the exact configuration and report multiple seeds with standard deviations.","section":"§5.3 vs. §5.4, Tables 3 and 5"},{"comment":"The consistency filtering ablation contradicts the paper's claim that filtering is uniformly beneficial. Using 10,000 synthetic samples, filtering improves Objaverse-LVIS and ScanObjectNN but decreases ModelNet40 by 2.1 points (73.4 without filtering vs. 71.3 with filtering). Section 5.2 states that the results 'indicate that the proposed consistency filtering of TeGA provides effective synthetic training,' which is only partially supported. Since the main result in Table 3 uses the filter at a much larger scale, the paper should explain why filtering helps at the scale used in Table 3, or qualify the claim if the effect is benchmark-dependent.","section":"§5.2, Table 1"},{"comment":"Table 5 shows that ScanObjectNN accuracy declines monotonically with the amount of synthetic data (48.2 at ×0.1, 47.1 at ×1, 45.1 at ×2), which is in direct tension with Table 3, where the ×1 setting yields 51.1. The paper acknowledges this decline in the text, but does not reconcile it with the large ScanObjectNN gain claimed in the main experiment. The authors need to state whether the Table 5 runs used the same filtered dataset as Table 3, and if so, explain why the identical configuration yields such different numbers.","section":"§5.4, Table 5"},{"comment":"The consistency filter assumes that alignment between the input text and the final rendered image implies alignment of the intermediate point cloud. This is load-bearing because the point cloud is never checked directly. The paper's own feature visualization in Appendix Figure C shows that real and synthetic features of the same class are well separated, which indicates a substantial domain gap between ShapeNet and Point-E data. The authors should discuss whether the text-image-based filter is sufficient to ensure point-cloud quality, or whether the domain gap explains why synthetic data sometimes hurts performance (e.g., ModelNet40 in Table 1 and ScanObjectNN in Table 5).","section":"§4, Consistency Filtering; Appendix Figure C"},{"comment":"No error bars, number of seeds, or code are reported for any of the tables. Given the contradictions between Tables 3 and 5 and the negative scaling trend in Table 5, single-run numbers are insufficient to support the paper's central claim. Reporting mean and standard deviation over at least three seeds, or releasing the code and generated dataset, is necessary to establish that the gains are not due to run-to-run variance.","section":"All experiments"}],"minor_comments":[{"comment":"The abstract reports gains of 4.6% on ScanObjectNN and 8.7% on ModelNet40, while Section 5.3 reports 4.4% and 8.8% for the same comparison; please harmonize these numbers.","section":"Abstract and §5.3"},{"comment":"The size of ShapeNet is given as 52,470 in the Introduction and 53,470 in Section 5.4; please correct the inconsistency.","section":"§1 and §5.4"},{"comment":"The notation in the definition of the synthetic dataset is confusing: the text refers to Dt and Ds in ways that appear swapped, and the tuple contains a typo 'x0T′ i'. Please clarify which dataset is the source and which is the generated one.","section":"§4, Synthetic Dataset Construction"},{"comment":"The text says 'In Table 1, we compare the Top1 accuracy with each guidance scale' but the results are in Table 2; please fix the cross-reference.","section":"§5.2, Effect of Guidance Scale"},{"comment":"The caption of Figure 5 refers to 'PE/SP' but the table and text use 'PE/SN'; please use a consistent abbreviation.","section":"§5.4, Figure 5 caption"},{"comment":"The phrase 'which achieves SoTA in zero-shot 3D classification' is unclear because it could refer to TeGA itself or to the base MixCon3D model; please rephrase to avoid ambiguity.","section":"Abstract"}],"recommendation":"major_revision","confidential_remarks":"The internal inconsistency between Table 3 and Table 5 is the main obstacle: the reader cannot trust the headline gains without an explanation of the differing configurations and a multi-seed evaluation. The paper's core idea is plausible and the exploratory ablations are informative, but the reporting needs substantial revision before the claims can be accepted. I would not reject outright, as the inconsistencies are fixable in principle, but they are load-bearing and must be resolved."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the idea is genuinely useful—use a text-to-3D generator to expand a 3D pretraining set for zero-shot classification—and the paper lays out a clean pipeline with a sensible filtering step. But the headline numbers don't hold together internally. Table 3 claims that doubling ShapeNet with TeGA-filtered Point-E data gives 12.4/51.1/73.3 on Objaverse-LVIS/ScanObjectNN/ModelNet40. The Table 5 scaling ablation, which is supposed to be the same configuration (×1), reports 11.2/47.1/73.4. That 4.0-point gap on ScanObjectNN is nearly the entire claimed gain. Section 5.4 never states whether Table 5 used the filter; if it did, the results conflict; if it didn't, the paper doesn't say which component produced the main result.\n\nThere are also smaller inconsistencies. The abstract says 4.6/8.7, the body says 4.4/8.8; the filtering ablation shows a 2.1-point drop on ModelNet40, and the scaling ablation shows ScanObjectNN monotonically decreasing with more synthetic data. No seeds, no error bars, no code. The filtering step's key assumption—that text-image alignment implies point-cloud alignment—is asserted, not validated, and the paper's own t-SNE shows real and synthetic features of the same class separated.\n\nCredit where it's due: the paper is the first in its cited literature to use text-to-3D generation to expand language-image-3D pretraining data. The method is simple, reproducible in spirit with off-the-shelf components, and the ablations (mixing ratio, scaling, guidance scale) are a reasonable start. The writing is clear and the limitations paragraph is honest about Point-E's closed training data.\n\nBut the central empirical claim—that synthetic expansion improves zero-shot 3D classification—is not currently backed by a stable set of measurements. This is a fixable problem: report multiple seeds and error bars, clarify the Table 5 configuration, and reconcile the numbers. As it stands, the paper is a good idea with an unproven headline. I'd send it to reviewers, because the idea deserves a fair test, but I'd expect major revision.","headline":"A sensible idea with an internally inconsistent main result; the headline claim that synthetic expansion helps 3D zero-shot classification needs more careful experiments before it can be trusted.","tokens_in":16389,"tokens_out":2127,"would_cite":false,"duration_ms":20068,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that synthetic 3D point clouds generated by Point-E and filtered by text-image consistency can be added to ShapeNet to improve zero-shot 3D classification, with gains of 3.0% on Objaverse-LVIS, 4.6% on ScanObjectNN, and…","keywords":["synthetic data","zero-shot 3D classification","text-to-3D","point cloud","consistency filtering","contrastive learning","TeGA","dataset expansion"],"falsifier":"A direct test would be to measure the alignment between the generated point clouds and the input text using a point-cloud-caption retrieval model, and compare alignment scores for samples that pass TeGA's image-text filter versus those that fail. If the point-cloud-text alignment is not higher for filtered samples, the filter's core assumption fails. A second check would be to train MixCon3D using unfiltered synthetic data on ModelNet40 alone and see whether accuracy remains above the filtered result, which would indicate that filtering removes useful geometric diversity in that setting.","tokens_in":15348,"feed_emoji":"🧊","tokens_out":3747,"duration_ms":34051,"temperature":0.7,"pith_summary":"The paper sets out to show that synthetic 3D data generated by a text-to-3D model can serve as effective training data for zero-shot 3D classification, alleviating the high cost of collecting real 3D data. It proposes TeGA, which generates point clouds from class names using Point-E, renders them into images, and filters out samples whose images do not match the source text. When these filtered synthetic samples are added to ShapeNet, the MixCon3D model improves by 3.0% on Objaverse-LVIS, 4.6% on ScanObjectNN, and 8.7% on ModelNet40 compared to training on ShapeNet alone. This matters because it suggests a practical route to scaling 3D training data without manual annotation, addressing a core bottleneck in zero-shot 3D vision.","feed_headline":"Synthetic 3D shapes lift zero-shot classification by 8.7%","feed_subtitle":"Filtered Point-E data added to ShapeNet improves zero-shot 3D classification on three benchmarks.","key_machinery":"The key machinery is TeGA's consistency filtering pipeline. For each generated point cloud, the method renders two images (front and back), generates captions with BLIP, merges them via GPT-4 into one unified caption, and then scores the match between this caption and the original text prompt using two metrics: word-level containment (score 5 or 1) and a GPT-4 semantic similarity score (1 to 5). The scores are summed and compared to a threshold (δ = 3.5); samples below the threshold are discarded. This filter is the component that keeps synthetic data aligned across text, image, and point cloud modalities during contrastive pretraining.","core_discovery":"The central claim is that a consistency-filtered synthetic dataset, constructed by feeding ShapeNet category names into Point-E and rendering the resulting point clouds from multiple viewpoints, can be combined with a real 3D dataset to improve zero-shot 3D classification performance. The authors show that doubling ShapeNet with TeGA-filtered synthetic data improves MixCon3D's Top-1 accuracy on three benchmarks: from 9.4% to 12.4% on Objaverse-LVIS, from 46.5% to 51.1% on ScanObjectNN, and from 64.6% to 73.3% on ModelNet40. They further show that the consistency filter, which removes roughly half of the generated samples, is beneficial on Objaverse-LVIS and ScanObjectNN but slightly harmful on ModelNet40, and that replacing 25% of real ShapeNet data with synthetic data beats ShapeNet alone while 100% synthetic data collapses accuracy.","pith_inferences":["The observed gains may stem more from increased data volume and stronger reliance on text-image alignment than from geometric fidelity, since the paper's own feature visualization shows that real and synthetic features of the same class remain separated.","The filter's assumption that text-image alignment implies point-cloud alignment is directly testable; if it fails, a point-cloud-aware alignment check could recover the ModelNet40 accuracy lost by filtering.","Applying TeGA to a larger and more diverse real dataset (e.g., the combined ShapeNet, ABO, 3D-FUTURE, and Objaverse setup used by state-of-the-art methods) could reveal whether synthetic expansion remains beneficial at higher data volumes or whether the domain gap between Point-E and real shapes becomes a limiting factor."],"forward_implications":["Doubling ShapeNet with TeGA-filtered Point-E data improves zero-shot classification accuracy on Objaverse-LVIS, ScanObjectNN, and ModelNet40.","Consistency filtering removes noisy samples and improves accuracy on Objaverse-LVIS and ScanObjectNN, though it slightly reduces ModelNet40 accuracy.","Replacing 25% of real ShapeNet data with synthetic data outperforms ShapeNet alone, while training on 100% synthetic data leads to dramatic performance collapse.","Scaling synthetic data to twice the ShapeNet size improves Objaverse-LVIS and ModelNet40 but degrades ScanObjectNN, which is more sensitive to noise.","The text-image alignment filter is sufficient to support modality alignment in language-image-3D contrastive learning, even without direct point-cloud verification."],"supporting_citations":[{"why":"Point-E is the text-to-3D model that generates the synthetic point clouds used for dataset expansion.","marker":"[26]"},{"why":"MixCon3D is the contrastive language-image-3D pretraining method that serves as both the baseline and the model being augmented.","marker":"[10]"},{"why":"ShapeNet is the real 3D dataset that provides class-name prompts for generation and is the dataset being expanded.","marker":"[3]"},{"why":"BLIP generates captions from rendered images, forming the basis for the consistency filter's text-image comparison.","marker":"[19]"},{"why":"T3Bench provides the alignment evaluation framework that inspires the GPT-4 semantic scoring used in consistency filtering.","marker":"[12]"},{"why":"OpenCLIP supplies the frozen image and text encoders used in MixCon3D's contrastive learning with the augmented dataset.","marker":"[5]"},{"why":"Objaverse-LVIS is one of the three zero-shot classification benchmarks used to evaluate the method.","marker":"[7]"}],"fun_headline_variants":["Synthetic 3D shapes lift zero-shot classification by 8.7%","TeGA: Consistency-filtered synthetic data boosts zero-shot 3D","Filtered synthetic 3D data improves zero-shot classification up to 8.7%","Synthetic augmentation from Point-E yields up to 8.7% gain in zero-shot 3D"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The consistency filter assumes that if the input text matches the final rendered image, the intermediate point cloud also matches the text, even though the point cloud itself is never directly checked.","fun_headline_variants_meta":{"raw":{"variants":["Synthetic 3D shapes lift zero-shot classification by 8.7%","TeGA: Consistency-filtered synthetic data boosts zero-shot 3D","Filtered synthetic 3D data improves zero-shot classification up to 8.7%","Synthetic augmentation from Point-E yields up to 8.7% gain in zero-shot 3D"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000738,"raw_usage":{"total_tokens":3331,"prompt_tokens":1017,"completion_tokens":2314,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":633,"completion_tokens_details":{"reasoning_tokens":2222}},"tokens_in":633,"tokens_out":2314,"duration_ms":15726,"temperature":1.0,"reasoning_tokens":2222,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T20:06:14.115613+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A direct test would be to measure the alignment between the generated point clouds and the input text using a point-cloud-caption retrieval model, and compare alignment scores for samples that pass TeGA's image-text filter versus those that fail. If the point-cloud-text alignment is not higher for filtered samples, the filter's core assumption fails. A second check would be to train MixCon3D using unfiltered synthetic data on ModelNet40 alone and see whether accuracy remains above the filtered result, which would indicate that filtering removes useful geometric diversity in that setting.","supporting_citations":[{"cited_title":"Sculpting holistic 3d representation in con- trastive language-image-3d pre-training","cited_arxiv_id":null,"evidence_quote":"MixCon3D is the contrastive language-image-3D pretraining method that serves as both the baseline and the model being augmented."},{"cited_title":"Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models","cited_arxiv_id":null,"evidence_quote":"BLIP generates captions from rendered images, forming the basis for the consistency filter's text-image comparison."},{"cited_title":"Reproducible scal- ing laws for contrastive language-image learning","cited_arxiv_id":null,"evidence_quote":"OpenCLIP supplies the frozen image and text encoders used in MixCon3D's contrastive learning with the augmented dataset."},{"cited_title":"Objaverse: A universe of annotated 3d objects","cited_arxiv_id":null,"evidence_quote":"Objaverse-LVIS is one of the three zero-shot classification benchmarks used to evaluate the method."}],"review_version":1}