{"id":"1c770963-25d3-4b58-948c-39cdf3baae17","arxiv_id":"2607.24436","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"Hierarchical point-shuffle densification plus local AVS-Conv multi-scale decoding lets compact VecSet VAEs approach voxel-level 3D reconstruction fidelity at much lower token and query cost.","lead":"MSVS-VAE is a hierarchical set-based 3D VAE that densifies compact anchored latents and decodes them with local geometry-aware operators. It aims to match voxel-level reconstruction fidelity while staying far more compact and faster to query than prior set-based VAEs.","discovery_kind":"new_method","skeptic_critique":{"model":"moonshotai/kimi-k3","headline":"The fixed-budget \"outperforms\" claim may be inflated by oracle anchor placement: Sec. 4.2 derives anchor positions from active voxels intersected by the mesh, injecting coarse occupancy structure that unanchored set baselines (Dora, HY3D2.1) do not receive at the same token count.","rationale":"The reader correctly flagged the external anchor dependency as the weakest assumption, and I agree it is the right neighborhood; I disagree only in emphasis. The reader framed it as a transfer risk (\"may not transfer to a pure set pipeline or shapes outside the anchor generator's support\"). I think the sharper issue is evaluation fairness on the paper's own numbers: the equal-token-budget comparisons that ground \"consistently outperforms\" give MSVS-VAE (and, to be fair, Lattice) structural side information that Dora/HY3D2.1 lack, and the compactness currency ignores anchor coordinates and any external structure input. Credit where due: the PSU ablation (Tab. 3, w/o PSU collapses F1@0.001 from 99.6 to 93.3) shows a genuine architectural effect independent of anchor provenance, and the latency scaling in Tab. 4 (latency roughly flat from 10k to 40k tokens vs Lattice's ~4x growth) is consistent with the claimed complexity advantage of local aggregation, not an artifact of anchors. So the architectural contribution looks real; what is unsettled is the size of the headline margin and the 10x-compactness framing. Because the reader's verdict is already CONDITIONAL with exactly this family of conditions attached (repro details, isolation from the external anchor source), my concern sharpens the conditions rather than moving the verdict; hence UNCHANGED, with the anchor-source ablation as the specific experiment that would settle it.","tokens_in":14892,"tokens_out":2041,"duration_ms":74198,"concrete_test":"Run an anchor-source ablation at fixed token budgets (4k and 20k) on the Objaverse test set: (i) anchors = active-voxel centers from the GT mesh (as in Sec. 4.2), (ii) anchors = farthest-point-sampled positions from the input surface point cloud (no occupancy oracle, no external model), (iii) anchors = perturbed/pruned versions of (i) simulating an imperfect external generator (e.g., drop 10–20% of active voxels, dilate by one voxel). If setting (ii) loses most of the MD/F1@0.001 gap over Lattice-4k, the equal-budget comparison is driven by oracle structure rather than the proposed modules; if (iii) degrades sharply, the generative-pipeline transfer claim fails. Additionally, report compactness in bits (tokens + anchor coordinates) so the 10x-compactness claim can be checked on equal footing.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Sec. 4.2 states anchors C are \"sampled ... as voxel queries from the centers of active voxels intersected by the mesh, obtained from an off-the-shelf generative model (e.g., HY3D2.1).\" The sentence is ambiguous between (a) voxelizing the input mesh itself and (b) importing a sparse structure from an external generator, but under either reading the anchor set encodes where geometry lives — i.e., a coarse occupancy map. This matters for the central claim in two ways. First, the marquee comparison at equal token budget (Tab. 1, 4k row: MSVS-VAE MD 5.644 vs Dora 17.352 vs HY3D2.1 11.485) is not apples-to-apples: MSVS-VAE's 4k tokens ride on 4k anchor positions that carry explicit spatial support information derived from the shape's occupancy, whereas Dora/HY3D2.1 tokens must encode both position and content. Anchored-vs-unanchored is a legitimate design choice (Lattice shares it, and MSVS beats Lattice too), but the abstract's framing \"consistently outperforms prior set-based ... VAEs\" and \"~10x higher compactness\" treats tokens as the sole currency of compactness while the anchor positions — 3 coordinates per token, plus, under reading (b), a dependence on an external model's structure output — are not counted. Second, under reading (b), reconstruction fidelity is conditioned on the external generator's active-voxel structure being correct and complete; the paper provides no ablation isolating anchor-source quality, so it is unknown how much of the PSU/MSQ gain survives when anchors are imperfect (the actual deployment case in a generative pipeline). This is not an internal contradiction — the architecture is coherent and the PSU ablation (Tab. 3) shows a real architectural contribution — but the quantitative margin that constitutes the headline claim is entangled with un-counted structural side information and an unablated external dependency.","agreement_with_reader":"partial"},"referee_report":{"model":"moonshotai/kimi-k3","summary":"The paper proposes MSVS-VAE, a set-based 3D VAE intended to close the reconstruction-fidelity gap between compact VecSet-style latents and sparse voxel-based representations. Three components are introduced: (1) hierarchical point-shuffle upsampling (PSU), which progressively densifies anchored VecSet latents (each latent split into r children with predicted geometric offsets and halved channels) to raise spatial capacity; (2) AVS-Conv, a PointConv-style KNN-based local aggregation operator that replaces global cross-attention in both encoder and decoder, reducing query cost; and (3) multi-scale query decoding (MSQ), which fuses AVS-Conv features queried independently at each latent scale to balance global context against local detail. On Objaverse, ABO, in-the-wild shapes, and Dora-Bench sharp-edge subsets, the method reports large gains over set-based baselines (Dora, HY3D2.1, Lattice) at equal token budgets, matches or exceeds voxel-based baselines (SparC3D, SparseFlex-1024, Direct3D-S2) at ~10–40% of their token counts, and achieves 5.5–14.4× lower query latency than Lattice. Ablations identify PSU as the critical component and MSQ as a consistent smaller gain; test-time scaling with token budget is demonstrated up to 80K tokens.","tokens_in":15363,"tokens_out":3802,"duration_ms":142723,"significance":"If the numbers hold, this is a meaningful step for latent-diffusion 3D generation: set-based VAEs have trailed voxel-based ones on fine geometry, and closing that gap at 20K tokens with 5.5–14.4× decoding speedups over the direct predecessor would matter to the many pipelines (CLAY, TripoSG, Hunyuan3D) built on VecSet-style latents. The paper ships the right kind of evidence for an empirical systems paper: multi-benchmark quantitative tables including sharp-edge metrics on Dora-Bench, component ablations that isolate PSU as the critical piece and MSQ as a consistent refinement, a latency table, and test-time scaling curves. The wins over Lattice — the only like-for-like anchored baseline — are large and consistent across all four token budgets and three datasets, which is the strongest and fairest evidence in the paper. No code or machine-checked artifacts are provided, and two of the baselines (Lattice, SparC3D) are the authors' own re-implementations, which tempers reproducibility credit.","major_comments":[{"comment":"The sentence \"we sample latent anchor points C ... from the centers of active voxels intersected by the mesh, obtained from an off-the-shelf generative model (e.g., HY3D2.1)\" is ambiguous between (a) voxelizing the input mesh itself (which is always available at VAE training/eval time and is presumably what Lattice does) and (b) importing the sparse structure predicted by an external generator. The distinction is load-bearing for the paper's framing as a self-contained set-based VAE: under reading (b), both reconstruction fidelity and the downstream generative pipeline depend on the external model's active-voxel structure being correct and complete, and no ablation isolates anchor-source quality. Please (i) state unambiguously where anchors come from at training time, at reconstruction-eval time, and in a full generation pipeline; (ii) add a robustness ablation with degraded anchors — e.","section":"§4.2 (anchor sampling)"},{"comment":"The abstract advertises \"~10x higher compactness than voxel-based baselines\" and Tab. 1 compares at equal token count (e.g., the 4k row: MSVS-VAE MD 5.644 vs Dora 17.352 vs HY3D2.1 11.485). But each MSVS-VAE token carries an explicit anchor position (3 coordinates) derived from the shape's occupancy structure — a coarse occupancy map that unanchored baselines (Dora, HY3D2.1) do not receive at the same token budget, and whose 3M additional scalars (plus, under reading (b) above, a dependence on an external structure model) are not counted in the compactness ledger. Anchored-vs-unanchored is a legitimate design axis, and the comparison against Lattice — which shares the anchored design and is beaten soundly at every budget — is fair and is in fact the strongest evidence in the paper. But the token-only accounting should be corrected: state the total latent dimensionality (positions + featu","section":"Abstract and Table 1 (compactness accounting)"},{"comment":"The abstract claims MSVS-VAE \"consistently outperforms prior set-based and voxel-based VAEs,\" but Table 2 does not support this at the 20K budget: on Level-3 S-MD, SparseFlex scores 3.21 vs MSVS-VAE-20K 3.22, and on Level-4 S-MD SparseFlex scores 3.08 vs 3.58 — i.e., a voxel-based baseline is better on the sharp-edge metric the table is designed to highlight, on both difficulty levels. The claim holds at 40K tokens. Please either soften the abstract wording (e.g., \"matches or outperforms ... at comparable or smaller budgets\") or make explicit in the text that the sharp-metric advantage over SparseFlex requires the 40K configuration.","section":"Table 2 vs Abstract (\"consistently outperforms\")"},{"comment":"The two most important comparators in Tab. 1 — Lattice (the design ancestor and fair anchored baseline) and SparC3D (the strongest voxel baseline on sharp metrics) — are the authors' own re-implementations (marked †). Their numbers presumably differ from the published ones, and no detail is given on training data, steps, or whether the re-implementations reproduce the published results on a common benchmark. Since the headline claim is a large margin over Lattice at every token budget, the fairness of that re-implementation is load-bearing. Please report (i) the re-implementation protocol, (ii) a sanity comparison of re-implemented vs published numbers where available, and (iii) whether re-implemented baselines were given the same 400k-mesh training set and comparable compute.","section":"§5.2 (re-implemented baselines, † in Tables 1–2)"}],"minor_comments":[{"comment":"Key hyperparameters are not reported in the main text: number of upsampling stages L, split factor r, KNN size K, latent channel dimensions C_l, and the resulting M_l at each scale. These are needed to reproduce the architecture and to interpret the token-budget rows in Tab. 1. Please add a configuration table.","section":"§4.3 / §5.1"},{"comment":"The hierarchy is defined as Z^{(l)}, l = 0,...,L with Z^{(0)} = Z, but the fusion in Eq. (5) concatenates h_i^{(1)} through h_i^{(L)}. Either the coarsest scale is excluded from fusion (in which case say so and explain) or the indexing is off by one.","section":"Eq. (5) (indexing)"},{"comment":"Unit conventions are inconsistent: Table 1 states MD is scaled by ×10^4, Table 2 states \"MD and S-MD are scaled by 10^{-4}\" — the latter is presumably meant to read ×10^4 as well. Also, no absolute length scale is given (are meshes normalized to a unit cube/sphere?), which is needed to interpret MD and the F1 thresholds.","section":"Tables 1–2 (MD scaling)"},{"comment":"No hardware is specified for the latency benchmark, and the comparison is only against Lattice; a voxel-based decoder latency row (even if unfavorable, as the Limitations section concedes) would make the efficiency story more complete. Also state whether cuBQL KNN build time is included in the reported latency.","section":"Table 4 (latency)"},{"comment":"The progressive crop fine-tuning stage supervises only near-surface queries within a crop, but evaluation metrics in Tab. 1 are computed on full shapes. Please clarify how full-shape latents are produced at test time (single global forward pass with M tokens, or chunked encoding with merging?), since this affects whether the reported numbers reflect the global or the locally fine-tuned regime.","section":"§5.1 (progressive crop training)"},{"comment":"Fig. 5's ablation differences are described as visible only on zoom; consider adding error heatmaps or insets. Typo: \"MSVS-V AE\" appears with a stray space in several places (title, §4.1 heading, Table 2). Fig. 7 is referenced before Fig. 6 in the text. No code/model release statement is included; given that two baselines required re-implementation, releasing at least evaluation code would substantially strengthen the paper.","section":"Figures 5–7 / presentation"}],"recommendation":"major_revision","confidential_remarks":"Two things the editor may wish to know. (1) Several key comparators (Lattice, Trellis2, FaithC, Hunyuan3D 2.5) are arXiv-only preprints from the last ~8 months, and two of the four main baselines are the authors' own re-implementations; the empirical SOTA claim is therefore resting on a fast-moving, partly unvetted baseline landscape. This does not diminish the technical contribution but counsels some caution about the strength of the \"state-of-the-art\" framing. (2) The author list is largely industrial (Lightspeed Studios); no code-release statement appears in the manuscript, so the reproducibility credit the field increasingly expects is currently absent. I recommend asking about release plans regardless of the revision outcome."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The useful takeaway is simple: densifying an anchored VecSet with point-shuffle upsampling, then decoding with local KNN aggregation (AVS-Conv) plus multi-scale fusion, closes most of the reconstruction gap to heavy sparse-voxel VAEs at far fewer tokens and with much faster set decoding.\n\nWhat is actually new is the composition, not any single operator. Anchored tokens (Lattice), sub-pixel/point-shuffle style expansion, PointConv-style local weights, and coarse-to-fine query fusion are all known. Putting them into a progressive hierarchy that keeps the base latent compact while growing spatial support only at decode time is the real move, and the numbers back it. Against Lattice at matched token budgets the gains are large and consistent (Objaverse/ABO/in-the-wild + Dora-Bench sharp metrics). The PSU ablation is decisive; MSQ is a smaller but real cleaner. Latency table shows local decoding scales far better than global cross-attention. Progressive global-then-crop training is a practical detail that lets them push high-frequency detail without blowing memory. Citation pattern is normal for the subfield.\n\nSoft spots, in proportion. The stress-test is half-right. Anchors are active-voxel centers derived from the mesh (or an off-the-shelf generator). That injects coarse occupancy that pure unanchored sets (Dora, HY3D) must encode inside the tokens themselves, so the 4k-token headline margins versus those two are not pure apples-to-apples. Lattice shares the anchor design and is still beaten, so the architectural claim survives; the abstract’s “consistently outperforms prior set-based” framing just over-sells the unanchored comparisons. There is also no ablation on imperfect or generator-derived anchors—the actual generative-pipeline case—and no code/data release, so transfer risk is real but secondary. Metrics lack error bars; a few baselines are re-implementations. None of this is load-bearing math failure.\n\nThis is for people building latent-diffusion 3D backbones who care about the set-vs-voxel tradeoff. Worth a serious referee. I would read the camera-ready and probably cite the PSU+local-decode pattern if I am in that lane.","headline":"Solid hierarchical densification of anchored VecSets that really does lift set-based VAE fidelity; the equal-token wins over unanchored baselines are partly inflated by free occupancy anchors, but the Lattice comparison and ablations still hold.","tokens_in":16515,"tokens_out":570,"would_cite":true,"duration_ms":11953,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"A compact set-based 3D VAE can match voxel-level reconstruction by densifying anchored latents and decoding them locally across scales.","keywords":["3D VAE","VecSet","latent diffusion","point-shuffle upsampling","local aggregation","multi-scale decoding","SDF reconstruction","compact 3D representation"],"falsifier":"Train and evaluate the same architecture with anchors sampled only from the input surface (no off-the-shelf voxel generator) on the same Objaverse/ABO/in-the-wild splits; if Mesh Distance and F1@0.001 collapse relative to the reported 20k–40k token numbers, the central claim fails.","tokens_in":16085,"feed_emoji":"🧊","tokens_out":889,"duration_ms":21269,"temperature":0.7,"pith_summary":"High-fidelity 3D generation is bottlenecked by the VAE that turns shapes into latents for diffusion. Voxel VAEs reconstruct fine geometry well but need huge latent budgets; set-based VAEs stay compact and continuous but blur details because the latent set is sparse and decoding is overly global. This paper claims that gap is closable without giving up compactness: start from an anchored VecSet, progressively densify it with hierarchical point-shuffle upsampling, decode with a local geometry-aware operator instead of global attention, and fuse coarse-to-fine queries so long-range structure and local detail both survive. On Objaverse, ABO, and in-the-wild tests the method beats prior set and voxel VAEs while reporting roughly 10× faster decoding than earlier set methods and roughly 10× more compact latents than voxel baselines. A sympathetic reader cares because latent diffusion only scales if the VAE is both faithful and cheap to sample from.","feed_headline":"Compact set latents match voxel 3D fidelity at ~10× less cost","feed_subtitle":"Densified anchored VecSets plus local multi-scale decoding close the set-vs-voxel reconstruction gap.","key_machinery":"Hierarchical Point-Shuffle Upsampling of anchored VecSets, decoded by AVS-Conv (KNN-local geometry-aware aggregation) and Multi-Scale Query Decoding that fuses coarse global context with fine local residuals.","core_discovery":"MSVS-VAE shows that hierarchical densification of anchored VecSet latents, paired with local AVS-Conv aggregation and multi-scale query fusion, lets a compact continuous set representation reach or surpass voxel-based reconstruction fidelity while remaining far smaller and decoding much faster than prior set-based VAEs.","pith_inferences":["If anchors must come from another generator, end-to-end pure-set pipelines may still need a learned anchor proposal stage before this densify-and-decode recipe fully replaces voxel pipelines.","The same multi-scale local query pattern could transfer to appearance or texture latents, which the paper flags as future work.","Sharp-feature metrics improve but the continuous implicit still struggles on knife-edge geometry; dual-contouring-style cues might be the next bottleneck to close."],"forward_implications":["Set-based latents become a practical backbone for latent diffusion without sacrificing fine geometry.","Token budgets around 20k can already beat voxel baselines that use 50k–210k tokens on the reported benchmarks.","Local neighborhood decoding (AVS-Conv) makes dense surface querying scale with latent size far better than global cross-attention.","Test-time increases in latent count can be used as a quality dial without changing the trained weights.","Watertight continuous surfaces remain available while approaching voxel-level detail, reducing topology brittleness from discrete sparse grids."],"fun_headline_variants":["Hierarchical VecSet densification closes set-vs-voxel 3D gap","Anchored multi-scale sets match voxel fidelity at far lower cost","Local AVS-Conv plus densified VecSets lift compact 3D VAE quality","MSVS-VAE: set latents hit voxel-level 3D recon with 10× speedups","Coarse-to-fine VecSet fusion beats prior set and voxel VAEs"],"cache_read_input_tokens":128,"weakest_assumption_plain":"The method depends on external active-voxel anchors from an off-the-shelf model; if those anchors miss structure or are unavailable, the claimed fidelity-compactness tradeoff may not hold in a pure set pipeline.","fun_headline_variants_meta":{"raw":{"variants":["Hierarchical VecSet densification closes set-vs-voxel 3D gap","Anchored multi-scale sets match voxel fidelity at far lower cost","Local AVS-Conv plus densified VecSets lift compact 3D VAE quality","MSVS-VAE: set latents hit voxel-level 3D recon with 10× speedups","Coarse-to-fine VecSet fusion beats prior set and voxel VAEs"]},"model":"grok-4.5","effort":"low","cost_usd":0.004592,"raw_usage":{"total_tokens":1333,"prompt_tokens":797,"num_sources_used":0,"completion_tokens":112,"cost_in_usd_ticks":45924000,"prompt_tokens_details":{"text_tokens":797,"audio_tokens":0,"image_tokens":0,"cached_tokens":128},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":424,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":797,"tokens_out":112,"duration_ms":9145,"temperature":1.0,"reasoning_tokens":424,"cache_read_input_tokens":128,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-31T14:56:38.626013+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"Train and evaluate the same architecture with anchors sampled only from the input surface (no off-the-shelf voxel generator) on the same Objaverse/ABO/in-the-wild splits; if Mesh Distance and F1@0.001 collapse relative to the reported 20k–40k token numbers, the central claim fails.","supporting_citations":[],"review_version":1}