{"id":"d9cc37ef-2978-4d33-a42a-66cd24635ff2","arxiv_id":"2608.03323","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":13,"one_line_summary":"A multi-view layout estimator that fits Manhattan 3D polygons to multiple rooms jointly, using learned feature alignment and adaptive topology refinement.","lead":"PolyLayout turns multi-view photos of indoor scenes into 3D room layouts shaped like Manhattan polygons, optimizing all rooms together. It outperforms earlier image-based methods on new multi-room benchmarks, though the gains are partly due to a stronger neural network backbone.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Coverage-dependent evaluation: visibility-based image sampling ensures all walls are observed, so reported gains may not generalize to arbitrary multi-view captures.","rationale":"The reader's weakest assumption points to the initialization and coverage precondition, and the paper's own supplementary material confirms that unobserved regions are a known failure mode. This is indeed the most load-bearing concern for the empirical claim: the benchmarks are constructed with visibility-based sampling that guarantees near-ideal coverage, so the reported 'outperforms prior approaches in robustness' is not demonstrated for arbitrary or degraded multi-view inputs. The concern is not fatal—the method may still be state-of-the-art under good coverage—but it narrows the claim. The reader's CONDITIONAL verdict already captures this, so no change to the verdict is needed. A concrete test with uniform random sampling would settle whether the advantage persists without the curated coverage.","tokens_in":422,"tokens_out":10092,"duration_ms":166209,"concrete_test":"Re-run the ASE evaluation with alternative image sets sampled uniformly at random from the full camera trajectory, without visibility-based scoring, without excluding images outside the room layout, and with the same 10 images per room. Compare PolyLayout and PixCuboid (and optionally SceneScript with the same COLMAP point clouds) on wall recall and IoU. If PolyLayout's wall recall drops by more than, say, 10 points or its IoU advantage over PixCuboid shrinks by more than half, the reported robustness and accuracy gains are coverage-dependent. Additionally, deliberately remove all views of one wall per room to confirm that the missed wall is unrecoverable.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that PolyLayout 'outperforms prior approaches, both in terms of accuracy and robustness' is supported by results on benchmarks whose image sets are explicitly curated to maximize wall coverage. In Sec. 7.4, ASE images are sampled with a visibility-based score (Eq. 9) that selects views to maximize coverage of floor, ceiling, and walls, and images are excluded if the camera is outside the ground-truth layout or if more than 10% of the view is a door to another room. ScanNet++ similarly excludes images whose camera center is outside the layout. This creates a favorable coverage distribution for PolyLayout, whose initialization (Sec. 3.4) and optimization require the walls to be observed: the initial polygon is derived from camera positions via an α-shape buffered by δ=3m, and unobserved walls cannot be pushed to the true geometry. The paper's own failure cases (Fig. 8) attribute errors to 'parts of the scene not being observed in any view.' Therefore, the empirical support for the robustness claim is conditional on a strong coverage precondition that is engineered into the evaluation, not a naturally occurring property of multi-view captures. This is not an internal inconsistency, but it makes the headline claim broader than the current evidence.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes PolyLayout, a multi-view multi-room Manhattan layout estimation method. Room layouts are represented as 3D Manhattan polygons and estimated by optimizing a learned featuremetric objective, with explicit, model-based camera projection and polygon updates, and adaptive wall split/simplification during optimization. Rooms are optimized jointly, sharing orientation and optionally floor/ceiling height. The authors introduce two new annotated benchmarks based on Aria Synthetic Environments and ScanNet++ v2, and report that PolyLayout outperforms prior multi-view and point-cloud baselines on accuracy and robustness. The main text includes ablations on parameter sharing, network backbone, cost terms, initialization, and topology updates, plus an experiment with predicted camera poses and a comparison to point-cloud cuboid fitting.","tokens_in":17158,"tokens_out":5668,"duration_ms":58813,"significance":"If the results hold, this is a useful advance: it generalizes cuboid-only multi-view layout estimation to Manhattan polygons, introduces an explicit geometry/learned-scoring separation that may transfer across datasets, and provides two new benchmarks with annotations. The ablation study is informative, and the paper is candid about failure cases. The core optimization formulation is plausible and the comparisons with alternative cost terms and networks are valuable. However, the evaluation protocol has several load-bearing weaknesses, described below, that prevent the current version from fully supporting the headline robustness and generalization claims.","major_comments":[{"comment":"The aggregation rule for the per-scene metrics is not defined. The text says 'we adopt ... metrics proposed in [13], but compute them per scene instead of per room for ASE and ScanNet++' without specifying whether IoU is the mean of per-room IoUs, a scene-level volume union/intersection, or another rule; the same ambiguity applies to Chamfer distance and the recall metrics. Since Table 1 mixes methods that run per room (PixCuboid, Plane-DUSt3R) with methods that run per scene (PolyLayout, RoomFormer, SceneScript), the comparison is only meaningful if the aggregation is exactly the same and reported. Please provide the precise formulas and report both per-scene and per-room aggregates for all methods.","section":"Sec. 4.3, Table 1"},{"comment":"The evaluation protocol encodes a strong coverage prior that is load-bearing for PolyLayout. ASE images are selected with a visibility-based sampling score (Eq. 9), and images whose camera center is outside the ground-truth layout or with >10% door pixels are excluded; ScanNet++ similarly excludes images whose camera center is outside the layout. PolyLayout's initialization is an alpha-shape of camera positions buffered by delta=3 m, and the failure cases in Fig. 8 show that walls unobserved in all views cannot be recovered. The reported numbers are therefore conditional on engineered coverage and do not support the Abstract's unqualified robustness claim. Please evaluate on naturally sampled or unfiltered image sets, and quantitatively vary the image coverage (number of views, visibility-aware vs. random sampling) to demonstrate robustness.","section":"Sec. 7.4, Sec. 3.4, Fig. 8"},{"comment":"Baseline comparisons involve oracle information or distribution leakage. SceneScript's point cloud is filtered to points inside the ground-truth layouts or within 1 m of them, and the ASE test set is stated to be a subset of SceneScript's training scenes. RoomFormer lifts 2D polygons to 3D using ground-truth floor/ceiling height. For 2D-3D-Semantics, single-view methods are evaluated by selecting the prediction with maximum IoU against ground truth. These protocol choices make the relative comparison difficult to interpret. The paper should either avoid ground-truth-derived filtering/lifting, or present an additional comparison where all baselines use the same amount of oracle information, and clearly disclose the effect of these choices.","section":"Sec. 5.1"},{"comment":"No uncertainty quantification is provided. All metrics are point estimates on a single split (100 ASE scenes, 80 ScanNet++ scenes, 160 rooms in 2D-3D-S). The large gaps over SceneScript and Plane-DUSt3R are clear, but several claims in Sec. 5.2 rely on small differences (e.g., orientation sharing vs. none, DINOv2 vs. ResNet, and the 2D-3D-S comparison where PixCuboid has higher wall/room recall in Table 1). Report standard deviations, bootstrap confidence intervals, or significance tests over scenes and/or image subset samplings so that the reader can assess whether the differences are meaningful.","section":"Tables 1 and 2"}],"minor_comments":[{"comment":"The statement 'No other method show competitive results on this dataset' (2D-3D-Semantics) is contradicted by PixCuboid's wall recall 93.8 vs. 92.2 and room recall 85.0 vs. 80.6 in the same table. Please soften or qualify this claim.","section":"Sec. 5.1, Table 1"},{"comment":"The perimeter cost includes a sum over i=4..p-1 of |d_{i+1} - d_{i-1}|, but the relationship between the plane offsets and the polygon edge lengths is not explicit. A short derivation would help the reader understand the geometric meaning of each term.","section":"Sec. 3.2, Eq. (7)"},{"comment":"Free parameters such as the alpha-shape parameter, rasterization pixel size (1 m), and buffer distance delta=3 m are stated only in the supplementary. Since the ablation study does not vary these parameters, the sensitivity of the method to them is unknown; please report at least a validation-set sensitivity check.","section":"Sec. 7.1, Sec. 3.4"},{"comment":"The prediction-time comparison excludes DeepLSD line detection and dense COLMAP reconstruction. This should be stated directly in the main text, not only in the table footnote, to avoid an unfair efficiency impression.","section":"Sec. 5.1, Table 1"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"PolyLayout is a real step forward for multi-view room layout estimation. It takes the featuremetric optimization idea from the authors' own PixCuboid and generalizes cuboids to Manhattan polygons with automatic wall split/merge during optimization, plus joint multi-room optimization with shared orientation and floor/ceiling height. That is a meaningful practical advance. The DINOv2-based network and the end-to-end training through LM steps are a natural and well-executed continuation. The ablations are genuinely informative: they show parameter sharing helps, the concave-hull initialization beats cuboid/circle inits, and the perimeter cost improves 3D metrics. The comparison against point-cloud baselines (RANSAC, ICP-style fitting) is fair and makes the case that learned featuremetric costs beat direct distance costs.\n\nThe main soft spot is not the method but the evaluation protocol. On the new ASE and ScanNet++ benchmarks the image sets are deliberately sampled to maximize wall coverage (Sec. 7.4), excluding images where the camera is outside the layout or sees too much of a door. That means the test distribution is favorable to a method whose initialization and optimization require every wall to be visible in at least one view. The paper's own failure cases (Fig. 8) attribute errors to unobserved walls. So the 'outperforms... robustness' claim is broader than the current evidence; it should say 'on image sets with sufficient coverage.' Also, no error bars or statistical tests are reported, and the per-scene aggregation for ASE/ScanNet++ is not fully specified. On 2D-3D-Semantics, PixCuboid actually achieves higher wall and room recall (93.8 vs 92.2 and 85.0 vs 80.6), so 'outperforms' is not uniform across all metrics there. The same holds against SceneScript on ASE, which beats PolyLayout on wall recall when given the semi-dense point cloud.\n\nNone of this undermines the core contribution. The optimization is mathematically sound, the ablations support the design choices, and the new benchmarks are a useful resource. The paper deserves a serious referee. I would ask the authors to add error bars, specify the aggregation, temper the robustness claim, and ideally report results on image sets with less curated coverage or at least analyze sensitivity to coverage. Letting this through with minor revisions after those clarifications seems right.","headline":"PolyLayout is a credible next step from the authors' own PixCuboid, generalizing cuboids to Manhattan polygons with adaptive topology and multi-room sharing; strong results on new benchmarks, but the coverage-curated evaluation makes the robustness claim broader than the evidence.","tokens_in":17655,"tokens_out":2328,"would_cite":true,"duration_ms":22605,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"PolyLayout recovers multi-room 3D layouts from posed images by optimizing a learned featuremetric cost over Manhattan polygons, and reports accuracy rivaling point-cloud methods on new benchmarks.","keywords":["room layout estimation","multi-view","Manhattan world","featuremetric alignment","polygon parameterization","multi-room optimization","ScanNet++","indoor scene understanding"],"falsifier":"Take a real room with known floor plan and place all cameras near its center, so the α-shape initialization is a small polygon far from the true walls; predict the layout with PolyLayout. If the method recovers the outer walls anyway, the learned featuremetric cost can overcome poor initialization; if, as the paper's design implies, it converges to a layout close to the small initial polygon, that confirms the initialization dependence.","tokens_in":16723,"feed_emoji":"🏠","tokens_out":9327,"duration_ms":82254,"temperature":0.7,"pith_summary":"The paper presents PolyLayout, a method that reconstructs the 3D layout of one or more rooms (walls, floor, ceiling) from a set of posed perspective images, without needing point clouds. Each room is represented as a Manhattan polygon whose edges align with three principal axes, and the layout is refined by minimizing a cost that combines multi-view consistency of learned deep features, edge alignment, vanishing-point agreement, and a perimeter regularizer. The core claim is that separating the learned scoring from explicit model-based geometry (camera projection and polygon updates) makes the method generalize to new datasets and camera parameters, and that jointly optimizing multiple rooms while sharing orientation and floor/ceiling height improves accuracy. On two new benchmarks derived from Aria Synthetic Environments and ScanNet++ v2, PolyLayout outperforms prior image-based methods, and on the synthetic set its accuracy approaches that of a point-cloud baseline (SceneScript) that uses a semi-dense SLAM point cloud.","feed_headline":"Image-only room layout method rivals point-cloud accuracy","feed_subtitle":"Jointly optimizing Manhattan polygons across rooms beats single-room and point-cloud methods on new benchmarks.","key_machinery":"The core machinery is the Manhattan polygon parameterization with a learned cost. In the local frame, a room is described by a rotation $R$ and plane offsets $d=(d_1,\\dots,d_p)$: floor $z=d_1$, ceiling $z=d_2$, and walls alternating between $x$ and $y$ planes. The optimization minimizes $E(P)=E_{\\mathrm{feat}}+\\alpha E_{\\mathrm{edge}}+\\beta E_{\\mathrm{VP}}+\\gamma E_{\\mathrm{per}}$, where $E_{\\mathrm{feat}}$ warps deep features between views via the polygon's induced homographies, $E_{\\mathrm{edge}}$ aligns polygon edges with predicted edge maps, $E_{\\mathrm{VP}}$ matches the three vanishing points to detected line segments, and $E_{\\mathrm{per}}$ penalizes perimeter complexity to prevent uno","core_discovery":"The central discovery is that a room layout can be estimated accurately by treating it as a parametric Manhattan polygon and optimizing a learned featuremetric cost with explicit, differentiable geometry. The method uses a DINOv2-based network to predict dense feature, edge, and confidence maps; these maps define an optimization objective that measures how well a candidate polygon explains the multi-view consistency of the features, the location of predicted edges, and the vanishing points implied by the Manhattan frame. The polygon is initialized from the camera positions using an α-shape concave hull, then refined with Levenberg-Marquardt steps while walls are adaptively split or merged. B","pith_inferences":["If the separation of learned scoring from model-based geometry is the true source of generalization, similar 'learned cost + explicit optimization' designs could improve other structured reconstruction tasks, such as floor-plan recovery from panoramas, where current methods either fix the model class or rely on black-box prediction.","The method's dependence on the camera-derived initial polygon suggests that a learned or search-based initialization might recover walls not visible in any single view, a limitation the paper's own failure cases attribute to unobserved scene parts.","The benchmarks cover only Manhattan (or near-Manhattan) rooms; adapting the polygon framework to Atlanta-world or non-orthogonal walls would be a natural next step and remains untested.","At roughly 3.5–5.5 seconds per scene, the method is not yet real-time, but the architecture is modular; replacing the DINOv2 encoder with a lighter backbone could trade accuracy for speed in robotics or AR applications."],"forward_implications":["PolyLayout can represent general Manhattan rooms (L-shaped, etc.) without fixing the number of walls in advance; the wall count is determined at inference by split and merge operations.","Jointly optimizing multiple rooms with shared orientation and floor/ceiling height improves accuracy and convergence compared with per-room estimation, so the method scales naturally to building-scale scenes.","Because the method uses only posed RGB images, it applies to data from SLAM or Structure-from-Motion pipelines, without requiring depth sensors or dense point clouds.","On the newly introduced ASE and ScanNet++ v2 benchmarks, PolyLayout outperforms PixCuboid, Plane-DUSt3R, and RoomFormer, and its ASE accuracy is close to that of SceneScript with a semi-dense SLAM point cloud.","The two new benchmarks provide the first multi-view multi-room layout evaluation sets with ground-truth 3D layouts, enabling future comparisons."],"supporting_citations":[{"why":"PixCuboid establishes the featuremetric alignment framework and training procedure that PolyLayout extends from cuboids to arbitrary Manhattan polygons.","marker":"[13]"},{"why":"DINOv2 provides the pretrained visual features whose multi-view consistency forms the core optimization objective.","marker":"[27]"},{"why":"The Manhattan world assumption defines the geometric model that the polygon parameterization relies on (walls aligned with three principal axes).","marker":"[6]"},{"why":"The α-shape algorithm is used to build the initial polygon from camera positions, which determines the reachable layout.","marker":"[8]"},{"why":"Barron's robust loss is used in the cost and training loss to downweight outliers during optimization.","marker":"[5]"},{"why":"The Visvalingam-Whyatt simplification algorithm is adapted to remove converged walls while preserving the Manhattan property.","marker":"[42]"},{"why":"ScanNet++ v2 provides one of the two new multi-view multi-room benchmarks and the training data for the network.","marker":"[49]"},{"why":"Aria Synthetic Environments provides the synthetic multi-room benchmark and the point-cloud baseline SceneScript that the method is compared against.","marker":"[3]"}],"fun_headline_variants":["Joint room layout optimization beats single-room methods","Multi-room Manhattan polygons improve layout accuracy","Learned scoring plus explicit geometry for room layouts","PolyLayout: Joint Manhattan layout across rooms","Robust multi-view room layout via adaptive polygons"],"cache_read_input_tokens":2816,"weakest_assumption_plain":"The method can only recover walls that are visible in at least one input image and whose location is reachable from the initial polygon built from the camera positions; if the cameras do not encircle the room footprint or a wall is never seen, that wall cannot be pulled to its true position.","fun_headline_variants_meta":{"raw":{"variants":["Joint room layout optimization beats single-room methods","Multi-room Manhattan polygons improve layout accuracy","Learned scoring plus explicit geometry for room layouts","PolyLayout: Joint Manhattan layout across rooms","Robust multi-view room layout via adaptive polygons"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000235,"raw_usage":{"total_tokens":1318,"prompt_tokens":710,"completion_tokens":608,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":454,"completion_tokens_details":{"reasoning_tokens":551}},"tokens_in":454,"tokens_out":608,"duration_ms":6358,"temperature":1.0,"reasoning_tokens":551,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T21:00:12.438154+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a real room with known floor plan and place all cameras near its center, so the α-shape initialization is a small polygon far from the true walls; predict the layout with PolyLayout. If the method recovers the outer walls anyway, the learned featuremetric cost can overcome poor initialization; if, as the paper's design implies, it converges to a layout close to the small initial polygon, that confirms the initialization dependence.","supporting_citations":[{"cited_title":"In: Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision (ICCV) Workshops (2025)","cited_arxiv_id":null,"evidence_quote":"PixCuboid establishes the featuremetric alignment framework and training procedure that PolyLayout extends from cuboids to arbitrary Manhattan polygons."},{"cited_title":"In: International Conference on Learning Representations (ICLR) (2025)","cited_arxiv_id":null,"evidence_quote":"DINOv2 provides the pretrained visual features whose multi-view consistency forms the core optimization objective."},{"cited_title":"In: International Conference on Computer Vision (ICCV) (1999)","cited_arxiv_id":null,"evidence_quote":"The Manhattan world assumption defines the geometric model that the polygon parameterization relies on (walls aligned with three principal axes)."},{"cited_title":"IEEE Transactions on information theory29(4), 551–559 (2003)","cited_arxiv_id":null,"evidence_quote":"The α-shape algorithm is used to build the initial polygon from camera positions, which determines the reachable layout."},{"cited_title":"In: Computer Vision and Pattern Recognition (CVPR) (2019)","cited_arxiv_id":null,"evidence_quote":"Barron's robust loss is used in the cost and training loss to downweight outliers during optimization."},{"cited_title":"Cartographic Journal30(1), 46–51 (1993)","cited_arxiv_id":null,"evidence_quote":"The Visvalingam-Whyatt simplification algorithm is adapted to remove converged walls while preserving the Manhattan property."},{"cited_title":"In: International Conference on Computer Vision (ICCV) (2023)","cited_arxiv_id":null,"evidence_quote":"ScanNet++ v2 provides one of the two new multi-view multi-room benchmarks and the training data for the network."},{"cited_title":"In: European Conference on Computer Vision (ECCV) (2024)","cited_arxiv_id":null,"evidence_quote":"Aria Synthetic Environments provides the synthetic multi-room benchmark and the point-cloud baseline SceneScript that the method is compared against."}],"review_version":1}