{"id":"0980d143-94cb-4aa2-aa7f-69c42c31c4ae","arxiv_id":"1908.07070","paper_version":1,"verdict":"ACCEPT","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A deep network that predicts per-pixel surface frames in two coordinate systems and solves a weighted least-squares alignment to recover camera orientation, improving accuracy by 20-30% over prior methods.","lead":"UprightNet estimates a camera's tilt and roll from a single indoor photo by predicting the shape of surfaces in both camera and world coordinates, and then finding the rotation that fits the two predictions together. It beats prior single-image orientation methods on synthetic, real, and cross-dataset benchmarks, with 20-30% lower average angular error.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Global upright frame's x/y axes are undefined, making the f_z^g supervision potentially scene-dependent.","rationale":"The reader's weakest assumption is the domain assumption about sufficient supporting structure, which is a real limitation but is acknowledged in Section 5 and does not threaten the average-case central claim. The more precise and load-bearing concern is the undefined global coordinate frame: the entire method hinges on supervising and predicting f_z^g, and if the global x/y axes are not canonically defined, the labels for the tangent components are arbitrary. The paper's reported success suggests a specific convention was used, but the text does not state it. A single consistency check on ground-truth label generation would settle whether this concern lands. Until then, the central claim is not fully reproducible, so the verdict should be CONDITIONAL: accept if the authors specify the convention, otherwise the published numbers cannot be independently verified.","tokens_in":12466,"tokens_out":28065,"duration_ms":292038,"concrete_test":"Compute ground-truth f_z^g labels for a sample of ScanNet test images using two conventions: (a) each scene's raw reconstruction y-axis and (b) the yaw-preserving upright frame y_g = R y_c. Measure the agreement of t_z^g and b_z^g on corresponding wall pixels. If the labels disagree, the paper's intermediate supervision is ambiguous and the reported Table 2 numbers depend on an unstated convention; training with convention (b) and checking whether the results still hold would settle the impact.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The load-bearing step is the construction of the global upright surface frames F_g (Eq. 1–2) and the training labels f_z^g. Section 3 defines the global upright coordinate system only by saying its camera up vector aligns with the scene up vector; it never specifies the global x/y axes. It then defines the tangent t as n × (camera y-axis) — a definition that is unambiguous in the local camera frame but not in the global frame. If the global y-axis is chosen as the per-scene reconstruction y-axis (ScanNet world frames are arbitrary per scene), then t_g^z and b_g^z for non-Manhattan surfaces, and even the sign of t_g^z for vertical walls, depend on an arbitrary scene convention. The supervision loss LF (Eq. 13) would then ask the network to predict a quantity that is not a function of the image. The paper's claim that f_z^g is 'consistent in the same supporting surfaces across images' holds only for n_g^z on ground/wall/ceiling; it is not established for the tangent components. If the authors instead used the natural convention y_g = R y_c (the yaw-preserving upright frame), they need to state it; otherwise the published accuracy numbers cannot be reproduced or checked.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces UprightNet, an end-to-end deep network for estimating 2DoF camera orientation (roll and pitch) from a single indoor RGB image. The central idea is to predict per-pixel surface frames in both local camera coordinates (F_c) and a global upright coordinate system (f_z^g), then solve for the camera up vector as the unit vector that best aligns these two predictions via a differentiable weighted constrained least-squares module. Training is end-to-end with a combination of an orientation loss, surface-frame losses, and gradient-consistency losses. Experiments on InteriorNet, ScanNet, and a SUN360 cross-dataset test show consistent improvements over regression, classification, and classical baselines, and ablations support the design choices of the predicted weights and the full surface-frame representation.","tokens_in":12742,"tokens_out":12466,"duration_ms":120403,"significance":"If the formulation is sound, this is a valuable contribution to single-image camera calibration. The paper convincingly demonstrates that injecting explicit geometric reasoning through dual surface-frame prediction and a differentiable constrained least-squares solver can improve both accuracy and cross-dataset generalization relative to black-box regression or classification. The differentiable solver itself is a reusable module, and the extensive experimental comparison and ablation study strengthen the empirical case. The main concern is a gap in the definition of the global upright coordinate system, which affects the mathematical foundation and reproducibility of the method.","major_comments":[{"comment":"The global upright coordinate system is not fully specified: the paper states only that its up vector aligns with the scene up vector, leaving the global x/y axes undefined. Because the tangent vector t is defined as n × (camera y-axis), the analogous definition in the global frame depends on the arbitrary global y-axis. For a vertical wall, t_g^z is the sine of the angle between the wall normal and the global y-axis, so its sign and magnitude change under a scene-level yaw rotation. Consequently, the ground-truth f_z^g used in the surface-frame loss L_F (Eq. 13) is not well-defined as a function of the image, and Eq. (1) F_g = R F_c holds only if R maps the camera y-axis to the global y-axis, which is generally false for a camera with nonzero yaw. The authors should specify a canonical global yaw convention (e.g., projecting the camera's right vector onto the horizontal plane) or redefine the tangent vectors in a yaw-invariant way, then re-derive the alignment equation and re-run the experiments under that convention.","section":"Section 3, Eqs. (1)-(2) and Eq. (13)"},{"comment":"The paper repeatedly describes improvements as 'significant,' but no statistical significance tests, confidence intervals, or variance across multiple training runs are reported. On ScanNet, the gain over Hold-Geoffroy et al. is about 0.8 degrees in mean angular error (3.74 vs. 4.55), which may not be robust given that the baselines were re-implemented by the authors and that only a single split is used. Please report error bars, significance tests, or at least a discussion of variance to support the claim of significant improvement.","section":"Section 4.3, Tables 1-5"}],"minor_comments":[{"comment":"The paper states that the derivative of the eigenvalue is computed in closed form using [45], but the actual derivative formula and its backpropagation details are not presented. A short derivation or a pointer to the supplementary material would improve reproducibility.","section":"Section 3.1, Eqs. (7)-(8)"},{"comment":"In the row 'Ours (w/o Lo)', the median pitch error (2.83°) is reported as larger than the mean pitch error (1.21°), which is atypical for an angular error distribution; please check this entry for a possible typo.","section":"Table 3"},{"comment":"The description of InteriorNet says 'Each scene includes 3 images randomly sampled from a rendered videos.' This should read 'a rendered video' or 'rendered videos.'","section":"Section 4.1"},{"comment":"The nested-norm notation in Eq. (9) is confusing; it should be a single L2 norm of the vector W(i)(u^T F_c(i) - f_z^g(i)).","section":"Eq. (9)"},{"comment":"The caption says the weight maps are overlaid on the input images, but the overlay alpha or blending mode is not specified; adding this detail would improve clarity.","section":"Figure 4 caption"}],"recommendation":"major_revision","confidential_remarks":"The main obstacle is the definition of the global upright frame. If the authors can provide a precise, consistent definition that makes the ground-truth f_z^g well-defined and independent of scene-level yaw, the paper is likely acceptable. The lack of statistical significance tests is a standard but important concern for the empirical claims."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"UprightNet is a solid, well-executed paper and worth engaging. The genuinely new piece is the representation: predicting per-pixel surface frames (normal plus two tangents) in both the camera and the upright coordinate system, then solving for the up vector as a differentiable weighted constrained least-squares alignment between the two. The least-squares solver itself is a known constrained eigenvalue problem (Gander et al. 1989), but the application, the predicted weight maps, and the end-to-end training are a clean contribution. The experiments back it up: consistent 20-30% relative improvements over re-implemented baselines on InteriorNet and ScanNet, and better cross-dataset generalization to SUN360. The ablations are thorough and support the design choices; it is genuinely interesting that the learned weights concentrate on line junctions.\n\nThe stress-test worry about undefined global x/y axes does not survive contact with Eq. (2): f_z^g is defined as u^T F_c, the projection of the camera-frame surface frame onto the scene up vector. The first two rows of the global rotation never enter, so the arbitrary yaw convention of the global upright coordinate system is irrelevant. The paper could state this explicitly; right now a reader might stumble on exactly that question.\n\nSoft spots are real but not load-bearing. There are no error bars or repeated-run statistics, so the claimed improvements are point estimates. The baselines are re-implemented rather than taken from released code, though using the same network architecture for all methods is a fair choice. No code or data are released, which hurts reproducibility. The method is inherently 2DoF and assumes enough structured indoor surface; the paper acknowledges the failure mode, and the assumption is a reasonable domain restriction, not a hidden flaw.\n\nThis is a paper for someone working on single-image calibration, AR, or geometry-aware deep learning. It deserves a serious referee. I would recommend accept with minor revision, mainly asking for error bars and a clearer statement of the coordinate conventions. Bring it to the reading group if you want a clean example of how to inject explicit geometry into an end-to-end training pipeline.","headline":"Solid geometry-aware calibration paper; the dual surface-frame representation and differentiable weighted solver are the real contributions, and the stress-test concern about global axes dissolves once you see Eq. (2).","tokens_in":13241,"tokens_out":4107,"would_cite":true,"duration_ms":42775,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A network that predicts two surface geometries at every pixel estimates camera roll and pitch more accurately than direct regression, by solving for the up vector that best aligns them.","keywords":["camera orientation estimation","surface frames","constrained least squares","single-image calibration","roll and pitch","indoor scene geometry","geometric deep learning","up vector"],"falsifier":"Build a test set of indoor images with little or no visible floor, ceiling, or wall plane (close-up furniture views, cluttered corners), run UprightNet and the direct-regression baseline on it, and check whether the up-vector error gap shrinks or reverses; the alignment mechanism predicts the method should lose its advantage exactly where planar supporting structure is absent.","tokens_in":12303,"feed_emoji":"🧭","tokens_out":9689,"duration_ms":83997,"temperature":0.7,"pith_summary":"The paper proposes that camera roll and pitch can be estimated more accurately from a single indoor image when the network is made to predict per-pixel surface geometry in two coordinate systems and solve for the camera's up vector as the rotation that aligns them, rather than regressing angles directly. At each pixel it predicts an orthonormal surface frame (surface normal plus two tangent directions) in local camera coordinates, and a matching scene-layout vector in a global upright coordinate system. The alignment is a weighted constrained least-squares problem that is solved analytically and differentiably, so the whole model can be trained end-to-end from ground-truth orientations while also being supervised on the intermediate geometry. The result is an average up-vector angular error of 1.17 degrees on the InteriorNet test set, against 1.76 degrees for the strongest prior CNN-based calibration method, with the advantage persisting on real ScanNet frames and on a cross-dataset SUN360 test. Learned per-pixel weights, which concentrate on line junctions and plane boundaries, are shown to be necessary for the improvement.","feed_headline":"Camera tilt error falls to 1.17 degrees","feed_subtitle":"Geometric alignment of local and global surface frames beats direct regression on roll and pitch.","key_machinery":"The load-bearing object is the per-pixel surface frame $\\mathbf{F}(i)=[\\mathbf{n}(i)\\ \\mathbf{t}(i)\\ \\mathbf{b}(i)]$, an orthonormal basis composed of the surface normal and two tangent vectors. The network predicts these frames in camera coordinates and, in global upright coordinates, only the third row $\\mathbf{f}^g_z$ of the global frame, a scene-layout vector whose normal component sets floor, wall, and ceiling pixels to fixed values. The identity $\\mathbf{f}^g_z(i)=\\mathbf{u}^T\\mathbf{F}_c(i)$ turns the unknown up vector into the solution of a constrained least-squares problem, and the solver reduces it to a quadratic eigenvalue problem $I\\lambda^2-2H\\lambda+H^2-\\mathbf{g}\\mathbf{g}^T=0$ with $H=A^TA$ and $\\mathbf{g}=A^T\\mathbf{b}$. Because the needed eigenvalue derivative is closed-form, the orientation loss propagates through the solver to the network, while separately predicted per-pixel weights make the fit robust to unreliable regions by concentrating on line junctions and structure edges.","core_discovery":"The central claim is that the information needed for single-image roll and pitch is already present in local surface geometry, but only becomes usable when the same geometry is also predicted in a global upright coordinate system. Writing the per-pixel surface frame as $\\mathbf{F}_c(i)$ and the third row of the global upright frame as $\\mathbf{f}^g_z(i)$, the scene up vector $\\mathbf{u}$ must satisfy $\\mathbf{f}^g_z(i)=\\mathbf{u}^T\\mathbf{F}_c(i)$ at every pixel, so orientation estimation reduces to a weighted constrained least-squares problem in $\\mathbf{u}$ alone. The paper shows this problem has an analytic solution through a quadratic eigenvalue problem, and that the eigenvalue derivative is available in closed form, making the entire alignment differentiable and trainable by back-propagating the orientation error. With this formulation, the average up-vector error drops from 1.76 degrees to 1.17 degrees on InteriorNet relative to the best prior CNN-based method, with consistent gains on ScanNet and a large margin on the SUN360 cross-dataset test.","pith_inferences":["The differentiable two-frame alignment is not tied to indoor scenes: replacing the global upright frame with a sky-versus-ground semantic prior could give outdoor roll and pitch estimation the same structure.","Because the orientation loss alone already improves accuracy, a self-supervised variant could be built that aligns predicted frames across generated rotations, reducing the need for dense ground-truth normals.","A testable consequence of the paper's mechanism is that accuracy should depend on how much of the image is covered by planar supporting structures; partitioning a test set by that coverage would draw the boundary where the method loses its advantage over direct regression."],"forward_implications":["Single-image roll and pitch can be framed as an alignment of two predicted geometry maps, and this framing is trainable end-to-end through a differentiable constrained least-squares solver.","Supervising the intermediate surface frames in both coordinate systems adds accuracy on top of the orientation loss, so the method can exploit partial geometric ground truth when full camera poses are available.","The learned weight maps form an interpretable by-product: they highlight line junctions and plane boundaries, the same regions that carry vanishing-point information in classical calibration.","Cross-dataset results on SUN360 indicate the geometric intermediate representation transfers to new indoor scene distributions better than direct regression or horizon classification."],"supporting_citations":[{"why":"Supplies the quadratic-eigenvalue formulation and closed-form eigenvalue derivative that make the constrained least-squares up-vector solver differentiable.","marker":"[14]"},{"why":"The prior CNN-based calibration method that serves as the strongest baseline in the comparison.","marker":"[19]"},{"why":"A CNN horizon-line classification baseline used throughout the experiments for accuracy comparison.","marker":"[48]"},{"why":"The classical vanishing-point-based upright-adjustment baseline representing geometry-based calibration.","marker":"[26]"},{"why":"Provides the synthetic InteriorNet images with ground-truth camera poses and surface normals used for training and evaluation.","marker":"[28]"},{"why":"Provides the real ScanNet RGB-D frames and 3D reconstructions used to train and test on real imagery.","marker":"[10]"},{"why":"Panorama source for the SUN360 cross-dataset test set, used to measure generalization to unseen indoor images.","marker":"[49]"},{"why":"Supplies rendered surface normals for ScanNet frames, used as the ground-truth supervision for the intermediate geometry.","marker":"[53]"}],"fun_headline_variants":["Geometry-aware network cuts camera tilt error to 1.17°","UprightNet: infer camera orientation via geometric alignment","Differentiable least squares improves single-image roll/pitch","Local-global surface alignment beats black-box orientation regression","Camera roll and pitch from geometric scene reasoning"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes a single indoor image contains enough visible flat, structured surfaces—floors, ceilings, walls, and their junctions—to give a clear alignment signal between the two predicted geometries; the paper's own limitations section says images without such supporting structure lead to inaccurate predictions.","fun_headline_variants_meta":{"raw":{"variants":["Geometry-aware network cuts camera tilt error to 1.17°","UprightNet: infer camera orientation via geometric alignment","Differentiable least squares improves single-image roll/pitch","Local-global surface alignment beats black-box orientation regression","Camera roll and pitch from geometric scene reasoning"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000988,"raw_usage":{"total_tokens":4163,"prompt_tokens":891,"completion_tokens":3272,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":507,"completion_tokens_details":{"reasoning_tokens":3194}},"tokens_in":507,"tokens_out":3272,"duration_ms":21241,"temperature":1.0,"reasoning_tokens":3194,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T12:26:47.535906+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Build a test set of indoor images with little or no visible floor, ceiling, or wall plane (close-up furniture views, cluttered corners), run UprightNet and the direct-regression baseline on it, and check whether the up-vector error gap shrinks or reverses; the alignment mechanism predicts the method should lose its advantage exactly where planar supporting structure is absent.","supporting_citations":[{"cited_title":"A con- strained eigenvalue problem","cited_arxiv_id":null,"evidence_quote":"Supplies the quadratic-eigenvalue formulation and closed-form eigenvalue derivative that make the constrained least-squares up-vector solver differentiable."},{"cited_title":"A perceptual measure for deep single image camera calibration","cited_arxiv_id":null,"evidence_quote":"The prior CNN-based calibration method that serves as the strongest baseline in the comparison."},{"cited_title":"Hori- zon lines in the wild","cited_arxiv_id":null,"evidence_quote":"A CNN horizon-line classification baseline used throughout the experiments for accuracy comparison."},{"cited_title":"Automatic upright adjustment of photographs with robust camera calibration","cited_arxiv_id":null,"evidence_quote":"The classical vanishing-point-based upright-adjustment baseline representing geometry-based calibration."},{"cited_title":"Interiornet: Mega-scale multi-sensor photo-realistic indoor scenes dataset","cited_arxiv_id":null,"evidence_quote":"Provides the synthetic InteriorNet images with ground-truth camera poses and surface normals used for training and evaluation."},{"cited_title":"Scannet: Richly- annotated 3d reconstructions of indoor scenes","cited_arxiv_id":null,"evidence_quote":"Provides the real ScanNet RGB-D frames and 3D reconstructions used to train and test on real imagery."},{"cited_title":"Recognizing scene viewpoint using panoramic place representation","cited_arxiv_id":null,"evidence_quote":"Panorama source for the SUN360 cross-dataset test set, used to measure generalization to unseen indoor images."},{"cited_title":"Deep depth comple- tion of a single rgb-d image","cited_arxiv_id":null,"evidence_quote":"Supplies rendered surface normals for ScanNet frames, used as the ground-truth supervision for the intermediate geometry."}],"review_version":1}