{"id":"6fd4a32a-0e6a-4d64-b2a7-0a73cc605d56","arxiv_id":"2506.14471","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"Introduces a 160K-panorama auto-annotated dataset, a dense captioning and grounding benchmark, and ERP-RoPE; fine-tuning Qwen2.5VL on the data lifts benchmark scores.","lead":"Dense360 assembles 160,000 panorama images with automatically generated dense captions, referring expressions, and scene descriptions, and adds a benchmark plus a position encoding tweak for panoramic inputs. The paper shows large score gains when fine-tuning Qwen2.5VL on this data, but the benchmark and training labels come from the same model-generated pipeline with no human verification.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The benchmark's ground truth and training annotations come from the same auto-generated pipeline, so Dense360VLM's gains may reflect mimicking the generator rather than true dense understanding; no human validation establishes benchmark validity.","rationale":"The central claim requires Dense360-Bench to be a valid instrument for measuring omnidirectional dense understanding. The reader's weakest assumption identifies exactly the vulnerability: the ground truth for both training and evaluation is produced by the same automatic pipeline. My reading confirms this and adds a specific technical detail: the 'reliability score' in Section 3.2 only checks region-level localizability, not semantic truth. InternVL3 first generates the caption, then grounds it, and SAM produces a mask; the IoU with the entity mask only verifies that the caption can be associated with the correct region of the ERP image. Attributes or events described in the caption are never independently checked, so hallucinations within the correct region pass the filter. Consequently, Dense360-Bench's captioning metric—which asks ChatGPT-4o to check phrase recall against these same auto-generated phrases—rewards models that mimic the generator. The reported gains of Dense360VLM over SA2VA (51.78 vs 47.80 captioning, 76.81 vs 74.39 grounding) are therefore consistent with overfitting to the generator, not with a genuine improvement in dense understanding. The paper reports no human validation or inter-annotator agreement, and no external benchmark is used. This is more fundamental than the incomplete ERP-RoPE derivation, since even a correct positional encoding could not rescue a claim measured on an invalid benchmark. The proposed human validation on a random subset would directly test whether the auto-generated labels agree with human judgment. If they do, the concern is largely resolved; if not, the central claim is unsupported. I therefore concur with the REJECT verdict.","tokens_in":13584,"tokens_out":5564,"duration_ms":53467,"concrete_test":"Take a random subset of 300 Dense360-Bench entities. Have two independent human annotators (i) judge whether each key phrase in the detailed caption is actually present/true in the ERP image, and (ii) validate the grounding mask by drawing the correct mask if it is wrong. Recompute captioning recall using only human-verified phrases and grounding IoU against human-corrected masks. If the automatic metrics disagree significantly with human-verified scores, or if the model ranking changes (e.g., Dense360VLM no longer beats SA2VA), then the benchmark is not a valid measure of dense understanding and the central claim is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim requires Dense360-Bench to be a valid measure of omnidirectional dense understanding. Section 3.2's verification pipeline computes a reliability score as the IoU between a mask produced by SAM after InternVL3 grounds the caption and the entity mask from Level-1. Because InternVL3 also generates the captions, high IoU only shows the caption can be localized to the entity's region; it does not verify the semantic content (attributes, predicates, counts) is true, so hallucinated details within the correct region pass. Level-3 then builds scene descriptions from these captions, and Dense360-Bench's captioning metric extracts key phrases from the same detailed captions and asks ChatGPT-4o whether a predicted caption mentions them. Thus both training targets and evaluation references derive from one automatic pipeline, and a model that imitates the generator's hallucination patterns scores high. The paper reports no human validation or agreement study, so the central claim that Dense360VLM achieves 'significant improvements for omnidirectional dense understanding' is not established; the benchmark may be measuring agreement with the generator, not true understanding.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper introduces Dense360, a large automatically generated dataset of 160K equirectangular panoramas with 5M entity-level captions, 1M referring expressions, and 100K entity-grounded scene descriptions; it also presents Dense360-Bench, a captioning/grounding benchmark built from 1,279 curated panoramas, and Dense360VLM, a Qwen2.5VL-based model extended with a proposed ERP-RoPE positional encoding. The authors report that Dense360VLM-3B outperforms a post-trained SA2VA-4B on the benchmark (51.78 vs 47.80 captioning; 76.81 vs 74.39 grounding) and that the proposed ERP-RoPE contributes a large grounding gain in ablations. The central claims are that the dataset, benchmark, and position encoding together constitute a significant step for omnidirectional dense understanding.","tokens_in":13781,"tokens_out":7913,"duration_ms":75457,"significance":"If the benchmark and annotations were independently validated, this would be a useful resource: it is one of the first attempts to scale dense entity captioning and grounding to omnidirectional panoramas, and the three-tier generation pipeline is clearly described. The paper's empirical results are internally consistent, and the geometric motivation for ERP-specific position encoding addresses a real problem. However, the significance is conditional on resolving the benchmark validity issue described below: because training targets and evaluation references come from the same unvalidated automatic pipeline, the reported gains may reflect distribution matching rather than genuine dense understanding. The paper also reports no human-validated subset or external validation, so the central claim is not established as written.","major_comments":[{"comment":"The validity of Dense360-Bench as a measure of dense understanding is not established. Section 3.2 uses InternVL3 to generate both brief and detailed entity captions; Section 3.3 uses GPT-4o to write scene descriptions from those captions; Section 5.1 evaluates captioning by extracting key phrases from the same detailed captions and asking ChatGPT-4o whether a predicted caption mentions them. Thus both the training targets and the evaluation references are produced by the same automatic pipeline. A model that imitates the generator's output distribution will score high even if it hallucinates semantic content, so the reported gains for Dense360VLM could measure agreement with the generator rather than true understanding. No human validation, human agreement study, or independent benchmark is reported. Please provide a human-validated evaluation subset (e.g., human ratings of caption correctness and human-written references) and rerun the headline results on it, or otherwise demonstrate that the generated references agree with human judgments.","section":"§3.2 Verification Pipeline"},{"comment":"The reliability score only verifies localizability, not semantic correctness. The pipeline computes IoU between a SAM mask obtained after InternVL3 grounds a brief caption and the Level-1 entity mask. This shows that the caption can be located in the entity's region, but it does not verify that the caption's attributes, relations, counts, or predicates are true, because hallucinated details inside the correct region can still yield high IoU. Since InternVL3 also generated the caption, high IoU can reflect self-consistency between the captioning and grounding heads of the same model. Therefore the description of the annotations as 'reliability-scored' overstates what is verified, and the benchmark's grounding and captioning references inherit this unresolved issue.","section":"§3.2"},{"comment":"The derivation of ERP-RoPE is not internally coherent. Equation (1) sums cosθ Wγ over an unspecified set of θ values and then divides by W to obtain γ = H/Σ cosθ, but the reparameterization (h, γf(w)) applies a single global γ while the vertical coordinate g(h)=h is unchanged, so latitude-dependent information density is not actually reflected in per-token encoding. In addition, the implementations of f(w) in Eqs. (3) and (4) do not satisfy the Table 2 properties in a clear way: for W=5, f(1)=1 and f(W)=2, so the periodic extension f(w)=f(w+W) fails at the boundary, and the 'maximum at ERP center' is only approximate. The authors should either correct the derivation, define f(w) on a circle with proper wrap-around boundary conditions, or reframe the contribution as an empirical heuristic with ablations against simpler alternatives such as circular padding of the visual feature map.","section":"§4.1"},{"comment":"The grounding evaluation protocol and the ablation interpretation need more support. In Section 5.1, Qwen2.5VL and InternVL3 are evaluated by prompting them to output textual bounding boxes and then running SAM on those boxes, while SA2VA and Dense360VLM directly output masks; this protocol mismatch can confound the comparisons, especially for the grounding metric. In Table 4, the full-data row without ERP-RoPE gives 60.43 grounding accuracy while the full-data row with ERP-RoPE gives 76.81, a 16.38-point jump that is unusually large; no failure-case analysis or comparison with alternative periodic position encodings is provided. I recommend reporting grounding under a common protocol (e.g., all models output masks, or all models output boxes followed by SAM) and adding ablations on a human-validated subset to rule out artifacts of the evaluation pipeline.","section":"§5.1, §5.2"}],"minor_comments":[{"comment":"The section heading reads 'Dens360 Dataset and Benchmark' and should be corrected to 'Dense360'.","section":"§3"},{"comment":"The text says 'as shown in Tab. 3' but the ablation study with data components and ERP-RoPE is reported in Table 4; the cross-reference is wrong.","section":"§5.2"},{"comment":"The table footnote defines the dagger symbol for post-trained MLLMs, but ChatGPT4o-latest has missing entries for all grounding columns without explanation; please add a note that grounding was not evaluated for that model.","section":"Table 3"},{"comment":"Many reference entries contain missing spaces in venue names, e.g., 'InNeurIPS' and 'InCVPR'; please correct the bibliography formatting.","section":"References"},{"comment":"The Limitations paragraph contains a typo, 'a'l the proposed', and the sentence structure should be revised for clarity.","section":"Limitations"}],"recommendation":"major_revision","confidential_remarks":"The main risk is that the benchmark's evaluation protocol is self-referential: the training targets, evaluation references, and judging process all derive from the same automatic pipeline with no human validation. I would not require the authors to abandon the benchmark, but a revision should include a substantial human validation study (e.g., several hundred entities with human caption-correctness ratings and human mask agreement) and rerun all headline numbers on the human-validated subset. If the authors instead argue that automatic references are acceptable, they should provide evidence that the generator's captions agree with human judgments. Without this, acceptance would set a risky precedent for generated benchmarks."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The short version: Dense360 is a real resource—160K panoramas with dense entity captions, referring expressions, and scene descriptions—and the authors have assembled a plausible first benchmark for panoramic captioning and grounding, plus an ERP-tuned RoPE variant. But the headline numbers (51.78 vs 47.80 captioning, 76.81 vs 74.39 grounding) are not yet evidence of 'dense understanding' because both the training targets and the evaluation references come out of the same automated pipeline, with no human validation reported.\n\nWhat's genuinely new: the scale and the combination—dense entity-level captions, referring expressions, and grounded scene descriptions for panoramas—are not present in KITTI-360, EGO-K360, PanoVOS, JRDB-PanoTrack, or 360+X. The three-tier annotation pipeline (CropFormer masks, InternVL3 captions with reliability scores, GPT-4o scene descriptions) is a reasonable way to bootstrap a large dataset, and the reliability-score concept is a sensible nod to the noise problem. The ERP-RoPE is a sensible adaptation of mRoPE; the intuition about horizontal continuity and latitude-dependent density is correct, and the ablation in Table 4 suggests it helps, particularly on back-direction captioning.\n\nWhere it gets soft. The circularity is the load-bearing issue. Section 3.2's verification pipeline uses InternVL3 to ground the caption and SAM to produce a mask, then compares IoU with the entity mask. That only checks that the caption can be localized to the region—it does not check whether the attributes, predicates, or counts in the caption are true. The same captions then become the training targets, and Dense360-Bench's captioning metric extracts key phrases from those same captions and uses ChatGPT-4o as judge. A model that imitates the generator's style will score well even if it reproduces its hallucinations. No human agreement study is reported. This is not necessarily fatal to the dataset as a resource, but it means the reported gains do not establish the central claim; they may only show agreement with the generator.\n\nThe ERP-RoPE derivation also has a gap. Equation 1 asserts a constraint on gamma and Equation 2 solves it, but the constraint itself is not derived from the stated geometry, and the triangular f(w) in Equations 3-4 is described as 'naive' with no proof that it satisfies the properties in Table 2. That is a minor issue if the empirical ablation is robust, but the ablation has no error bars and the benchmark uses a single split.\n\nWho this is for: people building panoramic or 360-degree MLLMs, and anyone working on automatic dataset annotation for dense understanding. As submitted, the comparison tables are internally consistent and the benchmark is usable as a stress test of ERP-aware models, but the validity argument needs a human-verified subset and a clear separation between the training pipeline and the evaluation references.\n\nMy take: send it to peer review, but reviewers should push for a human-validated subset, error bars, and a proper justification of the position encoding. If the authors can show the benchmark tracks human judgment, the resource is valuable enough to warrant publication despite the current circularity.","headline":"Dense360 builds a substantial panoramic dataset and benchmark, but the headline gains are undermined by a circular training/evaluation pipeline and a missing human-validated subset.","tokens_in":14374,"tokens_out":2076,"would_cite":false,"duration_ms":20718,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that equirectangular panoramas, read through a position encoding that respects their wrap-around and latitude geometry, give multimodal language models dense 360° scene understanding.","keywords":["omnidirectional panorama","equirectangular projection","dense captioning","referring expression grounding","multimodal large language model","position encoding","reliability-scored annotations","360-degree benchmark"],"falsifier":"Re-annotate a random sample of Dense360-Bench entities with human verifiers, producing human-checked masks and captions, then recompute Dense360VLM's captioning and grounding scores on that subset. If agreement with the automatic labels is low, or if the model's margin over baselines shrinks on the human-validated subset, the benchmark would be measuring agreement with the generator rather than true dense understanding.","tokens_in":13320,"feed_emoji":"🌐","tokens_out":8663,"duration_ms":77494,"temperature":0.7,"pith_summary":"The paper claims that multimodal language models can be taught dense understanding of 360° scenes by feeding them a single equirectangular panorama instead of many perspective views, provided the model's position encoding respects the geometry of that projection. To support the claim it contributes three things: a 160K-panorama dataset with 5M entity-level captions, 1M referring expressions, and 100K entity-grounded scene descriptions, each caption carrying an automatic reliability score; Dense360-Bench, a first benchmark for omnidirectional captioning and grounding; and ERP-RoPE, a position encoding that makes the horizontal axis wrap around and corrects for latitude-dependent pixel density. With ERP-RoPE and the dataset, a 3B model reaches 51.78 captioning and 76.81 grounding on the benchmark, above a 4B grounding-specialist model post-trained on the same data (47.80 and 74.39). If these numbers hold, a compact single-image input can replace multi-view feature fusion for panoramic scene understanding.","feed_headline":"A 3B model with 360° position encoding outperforms larger baselines","feed_subtitle":"New reliability-scored dataset and benchmark teach models to read 360° panoramas as one continuous scene.","key_machinery":"ERP-RoPE is the load-bearing object: a rotary position encoding for equirectangular images that replaces the ordinary horizontal coordinate with a periodic, center-peaked coordinate f(w) and rescales it by gamma = H / sum(cos theta) so that all latitude circles receive a matched total positional budget. These two choices encode wrap-around continuity (the left and right borders are adjacent) and latitude-dependent information density (pixels near the poles are physically closer together). The paper plugs ERP-RoPE into the multimodal rotary position embedding of a vision-language model, adds a [SEG] token decoded into segmentation masks by SAM, and trains with LoRA on the Dense360 dataset, while perspective images and text keep the standard mRoPE encoding.","core_discovery":"On its own terms, the central discovery is that the two geometric facts of equirectangular projection—horizontal continuity, where the image's left and right edges are adjacent in the scene, and latitude-dependent distortion, where a pixel row near a pole covers less physical circumference than the equator—are not handled by standard rotary position embeddings, and that encoding them directly into the position embedding is enough to unlock dense panorama understanding. The paper integrates ERP-RoPE into a vision-language model by reparameterizing each pixel's horizontal position as gamma times f(w), where f(w) is a periodic coordinate peaked at the horizontal center and gamma equals H divided by the sum of cos(theta) over all latitudes, equalizing information density. Trained on the Dense360 dataset, this Dense360VLM-3B improves captioning by 5.92 points and grounding by 16.38 points over the same model without ERP-RoPE, and outperforms a post-trained 4B baseline. The paper also presents Dense360-Bench, with 3,000 grounding and 3,000 captioning questions across 1,279 ERP images balanced across the front, back, left, and right directions, as the first shared evaluation suite for this task.","pith_inferences":["A natural extension this paper does not test is that the same wrap-around position encoding could transfer to 360° video, cylindrical projections, or panoramic depth and range images, where the same two geometric facts hold.","Because the benchmark's captioning score is phrase recall judged by a language model, scores may partly reflect the judge's language preferences; a small human-validated subset would separate caption quality from judge style, but the paper reports no such check.","The paper's argument that one ERP image beats multi-view inputs at the input level is a claim about representation efficiency; a direct comparison against multi-view methods at matched compute and data would settle it, but the paper does not run that comparison.","If the reliability scores are faithful, the verification pipeline itself is a reusable method for hallucination filtering in dense caption generation, independent of the panorama setting."],"forward_implications":["Any multimodal language model that adopts ERP-RoPE can consume a single equirectangular image as input, avoiding the cost of multi-view feature fusion for 360° scenes.","Dense360-Bench gives a standard way to compare future models on omnidirectional captioning and grounding, with explicit back-direction evaluation for entities split across the image seam.","The reliability-score mechanism in the data pipeline can flag low-confidence caption-mask pairs at scale, making automatic dense annotation more trustworthy.","Training with all three data components together yields the best results, suggesting that captioning, referring, and scene-description tasks reinforce each other rather than compete.","The back-direction gain supports the specific claim that wrap-around encoding repairs a known failure mode: entities straddling the ERP image's left and right borders."],"supporting_citations":[{"why":"Supplies the base vision-language model and multimodal rotary position embedding that ERP-RoPE modifies, and the strongest open-source baseline in the benchmark.","marker":"[2]"},{"why":"Generates the dense entity captions in the Level-2 pipeline and grounds brief captions for reliability scoring; also serves as an open-source baseline.","marker":"[79]"},{"why":"Reference small-scale grounding-specialist model that is post-trained on the Dense360 dataset and compared against Dense360VLM.","marker":"[65]"},{"why":"Performs entity segmentation on ERP slice views and the full image, producing the candidate entity masks for the Level-1 pipeline.","marker":"[51]"},{"why":"Converts grounded boxes into masks for caption reliability scores and decodes the [SEG] token into grounding masks for Dense360VLM.","marker":"[54]"},{"why":"Supplies the visual instruction data, 30 percent of which is mixed with the Dense360 dataset during training.","marker":"[41]"},{"why":"Closest prior panorama dataset, used to position Dense360's dense entity-level annotations against existing scene and action labels.","marker":"[7]"},{"why":"Recognition model in the tag pipeline that produces semantic tag information for each entity-centric view.","marker":"[25]"},{"why":"Grounding model that generates tag-conditioned masks, matched to entities by mask IoU to attach semantic priors.","marker":"[56]"}],"fun_headline_variants":["360° position encoding unlocks dense panorama understanding","Panorama-aware encoding beats bigger baselines in grounding","Dense360: dataset and benchmark for panoramic vision","ERP-RoPE: position encoding for equirectangular images","Position encoding that respects 360° geometry wins"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole approach assumes that the automatically generated, reliability-scored captions and masks are faithful ground truth for both training and evaluation, yet the paper reports no human validation of them.","fun_headline_variants_meta":{"raw":{"variants":["360° position encoding unlocks dense panorama understanding","Panorama-aware encoding beats bigger baselines in grounding","Dense360: dataset and benchmark for panoramic vision","ERP-RoPE: position encoding for equirectangular images","Position encoding that respects 360° geometry wins"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000647,"raw_usage":{"total_tokens":3005,"prompt_tokens":1013,"completion_tokens":1992,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":629,"completion_tokens_details":{"reasoning_tokens":1916}},"tokens_in":629,"tokens_out":1992,"duration_ms":15369,"temperature":1.0,"reasoning_tokens":1916,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T00:17:57.243772+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-annotate a random sample of Dense360-Bench entities with human verifiers, producing human-checked masks and captions, then recompute Dense360VLM's captioning and grounding scores on that subset. If agreement with the automatic labels is low, or if the model's margin over baselines shrinks on the human-validated subset, the benchmark would be measuring agreement with the generator rather than true dense understanding.","supporting_citations":[{"cited_title":"High quality entity segmentation","cited_arxiv_id":null,"evidence_quote":"Performs entity segmentation on ERP slice views and the full image, producing the candidate entity masks for the Level-1 pipeline."},{"cited_title":"360+ x: A panoptic multi-modal scene understanding dataset","cited_arxiv_id":null,"evidence_quote":"Closest prior panorama dataset, used to position Dense360's dense entity-level annotations against existing scene and action labels."},{"cited_title":"Aligning and prompting everything all at once for universal visual perception","cited_arxiv_id":null,"evidence_quote":"Grounding model that generates tag-conditioned masks, matched to entities by mask IoU to attach semantic priors."}],"review_version":1}