{"id":"d75dba86-9040-4441-a79c-62319fe9a3cf","arxiv_id":"2507.12166","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"A new 3D radio map dataset with DoA and ToA labels and a 3D diffusion benchmark are introduced, but the evaluation lacks baselines and covers only part of the modalities.","lead":"UrbanRadio3D adds a large simulated 3D radio map dataset with pathloss, arrival angle, and arrival time labels at 20 heights. The paper also presents a diffusion model benchmark, but the reported experiments are too incomplete to support the claimed performance.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The abstract's 'superior performance' claim for RadioDiff-3D is unsupported: the experiments contain no baseline comparisons, so no superiority is established by the reported metrics.","rationale":"The reader's stated weakest assumption was that stacking 20 independent 2D DPM simulations yields a physically consistent 3D volume; that is a legitimate secondary concern about dataset fidelity. However, the most load-bearing gap for the paper's actual abstract claim is the complete absence of baselines in the evaluation. Section V reports only absolute numbers for RadioDiff-3D and 3D-UNet, with no comparison to prior art or simple interpolation methods, so the word 'superior' has no empirical referent. This is a correctness risk in the strongest sense: the central claim cannot be true on the evidence presented, regardless of whether the dataset itself is useful. The reader's rationale already notes the missing baselines and unsupported claims, so the verdict of REJECT remains appropriate. The dataset release and public code are real contributions and should be credited, but they do not establish the benchmark's headline claim. No ad hominem is intended; the issue is the incomplete experimental design, not the integrity of the authors.","tokens_in":23444,"tokens_out":6570,"duration_ms":77525,"concrete_test":"Reproduce the pathloss experiment from Tables V and VI on the same test split and run at least (i) RadioUNet, (ii) the proposed 3D-UNet, and (iii) a classical baseline such as Kriging or IDW on the same four height slices (1-4 m), computing RMSE, NMSE, SSIM, and PSNR for all methods. If RadioDiff-3D's NMSE is not below the best baseline with non-overlapping confidence intervals, the abstract's 'superior performance' claim should be removed or qualified. Additionally, run RadioDiff-3D on ToA and DoA at heights 1-4 m; if the generative model is not evaluated on these modalities, the claim of high-dimensional 3D RM construction remains unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that 'extensive evaluations on UrbanRadio3D validate that RadioDiff-3D achieves superior performance' in constructing 3D radio maps. For this claim to hold, RadioDiff-3D must be shown to outperform existing or alternative RM construction methods on the same data and metrics. That condition is never tested. Section V reports only absolute metrics for RadioDiff-3D (Tables V and VI) and for 3D-UNet (Table VIII), with no comparison to RadioUNet, RadioDiff, RME-GAN, IDW, Kriging, or even the proposed 3D-UNet on the pathloss task. Table VI shows that DDIM with 200 steps and no sampling gives NMSE 0.3472, while adding 10% sampling gives NMSE 0.0550; neither number is anchored to a baseline, so 'superior' cannot be evaluated. Additionally, RadioDiff-3D is tested only on pathloss at heights 1-4 m, while ToA and DoA are evaluated only on 3D-UNet. Thus the claim of superiority for the generative model across 'rich, high-dimensional radio maps' is not empirically supported. The dataset release is a useful artifact, but the benchmark's headline claim is unverifiable from the reported experiments.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces UrbanRadio3D, a large-scale dataset of radio maps with pathloss, DoA (azimuth/elevation), and ToA across 20 receiver heights, generated from simulations over 701 urban regions with 200 transmitter locations each. To benchmark 3D radio map construction, the authors propose a 3D-UNet baseline and RadioDiff-3D, a diffusion model with 3D convolutions supporting radiation-aware and radiation-unaware (sparse-sampling) settings. The paper claims that extensive evaluations validate superior performance of RadioDiff-3D, but the experimental section reports only absolute metrics for RadioDiff-3D on pathloss at heights 1–4 m and for 3D-UNet on ToA/DoA, with no comparisons to prior methods.","tokens_in":23730,"tokens_out":3143,"duration_ms":36364,"significance":"If substantiated, the dataset would be a useful community resource for 3D radio map research, and the diffusion benchmark would provide a reference point for future volumetric channel modeling. The paper's strengths are the scale of the released dataset (701 maps, 200 transmitters, 20 height slices, multiple channel metrics) and the inclusion of a working 3D convolutional diffusion pipeline with publicly available code. However, the central performance claim is not supported by the reported experiments, and the physical consistency of the '3D' data is questionable. The contribution therefore remains at the level of a potentially valuable dataset release, not a validated benchmark showing superiority of the proposed method.","major_comments":[{"comment":"The abstract and introduction claim that 'RadioDiff-3D achieves superior performance' in constructing 3D radio maps, but no baseline comparison is reported anywhere in Section V. Tables V and VI give only RadioDiff-3D's own RMSE/NMSE/SSIM/PSNR on pathloss, and Table VIII gives metrics only for 3D-UNet on ToA and DoA. There is no comparison against RadioUNet, RadioDiff, RME-GAN, IDW, Kriging, or even the proposed 3D-UNet on the same pathloss task. Without such comparisons, 'superior performance' cannot be verified from the reported experiments.","section":"Section V, Tables V, VI, VIII"},{"comment":"The construction of the 3D dataset is explicitly described as separate 2D simulations at 20 height levels that are 'stacked to form a complete 3D spatial distribution.' Since each slice is simulated independently with 2D propagation paths, the resulting volume does not model vertical propagation paths between heights, so DoA/ToA values are not physically consistent across the vertical dimension. This assumption is load-bearing for both the dataset's validity as a true 3D radio map and for the 3D models trained on it. The paper should either provide evidence that the stacked slices approximate a physically valid 3D volume or clearly qualify the dataset as a set of co-registered 2D slices.","section":"Section III-A"},{"comment":"The abstract and introduction state that the dataset is 'constructed via ray tracing,' but Section III-A states that 'All data have been generated using the Dominant Path Model (DPM)' and the simulation parameters in Table III list 'Simulation type DPM.' The dominant path model is an approximate propagation model, not ray tracing. This inconsistency misrepresents the fidelity of the dataset and should be corrected in both the abstract and the main text.","section":"Abstract and Section III-A"},{"comment":"In the sparse-sampling experiments, the input includes a sampling map where 'the sampled pixels retained as they are from the original tensor, while non-sampled pixels are set to a value of 0.' Thus the model receives 10% of the ground-truth radio map values at inference, so the improvement from 0.3472 to 0.0550 NMSE in Table VI is a reconstruction from observed samples, not a prediction. The paper does not compare this setting against established interpolation or completion baselines such as Kriging or matrix completion, so it does not demonstrate that the diffusion model adds value beyond merely copying the provided samples.","section":"Section V-B, V-C, Table VI"},{"comment":"RadioDiff-3D is evaluated on pathloss only, at the four heights 1–4 m, while ToA, DoA_Azi, and DoA_Ele are evaluated only for the 3D-UNet baseline (Table VIII). The central claim of 'rich, high-dimensional radio maps' therefore lacks direct evidence: no generative-model results are reported for the angular and delay modalities. To support the headline claim, the paper needs to provide RadioDiff-3D's metrics on ToA and DoA, or explicitly restrict the claim to pathloss.","section":"Section V, Tables V–VIII"}],"minor_comments":[{"comment":"The dataset size is inconsistent: the abstract reports 2.84 million simulation instances, Table I lists a dataset size of 11.2M, and the introduction states 'over ten million labeled data points.' The relationship between simulation instances, labeled data points, and the 11.2M figure should be clarified.","section":"Abstract and Table I"},{"comment":"The text mentions both 'the WinProp module in the Altair software suite' and 'the electromagnetic simulator FEKO.' It should be stated explicitly which simulator produced the dataset and how the two tools are related.","section":"Section III-A and Table III"},{"comment":"Equation (6) is misprinted: the expression for x_{t-1} should involve the predicted noise ϵθ rather than μθ in the shown combination, and the factor structure of the DDPM update is not written consistently with Eq. (5).","section":"Section II-C, Eq. (6)"},{"comment":"The term 'GIF tensor' is used without definition; it appears to refer to the multi-frame input tensor and should be defined or replaced with a standard term such as 'input tensor.'","section":"Section V-C"},{"comment":"The row 'Sampling Information' contains the unclear entry 'Alternative Cannot Alternative Necessary'; this should be rephrased to describe whether each method requires sparse measurements as input.","section":"Table II"},{"comment":"The captions of Figures 8 and 9 refer to 'video frames,' but the data are individual height slices of a 3D volume; the wording should be changed to 'height slices' for clarity.","section":"Section V-B and Fig. 8/9"}],"recommendation":"reject","confidential_remarks":"The paper has a potentially useful dataset, but the experimental evaluation does not support the central claim of superiority, and the '3D' nature of the data rests on a stack-of-2D-slices assumption that is not physically validated. Given that these issues are load-bearing and would require new experiments and dataset validation, I do not see them as addressable by a minor revision. The manuscript may be stronger if reframed purely as a dataset paper with transparent limitations."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The dataset is the contribution; the headline performance claim is not. UrbanRadio3D looks genuinely new and useful: 20 height layers at 1 m resolution with pathloss, DoA (azimuth/elevation), and ToA co-registered, across 701 urban regions and 200 transmitter locations. That is a real step beyond RadioMapSeer, SpectrumNet, RadioGAT, and CKMImageNet, and the release is the kind of artifact worth citing and building on.\n\nWhat the paper does well: it documents the simulation pipeline, normalization thresholds, file naming, and visualization; it includes a 3D-UNet baseline with quantitative results on ToA/DoA; and it reports inference times for DDIM steps, which is useful for deployment thinking. The diffusion model itself is a fairly direct 3D extension of RadioDiff, so I would not call the framework novel.\n\nThe soft spots are real and load-bearing for the abstract. There is no comparison against any existing method—no RadioUNet, RadioDiff, RME-GAN, IDW, Kriging, or even the paper's own 3D-UNet on the pathloss task. Tables V and VI are absolute numbers, so 'superior performance' cannot be verified. The evaluation is also narrow: RadioDiff-3D is tested on pathloss at only heights 1–4 m, and ToA/DoA are evaluated only on the 3D-UNet, not on the diffusion model.\n\nI flag two internal consistency problems. The abstract says 'constructed via ray tracing,' but Section III-A states all data were generated with the Dominant Path Model, and it also mentions FEKO while the pipeline is WinProp. DPM is a fast approximate model, not full ray tracing; the discrepancy should be fixed. More conceptually, the 3D volume is built by running twenty independent 2D simulations and stacking slices. That gives a volumetric tensor, but vertical propagation paths and inter-height correlations are not modeled. If the paper claims true volumetric propagation, this limitation needs to be stated, not hidden.\n\nOne more small thing: the sparse-sampling gain partly comes from feeding 10% of true map values back into the model. That is mechanical inpainting, so it should not be presented as a discovery.\n\nWho is this for? Researchers working on radio map construction, channel knowledge maps, and 6G environment-aware communication will find the dataset useful. The paper deserves a serious referee because the artifact matters and the benchmark questions are worth getting right. I would send it out, with a strong request to add baselines, evaluate across all heights and modalities, resolve the DPM/ray-tracing wording, and discuss the slice-stacking limitation.","headline":"UrbanRadio3D is a genuinely useful dataset, but RadioDiff-3D's 'superior performance' claim is not backed by its own experiments.","tokens_in":24272,"tokens_out":3118,"would_cite":true,"duration_ms":35283,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that radio map construction should move from 2D pathloss planes to full 3D volumes carrying pathloss, arrival angles, and arrival times, and provides both a dataset and a diffusion model to make that possible.","keywords":["radio map","3D radio map dataset","pathloss","direction of arrival","time of arrival","diffusion model","environment-aware communication","ray tracing"],"falsifier":"Run a full 3D ray tracer that allows propagation paths to change height, including vertical rays and height-changing reflections, on a subset of UrbanRadio3D scenes, and compare voxel-level pathloss, DoA, and ToA with the stacked-slice dataset; disagreement beyond simulator tolerance near building rooflines would show that the slice-stacked construction does not represent true volumetric propagation.","tokens_in":23256,"feed_emoji":"📡","tokens_out":8971,"duration_ms":95576,"temperature":0.7,"pith_summary":"This paper argues that radio map construction should move from 2D pathloss planes to full 3D volumes carrying multiple channel descriptors, and provides the data and models to make that move possible. It introduces UrbanRadio3D, a ray-traced dataset covering 701 urban regions at 1-meter resolution across 20 height layers, with pathloss, direction of arrival (DoA) in azimuth and elevation, and time of arrival (ToA) for 200 transmitter locations each, totaling 2.84 million simulation instances. To benchmark the task, the paper proposes RadioDiff-3D, a conditional denoising diffusion model built on 3D convolutions, which can construct 3D radio maps when the transmitter is known and when only sparse observations are available. If the central claim holds, the work supplies a foundation for environment-aware 6G tasks such as altitude-aware beamforming, UAV navigation, and interference-avoidance planning.","feed_headline":"A diffusion model now builds 3D radio maps with angles and delays","feed_subtitle":"UrbanRadio3D supplies 2.84 million ray-traced instances across 20 heights to train it.","key_machinery":"The central object is the 4D tensor $R \\in \\mathbb{R}^{H \\times W \\times D \\times C}$ that represents a radio map, with $D$ the height dimension and $C$ the channel modalities: pathloss, DoA azimuth, DoA elevation, and ToA. In the dataset, each 256×256 m scene is ray-traced at 20 separate 1 m receiver heights using the Dominant Path Model, and the resulting horizontal slices are stacked into a volume with 1 m cubic voxels. The generative model is a conditional denoising diffusion probabilistic model whose denoising U-Net uses 3D convolutions; conditioning injects the environment occupancy grid, building height map, and transmitter location or an interpolated sparse sampling map into residual blocks via cross-attention or FiLM-like modulation. Reconstruction-guided sampling refines the generated volume to match observed samples, and autoregressive height-wise generation reduces memory cost by generating one slice conditioned on the previous one.","core_discovery":"The central claim is that a 3D×3D radio map—pathloss, direction of arrival (azimuth and elevation), and time of arrival spread through a full three-dimensional volume—can be captured in a large ray-traced dataset and reconstructed by a generative model that treats the map as a volumetric tensor. The paper says UrbanRadio3D is over 37 times larger than previous radio map datasets and has 7 times more height layers than the prior state of the art, covering 701 urban regions, 200 transmitter locations per region, and receiver heights from 1 m to 20 m in 1 m steps. It also says RadioDiff-3D achieves superior performance in constructing these rich high-dimensional maps under diverse environmental dynamics, in both radiation-aware settings with a known transmitter and radiation-unaware settings driven by sparse spatial samples. Alongside the diffusion model, a 3D-UNet baseline is provided, and the paper reports standard image-quality metrics for pathloss, ToA, DoA azimuth, and DoA elevation.","pith_inferences":["A natural extension is to train and evaluate the model on all 20 height layers at once; the visible experiments cover only the lower 1–4 m band, so the paper's full-volume claim is an extrapolation from that subset.","Because each height slice is simulated independently, a full-3D ray-tracing comparison would reveal how much vertical multipath the dataset misses and whether DoA elevation predictions need a true volumetric simulator.","The same conditional generation setup could be inverted to localize an unknown transmitter from sparse 3D samples, a task the paper motivates but does not evaluate.","The uniform default facade materials mean models trained on UrbanRadio3D may need additional simulation or fine-tuning before transferring to cities with different construction materials or frequency bands, an open question the paper's future-work section implicitly acknowledges."],"forward_implications":["Existing 2D radio map models, which predict pathloss on a fixed plane, can be replaced by models that learn joint spatial dependencies across all three dimensions because the dataset supplies the missing vertical supervision.","The same generative framework can produce radio maps for cooperative transmitters and estimate interference fields from non-cooperative transmitters, reducing reliance on dense pilot measurements.","Adding sparse sampling information at a 10% rate improves reconstruction quality over environment-only conditioning, so partial measurements can be converted into complete volumetric maps.","With 20 height layers at 1 m resolution, the dataset opens the way to altitude-sensitive tasks such as UAV trajectory planning, 3D positioning, and elevation-aware beamforming that 2D radio maps cannot support."],"supporting_citations":[{"why":"Supplies SpectrumNet, the prior multi-height radio map dataset that UrbanRadio3D compares against in scale and number of height layers.","marker":"[16]"},{"why":"Provides RadioMapSeer and the RadioUNet baseline that define the 2D fixed-height pathloss paradigm this work extends.","marker":"[17]"},{"why":"Provides RadioGAT, a prior multi-band radio map dataset and reconstruction method used as a comparison point.","marker":"[18]"},{"why":"Provides RME-GAN, a GAN-based radio map estimation baseline that requires sampling information.","marker":"[22]"},{"why":"Introduces RadioDiff, the diffusion-based radio map construction approach that RadioDiff-3D extends from 2D to 3D.","marker":"[24]"},{"why":"Defines the denoising diffusion probabilistic model that forms the generative core of RadioDiff-3D.","marker":"[68]"},{"why":"Defines DDIM deterministic sampling, which gives the fast inference procedure used in the experiments.","marker":"[69]"},{"why":"Defines the Dominant Path Model, the propagation model used to simulate UrbanRadio3D.","marker":"[70]"},{"why":"Defines the 3D U-Net architecture used both as the benchmark baseline and as the denoising backbone.","marker":"[71]"}],"fun_headline_variants":["Diffusion model now maps 3D radio signals with angles and delays","UrbanRadio3D: 37x larger dataset for 3D radio map diffusion","RadioDiff-3D generates 3D radio maps with DoA and ToA","Full 3D radio maps via diffusion, from 1m to 20m heights"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that stacking 20 independently simulated 2D horizontal slices, each computed by a propagation model that treats the scene slice by slice, yields a physically consistent 3D radio map; if true 3D propagation paths that change height are needed for accurate DoA and ToA, the dataset's volume is not a genuine volumetric electromagnetic simulation.","fun_headline_variants_meta":{"raw":{"variants":["Diffusion model now maps 3D radio signals with angles and delays","UrbanRadio3D: 37x larger dataset for 3D radio map diffusion","RadioDiff-3D generates 3D radio maps with DoA and ToA","Full 3D radio maps via diffusion, from 1m to 20m heights"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000643,"raw_usage":{"total_tokens":3029,"prompt_tokens":1088,"completion_tokens":1941,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":704,"completion_tokens_details":{"reasoning_tokens":1852}},"tokens_in":704,"tokens_out":1941,"duration_ms":15303,"temperature":1.0,"reasoning_tokens":1852,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T16:52:10.933411+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a full 3D ray tracer that allows propagation paths to change height, including vertical rays and height-changing reflections, on a subset of UrbanRadio3D scenes, and compare voxel-level pathloss, DoA, and ToA with the stacked-slice dataset; disagreement beyond simulator tolerance near building rooflines would show that the slice-stacked construction does not represent true volumetric propagation.","supporting_citations":[{"cited_title":"Generative ai on spectrumnet: An open benchmark of multiband 3d radio maps,","cited_arxiv_id":null,"evidence_quote":"Supplies SpectrumNet, the prior multi-height radio map dataset that UrbanRadio3D compares against in scale and number of height layers."},{"cited_title":"Radiogat: A joint model-based and data-driven framework for multi-band radiomap reconstruction via graph attention networks,","cited_arxiv_id":null,"evidence_quote":"Provides RadioGAT, a prior multi-band radio map dataset and reconstruction method used as a comparison point."},{"cited_title":"RME-GAN: A learning framework for radio map estimation based on conditional generative adversarial network,","cited_arxiv_id":null,"evidence_quote":"Provides RME-GAN, a GAN-based radio map estimation baseline that requires sampling information."},{"cited_title":"Radiodiff: An effective generative diffusion model for sampling-free dynamic radio map construction,","cited_arxiv_id":null,"evidence_quote":"Introduces RadioDiff, the diffusion-based radio map construction approach that RadioDiff-3D extends from 2D to 3D."},{"cited_title":"Denoising diffusion probabilistic models,","cited_arxiv_id":null,"evidence_quote":"Defines the denoising diffusion probabilistic model that forms the generative core of RadioDiff-3D."},{"cited_title":"Denoising diffusion implicit models,","cited_arxiv_id":null,"evidence_quote":"Defines DDIM deterministic sampling, which gives the fast inference procedure used in the experiments."},{"cited_title":"Dominant path prediction model for urban scenarios,","cited_arxiv_id":null,"evidence_quote":"Defines the Dominant Path Model, the propagation model used to simulate UrbanRadio3D."},{"cited_title":"3d u-net: Learning dense volumetric segmentation from sparse annotation,","cited_arxiv_id":null,"evidence_quote":"Defines the 3D U-Net architecture used both as the benchmark baseline and as the denoising backbone."}],"review_version":1}