{"id":"19a96b8f-d1f5-454a-9fb9-4bbb2024d619","arxiv_id":"2502.10259","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A new mmWave imaging dataset and simulator enable segmentation and classification of everyday objects hidden behind occluders.","lead":"MITO introduces a dataset of millimeter-wave radar images of everyday objects captured with a robotic arm in line-of-sight and fully occluded settings, plus an open-source simulator and two non-line-of-sight perception benchmarks. The work targets robotic and logistics applications where cameras cannot see through boxes or cloth.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"SAR focusing depends on unvalidated antenna-position accuracy; at 77 GHz, sub-millimeter trajectory errors from Sec. 4.3's NTP/interpolation pipeline would defocus all 550 images, so the dataset and all downstream numbers rest on an unmeasured quantity.","rationale":"The reader's weakest_assumption identifies the same load-bearing concern I find: the coherent SAR pipeline in Eq. 1 rests on unmeasured antenna-position accuracy. I agree with the reader, and I sharpen the concern by quantifying the phase-error budget at 77 GHz and by specifying a concrete motion-capture test. The simulator validation is also weakened by Eq. 11, which selects the best per-object weights before computing the 94% median F-score; that is a real overfitting risk. However, the sim2real classification in Table 3 provides independent evidence that the simulator captures enough signal physics to train a real-world classifier, whereas no independent check protects the real-data SAR pipeline. If the antenna positions are wrong, the real images themselves are corrupted, and both the simulator comparison and the downstream benchmarks inherit that corruption. Therefore the trajectory/synchronization assumption is the single most load-bearing point. This is not an internal inconsistency in the paper's math, but it is a missing empirical validation of a physical precondition; the CONDITIONAL verdict remains appropriate. The paper is promising, the claims are plausible, and the release of trajectory-error statistics, raw poses, and a motion-capture validation would resolve the concern.","tokens_in":14160,"tokens_out":4611,"duration_ms":51476,"concrete_test":"Perform a motion-capture validation on a subset (e.g., 10–20 scans) of the MITO acquisition pipeline: place retroreflective markers on the radar fixture, record ground-truth 6-DoF poses at 100+ Hz synchronized to radar frame triggers, and compare against the UR5e/NTP-interpolated positions used in the released dataset. Report the per-sample position error distribution (median and 95th percentile) and re-run Eq. 1 with corrected positions. If median error exceeds ~0.25 mm at 77 GHz, or if corrected images differ materially in sharpness or F-score against the RGB-D/optical ground truth, the SAR images in MITO are not reliably focused and the dataset claim needs revision; if the error is below that threshold and the images are essentially unchanged, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"MITO's central value is the 550 coherent SAR images, formed by Eq. 1 as I = Σ S_{j,k} exp(j2π d(x,y,z,k)/λ_j), with d = 2||(x,y,z) − p_k||. The phase in that sum is directly sensitive to the antenna position p_k. At 77 GHz, λ ≈ 3.9 mm, so a position error δr changes the round-trip phase by 4πδr/λ ≈ 3.2 rad/mm; keeping the phase error below π/4 requires δr ≲ 0.25 mm. The manuscript (Sec. 4.3) reconstructs p_k by interpolating timestamped UR5e poses to radar timestamps after NTP synchronization between two computers. No trajectory accuracy, NTP offset/jitter, or interpolation error is reported. The UR5e's nominal repeatability alone does not bound absolute accuracy under motion, and NTP on general-purpose computers is not a hard-real-time synchronization mechanism. If the true per-sample position error exceeds a few tenths of a millimeter for a significant fraction of the 24 million frames, the coherent sum in Eq. 1 partially or fully decorrelates, blurring the 550 SAR images and undermining every downstream result: Table 2 segmentation, Table 3 classification, and Table 1 simulator F-scores all compare against these real images. This is not an internal inconsistency, but it is an unvalidated physical precondition for the central claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents MITO, a dataset of millimeter-wave (mmWave) synthetic-aperture-radar (SAR) images of everyday objects, collected with a robotic arm carrying two radars (24 GHz and 77 GHz) and an RGB-D camera. The dataset contains over 24 million raw frames processed into 550 high-resolution, complex-valued 3D images of 76 YCB objects in both line-of-sight (LOS) and non-line-of-sight (NLOS) settings, together with RGB-D images, segmentation masks, and raw signals. The paper also introduces an open-source simulator that renders synthetic mmWave images from arbitrary 3D meshes using specular and edge-reflection models, and reports a median 3D F-score of 94% when comparing simulated and real point clouds. Two downstream applications are demonstrated: NLOS object segmentation using SAM with power-based prompting, and shape classification trained exclusively on synthetic images, achieving 85.2% overall accuracy on real LOS and NLOS images. The paper argues that MITO is the first diverse everyday-object mmWave dataset and that the simulator produces data matching real-world measurements, enabling broader NLOS perception.","tokens_in":14461,"tokens_out":3684,"duration_ms":37759,"significance":"If the dataset and simulator are validated, this work would be a significant contribution: it is the first open mmWave dataset of diverse everyday objects at multiple frequencies with raw data and code, and the sim2real classification result is a strong, independent demonstration that the simulator captures enough physical fidelity to be useful for training perception models. The segmentation benchmark provides a concrete baseline for NLOS perception, and the open-source simulator could enable large-scale synthetic data generation for future research. However, the headline simulator accuracy is weakened by the fact that it is computed after per-object fitting of the combination weights (Eq. 11), and the entire dataset rests on an unvalidated assumption about antenna-position accuracy for coherent SAR focusing. These issues must be resolved before the claims of 'matches real-world collected data' can be accepted as stated.","major_comments":[{"comment":"The reported simulator F-scores, including the median of 94% in Table 1, are computed after selecting the best (α1, α2) weights for each object individually via Eq. (11). This makes the result a measure of the simulator's representational flexibility, not its predictive accuracy: the weights are fit to the same real images used for evaluation. The claim in Sec. 2 that the simulator 'generates synthetic data that matches real-world collected data' is therefore overstated. Please report F-scores for fixed weights (e.g., equal weights), or use a cross-validation scheme where weights are selected on a training subset and evaluated on held-out objects, and include the distribution of the selected weights.","section":"Sec. 5.2.2, Eq. (11)"},{"comment":"Coherent SAR focusing via Eq. (1) requires accurate antenna positions p_k in the phase term exp(j2π d/λ). At 77 GHz (λ ≈ 3.9 mm), a position error of just a few tenths of a millimeter produces a phase error on the order of a radian, defocusing the image. The paper states that robot locations are interpolated to radar timestamps after NTP synchronization between two computers, but it reports no trajectory accuracy, synchronization offset/jitter, or interpolation error. Without a quantitative validation of the antenna-position error (e.g., a calibration measurement with a point reflector, or a comparison of focused images against a ground-truth target), the physical validity of all 550 real SAR images—and hence every downstream result in Tables 2 and 3—is unestablished.","section":"Sec. 4.3, Eq. (1)"},{"comment":"The simulator and its evaluation depend on several free thresholds: the specular angle threshold τ (Eq. 6), the edge angle threshold τ_e (Sec. 5.1.2), the point-cloud power threshold τ_P (Sec. 5.2.1), and the F-score distance threshold τ_F (Eqs. 9-10). No values are reported for these thresholds, and no sensitivity analysis is provided. Since the reported F-scores and the point-cloud comparisons can be expected to vary substantially with these choices, the simulator accuracy claim is under-specified. Please state the threshold values and show that the conclusions (e.g., the superior performance of the combined model) are robust to reasonable variations.","section":"Sec. 5.2.1, Eqs. (9)-(10), Sec. 5.1.1-5.1.2"}],"minor_comments":[{"comment":"The phrase 'non-light-of-sight' should be 'non-line-of-sight' for consistency with the rest of the text.","section":"Abstract"},{"comment":"The subsection title 'Combing Reflections' is a typo; it should be 'Combining Reflections'.","section":"Sec. 5.1.3"},{"comment":"The asterisk next to the camera NLOS recall value (100%*) is never explained in the caption or body text; please add a footnote explaining that the mask covers the occluder rather than the object.","section":"Table 2"},{"comment":"The threshold τ in Eq. (6) is described as being on the angle between the normal and the radar-to-vertex vector, but the condition is written without explicit units or a default value; please clarify the range and units of τ.","section":"Sec. 5.1.1"},{"comment":"The paper mentions that the tripod/styrofoam background can be removed by subtracting an empty scene, but it does not state how many of the 550 images actually use this background subtraction. This is useful information for users of the dataset.","section":"Sec. 4.2.1"}],"recommendation":"major_revision","confidential_remarks":"This paper is a potentially valuable dataset contribution, but the central validation claims need substantial strengthening. The simulator F-score is partly a fitted value, and the dataset's coherence depends on an unvalidated trajectory-accuracy assumption. The sim2real classification result is the strongest evidence for the simulator's usefulness and should be highlighted once the trajectory issue is addressed. The 'first' claim in the abstract is appropriately scoped to diverse everyday objects, but the authors should make sure the comparison to prior work in Sec. 2 is exhaustive. I would be willing to review a revised version."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Three things to know. First, the paper delivers something genuinely missing: a public mmWave dataset of 76 everyday YCB objects with 24M raw frames, 550 SAR images in LOS and NLOS, plus RGB-D images and masks, and an open-source simulator for arbitrary triangle meshes. That is a real resource contribution, not a restatement of prior work, which is mostly humans, vehicles, or weapons. Second, the sim2real classification result — training a small CNN entirely on synthetic images and getting 85.2% overall accuracy on real LOS/NLOS images — is the strongest evidence that the simulator captures enough physics to be useful. It is independent of the fitted simulator F-score and it is the honest headline of the paper. Third, the paper’s own numbers overstate simulator accuracy. The 94% median F-score in Table 1 comes from per-object selection of the alpha1, alpha2 weights (Eq. 11), so it is a fit, not a prediction. The paper is transparent about this, but it means the simulator’s accuracy claim should be read as “best case over weights,” not as an independent match.\n\nThe soft spots beyond that: several thresholds (tau_P, tau_F, the specular and edge angle thresholds tau and tau_e) and the SAM prompt-point selection are never given values or sensitivity analysis. There are no error bars on the segmentation or classification percentages, which are medians over 76 objects. The SAR focusing concern is real and worth taking seriously: at 77 GHz, Eq. 1’s phase depends on antenna positions to fractions of a millimeter, and Sec. 4.3 reports no trajectory, interpolation, or NTP synchronization error. That is a measurable experimental parameter and the authors should report it; without it you cannot rule out partial defocus. But the sim2real transfer succeeding suggests the images are not badly defocused, so I would treat this as a gap to fix, not a fatal flaw.\n\nThe citation pattern looks fine — the prior mmWave datasets and simulators are cited, and the claims about prior work match what I know. There are minor typos and malformed links in the reference list; those are cosmetic.\n\nWho this is for: robotics perception, mmWave sensing, and anyone training sim2real models on radar. It deserves a serious referee: the resource is substantial, the code and data are promised, and the sim2real result is evidence that the simulator has some real predictive value. My recommendation is to send it to peer review, with the request that the authors disclose threshold values, report error bars, and provide trajectory/synchronization accuracy for the SAR pipeline.","headline":"A genuinely useful public mmWave dataset and simulator for everyday-object NLOS perception, with an honest sim2real result; the headline simulator F-score is partly a fit and trajectory accuracy is unreported.","tokens_in":15025,"tokens_out":2236,"would_cite":true,"duration_ms":24478,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper establishes MITO, a millimeter-wave dataset and simulator that enable non-line-of-sight perception of everyday objects.","keywords":["millimeter-wave radar","non-line-of-sight perception","synthetic aperture radar","mmWave dataset","sim-to-real transfer","object segmentation","object classification","robotic manipulation"],"falsifier":"Compare the UR5e's recorded poses against an independent motion-capture system while the arm moves the radar; if the position error exceeds roughly a millimeter, the 77 GHz synthetic-aperture images would visibly defocus, and both the simulator's 94% median F-score and the classifier's 85.2% accuracy would drop. Re-running the classification benchmark with a different trajectory or aperture size would also reveal how sensitive the claim is to trajectory accuracy.","tokens_in":13926,"feed_emoji":"📡","tokens_out":5965,"duration_ms":52863,"temperature":0.7,"pith_summary":"The paper presents MITO, a dataset of millimeter-wave (mmWave) radar images of 76 everyday objects, captured both in direct view and hidden inside cardboard or fabric, together with an open-source simulator that turns any 3D triangle mesh into matching synthetic mmWave images. The central claim is that these two pieces together give computer vision a workable path to non-line-of-sight perception: seeing and recognizing objects that visible-light cameras cannot. The authors demonstrate that the simulator matches real captures with a median 3D F-score of 94% (Table 1), and that a classifier trained only on synthetic images reaches 85.2% overall accuracy on real images, including 81.2% on fully occluded objects (Table 3). If correct, this means robots could verify packaged orders, retrieve occluded items, and classify hidden objects using signals that penetrate common packaging materials.","feed_headline":"Radar dataset lets robots see objects hidden in boxes","feed_subtitle":"MITO pairs 24 million real mmWave frames with an open-source simulator that trains classifiers to 85% accuracy on occluded objects.","key_machinery":"The load-bearing identity is the coherent SAR sum $I(x,y,z)=\\sum_{k=1}^{K}\\sum_{j=1}^{N}S_{j,k}\\,e^{j2\\pi d(x,y,z,k)/\\lambda_j}$ (Eq. 1), which adds up low-resolution radar frames from many antenna positions into one high-resolution complex image, with the phase term $2\\pi d/\\lambda$ depending on the round-trip distance from each antenna to each voxel. The simulator feeds this same pipeline with predicted reflections from a mesh: for each radar location it finds visible vertices, assigns each a phase $s_j(l,v)=e^{-j4\\pi |l-v|/\\lambda_j}$, and optionally suppresses reflections whose surface normal points away from the radar (specular model, Eq. 6) or keeps only edge vertices (edge model, Eq. 7); the two synthetic images are then blended as $I_{\\mathrm{syn}}(\\alpha_1,\\alpha_2)=\\frac{\\alpha_1}{\\alpha_1+\\alpha_2}I_s+\\frac{\\alpha_2}{\\alpha_1+\\alpha_2}I_e$ (Eq. 8) to cover different materials.","core_discovery":"The paper's central discovery is that coherently summing the complex-valued reflections from a radar moving along a known robot trajectory - synthetic aperture radar, Eq. (1) - yields high-resolution images of everyday objects even when they sit inside a closed cardboard box, and that a lightweight simulator which predicts only two mechanisms of reflection, specular surface returns (Eq. 6) and edge diffraction (Eq. 7), reproduces these images closely enough to train a real-world classifier. With both simulation types blended by random weights during training, the classifier achieves 88.8% accuracy in line-of-sight and 81.2% in non-line-of-sight, for an overall 85.2% on real images (Table 3), and the simulator's best weighted output reaches a median 3D F-score of 94% against real point clouds (Table 1).","pith_inferences":["The paper's simulator models geometry only; objects whose reflections are dominated by volume scattering or strong absorption (for example, very wet or rough materials) may fall outside the two-mechanism model, so the claimed sim-to-real match is best understood as demonstrated on the 76 tested objects.","If the simulator generalizes as claimed, synthetic-aperture mmWave imaging becomes a standard pipeline for through-occlusion perception: generate any object's mesh, render synthetic radar images, and train a classifier without physical data collection.","A natural testable extension is classification accuracy as a function of occluder thickness and material; the paper tests cardboard and four layers of fabric, but the simulator does not model attenuation through occluders, so very thick or highly reflective packaging should degrade 77 GHz performance first."],"forward_implications":["Robotic systems can segment and classify objects inside closed cardboard boxes using mmWave images alone, with non-line-of-sight accuracy close to their line-of-sight accuracy.","Any 3D triangle mesh - from object dataset or phone scan - becomes training data for mmWave perception, removing the need for expensive real-world radar capture for each new object.","Fusing 24 GHz and 77 GHz images improves segmentation over either band alone, indicating that frequency diversity is a first-class design lever for through-occlusion perception.","The two-mechanism reflection model (specular plus edge) with random weight blending acts as a material augmentation, letting a network trained on synthetic images generalize to real objects of unmeasured material properties.","The dataset supplies a benchmark for two new vision tasks - NLOS object segmentation and NLOS shape classification - that are impossible for RGB-D cameras alone."],"supporting_citations":[{"why":"Supplies the 76 everyday objects and their 3D meshes, the dataset's core content and the simulator's input geometry.","marker":"[14]"},{"why":"Provides the segment-anything foundation model used both to create ground-truth segmentation masks and as the backbone for the NLOS segmentation benchmark.","marker":"[27]"},{"why":"Defines the 3D F-score metric used to compare simulator output against real-world radar images.","marker":"[43]"},{"why":"Provides the point-cloud generation procedure that converts mmWave voxel images into the point clouds scored by the 3D F-score.","marker":"[18]"},{"why":"The 77 GHz radar (TI IWR1443Boost) that collects the higher-resolution LOS and NLOS images in the dataset.","marker":"[6]"},{"why":"The 24 GHz radar (Infineon Position2Go) that supplies the lower-frequency images used for multi-spectral fusion.","marker":"[4]"},{"why":"The UR5e robotic arm whose mobility creates the synthetic aperture and whose timestamped positions are needed for coherent focusing.","marker":"[44]"},{"why":"Supplies the mesh-visibility computation used by the simulator to decide which vertices reflect toward each radar location.","marker":"[3]"}],"fun_headline_variants":["MITO dataset+simulator lets robots see hidden boxed objects","Synthetic aperture radar sees objects inside cardboard boxes","Radar + simulation lets robots identify objects inside closed boxes","24M radar frames plus simulator teach robots NLOS perception"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Coherent focusing assumes the radar's position at each measurement is known to within a small fraction of the 3.9-millimeter wavelength at 77 GHz, and the paper relies on interpolating timestamped robot poses after network-time synchronization without reporting the resulting trajectory or synchronization error.","fun_headline_variants_meta":{"raw":{"variants":["MITO dataset+simulator lets robots see hidden boxed objects","Synthetic aperture radar sees objects inside cardboard boxes","Radar + simulation lets robots identify objects inside closed boxes","24M radar frames plus simulator teach robots NLOS perception"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001182,"raw_usage":{"total_tokens":4891,"prompt_tokens":966,"completion_tokens":3925,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":582,"completion_tokens_details":{"reasoning_tokens":3858}},"tokens_in":582,"tokens_out":3925,"duration_ms":29475,"temperature":1.0,"reasoning_tokens":3858,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T18:45:00.470208+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compare the UR5e's recorded poses against an independent motion-capture system while the arm moves the radar; if the position error exceeds roughly a millimeter, the 77 GHz synthetic-aperture images would visibly defocus, and both the simulator's 94% median F-score and the classifier's 85.2% accuracy would drop. Re-running the classification benchmark with a different trajectory or aperture size would also reveal how sensitive the claim is to trajectory accuracy.","supporting_citations":[{"cited_title":"Yale-cmu-berkeley dataset for robotic manipulation research","cited_arxiv_id":null,"evidence_quote":"Supplies the 76 everyday objects and their 3D meshes, the dataset's core content and the simulator's input geometry."},{"cited_title":"Richter*, Ren ´e Ranftl, Zhuwen Li, Vladlen Koltun, and Thomas Brox","cited_arxiv_id":null,"evidence_quote":"Defines the 3D F-score metric used to compare simulator output against real-world radar images."},{"cited_title":"Around the corner mmwave imaging in practical environments","cited_arxiv_id":null,"evidence_quote":"Provides the point-cloud generation procedure that converts mmWave voxel images into the point clouds scored by the 3D F-score."},{"cited_title":"https://www.ti.com/product/ IWR1443#tech-docs","cited_arxiv_id":null,"evidence_quote":"The 77 GHz radar (TI IWR1443Boost) that collects the higher-resolution LOS and NLOS images in the dataset."},{"cited_title":"https://www.infineon.com/ cms / en / product / evaluation - boards / demo - position2go/","cited_arxiv_id":null,"evidence_quote":"The 24 GHz radar (Infineon Position2Go) that supplies the lower-frequency images used for multi-spectral fusion."},{"cited_title":"https://www.universal- robots.com/products/ur5-robot/, 2021","cited_arxiv_id":null,"evidence_quote":"The UR5e robotic arm whose mobility creates the synthetic aperture and whose timestamped positions are needed for coherent focusing."},{"cited_title":"https://ps.is","cited_arxiv_id":null,"evidence_quote":"Supplies the mesh-visibility computation used by the simulator to decide which vertices reflect toward each radar location."}],"review_version":1}