{"id":"de0e5a67-07e7-481b-acd1-b06a58d7003a","arxiv_id":"2508.09415","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":10,"one_line_summary":"A two-stage pipeline converts open government curb ramp coordinate lists into about 850k labeled street view images and trains a detector reaching 0.924 AP, well above prior work.","lead":"This paper builds a large dataset of curb ramps seen in Google Street View by automatically turning city government lists of ramp coordinates into pixel locations inside the panoramas, then trains a model that finds ramps in new street views. The resulting detector reports far higher accuracy than prior attempts, which matters for cities that audit sidewalks for accessibility.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Headline precision/recall/AP numbers rest on a single arbitrary 88-pixel matching radius (Sec. 3.3), with no sensitivity analysis; if the radius is lenient, the 'high-quality dataset' and '0.9236 AP' claims could be substantially inflated.","rationale":"I considered two candidate concerns: (1) the government-data dependency and (2) the matching-radius sensitivity. The reader's weakest_assumption focuses on (1) and on the lack of cross-city evaluation. Those are valid limitations but they primarily affect the generalization claim ('benefits all cities') and the scalability of the bootstrapping approach; the constructed dataset from three vetted cities can still be high-quality even if only 10% of cities publish metadata. In contrast, the arbitrary 88-pixel radius is used to define every headline number in the abstract and the SOTA comparison. If the radius is too generous, the dataset could contain many labels that are not actually at the curb-ramp centers, undermining the 'high-quality' contribution and the AP comparison. The paper provides no analysis of how the metrics vary with this tolerance. This is a more direct, internally focused threat to the central quantitative claims. My proposed test would settle it by recomputing all metrics at multiple radii. If the results hold at stricter radii, the central claim is strengthened; if not, the paper must qualify its numbers. I therefore recommend keeping the CONDITIONAL verdict, with the condition being a robustness analysis of the matching threshold.","tokens_in":12426,"tokens_out":8450,"duration_ms":84123,"concrete_test":"Recompute Stage 1 precision/recall and Stage 2 AP on the 1,000-panorama manual ground truth (3,919 ramps) using matching radii of 22, 44, 88, and 132 px in the 4096x2048 equirectangular image, and report PR curves for each radius. Also re-run Weld et al.'s model under the same radii. If the AP at 44 px drops by more than, say, 10 percentage points relative to the 88-px value, the headline numbers are not robust and the 'high-quality' / 'far exceeding' claims need qualification. Additionally, compute the AP with a radius that scales with panoramic latitude/pitch to assess whether the fixed-pixel tolerance biases results.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central quantitative claims—Stage 1 94.0% precision/92.5% recall and Stage 2 0.9236 AP—are computed by matching predicted curb-ramp points to manual labels within an 88-pixel radius in 4096x2048 panoramas (Sec. 3.3). This threshold is introduced without justification or sensitivity analysis. In equirectangular imagery, 88 px corresponds to roughly 7.7° of longitude at the equator and varies with pitch; for a curb ramp imaged at moderate distance, this tolerance could span a substantial portion of the ramp and its surrounding sidewalk. A more stringent radius (e.g., 44 px) could change the TP/FP assignment and materially reduce the reported precision/recall and AP. Since the same radius is used for both stages and for the comparison against Weld et al., the 'far exceeding prior work' conclusion is contingent on this arbitrary tolerance. The paper also does not report error bars or multiple evaluation runs, so the 0.9236 AP may not be stable. This is load-bearing because the paper's central contribution is a 'high-quality' dataset and a new SOTA detector; if the metric is lenient, the qualitative claims weaken even though the pipeline itself may be sound.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces RampNet, a two-stage pipeline for curb ramp detection in Google Street View panoramas. Stage 1 translates government-provided curb ramp coordinates into pixel labels by extracting directional crops and using a fine-tuned ConvNeXt V2 heatmap model to localize ramps. This produces an auto-labeled dataset of 849,895 ramp points across 214,376 panoramas from NYC, Portland, and Bend. Stage 2 trains a separate ConvNeXt V2 detector on these auto-generated labels to detect curb ramp points directly from full panoramas. The authors evaluate both stages against a manually labeled ground truth of 1,000 held-out panoramas containing 3,919 ramps, reporting Stage 1 precision/recall of 94.0%/92.5% and Stage 2 AP of 0.9236, compared with 0.3803 for the prior Weld et al. system. The paper contributes a new dataset, benchmark, and model, with code and data open-sourced.","tokens_in":12645,"tokens_out":4112,"duration_ms":44141,"significance":"If the reported numbers are robust, this is a substantial contribution. The manual ground-truth evaluation is a genuine strength: the 1,000-panorama set is independent of the 312-panorama set used to fine-tune the Stage 1 crop model, and the Stage 2 model is evaluated on manually labeled images it never saw during training. The scale of the auto-generated dataset far exceeds prior curb ramp detection resources, and the two-stage bootstrapping idea is both practical and clearly described. The paper also ships open-source code and data, which supports reproducibility and follow-up work. However, the headline metrics rest on an arbitrarily chosen 88-pixel matching radius with no sensitivity analysis, so the magnitude of the claimed improvement over prior work is not yet established. The generalization claim for Stage 2 is also untested outside the three training cities.","major_comments":[{"comment":"All central quantitative claims—Stage 1 precision/recall, Stage 2 AP, and the comparison with Weld et al.—are computed by matching predicted points to manual labels within an 88-pixel radius in 4096×2048 equirectangular panoramas. This radius is introduced without justification or sensitivity analysis. In equirectangular imagery, 88 px corresponds to about 7.7° of longitude at the equator, which can span a substantial portion of a curb ramp and its surrounding sidewalk at typical street-view distances. A stricter radius (e.g., 44 px) could change true/false positive assignments and materially lower the reported numbers. Since the same radius is used for both stages and for the prior-work comparison, the 'far exceeding prior work' conclusion is contingent on this arbitrary tolerance. The authors should report precision, recall, and AP for a range of radii (e.g., 44, 66, 88, 110 px) and ju","section":"§3.3 (Correctness metrics)"},{"comment":"The Stage 2 model is trained for a single epoch with batch size one on 16 GPUs, and AP is reported as a single number (0.9236) with no error bars, multiple seeds, or stability analysis. Given the evaluation pipeline’s dependence on the 88-pixel matching radius and confidence threshold (0.55), the absence of uncertainty quantification makes it difficult to assess whether the reported AP is a reliable point estimate. At minimum, the authors should report variance across a few training runs or bootstrapped evaluation samples, and state how the reported AP is affected by the peak-extraction threshold.","section":"§4.3 (Results) and §4.2 (Training)"},{"comment":"The paper claims that the Stage 2 model 'benefits all cities with GSV availability' and emphasizes real-world applicability, but the only evaluation is on panoramas from the same three cities used to train the Stage 1 auto-labeler. The authors acknowledge this limitation in §5.1, but the claim in the introduction and abstract goes beyond the evidence. To support the transferability claim, the authors should evaluate the Stage 2 model on at least one held-out city (e.g., Austin, Los Angeles, or Nashville, whose government data was screened but not used) with a small manually labeled set. Without such a test, the model's performance is only demonstrated on its training distribution.","section":"§1 and §5.1"}],"minor_comments":[{"comment":"The 10-meter panorama selection radius, 35-meter label candidate radius, and 60-meter null-image exclusion distance are presented as fixed choices without sensitivity analysis or a rationale beyond practical necessity. A brief discussion of how these were selected would be helpful.","section":"§3.1"},{"comment":"The crop-model evaluation (76.7% recall/77.2% precision on Project Sidewalk alone; 89.0%/87.0% after fine-tuning) does not state which held-out set or matching criterion was used. This should be specified for reproducibility.","section":"§3.2"},{"comment":"The training setup would benefit from more detail: whether the validation split was used for early stopping, how the learning rate was chosen, and whether the single-epoch choice was based on convergence curves. As written, it is hard to judge whether the model is undertrained or overtrained.","section":"§4.2"},{"comment":"When comparing against Weld et al., the authors use their released checkpoints rather than retraining. This is reasonable, but it should be stated explicitly in the table caption or text that the comparison may be affected by differences in training data and input modalities (depth/image/geo vs. image-only).","section":"Table 4"},{"comment":"The text says 'we pick the one with highest confidence and ignore the others entirely' when multiple predictions match one ground-truth point, but it does not specify whether 'confidence' refers to the heatmap peak value or a separate score. Please clarify.","section":"§3.3"},{"comment":"The precision-recall curves would be more informative if the matching radius used to compute TP/FP was stated directly in the caption, since the curves depend on that radius.","section":"Figure 6"}],"recommendation":"major_revision","confidential_remarks":"The main risk is not circularity—the manual evaluation is properly held out and independent of the Stage 1 fine-tuning set. The load-bearing issue is the single arbitrary 88-pixel matching radius and the lack of sensitivity analysis around it. If the authors supply a radius sweep and show that the conclusions are stable (or quantify how they degrade with stricter radii), the paper would be much stronger. The generalization claim also needs a held-out city evaluation before the 'any city' statement can be accepted. These are fixable with additional experiments, so I am not recommending rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The reader's take lines up with mine. The real contribution is Stage 1: turning government curb-ramp coordinate lists into pixel labels via a learned crop model. That is new for this task, and it gets you a 214k-panorama dataset that nobody else has. The 1,000-panorama manual ground truth (3,919 ramps) is a genuine strength—it breaks the circular evaluation that would otherwise haunt an auto-labeled dataset. The paper does this openly, and the error analysis (17% government data disagreements, 43% visually ambiguous) is honest about where the method struggles.\n\nThe soft spots are real but not fatal. The 88-pixel matching radius is the biggest one. It is introduced without justification, no sensitivity analysis, and it gates every headline number. In a 4096x2048 equirectangular pano, that is a fairly generous tolerance. The stress-test note is right to flag it. The fix is easy: report precision/recall/AP at several radii (44, 88, 132) and show the conclusions hold. I suspect they will—the prior work numbers are so much lower that the qualitative ordering probably survives—but the paper should prove that.\n\nSecond, all evaluation is in the same three cities used for training. The claim that Stage 2 works on any city with GSV is plausible but untested. A cross-city test on a city not in the training set (even one with government data, e.g., Austin) would strengthen it a lot. The single-run results without error bars are also a minor concern; the AP of 0.9236 might move a bit.\n\nThe comparison to Weld et al. is the same lab's prior model, which is fine given they released checkpoints, but it is worth stating clearly that this is a same-lab comparison. The bootstrapping dependence on rare government data is acknowledged in the limitations, and it does constrain who can use Stage 1. That is a scope issue, not a flaw.\n\nOverall: the pipeline is sound, the evaluation is honest, and the dataset plus benchmark are real contributions. The paper deserves serious peer review. A good revision would add sensitivity analysis, error bars, and a cross-city transfer experiment.","headline":"A genuinely useful auto-labeling pipeline with an honest independent evaluation, but the headline numbers rest on a single hand-picked matching radius that needs a sensitivity check.","tokens_in":13306,"tokens_out":2063,"would_cite":false,"duration_ms":24044,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"RampNet claims open government curb ramp coordinates can be auto-translated into pixel labels, yielding a 214,376-panorama dataset and a detector that reaches 0.924 AP—more than double the previous best.","keywords":["curb ramp detection","street-view imagery","government metadata","heatmap regression","ConvNeXt V2","dataset generation","urban accessibility","point-based object detection"],"falsifier":"Run Stage 1 in a city with published curb ramp coordinates not used in this work and manually verify a random sample of generated labels: if a substantial share of labels fall on non-ramps or on ramps that no longer exist, the bootstrapping premise fails. Also evaluate the Stage 2 model on a held-out city with independent manual labels; a drop below the crowdsourced baseline (0.380 AP) would contradict the generalization claim.","tokens_in":12169,"feed_emoji":"♿","tokens_out":6546,"duration_ms":60158,"temperature":0.7,"pith_summary":"This paper claims that open government curb ramp location lists can anchor pixel-level labels in street-view panoramas at scale, producing a training set of 214,376 panoramas with 849,895 ramp labels, and that a model trained on these auto-labels detects curb ramps at 0.924 average precision—more than double the previous best system. The two-stage pipeline first converts each governmental latitude/longitude into a directional perspective crop, locates the ramp point with a heatmap-regressing ConvNeXt V2 model, and reprojects it to the panorama. A second ConvNeXt V2 model regresses heatmaps over full panoramas to predict ramp points directly. Evaluated against 1,000 manually labeled panoramas, Stage 1 reaches 94.0% precision and 92.5% recall, and Stage 2 reaches 0.924 AP. If correct, the approach removes the need for exhaustive manual labeling in cities that publish curb ramp coordinates.","feed_headline":"Auto-labeled panoramas power curb ramp detection at 0.924 AP","feed_subtitle":"Open city lat/long lists become 214,000 labeled street panoramas, beating prior crowdsourced detectors.","key_machinery":"The load-bearing mechanism is the auto-translation of government <lat,long> curb ramp metadata into pixel labels, powered by two heatmap-regressing ConvNeXt V2 models. The first (crop-level) model localizes ramp points within 1024×1024 directional perspective crops; the second (panorama-level) model takes the full 4096×2048 equirectangular panorama and outputs a 1024×512 heatmap with Gaussian centers (σ=10) at ramp points. The 35-meter label-candidate radius and installation-date-before-capture filtering align metadata with imagery; a 60-meter spatial split prevents leakage.","core_discovery":"The central discovery is that coarse government metadata can be auto-translated into pixel-accurate labels, and a detection model trained solely on those auto-labels approaches manual-labeling quality in the cities studied. Stage 1 computes the bearing from a panorama to each government-listed curb ramp, extracts a 341×1024 perspective crop, and uses a ConvNeXt V2 model to produce a heatmap whose peak marks the ramp point; these points are projected back to equirectangular coordinates. Stage 2 trains a separate ConvNeXt V2 heatmap regressor on full panoramas. On 1,000 manually labeled test panoramas comprising 3,919 ramps, Stage 1 achieves 92.5% recall and 94.0% precision, and the Stage 2 de","pith_inferences":["The auto-translation step could be inverted: detected pixel points back-projected to geocoordinates would let cities update their own curb ramp inventories from imagery alone, an extension the paper mentions as future work.","The same bootstrapping recipe may transfer to other street features with open GIS coordinates—pedestrian signals, crosswalks, fire hydrants—subject to the same metadata-precision constraint.","Because the detector is trained and tested only in three U.S. cities, its performance on other countries' ramp designs or on non-street-view imagery is an open empirical question that the current benchmark does not settle.","The single-point label format hides ramp width and slope; extending the heatmap head to output bounding boxes or orientation would make automatic quality assessment (tactile warnings, steepness) feasible."],"forward_implications":["Cities that publish curb ramp coordinates can generate pixel-labeled training data automatically, eliminating the manual-labeling bottleneck.","The Stage 2 detector needs only street-view imagery, so cities without open curb ramp data can still receive whole-city audits after a one-time model deployment.","At 0.924 AP on manual labels, the detector approaches human-level labeling, making automated curb ramp inventories viable for ADA compliance monitoring.","The 849,895-label dataset and 214,376-panorama benchmark give the community a standardized point-detection task for curb ramps, analogous to how large public benchmarks advanced face detection.","The 20% null-image infusion and viewpoint duplication (4.5 panoramas per ramp) make the trained detector more robust to empty scenes and varied angles."],"supporting_citations":[{"why":"shows only 10% of US cities publish curb ramp data, motivating the bootstrapping approach.","marker":"[10]"},{"why":"provides the previous state-of-the-art detector and crowdsourced dataset that RampNet compares against and surpasses.","marker":"[45]"},{"why":"supplies the crowdsourced sidewalk labels used to pretrain the crop model and serves as the reference dataset.","marker":"[35]"},{"why":"defines the ConvNeXt V2 architecture used in both the crop-localizing and panorama-detecting models.","marker":"[46]"},{"why":"prior system whose precision/recall and resolution choices this work extends.","marker":"[17]"},{"why":"labeling tool used to produce both the fine-tuning set and the 1,000-panorama ground truth.","marker":"[41]"},{"why":"ImageNet-1k pretrained weights initialize the Stage 2 detector for faster convergence.","marker":"[11]"},{"why":"provides the heatmap regression formulation adapted for point-based ramp detection.","marker":"[47]"}],"fun_headline_variants":["Curb ramp detection from street view, auto-labeled via city data","RampNet: Turning city data into 210k labeled ramps","Auto-labels from government data yield SOTA curb ramp detection","0.924 AP: Curb ramps found via auto-labeled panoramas","Open data to pixel labels: RampNet scales curb ramp detection"],"cache_read_input_tokens":2816,"weakest_assumption_plain":"The pipeline's quality depends on government curb ramp coordinates being accurate and current enough that a camera aimed at them actually shows the ramp; one of eight surveyed cities was rejected for poor precision, and 17% of Stage 1 errors trace to government-data disagreements.","fun_headline_variants_meta":{"raw":{"variants":["Curb ramp detection from street view, auto-labeled via city data","RampNet: Turning city data into 210k labeled ramps","Auto-labels from government data yield SOTA curb ramp detection","0.924 AP: Curb ramps found via auto-labeled panoramas","Open data to pixel labels: RampNet scales curb ramp detection"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000867,"raw_usage":{"total_tokens":3608,"prompt_tokens":771,"completion_tokens":2837,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":515,"completion_tokens_details":{"reasoning_tokens":2754}},"tokens_in":515,"tokens_out":2837,"duration_ms":20382,"temperature":1.0,"reasoning_tokens":2754,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T21:04:54.712775+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run Stage 1 in a city with published curb ramp coordinates not used in this work and manually verify a random sample of generated labels: if a substantial share of labels fall on non-ramps or on ramps that no longer exist, the bootstrapping premise fails. Also evaluate the Stage 2 model on a held-out city with independent manual labels; a drop below the crowdsourced baseline (0.380 AP) would contradict the generalization claim.","supporting_citations":[{"cited_title":"Squeaky wheels: Missing data, disability, and power in the smart city","cited_arxiv_id":null,"evidence_quote":"shows only 10% of US cities publish curb ramp data, motivating the bootstrapping approach."},{"cited_title":"Froehlich","cited_arxiv_id":null,"evidence_quote":"provides the previous state-of-the-art detector and crowdsourced dataset that RampNet compares against and surpasses."},{"cited_title":"Project Sidewalk: A Web-based Crowdsourcing Tool for Collecting Sidewalk Accessibility Data At Scale","cited_arxiv_id":null,"evidence_quote":"supplies the crowdsourced sidewalk labels used to pretrain the crop model and serves as the reference dataset."},{"cited_title":"ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoders, 2023","cited_arxiv_id":null,"evidence_quote":"defines the ConvNeXt V2 architecture used in both the crop-localizing and panorama-detecting models."},{"cited_title":"Tohme: Detecting curb ramps in google street view using crowdsourcing, computer vision, and machine learning","cited_arxiv_id":null,"evidence_quote":"prior system whose precision/recall and resolution choices this work extends."},{"cited_title":"Label Studio: Data labeling soft- ware, 2020-2025","cited_arxiv_id":null,"evidence_quote":"labeling tool used to produce both the fine-tuning set and the 1,000-panorama ground truth."},{"cited_title":"ImageNet: A large-scale hierarchical im- age database","cited_arxiv_id":null,"evidence_quote":"ImageNet-1k pretrained weights initialize the Stage 2 detector for faster convergence."},{"cited_title":"Simple Baselines for Human Pose Estimation and Tracking","cited_arxiv_id":null,"evidence_quote":"provides the heatmap regression formulation adapted for point-based ramp detection."}],"review_version":1}