{"id":"c10b8bb5-b741-4926-a3c4-51f8fbd30a19","arxiv_id":"2608.06934","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A user-conditioned deep learning model that combines sidewalk images with rater attributes predicts individual walkability ratings better than image-only models, and sidewalk imagery yields higher walkability scores than street-view imagery.","lead":"This paper collects nearly 30,000 walkability ratings from 1,196 people viewing sidewalk-level images, and trains a model that uses who the rater is, not just the image, to predict each rating. Ratings from a pedestrian viewpoint were higher than from a car viewpoint, and adding rater details improved prediction agreement by 65% over an image-only model.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The user-conditioned QWK gain may reflect memorization of near-unique rater attribute vectors rather than generalizable attribute-driven perception; a rater-disjoint split is required to test this.","rationale":"The reader's weakest assumption correctly identifies the most load-bearing concern: the rating-completion protocol allows the same raters to appear in both training and test, and if attribute combinations are near-unique, the model can memorize rater identity through the attribute vector. This directly threatens the central claim that respondent attributes, rather than identity, carry predictive signal beyond image content. The paper's own Section IV-C1 frames results as 'within the sampled image pool' and the conclusion admits that fully distinctive preferences are beyond reach, yet the abstract and conclusion present the improvement as evidence about who is evaluating. The permutation importance results are consistent with identity memorization, as permuting an attribute breaks the unique mapping. A rater-disjoint split is the decisive test: if the improvement collapses, the headline claim must be softened. Other concerns, such as the viewpoint-survey sample size, are secondary and do not affect the central claim. The paper does contribute a valuable dataset, a viewpoint comparison, and interaction analyses that partially support attribute-driven perception, so the finding is not worthless. The conditional verdict remains appropriate: the claim is plausible but not yet established. No change to the reader's verdict is needed.","tokens_in":15820,"tokens_out":5105,"duration_ms":56795,"concrete_test":"Retrain the user-conditioned model (Swin-Tiny + FT-Transformer + CORAL) under a rater-disjoint split: assign all ratings from a held-out subset of respondents (e.g., 20%) to the test set, ensuring no respondent appears in training, and compare QWK against the image-only baseline on the same held-out raters. If the user-conditioned advantage over the image-only baseline is no longer significant or is substantially below the reported 0.184, the gain is attributable to memorizing near-unique attribute vectors rather than to generalizable attribute-driven perception.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section IV-C1 defines a rating-completion protocol in which raters and images may appear in both training and test partitions, with only the individual image-respondent rating held out. The paper argues that because conditioning is on respondent attributes rather than identity, direct memorization is impossible. This argument fails if attribute combinations are near-unique across the 1,196 respondents. With seven attributes (age, gender, residence, childhood area, upbringing country, walking frequency, disability), each respondent may have a rare or unique attribute profile; the model can then use the attribute vector as a de facto rater ID and learn each rater's average rating tendency, including idiosyncratic offsets. On test ratings from those same raters, this inflates QWK without providing evidence that the attributes generalize to unseen raters. The abstract's claim that 'who is evaluating an environment carries predictive indication beyond image content alone' is only established in-sample under this protocol. The permutation importance in Section IV-D is consistent with memorization: permuting an attribute destroys the identity mapping and causes a large QWK drop, which the paper itself notes can be inflated for correlated predictors. The conclusion acknowledges that 'fully distinctive preferences... are beyond the reach' of the dataset, but the headline result is still framed as evidence about respondent attributes. Therefore, the central claim that attributes, rather than identity, drive the improvement is not yet supported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces a new sidewalk-view walkability perception dataset with 29,870 ratings from 1,196 respondents across Australian urban, suburban, and regional settings, links each rating to seven respondent attributes, and proposes a multimodal deep learning framework that conditions walkability prediction on both image content and respondent attributes. A supplementary viewpoint comparison shows that matched sidewalk-view images receive higher walkability ratings than street-view images. The user-conditioned model is reported to improve quadratic weighted kappa from 0.285 to 0.469 over an image-only baseline, and the paper argues that respondent attributes carry predictive signal beyond image content alone.","tokens_in":16101,"tokens_out":2888,"duration_ms":34818,"significance":"If the central claim is valid, this is a useful contribution: the dataset is novel in pairing sidewalk-view imagery with individual-level attributes and covers suburban and regional contexts that are rare in urban perception datasets. The viewpoint comparison is a clean demonstration that imagery source matters for perceptual ratings. The modeling framework is also, to my knowledge, a first for walkability perception and could inform personalized pedestrian routing. The paper reports five-seed runs, ablations, and clearly disclosed limitations, which are strengths. However, the headline claim that 'who is evaluating' matters beyond image content is not yet established because the evaluation protocol allows the same respondents to appear in both training and test partitions, so the attribute vector may act as a near-unique identifier rather than as a generalizable descriptor.","major_comments":[{"comment":"The rating-completion protocol explicitly allows raters and images to recur across training and test partitions, with only the individual image-respondent rating held out. The paper argues that because conditioning is on respondent attributes rather than respondent identity, 'individual raters could not be directly memorized.' This argument fails if attribute combinations are near-unique across the 1,196 respondents: with seven categorical attributes, many respondents may have rare or unique attribute profiles, and the model can use the attribute vector as a de facto rater ID to learn per-rater average rating tendencies. The 0.184 QWK gain in Table I could then reflect memorization of rater-specific offsets rather than generalizable attribute-driven perception. To support the abstract's claim that 'who is evaluating an environment carries predictive indication beyond image content alone,' the authors should report a rater-disjoint (or rater-grouped) split, where no respondent appears in both training and test, and also report the number of unique attribute profiles in the dataset. Without this experiment, the central claim is only established in-sample.","section":"IV-C1"},{"comment":"The permutation-importance analysis is interpreted as a relative ranking of attribute influence, but under the same leakage concern, permuting any one attribute breaks the potential rater-identification mapping and can cause a large QWK drop regardless of the attribute's true psychological relevance. The paper acknowledges that permutation importance can be inflated for correlated predictors, yet it still frames the ranking as meaningful for understanding which respondent attributes matter. A more robust alternative would be conditional permutation importance or a model trained with a rater-disjoint split before computing importance. As written, Table IV is consistent with memorization of unique attribute combinations and does not provide independent evidence for the causal role of individual attributes.","section":"IV-D"}],"minor_comments":[{"comment":"The heading 'View point comparative analysis' should be 'Viewpoint comparative analysis' for consistency and to avoid a typo.","section":"IV-B"},{"comment":"The phrase 'learning rate of1×10 −4' in the experimental setup is missing a space; it should read 'learning rate of 1 × 10⁻⁴'.","section":"III-C1"},{"comment":"The caption for Fig. 4 is slightly ambiguous: '(d–f) tree pixel coverage difference' can be read as applying to all three panels; consider clarifying that (a–c) show sidewalk coverage and (d–f) show tree coverage.","section":"Fig. 4"}],"recommendation":"major_revision","confidential_remarks":"The rater-leakage issue is the primary obstacle to publication: the current protocol cannot distinguish genuine attribute-driven generalization from memorization of near-unique rater profiles. The authors should be asked to run a rater-disjoint split and report those results; if the QWK gain largely disappears, the abstract and conclusions must be revised accordingly. I also note that the conflict of interest for the imagery provider is disclosed, which is appropriate, and the data/code availability statement is a positive feature."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Good news first: this paper ships a genuinely new artifact. The sidewalk-view walkability dataset (29,870 ratings, 1,196 respondents, 974 images across urban, suburban, and regional Australia) is exactly the kind of resource the field needs, and the viewpoint-comparison study is a clean, useful result: matched sidewalk images get higher walkability ratings than street view, with the effect modest in magnitude and the paper honest about that. The architecture is standard—Swin-Tiny, FT-Transformer, CORAL—but the ablations are careful, the five-seed reporting is good practice, and the paper is unusually candid about limitations. The data and code are promised publicly. That part deserves a serious referee.\n\nThe soft spot is exactly where your reader puts it: the rating-completion protocol. Because the same raters appear in train and test, the 65% QWK gain is plausibly driven by the model memorizing each rater's average tendency through the attribute vector. Seven attributes with sparse categories can be near-unique identifiers. The paper says direct memorization is impossible because conditioning is on attributes, not identity, but that is only true if attribute combinations are not unique per rater; with 1,196 respondents and these cells, they likely are. So the headline claim—that \"who is evaluating\" carries predictive signal beyond image content—is not established by the deep learning evaluation alone. I would not call this fatal: the mixed-effects interaction analysis (settlement type × respondent attributes) provides independent evidence that attributes shape perception, and the paper explicitly frames the deep model as rating-completion within the sampled pool. But the abstract overreaches, and the permutation importance results are consistent with memorization, as the paper itself acknowledges for correlated predictors.\n\nWhat would fix it: a rater-disjoint split (train on 80% of raters, test on the remaining 20%) or at minimum an identity-ablation where a one-hot rater ID replaces the attribute vector. If QWK stays near 0.47 with unseen raters, the claim holds. If it drops to roughly the image-only baseline, the contribution shifts to the dataset and viewpoint study, which are still solid. Either way, this paper deserves peer review, but the rater-disjoint experiment should be required before acceptance.","headline":"Great dataset and a clean viewpoint comparison, but the headline user-conditioning gain is likely inflated by rater leakage; the paper needs a rater-disjoint split before the \"who is evaluating\" claim can stand.","tokens_in":16587,"tokens_out":2674,"would_cite":true,"duration_ms":29066,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"User-conditioned deep learning predicts walkability ratings better when it knows who is rating.","keywords":["walkability perception","user-conditioned prediction","multimodal deep learning","sidewalk-view imagery","street-view imagery","respondent attributes","ordinal regression","urban perception"],"falsifier":"Retrain the user-conditioned model on one disjoint set of respondents and evaluate it on a different set of respondents who rated the same images; if the quadratic weighted kappa gain over the image-only baseline shrinks to near zero, the reported 65% improvement does not represent a transferable attribute-conditioning effect.","tokens_in":15645,"feed_emoji":"🚶","tokens_out":2384,"duration_ms":26735,"temperature":0.7,"pith_summary":"This paper argues that walkability perception is not a single objective property of a street: the same sidewalk is rated differently by different people, and those differences follow systematic patterns tied to who the rater is. To make this case, the authors built a dataset of 29,870 walkability ratings from 1,196 respondents, paired with sidewalk-view imagery across urban, suburban, and regional Australian settings and a set of respondent attributes. They then trained a multimodal model that takes both the image and the respondent's profile as input, and found that adding the respondent profile raises rank agreement with observed ratings by 65% over an image-only baseline (quadratic weighted kappa 0.47 versus 0.29). A secondary study showed that the same location photographed from the sidewalk receives significantly higher walkability ratings than from the street, so the choice of imagery source changes what is being measured. If the central claim holds, aggregated observer-independent walkability scores hide meaningful variation, and future tools could predict walkability separately for different kinds of pedestrians.","feed_headline":"Who's rating matters: user-aware model lifts walkability prediction 65%","feed_subtitle":"A multimodal model that includes the rater's profile beats image-only baselines for perceived walkability, and sidewalk views score higher…","key_machinery":"The load-bearing mechanism is a multimodal fusion architecture that jointly encodes a sidewalk-view panorama and a respondent profile. The image encoder is a Swin-Tiny vision transformer pretrained on ImageNet-1K; the respondent encoder is an FT-Transformer that tokenises each categorical attribute (age, gender, residence, upbringing, walking frequency, disability, childhood country) into dedicated embeddings; a three-block Transformer fusion module combines the two 768-dimensional representations; and a CORAL ordinal head converts the joint embedding into a rank-consistent prediction on the five-point scale. The same framework is used to test the viewpoint hypothesis through a matched sidewalk-versus-street image comparison, and the dataset itself is constructed via Threshold-Constrained Stratified Sampling so that the sampled images preserve the real-world distribution of walkability-relevant street attributes.","core_discovery":"The paper's central discovery is that who is evaluating an environment carries predictive signal beyond what the pixels alone provide. The authors formulate visual walkability perception as a user-conditioned prediction task, where the target rating for an image is conditioned on an explicit respondent attribute vector rather than on a shared mean score. In their experiments, conditioning on respondent attributes improves quadratic weighted kappa from 0.285 to 0.469 on a held-out rating test, a relative improvement of roughly 65%, with the largest gains concentrated at the extremes of the five-point scale where the image-only model fails most. They interpret this as evidence that individual-level variation in perceived walkability is learnable and systematic, not merely noise to be averaged away, and they position the work as the first to apply user-conditioned prediction to walkability perception.","pith_inferences":["If the same attribute-conditional gains replicate on a held-out set of raters who never appear in training, the approach would generalise beyond the current sample; if not, the gain may partly reflect raters' idiosyncratic scale usage rather than generalisable demographic structure.","The attribute set used here (demographics, residence, walking habits) is coarse relative to the full space of individual preferences, so true personalisation would likely require richer signals such as personality traits or direct per-user feedback.","The viewpoint result suggests that any street-view-based walkability dataset, including large public benchmarks, carries an inherent bias that could be corrected by collecting matched sidewalk-view imagery.","A practical extension would be to use the user-conditioned model to generate 'walkability maps for demographic profiles', giving planners a way to see where different populations would feel comfortable walking."],"forward_implications":["Aggregated walkability scores that ignore the evaluator will understate disagreement and may bias planning decisions toward the average user.","Sidewalk-view imagery should be preferred over vehicle-mounted street-view imagery in walkability perception surveys, since the chosen viewpoint materially changes the ratings obtained.","User-conditioned predictions can be inserted into route-choice cost functions, enabling personalised pedestrian routing that reflects an individual's own perception of walkability rather than a city-wide mean.","The model's improved recall at the low and high ends of the rating scale suggests that respondent attributes help correct the central-tendency bias that plagues image-only predictors.","Respondent attributes shape how people differentiate between environments (e.g., urban versus non-urban) more than they shift overall rating levels, so future models should capture attribute-by-image interactions rather than simple additive offsets."],"supporting_citations":[{"why":"Establishes the pairwise-comparison paradigm and Place Pulse 2.0 benchmark that the paper's image-only baseline builds on.","marker":"[11]"},{"why":"Provides evidence that streetscape perceptions vary across demographics and personality, motivating the user-conditioned formulation.","marker":"[6]"},{"why":"Shows that conditioning image aesthetics prediction on subject attributes improves performance, the direct methodological precedent for user-conditioned scoring.","marker":"[15]"},{"why":"Supplies the recommended number of ratings per image and the use of quadratic weighted kappa for Likert-scale perception surveys.","marker":"[9]"},{"why":"Documents weak correspondence between street-view and sidewalk-view imagery, the basis for the viewpoint-comparison hypothesis.","marker":"[8]"},{"why":"Provides the CORAL ordinal regression loss that preserves the rank structure of the five-point rating scale.","marker":"[19]"},{"why":"Defines the Swin Transformer backbone used as the image encoder in the final model.","marker":"[31]"},{"why":"Introduces the FT-Transformer tabular encoder that tokenises respondent attributes and learns inter-attribute interactions.","marker":"[18]"}],"fun_headline_variants":["Rater profile boosts walkability prediction by 65% over image-only","User-aware model scores walkability 65% better than images alone","Walkability perception is personal: AI conditioned on viewer improves 65%","Sidewalk views rate higher, and who's viewing changes scores by 65%","Just pixels? No: adding rater profile lifts walkability accuracy 65%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The model is said to generalise to new raters because it conditions on attributes rather than on identity, yet the same raters appear in both training and test sets; if a rater's attribute combination is nearly unique, the model could be memorising that rater's average tendency rather than learning a general mapping from attributes to perception.","fun_headline_variants_meta":{"raw":{"variants":["Rater profile boosts walkability prediction by 65% over image-only","User-aware model scores walkability 65% better than images alone","Walkability perception is personal: AI conditioned on viewer improves 65%","Sidewalk views rate higher, and who's viewing changes scores by 65%","Just pixels? No: adding rater profile lifts walkability accuracy 65%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000356,"raw_usage":{"total_tokens":1923,"prompt_tokens":929,"completion_tokens":994,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":545,"completion_tokens_details":{"reasoning_tokens":895}},"tokens_in":545,"tokens_out":994,"duration_ms":8957,"temperature":1.0,"reasoning_tokens":895,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T18:02:57.805817+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain the user-conditioned model on one disjoint set of respondents and evaluate it on a different set of respondents who rated the same images; if the quadratic weighted kappa gain over the image-only baseline shrinks to near zero, the reported 65% improvement does not represent a transferable attribute-conditioning effect.","supporting_citations":[{"cited_title":"Deep learning the city: Quantifying urban perception at a global scale,","cited_arxiv_id":null,"evidence_quote":"Establishes the pairwise-comparison paradigm and Place Pulse 2.0 benchmark that the paper's image-only baseline builds on."},{"cited_title":"Global urban visual perception varies across demograph- ics and personalities,","cited_arxiv_id":null,"evidence_quote":"Provides evidence that streetscape perceptions vary across demographics and personality, motivating the user-conditioned formulation."},{"cited_title":"Per- sonalized image aesthetics assessment with rich attributes,","cited_arxiv_id":null,"evidence_quote":"Shows that conditioning image aesthetics prediction on subject attributes improves performance, the direct methodological precedent for user-conditioned scoring."},{"cited_title":"Designing effective image-based surveys for urban visual perception,","cited_arxiv_id":null,"evidence_quote":"Supplies the recommended number of ratings per image and the use of quadratic weighted kappa for Likert-scale perception surveys."},{"cited_title":"Translat- ing street view imagery to correct perspectives to enhance bikeability and walkability studies,","cited_arxiv_id":null,"evidence_quote":"Documents weak correspondence between street-view and sidewalk-view imagery, the basis for the viewpoint-comparison hypothesis."},{"cited_title":"Swin Transformer: Hierarchical vision transformer using shifted windows,","cited_arxiv_id":null,"evidence_quote":"Defines the Swin Transformer backbone used as the image encoder in the final model."},{"cited_title":"Revisiting deep learning models for tabular data,","cited_arxiv_id":null,"evidence_quote":"Introduces the FT-Transformer tabular encoder that tokenises respondent attributes and learns inter-attribute interactions."}],"review_version":1}