{"id":"d6162f6d-0688-4d2a-b780-e7d5fcebc19b","arxiv_id":"2506.02358","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"RoadFormer, a hybrid convolutional-transformer network with a foreground-background training module, reports top-1 accuracies of 92.52% and 96.50% on the RSCD pavement datasets.","lead":"RoadFormer is a hybrid neural network that combines convolutional and transformer modules to classify road surfaces into 27 detailed categories or 5 simplified conditions from camera images. The paper reports top-1 accuracies of 92.52% and 96.50% on the RSCD datasets, which could matter for autonomous vehicles adjusting speed and braking on wet, icy, or rough roads.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported SOTA gains may stem from undertrained baselines: all models receive only 40 epochs with no pretraining, a regime that disadvantages ViT/Swin; the central superiority claim needs baseline calibration.","rationale":"The reader's weakest assumption identifies the same load-bearing risk: the comparison assumes all baselines were trained to a representative state under an identical protocol, and that the RSCD split matches the official benchmark. My stress-test pass agrees that this is the most serious threat to the paper's central claim. The paper's own ablation (Table V) shows internal consistency: removing FBM and/or the hybrid stacking lowers accuracy by modest amounts, which is plausible and not self-contradictory. Table IV also shows FBM transferring to other models with small gains. These internal results give some credit to the architectural ideas. However, the external claim of SOTA-level superiority is only as strong as the baseline training protocol, and that protocol is described in one sentence with no pretraining mention, no augmentation details, no seeds, and no error bars. The 5.69-12.84% margins are large enough to be suspicious of undertrained baselines. The concrete test I propose would settle this by calibrating the baselines under longer training or pretrained initialization. A secondary issue is Eq. (3), which mislabels Hardtanh; this is a genuine technical error noted by the reader, but it is not load-bearing for the empirical superiority claim because the auxiliary loss could still function with the stated formula as written (it resembles tanh(2x)). I see no reason to change the reader's CONDITIONAL verdict; the concern fully overlaps with the reader's weakest assumption, and the evidence remains insufficient to ACCEPT without baseline calibration and released splits.","tokens_in":14158,"tokens_out":2739,"duration_ms":28131,"concrete_test":"Train ConvNeXt-B, Swin-B, and ViT-B on the same RSCD train/test split used for RoadFormer-B under three conditions: (a) the paper's 40-epoch recipe without pretraining, (b) a 300-epoch cosine schedule with identical augmentation, and (c) ImageNet-1k pretrained initialization followed by 40 epochs of fine-tuning. If RoadFormer-B's 92.52% in Table II remains at least 3 points above all calibrated baselines, the superiority claim survives; if the gaps shrink below the reported 5.69-12.84% or reverse, the central claim is a training-protocol artifact. Report mean and standard deviation over three seeds and publish the exact split to confirm comparability.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that RoadFormer outperforms SOTA classifiers rests on a single comparison protocol described in Section IV (Comparative experiment): 'all models were trained for 40 epochs on the same device' at 224x224. The paper does not state whether ImageNet-pretrained weights were used, nor does it report per-model hyperparameter tuning, augmentation, or repeated seeds. This matters because ConvNeXt, Swin, and especially ViT are known to require substantially longer schedules or pretrained initialization to reach representative accuracy, whereas RoadFormer's convolutional stem and hybrid stages may converge faster. The reported gaps of 5.69-12.84% are unusually large for a 27-class fine-grained task and are consistent with a training-recipe artifact: the baselines may be undertrained rather than architecturally inferior. Since the abstract claims improvement 'compared to SOTA methods' and Table II only shows self-trained baselines, the empirical support for the headline claim is contingent on the fairness of this 40-epoch protocol. Without baseline calibration, the architectural contribution of RoadFormer (local-global stacking and FBM) is not cleanly isolated. A secondary but non-central issue is Eq. (3), which defines 'Hardtanh' as a double-angle tanh identity rather than the standard clamp function; this appears to be a typo or mislabeling and does not by itself overturn the results, but it undermines confidence in the precision of the methods write-up.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes RoadFormer, a hybrid CNN-Transformer architecture for fine-grained road surface classification, combining local convolutional stages, mixed stages, and a global transformer stage. A Foreground-Background Module (FBM) selects high-scoring spatial locations from intermediate classifiers and pushes the remaining locations toward a background target value via an auxiliary MSE loss. Experiments on the RSCD dataset report 92.52% top-1 accuracy on the 27-class version and 96.50% on a simplified 5-class version, with claimed improvements of 5.69% to 12.84% over ConvNeXt, Swin Transformer, and ViT trained for 40 epochs at 224x224 resolution. The paper also includes stacking-structure comparisons, ablations, and confusion-matrix analyses.","tokens_in":14467,"tokens_out":3043,"duration_ms":30756,"significance":"If the reported results hold, RoadFormer would be a strong practical result for road-surface classification, and the systematic exploration of local-global stacking plus a plug-in foreground-background module is a useful contribution. The paper has several strengths: it evaluates four model variants across two dataset granularities, provides ablations separating the contributions of the stacking structure and the FBM, and includes confusion-matrix analyses that point to the dry-wet and wet-water confusions. However, the empirical support for the headline superiority claim is conditional on the fairness of the baseline training protocol, which is not fully specified, and the FBM's self-referential patch-selection mechanism raises a validity concern that the current experiments do not resolve. The absence of error bars, repeated runs, and code also limits the reliability and reproducibility of the quantitative claims.","major_comments":[{"comment":"The central claim of 5.69% to 12.84% improvements over SOTA rests entirely on a single 40-epoch training protocol with no stated pretraining for the baselines, no per-model hyperparameter tuning, and no repeated seeds. ConvNeXt, Swin, and ViT are known to require substantially longer schedules or ImageNet-pretrained initialization to reach representative accuracy, so the reported gaps may reflect training-recipe artifacts rather than architectural superiority. Please report results from the original RSCD paper [62], add a longer-schedule or pretrained-initialization calibration for the baselines, and give per-model hyperparameters and augmentation details so that the comparison is demonstrably fair.","section":"Section IV (Comparative experiment)"},{"comment":"The paper cites the RSCD dataset paper [62] but never compares against the results reported there, so the 'SOTA' label is not grounded in the published literature on this dataset. If the original paper uses a different train/test split or protocol, the discrepancy must be reconciled; if it does not, those numbers should be included in Table II or discussed explicitly. Without this comparison, readers cannot assess whether the reported accuracy is an advance over established results or merely an artifact of the local evaluation setup.","section":"Section IV (Dataset and Table II)"},{"comment":"The FBM loss selects the top-K_i spatial locations according to the stage classifier's own maximum score and then trains those locations toward the foreground while pushing all others toward -1. This creates a self-referential training signal that may reinforce the classifier's existing biases rather than learn a meaningful foreground-background separation. Please add a control experiment that ablates the selection mechanism, for example random selection or selection based on a fixed center crop, and show visualizations of the selected regions to demonstrate that they correspond to pavement rather than to arbitrary high-confidence patches.","section":"Section III (Foreground Background Module, Eq. (5))"},{"comment":"All accuracy, precision, recall, and F1 numbers are single-run point estimates with no error bars, no number of seeds, and no significance tests. The FBM gains in Table IV are about 0.1% in top-1 accuracy, which is within the typical run-to-run variation for this type of model, so the claim that FBM generalizes to other architectures is not statistically supported. Please provide mean and standard deviation over at least three seeds, or otherwise justify that the reported differences exceed training noise.","section":"Tables II-VI"}],"minor_comments":[{"comment":"The definition of Hardtanh appears incorrect or misformatted: the standard Hardtanh is a clamp function, while Eq. (3) resembles a double-angle identity for tanh rather than a hard tanh. Please correct the equation and the surrounding description, as the current text undermines confidence in the methods write-up.","section":"Eq. (3)"},{"comment":"The caption reads 'F represents Front Background Module'; this should be 'Foreground-Background Module' for consistency with the rest of the paper.","section":"Table V caption"},{"comment":"The caption 'Comparison among RoadFormer and efficient Networks' is vague and the figure is not referenced in the main text; please either reference it or remove it.","section":"Fig. 2 caption"},{"comment":"The sentence 'the number of satges is kept constant at 4' contains a typo ('satges' should be 'stages').","section":"Section IV (Stacking Structure)"},{"comment":"The definitions of Mean Precision, Mean Recall, and Mean F1 are given in prose but not as explicit formulas, and it is unclear whether they are macro-averaged or weighted by class frequency; please state the averaging scheme explicitly.","section":"Section IV (Experimental Setup)"}],"recommendation":"major_revision","confidential_remarks":"The paper is likely to be of interest to the intelligent-transportation and autonomous-driving audience, and the architecture exploration is reasonable. The main risk is that the reported superiority over baselines is not yet convincingly established because of the undertrained-baseline concern and the absence of comparison with the dataset paper's own results. I would encourage the editor to require the baseline-calibration and repeated-run experiments described in the major comments before considering acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"RoadFormer is a hybrid CNN-transformer for fine-grained road surface classification. The new bits are the LMMG stacking pattern (local, then two mixed, then global stages) and a foreground-background module (FBM) that adds a per-stage auxiliary loss pushing low-confidence patches toward -1. The paper does a few things right. The ablation is there: without both the stacking and FBM the model gets 91.90, with only stacking 92.34, with only FBM 92.23, with both 92.52. So the components each contribute something, and the internal logic is consistent. Testing on a simplified 5-class split (simple-RSCD) is useful for deployment.\n\nThe soft spots are real and mostly about evidence. The headline comparison (Table II) trains all models for 40 epochs at 224x224, with no mention of pretrained weights, no per-model tuning, no seeds, no error bars. ConvNeXt, Swin, and ViT typically need longer schedules or pretrained initialization to reach representative accuracy, so the 5.69-12.84% gaps look like they could be training-recipe artifacts. The paper also never reports the original RSCD paper's baseline numbers [62], which is a straightforward omission. I can't reconcile Table IV with Table II: the same model names have different accuracies (ConvNeXt 83.69 vs 82.27, Swin 82.91 vs 85.39, ViT 84.19 vs 86.83), so the FBM experiments appear to use different settings, but the paper doesn't say. Equation (3) defines Hardtanh as the double-angle tanh identity, which is not the standard clamp function; it's a typo, but one that should have been caught.\n\nOn the FBM itself: it selects high-confidence patches and pushes the rest to -1, which could reinforce the classifier's biases, but that's a training dynamic, not circular logic. Still, its effect on other models is tiny (about 0.1% top-1), so the module's contribution is modest.\n\nWho is this for? Someone working on road surface perception or hybrid backbone design. It's a reasonable engineering paper, not a paradigm shift. The central claim is plausible but not proven. I'd send it to peer review, but as major revision, with a request for code, seeds, error bars, per-model tuning or pretraining details, and a comparison against the published RSCD numbers.","headline":"Plausible hybrid architecture with honest ablations, but the SOTA claim rests on an under-specified baseline protocol and missing reproducibility artifacts.","tokens_in":14975,"tokens_out":2963,"would_cite":false,"duration_ms":27829,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"RoadFormer, a hybrid convolution-transformer network with a foreground-background auxiliary loss, claims 92.52% top-1 accuracy on the 27-class RSCD road surface benchmark, beating ConvNeXt, Swin, and ViT by 5.69 to 12.84 points.","keywords":["road surface classification","fine-grained classification","local-global feature fusion","CNN-transformer hybrid","foreground-background module","autonomous driving","RSCD dataset","pavement perception"],"falsifier":"Retrain ConvNeXt, Swin, and ViT under the paper's exact protocol, or use the baselines' official pretrained weights fine-tuned on RSCD, and compare on the official RSCD test split; if the accuracy gaps shrink to near zero, the claimed advantage is not architectural. A second check is to overlay FBM's selected foreground regions on input images and measure whether they align with actual road pixels via a segmentation mask.","tokens_in":13944,"feed_emoji":"🛣️","tokens_out":5388,"duration_ms":40995,"temperature":0.7,"pith_summary":"This paper claims that RoadFormer, a vision-only network that stacks convolutional (local) and transformer (global) stages in a specific order, outperforms state-of-the-art classifiers on fine-grained road surface classification. The paper reports 92.52% top-1 accuracy on the 27-class RSCD benchmark and 96.50% on a 5-class simplified version, improvements of 5.69 to 12.84 percentage points over ConvNeXt, Swin Transformer, and ViT baselines. The authors attribute the gain to two mechanisms: a local-global stacking that captures both texture details and long-range context, and a Foreground-Background Module that pushes intermediate features to separate road from irrelevant surroundings. If correct, this would make RoadFormer the best reported vision-only classifier on this dataset and a candidate backbone for autonomous driving perception.","feed_headline":"Hybrid CNN-transformer tops road surface classification at 92.5%","feed_subtitle":"Stacking local and global stages plus a foreground-background loss beats ConvNeXt, Swin, and ViT on RSCD.","key_machinery":"The central object is the four-stage hybrid backbone with a Foreground-Background Module (FBM). The backbone passes features through a Stem, then Stage1-L (conv blocks only), Stage2-M and Stage3-M (mixed conv and transformer blocks), and Stage4-G (transformer blocks only), with patch embedding between stages, so local texture detail and global pixel relationships are both represented at multiple resolutions. The FBM attaches a linear classifier to each stage's feature map, softmaxes it into a classification map, takes the max score per spatial position, selects the top-K positions as foreground (with K decreasing from 256 to 32 across stages), and trains a Hardtanh-mapped background prediction toward -1 with a mean-squared-error loss. This loss is meant to expand the separation between road-relevant and road-irrelevant regions without requiring segmentation labels.","core_discovery":"The central discovery is that a four-stage hybrid backbone composed of a local stage, two mixed stages, and a global stage combined with an auxiliary foreground-background separation loss is enough to lift fine-grained road surface classification accuracy well above pure CNN and pure transformer models of comparable size. On RSCD, RoadFormer-B reaches 92.52% top-1 accuracy, beating ConvNeXt-B by 8.44 points, Swin-B by 6.84 points, and ViT-B by 5.69 points, with fewer parameters. The ablation shows the hybrid stacking alone raises accuracy from 91.90% to 92.34%, the FBM alone to 92.23%, and both together to 92.52%, indicating the two contributions combine. The FBM also transfers: adding it to ConvNeXt, Swin, and ViT improves their top-1 accuracy by about 0.1 points, showing the foreground-background prior is not architecture-specific.","pith_inferences":["If the FBM's foreground-background prior generalizes, the same auxiliary-loss trick could transfer to other fine-grained classification tasks such as material recognition or terrain type classification, which the paper does not test.","The small FBM gains on other models (~0.1 points) relative to the stacking-structure gain (~0.4 points in the ablation) suggest that the local-global stacking, not the FBM, carries most of RoadFormer's advantage; the paper does not draw this conclusion.","The decreasing foreground selection K values (256, 128, 64, 32) were chosen heuristically; a learned or resolution-adaptive selection rule could be tested and might yield further gains.","Because label pairs such as dry-wet and wet-water are inherently ambiguous, the reported accuracy may partly reflect annotation consistency rather than perceptual separability; a re-annotation study could quantify this."],"forward_implications":["At the base size, RoadFormer reaches 92.52% top-1 accuracy on the 27-class RSCD and 96.50% on the 5-class simple-RSCD, with 80M parameters.","RoadFormer-L reaches 92.86% on RSCD, the best result the paper reports.","The FBM is transferable: adding it to ConvNeXt, Swin, and ViT raises their top-1 accuracy by about 0.1 points, suggesting the auxiliary separation loss is a plug-in module.","Confusion-matrix analysis shows residual confusion between dry-wet and wet-water label pairs, pointing to label ambiguity as a remaining bottleneck.","The hybrid stacking structure outperforms pure-CNN and pure-transformer orderings, supporting the paper's claim that local-global mixing at multiple scales is beneficial for fine-grained texture tasks."],"supporting_citations":[{"why":"Supplies the ConvNeXt baseline and its variant configurations that RoadFormer is directly compared against.","marker":"[64]"},{"why":"Supplies the Swin Transformer baseline and its variant configurations that RoadFormer is directly compared against.","marker":"[37]"},{"why":"Supplies the ViT baseline that RoadFormer is directly compared against at the base and large sizes.","marker":"[36]"},{"why":"Provides the RSCD dataset and its benchmark protocol, which is the primary testbed for the claimed accuracy improvements.","marker":"[62]"},{"why":"Documents the original road surface image dataset with detailed annotations that was later expanded to create RSCD.","marker":"[63]"}],"fun_headline_variants":["Hybrid CNN-transformer hits 92.5% on fine-grained road surface classification","RoadFormer fuses local-global stages for 92.5% top-1 accuracy","CNN-transformer hybrid with FBM beats SOTA by up to 12.8% on roads","Foreground-background module gives RoadFormer the edge on fine-grained roads","Stacked local-global features push road surface accuracy to 92.5%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The accuracy comparisons assume the baseline models were trained to a comparable state using the same untuned 40-epoch recipe and the same RSCD train/test split; if the baselines were undertrained or the split differs, the reported gains could come from training protocol or data selection rather than from RoadFormer's architecture.","fun_headline_variants_meta":{"raw":{"variants":["Hybrid CNN-transformer hits 92.5% on fine-grained road surface classification","RoadFormer fuses local-global stages for 92.5% top-1 accuracy","CNN-transformer hybrid with FBM beats SOTA by up to 12.8% on roads","Foreground-background module gives RoadFormer the edge on fine-grained roads","Stacked local-global features push road surface accuracy to 92.5%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00072,"raw_usage":{"total_tokens":3287,"prompt_tokens":1053,"completion_tokens":2234,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":669,"completion_tokens_details":{"reasoning_tokens":2123}},"tokens_in":669,"tokens_out":2234,"duration_ms":16571,"temperature":1.0,"reasoning_tokens":2123,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T11:24:57.595107+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain ConvNeXt, Swin, and ViT under the paper's exact protocol, or use the baselines' official pretrained weights fine-tuned on RSCD, and compare on the official RSCD test split; if the accuracy gaps shrink to near zero, the claimed advantage is not architectural. A second check is to overlay FBM's selected foreground regions on input images and measure whether they align with actual road pixels via a segmentation mask.","supporting_citations":[{"cited_title":"A convnet for the 2020s,","cited_arxiv_id":null,"evidence_quote":"Supplies the ConvNeXt baseline and its variant configurations that RoadFormer is directly compared against."},{"cited_title":"Swin transformer: Hierarchical vi- sion transformer using shifted windows,","cited_arxiv_id":null,"evidence_quote":"Supplies the Swin Transformer baseline and its variant configurations that RoadFormer is directly compared against."},{"cited_title":"A compre- hensive implementation of road surface classification for vehicle driving assistance: Dataset, models, and deploy- ment,","cited_arxiv_id":null,"evidence_quote":"Provides the RSCD dataset and its benchmark protocol, which is the primary testbed for the claimed accuracy improvements."},{"cited_title":"A road surface image dataset with detailed annotations for driving assistance applications,","cited_arxiv_id":null,"evidence_quote":"Documents the original road surface image dataset with detailed annotations that was later expanded to create RSCD."}],"review_version":1}