{"id":"dedcbec9-51ac-4d80-8bd0-8bfbc6a9744c","arxiv_id":"2505.16352","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A visuo-tactile deep network using a CNN autoencoder and ConvLSTM predicts four haptic attribute ratings from images and tool vibrations, beating single-modality baselines in leave-one-out tests.","lead":"A deep learning system that combines camera images and tool vibration data predicts how people rate surface textures such as rough-smooth and hard-soft, and it outperforms models that use only one modality. The approach could make VR and remote robot touch feel more realistic without streaming raw sensor data.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Unspecified LOOCV unit and post-hoc model/feature selection could make the reported MAE/RMSE gains optimistic; a texture-level reproduction with explicit aggregation is needed.","rationale":"The reader's weakest assumption concerns the validity of LOOCV as a generalization estimate, specifically the unspecified aggregation and normalization procedure. My stress test sharpens this into the most load-bearing issue: the unit of observation is ambiguous, and the architecture and features appear to have been selected using the full dataset before the reported LOOCV. This matters because the quantitative headline is a set of MAE/RMSE numbers, and those numbers can only be trusted if the train/test separation is truly texture-level and if model selection is accounted for. I do not see an internal contradiction in the method; the qualitative claim that multimodal fusion helps is plausible and supported by the ablation structure. The concern is about the reliability of the reported effect size, not about whether the approach has merit. Since the reader already issued a CONDITIONAL verdict, my read does not change that verdict; it reinforces the need for the requested procedural details and a cleaner texture-level evaluation.","tokens_in":18141,"tokens_out":3126,"duration_ms":29665,"concrete_test":"Re-run the full pipeline under strict leave-one-texture-out: for each held-out texture, fit all preprocessing, normalization, and early-stopping validation on the 49 training textures only; predict every segment and image of the held-out texture; aggregate to one texture-level prediction using an explicitly stated rule (e.g., mean of segment predictions); compute texture-level MAE/RMSE. Repeat with at least 3 random seeds and report mean and standard deviation. If the texture-level errors increase materially or overlap the best single-modality baseline within noise, the reported superiority is not established.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim stands or falls on whether the reported LOOCV errors are unbiased estimates for unseen textures. The paper never specifies the evaluation unit. Tactile data are segmented into overlapping 0.5-s windows, and each image is divided into patches and augmented; if errors are computed per segment or per patch rather than per texture, or if overlapping windows straddle the train/test boundary, the MAE/RMSE values in Tables 2-4 are optimistically biased. The paper also states in Section III-C that the architecture was determined through 'extensive empirical experiments', and Table 4 compares feature sets after initial trials, so feature and model selection appear to have used all 50 textures. With n=50 and no error bars or random-seed variation, the 1-3 point gaps over the best tactile baseline may be within selection noise. The qualitative direction of the result is plausible and the psychophysical dataset is a real asset, but the specific quantitative claims are not yet supported without clarifying the evaluation protocol.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes a multimodal visuo-tactile deep-learning framework for predicting four perceptual haptic attributes (rough-smooth, flat-bumpy, sticky-slippery, hard-soft) of 50 real-world textures. Visual features are extracted with pre-trained ResNet-50 plus GLCM descriptors and processed by a 1D-CNN autoencoder; tactile features are MFCCs and statistical descriptors of speed and force, processed by a ConvLSTM. The two 128-dimensional streams are fused and regressed to the four attribute ratings. Evaluation uses leave-one-out cross-validation (LOOCV) against several unimodal and multimodal baselines, reporting MAE and RMSE. The authors claim the proposed method outperforms all baselines on every attribute pair, with the lowest MAE values of 5.23 (R-S), 4.48 (F-B), 6.67 (S-S), and 5.21 (H-S). The paper also presents an individual-feature analysis and discusses class-level error patterns.","tokens_in":18320,"tokens_out":4462,"duration_ms":37418,"significance":"If the quantitative claims are supported, the paper would make a useful contribution to haptic texture perception modeling: it applies ConvLSTM to tactile haptic signals for the first time, combines visual and tactile features in a relatively simple fusion architecture, and evaluates on a real psychophysical dataset with 50 physical textures and four rated attribute pairs. The qualitative direction of the result—multimodal fusion outperforming single-modality baselines—is plausible and consistent with the literature. The paper also provides a useful four-dimensional perceptual-space visualization. However, the central quantitative claims currently rest on an incompletely specified evaluation protocol: the LOOCV unit (texture vs. segment vs. patch) is not stated, early stopping appears to require a validation split that is not described, and feature and architecture selection were conducted using all 50 textures. These issues make the reported MAE/RMSE values potentially optimistic, and the absence of uncertainty measures or significance tests means the margins over baselines may not be meaningful.","major_comments":[{"comment":"The LOOCV evaluation unit is unspecified. Section VI-B says that each iteration trains on 49 textures and validates on one texture, but the input instances are 0.5-second overlapping tactile segments (Section V-A) and augmented image patches (Section V-B). The paper never states whether the errors in Tables 2-3 are computed per segment/patch and pooled, or aggregated to a texture-level prediction, nor what aggregation rule is used (e.g., averaging segment-level predictions). If overlapping segments from the held-out texture appear in the training set, or if per-segment errors are pooled, then Eq. (1) with n=50 in Section VI-A is not a correct texture-level estimator and the reported MAE/RMSE values are optimistically biased. Please specify the exact evaluation unit and aggregation procedure, and ensure no temporal or spatial overlap crosses the train/test boundary.","section":"VI-B and VI-C; Eqs. (1)-(2)"},{"comment":"Early stopping appears to be incompatible with the described LOOCV protocol. Section III-C states that training is run for up to 200 epochs with early stopping based on validation performance and a patience of 10 epochs. In the LOOCV scheme of Section VI-B, each fold has only one held-out texture; if that texture is used as the validation set for early stopping, test information leaks into model selection. If instead an inner validation split is used, that split is not described anywhere. This issue affects every reported MAE/RMSE value in Tables 2-3. Please clarify which validation data control early stopping, or replace early stopping with a fixed epoch count.","section":"III-C with VI-B"},{"comment":"Feature and architecture selection appear to have been performed using all 50 textures, making the reported results selection-optimized rather than independent. Section III-C says the architecture configuration was determined through 'extensive empirical experiments,' and Section VI-E states that 1D-DWT and DFT were dropped after 'initial trials' because of poor performance. These choices were evidently made under the same LOOCV error metric that is later used to report final performance. With n=50 and no held-out test set or nested cross-validation, the 1-3 point MAE gains over the best tactile baseline in Tables 2-3 may be within selection noise. Please either implement nested LOOCV, or explicitly disclose the selection process and provide selection-robust estimates (e.g., reporting performance for the full feature set as well as the selected set).","section":"III-C and VI-E"},{"comment":"No measures of uncertainty or statistical significance are reported. Tables 2 and 3 give single-point MAE/RMSE values with no standard deviations, per-fold errors, random-seed variation, or paired significance tests across textures. For example, the proposed method's 5.23 MAE for R-S is compared with Tactile SVM's 9.40, but the reader cannot tell whether this gap is stable across the 50 LOOCV folds. Please report per-texture error distributions, confidence intervals (e.g., bootstrap), and paired tests (e.g., Wilcoxon signed-rank) for comparisons against the strongest baselines. This is a necessary complement to the LOOCV point estimates.","section":"Tables 2-3 and VI-C"}],"minor_comments":[{"comment":"The text describing Table 4 appears to misread the table: Section VI-E says the combined visual features achieved an RMSE of 10.11 for R-S and outperformed ResNet (18.29) and GLCM (19.11), but Table 4 shows the concatenated visual features have R-S RMSE 13.26, while 10.11 is the F-B value. Please correct the text or the table.","section":"VI-E and Table 4"},{"comment":"Section IV explicitly states that the experimental setup and dataset were adopted from the authors' previous study [14], while the abstract and introduction present the psychophysical experiments and data collection as part of this work. Please clarify what data are reused, what are newly collected, and what the novel contribution is relative to [14].","section":"IV and I"},{"comment":"The claim that class-wise errors fall within the 'Just Noticeable Difference (JND) threshold for perceptual similarity, often estimated at around 10 out of 100 [35]' is not well supported: reference [35] is not a standard source for JND on this type of rating scale, and no experiment here measures JND. Please provide a proper citation or temper the claim.","section":"VII"},{"comment":"Figure 8 plots actual versus predicted attribute values without any indication of inter-participant variability in the averaged ratings; adding per-texture rating spreads or confidence intervals would help assess how much of the residual error is attributable to the model versus human rater noise.","section":"Figure 8"},{"comment":"The statement that the HPS is the 'first visualization to consolidate multi-dimensional haptic attributes into a unified 2D framework' is a strong claim; the 'to the best of our knowledge' qualifier is present, but a brief comparison with existing multidimensional visualization approaches in prior haptic studies would strengthen it.","section":"IV-C"}],"recommendation":"major_revision","confidential_remarks":"The paper's core claim is defensible and the application domain is relevant to the journal, but the evaluation protocol needs substantial clarification before the quantitative results can be accepted. The papers relies heavily on the authors' own prior work [13,14] for the dataset and baselines; the provenance of the ratings and physical signals should be made transparent in the revision. I would not reject on grounds of novelty: applying ConvLSTM to tactile haptic signals and the explicit multimodal fusion are reasonable incremental contributions. The main risk is that the reported improvements are an artifact of selection or leakage; the authors should be given the opportunity to address this with a clear evaluation protocol and uncertainty reporting."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things you should know. First, the paper is a credible, incremental application of known building blocks—ConvLSTM on MFCC tactile features plus a CNN autoencoder on ResNet/GLCM visual features—to continuous haptic attribute prediction. Second, the headline numbers are probably optimistic: the evaluation protocol is underspecified and the model/feature selection used all 50 textures, so the exact MAE/RMSE gaps should not be trusted yet.\n\nWhat's actually new: ConvLSTM hasn't been used for haptic texture attribute regression before, and the two-stream fusion with handcrafted plus deep visual features is a sensible combination. The psychophysical dataset, borrowed from the group's prior work, is a real asset: 50 physical textures with four bipolar ratings. The LOOCV comparison against five baselines is the right kind of evidence for a small dataset.\n\nThe stress-test concern mostly lands, with one correction. Section VI-B explicitly says each fold leaves out an entire texture, so the evaluation unit is textures, not segments. That removes the worst-case leakage story. But the paper never says how per-segment tactile predictions are converted into a texture-level prediction, and no normalization details are given. Architecture choices were 'determined through extensive empirical experiments' (Section III-C), and DWT/DFT were dropped after poor initial trials, so feature selection is post-hoc. With n=50 and no error bars or repeated runs, the 1–3 point improvements over the best tactile baseline could be selection noise. The qualitative direction is credible: MFCC beats DWT/DFT by a mile, and the fused model beats either modality alone. I just wouldn't quote the specific values.\n\nThere are also two concrete presentation problems. Table 4 lists the concatenated visual R-S RMSE as 13.26, but the text says 10.11 (which is actually the F-B value). And reference [50], cited as a vision 1D-CNN baseline, is a heart-rate variability paper, which makes me wonder about the other reproduced baselines.\n\nBottom line: the paper is a decent conference-level contribution with a plausible qualitative result and an honest discussion of class-level errors. It doesn't reshape anything, but it's a fair step for the haptics subfield. I'd send it to peer review, not desk reject it, and ask for major revision: repeat runs with error bars, explicit aggregation and normalization protocol, transparent feature/model selection, and fixes to the table/text mismatch and the citation.","headline":"A plausible incremental visuo-tactile haptic attribute predictor whose qualitative conclusion is probably right, but whose reported error numbers are not yet trustworthy.","tokens_in":18859,"tokens_out":3973,"would_cite":true,"duration_ms":33866,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A two-stream network fusing camera images and tool vibrations predicts human ratings of surface textures—rough/smooth, flat/bumpy, sticky/slippery, hard/soft—with mean errors of 4.5–6.7 on a 100-point scale, beating single-modality…","keywords":["haptic texture attributes","visuo-tactile learning","ConvLSTM","CNN autoencoder","perceptual attribute prediction","leave-one-out cross-validation","MFCC","texture perception"],"falsifier":"Run the same trained pipeline on ten new textures rated with the same protocol and compare per-attribute MAE and RMSE to the reported values; if the errors climb above the roughly 10-point just-noticeable difference cited in the paper, the claimed generalization to unseen textures would not hold.","tokens_in":17919,"feed_emoji":"🖐️","tokens_out":5799,"duration_ms":41287,"temperature":0.7,"pith_summary":"The paper tries to establish that a machine can predict how people perceive surface textures from physical signals, and that combining vision and touch does this better than either modality alone. It builds a dataset of 50 real textures, collects images and tool-recorded vibrations, force, and speed, and obtains human ratings along four bipolar attribute pairs through psychophysical experiments. Under leave-one-out cross-validation, the proposed two-stream network reports mean absolute errors of 5.23, 4.48, 6.67, and 5.21 for rough–smooth, flat–bumpy, sticky–slippery, and hard–soft, with RMSE values of 6.81, 5.67, 7.52, and 6.13, all lower than the tested single-modality baselines. If correct, this offers a scalable substitute for expensive human rating studies and a way to select or compress haptic feedback in VR, teleoperation, and robotic perception.","feed_headline":"Sight plus touch predicts texture feel within 5-7 points","feed_subtitle":"Merging camera images with tool vibrations beats vision-only or touch-only models on all four perceived texture axes.","key_machinery":"The load-bearing object is the visuo-tactile Net, a two-stream architecture. The visual stream (HV-Net) is a CNN-based autoencoder that compresses a 2304-dimensional vector of ResNet-50 and GLCM features into a 128-dimensional representation; the tactile stream (HT-Net) is a six-layer 1D-ConvLSTM, a recurrent network whose internal state transitions use convolutions so it learns spatial and temporal structure together, ingesting per-segment MFCC features from acceleration plus statistical descriptors of speed and force to produce another 128-dimensional representation. The two representations are concatenated into 256 dimensions and passed through fully connected layers to regress the four attribute scores. The ConvLSTM is the mechanism that lets the tactile stream preserve spatial structure while learning temporal dependencies in vibration signals, and the autoencoder is what lets the visual stream denoise and compress high-dimensional image features.","core_discovery":"The paper claims that perceptual ratings of surface texture can be predicted from physical signals by a two-stream deep network, and that combining visual and tactile input is strictly better than using either modality alone. On a dataset of 50 real textures, human ratings along four bipolar attribute pairs (rough–smooth, flat–bumpy, sticky–slippery, hard–soft) serve as targets; images supply the visual stream and tool-recorded acceleration, speed, and force supply the tactile stream. Under leave-one-out cross-validation, the visuo-tactile network reports mean absolute errors of 5.23 (R-S), 4.48 (F-B), 6.67 (S-S), and 5.21 (H-S) and RMSE values of 6.81, 5.67, 7.52, and 6.13, lower on every attribute than the five baselines tested. The authors interpret this as evidence that a perceptually aligned mapping from physical signals to human ratings is learnable and can generalize to textures left out of training.","pith_inferences":["A direct test the paper does not run: take a fresh set of textures, record only images, and check how much of the multimodal accuracy survives; the authors' own feature table suggests vision alone may already capture flat-bumpy, so the tactile stream's unique contribution may vary by attribute.","If the reported errors are reproducible under properly leakage-free leave-one-out evaluation, the same feature pipeline could be turned into a label generator for psychophysical studies, letting researchers rate textures without new human experiments.","The framework's dependence on a 50-texture dataset and a lab-grade tool may not transfer to different sensors or bare-finger interaction; testing with consumer-grade hardware would bound its practical reach.","The unspecified per-segment aggregation and normalization steps make the reported LOOCV numbers likely optimistic; a user should verify performance on genuinely new textures before relying on the model."],"forward_implications":["A model that predicts four perceptual ratings from physical signals can serve as a fast proxy for psychophysical rating sessions, which are time-consuming and costly.","In model-mediated teleoperation, predicted attributes could select or reconstruct haptic feedback from a texture library when raw signals cannot be transmitted in real time.","Perception-based compression becomes possible: store or send attribute vectors instead of raw acceleration and image data, then resynthesize feedback from those vectors.","The hybrid features (ResNet+GLCM for vision, MFCC for touch) improve over either feature family alone, so feature fusion matters even before network fusion.","Reported errors sit mostly below the roughly 10-point just-noticeable difference cited in the paper, meaning predicted ratings would often be perceptually indistinguishable from human ratings."],"supporting_citations":[{"why":"Supplies the 50-texture dataset, the four attribute pairs, and the vision-based Haptic CNN baseline that this work extends.","marker":"[14]"},{"why":"Provides the tactile CNN-LSTM baseline and the prior approach this work replaces with ConvLSTM and multimodal fusion.","marker":"[13]"},{"why":"Supplies the pretrained ResNet-50 features used as deep visual input to the HV-Net autoencoder.","marker":"[15]"},{"why":"Defines the ConvLSTM formulation that gives the tactile stream its spatio-temporal modeling.","marker":"[16]"},{"why":"Provides the MFCC method used to convert acceleration signals into compact, noise-resilient tactile features.","marker":"[45]"},{"why":"Supplies the DFT321 algorithm used to project 3-axis acceleration onto a single axis.","marker":"[44]"},{"why":"Establishes the data acquisition and preprocessing setup for tactile signal recording.","marker":"[43]"},{"why":"Defines the GLCM texture descriptor combined with ResNet features in the visual input vector.","marker":"[46]"},{"why":"Serves as the vision-only 1D-CNN baseline against which the multimodal network is compared.","marker":"[50]"},{"why":"Serves as the tactile SVM baseline against which the multimodal network is compared.","marker":"[51]"}],"fun_headline_variants":["Vision plus vibration bests single senses at predicting texture feel","Two-modality network outpredicts vision-only or touch-only for texture ratings","Deep fusion of sight and touch nails how hard, sticky, rough, or bumpy a surface feels","AI that sees and feels surfaces predicts human texture ratings on four axes","Sight + touch deep net beats unimodal models on all 4 haptic attribute axes"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The headline error numbers rest on the assumption that leave-one-out cross-validation on 50 textures is a trustworthy estimate of performance on new textures, which presumes no information from the held-out texture leaks into preprocessing, feature normalization, or model selection; the paper does not specify how per-segment tactile predictions are aggregated into one texture rating.","fun_headline_variants_meta":{"raw":{"variants":["Vision plus vibration bests single senses at predicting texture feel","Two-modality network outpredicts vision-only or touch-only for texture ratings","Deep fusion of sight and touch nails how hard, sticky, rough, or bumpy a surface feels","AI that sees and feels surfaces predicts human texture ratings on four axes","Sight + touch deep net beats unimodal models on all 4 haptic attribute axes"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000377,"raw_usage":{"total_tokens":2026,"prompt_tokens":980,"completion_tokens":1046,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":596,"completion_tokens_details":{"reasoning_tokens":943}},"tokens_in":596,"tokens_out":1046,"duration_ms":7955,"temperature":1.0,"reasoning_tokens":943,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T15:02:16.815015+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same trained pipeline on ten new textures rated with the same protocol and compare per-attribute MAE and RMSE to the reported values; if the errors climb above the roughly 10-point just-noticeable difference cited in the paper, the claimed generalization to unseen textures would not hold.","supporting_citations":[{"cited_title":"Establishing haptic texture attribute space and predicting haptic attributes from image features using 1d-cnn,","cited_arxiv_id":null,"evidence_quote":"Supplies the 50-texture dataset, the four attribute pairs, and the vision-based Haptic CNN baseline that this work extends."},{"cited_title":"Predicting perceptual haptic at- tributes of textured surface from tactile data based on deep cnn-lstm network,","cited_arxiv_id":null,"evidence_quote":"Provides the tactile CNN-LSTM baseline and the prior approach this work replaces with ConvLSTM and multimodal fusion."},{"cited_title":"Convolutional lstm network: A machine learning approach for precipi- tation nowcasting,","cited_arxiv_id":null,"evidence_quote":"Defines the ConvLSTM formulation that gives the tactile stream its spatio-temporal modeling."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the MFCC method used to convert acceleration signals into compact, noise-resilient tactile features."},{"cited_title":"Di- mensional reduction of high-frequency accelerations for haptic rendering,","cited_arxiv_id":null,"evidence_quote":"Supplies the DFT321 algorithm used to project 3-axis acceleration onto a single axis."},{"cited_title":"Data-driven modeling of anisotropic haptic tex- tures: Data segmentation and interpolation,","cited_arxiv_id":null,"evidence_quote":"Establishes the data acquisition and preprocessing setup for tactile signal recording."},{"cited_title":"Textural features for image classification,","cited_arxiv_id":null,"evidence_quote":"Defines the GLCM texture descriptor combined with ResNet features in the visual input vector."},{"cited_title":"Application of a convolutional neural network for predicting the occurrence of ventricular tachyarrhyth- mia using heart rate variability features,","cited_arxiv_id":null,"evidence_quote":"Serves as the vision-only 1D-CNN baseline against which the multimodal network is compared."},{"cited_title":"Haptic recognition of texture surfaces using semi-supervised feature learning based on sparse representation,","cited_arxiv_id":null,"evidence_quote":"Serves as the tactile SVM baseline against which the multimodal network is compared."}],"review_version":1}