{"id":"752fbcf9-bef0-48f3-a2ea-e63f86ca272d","arxiv_id":"1908.08855","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A median-absolute-deviation reweighting rule improves neural network regression on datasets with output outliers, but the paper omits the loss function, code, and an IRLS baseline.","lead":"The paper proposes a robust regression method that repeatedly down-weights measurements with large errors, using a median-based threshold, and tests it on simulated and real robot data. It is a modest variant of existing robust regression techniques and would matter mainly for making neural network models less sensitive to dirty sensor data.","discovery_kind":"incremental","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'no distribution assumption' claim is undermined by the fixed gamma=2*MAD threshold and near-hard weight e^{-7v^8}, which implicitly assumes a residual scale and discards many clean Gaussian inliers.","rationale":"The reader's weakest assumption was that outliers are detectable as large output residuals after an initial fit and that gamma=2 separates inliers from outliers, with input-space leverage points untested. My concern is closely related but more specific: the fixed threshold with unnormalized MAD and the near-hard weight e^{-7v^8} impose an implicit Gaussian scale assumption, so the paper's claim of being distribution-free is internally overstated. This is load-bearing because the entire outlier-identification mechanism reduces to this threshold. However, the concern is addressable by additional experiments or by reformulating the claim, so it does not require changing the reader's CONDITIONAL verdict. The real-data Table III, where the robust model is worse than the traditional baseline on all metrics, further weakens the central claim, but the simulated outlier experiments do provide some evidence of robustness, so the appropriate outcome remains conditional acceptance pending clarification and validation.","tokens_in":5098,"tokens_out":6037,"duration_ms":71684,"concrete_test":"Run the proposed method on clean synthetic data containing no outliers (e.g., the nonlinear Dataset 1 setup with Gaussian noise only) and record the fraction of true inliers assigned weight < 0.01 in the final iteration. For Gaussian residuals with gamma=2 and unnormalized MAD, theory predicts approximately 18% of inliers will be downweighted to near zero; if the observed fraction is close to this value, the method is discarding a substantial share of valid data, contradicting the 'no distribution assumption' claim. Also compare the resulting model RMSE against the unweighted NN on the same clean data to quantify the bias introduced.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that the method performs outlier identification 'without assuming a data distribution' rests entirely on the median/MAD threshold with a fixed multiplier gamma=2 and the weight w_{i,j}=e^{-7v^8}. This mechanism is not distribution-free: for Gaussian residuals, MAD is approximately 0.6745 sigma, so the threshold is 2*MAD = 1.349 sigma, not a typical 2- or 3-sigma rule. Because e^{-7} ≈ 0.0009, any point whose residual is even slightly beyond 1.349 sigma from the median is effectively discarded. On clean Gaussian data, this rejects about 18% of genuine inliers in every iteration, meaning the method is not simply 'minimally influenced by outliers' but is also strongly influenced by an implicit Gaussian-scale assumption. The paper never states the loss minimized by the weighted NN, so this threshold is the entire outlier-detection mechanism; it is neither derived from a robust loss nor validated across non-Gaussian inlier distributions. The real-data results are consistent with this concern: in Table III the robust NN has worse R2 and RMSE than the traditional NN for all four outputs on train, test, and validation, so the claimed practical advantage is not demonstrated.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes an iterative reweighting scheme for robust regression with neural networks. For each output dimension, residuals from an initial fit are converted into sample weights via w_{i,j} = exp(-7 v^8), where v = (r_{i,j} - m_i) / t_i, m_i is the residual median, and t_i = gamma * MAD_i with gamma = 2; the model is retrained over a fixed number of refinements. The authors claim that the method identifies and rejects outliers during regression without any assumption on the data distribution. The method is tested on synthetic linear and nonlinear regression problems with Gaussian noise and injected output outliers, and on a real tendon-driven surgical robot dynamics dataset, comparing against traditional neural networks, RANSAC, and Gaussian process regression.","tokens_in":5331,"tokens_out":4157,"duration_ms":44321,"significance":"If the central claims were fully supported, the paper would offer a simple plug-in robustification for neural-network regression that requires no preprocessing, which would be practically useful in robot model learning. The paper has genuine strengths: it evaluates on external synthetic data and real sensory data, it includes comparisons to RANSAC and Gaussian process regression, and the proposed reweighting idea is simple and potentially generalizable to other regressors. However, as written, the method is not fully specified, the 'no distribution assumption' claim is questionable, and the real-data experiments do not demonstrate an advantage over the traditional neural network. The contribution is therefore plausible but not yet established.","major_comments":[{"comment":"The paper never states the loss function that the weights W_{i,j} actually enter. The text says only that 'the model is refined with the new weights,' but it does not specify whether the weights multiply the squared error in a weighted mean-squared-error objective, whether they are used to resample the training data, or whether they scale the gradient in some other way. Without this equation, the experiments are not reproducible, and the reader cannot verify that the reported improvement is due to Eq. (2) rather than to some implementation detail. Please provide the exact weighted training objective and the stopping criterion for the iterative refinements.","section":"Section II, Eq. (2)"},{"comment":"The claim that the method 'doesn't make any assumption on the data distribution' is not supported by the mechanism. With gamma = 2 and MAD as the scale, the threshold is t = 2 * MAD. For Gaussian inlier residuals, MAD is approximately 0.6745 sigma, so the threshold is roughly 1.349 sigma; combined with the near-hard weight e^{-7 v^8}, any point with residual more than about one median absolute deviation from the median receives a weight near 0.0009. On clean Gaussian data, this effectively discards a substantial fraction of genuine inliers at every iteration (about 18% for a Gaussian if the residuals are exactly Gaussian). This is a strong implicit scale and distribution assumption. The paper provides no sensitivity analysis for the free parameters gamma = 2, coefficient 7, exponent 8, or the number of refinements, so the robustness claim is not demonstrated across inlier distributions. Please either add such an analysis or substantially soften the distribution-free claim.","section":"Section II and Section IV"},{"comment":"The real-robot results do not support the claimed practical advantage. In Table III, the robust NN has worse R2 and RMSE than the traditional NN for all four outputs on train, test, and validation (e.g., output 4: R2 0.969 vs 0.972 and RMSE 0.0649 vs 0.0609). The text states that the robust method 'allows to have smoother mapping, being less influenced by unwanted data,' but the reported metrics contradict any improvement. Please discuss this discrepancy directly, report a quantitative measure of smoothness if that is the intended advantage, and consider adding a real-data experiment with injected outliers to show where the robust method helps.","section":"Table III and Section III.B"},{"comment":"The metric definitions are ambiguous and the reported numbers are internally hard to interpret. The captions say 'R2 and RMSE between the computed models and the desired mapping,' but for the nonlinear case with outliers, the robust NN has RMSE 0.0521 yet R2 0.1722, while the traditional NN has RMSE 0.3825 yet R2 0.2302. If both metrics are computed against the same clean desired mapping, the R2 ordering is counterintuitive given the RMSE ordering. Please state precisely what target each metric is computed against (contaminated observations, clean mapping, or held-out data), give the formulas used, and ensure that all methods are compared on the same target. This is essential because the headline comparisons in the abstract and conclusions rest on these numbers.","section":"Tables I and II captions"},{"comment":"The experimental comparison omits the most natural baseline for an iteratively reweighted method, namely iteratively reweighted least squares (IRLS), even though IRLS is cited in the introduction. Since the proposed scheme is essentially a robust reweighting scheme, a comparison with IRLS on the linear problems, and with a robust-loss neural network on the nonlinear problems, would be needed to quantify the claimed improvement over existing robust regression approaches. Without such a baseline, it is difficult to assess the novelty and practical value of the contribution.","section":"Section III.A"}],"minor_comments":[{"comment":"The notation is inconsistent: r_i is defined as a vector, but Eq. (2) uses r_{i,j}, and the expression should be parenthesized as v = (r_{i,j} - m_i) / t_i to avoid ambiguity about operator precedence.","section":"Section II, Eq. (2)"},{"comment":"The text says the dataset is 'divided randomly into train set (80%,10%, 10%)' but the two 10% portions are not labeled; please specify which is validation and which is test.","section":"Section III.A"},{"comment":"There are several typos and unclear figure labels: 'datset', 'mange', 'Guassian', 'therms', and 'nolinearities' should be corrected, and the figure legends such as '10 nodes20 nodes' should be made readable.","section":"Section III.A and Figures 1-3"},{"comment":"The row labels '10 nodes' and '20,10 nodes' in the nonlinear tables do not match the text's description of 'one single hidden layer with 20 nodes and a two-hidden-layer structure with 20 and 10 nodes'; please make the architecture naming consistent.","section":"Table II"},{"comment":"The introduction says outlier detection methods 'rely on only one particular method, or on the knowledge of the data statistical distribution,' but the proposed method also relies on a threshold and a fixed weight function; the distinction should be stated more carefully.","section":"Section II"}],"recommendation":"major_revision","confidential_remarks":"This appears to be a lightly revised version of an IROS 2019 preprint. The main technical idea is reasonable and potentially publishable after major revision, but the missing loss specification, the overclaimed distribution-free property, and the unconvincing real-data metrics need to be addressed before it can be accepted. I recommend major revision rather than rejection because none of the issues appears to be a fundamental flaw in the core reweighting concept; they are fixable with additional experiments, a precise problem statement, and revised claims."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is an IRLS variant with an ad hoc weight function w=e^{-7v^8} and median/MAD thresholding. It works on the synthetic examples shown, but the paper overclaims distribution-freeness, the metrics are internally inconsistent, and the real-data test doesn't support the claimed benefit.\n\nWhat's actually new: the specific weight shape and the iterative scheme with unit-weight initialization and t=2*MAD. That's a minor extension of iteratively reweighted least squares, which they cite. The paper is honest that IRLS exists, and the synthetic linear experiment with a cluster of input-region noise is a decent demonstration that the method can ignore a block of outliers. The nonlinear experiment with 150 injected outliers also shows the robust model keeps RMSE relatively low compared to the traditional NN and GPR, though the R2 numbers are strange.\n\nSoft spots, in rough order of severity:\n\n1. The loss function is never stated. The paper says weights are applied, but not what weighted objective is minimized. That's a reproducibility gap you could drive a truck through.\n\n2. The metric definitions are inconsistent. Table I reports robust NN with R2=0.9927 and RMSE=0 against the desired mapping—if RMSE is exactly zero, R2 should be 1. In Table II, robust NN has lower RMSE but markedly lower R2 than traditional NN on the outlier cases. Either R2 and RMSE are computed against different references, or something is mislabeled. The reader can't tell which.\n\n3. No IRLS baseline. Since the method is IRLS with a different weight function, you need to compare against standard IRLS with a Huber or bisquare loss to show the ad hoc weight is actually better. Without that, the 'novelty' is just a parameterization.\n\n4. The 'no distribution assumption' claim doesn't survive contact with the experiment. With gamma=2 and MAD, on Gaussian residuals the threshold is about 1.35 sigma, and e^{-7} ~ 0.0009 at that distance. So on clean Gaussian data the method effectively discards ~18% of genuine inliers every iteration. That is not distribution-free; it's an implicit assumption about residual scale. The paper never acknowledges this.\n\n5. The real robot experiment actually favors the traditional NN on every reported R2 and RMSE for train, test, and validation. The paper says the robust method gives a smoother mapping, but the numbers don't support any advantage.\n\n6. No code, no data, no error bars, no sensitivity analysis for gamma or the exponent. For a method paper, that's a serious omission.\n\nWho is this for? Someone who wants a simple plug-in robust reweighting rule for NN regression and is willing to treat the specific weight as a tuning knob. It's a plausible engineering trick, not a major advance. It deserves a serious referee because the idea is coherent and testable, but it needs major revision: state the loss, fix the metrics, add the IRLS baseline and sensitivity analysis, and dial back the distribution-free claim.","headline":"An IRLS variant with a hard threshold weight that works on synthetic outlier examples but overclaims distribution-freeness and is under-specified on the loss and metrics.","tokens_in":5863,"tokens_out":3843,"would_cite":false,"duration_ms":34303,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a residual-based reweighting rule can identify outliers while a regression model is being learned, keeping robot models accurate even when sensor data are corrupted, without assuming a particular data distribution.","keywords":["robust regression","outlier detection","iterative reweighting","robot model learning","neural networks","median absolute deviation","inverse dynamics","sensor noise"],"falsifier":"Build a regression dataset in which 15% of the points are leverage points—inputs far outside the bulk of the data—with output values deliberately set so that their residuals after a first unweighted fit stay below $2\\,\\mathrm{MAD}$; if the final model is pulled toward those points, the residual-only detection rule is not sufficient, and the paper's claim that outliers are identified during regression would fail for input-space outliers.","tokens_in":4846,"feed_emoji":"🤖","tokens_out":5895,"duration_ms":58961,"temperature":0.7,"pith_summary":"Many robot models are learned from sensor measurements that contain outliers, and standard regression methods such as neural networks are pulled off course by those bad points. This paper presents an iterative reweighting scheme that assigns each training sample a weight based on how far its prediction residual lies from the median residual, measured in median absolute deviations. The claim is that this identifies and suppresses outliers during regression itself, with no separate preprocessing step and no assumption about the data's statistical distribution. The approach is demonstrated with neural networks on simulated linear and nonlinear regressions and on the inverse-dynamics model of a tendon-driven surgical robot. If the claim holds, adding this weighting rule to an existing regression method would make the learned model minimally influenced by bad sensor data.","feed_headline":"Weighting rule finds outliers while fitting robot models","feed_subtitle":"A residual-based reweighting scheme rejects bad sensor data without assuming a distribution; tested on a surgical robot.","key_machinery":"The load-bearing object is the weight function $W_{i,j} = e^{-7 v^8}$ applied to the normalized residual $v = (r_{i,j} - m_i)/(\\gamma\\,\\mathrm{MAD}_i)$. The eighth power makes the weight almost exactly one for inliers and drives it rapidly toward zero once the residual passes the threshold, giving a smooth downweighting rather than a hard cut. The median and the median absolute deviation play the role of robust location and scale estimates, so the threshold adapts to each output dimension without knowing the noise distribution. The iterative loop—train, compute residuals, reweight, retrain—is what turns the static formula into a regression procedure.","core_discovery":"The central discovery is that a single reweighting formula, applied iteratively while training a regressor, is enough to make the learned model robust to output outliers. For each output component the paper computes residuals $r_{i,j}$, their median $m_i$, and the median absolute deviation $\\mathrm{MAD}_i$; the threshold is $t_i = \\gamma\\,\\mathrm{MAD}_i$ with $\\gamma=2$ in the experiments. Each sample's weight is $W_{i,j} = e^{-7 v^8}$, where $v = (r_{i,j} - m_i)/t_i$, so points whose residual is far from the median are heavily downweighted. Re-estimating the model with these weights over several refinements yields models that, in the reported simulations, stay close to the true mapping even when 150 of 2000 nonlinear-regression points are injected outliers, while traditional neural networks and Gaussian process regression are visibly degraded. On the robot inverse-dynamics data the robust model tracks the measured torques with similar $R^2$ to the traditional network but produces a smoother mapping, which the paper interprets as being less influenced by unwanted measurements.","pith_inferences":["The simulations only corrupt outputs; a natural next test would be leverage points in the input space, since residual-based reweighting is blind to an outlier that manages to keep its residual small.","The fixed multiplier $\\gamma=2$ and the fixed number of refinements are not tuned to contamination level; users facing heavier or asymmetric outlier distributions would likely need a rule for adjusting them, which the paper leaves open.","The per-output-component weighting suggests a direct extension to multi-output regression with heterogeneous noise: each output keeps its own median and MAD threshold, and the same iteration scheme should carry over."],"forward_implications":["The same weighting rule can be attached to any regression estimator, not just neural networks, turning ordinary model fitting into an outlier-robust procedure without changing the underlying learner.","Separate preprocessing for outlier removal becomes unnecessary: detection happens during training, which simplifies pipelines for learned robot models.","Robot inverse-dynamics models trained on tendon-driven systems can be smoothed against transient sensor glitches, which matters for control applications that differentiate or integrate the learned torques.","The price is computation: the robust models required more training time than plain neural networks, comparable to Gaussian process regression in the nonlinear experiments."],"supporting_citations":[{"why":"Supplies the definition of an outlier as a data point significantly different from the others, which motivates the residual-based criterion.","marker":"[4]"},{"why":"Supplies the background that robust regression and outlier detection are coupled, the problem this method targets.","marker":"[5]"},{"why":"Supplies the iteratively reweighted least-squares tradition that the proposed iterative reweighting scheme extends.","marker":"[7]"},{"why":"Provides RANSAC, used as the linear-regression baseline the method must beat.","marker":"[8]"},{"why":"Supplies the universal approximation result that justifies using neural networks to represent the mapping.","marker":"[9]"},{"why":"Supports the premise that neural network behaviour is influenced by outliers, the gap the method addresses.","marker":"[10]"},{"why":"Also supports the premise that outliers degrade neural network training, motivating a robust error measure.","marker":"[11]"},{"why":"Provides the Micro-IGES tendon-driven surgical robot whose inverse-dynamics data are used for the real-data validation.","marker":"[13]"}],"fun_headline_variants":["Single reweighting formula makes robot models robust to outliers","Residual-based downweighting yields robust robot models from noisy sensor data","One formula reweights residuals to ignore outliers in robot learning","Median residual reweighting makes robot models robust to outliers","Iterative downweighting rejects outliers without distribution assumptions"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that outliers are visible as large output residuals after an initial fit and that the median plus twice the median absolute deviation of those residuals reliably separates bad points from good ones.","fun_headline_variants_meta":{"raw":{"variants":["Single reweighting formula makes robot models robust to outliers","Residual-based downweighting yields robust robot models from noisy sensor data","One formula reweights residuals to ignore outliers in robot learning","Median residual reweighting makes robot models robust to outliers","Iterative downweighting rejects outliers without distribution assumptions"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001418,"raw_usage":{"total_tokens":5686,"prompt_tokens":867,"completion_tokens":4819,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":483,"completion_tokens_details":{"reasoning_tokens":4736}},"tokens_in":483,"tokens_out":4819,"duration_ms":36584,"temperature":1.0,"reasoning_tokens":4736,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T11:28:05.464384+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Build a regression dataset in which 15% of the points are leverage points—inputs far outside the bulk of the data—with output values deliberately set so that their residuals after a first unweighted fit stay below $2\\,\\mathrm{MAD}$; if the final model is pulled toward those points, the residual-only detection rule is not sufficient, and the paper's claim that outliers are identified during regression would fail for input-space outliers.","supporting_citations":[{"cited_title":"A Note on Computing Robust Regression Estimates Via Iteratively Reweighted Least Squares,","cited_arxiv_id":null,"evidence_quote":"Supplies the iteratively reweighted least-squares tradition that the proposed iterative reweighting scheme extends."},{"cited_title":"Random sample consensus: a paradigm for model fitting with applications to image analy- sis and automated cartography,","cited_arxiv_id":null,"evidence_quote":"Provides RANSAC, used as the linear-regression baseline the method must beat."},{"cited_title":"Approximation capabilities of multilayer feedfor- ward networks,","cited_arxiv_id":null,"evidence_quote":"Supplies the universal approximation result that justifies using neural networks to represent the mapping."},{"cited_title":"The Ef- fects of Outliers Data on Neural Network Performance,","cited_arxiv_id":null,"evidence_quote":"Supports the premise that neural network behaviour is influenced by outliers, the gap the method addresses."},{"cited_title":"Robust error measure for supervised neural network learning with outliers,","cited_arxiv_id":null,"evidence_quote":"Also supports the premise that outliers degrade neural network training, motivating a robust error measure."},{"cited_title":"A Single-Port Robotic System for Transanal MicrosurgeryDe- sign and Validation,","cited_arxiv_id":null,"evidence_quote":"Provides the Micro-IGES tendon-driven surgical robot whose inverse-dynamics data are used for the real-data validation."}],"review_version":1}