{"id":"de58f3c4-25f8-48e2-b9cc-b922341a2d20","arxiv_id":"2501.14012","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A CMA-ES-based affine transformation lets pre-trained random forest surrogates transfer to related target tasks using tens of samples, beating from-scratch models in many low-data benchmarks.","lead":"The paper adapts a transfer learning trick for Gaussian processes to random forest surrogates, tuning a rotation and translation so a model trained on one task can be reused on a related task with only a few new samples. It reports lower error than training from scratch on most BBOB functions and on several real-world benchmarks, especially when target data is scarce.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Real-world benchmarks are never checked for the assumed affine symmetry; the paper's broad 'across real-world benchmarks' claim rests on an unvalidated assumption, and mixed real-world results (e.g., torque7→torque1) show the transfer mechanism is not understood.","rationale":"The reader's weakest-assumption analysis already identifies the affine symmetry as the load-bearing premise, and I agree that this is the most critical point. My emphasis is slightly different: rather than focusing on the SO(d) restriction excluding scaling/reflections, I stress that the real-world benchmarks are never verified to satisfy any affine relation at all, and that the mixed results (particularly robot-arm torque transfers) suggest the method may work or fail for reasons unrelated to the assumed symmetry. This does not overturn the reader's CONDITIONAL verdict; it reinforces it by pointing to a concrete missing control. The proposed negative-control test would determine whether the observed real-world transfer gains actually depend on the affine assumption or arise from generic model reuse and RFR flexibility. If the negative control shows random pairs also benefit from transfer, the paper's central mechanism is not what drives the real-world results, and the claims would need substantial revision. If it does not, the assumption is likely necessary and the paper's conditional statements are appropriate. I therefore recommend keeping the verdict unchanged while adding this test to the list of required revisions.","tokens_in":40805,"tokens_out":9772,"duration_ms":92507,"concrete_test":"Run a negative-control experiment on each real-world task: randomly permute the source-target pairs (or use a pre-trained model from a different benchmark as the source) and apply the same TL-CMA-ES transfer procedure. If the transferred model beats scratch training on a substantial fraction of random pairs, the real-world wins are not attributable to the affine domain relation. If it does not, the affine premise is likely load-bearing. Additionally, for each original pair, fit (W,v) on a large random subset (e.g., 1,000 points) and compute the held-out SMAPE of f_S(Wx+v) against f_T(x); if the best affine fit is not clearly better than the identity map and does not approach the scratch-trained model's error, that pair does not exhibit the assumed affine symmetry.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that a pre-trained RFR can be transferred to a target satisfying f_T(x)=f_S(Wx+v) with W∈SO(d), and that this beats scratch training with 50–100 target samples, especially in high dimensions. On the BBOB synthetic tasks this premise is enforced by construction, so those results only demonstrate that CMA-ES can recover an affine transform when one is guaranteed to exist. The real-world section is presented as validation, but no evidence is given that any source-target pair actually satisfies the affine relation. For robot-arm torque pairs the input domain is identical and the outputs are different physical quantities; for porkchop plots different launch windows are not known to be affine-related; and the authors explicitly concede the Vehicle Dynamics task violates the assumption. The mixed real-world outcomes, such as torque7→torque1 losing to scratch at every transfer size while torque1→torque3 wins, are consistent with the method being sensitive to unmeasured structural properties rather than a validated affine relationship. Consequently, the paper demonstrates a working optimizer for an exactly affine synthetic setup, but the title-level claim of transferring 'across synthetic and real-world benchmarks' remains conditional on an assumption that is never tested on the real-world data.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a method for transferring random forest regression (RFR) surrogate models from a source function to a target function assumed to be related by an input-domain transformation f_T(x) = f_S(Wx + v), where W is a rotation in SO(d) and v is a translation. The transformation parameters are optimized on a small transfer set using CMA-ES over the Lie algebra representation of SO(d). The method is evaluated on BBOB synthetic benchmarks and on four real-world transfer learning problems (porkchop plots, SARCOS robot arm torques, vehicle dynamics, and MarioGAN). The reported results show that the transferred RFR often outperforms an RFR trained from scratch when the transfer set is small, especially on BBOB, with notable exceptions on several BBOB functions and on the vehicle dynamics benchmark.","tokens_in":41024,"tokens_out":7397,"duration_ms":71285,"significance":"If the central claim holds, the paper extends affine-domain transfer learning, previously limited to differentiable surrogates such as Gaussian processes, to non-differentiable random forest surrogates. This is a useful practical contribution because random forests are common in low-data engineering settings. The paper is honest in reporting failure cases, and it provides ten repetitions, statistical significance testing, and a Zenodo repository for reproducibility. However, the strongest evidence is obtained on synthetic BBOB tasks where the affine assumption is enforced by construction. The real-world benchmarks are presented as validation but do not verify the affine symmetry, and at least one benchmark appears to violate it. The significance of the real-world results is therefore conditional on an unmeasured assumption.","major_comments":[{"comment":"The synthetic BBOB targets are constructed by randomly generating rotation matrices and translation vectors applied to the source function, so the affine symmetry f_T(x) = f_S(Wx+v) holds exactly by construction. The positive results in Section V.A therefore validate only that CMA-ES can recover the transformation under the method's own assumption; they provide no evidence that the assumption is plausible for any real-world task. The real-world benchmarks in Section V.B are the only external evidence, yet no diagnostic is reported that checks whether any source-target pair actually satisfies the affine relation. This gap is load-bearing for the title's and abstract's claim of transferring 'across synthetic and real-world benchmarks' and for the conclusion that the method offers 'significant practical advantages.' The authors should either add a validation analysis for the real-world benchmarks (e.g., fit the transformation on a held-out subset and measure residual SMAPE against a no-transfer baseline, or inspect the recovered W and v for consistency with domain knowledge), or explicitly reclassify the real-world results as exploratory evidence that does not establish the affine assumption.","section":"Section IV.A, V.B"},{"comment":"For the SARCOS robot arm benchmark, the source and target functions share the same 21-dimensional input space (joint positions, velocities, accelerations) and differ only in which joint's torque is predicted. Under an exact input-domain affine symmetry, the only admissible transformation is W=I and v=0, making the transfer equivalent to using the source model without adaptation. The mixed results, most notably torque7→torque1 losing to scratch-trained RFR at every transfer size, are therefore unlikely to be explained by an input-domain affine shift. The paper should either justify why this benchmark is a meaningful test of the affine domain transformation assumption, or report and analyze the optimized W and v to show what structure the method actually exploits. Without this, the robot arm experiment does not support the paper's central claim.","section":"Section IV.B.2, Fig. 4"},{"comment":"The paper uses the term 'affine transformation' but restricts W to the special orthogonal group SO(d), which together with a translation vector v gives only rigid motions (rotations and translations), not general affine transformations that include scaling, shear, and reflection. Real-world surrogate modeling tasks often have different unit scales, input ranges, or parameterizations between source and target, so the restriction to SO(d) may be exactly the wrong symmetry for those tasks. The conclusion acknowledges nonlinear domain changes in Vehicle Dynamics but does not acknowledge the scaling/reflection limitation. At minimum, the authors should explicitly state that the method handles only rotation-translation shifts, and ideally they should compare against a version that allows a general linear map (e.g., full GL(d)) to determine whether the SO(d) restriction is a practical bottleneck.","section":"Section III"}],"minor_comments":[{"comment":"The text contains a typo: 'instacne' should be 'instance' in the sentence 'The results underscore the unique difficulty of transferring from instacne3 to other instances.'","section":"Section V.B.3"},{"comment":"The Introduction uses the phrase 'non-differential models' and 'differential models' where 'non-differentiable models' and 'differentiable models' are meant; this terminology should be corrected for consistency with Section III.","section":"Section I"},{"comment":"Line 12 of Algorithm 1 reads 'Update m, C and σ with and {xi}λ i=1 and {yi}λ i=1'; the word 'and' appears to be a typo, and the sentence should be rephrased to 'Update m, C and σ using {xi} and {yi}'.","section":"Algorithm 1"},{"comment":"The displayed equation for the loss L(v, W) is rendered as 'L(v, W) = n−1 T P ...', which is hard to read; it should be typeset as n_T^{-1} \\sum ... for clarity.","section":"Section III"},{"comment":"The caption of Fig. 2 does not define the color scale; please state explicitly that red indicates positive percentage difference (transfer better) and blue indicates negative, since the reader cannot infer the mapping from the grayscale version.","section":"Fig. 2"},{"comment":"The comparison with transferred GPR results from [13] would be more informative if the experimental settings (transfer set sizes, dimensions, number of repetitions) were aligned; as reported, the comparison is only qualitative and should be labeled as such.","section":"Section V.A"}],"recommendation":"major_revision","confidential_remarks":"The paper's main gap is that it validates the method under its own affine assumption on synthetic data and then applies it to real-world benchmarks without testing the assumption. The SARCOS benchmark is particularly problematic because the shared input space makes a nontrivial rotation/translation mapping implausible. I believe the central idea is sound and the experimental study is honest, but the claims need to be scaled back or the real-world validation needs to be strengthened with a diagnostic of the affine relation. This is fixable within the scope of the manuscript, hence major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper does one thing well: it takes the affine-transfer idea from the authors' GECCO 2024 paper and makes it work for random forests by optimizing the rotation on the Lie algebra so(d) with CMA-ES. That is a real, if incremental, contribution. The experimental setup is also good: independent test sets, ten repetitions, Kruskal-Wallis and Dunn tests, and a Zenodo repository with code. They report failures honestly—F16 and F21–F24 on BBOB, several vehicle dynamics transfers, and the torque7→torque1 case where transfer loses at every data size. The conclusion is appropriately cautious about nonlinear shifts, and the Vehicle Dynamics task is flagged as likely violating the affine assumption.\n\nThe soft spot is in the real-world validation. On BBOB, the target functions are constructed by applying a random affine transformation to each source, so the assumption is true by construction. Those results only show that CMA-ES can recover a rotation and translation when one exists. The real-world benchmarks never check whether the source–target pairs actually satisfy f_T(x) = f_S(Wx+v). For the robot arm, the input domains are the same and the outputs are different physical torques; for porkchop plots, different launch windows are not known to be affine-related; and the authors concede Vehicle Dynamics violates the assumption. The mixed outcomes—torque1→torque3 consistently good, torque7→torque1 consistently bad—are reported but not explained. There is also no comparison to existing random forest transfer baselines (Segev et al. [14]), only three of the 28 MarioGAN functions are used, and the SMAC3 tuning for vehicle dynamics is underspecified. These are not fatal flaws; they are gaps in validation and comparison.\n\nOverall, the paper is honest, the method is plausible, and the BBOB result is solid under its own premise. The title-level claim of transferring \"across synthetic and real-world benchmarks\" is conditional on an assumption that is never tested on the real data. That makes it a paper for the surrogate-modeling community: readers who want a concrete recipe for adapting an RFR under affine shifts will get value, but anyone hoping for a general transfer method should be careful.\n\nI would send this to review with major revisions. The key asks would be a diagnostic for the affine assumption on real-world tasks, a baseline comparison with Segev et al., and a more complete description of the tuning setup. The paper deserves referee time because the code, honesty, and reproducible experiments give it a solid floor, even if the ceiling needs work.","headline":"A clean, reproducible extension of affine transfer learning to random forests, but the real-world benchmarks never test the affine assumption, so the title overclaims.","tokens_in":41602,"tokens_out":1765,"would_cite":true,"duration_ms":18819,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A random forest surrogate trained on one function can be reused on an unknown rotated and translated version of the same function using only tens of target samples.","keywords":["transfer learning","surrogate models","random forest regression","affine transformation","domain shift","CMA-ES","Black-Box Optimization Benchmark","real-world regression"],"falsifier":"Take a source-target pair generated by a scaling such as $f_T(x)=f_S(2x)$ or $f_S(x/2)$, so the domain map is not in $\\mathrm{SO}(d)$, and run the transfer with 50 samples: if the transferred model does not beat scratch training on most instances, the rotation-only affine assumption is the limiting condition.","tokens_in":40592,"feed_emoji":"🔄","tokens_out":7399,"duration_ms":63314,"temperature":0.7,"pith_summary":"The paper tries to establish that a random forest regression model trained on one function can be transferred to a second function whose input domain is an unknown rotation and translation of the first, by fitting that affine map on a small set of target samples. The fitting is done with the covariance matrix adaptation evolution strategy, which optimizes the rotation through its skew-symmetric matrix representation. The reported evidence is that the transferred random forest beats a model trained from scratch with only 50 to 100 target samples across most functions of the Black-Box Optimization Benchmark and in three of the four real-world tasks. If correct, this gives a data-efficient route to surrogate modeling for expensive simulations, since the expensive source model is trained once and then reused.","feed_headline":"Tens of samples transfer a random forest to a rotated task","feed_subtitle":"Pre-trained surrogates outdo scratch models on affine-shifted tasks, cutting the data needed for expensive real-world simulation.","key_machinery":"The machinery is an affine reparameterization of the source model, $\\hat{f}_T(x)=\\hat{f}_S(Wx+v)$, with the rotation searched in the Lie algebra $\\mathfrak{so}(d)$: a vector of $d(d-1)/2$ real numbers is packed into a skew-symmetric matrix $A$, and the rotation is $W=\\exp(A)$. This converts a constrained search over the rotation group into an unconstrained Euclidean search that CMA-ES can handle, while the translation $v$ is searched directly. The same transfer loss is used as in the differentiable case, but the optimizer no longer requires gradients.","core_discovery":"The paper's central claim is that non-differentiable surrogates, specifically random forest regression, can be transferred under an affine domain shift by solving $\\min_{W \\in \\mathrm{SO}(d), v \\in \\mathbb{R}^d} \\frac{1}{|T|}\\sum_{x \\in T}(\\hat{f}_S(Wx+v) - f_T(x))^2$, where $\\hat{f}_S$ is the source-trained model and $T$ is a small transfer set from the target. Because random forests have no gradients, the paper replaces the Riemannian gradient optimizer used for Gaussian process regression with CMA-ES, representing rotations in the Lie algebra $\\mathfrak{so}(d)$ of skew-symmetric matrices and recovering $W$ by the matrix exponential. It reports that the transferred random forest achieves lower symmetric mean absolute percentage error than a scratch-trained model with as few as 50 to 100 transfer samples, especially in higher dimensions, and that this advantage shrinks as transfer data grows.","pith_inferences":["An extension not tested in the paper: because CMA-ES only needs function evaluations, the same wrapper should apply to other non-differentiable surrogates, such as k-nearest-neighbor models or decision tree ensembles beyond random forests.","The paper's failure on vehicle dynamics suggests a cheap diagnostic: fit the affine transfer and compare its loss on the transfer set to a scratch model's loss; if the transfer loss does not drop, the affine assumption is likely violated.","A testable improvement would be to allow scaling or reflection in the affine map, for example $W \\in \\mathrm{GL}(d)$ or orthogonal matrices with determinant $-1$, since the paper restricts to rotations and lists nonlinear warping as future work.","If confirmed, the method changes the economics of simulation-driven design: one expensive training run can serve many related configurations, with only a tiny target sample needed for adaptation."],"forward_implications":["At transfer sizes of 50 to 100 samples, the transferred random forest is reported to beat the scratch-trained model on most BBOB functions, with the gap larger in ten dimensions than in two dimensions.","The benefit of transfer shrinks as the transfer set grows, with scratch-trained models catching up once hundreds of target samples are available.","Transfer is ineffective when the source surrogate is a poor fit to its own source function, as with F23, and is limited on complex multimodal functions such as F16 and F21 through F24.","Real-world porkchop plot, robot arm, and MarioGAN tasks show transferred random forests winning in low-data regimes, while the vehicle dynamics benchmark largely violates the affine assumption and transfer fails there."],"supporting_citations":[{"why":"Defines the affine-transfer objective and the Gaussian process baseline that this paper extends to random forests.","marker":"[13]"},{"why":"Provides the CMA-ES update equations used to optimize the translation and rotation parameters.","marker":"[22]"},{"why":"Defines the BBOB functions and their properties used to construct the synthetic source and target tasks.","marker":"[27]"},{"why":"Supplies the Lambert problem solver that generates the porkchop plot energy functions.","marker":"[28]"},{"why":"Supplies the SARCOS robot arm data used as one of the real-world transfer tasks.","marker":"[31]"},{"why":"Supplies the vehicle dynamics benchmark whose instances serve as source and target functions.","marker":"[32]"},{"why":"Supplies the MarioGAN fitness functions F1, F5, and F6 used as game-benchmark transfer tasks.","marker":"[33]"},{"why":"Defines SMAPE, the scale-invariant error metric used to compare model accuracy across tasks.","marker":"[34]"}],"fun_headline_variants":["Random forests transfer across rotated tasks with just 50 samples","Affine transfer learning adapts random forests on a tiny dataset","CMA-ES fits affine shifts so random forests transfer to new domains","Few transfer points turn a random forest into a new-task surrogate","Random forest transfer: 50 points beat training from scratch"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes the target function is exactly the source function composed with a rotation and translation of its input domain, and the real-world benchmarks do not verify that relation.","fun_headline_variants_meta":{"raw":{"variants":["Random forests transfer across rotated tasks with just 50 samples","Affine transfer learning adapts random forests on a tiny dataset","CMA-ES fits affine shifts so random forests transfer to new domains","Few transfer points turn a random forest into a new-task surrogate","Random forest transfer: 50 points beat training from scratch"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000235,"raw_usage":{"total_tokens":1501,"prompt_tokens":950,"completion_tokens":551,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":566,"completion_tokens_details":{"reasoning_tokens":466}},"tokens_in":566,"tokens_out":551,"duration_ms":5662,"temperature":1.0,"reasoning_tokens":466,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T15:28:45.838569+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a source-target pair generated by a scaling such as $f_T(x)=f_S(2x)$ or $f_S(x/2)$, so the domain map is not in $\\mathrm{SO}(d)$, and run the transfer with 50 samples: if the transferred model does not beat scratch training on most instances, the rotation-only affine assumption is the limiting condition.","supporting_citations":[{"cited_title":"Revisiting lambert’s problem,","cited_arxiv_id":null,"evidence_quote":"Supplies the Lambert problem solver that generates the porkchop plot energy functions."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the SARCOS robot arm data used as one of the real-world transfer tasks."},{"cited_title":"Real-world optimization benchmark from vehicle dynamics: Specification of problems in 2d and methodology for transferring (meta-)optimized algorithm parameters,","cited_arxiv_id":null,"evidence_quote":"Supplies the vehicle dynamics benchmark whose instances serve as source and target functions."}],"review_version":1}