{"id":"1f829d80-69e0-42c5-81ca-efe4075affb9","arxiv_id":"2608.06137","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"SkillTFM adapts tabular foundation models at deployment time by gating evidence-triggered repair skills, improving AUC by up to 0.142 on boundary-shifted tasks with zero observed harm.","lead":"This paper introduces SkillTFM, a training-free adaptation layer for tabular foundation models that applies repair skills only when runtime evidence supports them and a gate certifies them as safe, reporting AUC gains of 0.128 to 0.142 on synthetic boundary-shift benchmarks and a large MAE reduction on one electricity-price test date.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central risk is that Eq. (5) evidence z_T may not distinguish repairable boundary failures from deployments where a candidate repair would harm, so the zero-harm claim is only as strong as the similarity between guard/promotion tasks and deployment.","rationale":"The paper is best read as a proposal for a pluggable, validation-gated adaptation system, not as a new base model. The controlled experiments are internally consistent: Table 2A cleanly separates evidence-conditioned retrieval from the runtime certificate, Figure 3 shows fallback increasing as base-model AUC approaches saturation, and the cross-model transfer results indicate the skill state is not tied to a single backbone. These are real strengths and support conditional acceptance rather than rejection. The load-bearing weakness is the transfer of the evidence-to-repair mapping. The runtime certificate is the only safeguard between a promoted skill and deployment harm, and its decisions are driven by an underspecified z_T and risk/memory scores. The paper demonstrates that SkillTFM works on tasks generated by the same boundary families used for validation, but it does not demonstrate that z_T can separate a repairable instance from a harmful instance when the two share observable evidence. Eq. (3) is stated as a constraint over D_guard; it is enforced at promotion time, not proven as a runtime invariant. The zero-harm numbers are measured, not implied, and the threshold delta is never reported. Without code, data, or threshold values, a reader cannot determine whether the certificate's 'evidence sufficient / risk acceptable' condition has any discriminative power beyond the suite. The electricity case is the only out-of-family test and it is small (288 points), which limits the strength of the transfer evidence. The twin test proposed above would settle whether the concern lands: if the same z_T can be realized in tasks with opposite repair effects and SkillTFM cannot separate them, the central claim about safe training-free adaptation is not established for novel deployment conditions; if it can separate them, the concern is resolved. I therefore keep the reader's conditional verdict.","tokens_in":14352,"tokens_out":7579,"duration_ms":146499,"concrete_test":"Release the evidence extractor, skill bank, thresholds, and exact promotion settings, then run a counterfactual twin test: for each promoted repair operator o, construct a deployment task whose z_T matches a beneficial task (same missingness pattern, prediction bias, trend/periodicity, and probe sensitivity) but whose target rule is perturbed so that o decreases AUC. Score with labels only after SkillTFM decides, and compare fallback and harm rates on beneficial versus harmful twins. If SkillTFM does not fall back on the harmful twin at a materially higher rate, Eq. (5) lacks the discriminative signal required by the zero-harm claim; as a secondary check, rerun Table 1 under leave-one-boundary-family-out promotion for T_sel and T_guard and report whether harm remains zero on the held-out family.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim requires that the runtime evidence vector z_T = E(T,M) (Eq. 5) carries enough information for the gating policy to distinguish repairable boundary failures from deployments where a candidate repair would harm. This is the only link between validation-gated promotion (Eq. 10), which uses labeled T_sel and T_guard tasks, and deployment, where runtime evidence is explicitly not allowed to use test labels or final-heldout feedback. The certificate in Eq. (7) ranks candidates using r_sup, r_hist, -r_risk, -r_mem, and -cost, and then applies 'evidence sufficient / risk acceptable / memory permits' checks, but the paper does not define the feature representation of z_T, the functional forms of these scores, or the thresholds used by the certificate. Because a candidate is promoted only when it improves T_sel and does not harm T_guard, a skill can still be unsafe on a deployment task that produces the same observable evidence but a different underlying mechanism. Table 1's zero harm over ten seeds is therefore evidence about tasks drawn from the same boundary-family suite used to build or validate the skills; it is not a runtime guarantee for arbitrary tasks. Table 2B illustrates the issue: static SkillTFM falls back 100% on the nonlinear boundary, and after evolution the AUC increases to 0.898 only on tasks from that same nonlinear family. The conclusion's stated limitation, that uncovered boundary families may still cause failure or harm, is precisely where the untested assumption lives.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"SkillTFM proposes a training-free, external 'skill state' that adapts tabular foundation models (TFMs) to boundary shift conditions by extracting runtime evidence, retrieving candidate repair skills, and executing a repair only when a runtime certificate approves it; otherwise it falls back to the base model. A second loop, validation-gated skill evolution, allows candidate edits from traces or LLM proposers to enter the deployed skill state only after utility-improvement, no-harm, regression, and schema checks. The paper reports AUC improvements of 0.128-0.142 with zero observed harm on controlled boundary suites, a nonlinear-boundary improvement from 0.699 to 0.898 after evolution, cross-model and cross-optimizer transfer, and MAE reductions in real-world electricity-price forecasting while modifying only a fraction of predictions.","tokens_in":14703,"tokens_out":4487,"duration_ms":44504,"significance":"If the claims hold, SkillTFM offers a plausible and useful paradigm: adapting frozen TFMs through a pluggable, auditable skill state rather than parameter updates. The paper has real strengths: controlled boundary-family construction, a clear ablation separating evidence-conditioned retrieval from runtime certification (Table 2A), held-out evaluation on a later electricity-price release with explicit target-overlap avoidance (Table 4), and cross-model/cross-optimizer experiments that give evidence of generality. However, the central safety claim is only partially supported because the runtime evidence representation, certificate thresholds, and promotion-gate constants are never specified, and the zero-harm results are confined to tasks generated from the same boundary-family suite used for skill construction and validation. The claimed significance therefore depends on closing these specification gaps and carefully re-scoping the 'zero harm' language.","major_comments":[{"comment":"The paper never defines the feature representation of z_T = E(T,M), the functional forms of r_sup, r_hist, r_risk, r_mem, and cost in Eq. (7), or the thresholds used by the certificate's 'evidence sufficient / risk acceptable / memory permits' checks. Since the runtime certificate is the only deployment-time safeguard against harmful repair, this omission makes the zero-harm claim unverifiable; in particular, the paper does not show that z_T can distinguish a repairable boundary failure from a deployment where the same observable evidence would lead to harm. This is the central load-bearing link between labeled promotion/guard validation (Eqs. 8-10) and unlabeled runtime decisions, and it must be specified before the safety claim can be evaluated.","section":"§Methodology, 'Evidence-Conditioned Boundary Expansion', Eq. (5) and Eq. (7)"},{"comment":"The values of epsilon (promotion improvement threshold), alpha (allowed harm rate), and delta (tolerated degradation margin) are never reported for any experiment. Because Eq. (10) requires G_sel >= epsilon and H_sel <= alpha, and Eq. (3) defines the no-harm guarantee in terms of alpha and delta, the reader cannot assess whether the no-harm condition is tight or vacuous; the manuscript should report these values and, if they are chosen by the authors, describe the selection procedure and report sensitivity to them.","section":"§Experiments, Table 1 and §Methodology, Eqs. (3), (9), (10)"},{"comment":"Table 1's zero observed harm is demonstrated on tasks generated from the same controlled boundary-family construction protocol used to build and validate the skills; Table 2B shows the same limitation, because the nonlinear boundary is 'unsupported' only until evolution is run on tasks from that same nonlinear family. The conclusion acknowledges that uncovered boundary families may still cause failure or harm, but the abstract and Table 1 present zero harm without this qualification. The claims should be restated as zero observed harm on the tested boundary families, not as a general runtime guarantee.","section":"§Experiments, 'Q1: Boundary Adaptation and Stratification' and Conclusion"},{"comment":"Because T_sel and T_guard are used to decide promotion and the held-out evaluation tasks are drawn from the same boundary-family suite (new tasks, mixed boundaries, generator/severity transfer), the reported gains partly reflect model selection on tasks very similar to the selection tasks. The paper should report the number and construction of tasks in T_sel and T_guard, the performance gap between selection tasks and held-out tasks, and a clear statement of what generalization claim the experiments actually support.","section":"§Methodology, 'Validation-Gated Skill Evolution', Eqs. (8)-(10), and Table 1"}],"minor_comments":[{"comment":"The two panels are labeled 'A' and 'B' but their y-axes and exact quantities are not described in the caption; please add axis labels and a legend so the reader can see whether the plot shows AUC improvement, fallback rate, or both.","section":"Figure 3"},{"comment":"The header 'Model Val. Harm Decision' is ambiguous; clarify that 'Valid' refers to schema-valid edits and 'Harm' to the guard-task harm rate, and make the column alignment consistent across all rows.","section":"Table 3"},{"comment":"The task distribution D in Eq. (2) and the guard distribution D_guard in Eq. (3) are not precisely defined; please state whether D_guard is a subset of D and how both relate to the controlled boundary suites.","section":"§Methodology, Eqs. (2)-(3)"},{"comment":"Several references are formatted inconsistently (e.g., 'RUBIN 1976' appears in all caps, and some 2026 arXiv preprints lack archive identifiers); the reference list should be normalized to the journal style.","section":"References"},{"comment":"The text contains run-together words such as 'andmodeldevelopment' and 'Tothisend' in the abstract and introduction; these should be fixed in the camera-ready version.","section":"Abstract and Introduction"}],"recommendation":"major_revision","confidential_remarks":"The central idea is timely and the controlled experiments are well organized, but the paper currently resembles a systems proposal more than a fully verified method. The main barrier is not scope fit but technical specificity: the evidence extractor, certificate thresholds, and promotion-task construction need to be specified before the reader can audit the safety and generalization claims. I would encourage the editor to seek a revised version with these details in a supplement, and to ensure the abstract's 'zero harm' is aligned with what the experiments can actually support."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my take on SkillTFM. The paper builds a genuinely new piece of engineering: instead of fine-tuning tabular foundation models, it wraps them in an external skill state that decides whether to repair or fall back based on runtime evidence, and it grows that skill state through a validation gate. That's a real contribution, and the ablation in Table 2A is the strongest part—it shows both evidence-conditioned retrieval and the certificate matter, with harm appearing when you drop the certificate. The cross-backbone transfer and cross-optimizer portability are also nice, and the authors are honest about the limitation of uncovered boundary families.\n\nThe soft spots are mostly about evaluation completeness, not the core idea. The thresholds epsilon, alpha, and delta are never reported, so 'zero harm' is only meaningful if we know the allowed harm rate. They also don't compare against standard baselines like calibration or imputation; the gains might shrink next to a well-tuned preprocessing pipeline. The real-world electricity test is a single held-out date with 288 points, so the 83% large-error coverage is suggestive but not strong evidence. And the deeper concern—whether the runtime evidence vector z_T carries enough signal to know when a repair will help—is real, but it's a known limitation of any selective system, and the paper explicitly flags it. That doesn't sink the paper, but it does mean the 'zero observed harm' is guaranteed only on guard tasks drawn from the same boundary family, not on arbitrary deployment tasks.\n\nI'd want code and exact parameters before trusting the numbers, and the authors should add standard baselines and multiple evaluation dates. As it stands, the paper deserves a serious referee: the framework is coherent, the ablations are informative, and the novelty is clear. I'd send it out with a request for reproducibility materials and a tighter evaluation. I wouldn't cite it until the code and thresholds are public.","headline":"A coherent system paper on gated skill adaptation for TFMs, with solid ablations but incomplete evaluation and missing thresholds.","tokens_in":15238,"tokens_out":2139,"would_cite":false,"duration_ms":16426,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"SkillTFM claims tabular foundation models can adapt to distribution shifts and structural boundary failures purely through an external, gated skill state, with no parameter updates, and reports AUC gains of 0.128 to 0.142 with zero…","keywords":["tabular foundation models","training-free adaptation","skill bank","gated skill evolution","boundary shift","runtime certificate","selective repair","electricity price forecasting"],"falsifier":"Run SkillTFM on a boundary family whose evidence signature is identical to a family where repair helps, but where the repair action actually hurts test performance. If the runtime certificate still certifies the repair because evidence is sufficient, risk is acceptable, and memory does not block the route, the central claim fails; if it falls back or rejects instead, observe the resulting AUC difference to quantify the evidence-transfer gap.","tokens_in":1959,"feed_emoji":"🧩","tokens_out":2273,"duration_ms":65161,"temperature":0.7,"pith_summary":"SkillTFM claims that tabular foundation models can be adapted to distribution shift and other boundary failures without any parameter updates, by attaching an external skill state that decides when to repair the base prediction and when to fall back. The system extracts runtime-observable evidence from the task and the base model, activates candidate repair skills, and executes a repair only when a runtime certificate finds the evidence sufficient, the risk acceptable, and no prior failure route blocks the action. New boundary experiences are converted into reusable skills through a promotion gate that requires measurable improvement, low harm rate on guard tasks, and regression and schema checks. On controlled boundary suites the paper reports AUC gains of 0.128 to 0.142 with zero observed harm, and on a nonlinear boundary it reports raising AUC from 0.699 to 0.898 after skill evolution. If these results hold, SkillTFM offers a pluggable way to expand the reliable operating region of a tabular foundation model without touching its weights.","feed_headline":"SkillTFM lifts tabular-model AUC by up to 0.142 with zero harm","feed_subtitle":"A pluggable skill bank repairs boundary failures and falls back when evidence is weak, with zero observed harm.","key_machinery":"The central object is the external skill state S = (E, O, R, G, F, B, P), a structured repair policy held outside the base model. E extracts boundary evidence z_T from feature distribution, label-feature relation, prediction bias, trend/periodicity, missingness pattern, and probe sensitivity. O is a bank of verified repair skills, each declaring required evidence and contraindications. R ranks candidate repairs lexicographically by evidence support, historical reliability, risk, memory similarity, and cost. G and F implement risk guards and fallback rules. B and P store rejected routes and principles that constrain future activation, ranking, and certification. The runtime certificate is the gate that executes a repair only when evidence is sufficient, risk is acceptable, and memory does not block the route, otherwise falling back to the base model. The promotion gate admits candidate edits only when intended improvement is at least epsilon, harm rate on guard tasks is at most alpha, and regression and schema checks pass.","core_discovery":"The paper's central claim is that TFM boundary failure is not a global property of a task but a local, evidence-conditioned pattern, and that a verifiable external skill state can expand the model's usable boundary by choosing between certified repair and fallback. The skill state S = (E, O, R, G, F, B, P) couples evidence extraction, a verified repair skill bank, ranking rules, risk guards, fallback rules, rejected-route constraints, and principle memory. At runtime the evidence vector z_T = E(T, M) activates only those repair skills whose applicability conditions match and whose contraindications are absent, then a runtime certificate scans ranked candidates and executes the first one that passes evidence, risk, and memory checks; if none passes, the system returns the base prediction. Validation-gated skill evolution then admits a candidate edit only when it improves intended capability on selection tasks, keeps harm on guard tasks below the tolerated margin, and passes regression and schema checks. The paper argues that this two-gate design is what lets SkillTFM improve AUC by 0.128 to 0.142 across held-out, mixed, and generator/severity transfer settings with zero observed harm, and convert an initially unsupported nonlinear boundary from 0.699 to 0.898 AUC.","pith_inferences":["The same evidence-conditioned certificate idea could be applied to other foundation-model domains, such as images, text, or time series, whenever a frozen predictor's failures leave observable signatures in input or prediction statistics; the paper's evidence channels are tabular-specific but the gating loop is not.","Because the zero-harm result is empirical rather than formal, a stronger test would be to run SkillTFM on boundary families whose evidence signatures are deliberately confounded with safe tasks, to see whether the certificate's evidence-to-repair transfer can be fooled.","The zero-harm result depends on the guard-task distribution and the tolerated margin delta; changing delta or adding adversarial guard tasks could expose the trade-off between fallback rate and improvement that the current headline numbers average over.","The token cost of skill evolution (roughly 19,000 to 35,000 tokens per optimizer alignment, and about 107,000 for the nonlinear skill) suggests a testable extension: measuring whether a smarter proposer reduces alignment tokens while preserving the promotion gate's harm bound."],"forward_implications":["SkillTFM can be attached to different tabular foundation models (TabPFN, TabICL, TabDPT, LimiX) with the same external skill state, with held-out AUC improvements around 0.139 to 0.148 per model.","The gating policy keeps intervention selective: fallback rates stay between 0.425 and 0.463 on a benchmark with 60 percent boundary-affected samples, and observed harm remains zero.","Skill evolution can add a capability that does not exist in the initial skill bank: the nonlinear boundary rises from 0.699 to 0.898 AUC after a gated promotion.","In real electricity-price forecasting, SkillTFM reduces MAE from 53.02 to 25.16 across three cities while modifying only 38.2 percent of predictions and covering 83.4 percent of large-error points.","Different LLM optimizers can propose candidate edits through a shared interface, but the local promotion gate decides admission, so proposal is separated from deployment."],"supporting_citations":[{"why":"Supplies TabPFN, one of the four frozen tabular foundation models whose boundary failures SkillTFM repairs.","marker":"Hollmann et al. 2023, 2025"},{"why":"Supplies TabICL, another base tabular foundation model used in the cross-model transfer evaluation.","marker":"Qu et al. 2025"},{"why":"Supplies TabDPT, a scaled tabular foundation model used to test SkillTFM's portability.","marker":"Ma et al. 2026"},{"why":"Supplies LimiX, the base model for both the cross-model transfer test and the real electricity-price forecasting case.","marker":"Zhang et al. 2025"},{"why":"Provides the selective-prediction formulation that underlies the runtime certificate's reject-or-repair decision.","marker":"Geifman and El-Yaniv 2019"},{"why":"Foundations of selective classification, cited to motivate deciding whether a task provides enough evidence to support repair.","marker":"El-Yaniv and Wiener 2010"},{"why":"Reflective prompt-evolution work that emphasizes validating generated edits before deployment, informing the promotion gate.","marker":"Agrawal et al. 2026"}],"fun_headline_variants":["SkillTFM: training-free tabular boost, AUC +0.142, zero harm","Gated skill bank repairs tabular boundary failures, +0.142 AUC","SkillTFM lifts tabular foundation model AUC 0.142, no training","Verifiable skills: tabular AUC up 0.142, fallback when weak","SkillTFM: tabular boundary repair, +0.142 AUC, zero drop"],"cache_read_input_tokens":17280,"weakest_assumption_plain":"The load-bearing premise is that the system can tell from the task's observable features and the base model's own predictions when a repair will help and when it will harm, without ever seeing test labels.","fun_headline_variants_meta":{"raw":{"variants":["SkillTFM: training-free tabular boost, AUC +0.142, zero harm","Gated skill bank repairs tabular boundary failures, +0.142 AUC","SkillTFM lifts tabular foundation model AUC 0.142, no training","Verifiable skills: tabular AUC up 0.142, fallback when weak","SkillTFM: tabular boundary repair, +0.142 AUC, zero drop"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001058,"raw_usage":{"total_tokens":4487,"prompt_tokens":1038,"completion_tokens":3449,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":654,"completion_tokens_details":{"reasoning_tokens":3340}},"tokens_in":654,"tokens_out":3449,"duration_ms":16908,"temperature":1.0,"reasoning_tokens":3340,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T14:13:06.473078+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run SkillTFM on a boundary family whose evidence signature is identical to a family where repair helps, but where the repair action actually hurts test performance. If the runtime certificate still certifies the repair because evidence is sufficient, risk is acceptable, and memory does not block the route, the central claim fails; if it falls back or rejects instead, observe the resulting AUC difference to quantify the evidence-transfer gap.","supporting_citations":[],"review_version":1}