{"id":"4df97511-4f63-4786-b800-cb15cc5eb55c","arxiv_id":"2501.03413","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A dataset of authentic, anonymized, linked ERP sales tables (2.3M item rows) with eight multiclass autocompletion targets and baseline evaluations.","lead":"SALT is a new dataset of over two million sales order records pulled from a real SAP ERP system, with four linked tables and eight prediction targets for sales autocompletion. It gives table-learning researchers a rare look at authentic enterprise data with foreign-key structure and temporal splits.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The Sec. A.4 filter keeping only fully processed orders may remove exactly the incomplete orders where autocompletion is needed, so SALT's real-enterprise representativeness is asserted but not demonstrated.","rationale":"The reader's weakest assumption correctly identifies the filtering and anonymization pipeline as the key risk to distributional representativeness. I agree that the 'fully processed orders only' filter is the most serious concern: it creates a train/inference mismatch for the autocompletion task and is supported by no empirical check in the paper. I disagree slightly on anonymization: the paper states categorical variables were encrypted, and bijective encryption would preserve univariate and join distributions by construction, so the filtering step is the more credible threat. The proposed concrete test would settle whether the excluded population differs materially from the included one, and the request for an auditable release artifact is a minimal condition for propagating SALT as a benchmark. Since the reader already conditioned the verdict on addressing these points, my read does not move the verdict; it sharpens the primary condition.","tokens_in":8727,"tokens_out":4344,"duration_ms":44543,"concrete_test":"Request that the authors report the number and percentage of orders excluded by the 'fully processed' filter, and rerun the eight-target MRR evaluation on a variant that includes incomplete orders (or on a held-out set of incomplete orders not used for training). Compute a distribution-shift measure, e.g., population stability index or chi-square statistic, for each target field between included and excluded orders. If any target shifts beyond a pre-registered threshold, the distribution-preservation claim fails; SALT should then be described as a fully-processed-order benchmark rather than a general enterprise autocompletion dataset. In addition, provide a commit hash and SHA-256 checksums for the released data so the exact filtered artifact can be independently audited.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that SALT is a curated ERP dataset that 'demonstrat[es] the characteristics of data within actual enterprise systems' (Sec. 4). For the sales-autocompletion task, the model is supposed to predict fields that are typically missing during sales-order creation. However, Sec. A.4 states: 'Only orders which have been fully processed are included, that is, sales orders which have gone through the entire business process.' This filter removes incomplete, rejected, or cancelled orders from the training and evaluation data. At inference time, autocompletion is needed precisely for orders that are not yet complete, so the model never observes the partial-order state it is meant to assist. Moreover, the target distributions are conditioned on final completion status; if excluded orders have systematically different target values or missingness patterns, the benchmark overstates real-world predictive performance. The paper's assertion in Sec. A.1 that the privacy protocol 'was designed to preclude any likelihood of data distribution distortion or introduction of bias' addresses encryption and sanitization, not the filtering step, and is an assertion rather than a measurement. Since the filtering decision directly affects whether the dataset represents real enterprise autocompletion, this is the most load-bearing unverified assumption. Secondary issues such as missing error bars and the 'seven critical variables' wording (Sec. 2 lists eight target fields) are addressable but do not threaten the central claim as directly.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces SALT, a publicly released dataset of four linked tables (sales documents, sales document items, customers, and addresses) extracted from a single enterprise ERP system, containing about 2.3 million sales-order-item rows spanning 2018–2020. The dataset is designed for table representation learning and is paired with a sales-autocompletion task: predict eight target fields (e.g., sales office, plant, shipping point, incoterms) that are typically filled in during later stages of sales-order creation. The authors describe the schema, statistics, and an SQL join procedure, and they report baseline evaluations using random/majority classifiers, XGBoost, LightGBM, CatBoost, CARTE, AutoGluon, and GraphSAGE, measured by mean reciprocal rank (MRR). The central claim is that SALT constitutes a realistic, anonymized, multi-table enterprise dataset with a well-defined task and temporal splits.","tokens_in":8929,"tokens_out":4536,"duration_ms":41055,"significance":"If the dataset is as described, it fills a real gap: most existing table corpora are web-scraped and lack the inter-table foreign-key structure and business semantics of operational ERP data. The paper ships a clear schema, a documented join query, per-field statistics, and baseline code, which are valuable for benchmarking tabular and relational learning methods. The inclusion of a real-world enterprise source and a concrete autocompletion task with temporal splits makes SALT potentially useful for the table-representation-learning community. However, the value of the contribution depends on the dataset faithfully representing the enterprise context in which autocompletion operates, and on the baseline comparisons being statistically meaningful.","major_comments":[{"comment":"The filtering step in Sec. A.4 keeps only fully processed sales orders ('sales orders which have gone through the entire business process'), and Sec. 2 motivates the task as predicting fields that are 'typically missing in sales orders' during order creation. Because the evaluation set contains only completed orders, the model never sees the partial-order state in which autocompletion would actually be used, and the target distributions are conditioned on final completion status. This is a load-bearing mismatch: the claim in Sec. 4 that SALT 'demonstrat[es] the characteristics of data within actual enterprise systems' is not established for the autocompletion use case. Please address this by (a) providing evidence that incomplete orders follow the same conditional target distributions, (b) including an analysis or separate split on incomplete orders if available, or (c) re-framing the task as end-of-process field prediction and explicitly documenting the limitation.","section":"Sec. A.4 and Sec. 2 (Task)"},{"comment":"All baseline results in Table 2 are reported as point estimates from a single run, with no error bars, standard deviations, or multiple seeds. Given the small margins separating the top methods (CARTE average MRR 0.79, AutoGluon 0.77, XGBoost 0.76) and the strong class imbalance noted in Sec. 2, the reported ranking may be within run-to-run variation. Please report means and standard deviations across at least five independent seeded runs for every method, or otherwise quantify the uncertainty of the comparisons. This is necessary to support the claim that CARTE performs best on SALT with a margin of +0.02.","section":"Sec. 3 (Experiments & Results), Table 2"},{"comment":"The statement that the privacy-sanitization protocol 'was designed to preclude any likelihood of data distribution distortion or introduction of bias' is an assertion rather than a measured property. Deterministic encryption or hashing can in principle alter cardinalities, introduce collisions, or shift value distributions, and the paper provides no before/after comparison or detailed description of the encryption scheme. Because the representativeness of SALT is a central claim, please provide empirical checks (e.g., distributional comparisons of key fields before and after anonymization) or, at minimum, a precise account of the type of encryption used and a discussion of its known properties and limitations.","section":"Sec. A.1 (Privacy & Anonymization)"}],"minor_comments":[{"comment":"The text says the model performs 'multiclass classification on seven critical variables,' but Table 2 lists eight target tasks and the bullet list contains eight field names (the two Incoterms fields are counted separately in the table). Please reconcile the count or clarify that the two Incoterms fields are grouped as one 'variable.'","section":"Sec. 2 (Task)"},{"comment":"The text states 'Table 8 provides statistics on the target fields' and then 'Table 8 provides detailed statistics of the table fields'; the second reference should presumably be to a different table (Table 9?), and the manuscript's actual Table 8 is titled 'Field statistics.' Please fix the table numbering and cross-references.","section":"Sec. A.3"},{"comment":"The caption reads 'Atomatic data type'; this should be 'Atomic data type.'","section":"Table 1 caption"},{"comment":"The paper states that the validation set was used for early stopping and that no hyperparameter tuning was performed, but it does not specify how early stopping was implemented (e.g., monitored metric, patience) or whether the validation set was also used for any model selection. Please clarify this procedure.","section":"Sec. 3 (Experiments & Results)"},{"comment":"The paper reports missing-value percentages for target fields (up to 1.09% for CustomerPaymentTerms), but it does not describe how rows with missing target labels are treated in training and evaluation. Please state whether such rows are dropped, imputed, or handled as a special class.","section":"Sec. 2 and Table 7"}],"recommendation":"major_revision","confidential_remarks":"The dataset appears to be a genuine and potentially useful contribution, and the authors provide reproducible code and clear schema documentation. The main risk is that the filtering to fully processed orders and the lack of uncertainty quantification make the benchmark claims weaker than presented. I would encourage the editor to require the revisions outlined in the major comments before considering acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: SALT is a genuinely useful dataset release — real ERP sales tables, linked by foreign keys, with a defined autocompletion task and temporal splits — and it is documented clearly enough to be a serious benchmark candidate. The baselines are off-the-shelf, but that is fine for a dataset paper.\n\nWhat is new is the dataset itself. The related-work comparison against WebTables, GitTables, TabLib, LakeBench, and SalesDB is fair, and the gap is real: no prior public dataset gives authentic linked ERP tables at this scale for table representation learning. The schemas, join SQL, row counts, missing-value rates, and entropy statistics are concrete and useful. The basic artifact appears to exist and be usable.\n\nWhere it is soft: the evaluation is a single run with no error bars or seeded repetitions. For a benchmark, MRR numbers without variance are weak but not disqualifying. More importantly, the Sec. A.4 filter keeps only “fully processed” orders. That filter could bias an autocompletion benchmark: the model is trained and tested on completed orders, but at inference autocompletion is needed for incomplete ones. The paper asserts the filtering and anonymization do not distort distributions, but provides no measurement. That is a genuine limitation, not a manufactured one, and it is addressable: report statistics before and after filtering, and ideally include incomplete orders in a held-out set. A smaller issue: the text says “seven critical variables” but names eight targets; minor. I would also want the GitHub repo to state a license and provide checksums or a commit hash before the data is propagated as a benchmark.\n\nThe citation pattern looks appropriate. This is a dataset contribution, not a methodological one, and it acknowledges the synthetic SalesDB and related corpora. Self-citation is not an issue here.\n\nBottom line: SALT deserves a serious referee at a datasets-and-benchmarks venue. The central claim — a real linked ERP dataset exists and is usable for table representation learning — holds. The autocompletion realism caveat should be fixed or explicitly scoped before the dataset is treated as a benchmark for enterprise autocompletion.","headline":"SALT is a genuinely useful linked-ERP dataset release with a clearly documented schema and task; the main caveat is that filtering to fully processed orders may undercut the autocompletion realism, though it is addressable.","tokens_in":9540,"tokens_out":1895,"would_cite":true,"duration_ms":19126,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A linked ERP sales dataset for table representation learning, with a field autocompletion task.","keywords":["linked business tables","ERP dataset","sales autocompletion","table representation learning","multi-table data","tabular benchmarks","data drift","enterprise data"],"falsifier":"Take a raw, unfiltered ERP sales-order snapshot from the same system and period, and compare the value distributions and target-field missing rates with SALT; if the excluded incomplete orders are disproportionately the ones lacking the target fields, then SALT under-represents the autocompletion scenario and the claim that it reflects actual enterprise business contexts loses support.","tokens_in":8488,"feed_emoji":"📊","tokens_out":3786,"duration_ms":35742,"temperature":0.7,"pith_summary":"This paper introduces SALT, a curated dataset sourced from a real Enterprise Resource Planning (ERP) system, designed to support research on table representation learning for linked business tables. The authors argue that existing table corpora come from web scraping and do not capture the complexity of multi-table enterprise data, and that SALT fills this gap by offering four interconnected tables covering sales documents, items, customers, and addresses. The dataset is accompanied by a sales autocompletion task: predicting eight target fields that are typically populated late in the sales order creation process. If the dataset is representative, it gives researchers a public, anonymized benchmark for studying how tabular models handle real-world linked business data, where class imbalance, high cardinality, noise, and data drift are the norm.","feed_headline":"Real ERP sales tables: a benchmark for autocompletion","feed_subtitle":"SALT links 2.3M sales-order rows across four tables, with a 7-field autocompletion task for tabular models.","key_machinery":"The key object is the dataset itself: four tables (I_SalesDocument, I_SalesDocumentItem, I_Customer, I_AddrOrgNamePostalAddress) linked by primary and foreign keys, which are filtered to sales orders only, anonymized through encryption of categorical variables, and joined into a flat table with one row per sales order item. The task is multiclass classification on seven critical sales-order fields (SalesOffice, SalesGroup, CustomerPaymentTerms, ShippingCondition, ShippingPoint, Plant, and IncotermsClassification), which are marked as target fields because they are often populated later in the order-creation workflow. This setup lets a single model trained on the joined table directly address the business problem of sales autocompletion.","core_discovery":"The central claim is that SALT constitutes a realistic, multi-table enterprise dataset: it contains 500,908 sales orders, 2,319,944 sales order items, 139,611 customers, and 1,788,887 addresses, spanning transactions from January 2018 to December 2020, with temporal splits for validation and test. The paper further claims that the dataset demonstrates the characteristics of actual enterprise systems, including skewed class distributions, long-tailed categorical fields, input noise from manual data entry, and potential data drift over time. Baseline experiments show that most current tabular models handle the prediction tasks reasonably well, with CARTE achieving the best average mean reciprocal rank of 0.79, followed by AutoGluon at 0.77 and XGBoost at 0.76, while majority-class baselines perform far worse on most targets.","pith_inferences":["A likely follow-up is to benchmark table foundation models on the original four-table schema rather than the flat join, since the paper's flattening discards the explicit relational structure that distinguishes SALT from web-scraped corpora.","The exclusion of incomplete sales orders may remove exactly the cases where autocompletion is most needed, so the dataset's difficulty could be understated compared with live ERP conditions.","The reported data drift and 2018–2020 time span make SALT a natural candidate for studying concept drift in enterprise tabular classification, a direction the paper mentions but does not explore.","The encryption-based anonymization preserves value cardinalities but not semantic meaning, so models trained on SALT cannot directly transfer to other ERP systems without re-encoding; this suggests the dataset is best used for method development rather than for pretraining deployable models."],"forward_implications":["Researchers gain a public, anonymized multi-table ERP dataset that can serve as a benchmark for table representation learning on linked business data, complementing web-scraped corpora like WebTables and GitTables.","Baseline results indicate that gradient-boosted trees and AutoML pipelines already reach high accuracy on some targets (e.g., Sales Office near 0.99 MRR), while high-cardinality fields like Sales Group remain challenging, so the dataset can separate easy saturation from hard prediction.","The temporal splits allow evaluation under realistic data drift, where category definitions may shift over time, providing a testbed for drift-robust tabular models.","The flat-table format with repeated header-level targets enables multi-task learning where a single model predicts all target fields simultaneously, as the paper explicitly designed for.","Because the raw table structure is also released, methods that exploit foreign-key linkages natively, such as graph-based approaches, can be compared against flat-table baselines."],"supporting_citations":[{"why":"Supplies the argument that existing table datasets from the web do not represent the complexity of enterprise database tables, motivating the need for SALT.","marker":"(Bodensohn et al., 2024)"},{"why":"Provides GitTables, a large-scale web-scraped table corpus used as a comparison point in Table 1 to highlight SALT's enterprise origin.","marker":"(Hulsebos et al., 2023)"},{"why":"Provides WebTables, another web-derived table corpus whose atomic data type distribution is compared with SALT in Table 1.","marker":"(Lehmberg et al., 2016)"},{"why":"Provides TabLib, a 627-million-table corpus that represents the current scale of web table collections, again used for comparison in Table 1.","marker":"(Eggert et al., 2023)"},{"why":"XGBoost is one of the baseline tabular models evaluated in Table 2, establishing a strong tree-based comparison point.","marker":"(Chen and Guestrin, 2016)"},{"why":"CARTE is the best-performing baseline on SALT in Table 2, making it a key reference for the paper's evaluation claim.","marker":"(Kim et al., 2024)"},{"why":"AutoGluon is the second-best baseline in Table 2, providing an AutoML comparison that shows how off-the-shelf pipelines fare on the dataset.","marker":"(Erickson et al., 2020)"}],"fun_headline_variants":["SALT: 2.3M sales orders for tabular autocompletion","Linked ERP tables: a real benchmark for autocompletion","SALT dataset: 500K orders, 2.3M items for autocompletion","ERP sales data for table representation learning","SALT: authentic business tables for autocompletion"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The filtering and anonymization pipeline preserves the distributional properties of real ERP data, including the missing-value patterns in the target fields that the autocompletion task is designed to predict.","fun_headline_variants_meta":{"raw":{"variants":["SALT: 2.3M sales orders for tabular autocompletion","Linked ERP tables: a real benchmark for autocompletion","SALT dataset: 500K orders, 2.3M items for autocompletion","ERP sales data for table representation learning","SALT: authentic business tables for autocompletion"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000788,"raw_usage":{"total_tokens":3431,"prompt_tokens":857,"completion_tokens":2574,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":473,"completion_tokens_details":{"reasoning_tokens":2485}},"tokens_in":473,"tokens_out":2574,"duration_ms":16929,"temperature":1.0,"reasoning_tokens":2485,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T21:52:07.076165+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a raw, unfiltered ERP sales-order snapshot from the same system and period, and compare the value distributions and target-field missing rates with SALT; if the excluded incomplete orders are disproportionately the ones lacking the target fields, then SALT under-represents the autocompletion scenario and the claim that it reflects actual enterprise business contexts loses support.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the argument that existing table datasets from the web do not represent the complexity of enterprise database tables, motivating the need for SALT."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides GitTables, a large-scale web-scraped table corpus used as a comparison point in Table 1 to highlight SALT's enterprise origin."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"CARTE is the best-performing baseline on SALT in Table 2, making it a key reference for the paper's evaluation claim."}],"review_version":1}