{"id":"03bc7d5d-f00e-43ad-bc17-44389b39d8e0","arxiv_id":"2508.14741","paper_version":1,"verdict":"ACCEPT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"low","formal_verification":"none","parameter_count":0,"one_line_summary":"A synthesis of data curation practices for trustworthy AI, framed around an actionable definition of trustworthiness and a decision tree.","lead":"This report offers practical guidance for data curation to make AI systems trustworthy, defining trustworthiness as optimizing for performance on the true distribution of inputs in deployment. It catalogs techniques for splitting, resampling, weighting, and cleaning data, with hands-on examples using open-source tools.","discovery_kind":"review","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Framework's central claim is conditional on knowing the true distribution; report offers no validation that elicited or sampled approximations suffice, so practical impact is unproven.","rationale":"The reader identified the same load-bearing assumption: that the true distribution can be known or approximated well enough to guide curation. My analysis agrees and sharpens it by noting the report's own concession (§1.4) that trustworthiness is limited by that knowledge, and by pointing to evidence in §7.1.2 that the principal technical tool (importance weighting) may not work in deep learning. This is a genuine soft spot because the entire decision tree (§4) routes through the mode of distributional knowledge—if that knowledge is absent or inaccurate, the recommended actions lose their grounding. I recommend CONDITIONAL rather than ACCEPT to signal that the report's central claim is a conditional framework: it should be accepted as a useful synthesis of techniques under the stated assumption, but not as an empirically validated guarantee. The proposed test would settle the practical significance of the concern by quantifying how sensitive the pipeline's benefit is to the quality of distributional knowledge.","tokens_in":41863,"tokens_out":7556,"duration_ms":95350,"concrete_test":"Use a benchmark with a known, realistic distribution shift (e.g., WILDS FMoW or iWildCam), treating the official test set as the true distribution. Construct approximations of the true distribution of varying quality: (a) an oracle representative sample drawn i.i.d. from the test set, (b) small or noisy samples, (c) SME-style parameter estimates with controlled error. Run the report's recommended pipeline (representative selection via rsw/MRS, then weighting) and compare the resulting model's test-set performance against standard ERM. If oracle-level knowledge does not consistently improve over ERM, or if performance degrades sharply with realistic noise/estimate error, the framework's central assumption fails to hold in practice.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The actionable definition of trustworthiness (§1.1) requires optimization on the 'true distribution' of deployed inputs. The report's methods for achieving this depend on knowledge of that distribution: either parametric estimates from SMEs (Ch. 3) or a representative sample (cases b–d, §4.1). Yet §1.4 concedes that 'demonstrable trustworthiness... is constrained by the extent to which the true distribution is known,' and that it is 'not assumed that the true distribution can be fully known and parameterized.' This means the framework's central claim is conditional on a premise it never validates. The report provides no empirical evidence that SME elicitation (e.g., via SHELF) or representative-sample selection (rsw, MRS) yields sufficiently accurate approximations to improve model performance on the true distribution. Moreover, §7.1.2 notes that importance weighting—the core technique for covariate shift—has been shown to be ineffective for deep networks under long training, undermining the main mechanism for aligning training with the true distribution in the most common modern setting. Without a demonstrated path to reliable distributional knowledge, the definition becomes untestable, and following the recommended pipeline does not guarantee trustworthiness.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The report defines a trustworthy AI-enabled system as one optimized for performance on the true distribution of inputs in the deployed environment. It then organizes the data-curation phase into a sequence of actions—eliciting domain knowledge, choosing among statistical tools via a decision tree, splitting data, resampling/reweighting, semantic typing, documenting information, correcting mischaracterized data, and handling pretrained models—with the goal of aligning development data with the deployed environment. The contributions are primarily synthetic and advisory: a set of 12 SME questions, a two-stage decision tree, a catalogue of open-source tools, and illustrative practitioner experiments on a computer-vision triage task and a PubMed NLP task. The report is explicit that the actionable definition is conditional on knowledge of the true distribution, and it cites both the promise and the known limitations of the recommended techniques.","tokens_in":42146,"tokens_out":6510,"duration_ms":86531,"significance":"If taken as a practitioner-oriented synthesis, the report is useful and largely sound. Its main value is not novelty but a coherent operational framework that connects scattered data-curation techniques to a single normative definition of trustworthiness, with reproducible code in the appendices and references to open-source implementations. The explicit acknowledgment in §1.4 that demonstrated trustworthiness is bounded by knowledge of the true distribution, and in §7.1.2 that importance weighting is ineffective for some deep-learning regimes, strengthens the report's credibility. The two practitioner experiments are illustrative rather than confirmatory, and the report should not be read as providing quantitative evidence for the framework's efficacy. Overall, the central framing is defensible as long as the conditional nature of the claims is kept prominent.","major_comments":[{"comment":"The entire decision tree is gated by four mutually exclusive assumptions (a)–(d) about what is known of the true distribution. The report itself states in §1.4 that the true distribution may not be fully known and that demonstrable trustworthiness is constrained by that knowledge. This is an honest limitation, and I do not regard it as a hidden flaw. However, the practical claim in the abstract—that the described steps 'can promote trustworthiness'—is stronger than what the framework can deliver without an explicit procedure for deciding which of (a)–(d) holds, or for auditing sensitivity when the chosen case is wrong. I recommend adding a short subsection on robustness and assumption-checking: for example, re-running the main curation choices under several plausible distributional assumptions and reporting the spread of validation performance. This would make the conditional premise ope","section":"§4.1, Figure 4.1, §1.4"},{"comment":"The report notes that importance-weighted empirical risk minimization (IW-ERM) has little effect on deep networks trained for long durations, citing empirical and theoretical results, with only a possible recovery via polynomial-tailed losses. Yet the decision tree in Figure 4.1 routes paths (1a)/(2c) and (1d)/(2c) to importance weighting without distinguishing model class or training duration. Since many deployed AI-enabled systems are deep networks, this is a load-bearing omission: a practitioner following the recommended path for a modern deep model could use a method the report itself says is ineffective. I recommend adding a branch or explicit caveat in the decision tree and §7.4 telling users to test importance weighting against an unweighted baseline, or to prefer alternatives (e.g., data valuation, dynamic weighting, curriculum learning) when training deep models for long duratio","section":"§7.1.2 and Figure 4.1"}],"minor_comments":[{"comment":"The quantitative evidence consists of one PubMed split with k=800 and no repeated trials or error bars. The observed differences for '% Receiving U.S. Money' are within one percentage point of each other, which is not strong evidence of effectiveness. These sections should be labeled explicitly as illustrative walk-throughs rather than empirical validations.","section":"§5.4.2, §7.5, Tables 5.1 and 7.1"},{"comment":"There are several typographical errors: 'scikit kearn' should be 'scikit-learn'; 'weigthing' should be 'weighting'; and the column header 'T raining' in Table 5.1 contains a stray space. These should be corrected.","section":"§5.4.1 and §7"},{"comment":"'Relative the actionable definition' is missing the word 'to'; it should read 'Relative to the actionable definition.'","section":"§1.2"},{"comment":"The decision tree is dense and may be difficult to parse in print. A tabular summary mapping each (decision 1, decision 2) combination to the relevant chapter sections and tools would aid reproducibility for practitioners.","section":"§4.2, Figure 4.1"}],"recommendation":"minor_revision","confidential_remarks":"This manuscript is a technical/practitioner report rather than a standard research paper. Its value lies in synthesis, organisation, and actionable guidance, not in new empirical results. The practitioner experiments should not be treated as evidence of effectiveness. The main revisions I request—qualifying the abstract's claim and adding a robustness/assumption-checking step—are local and do not undermine the central framework, which is already careful in stating its limitations. The paper would benefit from an explicit statement that following the pipeline does not guarantee trustworthiness when the true distribution is unknown, despite the abstract's confident phrasing."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things. First, this is a genuinely useful synthesis: it organizes a diffuse literature on data curation around an explicit definition—optimize for the true distribution in the deployed environment—and gives practitioners a decision tree, twelve SME questions, and a tool-by-tool catalog with strengths, weaknesses, and preconditions. That is real value, especially for teams that do not have a curation playbook. Second, the stress-test concern lands, but the report already concedes it. Section 1.4 says plainly that demonstrable trustworthiness is constrained by how well the true distribution is known, and it does not assume the distribution can be fully parameterized. The authors are honest that their framework is conditional on a premise they do not validate.\n\nThe paper's strengths are real. The decision tree in Chapter 4 is a sensible way to route between sampling, weighting, and splitting approaches. The 12 questions in Chapter 3 are concrete and well grounded in the elicitation and data-centric AI literature. The practitioner sections, especially the rsw and SSS examples, are transparent about sample sizes and do not dress up small demonstrations as large-scale evidence. The report also flags the known weakness of importance weighting for deep networks (Section 7.1.2) and points to polynomial-tailed losses as a possible fix—so it is not hiding the load-bearing issues.\n\nWhere I would push back is on the implied practicality. The central claim—follow these steps and you promote trustworthiness—is essentially untested. The two use cases are illustrative, not evaluations; there are no error bars and no comparison against a control pipeline that skipped the curation steps. So the report is a good map, but it does not show that following the map gets you to a trustworthy system. That is a limitation in proportion to the authors' own framing: they call it a synthesis and a set of practices, not a result. Still, a serious referee should ask for a sharper statement of when the true distribution can be approximated well enough, and for any evidence—even qualitative—that the decision tree leads to better outcomes than common practice.\n\nWho is this for? Data scientists and project managers who want a structured way to think about curation for deployment, and researchers looking for a compact survey of the tool landscape. It deserves peer review as a systems/report-style contribution, with revision focused on scoping the claims and possibly adding a validation study. I would not desk reject it. I would keep it in my reference list for the decision tree and the definition, though I would not cite it as evidence that the pipeline improves trustworthiness.","headline":"A thorough, honest synthesis of data curation practice organized around a clear definition of trustworthiness; the central conditionality on knowing the true distribution is the main soft spot, but the report never overclaims beyond it.","tokens_in":42546,"tokens_out":1878,"would_cite":true,"duration_ms":25118,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The report argues trustworthiness means optimizing for the true distribution of deployed inputs.","keywords":["AI trustworthiness","data curation","true distribution","distribution shift","validation set selection","resampling","data weighting","domain knowledge elicitation"],"falsifier":"Take two models with identical architecture and training budget. Train one on data curated with the report's decision tree and the other on the raw development data. Evaluate both on a large, independently collected sample from the true deployment distribution. If the curated model's performance is not higher, the report's central claim that curation guided by the true distribution promotes trustworthiness is not supported.","tokens_in":1556,"feed_emoji":"🎯","tokens_out":4302,"duration_ms":88314,"temperature":0.7,"pith_summary":"The report argues that trustworthiness is not a vague aspiration but a concrete target: a trustworthy AI-enabled system must be optimized for performance on the true distribution of inputs it will encounter once deployed. It places this definition at the center of data curation, the phase between data acquisition and model training, and shows how data scientists can act on it. The authors identify two decision points—what is known about the true distribution, and whether interpretability or utility matters more—and connect those decisions to specific curation techniques: splitting, resampling, weighting, cleaning, and documentation. If the definition is accepted, the report's contribution is a practical workflow that turns 'trustworthy AI' into a set of concrete data-engineering choices rather than a slogan.","feed_headline":"Trustworthy AI starts with data from the true input distribution","feed_subtitle":"A practical workflow splits, resamples, and weights data based on knowledge of the deployed environment.","key_machinery":"The load-bearing concept is the 'true distribution' of deployed inputs, and the operating mechanism is the two-stage decision tree in Chapter 4. That tree maps four kinds of distributional knowledge (parametric estimates, a representative dataset, a prespecified representative subset, or an unlabeled representative subset) and five priority settings (from fully interpretable equal-weight data to opaque utility-maximizing weights) to specific curation actions. Every recommended technique is framed as a way to bring the training distribution closer to the true distribution, making the definition of trustworthiness executable.","core_discovery":"The central claim is that a trustworthy AI-enabled system must be optimized for performance on the true distribution of inputs it will encounter once deployed, and that this definition should guide data curation. Curation is not just preparation: it is the phase where developers translate the empirical distribution of available data toward the true distribution. The report organizes curation actions into a decision tree driven by two questions: what is known or assumed about the true distribution, and whether interpretability or utility matters more. Each branch leads to concrete techniques—representative sample selection, maximal representative subsampling, stratified splitting, resampling,","pith_inferences":["The framework's usefulness depends on how well the true distribution can be elicited or sampled; a natural extension would pair the decision tree with post-deployment monitoring to detect drift and trigger re-curation.","The report's definition of trustworthiness is performance-centric; user perceptions, ethics, and accountability are acknowledged but left largely outside. An extension could treat those as additional selected characteristics or as constraints within the same decision structure.","The decision tree could be operationalized as a checklist or recommendation tool for data scientists, and automated distribution-shift detection might substitute for expert elicitation in settings where domain experts are unavailable.","The 'true distribution' framing assumes a single, relatively stable deployment environment; in nonstationary environments the framework would need a time-varying or scenario-based extension."],"forward_implications":["Validation sets should be constructed to represent the true distribution, not just randomly held out; otherwise the demonstrated trustworthiness of the model is limited.","When no representative sample exists, domain experts must supply distributional parameters, and structured elicitation methods make that knowledge usable for splitting, weighting, and selection.","Resampling and weighting are interchangeable translations between distributions, with interpretability and calibration trade-offs; the right choice depends on the strength of the available model and the project's need for transparency.","Fairness is a special case of the framework: selected characteristics are the aspects across which performance must be equal, and fairness-oriented curation techniques apply to any mission-relevant characteristic.","Pretrained models and large language models enter the curation picture as feature generators: embeddings should be treated as features, and biased embeddings should be corrected during curation."],"supporting_citations":[{"why":"Supplies the framing that data are purposeful simplifications created through human choices, and that data scientists spend much of their time preparing data.","marker":"[112]"},{"why":"Provides the Integrated Model of Trust whose four components (competence, positive intentions, ethics, predictability) the report uses to connect its definition to trust scholarship.","marker":"[82]"},{"why":"Grounds the actionable definition in the Department of Defense principle that AI capabilities must be reliable and tested within defined uses.","marker":"[116]"},{"why":"Defines optimal representative sample weighting, the optimization method behind the report's representative sample selection and weighting guidance.","marker":"[19]"},{"why":"Introduces maximal representative subsampling, the positive-unlabeled learning technique the report recommends when a representative sample exists but selected characteristics are not specified.","marker":"[63]"},{"why":"Catalogs splitting strategies such as simple random sampling, systematic sampling, and stratified sampling that the report adapts for creating representative validation sets.","marker":"[129]"},{"why":"Introduces SMOTE, the synthetic minority over-sampling technique the report presents for up-sampling underrepresented classes.","marker":"[31]"},{"why":"Supplies the theoretical foundation of importance-weighted empirical risk minimization for covariate shift, a central model-agnostic weighting approach.","marker":"[127]"},{"why":"Brings Shapley values from cooperative game theory into data valuation, supporting the report's discussion of model-architecture-specific weighting.","marker":"[141]"},{"why":"Describes the Sheffield Elicitation Framework used to elicit distributional parameters from subject-matter experts.","marker":"[153]"}],"fun_headline_variants":["Trustworthy AI starts with data from the true distribution","Curation for the real world: the path to trustworthy AI","True distribution data: the foundation of AI trust","AI trust is built by curating data toward reality","To earn trust, AI must be trained on the right data"],"cache_read_input_tokens":44416,"weakest_assumption_plain":"The whole workflow assumes the true distribution of inputs in the deployed environment can be known or approximated well enough—through expert estimates, a representative sample, or a pre-specified subset—to guide curation choices.","fun_headline_variants_meta":{"raw":{"variants":["Trustworthy AI starts with data from the true distribution","Curation for the real world: the path to trustworthy AI","True distribution data: the foundation of AI trust","AI trust is built by curating data toward reality","To earn trust, AI must be trained on the right data"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000181,"raw_usage":{"total_tokens":1080,"prompt_tokens":616,"completion_tokens":464,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":360,"completion_tokens_details":{"reasoning_tokens":385}},"tokens_in":360,"tokens_out":464,"duration_ms":6011,"temperature":1.0,"reasoning_tokens":385,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T18:16:40.166061+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take two models with identical architecture and training budget. Train one on data curated with the report's decision tree and the other on the raw development data. Evaluate both on a large, independently collected sample from the true deployment distribution. If the curated model's performance is not higher, the report's central claim that curation guided by the true distribution promotes trustworthiness is not supported.","supporting_citations":[{"cited_title":"Optimal representative sample weighting","cited_arxiv_id":null,"evidence_quote":"Defines optimal representative sample weighting, the optimization method behind the report's representative sample selection and weighting guidance."}],"review_version":1}