{"id":"ea918070-d32d-443f-a034-53b1a38fe762","arxiv_id":"2607.22031","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"IDSTune jointly tunes knobs, indexes, and materialized views via coordinated LLM agents, reporting up to 38% latency gains and 57% faster tuning on benchmarks.","lead":"IDSTune uses coordinated LLM agents to tune database knobs, indexes, and materialized views at the same time, instead of running separate tuners that can fight each other. On benchmarks it reports up to 38% latency reduction and 57% faster tuning than sequential combinations of existing tools.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Superiority may hinge on hand-assembled triple baselines; no independent integrated triple tuner is compared, and chosen composition orders are not tested against alternatives.","rationale":"The reader's weakest_assumption is the same as the concern I find most load-bearing: the triple-configuration baselines are constructed for this paper and may be unrepresentative. The paper has real strengths—repeated runs with IQR, component-wise ablations, cost analysis, cross-engine/hardware experiments, and public code—so I would keep CONDITIONAL rather than reject. The Single-Agent ablation partially counters the concern by showing a single monolithic LLM proposing all three component types is much worse (617s vs 379s on JOB), suggesting coordination matters, but it is not a published or tuned integrated triple tuner. I also note a secondary numerical red flag in §6.2: the SDSS '32.7%' formula (783.56−527.39)/783.56 has no connection to the stated 965.48→465.43 numbers; this should be checked against raw logs. My verdict recommendation is UNCHANGED: keep CONDITIONAL with moderate confidence until a stronger integrated baseline is evaluated.","tokens_in":24320,"tokens_out":8882,"duration_ms":100328,"concrete_test":"Re-run the main JOB experiments against a fair integrated triple baseline: use the same GPT-4.1 backbone to perform best-of-k joint search over {knobs, indexes, MVs} with the same workload-replay budget (e.g., SMAC/BO or an LLM-driven sampler allowed to revise all three components in each round), and additionally record the reverse orderings of UniView+λ-Tune and AgentTune+HMAB. If IDSTune still beats the best of these by the reported margins, the baseline-fairness concern is resolved; if the margin drops substantially, the headline should be weakened.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—that IDSTune jointly optimizes knobs, indexes, and materialized views better than any alternative—rests on comparisons in §6.2 against three author-constructed sequential baselines: UniView→λ-Tune, Proto-X→UniView, and AgentTune→HMAB. Because the paper asserts in §6.1 that IDSTune is the first triple-component tuner, no existing integrated system is available for comparison; the baselines are combinations of single/pair tuners run in fixed orders. The authors say some orders were chosen because they 'yield better results than the reverse', but no reverse-order or alternative-composition results are shown. This is load-bearing: the 38.3%, 34.7%, and 56.9% headline numbers in §6.2 are measured against these hand-assembled pipelines. A baseline that jointly searches the same three configuration spaces within the same wall-clock budget—rather than fixing one component's output before invoking the next expert—could plausibly close much of the gap. The Single-Agent ablation in §6.5.2 does test joint reasoning, but it is a single LLM call without workload compression or iterative revision, so it does not fully establish representativeness of the main triple baselines.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes IDSTune, an LLM-driven multi-agent framework for jointly tuning three database configuration components: knobs, indexes, and materialized views. The system first compresses workloads into query/data/system features with an LLM-based selection agent, then uses specialist agents (knob, index, view) to propose configurations and a supervisor agent to iteratively resolve conflicts and refine recommendations, with hybrid rule/LLM safety guardrails and optional web search. Experiments on TPC-H, JOB, SDSS, TPC-C, SYSBENCH, Birds, and Redbench compare IDSTune against single-, dual-, and triple-component baselines and report up to 38% latency reduction and 57% faster tuning, with additional ablation, cost, scalability, drift, and cross-engine studies. Code and prompts are publicly available.","tokens_in":24549,"tokens_out":6655,"duration_ms":69614,"significance":"If the results hold, IDSTune is a meaningful step beyond prior work, which has focused on single components or pairs (knob+index, index+view). The paper's claimed contribution—integrated three-component tuning with explicit conflict resolution—is plausible and would be useful for both OLAP and OLTP settings. The evaluation is more thorough than typical systems papers in several respects: five repeated sessions with median/IQR, detailed component ablations, cost analysis, time-budget sensitivity, and robustness to data/query drift. The public release of code and prompt templates is a concrete reproducibility strength. However, the headline quantitative claims rest on comparisons against hand-assembled sequential baselines, since no independent integrated triple tuner exists. The fairness and representativeness of those baselines is the main correctness risk, and at least one reported real-world improvement number is internally inconsistent.","major_comments":[{"comment":"The triple-configuration comparison is the load-bearing support for the abstract's 'up to 38%' and '57% faster' claims, but all three triple baselines are sequential compositions of prior single/pair tuners (UniView→λ-Tune, Proto-X→UniView, AgentTune→HMAB). The text states that these orders 'yield better results than the reverse' but no reverse-order or alternative-composition results are reported. A fair joint-search baseline—one that explores the same three configuration spaces within the same wall-clock budget—could plausibly close much of the gap. Please provide order-sensitivity analysis (at least for the reported orders) or an integrated baseline, or explicitly qualify the claims as being 'vs. sequential compositions' rather than 'vs. state-of-the-art triple tuning.'","section":"§6.1, §6.2"},{"comment":"The real-world SDSS claim contains an arithmetic inconsistency. The text states IDSTune reduces latency to 465.43 s and outperforms all baselines by an average of 32.7%, but the displayed formula (783.56 − 527.39)/783.56 = 32.7% uses 527.39 s, not 465.43 s. If 465.43 s is the correct final latency, the improvement over the 783.56 s baseline average is 40.6%, not 32.7%. Please correct the formula or clarify what 527.39 represents, and specify which baselines are included in the average.","section":"§6.2, SDSS result"},{"comment":"The Single-Agent ablation does not fully address the concern that a competent joint optimizer could outperform the sequential baselines used in §6.2. The Single-Agent variant is a single LLM call without workload compression or iterative revision, so it conflates 'joint search' with 'single-shot monolithic prompting.' To isolate the value of multi-agent specialization, report a variant that performs the same iterative feedback loop with a single unified agent, and ideally a variant with the same workload compression and search budget as the full system.","section":"§6.5.2"}],"minor_comments":[{"comment":"The memory-budget experiment is described as 'shown in Figure 11(a)' and 'Figure 11(b)', but those panels belong to the time-budget experiment in §6.6.2. The references should be to Figure 10(a) and Figure 10(b).","section":"§6.6.1"},{"comment":"The sentence 'On JOB, it achieves at least a 38.3% and 34.7% reduction ... (614.87−379.25/614.87 = 38.3%, 580.66−379.25/580.66 = 34.7%)' is grammatically ambiguous: it is unclear whether the displayed numbers refer to JOB or to TPC-H/SDSS. Please clarify which baselines and benchmarks each percentage refers to.","section":"§6.2"},{"comment":"The name 'Uniview' is used inconsistently with 'UniView' across the baselines section, figure captions, and appendix. Please standardize.","section":"Throughout"},{"comment":"In the 'No Workload' row, '799.5327.59' lacks a separator between the best-found performance and the runtime overhead; it should read '799.53' and '27.59'.","section":"Table 2"},{"comment":"The text states that Figure A.1 reports per-query execution times and that 'the performance gain ... translate to gains or at least equal performance ... for each single query,' but the figure is not legibly present in the manuscript. Please ensure the figure is included or qualify the claim.","section":"Appendix A"}],"recommendation":"major_revision","confidential_remarks":"To the editor: The paper's novelty claim ('first triple-component tuner') and its reliance on the authors' own prior work (AgentTune, E2ETune) are not disqualifying, but the main evaluation risk is the absence of any independent integrated triple tuner. The constructed sequential baselines need order-sensitivity checks or an explicit qualification. The SDSS arithmetic inconsistency discussed above should be corrected before publication. I did not audit the released code; the reproducibility claim rests on the public repository and prompt templates."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my honest take. IDSTune is the first framework I know of that jointly tunes knobs, indexes, and materialized views in one LLM-driven multi-agent loop. That's a real contribution, not just an incremental wrapper: the supervisor-specialist architecture with conflict resolution and revision is sensible, and the workload compression (feature extraction plus selection) is a good answer to token limits. The evaluation is also noticeably more careful than typical LLM-tuning papers—five repeated sessions with IQR, detailed ablations, cost breakdowns, cross-engine tests on SQL Server, drift scenarios. The case study in Figure 1 shows the kind of cross-component conflict that motivates the work. I believe the central claim is plausible: coordinated tuning beats sequential single/pair tuners.\n\nThe weak spot is exactly what the stress-test flags. Because no existing system tunes all three components, the baselines are author-assembled sequential pipelines (UniView→λ-Tune, Proto-X→UniView, AgentTune→HMAB). The paper says the chosen orders 'yield better results than the reverse' but never shows the reverse or any alternative composition. That matters because the headline 38% improvement is measured against these specific pipelines. It's not fatal—the sequential order is a natural default, and the single-agent ablation in 6.5.2 partially supports the coordination story—but it means the strongest claim ('consistently finds the best configurations') is not fully established. A revision should add reverse-order runs and, ideally, a joint-search baseline that shares the same wall-clock budget.\n\nThere are also minor arithmetic slips: the SDSS 32.7% formula uses 527.39 while the text says 465.43, and '1.6× times' is sloppy. These are typos, not load-bearing.\n\nBottom line: this is a solid, useful paper for the DBMS tuning community. The architecture is new, the experiments are thorough, and the limitations are honest. I'd send it to serious peer review. I'd also bring it to a reading group—it's a good example of LLM multi-agent systems applied to a concrete systems problem, with evaluation quality above the current average. I'd cite it if I worked on tuning.","headline":"IDSTune is a real first for integrated tuning of knobs, indexes, and materialized views, but its headline gains are measured against hand-assembled sequential baselines and a few arithmetic slips.","tokens_in":25163,"tokens_out":2747,"would_cite":true,"duration_ms":28176,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A coordinated multi-agent LLM framework can jointly tune knobs, indexes, and materialized views, cutting database latency by up to 38% and tuning time by 57%.","keywords":["database tuning","large language models","multi-agent systems","knob configuration","index selection","materialized views","workload compression","physical database design"],"falsifier":"Run IDSTune and the strongest baseline on a workload engineered to contain a known cross-component conflict (e.g., a query where a materialized view plus a raised per-query memory cap flips the optimizer from an index plan to a hash join). If a natively integrated three-component tuner, or even a carefully ordered combination, matches or beats IDSTune's latency within the same tuning budget, the coordination mechanism is not what carries the gains.","tokens_in":24124,"feed_emoji":"⚙️","tokens_out":8177,"duration_ms":81567,"temperature":0.7,"pith_summary":"Database tuning tools usually optimize one knob, index, or materialized view at a time, and chaining separate tools together can backfire because their recommendations conflict. IDSTune is an attempt to tune all three configuration families together using specialized large language model agents coordinated by a supervisor. The paper claims this joint approach avoids harmful interactions, finds globally consistent configurations faster, and adapts to workload and data drift. If right, it points toward a new class of tuning systems that treat the database configuration as one coordinated decision space rather than three independent ones.","feed_headline":"AI agents jointly tune knobs, indexes, and views, cutting latency 38%","feed_subtitle":"A supervisor agent reconciles the three, so settings, indexes, and views reinforce each other instead of fighting.","key_machinery":"The load-bearing mechanism is a centralized multi-agent loop: specialist agents for knobs, indexes, and materialized views generate proposals; a supervisor agent arbitrates, detects conflicts and redundancies, and requests revisions; and hybrid safety guardrails (rule-based range checks plus LLM-based semantic checks) approve the final configuration. Around this loop sits workload compression—an LLM-based selection agent that picks a small, task-relevant subset of query, data, and system features so prompts stay compact—and optional web search that supplies external domain knowledge. The coordinator's arbitration is what converts three locally sensible recommendations into one globally consi","core_discovery":"IDSTune claims to be the first database tuning framework to jointly recommend knobs, indexes, and materialized views in a single coordinated process. It compresses the incoming workload into a compact set of query, data, and system features, then lets three specialist agents—one per configuration component—propose candidate settings. A supervisor agent merges the proposals, rejects conflicting or redundant ones (for instance, a view that duplicates an index or a memory setting that would destabilize the plan), and sends revision instructions until the report is coherent. A safety layer filters invalid configurations before deployment, and the resulting configuration is executed and fed back","pith_inferences":["A natural next test is to apply the same supervisor pattern to other configuration families, such as query rewrite rules, table partitioning, or vacuum policies, where optimizer interactions are equally coupled.","Because the supervisor's role is mostly conflict detection rather than deep domain expertise, a smaller and cheaper model might play that role while large specialist models generate proposals—a trade-off the paper does not explore.","A sharper evaluation will appear once other groups build native three-component tuners; comparing against those, rather than sequenced single-component tools, will isolate how much of the gain comes from the coordination mechanism itself.","The interpretable feature-selection and traceable agent dialogue suggest the same machinery could be reused for explaining a chosen configuration or diagnosing performance regressions, not just for generating one."],"forward_implications":["A DBA can tune all three configuration families in one pass, eliminating the need to chain separate knob, index, and view tools.","Tuning cost drops because a large share of refinement happens inside the agent dialogue, so fewer expensive workload replays are needed to reach a good configuration.","The framework is training-free and feature-based, so it transfers to new hardware and database engines without a retraining phase.","Under data and query drift, the tuner degrades less and recovers faster than physical-design-only methods, and it was the only method in the paper's real-world trace with consistently positive gains.","The coordination yields a globally consistent final configuration—memory settings that amplify the benefit of indexes and materialized views—rather than a union of local optima."],"fun_headline_variants":["One AI supervisor gets knobs, indexes, views to agree—38% faster","Multi-agent tuning cuts latency 38% and tuning time 57%","First to tune knobs, indexes, and views together: 38% gain","Three AI agents, one coordinated DB tune: 38% less latency","Joint DB tuning: knobs, indexes, views in sync—38% faster"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The headline performance advantage assumes that the comparison baselines—built by chaining existing separate tuners—fairly represent the best available multi-component tuning; if a stronger integrated baseline exists, the reported gap could shrink.","fun_headline_variants_meta":{"raw":{"variants":["One AI supervisor gets knobs, indexes, views to agree—38% faster","Multi-agent tuning cuts latency 38% and tuning time 57%","First to tune knobs, indexes, and views together: 38% gain","Three AI agents, one coordinated DB tune: 38% less latency","Joint DB tuning: knobs, indexes, views in sync—38% faster"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000797,"raw_usage":{"total_tokens":3339,"prompt_tokens":735,"completion_tokens":2604,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":479,"completion_tokens_details":{"reasoning_tokens":2502}},"tokens_in":479,"tokens_out":2604,"duration_ms":17124,"temperature":1.0,"reasoning_tokens":2502,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T05:59:58.125433+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run IDSTune and the strongest baseline on a workload engineered to contain a known cross-component conflict (e.g., a query where a materialized view plus a raised per-query memory cap flips the optimizer from an index plan to a hash join). If a natively integrated three-component tuner, or even a carefully ordered combination, matches or beats IDSTune's latency within the same tuning budget, the coordination mechanism is not what carries the gains.","supporting_citations":[],"review_version":1}