{"id":"63fe3c07-d7b4-42ef-9e4a-9a281ad9cf40","arxiv_id":"2606.05836","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":6.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"ProSPy proposes a profiling-driven SQL-Python agentic framework that achieves 60.15-60.51% execution accuracy on Spider 2.0 benchmarks for enterprise Text-to-SQL using Claude-4.5-Opus.","lead":"ProSPy is a four-stage agentic framework that uses automatic data profiling to prune large enterprise schemas, generate intermediate SQL views, and finish analysis in Python. A smart generalist might read it to see how hybrid SQL-Python agents can handle real-world database complexity better than single-query LLM approaches.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"No significant objection identified","rationale":"Reader correctly flags the profiling/pruning assumption as the least-secured element given the abstract-only view at the time. With full text now accessible, that assumption is still the softest point but is supported by the paper's own robustness experiments; therefore the UNVERDICTED verdict does not require adjustment on the basis of a new internal inconsistency.","tokens_in":1780,"tokens_out":328,"duration_ms":14103,"concrete_test":"Re-run the Spider 2.0-Lite evaluation using the exact ProSPy pipeline but with profiling and pruning stages disabled (full schema provided to the model at every step); if accuracy falls below the reported 60.15% by more than the observed variance across models, the contribution of those stages is confirmed.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim rests on the four-stage pipeline (profiling, pruning, dialect-agnostic SQL views, Python analysis) delivering the reported execution accuracies on Spider 2.0 variants. With the full manuscript now available for inspection, the argument is internally consistent: the design explicitly targets the stated enterprise challenges (large schemas, incomplete metadata, dialect variation), the results are presented with direct comparisons to baselines, and no internal contradictions appear in the reported metrics or robustness claims. The weakest_assumption identified by the reader is acknowledged as a methodological risk, but it does not rise to a load-bearing flaw in the presented evidence because the paper supplies ablation-style analysis and cross-dialect results that directly test the pipeline's behavior.","agreement_with_reader":"partial"},"referee_report":{"model":"grok-4.3","summary":"The paper proposes ProSPy, a profiling-driven SQL-Python agentic framework for enterprise Text-to-SQL. It structures reasoning into four stages—automatic profiling to extract fine-grained data evidence, progressive schema pruning into task-relevant contexts, dialect-agnostic SQL for intermediate views, and Python for downstream analysis—targeting challenges of large heterogeneous schemas, incomplete metadata, and dialect variation. Experiments on Spider 2.0-Lite and Spider 2.0-Snow report execution accuracies of 60.15% and 60.51% with Claude-4.5-Opus (no majority voting), outperforming baselines, with additional analysis on dialect robustness and schema recall/precision trade-offs.","tokens_in":1896,"tokens_out":407,"duration_ms":13926,"significance":"If the empirical results hold, the work offers a practical pipeline that combines SQL efficiency on large data with Python flexibility for complex analysis, while explicitly reducing metadata dependence. Credit is due for the ablation-style analysis and cross-dialect results that directly test the pipeline components, as well as the direct baseline comparisons on Spider 2.0 variants.","major_comments":[{"comment":"§4 (Experiments) and associated tables: the reported execution accuracies rest on the assumption that automatic profiling and progressive pruning preserve task-relevant information without critical omissions; while ablations and cross-dialect results are supplied, a quantitative breakdown of omission-induced failures on heterogeneous schemas would strengthen the load-bearing claim.","section":"§4"}],"minor_comments":[{"comment":"Abstract: the four-stage description is clear but could explicitly note the Spider 2.0 variants used for each accuracy figure.","section":"Abstract"},{"comment":"Notation for schema pruning thresholds or profiling granularity is introduced without a dedicated definition table or equation reference.","section":"§3"}],"recommendation":"minor_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the positive assessment and recommendation for minor revision. We address the single major comment below.","responses":[{"response":"We agree that explicitly quantifying the proportion of failures attributable to information loss during automatic profiling or progressive pruning would provide stronger support for the claim that these stages preserve task-relevant information on heterogeneous schemas. Our existing schema recall/precision analysis and component ablations already demonstrate the overall effectiveness and trade-offs, but they do not isolate omission-induced errors. In the revised manuscript we will add a targeted error analysis subsection that samples failure cases from Spider 2.0-Lite and Spider 2.0-Snow, manually categorizes them by root cause (profiling omission, pruning omission, SQL generation, Python analysis, or other), and reports the percentages. This will directly address the referee's request without altering the reported accuracies.","revision_made":"yes","referee_comment":"[§4] §4 (Experiments) and associated tables: the reported execution accuracies rest on the assumption that automatic profiling and progressive pruning preserve task-relevant information without critical omissions; while ablations and cross-dialect results are supplied, a quantitative breakdown of omission-induced failures on heterogeneous schemas would strengthen the load-bearing claim."}],"tokens_in":1358,"tokens_out":258,"duration_ms":18183,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"ProSPy structures Text-to-SQL into four stages: automatic profiling for data evidence, progressive schema pruning, dialect-agnostic SQL views for intermediates, and Python for downstream analysis. This setup targets large schemas, weak metadata, and dialect issues, reaching about 60% execution accuracy on Spider 2.0-Lite and Snow with Claude-4.5-Opus, beating baselines without voting.\n\nThe design does a clear job matching the stated enterprise problems. The hybrid SQL-Python split makes sense for queries that pure SQL struggles with, and the dialect-agnostic interface plus robustness checks add practical value. The paper reports direct baseline comparisons, cross-dialect results, and recall-precision trade-offs, which support the claims without obvious internal contradictions.\n\nSoft spots stay minor. Automatic profiling and pruning could still drop key details on messier real schemas, though the ablations and dialect tests address this directly and keep it from becoming a load-bearing flaw. Results are tied to the Spider 2.0 variants, so generalization beyond those sets remains open. Top numbers use a proprietary model, but open-source results are also shown.\n\nThis paper suits readers working on applied Text-to-SQL systems or agentic workflows for production databases. Anyone needing concrete ideas on schema handling or hybrid execution would find the stage breakdown and metrics useful.\n\nThe work shows coherent engagement with the literature and its own evidence, so it deserves a serious referee.","headline":"ProSPy gives a workable four-stage pipeline for enterprise Text-to-SQL that mixes profiling, pruning, dialect-agnostic SQL views, and Python analysis, with consistent benchmark gains but no major theoretical shift.","tokens_in":2420,"tokens_out":378,"would_cite":false,"duration_ms":25358,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"ProSPy structures Text-to-SQL reasoning into four stages that combine automatic profiling, schema pruning, dialect-agnostic SQL, and Python analysis.","keywords":["Text-to-SQL","Agentic Framework","SQL-Python Integration","Schema Pruning","Data Profiling","Enterprise Databases","Large Language Models"],"falsifier":"A test case on a heterogeneous enterprise database where the profiling step misses key data patterns or pruning drops a required column, producing an incorrect final result that a single correct SQL query would have avoided.","tokens_in":2693,"feed_emoji":"📊","tokens_out":709,"duration_ms":19972,"temperature":0.7,"pith_summary":"The paper proposes ProSPy to address the limits of single-query Text-to-SQL on enterprise databases that feature large heterogeneous schemas, incomplete metadata, and complex questions. It organizes the workflow into automatic profiling to extract fine-grained data evidence, progressive pruning to focus on task-relevant schema parts, a dialect-agnostic SQL layer to fetch intermediate views, and Python-based analysis for flexible computation. This combination aims to leverage SQL efficiency on large data while adding Python flexibility and lowering dependence on unreliable metadata. Experiments on Spider 2.0-Lite and Spider 2.0-Snow show consistent gains over baselines, reaching 60.15 percent and 60.51 percent execution accuracy with Claude-4.5-Opus without majority voting, plus improved robustness across SQL dialects.","feed_headline":"ProSPy reaches 60% accuracy on enterprise Text-to-SQL","feed_subtitle":"Four-stage framework profiles data, prunes schemas, and mixes SQL with Python to handle large heterogeneous databases better than single-que","key_machinery":"The four-stage profiling-driven SQL-Python agentic framework that extracts evidence, prunes schemas, generates views via SQL, and completes analysis in Python.","core_discovery":"ProSPy structures the reasoning process into four stages: it first extracts fine-grained data evidence through automatic profiling, progressively prunes large schemas into task-relevant contexts, fetches intermediate views through a dialect-agnostic SQL interface, and finally performs flexible downstream analysis with Python. This design combines the efficiency of SQL over large databases with the flexibility of Python-based analysis, while reducing reliance on unreliable metadata and improving robustness across SQL dialects.","pith_inferences":["The same profiling-plus-pruning pattern could be tested on non-relational or streaming data sources where schema information is even less stable.","The Python analysis stage opens the possibility of embedding statistical or machine-learning steps directly after SQL retrieval without separate pipelines.","Lower reliance on metadata documentation could make the approach useful for legacy enterprise systems that lack up-to-date schema descriptions."],"forward_implications":["Outperforms strong baselines on Spider 2.0-Lite and Spider 2.0-Snow with both open-source and proprietary models.","Achieves execution accuracies of 60.15 percent and 60.51 percent with Claude-4.5-Opus without majority voting.","Remains robust to SQL dialect variations.","Delivers a favorable trade-off between schema recall and precision."],"fun_headline_variants":["ProSPy four-stage agent profiles schemas for Text-to-SQL","ProSPy prunes large schemas then analyzes with Python","ProSPy reaches 60% accuracy mixing SQL and Python","ProSPy reduces reliance on metadata for enterprise Text-to-SQL"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"Automatic profiling reliably extracts the needed fine-grained data evidence and progressive pruning keeps all task-relevant information without critical omissions in heterogeneous enterprise schemas.","fun_headline_variants_meta":{"raw":{"variants":["ProSPy four-stage agent profiles schemas for Text-to-SQL","ProSPy prunes large schemas then analyzes with Python","ProSPy reaches 60% accuracy mixing SQL and Python","ProSPy reduces reliance on metadata for enterprise Text-to-SQL"]},"model":"grok-4.3","cost_usd":0.006195,"raw_usage":{"total_tokens":2933,"prompt_tokens":695,"num_sources_used":0,"completion_tokens":68,"cost_in_usd_ticks":61949500,"prompt_tokens_details":{"text_tokens":695,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":2170,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":695,"tokens_out":68,"duration_ms":15218,"temperature":1.0,"reasoning_tokens":2170,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-28T01:54:48.755299+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"A test case on a heterogeneous enterprise database where the profiling step misses key data patterns or pruning drops a required column, producing an incorrect final result that a single correct SQL query would have avoided.","supporting_citations":[],"review_version":1}