{"id":"90c8fde3-7617-4c9e-8202-f8ec298ff4c1","arxiv_id":"2506.02683","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"DPPM, a decompose-plan-in-parallel-and-merge framework with verify-and-refine feedback, improves final pass rates on travel-planning benchmarks over Direct, CoT, and LLM-Modulo.","lead":"DPPM is a planning method that splits travel-planning tasks into four parallel sub-problems (transport, lodging, sights, meals), plans each with a separate large language model agent, then merges the local plans into a final itinerary with a verify-and-refine step. On TravelPlanner and a modified ChinaTravel set, it reports clearly higher final pass rates than Direct, CoT, and LLM-Modulo.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"ChinaTravel-M is an unreleased, model-assisted reformatting of ChinaTravel, so the 13–32 point gains may not transfer to the original benchmark; the four-aspect decomposition assumption is untested on original constraints.","rationale":"The reader's conditional verdict already flags the modified ChinaTravel benchmark and the four-aspect decomposition as weaknesses. My stress-test agrees with the conditional verdict but sharpens the issue: the two weaknesses are connected. Appendix A's 'model-guided' adaptation may have rebuilt the original ChinaTravel constraints to conform to exactly the four-aspect schema DPPM assumes, which would make the benchmark modification the load-bearing factor behind the ChinaTravel-M improvements. The TravelPlanner validation results remain evidence for the method on that standard benchmark, so I would not reject the paper; however, the headline claim about 'travel planning tasks' should be conditional on release of the adaptation and on confirmation that the original constraints are compatible with the decomposition. This is a change in emphasis rather than a change in verdict: the paper should still be accepted only with the adaptation and data released and the baselines clarified. The reader's weakest_assumption (decomposability of constraints) is partially the same concern, but the distinct load-bearing point is that the paper may have modified the benchmark until that assumption held.","tokens_in":13470,"tokens_out":8665,"duration_ms":92153,"concrete_test":"Ask the authors to release the ChinaTravel-M adaptation script, including the prompts and the model used for the 'model-guided multi-round adjustments' in Appendix A. Then take the original ChinaTravel validation splits and determine, for each original Constraint Expression, whether it can be assigned to exactly one of the four DPPM aspects or to the global budget/stay categories. If any CE is cross-aspect or unassignable, DPPM's §3.1 decomposition cannot be instantiated on the original benchmark and the ChinaTravel-M results do not support the headline claim. If all CEs are assignable, run DPPM and LLM-Modulo on the original ChinaTravel evaluation without reformatting; a materially smaller advantage there would show the reported gains are specific to the adapted format.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim rests on two benchmarks, and the second one is not reproducible. Appendix A defines ChinaTravel-M through 'semantic parsing', 'iterative refinement' via 'model-guided multi-round adjustments', and 'standardized mapping', with no prompts, no model, no code, and no equivalence check between the original Constraint Expressions and the resulting TravelPlanner-style constraints. Since DPPM's decomposition in §3.1 assumes every constraint belongs to exactly one of four aspects (transportation, accommodation, attraction, meals), a reformatting that reshapes the original CEs into this schema would make the ChinaTravel-M comparison favorable by construction. The paper's stated limitation covers only single-constraint tasks, not cross-aspect or non-decomposable constraints. Because LLM-Modulo uses the same constraint evaluators and the same verification/refinement loop, the distinctive 12.8–32.0 point advantage claimed on ChinaTravel-M depends on exactly this unverified adaptation. If original ChinaTravel contains constraints that couple aspects or cannot be assigned to a single aspect, the reported gains on ChinaTravel-M may be an artifact of the adaptation rather than evidence for the paradigm.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DPPM, a three-stage planning paradigm for LLM-based travel planning: constraint-aware task decomposition (grouping all constraints into transportation, accommodation, attraction, and meals), parallel generation of multiple candidate subplans by local agents, and incremental merging of subplans via a Cartesian product with verification and refinement. The method is evaluated on the TravelPlanner validation set and on an adapted, unreleased version of ChinaTravel called ChinaTravel-M, using Qwen2.5-32B, Qwen2.5-72B, DeepSeek-V3, and DeepSeek-R1. The central empirical claim is that DPPM substantially outperforms Direct, CoT, and LLM-Modulo in final pass rate, with reported gains of about 22 to 32 percentage points over LLM-Modulo on TravelPlanner and 13 to 32 points on ChinaTravel-M. The paper also reports ablations showing the contribution of the global constraint instruction, the verification-and-refinement module, and the Cartesian product combination.","tokens_in":13625,"tokens_out":4387,"duration_ms":46262,"significance":"The proposed paradigm is a sensible response to two real limitations of LLM planning: constraint overload and cascading errors. If the reported results are reproducible, the work would be a useful contribution to the travel-planning benchmark literature, with a simple and generalizable decomposition idea. The paper has several strengths: it evaluates across multiple backbone families; it compares against LLM-Modulo using the same constraint evaluation functions and verification loop, making that comparison fair; the ablations isolate the contribution of each component; and the consistency of the TravelPlanner results across backbones is encouraging. However, the current evidence base is not yet sufficient to fully support the abstract's broad claim of significant superiority. The ChinaTravel-M benchmark is model-constructed, undocumented, and unreleased, and the quantitative claims are based on single runs on validation sets without significance tests or error bars. The significance of the work is therefore conditional on the authors providing reproducible benchmark artifacts and statistical grounding.","major_comments":[{"comment":"The ChinaTravel-M adaptation is not reproducible and may be favorable to DPPM by construction. Appendix A describes only three high-level steps: semantic parsing, iterative refinement, and standardized mapping, with no prompts, no model, no code, no equivalence checks between the original Constraint Expressions and the resulting TravelPlanner-style constraints, and no release of the converted benchmark. Since DPPM's decomposition stage in Section 3.1 assumes every constraint belongs to exactly one of four aspects, a reformatting that reshapes the original CEs into this schema would make the comparison favorable by construction. The authors should release the exact transformation artifacts, report conversion statistics (how many constraints were kept, split, merged, or dropped), verify that no cross-aspect coupling is introduced or removed, and ideally also evaluate on the original ChinaTravel benchmark with its original evaluation process.","section":"Section 4.1 / Appendix A"},{"comment":"The central claim of 'significantly outperforms' is not supported by statistical evidence. Table 1 reports point estimates on the TravelPlanner validation set only, with no error bars, no multiple seeds, and no significance tests. The same is true for Table 2 and Figure 5. Given that the headline differences are large, the claim may survive such testing, but the manuscript should either include standard deviations and statistical tests across at least three independent runs, or weaken the wording from 'significantly' to 'consistently outperforms on the validation set'. The authors should also report results on the TravelPlanner test set or justify why validation-only results are sufficient.","section":"Table 1 / Section 4.2"},{"comment":"The four-aspect decomposition and the fixed merge order assume that all constraints can be cleanly assigned to one aspect, with only budget and minimum-stay treated as global. The paper does not analyze what happens when constraints couple aspects in ways beyond these two examples, such as 'accommodation must be near a chosen attraction' or 'meals should be in the same city as the day's activities.' Because the ChinaTravel-M adaptation is undocumented, it is unclear whether such cross-aspect constraints existed in the original ChinaTravel and were removed or altered during conversion. Without evidence on non-decomposable or strongly coupled constraints, the paradigm's claimed generality over 'multiple constraints' is not established. The authors should either provide a formal or empirical analysis of the decomposition assumption on the original ChinaTravel constraints, or explicitly scope the method to constraints that satisfy the four-aspect decomposition.","section":"Section 3.1 / Section 3.3"},{"comment":"The introduction motivates the work by stating that ReAct and Reflexion achieve a final pass rate below 1% on multi-constraint travel planning, but neither baseline appears in Tables 1 or 2. Since the abstract claims DPPM 'significantly outperforms existing methods,' the comparison set is incomplete. The authors should either include these baselines under the same experimental settings or qualify the claim to 'outperforms the compared baselines' throughout the abstract and conclusion.","section":"Section 1 / Section 4.1"}],"minor_comments":[{"comment":"The table title says 'ChinaTravel dataset' but the body and Section 4.1 refer to the adapted 'ChinaTravel-M' dataset; the title should be corrected to avoid ambiguity.","section":"Table 2"},{"comment":"The introduction reports improvements on 'hard-level samples,' but Table 2 labels the difficulty levels as Easy, Medium, and Human. The terminology should be aligned, and the 'Human' level should be described more clearly.","section":"Introduction / Table 2"},{"comment":"The constraint definitions appendix says the definitions are 'shown in Figure 4,' but the corresponding figure appears to be Figure 7, and the actual constraint definitions are not reproduced in the text. The cross-reference should be fixed and the definitions should be included for self-containedness.","section":"Appendix C"},{"comment":"The authors remove phone numbers, website links, and geocoordinates from the reference information for all methods. Since this changes the inputs to all methods, the paper should document whether any constraint evaluation functions depend on the removed fields and should report whether results on the unmodified reference information differ.","section":"Section 4.2"},{"comment":"There are several typographical and grammatical issues, including 'TavelPlanner' in Section 4.1, 'we delegates different types of subtasks' in Section 3.2, and 'an significant step forward' in the Limitations section. These should be corrected.","section":"Throughout"},{"comment":"References Gundawar et al. 2024a and 2024b are identical (arXiv:2405.20625); they should be unified into a single reference or the two distinct papers should be cited separately.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The central issue is the unreleased and undocumented ChinaTravel-M adaptation: because DPPM's decomposition assumes a fixed four-aspect schema, the conversion process could inadvertently make the benchmark conform to the method's assumptions. I would ask the editor to require either release of the conversion pipeline and the adapted dataset or a holdout evaluation on the original ChinaTravel before considering the ChinaTravel-M claims as evidence. The TravelPlanner validation results are encouraging and may be sufficient for a revised claim if statistical robustness is added."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The thing to know: DPPM is a tidy engineering contribution. It splits travel planning into four aspect-wise subtasks, plans each in parallel, merges with a fixed order, and wraps everything in a verify-and-refine loop. On the TravelPlanner validation set it beats LLM-Modulo by 22 points with Qwen2.5-32B and by 19 with DeepSeek-V3, and the gains hold across four backbones. That is a real result, and the ablation makes sense: removing the verification module collapses performance, and the Cartesian product search over subplans buys another 13 points.\n\nWhat is actually new is the specific combination: constraint-aware decomposition, parallel local planning, global-constraint-aware prompts, and the incremental merge order. The components are all known — generate-test, self-refinement, merging — and the authors are honest that the verification module is inherited from LLM-Modulo. The contribution is a combination, not a new principle. That is fine, but it sets the bar for evidence higher, and the evidence is not quite there.\n\nThe soft spots, in descending order. First, ChinaTravel-M. The appendix describes the adaptation in three vague bullets — 'semantic parsing,' 'iterative refinement,' 'standardized mapping' — with no prompts, no model, no code, and no equivalence check between the original constraint expressions and the TravelPlanner-style constraints. Since DPPM's decomposition assumes every constraint belongs to exactly one of four aspects, a reformatting that imposes that schema makes the comparison favorable by construction. The stress-test note is right, and this is the load-bearing problem: the 13–32 point gains on ChinaTravel-M cannot be assessed until the benchmark is released and the adaptation is verifiable. The paper's own limitation section only mentions single-constraint tasks, which misses the deeper concern about cross-aspect constraints.\n\nSecond, the baselines. The introduction cites ReAct and Reflexion as failing below 1% final pass rate, but neither is included in the experiments. That is a conspicuous omission, especially since the authors mention them. Third, there are no error bars, no significance tests, and only the validation split. The word 'significantly' appears in the abstract, but no statistical support is provided.\n\nIf a referee is assigned, the verdict should be conditional: the TravelPlanner result is plausible and worth publishing after a solid revision, but the ChinaTravel-M claim needs to be backed by a released benchmark and an equivalence guarantee. Whoever runs the review should also ask for ReAct/Reflexion numbers and for confidence intervals. This paper is for people working on LLM planning benchmarks; they will want to know the DPPM arrangement works on the standard TravelPlanner testbed. I would not desk reject it, but I would not let the ChinaTravel-M claims stand as-is.","headline":"DPPM is a clean combination of known building blocks that shows large, consistent gains on TravelPlanner, but the unreleased ChinaTravel-M adaptation is too shaky to count as evidence.","tokens_in":14218,"tokens_out":2607,"would_cite":false,"duration_ms":24952,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"By splitting a trip into four aspect-specific subtasks, planning those in parallel, and merging with verification, DPPM claims far higher all-constraint satisfaction than prior LLM planners.","keywords":["LLM planning","travel planning","constraint satisfaction","task decomposition","parallel planning","plan merging","verification and refinement","planning benchmarks"],"falsifier":"Take a set of planning queries whose constraints are deliberately cross-aspect, for example 'the hotel must be within walking distance of the lunch restaurant, and the total daily walking distance under the attraction plan is capped', and count the final pass rate. If DPPM's advantage over a sequential planner with the same refinement budget disappears on such queries, the aspect-separability assumption is the failing premise.","tokens_in":13219,"feed_emoji":"🧳","tokens_out":8731,"duration_ms":73538,"temperature":0.7,"pith_summary":"This paper claims that LLM-based planning fails under many constraints because existing methods either pile all constraints onto one step or let one subtask's mistakes contaminate the next. It proposes DPPM, a decomposition-plan-merge pipeline for travel planning: split the trip into four subtasks (transportation, accommodation, attractions, meals), have a separate local agent plan each subtask in parallel with only its own constraints, then merge the subplans in a fixed order with a verification-and-refinement loop that catches violations. On TravelPlanner, DPPM's final pass rate is 58.9% with Qwen2.5-32B versus 36.7% for LLM-Modulo, and on ChinaTravel-M it beats LLM-Modulo by 12.8 to 32.0 points across difficulty levels. The claim matters because it suggests that decomposing constraints by aspect, rather than by time, removes the cascade and overload that plague sequential and interleaved planners.","feed_headline":"Parallel subplans lift LLM travel planning by 22–32 points","feed_subtitle":"Split trips into four aspects, plan each in parallel, then merge to avoid constraint overload and cascading errors.","key_machinery":"The load-bearing mechanism is the constraint-aware decomposition of the overall trip into four aspect subtasks. Each subtask is handled by its own local agent with a custom prompt, produces multiple candidate subplans under high-temperature sampling, and is checked by a per-aspect critic function. The incremental merge then combines transportation with attractions first, adds accommodation, and finishes with meals, cycling through the Cartesian product of candidate subplans until a combination passes all checks. A verification and refinement module feeds critic output back to the generating agent for up to ten re-planning iterations, and the Global Constraint Instruction pushes each agent to keep budget and stay flexible enough for the other aspects.","core_discovery":"The central discovery is that a travel plan's constraints can be assigned to four independent aspects and planned in parallel. Each local agent receives only its aspect's constraints plus a global constraint instruction to leave room for other aspects; this avoids the heavy-constraint bottleneck. The subsequent incremental merge proceeds from transportation-plus-attractions to accommodation to meals, using the Cartesian product of candidate subplans to find a consistent combination, and a verification/refinement module checks hard and commonsense constraints at both local and merged stages. Reported results: final pass rate rises to 58.9%, 64.4%, 76.7%, and 87.2% on TravelPlanner across Qwen2.5-32B, Qwen2.5-72B, DeepSeek-V3, and DeepSeek-R1, versus 36.7%, 39.4%, and 57.8% for LLM-Modulo where it was run; ChinaTravel-M with Qwen2.5-32B reaches 87.1%, 70.7%, and 52.5% on easy, medium, and human difficulties, versus 55.8%, 38.7%, and 39.7% for LLM-Modulo.","pith_inferences":["An implication the authors leave implicit is that the same aspect-based decomposition should transfer to other constrained scheduling domains whose constraints factor into a few independent dimensions, such as event scheduling, logistics routing, or itinerary timetabling, provided cross-dimension interactions remain sparse; the paper only demonstrates travel.","Because DPPM gains most on macro pass rates, the harder failure mode for LLMs may be satisfying many constraints at once rather than reasoning about any one; a testable prediction is that the gap over sequential baselines widens as the number of constraints per sample grows.","The method is effectively a small implicit search over the Cartesian product of candidate subplans; a cheap extension would be to score candidate combinations by estimated cost before invoking the merge LLM, reducing the number of merge calls.","The ablation attributes most of the gain to verification and refinement, so a direct comparison that gives LLM-Modulo the same number of refinement iterations and candidate diversity would isolate how much the parallel decomposition itself contributes."],"forward_implications":["On TravelPlanner, DPPM reaches 58.9% final pass rate with Qwen2.5-32B and 76.7% with DeepSeek-V3, well above Direct, CoT, and LLM-Modulo baselines.","On ChinaTravel-M with Qwen2.5-32B, DPPM surpasses LLM-Modulo by 31.3 points on easy, 32.0 on medium, and 12.8 on human-level samples.","The verification and refinement module is the largest single contributor: removing it cuts final pass rate from 58.9% to 20.0% on TravelPlanner; removing the Cartesian product cuts it to 46.1% and removing the global-constraint instruction to 49.4%.","DPPM degrades more slowly as constraints accumulate: on hard 12-constraint samples it satisfies all constraints in twice as many plans as LLM-Modulo.","The authors state the method is tailored to multi-constraint tasks; for single-constraint planning it has no advantage, so the applicability is deliberately bounded."],"supporting_citations":[{"why":"Supplies the TravelPlanner benchmark, the Direct baseline, and the output-format and reference-information conventions the experiments follow.","marker":"Xie et al., 2024"},{"why":"Supplies the ChinaTravel benchmark that is re-formatted into ChinaTravel-M for evaluation.","marker":"Shao et al., 2024"},{"why":"Is the LLM-Modulo baseline and a source of the generate-test-refine idea that the verification module adapts.","marker":"Kambhampati et al., 2024"},{"why":"Provides the LLM-Modulo travel-planning implementation whose constraint evaluation functions DPPM reuses.","marker":"Gundawar et al., 2024b"},{"why":"Motivates the iterative self-feedback mechanism behind the verification and refinement module.","marker":"Madaan et al., 2023"},{"why":"Is the CoT baseline and represents the interleaved reasoning approach DPPM contrasts with.","marker":"Wei et al., 2022"},{"why":"Is the Plan-and-Solve sequential decomposition approach that suffers from cascading errors, which DPPM aims to avoid.","marker":"Wang et al., 2023"}],"fun_headline_variants":["Parallel subplans yield 22–32 point gains in LLM travel planning","Divide and conquer: parallel planning boosts LLM travel accuracy","LLM travel planning improved by decomposing constraints and merging","Constraint splitting and parallel planning lift LLM travel scores","Parallel merge strategy lifts LLM travel planning pass rates"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a travel plan's constraints split cleanly into four independent aspects, transportation, accommodation, attractions, and meals, so every cross-aspect conflict can be absorbed by the incremental merge and refinement; for queries where one constraint inherently belongs to two aspects, the subplans may be mutually incompatible from the start.","fun_headline_variants_meta":{"raw":{"variants":["Parallel subplans yield 22–32 point gains in LLM travel planning","Divide and conquer: parallel planning boosts LLM travel accuracy","LLM travel planning improved by decomposing constraints and merging","Constraint splitting and parallel planning lift LLM travel scores","Parallel merge strategy lifts LLM travel planning pass rates"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000771,"raw_usage":{"total_tokens":3408,"prompt_tokens":929,"completion_tokens":2479,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":545,"completion_tokens_details":{"reasoning_tokens":2395}},"tokens_in":545,"tokens_out":2479,"duration_ms":16756,"temperature":1.0,"reasoning_tokens":2395,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T11:17:53.372760+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a set of planning queries whose constraints are deliberately cross-aspect, for example 'the hotel must be within walking distance of the lunch restaurant, and the total daily walking distance under the attraction plan is capped', and count the final pass rate. If DPPM's advantage over a sequential planner with the same refinement budget disappears on such queries, the aspect-separability assumption is the failing premise.","supporting_citations":[],"review_version":1}