{"id":"0b6bdf60-0a27-4e9f-94fa-8171b749af1a","arxiv_id":"2501.14742","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A two-stage sequential exhaustive-search optimization found the same Pareto-optimal designs as a full factorial search on a multi-scale office building case study, with a 91.2% reduction in function evaluations and better performance than NSGA-II.","lead":"This paper tests a staged optimization method for building design, where one group of design choices is optimized at a time and only the best solutions carry into the next stage. On an office-building case study, the two-stage version found the same set of optimal designs as an exhaustive search while using 91.2% fewer simulation runs.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 100% equivalence claim rests on an unproven stage-wise Pareto preservation property: every global optimum's architecture must survive the baseline-HVAC Stage 1 filter, a condition the paper neither states nor separately verifies.","rationale":"The paper's strongest claim is the exact equivalence between the field-grouped iterative sequential search and the full factorial search. The only mechanism by which the sequential method can lose global optima is the stage-wise filter that discards all solutions not Pareto-optimal in the current stage under baseline settings for future-stage variables. This is precisely the reader's weakest assumption, and it is load-bearing: if the property fails, the method has no guarantee of finding global optima, and the reported 100% effectiveness would be coincidental rather than systematic. The paper provides no proof, no condition characterizing when the property holds, and no explicit verification that all global optima's architecture vectors survive Stage 1. However, the empirical evidence on the case study is substantial and internally consistent: the method uses exhaustive searches within each stage, is deterministic, and is benchmarked against a full factorial baseline. The lack of code/data limits independent replication but does not by itself invalidate the reported observations. The appropriate verdict remains CONDITIONAL: the central claim is credible for this case study but is conditional on a verified-but-unstated preservation property, and the generalizability claim exceeds the evidence. No change to the reader's verdict is needed.","tokens_in":23624,"tokens_out":5542,"duration_ms":67999,"concrete_test":"Using the already-generated exhaustive dataset, recompute the Stage 1 Pareto set P1 over all geometry+fabric combinations with HVAC/controls fixed at baseline, per Equation (1). Then check whether every full-factorial global optimum's architecture vector belongs to P1, for each of the four scales and four large-scale variants. If any global optimum's architecture is absent from P1, the sequential method could not have found that solution and the 100% claim fails. As an additional check, run the same field-grouped sequence in the reversed order (HVAC+controls first, architecture at baseline) and compare the final Pareto sets; if the two orders disagree, the equivalence is ordering-dependent and not a general property. Both checks are pure post-processing of existing simulation outputs and require no new simulations.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Equations (1)-(3) in Section 3 fix future-stage variables at baseline during earlier stages and restrict each stage to the previous stage's Pareto set. The paper asserts that 'the fixed baseline setup ensures that interdependencies among variables are always inherently considered,' but the equations do not establish this. An architecture that is dominated under the baseline HVAC/control configuration is discarded in Stage 1 and can never be revisited, even if it becomes globally optimal when paired with a non-baseline HVAC/control combination. The reported 100% effectiveness therefore requires that, for every full-factorial global optimum in every tested scale and variant, the architecture vector is non-dominated in Stage 1 under baseline HVAC/controls. That is a strong, dataset-specific condition on the objective landscape; the paper does not state it, prove it, or check it independently of the final equality. If the condition fails on any tested formulation, the central claim 'same Pareto optimal solutions as full factorial search' is false even as an empirical statement. If it holds on these datasets, the mechanism is still not guaranteed by construction, so the discussion's generalization to other building types, climates, and objectives goes beyond what the evidence supports. The concern is not about the reported numbers being miscalculated; it is that the core mechanism that makes the method efficient is also the unexamined assumption that makes it reliable.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a sequential, stage-wise multi-objective optimization method for building performance design, in which each stage performs a full factorial (exhaustive) search over a subset of design variables while later-stage variables are held at baseline values, and only Pareto-optimal solutions from a stage are passed forward. The method is evaluated on an office-building case study with 11 design variables spanning geometry, fabric, HVAC system, and controls, at four problem scales (874 to 1,036,800 options) and four variations of large-scale boundary conditions. Twenty-four configurations are tested, varying starting bounds, grouping strategy, and initial versus iterative runs. The headline result is that the field-grouped, iterative two-stage configuration identifies the same Pareto-optimal solutions as the full factorial search in all tested scales and variants, with 100,700 function evaluations versus 1,036,800, a 91.2% reduction, and outperforms NSGA-II at equal evaluation budgets. The paper also provides pseudocode, a detailed simulation workflow, and supplementary sensitivity analysis of design variables.","tokens_in":23884,"tokens_out":3853,"duration_ms":47756,"significance":"If the reported result holds, the paper makes a useful contribution to simulation-based building optimization: a deterministic, tuning-free sequential procedure that matches exhaustive search on a nontrivial problem class would be practically attractive. The strengths of the study are that the effectiveness claim is benchmarked against an independent full-factorial ground truth, the sequential method itself has no fitted parameters, the simulation pipeline is described in enough detail to be reproduced, and Table 5 reports internally consistent counts across scales and starting points. The comparison with NSGA-II is informative, though the NSGA-II configuration is less rigorously justified. The main limitation is that the 100% reliability claim rests on an unproven structural property of the optimization landscape: stage-one dominance under baseline HVAC/control values must imply that no discarded architecture can become globally optimal once those later-stage variables vary.","major_comments":[{"comment":"The central reliability claim depends on a stage-wise Pareto preservation property that is nowhere stated, proved, or separately verified. Equations (1)-(3) fix future-stage variables at baseline during earlier stages and restrict each stage to the previous stage's Pareto set. Consequently, an architectural solution that is dominated under the baseline HVAC/control configuration is discarded in Stage 1 and can never be revisited, even if it would be globally optimal when paired with a non-baseline HVAC/control combination. The paper's assertion, immediately after Figure 1, that 'the fixed baseline setup ensures that interdependencies among variables are always inherently considered' is not established by Eqs. (1)-(3). For the reported 100% effectiveness to hold empirically, every full-factorial global optimum at every tested scale and variant must have its architecture vector survive the Stage 1 filter under baseline HVAC/controls. The authors should either provide conditions under which this preservation holds or report an explicit check: for each full-factorial global optimum, verify that its architecture prefix is non-dominated in Stage 1. Without such a check, the mechanism that creates the computational saving is also the unexamined assumption on which the reliability claim rests.","section":"Section 3, Eqs. (1)-(3)"},{"comment":"The effectiveness metric in Eq. (4) is a count ratio, not a set-equality measure. Table 5 reports only the number of global optima found, so a configuration that finds 28 of 28 optima but a different set of 28 solutions would receive 100% effectiveness. The text in Sections 5.1.1, 5.2.1, 5.3, and 6 nevertheless claims that the field-grouped iterative search 'identified the same Pareto optimal solutions as the full factorial search.' The supporting evidence in Figures 7 and B.3 is qualitative. To substantiate the 'same solutions' claim, the authors should report an explicit set-overlap metric, such as the fraction of full-factorial Pareto solutions that are exactly matched in the variable space, or provide the complete list of matched solution indices. This is load-bearing because the abstract and conclusions repeatedly assert set identity, not just count equality.","section":"Section 4.1.1, Eq. (4), and Table 5"},{"comment":"The efficiency comparison with NSGA-II is weakened by the tuning and selection protocol. Section 4.2 states that NSGA-II was run 20 times and only the four runs with the highest performance were used for comparison. This best-of-20 selection gives NSGA-II an advantage rather than a disadvantage, but it also means the comparison is not a standard single-run or fixed-budget assessment, and the hand-chosen population size, crossover probability, and mutation probability (Table 2) are not tested for sensitivity. The central 100%-versus-full-factorial claim does not depend on the NSGA-II comparison, but the paper's broader conclusion that the sequential approach is more efficient than NSGA-II should be phrased more cautiously, or supplemented with a sensitivity analysis over NSGA-II operator settings and a clear statement of how the best-of-20 selection affects the reported 73.5% figure.","section":"Section 4.2 and Section 5.4"},{"comment":"The generalization claims exceed the evidence. Section 6 states that the approach 'can be applied to diverse building types, climate conditions, and performance objectives,' and Section 7 repeats this as a conclusion. The empirical support is a single open-plan office in Nottingham with two objectives, 11 design variables, and four boundary-condition variants. The authors should either explicitly limit the conclusions to the tested problem class or provide a constructive argument, beyond the unproven baseline-preservation assertion, for why the stage-wise dominance filter is expected to preserve global optima under different objective definitions and coupling structures between stages.","section":"Section 6 and Section 7"}],"minor_comments":[{"comment":"The text contains a typo: 'This represents a shit from conventional multi-objective optimization' should read 'shift.'","section":"Section 2"},{"comment":"The placeholder 'Figure x' appears in the sentence describing jEPlus; this should be replaced with the actual figure number.","section":"Section 4.5"},{"comment":"The final sentence of the first paragraph refers to 'the very large problem formulation,' but the tables and earlier text label the largest scale as 'large.' Please align the terminology.","section":"Section 5.2.2"},{"comment":"The sentence beginning 'The iterative run increased the deviation in effectiveness across variations of problem formulations of the ungrouped and element grouped search increased to 12.3% and 20.5%, respectively' contains a duplicated verb; rewording would clarify the reported deviations.","section":"Section 5.3"}],"recommendation":"major_revision","confidential_remarks":"The core issue is not a miscalculation of the reported numbers, which are internally consistent, but the missing verification of the stage-wise Pareto preservation condition that underpins the 100% reliability claim. If the authors add an explicit empirical check or a proof of the preservation condition, and tighten the set-equality reporting, I would view the paper as suitable for publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nQuick take: this paper deserves a real referee. The specific result—a two-stage, field-grouped sequential exhaustive search exactly recovering the full-factorial Pareto set on an office-building problem, with a 91.2% evaluation reduction—is new as far as I know, and the reported numbers are internally consistent. The effectiveness metric against an independent full-factorial ground truth is the right way to benchmark, and the method has no fitted parameters, so the central claim doesn't depend on tuning choices.\n\nWhat the paper does well: it tests 24 configurations across four scales and four variants, reports exact counts of global optima, and is transparent about where the search and global optima differ. The NSGA-II comparison is the weakest part—best-of-20 selection and hand-picked parameters without error bars make the 'outperforms NSGA-II' headline softer than the abstract suggests—but that comparison isn't load-bearing for the main claim.\n\nThe real soft spot is the stage-wise Pareto-preservation assumption. Equations (1)–(3) fix future-stage variables at baseline and restrict each stage to the previous stage's Pareto set. The paper asserts that this 'ensures that interdependencies among variables are always inherently considered,' but it doesn't. An architecture that is dominated under baseline HVAC/controls is discarded permanently, even if it would be globally optimal paired with a non-baseline HVAC/control combination. The 100% result therefore requires that every full-factorial global optimum's architecture survive the Stage-1 filter. That's a strong, dataset-specific condition. The paper never states it, proves it, or checks it separately; it just observes the final equality. On the case study, the equality seems to hold—I don't see evidence the numbers are wrong—but the mechanism that makes the method efficient is the same unexamined assumption that makes it reliable. The discussion's generalization to other building types, climates, and objectives goes beyond what the evidence supports.\n\nAlso, no code or data are shipped, which matters for a method paper that claims full reproducibility. The pseudocode is fine, but the exhaustive dataset is the crux.\n\nBottom line: the paper is a credible empirical contribution that should go to review, probably with conditional acceptance. The fixes are addressable: restate and test the preservation condition on the existing data (at minimum check how many discarded architectures become non-dominated after stage 2), add error bars to the NSGA-II comparison, ship the dataset, and temper the generalizability claims. I'd bring it to a reading group if building optimization is on the agenda, and I'd cite it as a benchmark with the caveat.\n\nRecommendation: send to peer review.","headline":"Worth refereeing: the 100% result is a solid empirical finding on this case study, but the efficiency mechanism rests on an unstated stage-wise Pareto-preservation assumption that limits generalization.","tokens_in":24411,"tokens_out":2843,"would_cite":true,"duration_ms":30890,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Two-stage sequential optimization matches exhaustive search for building performance design, using 91.2% fewer evaluations.","keywords":["sequential optimization","building performance simulation","multi-objective optimization","Pareto optimality","full factorial search","NSGA-II","building design","computational efficiency"],"falsifier":"Construct a building model where one envelope option is dominated under the baseline HVAC system but, paired with a different HVAC system, gives the best joint objective; if the field-grouped sequential search then misses that global Pareto point, the baseline-fixing assumption fails. A practical version of this test is to run the same two-stage search on a naturally ventilated or mixed-mode building, where envelope and controls interact strongly, and compare its output to an exhaustive search of all combinations.","tokens_in":1608,"feed_emoji":"🏢","tokens_out":1684,"duration_ms":93294,"temperature":0.7,"pith_summary":"This paper claims that a deterministic, tuning-free sequential optimizer can replace exhaustive search for multi-objective building performance design without sacrificing any Pareto-optimal solutions. The proposed two-stage 'field-grouped' process first optimizes building geometry and fabric together while HVAC and controls are held at baseline values, then optimizes the HVAC and control variables with the architecture Pareto set carried forward, and then repeats the whole sequence starting from each optimal solution found in the first pass. On an office-building case study spanning 874 to 4,147,200 design options, this procedure found exactly the same Pareto-optimal solutions as a full factorial search across all problem scales and formulation variants, using 100,700 function evaluations instead of 1,036,800 (a 91.2% reduction). With the same evaluation budget, NSGA-II found only 73.5% of the global optima. If the result transfers to other buildings, it gives practitioners a low-cost route to an exact optimal design set rather than an approximate evolutionary one.","feed_headline":"Two-stage search matches exhaustive search at a tenth of the cost","feed_subtitle":"A deterministic method found every optimal design on four problem scales, beating NSGA-II on the same budget.","key_machinery":"The load-bearing mechanism is a sequence of Pareto-filtered exhaustive searches. At stage $z$, variables belonging to later stages are fixed to baseline values (Equation 1); the current stage's grouped variables are enumerated over all combinations; and only the non-dominated set $P_z$ from that exhaustive search is passed to the next stage (Equations 2 and 3). Grouping variables by field yields two stages—building geometry plus fabric, then HVAC system plus controls—and the iterative run restarts the entire sequence from each Pareto-optimal solution found in the initial run, which is what lets the search escape fronts caused by the initial baseline-fixing and expand the set of optima found. The exhaustive within-stage search provides the guarantees of full-factorial exploration, while the grouping and iteration provide the breadth needed to respect cross-field dependencies.","core_discovery":"The central claim is that grouping the eleven design variables into two fields—architecture (building geometry and fabric) and engineering (HVAC system and controls)—and optimizing them sequentially, with an exhaustive full-factorial search inside each stage and an iterative restart from the stage-one Pareto set, recovers the global Pareto frontier of the joint problem. The authors verify this by simulating all combinations of an open-plan office in Nottingham and comparing 24 configurations of their algorithm against the full-factorial benchmark. The field-grouped iterative configuration achieved 100% effectiveness across the very small, small, medium, large, and four large-scale problem variants, matching every global optimum; in the large problem it found all 28 global optima with 100,700 evaluations versus the benchmark's 1,036,800. The paper interprets the result as showing that carrying only non-dominated solutions from stage to stage preserves the interactions that matter, while the baseline-fixed setting accounts for interdependencies with the variables optimized later. It also stresses that the deterministic full-factorial steps make repeated runs identical, in contrast to the stochastic NSGA-II, which reached 73.5% effectiveness on the same budget.","pith_inferences":["The baseline-fixing assumption is likely to be violated when cross-field interactions are strong, such as in naturally ventilated or mixed-mode buildings where envelope and controls jointly drive comfort; applying the method to such a case is a natural test.","The success of the field-grouped search suggests a practical workflow in which architects optimize the envelope early against a fixed system, then engineers optimize the system given the envelope; this lowers barriers to optimization-based design but inherits the same assumption.","The sequential Pareto set could serve as a cheap deterministic warm start or surrogate target for other optimization algorithms, since it approximates the true frontier without requiring parameter tuning.","The 91.2% figure counts function evaluations, not wall-clock time; if simulation time per evaluation varies with the design, a runtime comparison under identical parallel resources would be a useful extension."],"forward_implications":["If the claim is correct, building designers can obtain the same global Pareto set as exhaustive search with roughly one-tenth of the simulation budget.","The deterministic, tuning-free nature of the sequential search means repeated runs give identical results, removing the run-to-run variability of population-based metaheuristics.","For this problem class, the field grouping—two stages rather than one variable per stage—is the configuration that preserves reliability, so the choice of grouping is a methodological decision, not a convenience.","The efficiency advantage grows with problem size, since computational load relative to the full factorial search fell from about 30% at the very small scale to 9% at the large scale.","With the same function-evaluation budget, the sequential search finds a larger fraction of global optima than NSGA-II on this case study."],"supporting_citations":[{"why":"Establishes exhaustive full-factorial search as the guaranteed benchmark for global Pareto optima, against which the sequential method is measured.","marker":"[17]"},{"why":"Defines NSGA-II, the evolutionary algorithm used as the efficiency baseline with the same function-evaluation budget.","marker":"[42]"},{"why":"Precedent for decoupling envelope and HVAC optimization in two stages, motivating the field-grouped split.","marker":"[9]"},{"why":"Precedent for a multi-stage optimization method that cuts computational cost while finding effective building designs.","marker":"[10]"},{"why":"Precedent for separating design-level and operational variables in a multi-level optimization framework.","marker":"[12]"},{"why":"The authors' earlier comparison of sequential versus simultaneous whole-building optimization, which this study extends with a full benchmark.","marker":"[7]"}],"fun_headline_variants":["Sequential search finds all optimal designs at 9% of the cost","Deterministic two-stage method beats NSGA-II on same budget","Sequential optimization matches exhaustive search exactly","100% global optima at 10% of the cost","Two-stage search: same answers, 91% less computation"],"cache_read_input_tokens":26496,"weakest_assumption_plain":"The method assumes that a design discarded because it is dominated at stage 1—when HVAC and controls are fixed at baseline values—could not become globally optimal once the HVAC and control variables are allowed to vary.","fun_headline_variants_meta":{"raw":{"variants":["Sequential search finds all optimal designs at 9% of the cost","Deterministic two-stage method beats NSGA-II on same budget","Sequential optimization matches exhaustive search exactly","100% global optima at 10% of the cost","Two-stage search: same answers, 91% less computation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000828,"raw_usage":{"total_tokens":3681,"prompt_tokens":1072,"completion_tokens":2609,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":688,"completion_tokens_details":{"reasoning_tokens":2525}},"tokens_in":688,"tokens_out":2609,"duration_ms":19485,"temperature":1.0,"reasoning_tokens":2525,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T16:33:17.185549+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Construct a building model where one envelope option is dominated under the baseline HVAC system but, paired with a different HVAC system, gives the best joint objective; if the field-grouped sequential search then misses that global Pareto point, the baseline-fixing assumption fails. A practical version of this test is to run the same two-stage search on a naturally ventilated or mixed-mode building, where envelope and controls interact strongly, and compare its output to an exhaustive search of all combinations.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Establishes exhaustive full-factorial search as the guaranteed benchmark for global Pareto optima, against which the sequential method is measured."},{"cited_title":"Hasan, K","cited_arxiv_id":null,"evidence_quote":"Precedent for a multi-stage optimization method that cuts computational cost while finding effective building designs."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The authors' earlier comparison of sequential versus simultaneous whole-building optimization, which this study extends with a full benchmark."}],"review_version":1}