Rapid Experimentation with Python Considering Optional and Hierarchical Inputs

Neil Ranly; Torrey Wagner

REVIEW 3 major objections 5 minor 38 references

Rapid Experimentation with Python Considering Optional and Hierarchical Inputs

T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read raxpy introduces five algorithms for space-filling designs over optional and hierarchical inputs, and its FSS-LHD-VP-MP variant ranks first on minimum interpoint distance across all tested spaces and sizes.

desk verdict A genuinely useful open-source tool for a real gap, but the algorithm comparison is built on a target allocation that ignores hierarchical dependencies, so the 'improved' claim needs revision before it stands. read the letter →

arxiv 2501.03398 v1 pith:VEC56SO5 submitted 2025-01-06 cs.MS stat.ME

Neil Ranly , Torrey Wagner This is my paper · ORCID

classification cs.MSstat.ME MSC 62K05

keywords space-fillingdesigncomputerexperimentsoptionaldimensionshierarchicalmaximumprojectionLatinhypercubePythonannotationsof

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces raxpy, a Python package that turns annotated Python functions into space-filling experimental designs, and proposes five algorithms for input spaces where some dimensions are optional and others are hierarchical. The central claim is that these algorithms create improved space-filling designs for such spaces, with the FSS-LHD-VP-MP variant ranking first on minimum interpoint distance and in the top three on every metric considered in the numerical study. A sympathetic reader should care because computer simulations, hyperparameter searches, and similar experiments routinely have optional or hierarchical inputs, while standard space-filling methods assume every input is always active. The package also automates the mapping from function signature to design to execution, removing a manual step in exploration experiments.

What carries the argument

The load-bearing machinery is the full-sub-space decomposition: feasible combinations of active dimensions are enumerated, target point allocations are computed from per-dimension null_portion values via the product formula in Eq. (7), and designs are built and evaluated sub-space by sub-space. A null-aware distance metric treats a null value as distance 1 from any active value and distance 0 from another null value, which lets criteria like minimum interpoint distance and MaxPro be applied to spaces with optional dimensions. The proposed algorithms layer these ideas: FSS-LHD and its value-pool variant build separate designs per full-sub-space, while TT-LHD and P-LHD flatten or traverse the hierarchy, and a simulated-annealing MaxPro variant (the -MP suffix) re-optimizes columns while preserving the design's full-sub-space allocation structure.

What would settle it

Run the same 12-algorithm comparison on spaces with real-valued optional dimensions whose null_portion is set to 0.05 or 0.5 instead of the default 0.25; if FSS-LHD-VP-MP no longer ranks near the top on minimum interpoint distance or star discrepancy, the claimed improvement depends on the heuristic rather than on the structural use of full-sub-spaces.

Watch

Extended reading notes

Core claim

The paper's discovery is that optional and hierarchical input spaces can be handled by decomposing the design into full-sub-spaces (FSS), allocating the trial points to those sub-spaces according to null-portion heuristics, filling each sub-space with Latin hypercube designs, pooling values, and optionally post-processing with simulated-annealing MaxPro optimization. Across four input spaces of increasing complexity and three design sizes, with 30 replications per combination, FSS-LHD-VP-MP achieves an average rank of 1.0 on minimum interpoint distance, 2.5 on average single-dimension projection distance, 2.5 on star discrepancy, and 4.25 on weighted full-sub-space discrepancy. The authors argue that algorithms which allocate points by full-sub-space match the target allocations derived from null-portion attributes, whereas random, tree-traversal, and whole-projection designs often diverge from those targets.

Load-bearing premise

The load-bearing premise is the default null_portion heuristic: every real-valued optional dimension is assumed to have complexity 3, so its null region occupies one quarter of that dimension; if the actual probability that an optional input is absent differs, the target allocations change and the reported algorithm rankings may not transfer.

Editorial extensions

If this is right

raxpy can take a Python function annotated with typing.Optional, typing.Union, and dataclasses and automatically produce and execute a full space-filling design with no separate space-specification file.
FSS-LHD-VP-MP gives the best average minimum interpoint distance across all tested input spaces and design sizes, and it ranks in the top three algorithms on average projection distance, star discrepancy, and weighted discrepancy.
Adding MaxPro optimization raises minimum interpoint distance and improves star discrepancy, but it can worsen the weighted full-sub-space discrepancy, so users who expect to focus on a single sub-space after exploration may prefer not to apply it.
Algorithms that build designs full-sub-space by full-sub-space match the target point allocations derived from null_portion attributes, while random, tree-traversal, and whole-projection designs often miss those targets and bias the resulting evaluation.
Because raxpy supports parallelism and distributed execution, the generated designs can be run on expensive external simulations, command-line programs, and web services as well as on ordinary Python functions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Beyond the paper: the same full-sub-space decomposition could be embedded in sequential or Bayesian optimization over hierarchical hyperparameter spaces, using a prior over configurations rather than a null_portion heuristic to set target allocations.
Beyond the paper: if the evaluation criterion were downstream surrogate model accuracy instead of geometric space-filling properties, the ranking of the proposed algorithms could change; this is a testable follow-up that the paper does not run.
Beyond the paper: when a user knows the true probability that an optional input is null, explicitly specifying null_portion or direct full-sub-space allocations should replace the default heuristic, and the algorithm ranking should be re-checked under those user-specified targets.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

You should know this paper is worth a look, but not for the reason the abstract says. The package raxpy addresses a real, underserved niche: space-filling experimental designs for Python functions with optional and hierarchical inputs. The annotation-based space definition is a nice idea, the code is open source under MIT, and the paper honestly reviews the existing landscape. That part is solid.

What is actually new: the combination of LHD-style value pooling with MaxPro optimization for hierarchical spaces, and the explicit handling of nullable dimensions in the metrics. The authors correctly identify that existing tools like ConfigSpace and scipy don't provide space-filling designs for these structured spaces. The software contribution is real and likely useful to people running simulation studies with conditional parameters.

Now the soft spot, and it is a load-bearing one. Equation (7) sets target point allocations as a product of per-dimension non-null probabilities, which assumes the null status of child dimensions is independent of their parent's activation. In the paper's own Ssimple example, x4 is a non-optional field of HierarchicalFactorOne, so it is automatically active whenever x3 is active. The product formula assigns probability mass to infeasible sub-spaces like {1,2,3} without x4, and it over-allocates to feasible sub-spaces by ignoring the conditional structure. The FSS-based algorithms are then explicitly designed to match those misspecified targets, and the evaluation rewards them for doing so. The asterisked algorithms in Table 1 are penalized partly for respecting the actual hierarchy rather than the incorrect target. This is not a minor quibble; it directly undermines the headline claim that the proposed algorithms produce 'improved' designs.

The secondary issues are the arbitrary null_portion heuristic (Chapter 3.3), which sets complexity to 3 for real dimensions and thus fixes the null region at a quarter of the dimension, and the lack of any confidence intervals or significance tests on the rankings. The evaluation is entirely internal, using self-defined metrics on synthetic spaces. These would be fixable with a revised evaluation, but the independence assumption in Eq. (7) requires a more fundamental rethinking of the target allocation.

Who is this for? Practitioners who want to run space-filling experiments on Python functions with optional and hierarchical inputs will find the package useful. The paper deserves peer review because the software and the identified gap are real, but the empirical claims need major revision. I would recommend the editor send it out, with the expectation that the authors rework the allocation targets to respect the hierarchy and add proper statistical comparisons.

Referee Report

3 major / 5 minor

Summary. raxpy, an open-source Python package, introspects annotated function signatures to define input spaces and automatically constructs space-filling designs for optional and hierarchical dimensions. The paper defines seven extended evaluation criteria (M_ocov, M_idis, M_adis, M_wdsr, M_sdsr, a MaxPro variant, and an FSS allocation-difference measure), proposes five algorithms (FSS-LHD, FSS-Random, FSS-LHD-VP, TT-LHD, P-LHD), and adds a simulated-annealing MaxPro post-optimization. It then compares twelve algorithm variants on four synthetic input spaces (basic, simple, modest, complex) at three design sizes with 30 replications each. Table 1 reports average rankings and the authors conclude that FSS-LHD-VP-MP is the best for M_idis and among the top three on all four metrics, supporting the claim that the proposed algorithms improve space-filling designs for optional and hierarchical inputs.

Significance. The software contribution is genuine: raxpy is MIT-licensed, documented, and lowers the barrier to space-filling experiments on Python-callable functions; the paper also identifies a real gap in the literature and proposes a concrete set of metric extensions. However, the numerical evidence for the headline claim is currently internal: the FSS-based algorithms are constructed to match target allocations computed in Section 3.3, and the evaluation metrics reward that match. In hierarchical spaces the target calculation is misspecified (major comment 1), so the reported rankings do not yet establish improved space-filling performance. The significance would be high if the comparisons are re-run with correctly conditioned targets and validated against independent criteria.

major comments (3)

[Section 3.3, Eq. (7)] Equation (7) cannot define a probability distribution over full sub-spaces as written. It multiplies only over dimensions in Do and omits null-portion factors for optional dimensions outside Do. In Sbasic, with alpha_bar=0.25, the raw targets for {1}, {1,2}, {1,3}, {1,2,3} would be 1, 0.75, 0.75, and 0.5625, which neither sum to 1 nor match the independent probabilities 0.0625, 0.1875, 0.1875, and 0.5625. Additionally, the product treats child null status as independent of parent activation; in Ssimple, x4 is a non-Optional field of HierarchicalFactorOne, so whenever parent x3 is active, x4 is non-null with probability 1, not 1-alpha_bar_4, and the target for {1,2,3,4} should therefore be 0.75*0.75*1=0.562 rather than 0.75^3=0.422. Since FSS-based algorithms are explicitly designed to match these targets and M_wdsr plus the FSS allocation-difference metric reward that match, the 'improved' ranking in Table 1 is partly an artifact of the misspecified target. Please recompute targets as probabilities over the feasible FSS tree (using the constraint set P), or require user-supplied allocations, and re-run the comparisons.
[Section 4.2, Table 1] The M_wdcr column in Table 1 (presumably M_wdsr from Section 3.1) is compared across algorithms with different FSS point allocations, which violates the restriction stated when M_wdsr is introduced: 'We only employ this metric for design comparisons given designs with the same FSS point allocations.' The caption itself marks Random*, TT-LHD*, and P-LHD* as having 'common divergence from the target number of points for each full-sub-space causing measurement biases,' yet their M_wdcr values are still reported and ranked. FSS-based algorithms match the target by construction, so this column partially measures agreement with a heuristic target rather than an independent space-filling property. If M_wdsr is used across algorithms, the comparison should be restricted to designs with identical FSS allocations or adjusted by conditioning on the realized FSS.
[Section 4.1] Table 1 reports average ranks over 30 replications without confidence intervals, standard errors, or significance tests. With 12 algorithms, four input spaces, and three sample sizes, differences such as average ranks of 1.0 versus 2.33 can easily be within replication noise, and Figure 5 shows visible variability. In addition, the default null_portion=1/4 heuristic in Section 3.3 sets the target allocations, and no sensitivity analysis is reported for other choices of null_portion or for user-supplied allocations. Please add uncertainty quantification and a robustness check across null_portion values, or temper the conclusion to claim a comparison only for the particular default configuration.

minor comments (5)

[Equation (3)] In the first case of dist_k, the expression reads |xik - xik|; it should be |xik - xjk|, otherwise the term is identically zero.
[Section 4.1 and Table 1] Metric names are inconsistent: Section 3.1 defines M_sdsr and M_wdsr, while Section 4.1 and Table 1 use M_sdcr and M_wdcr. Please unify the notation.
[Figure 5 caption] The caption refers to 'FSS-LHP-VP-MP'; this should be 'FSS-LHD-VP-MP'.
[Section 3.3] The sentence 'one unit of the complexity represents the null region' is ambiguous; near the end of the paragraph it would be clearer to state explicitly that for real optional dimensions the default is null_portion = 1/4 and that this value drives the FSS allocations in Eq. (7).
[References] The bibliography entry for Lindauer et al. (2019) lists the tool as 'Boah', but the text refers to it as the ConfigSpace package; please correct the title.

Circularity Check

1 steps flagged · score 4.0 of 10

FSS algorithms are constructed to match the Eq. (7) target allocation, and the same target weights the M_wdsr criterion and defines the allocation-difference metric, making part of the ranking self-referential; the M_idis result remains independent evidence.

self definitional [Section 3.1 (M_wdsr definition), Section 3.3 Eq. (7), Section 4.2 (Table 1 and Figure 4 discussion)]
"We weight and sum the discrepancies of the sub-designs, with weights representing the target number of points allocated to each FSS over n. ... the percentage of points to allocate to each SDo is computed as: Prod_{k in Do}(1 - alpha_k). ... The FSS-based algorithms, by design, create designs that match the target point allocations."

The same Eq. (7) target is used twice: once as the allocation rule inside FSS-LHD and FSS-LHD-VP, and once as the weighting function for M_wdsr and as the reference for the full-sub-space allocation-difference metric. Consequently, the low allocation differences and favorable M_wdsr rankings of the FSS algorithms are consequences of their construction rather than independent empirical findings. The default null_portion is itself heuristic (real-valued complexity set to 3 plus 1 null unit, giving alpha=1/4), so this portion of the comparison measures self-consistency with the heuristic, not externally validated space-filling quality. The central claim is not wholly circular because M_idis, M_adis, and M_sdcr do not depend on Eq. (7).

full rationale

The paper's main evaluation loop contains a partially self-referential component. Section 3.3 defines target full-sub-space point percentages via Eq. (7), a product of per-dimension non-null fractions. Section 3.2 states that FSS-based algorithms use these targets to allocate points, and Section 4.2 confirms that 'The FSS-based algorithms, by design, create designs that match the target point allocations.' Separately, Section 3.1 defines M_wdsr with 'weights representing the target number of points allocated to each FSS over n,' i.e., the same Eq. (7) targets, and Figure 4's allocation-difference metric uses the same targets as the reference. Thus the M_wdsr and allocation-difference rankings partly reduce to agreement with an internally chosen heuristic. This is a genuine partial circularity in the evaluation, not a fabricated one. However, the paper also reports M_idis, M_adis, and M_sdcr, which do not use Eq. (7); the best overall algorithm, FSS-LHD-VP-MP, ranks first on M_idis and highly on M_adis/M_sdcr, so the headline claim has independent content. Separately, the skeptic's concern that Eq. (7) assumes independence of null events across hierarchical dimensions is a correctness/validity issue rather than a circularity: it means the target allocations are misspecified for Ssimple and other hierarchical spaces, but that does not make the derivation circular. No self-citations are load-bearing, and no uniqueness theorem or ansatz is imported from the authors' prior work. Overall score 4 reflects the partial reduction of some evaluation metrics to an algorithmic design choice while the central claim retains independent support.

Assumptions & free parameters 2 free parameters · 6 assumptions · 0 invented entities

The paper's contribution rests on several modeling choices presented as definitions: the null_portion default, the extended metrics, and the distance encoding. These are not externally benchmarked, so they carry the burden of the 'improved' claim.

free parameters (2)

null_portion heuristic constant = 0.25 for real-valued dimensions (1/(3+1))
Chosen by hand in Section 3.3; sets the target FSS allocation via Eq (7) and therefore determines which algorithms are judged better.
MaxPro complexity estimate for optional/parent dimensions = For finite sets, |D_k|+1; for real dims, 3+1
Used in Eq (6) to avoid division by zero and weight differences; chosen by hand.

assumptions (6)

domain assumption Space-filling designs improve exploration of computer models
Section 1 motivates the entire work on this premise; no external validation that these designs lead to better downstream analysis.
ad hoc to paper The seven proposed metrics are appropriate criteria for evaluating designs in optional/hierarchical spaces
Section 3.1 introduces the metrics; their validity is asserted rather than benchmarked against existing DOE criteria.
ad hoc to paper The distance encoding in Eq (3), where null-null pair contributes 0 and null-real pair contributes 1, captures meaningful separation
Equation 3 defines distances with arbitrary values for null interactions; different encodings could change design rankings.
domain assumption The full-sub-space power set PP(Doptional) is the correct discrete structure for optional dimensions
Section 3.1 defines FSSs and uses them for coverage and allocation targets; this is a modeling choice.
standard math The scipy.stats.qmc.LatinHypercube algorithm with centered discrepancy is a valid baseline TSFD
Section 3.2 uses this standard library as the underlying space-filling algorithm for all proposed designs.
ad hoc to paper Null values are treated as preceding 0 on the number line in M_sdsr
Equation 4 orders null before 0; this is a convention that affects discrepancy values.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Rapid Experimentation with Python Considering Optional and Hierarchical Inputs." pith.science (2026). https://pith.science/paper/VEC56SO5

@misc{pith2026250103398,
  author       = {Pith},
  title        = {Pith review of: Rapid Experimentation with Python Considering Optional and Hierarchical Inputs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VEC56SO5}},
  note         = {Machine review of arXiv:2501.03398}
}

read the original abstract

Space-filling experimental design techniques are commonly used in many computer modeling and simulation studies to explore the effects of inputs on outputs. This research presents raxpy, a Python package that leverages expressive annotation of Python functions and classes to simplify space-filling experimentation. It incorporates code introspection to derive a Python function's input space and novel algorithms to automate the design of space-filling experiments for spaces with optional and hierarchical input dimensions. In this paper, we review the criteria for design evaluation given these types of dimensions and compare the proposed algorithms with numerical experiments. The results demonstrate the ability of the proposed algorithms to create improved space-filling experiment designs. The package includes support for parallelism and distributed execution. raxpy is available as free and open-source software under a MIT license.

Figures

Figures reproduced from arXiv: 2501.03398 by the authors.

**Figure 1.** X to Xopt point mapping P ⊆ Dparent × D × Dp denote a set of hierarchical constraints such that if (p, c, v) ∈ P then the child dimension c for a point is constrained to be null if the parent dimension p’s value is not v. This implies c ∈ Doptional , ∀(p, c, v) ∈ P. The first criteria we consider is the extent of a design’s coverage over the different possible combinations of optional parameters. To compute, we defi… view at source ↗

**Figure 2.** Scatter-plots of the points from a random DOE (left) and a FSS-LHD-VP-MP DOE (right) [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗

**Figure 3.** Algorithm Space-Filling Results, normalized [PITH_FULL_IMAGE:figures/full_fig_p012_3.png] view at source ↗

Figures from the paper (2 more)

**Figure 4.** Figure 4: Algorithm Full-Sub-Space Point Allocation Differences from Target Counts (lower is better) [PITH_FULL_IMAGE:figures/full_fig_p013_4.png]

**Figure 5.** Figure 5: Algorithm Space-Filling Results, normalized [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

38 extracted references · 28 canonical work pages

[1]

Sanchez, Paul J

Susan M. Sanchez, Paul J. Sanchez, and Hong Wan. Work Smarter , Not Harder : A Tutorial on Designing and Conducting Simulation Experiments . In 2020 Winter Simulation Conference ( WSC ) , pages 1128--1142, Orlando, FL, USA, December 2020. IEEE. ISBN 978-1-72819-499-8. doi:10.1109/WSC48552.2020.9384057. URL https://ieeexplore.ieee.org/document/9384057/

arXiv 2020
[2]

Isaac Gym : High Performance GPU - Based Physics Simulation For Robot Learning , 2021

Viktor Makoviychuk, Lukasz Wawrzyniak, Yunrong Guo, Michelle Lu, Kier Storey, Miles Macklin, David Hoeller, Nikita Rudin, Arthur Allshire, Ankur Handa, and Gavriel State. Isaac Gym : High Performance GPU - Based Physics Simulation For Robot Learning , 2021. URL https://arxiv.org/abs/2108.10470. Version Number: 2

arXiv 2021
[3]

G. E. P. Box and H. L. Lucas. Design of Experiments in Non - Linear Situations . Biometrika, 46 0 (1/2): 0 77, June 1959. ISSN 00063444. doi:10.2307/2332810. URL https://www.jstor.org/stable/2332810?origin=crossref

work page arXiv 1959
[4]

Sanchez and Paul J

Susan M. Sanchez and Paul J. Sánchez. Better Big Data via Data Farming Experiments . In Andreas Tolk, John Fowler, Guodong Shao, and Enver Yücesan, editors, Advances in Modeling and Simulation , pages 159--179. Springer International Publishing, Cham, 2017. ISBN 978-3-319-64181-2 978-3-319-64182-9. doi:10.1007/978-3-319-64182-9_9. URL http://link.springer...

work page doi:10.1007/978-3-319-64182-9_9 2017
[5]

Design choice and machine learning model performances

Rosa Arboretti, Riccardo Ceccato, Luca Pegoraro, and Luigi Salmaso. Design choice and machine learning model performances. Quality and Reliability Engineering International, 38 0 (7): 0 3357--3378, November 2022. ISSN 0748-8017, 1099-1638. doi:10.1002/qre.3123. URL https://onlinelibrary.wiley.com/doi/10.1002/qre.3123

work page doi:10.1002/qre.3123 2022
[6]

Design of experiments and machine learning with application to industrial experiments

Roberto Fontana, Alberto Molena, Luca Pegoraro, and Luigi Salmaso. Design of experiments and machine learning with application to industrial experiments. Statistical Papers, 64 0 (4): 0 1251--1274, August 2023. ISSN 0932-5026, 1613-9798. doi:10.1007/s00362-023-01437-w. URL https://link.springer.com/10.1007/s00362-023-01437-w

work page doi:10.1007/s00362-023-01437-w 2023
[7]

Weese, Jonathan W

Kade Young, Maria L. Weese, Jonathan W. Stallrich, Byran J. Smucker, and David J. Edwards. A graphical comparison of screening designs using support recovery probabilities. Journal of Quality Technology, 56 0 (4): 0 355--368, August 2024. ISSN 0022-4065, 2575-6230. doi:10.1080/00224065.2024.2356127. URL https://www.tandfonline.com/doi/full/10.1080/0022406...

work page arXiv 2024
[8]

Roshan Joseph

V. Roshan Joseph. Space-filling designs for computer experiments: A review. Quality Engineering, 28 0 (1): 0 28--35, January 2016. ISSN 0898-2112, 1532-4222. doi:10.1080/08982112.2015.1100447. URL http://www.tandfonline.com/doi/full/10.1080/08982112.2015.1100447

work page arXiv 2016

Show all 38 references

[9]

Johnson, L.M

M.E. Johnson, L.M. Moore, and D. Ylvisaker. Minimax and maximin distance designs. Journal of Statistical Planning and Inference, 26 0 (2): 0 131--148, October 1990. ISSN 03783758. doi:10.1016/0378-3758(90)90122-B. URL https://linkinghub.elsevier.com/retrieve/pii/037837589090122B

1990
[10]

Art B. Owen. ORTHOGONAL ARRAYS FOR COMPUTER EXPERIMENTS , INTEGRATION AND VISUALIZATION . Statistica Sinica, 2 0 (2): 0 439--452, 1992. ISSN 10170405, 19968507. URL http://www.jstor.org/stable/24304869. Publisher: Institute of Statistical Science, Academia Sinica

1992
[11]

Numerical studies of space-filling designs: optimization of Latin Hypercube Samples and subprojection properties

G Damblin, M Couplet, and B Iooss. Numerical studies of space-filling designs: optimization of Latin Hypercube Samples and subprojection properties. Journal of Simulation, 7 0 (4): 0 276--289, November 2013. ISSN 1747-7778, 1747-7786. doi:10.1057/jos.2013.16. URL https://www.t...

2013 doi
[12]

Roshan Joseph, Evren Gul, and Shan Ba

V. Roshan Joseph, Evren Gul, and Shan Ba. Designing computer experiments with multiple types of factors: The MaxPro approach. Journal of Quality Technology, 52 0 (4): 0 343--354, October 2020. ISSN 0022-4065, 2575-6230. doi:10.1080/00224065.2019.1611351. URL https://www.tandfo...

2020
[13]

Fast flexible space‐filling designs with nominal factors for nonrectangular regions

Ryan Lekivetz and Bradley Jones. Fast flexible space‐filling designs with nominal factors for nonrectangular regions. Quality and Reliability Engineering International, 35 0 (2): 0 677--684, March 2019. ISSN 0748-8017, 1099-1638. doi:10.1002/qre.2429. URL https://onlinelibrary...

2019 doi
[14]

Fast Flexible Space - Filling Designs for Nonrectangular Regions

Ryan Lekivetz and Bradley Jones. Fast Flexible Space - Filling Designs for Nonrectangular Regions . Quality and Reliability Engineering International, 31 0 (5): 0 829--837, July 2015. ISSN 07488017. doi:10.1002/qre.1640. URL https://onlinelibrary.wiley.com/doi/10.1002/qre.1640

2015 doi
[15]

Optimal Noncollapsing Space - Filling Designs for Irregular Experimental Regions

Ray-Bing Chen, Chi-Hao Li, Ying Hung, and Weichung Wang. Optimal Noncollapsing Space - Filling Designs for Irregular Experimental Regions . Journal of Computational and Graphical Statistics, 28 0 (1): 0 74--91, January 2019. ISSN 1061-8600, 1537-2715. doi:10.1080/10618600.2018...

2019
[16]

Okolo, and Weihua Zhang

Zeping Wu, Donghui Wang, Wenjie Wang, Kun Zhao, Patrick N. Okolo, and Weihua Zhang. Space-filling experimental designs for constrained design spaces. Engineering Optimization, 51 0 (9): 0 1495--1508, September 2019. ISSN 0305-215X, 1029-0273. doi:10.1080/0305215X.2018.1542691....

2019
[17]

Robust design modeling and optimization for dealing with a non-convex design space

Akın Özdemir and Mehmet Turkoz. Robust design modeling and optimization for dealing with a non-convex design space. Computers & Industrial Engineering, 185: 0 109688, November 2023. ISSN 03608352. doi:10.1016/j.cie.2023.109688. URL https://linkinghub.elsevier.com/retrieve/pii/...

2023
[18]

Particle swarm optimization for searching efficient experimental designs: A review

Ping‐Yang Chen, Ray‐Bing Chen, and Weng Kee Wong. Particle swarm optimization for searching efficient experimental designs: A review. WIREs Computational Statistics, 14 0 (5): 0 e1578, September 2022. ISSN 1939-5108, 1939-0068. doi:10.1002/wics.1578. URL https://wires.onlineli...

2022 doi
[19]

Orthogonal uniform composite designs

Xue-Ru Zhang, Min-Qian Liu, and Yong-Dao Zhou. Orthogonal uniform composite designs. Journal of Statistical Planning and Inference, 206: 0 100--110, May 2020. ISSN 03783758. doi:10.1016/j.jspi.2019.08.007. URL https://linkinghub.elsevier.com/retrieve/pii/S0378375819300898

2020 doi
[20]

Peter Z. G. Qian. Sliced Latin Hypercube Designs . Journal of the American Statistical Association, 107 0 (497): 0 393--399, March 2012. ISSN 0162-1459, 1537-274X. doi:10.1080/01621459.2011.644132. URL http://www.tandfonline.com/doi/abs/10.1080/01621459.2011.644132

2012
[21]

Devon Lin

Xinwei Deng, Ying Hung, and C. Devon Lin. Design for computer experiments with qualitative and quantitative factors. Statistica Sinica, 2015. ISSN 10170405. doi:10.5705/ss.2013.388. URL http://www3.stat.sinica.edu.tw/statistica/J25N4/J25N414/J25N414.html

2015 doi
[22]

Crombecq, E

K. Crombecq, E. Laermans, and T. Dhaene. Efficient space-filling and non-collapsing sequential design strategies for simulation-based modeling. European Journal of Operational Research, 214 0 (3): 0 683--696, November 2011. ISSN 03772217. doi:10.1016/j.ejor.2011.05.032. URL ht...

2011 doi
[23]

Progressive Latin Hypercube Sampling : An efficient approach for robust sampling-based analysis of environmental models

Razi Sheikholeslami and Saman Razavi. Progressive Latin Hypercube Sampling : An efficient approach for robust sampling-based analysis of environmental models. Environmental Modelling & Software, 93: 0 109--126, July 2017. ISSN 13648152. doi:10.1016/j.envsoft.2017.03.010. URL h...

2017 doi
[24]

Parker, T.W

J.D. Parker, T.W. Lucas, and W.M. Carlyle. Sequentially extending space-filling experimental designs by optimally permuting and stacking columns of the design matrix. European Journal of Operational Research, 319 0 (2): 0 600--610, December 2024. ISSN 03772217. doi:10.1016/j.e...

2024 doi
[25]

Constrained space-filling and non-collapsing sequential design of experiments and its application for the lightweight design of cylindrical stiffened shells

Zhixiang Wang, Dapeng Zhang, Yongjun Lei, Zeping Wu, Jie Wang, Xing OuYang, and Jun Wang. Constrained space-filling and non-collapsing sequential design of experiments and its application for the lightweight design of cylindrical stiffened shells. Structural and Multidisciplin...

2021 doi
[26]

Anderson‐Cook, Miranda Martin, and Towfiq Ahmed

Lu Lu, Christine M. Anderson‐Cook, Miranda Martin, and Towfiq Ahmed. Practical choices for space‐filling designs. Quality and Reliability Engineering International, 38 0 (3): 0 1165--1188, April 2022. ISSN 0748-8017, 1099-1638. doi:10.1002/qre.2884. URL https://onlinelibrary.w...

2022 doi
[27]

mlrMBO : A Modular Framework for Model - Based Optimization of Expensive Black - Box Functions , 2017

Bernd Bischl, Jakob Richter, Jakob Bossek, Daniel Horn, Janek Thomas, and Michel Lang. mlrMBO : A Modular Framework for Model - Based Optimization of Expensive Black - Box Functions , 2017. URL https://arxiv.org/abs/1703.03373. Version Number: 3

2017 arXiv
[28]

Algorithms for hyper-parameter optimization

James Bergstra, R\' e mi Bardenet, Yoshua Bengio, and Bal\' a zs K\' e gl. Algorithms for hyper-parameter optimization. In J. Shawe-Taylor, R. Zemel, P. Bartlett, F. Pereira, and K.Q. Weinberger, editors, Advances in Neural Information Processing Systems, volume 24. Curran Ass...

2011
[29]

VolcanoML : speeding up end-to-end AutoML via scalable search space decomposition

Yang Li, Yu Shen, Wentao Zhang, Ce Zhang, and Bin Cui. VolcanoML : speeding up end-to-end AutoML via scalable search space decomposition. The VLDB Journal, 32 0 (2): 0 389--413, March 2023. ISSN 1066-8888, 0949-877X. doi:10.1007/s00778-022-00752-2. URL https://link.springer.co...

2023 doi
[30]

Oliphant, Matt Haberland, Tyler Reddy, David Cournapeau, Evgeni Burovski, Pearu Peterson, Warren Weckesser, Jonathan Bright, St \'e fan J

Pauli Virtanen, Ralf Gommers, Travis E. Oliphant, Matt Haberland, Tyler Reddy, David Cournapeau, Evgeni Burovski, Pearu Peterson, Warren Weckesser, Jonathan Bright, St \'e fan J. van der Walt , Matthew Brett, Joshua Wilson, K. Jarrod Millman, Nikolay Mayorov, Andrew R. J. Nels...

2020
[31]

Mixture discrepancy for quasi-random point sets

Yong-Dao Zhou, Kai-Tai Fang, and Jian-Hui Ning. Mixture discrepancy for quasi-random point sets. Journal of Complexity, 29 0 (3-4): 0 283--301, 2013

2013
[32]

Roshan Joseph

Shan Ba and V. Roshan Joseph . MaxPro : Maximum Projection Designs , January 2015. URL https://CRAN.R-project.org/package=MaxPro. Institution: Comprehensive R Archive Network Pages: 4.1-2

2015
[33]

SlicedLHD : Sliced Latin Hypercube Designs , February 2024

A Anil Kumar, Baidya Nath Mandal, Rajender Parsad, Sukanta Dash, and Mukesh Kumar. SlicedLHD : Sliced Latin Hypercube Designs , February 2024. URL https://CRAN.R-project.org/package=SlicedLHD. Institution: Comprehensive R Archive Network Pages: 1.0

2024
[34]

JMP ® Pro

SAS Institute Inc. JMP ® Pro . Cary, NC , 1989

1989
[35]

Lucas and Jeffrey D

Thomas W. Lucas and Jeffrey D. Parker. The Variability in Design - Quality Measures for Multiple Types of Space - Filling Designs Created by Leading Software Packages . In 2023 Winter Simulation Conference ( WSC ) , pages 516--527, San Antonio, TX, USA, December 2023. IEEE. IS...

2023
[36]

PEP 484 – Type Hints

Guido van Rossum , Jukka Lehtosalo, and Łukasz Langa. PEP 484 – Type Hints. Python Software Foundation, 2014. URL https://peps.python.org/pep-0484/. Accessed: 2024-09-03

2014
[37]

PEP 593 – Flexible function and variable annotations

Till Varoquaux and Konstantin Kashin. PEP 593 – Flexible function and variable annotations. Python Software Foundation, 2019. URL https://peps.python.org/pep-0593/. Accessed: 2024-09-03

2019
[38]

Lindauer, K

M. Lindauer, K. Eggensperger, M. Feurer, A. Biedenkapp, J. Marben, P. Müller, and F. Hutter. Boah: A tool suite for multi-fidelity bayesian optimization & analysis of hyperparameters. arXiv:1908.06756 [cs.LG] , 2019

1908 arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.

[1] [1]

Sanchez, Paul J

Susan M. Sanchez, Paul J. Sanchez, and Hong Wan. Work Smarter , Not Harder : A Tutorial on Designing and Conducting Simulation Experiments . In 2020 Winter Simulation Conference ( WSC ) , pages 1128--1142, Orlando, FL, USA, December 2020. IEEE. ISBN 978-1-72819-499-8. doi:10.1109/WSC48552.2020.9384057. URL https://ieeexplore.ieee.org/document/9384057/

arXiv 2020

[2] [2]

Isaac Gym : High Performance GPU - Based Physics Simulation For Robot Learning , 2021

Viktor Makoviychuk, Lukasz Wawrzyniak, Yunrong Guo, Michelle Lu, Kier Storey, Miles Macklin, David Hoeller, Nikita Rudin, Arthur Allshire, Ankur Handa, and Gavriel State. Isaac Gym : High Performance GPU - Based Physics Simulation For Robot Learning , 2021. URL https://arxiv.org/abs/2108.10470. Version Number: 2

arXiv 2021

[3] [3]

G. E. P. Box and H. L. Lucas. Design of Experiments in Non - Linear Situations . Biometrika, 46 0 (1/2): 0 77, June 1959. ISSN 00063444. doi:10.2307/2332810. URL https://www.jstor.org/stable/2332810?origin=crossref

work page arXiv 1959

[4] [4]

Sanchez and Paul J

Susan M. Sanchez and Paul J. Sánchez. Better Big Data via Data Farming Experiments . In Andreas Tolk, John Fowler, Guodong Shao, and Enver Yücesan, editors, Advances in Modeling and Simulation , pages 159--179. Springer International Publishing, Cham, 2017. ISBN 978-3-319-64181-2 978-3-319-64182-9. doi:10.1007/978-3-319-64182-9_9. URL http://link.springer...

work page doi:10.1007/978-3-319-64182-9_9 2017

[5] [5]

Design choice and machine learning model performances

Rosa Arboretti, Riccardo Ceccato, Luca Pegoraro, and Luigi Salmaso. Design choice and machine learning model performances. Quality and Reliability Engineering International, 38 0 (7): 0 3357--3378, November 2022. ISSN 0748-8017, 1099-1638. doi:10.1002/qre.3123. URL https://onlinelibrary.wiley.com/doi/10.1002/qre.3123

work page doi:10.1002/qre.3123 2022

[6] [6]

Design of experiments and machine learning with application to industrial experiments

Roberto Fontana, Alberto Molena, Luca Pegoraro, and Luigi Salmaso. Design of experiments and machine learning with application to industrial experiments. Statistical Papers, 64 0 (4): 0 1251--1274, August 2023. ISSN 0932-5026, 1613-9798. doi:10.1007/s00362-023-01437-w. URL https://link.springer.com/10.1007/s00362-023-01437-w

work page doi:10.1007/s00362-023-01437-w 2023

[7] [7]

Weese, Jonathan W

Kade Young, Maria L. Weese, Jonathan W. Stallrich, Byran J. Smucker, and David J. Edwards. A graphical comparison of screening designs using support recovery probabilities. Journal of Quality Technology, 56 0 (4): 0 355--368, August 2024. ISSN 0022-4065, 2575-6230. doi:10.1080/00224065.2024.2356127. URL https://www.tandfonline.com/doi/full/10.1080/0022406...

work page arXiv 2024

[8] [8]

Roshan Joseph

V. Roshan Joseph. Space-filling designs for computer experiments: A review. Quality Engineering, 28 0 (1): 0 28--35, January 2016. ISSN 0898-2112, 1532-4222. doi:10.1080/08982112.2015.1100447. URL http://www.tandfonline.com/doi/full/10.1080/08982112.2015.1100447

work page arXiv 2016