pith. sign in

arxiv: 1907.06912 · v1 · pith:2YXTAFYBnew · submitted 2019-07-16 · 💻 cs.NE

Modeling User Selection in Quality Diversity

Alexander Hagg , Alexander Asteroth , Thomas B\"ack This is my paper

Pith reviewed 2026-05-24 20:42 UTC · model grok-4.3

classification 💻 cs.NE
keywords quality diversityinteractive optimizationuser selection modelingmultimodal optimizationneuroevolutionpenalty termevolutionary algorithmspreference drift
0
0 comments X

The pith

An interactive quality diversity algorithm models user selections to add a penalty when search drifts from preferences.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper develops a method for users to interact with quality diversity algorithms during the discovery phase of engineering design, where requirements are initially vague. It builds a model of user selections to detect if the optimization is moving away from those preferences and then adds a penalty term to the objective function to steer solutions back. The method is tested against a state-of-the-art alternative using a new multimodal optimization benchmark on both a planning task and a neuroevolution control task. A sympathetic reader would care because this turns quality diversity into a practical aid that incorporates human judgment without requiring all criteria to be formalized in advance.

Core claim

By modeling a user's selection it can be determined whether the optimization is drifting away from the user's preferences. The optimization is then constrained by adding a penalty to the objective function. We present an interactive quality diversity algorithm that can take into account the user's selection. The approach is evaluated in a new multimodal optimization benchmark that allows various optimization tasks to be performed. The user selection drift of the approach is compared to a state of the art alternative on both a planning and a neuroevolution control task, thereby showing its limits and possibilities.

What carries the argument

Model of user selection that detects drift and triggers a penalty term added to the objective function.

If this is right

  • The algorithm can support engineers by keeping high-performing solutions aligned with evolving preferences during design exploration.
  • A new multimodal optimization benchmark enables testing across varied tasks including planning and control.
  • User selection drift can be measured and compared directly to non-interactive methods on concrete tasks.
  • The penalty approach shows concrete limits when applied to neuroevolution and planning problems.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same user-modeling idea could be tested in other diversity-preserving search methods beyond quality diversity.
  • If the penalty preserves archive properties, interactive sessions might require fewer total evaluations than manual steering.
  • The benchmark could be extended to measure how quickly user preferences stabilize under the modeled penalty.
  • Real design software could integrate the drift detector to prompt users only when the model signals misalignment.

Load-bearing premise

The model of user selection can reliably detect drift from preferences so the penalty term usefully constrains the search without destroying diversity or performance.

What would settle it

An experiment in which the penalty is applied but the archive loses coverage or performance relative to standard quality diversity, or the model fails to flag actual preference changes shown by new user choices.

Figures

Figures reproduced from arXiv: 1907.06912 by Alexander Asteroth, Alexander Hagg, Thomas B\"ack.

Figure 1
Figure 1. Figure 1: QD searches through genotypic space R n (b) to fill an archive A of diverse, high-performing phenotypes (a) in a low-dimensional phenotypic (or behavior) space. The genotypic dimensionality n can be very high. By projecting the archive’s members onto a low-dimensional similarity space (c), the user’s selection can be modeled. The projection model Tˆ allows making comparisons of candidate solutions to the u… view at source ↗
Figure 2
Figure 2. Figure 2: User selection drift dM is based on the distance to the closest selected point δS and the distance to the closest deselected point δ S . 3.2 User Driven Quality Diversity To make use of the UDHM, QD is extended by including the UDHM to the user-seeded version of MAP-Elites [9]. This user driven quality diversity (UDQD) algorithm is interactive, although it is evaluated based on predefined rules that repres… view at source ↗
Figure 3
Figure 3. Figure 3: The multimodal maze (a) has a starting location in [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 5
Figure 5. Figure 5: Neurocontrol task before (top) and after (bottom) [PITH_FULL_IMAGE:figures/full_fig_p006_5.png] view at source ↗
Figure 4
Figure 4. Figure 4: Path planning task before (top) and after (bot [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 6
Figure 6. Figure 6: Influence of penalty weight derived from UDHM [PITH_FULL_IMAGE:figures/full_fig_p006_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Median percentage of correct and incorrect solu [PITH_FULL_IMAGE:figures/full_fig_p007_7.png] view at source ↗
read the original abstract

The initial phase in real world engineering optimization and design is a process of discovery in which not all requirements can be made in advance, or are hard to formalize. Quality diversity algorithms, which produce a variety of high performing solutions, provide a unique chance to support engineers and designers in the search for what is possible and high performing. In this work we begin to answer the question how a user can interact with quality diversity and turn it into an interactive innovation aid. By modeling a user's selection it can be determined whether the optimization is drifting away from the user's preferences. The optimization is then constrained by adding a penalty to the objective function. We present an interactive quality diversity algorithm that can take into account the user's selection. The approach is evaluated in a new multimodal optimization benchmark that allows various optimization tasks to be performed. The user selection drift of the approach is compared to a state of the art alternative on both a planning and a neuroevolution control task, thereby showing its limits and possibilities.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 0 minor

Summary. The paper proposes an interactive quality diversity (QD) algorithm that models a user's selections to detect when optimization is drifting from user preferences, then constrains the search by adding a penalty term to the objective function. It introduces a new multimodal optimization benchmark and evaluates user selection drift on planning and neuroevolution control tasks against a state-of-the-art alternative, aiming to demonstrate the approach's limits and possibilities for turning QD into an interactive innovation aid.

Significance. If the central claim holds, the work could meaningfully extend QD algorithms beyond fully automated settings into interactive design and engineering workflows where requirements emerge during discovery. The new benchmark may also provide a reusable testbed for multimodal tasks. However, the absence of equations, quantitative results, or implementation details in the provided description prevents assessing whether these benefits are realized.

major comments (2)
  1. [Abstract] Abstract: The central claim that adding a penalty term usefully constrains the search without destroying QD diversity or performance properties rests on the unelaborated user selection drift model, but the abstract supplies no equations, quantitative results, error analysis, or implementation details, making it impossible to determine whether the data or derivations support the claim.
  2. [Abstract] Abstract: The evaluation plan compares user selection drift on planning and neuroevolution tasks, yet without reported metrics, controls for the penalty weight (a free parameter), or ablation of the drift model, it is not possible to verify that the approach improves over the state-of-the-art alternative while preserving archive quality.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their comments on the abstract. The full manuscript elaborates the model, reports metrics, and includes implementation details in the body and experiments, but we address the concerns about the abstract's brevity below and are open to revisions.

read point-by-point responses

  1. Referee: [Abstract] Abstract: The central claim that adding a penalty term usefully constrains the search without destroying QD diversity or performance properties rests on the unelaborated user selection drift model, but the abstract supplies no equations, quantitative results, error analysis, or implementation details, making it impossible to determine whether the data or derivations support the claim.

    Authors: Abstracts are space-constrained and omit equations/results by design. The full paper details the user selection drift model with equations in the methods, provides quantitative results and error analysis in the experiments section, and includes implementation details. We can revise the abstract to reference the model more explicitly if the editor permits. revision: partial

  2. Referee: [Abstract] Abstract: The evaluation plan compares user selection drift on planning and neuroevolution tasks, yet without reported metrics, controls for the penalty weight (a free parameter), or ablation of the drift model, it is not possible to verify that the approach improves over the state-of-the-art alternative while preserving archive quality.

    Authors: The full manuscript reports specific metrics for drift on both tasks, includes controls via penalty weight sweeps, and provides ablations of the drift model showing effects on archive quality. These appear in the results and evaluation sections. We can add a brief mention of key metrics to the abstract in revision. revision: partial

Circularity Check

0 steps flagged

No significant circularity

full rationale

The paper introduces an interactive quality diversity algorithm that models user selections to detect preference drift and applies a penalty term to the objective. The abstract and described approach present this as an independent modeling addition evaluated on new multimodal benchmarks and control tasks, with no equations or steps shown that reduce the claimed performance or drift detection to quantities defined by the authors' own prior fits, self-citations, or ansatzes. The central mechanism is presented as a new constraint rather than a renaming or self-referential derivation, making the result self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

1 free parameters · 1 axioms · 1 invented entities

The central claim rests on the ability to build an accurate user selection model and on the effectiveness of the penalty term; both are introduced without independent evidence or derivation in the abstract.

free parameters (1)
  • penalty weight
    Coefficient that scales the penalty added to the objective function when user selection drift is detected; must be chosen or tuned.
axioms (1)
  • domain assumption User selections can be modeled sufficiently well to detect meaningful drift from preferences
    Invoked when the paper states that the optimization can be constrained by adding a penalty based on the model.
invented entities (1)
  • user selection drift model no independent evidence
    purpose: To determine whether the quality diversity search is moving away from user preferences
    New component introduced to enable the interactive penalty mechanism.

pith-pipeline@v0.9.0 · 5692 in / 1370 out tokens · 27162 ms · 2026-05-24T20:42:49.498214+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

25 extracted references · 25 canonical work pages · 1 internal anchor

  1. [1]

    Richard Balling. 1999. Design by shopping: A new paradigm?. InProceedings of the Third World Congress of structural and multidisciplinary optimization (WCSMO-3) , Vol. 1. International Soc. for Structural and Multidisciplinary Optimization Berlin, 295–297

    work page 1999

  2. [2]

    nearest neighbor

    Kevin Beyer, Jonathan Goldstein, Raghu Ramakrishnan, and Uri Shaft. 1999. When is "nearest neighbor" meaningful?. In International conference on database theory. Springer, 217–235

    work page 1999

  3. [3]

    Erin Bradner, Francesco Iorio, and Mark Davis. 2014. Parameters tell the design story: Ideation and abstraction in design optimization. Simulation Series 46, 7 (2014), 172–197

    work page 2014

  4. [4]

    Jeff Clune and Hod Lipson. 2004. Evolving Three-Dimensional Objects with a Generative Encoding Inspired by Developmental Biology. Methods (2004)

    work page 2004

  5. [5]

    Antoine Cully, Jeff Clune, Danesh Tarapore, and Jean-Baptiste Mouret. 2015. Robots that can adapt like animals. Nature 521, 7553 (2015), 503–507

    work page 2015

  6. [6]

    Antoine Cully and Yiannis Demiris. 2017. Quality and Diversity Optimization: A Unifying Modular Framework. IEEE Transactions on Evolutionary Computation (2017), 1–15

    work page 2017

  7. [7]

    Jeffrey L. Elman. 1990. Finding structure in time. Cognitive Science 14, 1 990 (1990), 179–211

    work page 1990

  8. [8]

    Adam Gaier, Alexander Asteroth, and Jean-Baptiste Mouret. 2018. Data-Efficient Design Exploration through Surrogate-Assisted Illumination. Evolutionary com- putation (2018), 1–30

    work page 2018

  9. [9]

    Alexander Hagg, Alexander Asteroth, and Thomas Bäck. 2018. Prototype Dis- covery using Quality-Diversity. In Parallel Problem Solving From Nature (PPSN)

    work page 2018

  10. [10]

    Hornby, Al Globus, Derek S

    Gregory S. Hornby, Al Globus, Derek S. Linden, and Jason D. Lohn. 2006. Auto- mated antenna design with evolutionary algorithms.Proc. AIAA Space Conference 5 (2006), 1–8

    work page 2006

  11. [11]

    Joel Lehman and Kenneth O. Stanley. 2011. Abandoning objectives: Evolution through the search for novelty alone. Evolutionary Computation 19, 2 (2011), 189–222

    work page 2011

  12. [12]

    Joel Lehman and Kenneth O. Stanley. 2011. Evolving a diversity of virtual creatures through novelty search and local competition. Proceedings of the 13th annual conference on Genetic and evolutionary computation - GECCO ’11 Gecco (2011), 211

    work page 2011

  13. [13]

    Jean-Baptiste Mouret. 2011. Encouraging Behavioral Diversity in Evolutionary Robotics: An Empirical Study. Evolutionary computation x (2011)

    work page 2011

  14. [14]

    Anh Nguyen, Jason Yosinski, and Jeff Clune. 2015. Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition . 427–436

    work page 2015

  15. [15]

    Harald Niederreiter. 1988. Low-discrepancy and low-dispersion sequences. Jour- nal of number theory 30, 1 (1988), 51–70

    work page 1988

  16. [16]

    Parmee and Christopher R Bonham

    Ian C. Parmee and Christopher R Bonham. 2000. Towards the support of inno- vative conceptual design through interactive designer/evolutionary computing strategies. Ai Edam 14, 1 (2000), 3–16

    work page 2000

  17. [17]

    2015.Multimodal Optimization by Means of Evolutionary Algorithms

    Mike Preuss. 2015.Multimodal Optimization by Means of Evolutionary Algorithms

    work page 2015

  18. [18]

    Pugh, Lisa B

    Justin K. Pugh, Lisa B. Soros, and Kenneth O. Stanley. 2016. Quality Diversity: A New Frontier for Evolutionary Computation. Frontiers in Robotics and AI 3, July (2016), 1–17

    work page 2016

  19. [19]

    Justin K. Pugh, L. B. Soros, and Kenneth O. Stanley. 2016. Searching for quality diversity when diversity is unaligned with quality. Lecture Notes in Computer Modeling User Selection in Quality Diversity GECCO ’19, July 13–17, 2019, Prague, Czech Republic Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinforma...

    work page 2016

  20. [20]

    Rasmussen

    Carl E. Rasmussen. 2004. Gaussian processes in machine learning. In Advanced lectures on machine learning . Springer, 63–71

    work page 2004

  21. [21]

    Uri Shaham and Stefan Steinerberger. 2017. Stochastic Neighbor Embedding separates well-separated clusters. (2017), 1–8. arXiv:1702.02670

    work page internal anchor Pith review Pith/arXiv arXiv 2017

  22. [22]

    Laurens van der Maaten. 2014. Accelerating t-SNE using Tree-Based Algorithms. Journal of Machine Learning Research 15 (2014), 3221–3245

    work page 2014

  23. [23]

    Laurens Van Der Maaten and Geoffrey Hinton. 2008. Visualizing high- dimensional data using t-sne. Journal of Machine Learning Research 9 (2008), 2579–2605

    work page 2008

  24. [24]

    Vassilis Vassiliades and Jean-Baptiste Mouret. 2018. Discovering the Elite Hyper- volume by Leveraging Interspecies Correlation. (2018)

    work page 2018

  25. [25]

    B G Woolley and Kenneth O. Stanley. 2014. A Novel Human-Computer Collab- oration: Combining Novelty Search with Interactive Evolution. Proceedings of the 16th annual conference on Genetic and evolutionary computation, GECCO ’14 (2014), 233–240

    work page 2014