pith. sign in

arxiv: 1906.10188 · v1 · pith:WD6QNWUJnew · submitted 2019-06-24 · 💻 cs.HC · cs.LG· stat.ML

Deep Learning in a Computational Model for Conceptual Shifts in a Co-Creative Design System

Pegah Karimi , Mary Lou Maher , Nicholas Davis , Kazjon Grace This is my paper

Pith reviewed 2026-05-25 16:57 UTC · model grok-4.3

classification 💻 cs.HC cs.LGstat.ML
keywords co-creative systemsconceptual shiftsdeep learningnovelty metricsketching interfaceuser studycreativity supportAI design partner
0
0 comments X

The pith

Increasing novelty in AI sketch contributions leads to higher creative outcomes in co-creative design.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper builds a computational model that applies a novelty metric to deep learning vector representations of sketches. This model is embedded in a system where an AI agent and human designer take turns sketching on a shared canvas, with the AI deliberately introducing varying levels of conceptual shift. A user study then shows that when the AI adds more novelty, the overall designs are rated as more creative, but when the AI stays too similar, creativity drops. Readers might care because this points to a practical way for AI to act as a creative partner rather than a copier or distractor.

Core claim

The paper claims that a deep learning-based novelty metric can be used to generate conceptual shifts in an AI agent's sketches within a co-creative system, and empirical results from a user study indicate that higher novelty in the AI's contributions is associated with higher creative outcomes while low novelty is associated with lower creative outcomes.

What carries the argument

A novelty metric computed from distances in deep learning vector embeddings of sketches, used to control the degree of conceptual shift in the AI's responses.

If this is right

  • Design systems can intentionally vary AI novelty to support better creative partnerships.
  • AI contributions that are too similar to the human's work may reduce overall creativity.
  • The vector-based approach provides a computational way to implement conceptual shifts without manual rules.
  • The user study results suggest that moderate to high novelty levels optimize creative output.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • This model could be tested in other visual or non-visual creative tasks to see if the novelty-creativity link holds.
  • If the vector distances do not align with human judgments of conceptual difference, the model's effectiveness would need re-evaluation.
  • Future systems might combine this with other metrics like usefulness or surprise to refine the AI's role.

Load-bearing premise

The distance in the deep learning vector space corresponds to meaningful conceptual shifts as perceived by humans rather than mere visual similarities.

What would settle it

Run an experiment where independent raters score the conceptual novelty of the generated sketches and check whether those scores correlate with the model's vector-based novelty values; a lack of correlation would undermine the claim.

Figures

Figures reproduced from arXiv: 1906.10188 by Kazjon Grace, Mary Lou Maher, Nicholas Davis, Pegah Karimi.

Figure 1
Figure 1. Figure 1: The Creative Sketching Partner interface. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Computational steps for identifying conceptual shifts. Top: Identifying visually similar categories to the user’s input. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: The total percentage of high, intermediate, and low survey responses for (a) inspired creative ideas, and (b) led to [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
read the original abstract

This paper presents a computational model for conceptual shifts, based on a novelty metric applied to a vector representation generated through deep learning. This model is integrated into a co-creative design system, which enables a partnership between an AI agent and a human designer interacting through a sketching canvas. The AI agent responds to the human designer's sketch with a new sketch that is a conceptual shift: intentionally varying the visual and conceptual similarity with increasingly more novelty. The paper presents the results of a user study showing that increasing novelty in the AI contribution is associated with higher creative outcomes, whereas low novelty leads to less creative outcomes.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 0 minor

Summary. The paper presents a computational model for conceptual shifts in a co-creative design system. A deep-learning novelty metric is applied to vector representations of sketches; the AI agent responds to a human designer's sketch by generating a new sketch with intentionally varied visual and conceptual similarity. A user study is reported in which increasing novelty in the AI contribution is associated with higher creative outcomes, while low novelty is associated with lower creative outcomes.

Significance. If the novelty metric is shown to track human-perceived conceptual distance rather than low-level visual statistics, and if the user-study measures are validated, the work could inform the design of AI partners that deliberately modulate conceptual distance to support creativity. The approach of embedding DL representations directly into an interactive sketching loop is a concrete step toward computational models of co-creativity.

major comments (2)
  1. [Abstract] Abstract: the central empirical claim—that the DL vector-distance novelty metric produces higher creative outcomes—rests on the untested assumption that Euclidean or cosine distance in the embedding space corresponds to human conceptual shifts. No human-rating correlation, ablation against pixel-level or edge-based baselines, or control for visual complexity is described.
  2. [User Study] User-study description: the creativity measure and its validation are not specified (e.g., how inter-rater reliability was assessed, how confounds such as sketch complexity or style were controlled). Without these details the reported association cannot be evaluated.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their detailed review and constructive comments on our manuscript. We address each of the major comments below.

read point-by-point responses

  1. Referee: [Abstract] Abstract: the central empirical claim—that the DL vector-distance novelty metric produces higher creative outcomes—rests on the untested assumption that Euclidean or cosine distance in the embedding space corresponds to human conceptual shifts. No human-rating correlation, ablation against pixel-level or edge-based baselines, or control for visual complexity is described.

    Authors: The referee is correct that the manuscript does not include a direct human validation of the embedding space distances as measures of conceptual shifts, nor ablations or controls for visual complexity. The reported association is between the AI-generated novelty levels (computed via the DL metric) and the creative outcomes in the user study. We will revise the abstract to clarify the scope of the empirical claim and add a discussion of this limitation, including suggestions for future validation studies. revision: yes

  2. Referee: [User Study] User-study description: the creativity measure and its validation are not specified (e.g., how inter-rater reliability was assessed, how confounds such as sketch complexity or style were controlled). Without these details the reported association cannot be evaluated.

    Authors: We agree that the description of the user study in the manuscript lacks sufficient detail regarding the creativity measure, its validation, inter-rater reliability, and controls for potential confounds. We will revise the user study section to provide a more complete account of the methodology, including how creativity was measured and any steps taken to ensure reliability and control for confounds. revision: yes

Circularity Check

0 steps flagged

No circularity: empirical association reported from user study with no derivations or self-referential reductions

full rationale

The paper describes a deep-learning novelty metric applied to sketches, integrates it into a co-creative system, and reports a user-study correlation between AI novelty level and creative outcomes. No equations, parameter fits, or derivation chains are present. The central claim is an observed statistical association, not a quantity derived from fitted inputs or self-citations. The metric's validity (whether vector distance tracks conceptual vs. visual distance) is an external assumption, not a circularity in the reported result itself. No load-bearing self-citations or renamings of known results appear in the provided text.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract-only review supplies almost no technical detail; no free parameters, axioms, or invented entities can be identified with certainty.

pith-pipeline@v0.9.0 · 5638 in / 1071 out tokens · 18670 ms · 2026-05-25T16:57:57.791161+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Unlearning with Asymmetric Sources: Improved Unlearning-Utility Trade-off with Public Data

    cs.LG 2026-05 unverdicted novelty 7.0

    Asymmetric Langevin Unlearning uses public data to suppress unlearning noise costs by O(1/n_pub²), enabling practical mass unlearning with preserved utility under distribution mismatch.

Reference graph

Works this paper leans on

22 extracted references · 22 canonical work pages · cited by 1 Pith paper · 4 internal anchors

  1. [1]

    [Bonazza 2019] Bonazza, M

    work page 2019

  2. [2]

    https://sketchtogether.com

    SketchTogether. https://sketchtogether.com. Accessed: 2019- 02-27. [Carbune 2017] Carbune, V

    work page 2019

  3. [3]

    https://www.tensorflow.org/tutorials/ sequences/recurrent_quickdraw

    Recur- rent neural networks for drawing classification. https://www.tensorflow.org/tutorials/ sequences/recurrent_quickdraw. [Colton et al. 2015] Colton, S.; Halskov, J.; Ventura, D.; Gouldstone, I.; Cook, M.; and Ferrer, B. P

    work page 2015

  4. [4]

    In ICCC, 189–196

    The paint- ing fool sees! new projects with the automated painter. In ICCC, 189–196. [Das and Gamb¨ack 2014] Das, A., and Gamb ¨ack, B

    work page 2014

  5. [5]

    In ICCC, 230–238

    Poetic machine: Computational creativity for automatic po- etry generation in bengali. In ICCC, 230–238. [Davis et al. 2015a] Davis, N.; Hsiao, C.-P.; Popova, Y .; and Magerko, B. 2015a. An enactive model of creativity for computational collaboration and co-creation. In Creativity in the Digital Age. Springer. 109–133. [Davis et al. 2015b] Davis, N. M.; H...

    work page 2009

  6. [6]

    [Gero 2000] Gero, J

    Imagenet: A large-scale hierarchical image database. [Gero 2000] Gero, J. S

    work page 2000

  7. [7]

    Technological fore- casting and social change 64(2-3):183–196

    Computational models of in- novative and creative design processes. Technological fore- casting and social change 64(2-3):183–196. [Grace et al. 2015] Grace, K.; Maher, M. L.; Fisher, D.; and Brady, K

    work page 2015

  8. [8]

    In Design Computing and Cogni- tion’14

    Modeling expectation for evaluating sur- prise in design creativity. In Design Computing and Cogni- tion’14. Springer. 189–206. [Guzdial et al. 2019] Guzdial, M.; Liao, N.; Chen, J.; Chen, S.-Y .; Shah, S.; Shah, V .; Reno, J.; Smith, G.; and Riedl, M

    work page 2019

  9. [9]

    Friend, Collaborator, Student, Manager: How Design of an AI-Driven Game Level Editor Affects Creators

    Friend, collaborator, student, manager: How de- sign of an ai-driven game level editor affects creators. arXiv preprint arXiv:1901.06417. [Ha and Eck 2017] Ha, D., and Eck, D

    work page internal anchor Pith review Pith/arXiv arXiv 1901

  10. [10]

    A Neural Representation of Sketch Drawings

    A neu- ral representation of sketch drawings. arXiv preprint arXiv:1704.03477. [Jacob et al. 2013] Jacob, M.; Coisne, G.; Gupta, A.; Sysoev, I.; Verma, G. G.; and Magerko, B

    work page internal anchor Pith review Pith/arXiv arXiv 2013

  11. [11]

    In Ninth Artificial Intelligence and Interactive Digital Enter- tainment Conference

    Viewpoints ai. In Ninth Artificial Intelligence and Interactive Digital Enter- tainment Conference. [Johnson et al. 2009] Johnson, G.; Gross, M. D.; Hong, J.; Do, E. Y .-L.; et al

    work page 2009

  12. [12]

    Foundations and Trends R⃝ in Human–Computer Interaction 2(1):1–93

    Computational support for sketching in design: a review. Foundations and Trends R⃝ in Human–Computer Interaction 2(1):1–93. [Jongejan et al. 2016] Jongejan, J.; Rowley, H.; Kawashima, T.; Kim, J.; and Fox-Gieg, N

    work page 2016

  13. [13]

    Deep Learning for Identifying Potential Conceptual Shifts for Co-creative Drawing

    The quick, draw!-ai experiment. Mount View, CA, accessed Feb17:2018. [Karimi et al. 2018a] Karimi, P.; Davis, N.; Grace, K.; and Maher, M. L. 2018a. Deep learning for identifying poten- tial conceptual shifts for co-creative drawing.arXiv preprint arXiv:1801.00723. [Karimi et al. 2018b] Karimi, P.; Grace, K.; Davis, N.; and Maher, M. L. 2018b. Creative sk...

    work page internal anchor Pith review Pith/arXiv arXiv 2018

  14. [14]

    nature 521(7553):436

    Deep learning. nature 521(7553):436. [Mikolov 2016] Mikolov, T

    work page 2016

  15. [15]

    https://code.google.com/p/word2vec

    word2vec: Tool for computing continuous distributed representations of words. https://code.google.com/p/word2vec. [Oh et al. 2018] Oh, C.; Song, J.; Choi, J.; Kim, S.; Lee, S.; and Suh, B

    work page 2018

  16. [16]

    In Proceedings of the 2018 CHI Confer- ence on Human Factors in Computing Systems,

    I lead, you help but only with enough de- tails: Understanding user experience of co-creation with ar- tificial intelligence. In Proceedings of the 2018 CHI Confer- ence on Human Factors in Computing Systems,

    work page 2018

  17. [17]

    [Purcell and Gero 1996] Purcell, A

    ACM. [Purcell and Gero 1996] Purcell, A. T., and Gero, J. S

    work page 1996

  18. [18]

    Design studies 17(4):363–383

    Design and other types of fixation. Design studies 17(4):363–383. [Shneiderman 2007] Shneiderman, B

    work page 2007

  19. [19]

    Commu- nications of the ACM 50(12):20–32

    Creativity sup- port tools: Accelerating discovery and innovation. Commu- nications of the ACM 50(12):20–32. [Simonyan and Zisserman 2014] Simonyan, K., and Zisser- man, A

    work page 2014

  20. [20]

    Very Deep Convolutional Networks for Large-Scale Image Recognition

    Very deep convolutional networks for large- scale image recognition. arXiv preprint arXiv:1409.1556. [Suwa and Tversky 1997] Suwa, M., and Tversky, B

    work page internal anchor Pith review Pith/arXiv arXiv 1997

  21. [21]

    [V oigt, Niehaves, and Becker 2012] V oigt, M.; Niehaves, B.; and Becker, J

    work page 2012

  22. [22]

    In International Conference on Design Science Research in Information Systems, 152–173

    Towards a unified design theory for creativity support systems. In International Conference on Design Science Research in Information Systems, 152–173. Springer. [Yannakakis, Liapis, and Alexopoulos 2014] Yannakakis, G. N.; Liapis, A.; and Alexopoulos, C

    work page 2014