REVIEW 3 major objections 5 minor 9 references
Playing magic tricks to deep neural networks untangles human deception
T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read This paper claims that a deep network trained to guess hidden coin locations can serve as an "artificial spectator" whose disagreements with humans expose the cognitive biases that make magic work.
desk verdict Original 'artificial spectator' idea, but the headline asymmetry is largely trained into the machine; worthy of peer review as a proof of concept, not as a validated result. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the "artificial spectator": a deep neural network trained not only to track visible coins but to infer hidden coin locations from the same human guesses a spectator would make. The mechanism is a two-by-two classification of each trick by whether the human is fooled and whether the machine is fooled, turning a magic performance into a confusion matrix. The inference training is the crucial part: because the network is taught to label occluded coins with human priors, its errors are not random tracking failures but structured departures from those priors. These departures, along with the tricks where the network and human agree, are what carry the argument.
What would settle it
Retrain the network on the same tricks using occlusion guesses from several different annotators, and also train a version with no occlusion labels at all. If the "machine not fooled" classifications change from one annotator to another, or disappear when no human priors are supplied, the reported asymmetries are an artifact of the training choice rather than a stable feature of human cognition.
Extended reading notes
Core claim
On the paper's own terms, the central discovery is that the difference between a human and a machine watching the same sleight of hand is not noise but information: it separates the motor mechanics of a trick from the cognitive priors that make it magical. The authors trained a deep-learning pose-estimation network to label coin positions even when coins were occluded, using a human annotator's guesses as the training target; the network thereby learned human-style expectations, such as assuming a coin remains in the closed fist. In tricks #1 and #2, human and machine were both fooled, showing that some cognitive illusions transfer to a network. In tricks #3, #4, and #5, the human was fooled while the machine was not, because the machine never suffered perceptual overload, attentional misdirection, or Gestalt symmetry assumptions. In trick #6, performed badly on purpose, the machine was fooled where a human would not be, because the flying coin looked like a rod rather than a circle. The central claim is that this human-machine confusion diagram is a readable map of human cognitive biases, made visible from the perspective of the machine.
Load-bearing premise
The argument assumes that the network's hidden-coin guesses are not just a stored copy of the human annotator's guesses used to train it; if the network only echoes that one person's prior, the human-machine differences reveal nothing new about human perception in general.
Editorial extensions
If this is right
- Magic can be decomposed into motor maneuvers whose perceptual effect is independently measurable: a network's inference errors tell which moments carry the deception.
- Training neural networks on occluded, not just visible, targets turns them from trackers into models of expectation, opening a general method for probing perception beyond magic.
- Tricks that fool both human and machine point to priors that are shared and learnable, while tricks that fool only humans point to attentional and Gestalt mechanisms that current models lack.
- The same comparison could guide the design of adversarial displays: routines would be iterated against a human-like network before being shown to people.
Reading between the lines
- Editorial inference: since training labels came from a single annotator, training the same network on many people's occlusion guesses would test whether the machine's "not fooled" verdicts are stable; if they are, the method measures individual differences in susceptibility to magic.
- Editorial inference: the authors' observation that the machine treats forward and backward video identically suggests a direct experiment—a "palindrome" trick—that would fool the machine in both directions while fooling humans in only one; a positive result would confirm the asymmetry comes from temporal expectation.
- Editorial inference: the results point to a design loop the authors only sketch: generate candidate movements, have a human-like network watch them, and use its misinferences to predict which routines will deceive a live audience.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript proposes using DeepLabCut, a deep-learning pose-estimation tool, as an "artificial spectator" for coin magic. The authors record a professional magician performing seven brief motor tricks, train DeepLabCut to track visible coins and to infer occluded coin positions from labels placed by a human annotator, and then classify each trick by whether it would fool a human, the machine, both, or neither (Figure 1F). The central claim is that this setup reveals human cognitive biases through human-machine asymmetries in which tricks succeed.
Significance. The idea of repurposing a deep-learning tracker as a model spectator for magic is creative and potentially useful; if the human-machine comparison were valid, it could open a new way to study cognitive illusions. The authors are commendably explicit about limitations, and they share a supplementary video and data. However, the reported evidence does not support the central interpretive claim: the machine's occluded-coin inference is trained on human guesses from the same videos, and "human fooled" is asserted rather than measured. At present the paper is a demonstrative proof-of-concept that DeepLabCut can be trained to extrapolate occluded locations, not a demonstration that machine errors reveal human cognitive biases.
major comments (3)
- [Methods (training the network on occluded coin locations)] The central claim requires the machine's occluded-coin inference to be an independent observer whose errors can be contrasted with human errors. However, the Methods state that when coins are not visible, "the human annotating frames clicked where she thought the coin was (i.e. in the left fist; under the right palm). The machine learned those human priors." Because the network is trained on the same annotator's guesses from the same videos used for evaluation (as the second stated limitation concedes), the machine's "fooled" state in tricks #1 and #2 is the supervised target supplied by a human who was also fooled. The human-machine asymmetry in Figure 1F is therefore a comparison between one human annotator's priors and the authors' interpretation of the true coin location, not between independent human and machine cognitive processes.
- [Figure 1F and first stated limitation] The quadrant classification in Figure 1F labels some tricks as "only fool a human," but no participant data are reported and the first stated limitation explicitly says that the quadrants "reflect what the human from the perspective of the machine would claim with respect to each trick." The textual descriptions of tricks #3, #4, and #5 (perceptual overload, misdirection, Gestalt symmetry) are plausible narrative interpretations, not measurements. Without independent human responses to the same video stimuli, the orange quadrant ("human fooled, machine not") is unsupported.
- [Overall evaluation (seven selected tricks)] The paper provides no statistical test, no error bars, no independent evaluation set, and no objective definition of "fooled." The seven tricks were selected after the fact to illustrate the desired quadrants, and the abstract's broad conclusion that "magic from the perspective of the machine reveals our own cognitive biases" goes beyond what a small, post hoc, qualitative set can establish. At minimum, the claim would require pre-registered stimuli, naive human participants, and a quantitative criterion for being fooled.
minor comments (5)
- [Methods] The Methods report that labeling was "typically of the order of 5% of the total frame number" with "at least 200K iters per video," but do not report the total number of frames, the fraction of frames used for validation, or the effect of training iterations; this is insufficient for reproducibility.
- [Main text, trick descriptions] The claim that the machine "analyzes the whole image pixel by pixel" is imprecise: DeepLabCut uses convolutional neural networks with local receptive fields, not exhaustive pixel-by-pixel matching. Please rephrase to describe the actual architecture.
- [Supplementary Video] The Supplementary Video is cited, but no timestamps or frame numbers are given for each of the seven tricks, making it difficult for a reader to verify the quadrant assignments in Figure 1F.
- [Abstract] There are typographical and spacing issues in the abstract (for example, "a p p e a r a n d d i s a p p e a r"), and the text would benefit from proofreading.
- [References] The Methods cite "DeepLabCut 2.0" but reference [6] describes the original DeepLabCut method; please specify the exact software version and, if applicable, the version-specific documentation.
Circularity Check
The machine's occluded-coin inferences are trained on the same human annotator's guesses, so the human-AI asymmetry in Figure 1F is largely built in by construction.
-
fitted input called prediction
[Main text, paragraph beginning 'The magic tricks performed, tracked and analyzed are shown in the Supplementary Video' (p. 2); Figure 1E]
"here, instead, we trained the network also with images in which the coins were not visible: the human annotating frames clicked where she thought the coin was (i.e. in the left fist; under the right palm). The machine learned those human priors (Figure 1E)."
The supervised target for occluded frames is the annotator's own guess ('where she thought the coin was'). The network is trained to output that guess. The paper later concludes from tricks #1 and #2 that 'Both infer it to be in the closed fist, where it is not. This shows that at least some cognitive tricks can in fact transfer from humans to machines.' But the machine's inference is the trained reproduction of the same human prior; agreement is guaranteed by the training objective, not an emergent similarity. Comparing this trained output to human behavior is comparing the machine to its own training target, so the claimed 'transfer' is an input-output tautology.
-
self definitional
[Limitations paragraph, p. 3]
"First, we did not train the network to label every frame as “fooled” versus “not fooled”. The quadrants in Figure 1F (specially the orange one) reflect what the human from the perspective of the machine would claim with respect to each trick."
The central Figure 1F classification is not derived from an independent machine measurement. The authors state that the quadrants 'reflect what the human from the perspective of the machine would claim.' Since the machine's occluded-location outputs were trained on that same human's guesses (previous step), the human-versus-machine asymmetry displayed in Figure 1F reduces to a comparison between a human's own labels and the authors' interpretation of the true coin location. The claimed revelation of 'our own cognitive biases' is therefore the input (human priors) recycled as the output, rather than a comparison of two independent observers.
full rationale
Score 7 reflects partial circularity. The paper's central 'artificial spectator' claim is substantially circular: the network is trained on the human annotator's guesses for occluded frames, and the Figure 1F quadrants are assigned by that same human perspective. Consequently, the machine's 'fooled' behavior in tricks #1 and #2 is a supervised replay of the annotator's prior, not an independent finding, and the paper explicitly concedes both of these points in its limitation paragraph. Not all content is circular: visible-coin tracking, motor-skill quantification, and the machine's failures on oddly oriented coins (e.g., trick #6) are independent of the human prior. Those non-circular elements, together with the honest statement of limitations, keep the paper from being a pure tautology. However, the headline conclusion that magic 'from the perspective of the machine reveals our own cognitive biases' relies on the circular comparison and is not established by the reported data as an independent empirical discovery.
Assumptions & free parameters
free parameters (2)
- Fraction of labeled frames per video =
~5%
- Training iterations per video =
>=200K
assumptions (3)
- domain assumption DeepLabCut can accurately track visible coins in naturalistic magic videos.
- domain assumption A network trained on the annotator's clicked locations for occluded coins behaves like a human spectator.
- domain assumption The authors' assignment of each trick to a quadrant in Figure 1F is unbiased.
Cite this review
Pith. "Pith review of Playing magic tricks to deep neural networks untangles human deception." pith.science (2026). https://pith.science/paper/LTFMPJVQ
@misc{pith2026190807446,
author = {Pith},
title = {Pith review of: Playing magic tricks to deep neural networks untangles human deception},
year = {2026},
howpublished = {\url{https://pith.science/paper/LTFMPJVQ}},
note = {Machine review of arXiv:1908.07446}
}
read the original abstract
Magic is the art of producing in the spectator an illusion of impossibility. Although the scientific study of magic is in its infancy, the advent of recent tracking algorithms based on deep learning allow now to quantify the skills of the magician in naturalistic conditions at unprecedented resolution and robustness. In this study, we deconstructed stage magic into purely motor maneuvers and trained an artificial neural network (DeepLabCut) to follow coins as a professional magician made them appear and disappear in a series of tricks. Rather than using AI as a mere tracking tool, we conceived it as an "artificial spectator". When the coins were not visible, the algorithm was trained to infer their location as a human spectator would (i.e. in the left fist). This created situations where the human was fooled while AI (as seen by a human) was not, and vice versa. Magic from the perspective of the machine reveals our own cognitive biases.
Figures
Reference graph
Works this paper leans on
-
[3]
Nature Reviews Neuroscience 9:871-9
Macknik S, King M, Randi J, Robbins A, Teller, Thompson J, & Martinez-Conde S (2008) Attention and awareness in stage magic: turning tricks into research. Nature Reviews Neuroscience 9:871-9
work page 2008
-
[4]
Kuhn G (2019) Experiencing the Impossible. The Science of Magic. MIT Press, Cambridge
work page 2019
-
[5]
Current Biology 26(10):R387-407
Quiroga RQ (2016) Magic and cognitive neuroscience. Current Biology 26(10):R387-407
work page 2016
-
[6]
Nature Neuroscience 21: 1281–1289
Mathis A, Mamidanna P, Cury KM, Abe T, Murthy VN, Mathis MW, & Bethge M (2018) DeepLabCut: markerless pose estimation of user-defined body parts with deep learning. Nature Neuroscience 21: 1281–1289
work page 2018
-
[7]
and adversarial tricks remains to be concretely drawn, our results are a prelude to further deployments of deep learning in the context of the demanding processes of a magic performance —a perspective to our knowledge not explored so far ("artificial illusionism"). AI still needs to be equipped with our familiar notions of causation [8]. And yet we can le...
work page 2015
-
[8]
Zhou Z & Firestone C (2019) Humans can decipher adversarial images. Nature Communications 10:1334
work page 2019
-
[9]
Pearl J & Mackenzie D (2018) The Book of Why: The New Science of Cause and Effect. Basic Books, NY
work page 2018
-
[10]
Behavioral and Brain Sciences 40:e253
Lake BM, Ullman TD, Tenenbaum JB, Gershman SJ (2017) Building machines that learn and think like people. Behavioral and Brain Sciences 40:e253
work page 2017
Show all 9 references
-
[11]
(2019) Machine behaviour
Rahwan I et al. (2019) Machine behaviour. Nature 568, 477–486. * Figure
2019
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.