REVIEW 4 major objections 6 minor 32 references
Breathless: An 8-hour Performance Contrasting Human and Robot Expressiveness
T0 review · 4 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read Video-captured dance gave a robot arm an 8-hour duet.
desk verdict A real 8-hour human-robot duet, honestly described, but the joint-mapping pseudocode contradicts the prose and needs a fix before the technical details are reliable. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is a staged motion-generation pipeline. OpenPose extracts human joint keypoints from video; a 1D blur convolution of size 15 smooths the angle trajectories; a fast Fourier transform with a hand-set threshold of 20 zeroes out high-frequency bins before inversion; and the resulting per-joint sinusoids are mapped onto the UR5e's six axes, with the unmapped shoulder-pan and wrist-2 joints filled by manually generated sinusoids. The pipeline's second half uses the UR5e's teachMode for compliant, dancer-guided recording at 500 Hz, forceMode with damping 0.2 to smooth the transition out of compliance, and a force-cue detector (movement-average threshold of 20 N) for triggered motifs such as the concluding bow. A URDF-based simulation library is used to verify trajectories for self-collision and sudden jolts before any motion is run on the robot.
What would settle it
Replay a fresh set of human dance videos through the exact OpenPose, FFT-threshold-20, and sinusoid-fitting pipeline on a UR5e and record whether any protective stop triggers, whether the robot enters a self-collision, or whether the resulting motion reads as a different motif; a single such failure would show the pipeline is not sufficient as described.
Extended reading notes
Core claim
On the paper's own terms, the discovery is that human arm motions can be captured as 2D joint angles, reduced by an FFT threshold to a small set of sinusoidal components, and replayed on a six-axis robot to evoke recognizable human-labour gestures. The shoulder, elbow, and wrist angles from OpenPose are mapped to the UR5e's shoulder-lift, elbow, wrist-1, and wrist-3 joints, while the remaining joints are driven by manually generated sinusoids with tuned amplitude, frequency, phase, offset, and an exponential amplitude envelope to create grow-and-fade motions. Live sections are handled by teach-mode 'zero-G' recording, a force-mode transition stage with damping 0.2 to avoid protective stops, and a force-cued waiting state with a 20 N threshold. The paper reports that this combination ran for the full eight-hour premiere without robot replacement or significant debugging, with only a few timeouts requiring resets.
Load-bearing premise
The assumption that carries the whole claim is that the shoulder, elbow, and wrist angles read off a flat video of a dancer, after smoothing and removing high-frequency wiggles, will drive a robot arm's joints in a way that is both safe and recognizable as human movement, even though the remaining joints are just filled with simple back-and-forth motions.
Editorial extensions
If this is right
- An artist with a UR5e and the released code suite can reproduce the full set of labor motifs without writing a trajectory planner.
- The pipeline's reliance on sinusoidal primitives means the same captured human motion can be ported to other six-axis arms by adjusting the hand-tuned amplitude and offset parameters.
- The teach-mode and force-cue interaction pattern gives a safe, non-verbal protocol for a human to improvise with an industrial robot in performance.
- Eight hours of continuous operation without major failure demonstrates that long-duration installation art is feasible with commodity industrial hardware.
Reading between the lines
- A natural extension is to replace the hand-set FFT threshold and manual sinusoids with an automated model that learns per-joint sinusoidal parameters from a corpus of dance videos, which would make the pipeline less dependent on human tuning.
- The force-cue and damping transitions could be adapted to collaborative manufacturing, where a worker briefly nudges a robot into a new pose and it resumes its task smoothly.
- The paper's mapping from 2D image-plane angles to robot joints ignores the third dimension, so motions with strong depth variation would likely need a multi-camera pose estimator to preserve expressiveness.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper describes the technology behind 'Breathless,' an eight-hour dance performance that paired the human dancer Catie Cuan with a Universal Robots UR5e industrial arm at National Sawdust Theater on December 16, 2023. The system combines several elements: an OpenPose-based pipeline that extracts 2D human joint angles, smooths them with a 15-tap convolution, applies FFT-domain filtering with a hand-tuned threshold of 20, and maps the resulting trajectories onto the UR5e joints; a library of manually constructed sinusoidal motifs (stirring, hammering, mopping, and others) with parameters A, omega, phi, gamma tuned in the urdfpy simulator; live interactive sections that use teach-mode recording and 500 Hz replay, a forceMode damping stage to smooth the compliant-to-rigid transition, and a 20 N force-threshold tap detector for cuing; and a Google Sheets, AppleScript, Keynote, and Zoom based stage-cueing system for music and lighting coordination. The paper claims a novel application of AI-based human pose tracking to sinusoidal robot motion planning, reports that the robot ran the full performance with occasional resets and that roughly 600 audience members attended, and offers the project website's video, code, and data as documentation.
Significance. If the system works as described, the paper documents a rare and useful data point: an industrial robot arm sustaining an eight-hour mixed scripted-and-interactive choreography in front of a live audience without hardware failure. The most valuable parts are the practical engineering details, including the damped forceMode transition around teach-mode replay, the moving-average tap detector with a 20 N threshold, and the simulation-first safety workflow, all of which are transferable to other human-robot performance and interaction settings. The stated plan to release video, code, and data is a genuine strength for reproducibility, and the Limitations section is candid about the operational complexity, which required three technical personnel, a stage manager, and a lighting technician. The significance is nonetheless bounded: the technical novelty beyond existing human-to-robot motion transfer and sinusoidal control work is incremental; expressiveness and operational robustness are asserted rather than measured; and the central signal-processing step is specified imprecisely. This is better framed as a systems and performance document than as a generalizable method.
major comments (4)
- [Section 3.1] The pseudocode's final joint assignment contradicts the mapping stated in the text. The text says 'H1 is mapped to shoulder lift, H2 to elbow, and H3 to wrist 1, and H4 to wrist 3,' but the pseudocode defines H0 as the absolute chest-shoulder angle, H1 as the relative shoulder angle, H2 as the relative elbow angle, H3 as wrist flexion, and H4 as wrist rotation, and then constructs theta_i = [theta_0(t), H_i0, H_i1, H_i2, theta_4(t), H_i3]. As written, shoulder lift follows the absolute chest-shoulder angle (which includes torso orientation) rather than shoulder flexion, wrist 3 follows wrist flexion rather than wrist rotation, and H4 never enters the joint vector. If the released code follows the pseudocode, the described wrist-rotation capability was never realized; if the code corrects the indexing, the pseudocode misrepresents the implemented system. The manuscript must align text, pseudocode, and code, and state which mapping actually produced the motions shown in the performance.
- [Section 3.1] The FFT-filtering step is internally inconsistent and not reproducible as written. The pseudocode computes H' = FFT^-1(f) after thresholding, but the final assignment uses H_i, not H'_i, so the inverse transform has no effect on the output. Moreover, the text describes low-pass filtering ('setting high-frequency bins to 0'), whereas the pseudocode zeroes components with |f_i| < 20, which is magnitude thresholding; the two operations are different, and the value 20 is not given in units that allow replication (a bin index, an amplitude in radians, or a pixel-derived unit). Since the paper's first contribution rests on converting tracked poses into sinusoidal trajectories, this step must be specified exactly and applied consistently.
- [Section 4] The Results section does not support the paper's central claims of expressiveness and of an eight-hour successful run. The evidence is one audience quote, a Forbes citation, and the statement that the robot, 'In a few cases, it timed out and needed to be reset,' while it 'largely ran' for eight hours. There are no operational statistics for the performance: no count of resets or protective stops, no downtime duration, no detection rate or latency for the force-tap cue across the full show, and no evaluation of expressiveness by a defined protocol (structured audience responses, movement-expert ratings, or even a documented observation rubric). Because the stated goal is to keep the arm moving 'safely and expressively for a full working day,' the manuscript should either add these basic measurements or explicitly reframe the paper as an artistic demonstration without robustness and expressiveness claims.
- [Sections 3.1-3.2] The claimed contribution of applying 'AI-based human pose tracking to robot motion planning using sinusoidal functions' is not exactly what the described pipeline does. The OpenPose branch smooths and FFT-thresholds the extracted joint trajectories and replays them; the paper reports no explicit fit of a small set of sinusoids of the form A*cos(omega*t + phi) + gamma to the human data, and no fit-quality measure (retained harmonics, residual error, variance explained) is given. The manual motifs in Section 3.2 are explicit sinusoids, but their parameter values (A, omega, phi, gamma, and the e^(-Bx) decay used for the bartender sequence) are not reported in a table or appendix, so that half of the pipeline is also not reproducible from the text. The authors should state how the FFT-filtered trajectories relate to the sinusoidal parameterization and provide representative parameter values for the motifs.
minor comments (6)
- [Section 3.1] The net-change computation for the right arm reads 'sl = sum_{i=1}^{n-1} = |H_ri - H_l(i+1)|'; it should be 'sr = sum_{i=1}^{n-1} |H_ri - H_r(i+1)|', and the stray equals sign in the summation notation should be removed.
- [Section 3.1] The manuscript mixes indexing conventions, listing the human joints as 1 through 6 in the extraction list but using 0 through 5 in the pseudocode; adopting a single convention throughout would prevent the kind of off-by-one confusion that occurs in the joint-mapping step.
- [Section 3.1] In the pseudocode, 'cordinates' should be 'coordinates', and the loop header 'j in [0,1,2,3,4]' includes an upper index that the loop body never uses.
- [Section 3.3] The sentence 'we add an section between the compliant and rigid sections of the robot' contains a grammar error, and the choice of the 20 N threshold and the 10-sample running-average window deserves a brief justification relative to the sensor-noise level illustrated in Figure 7.
- [Figure 7 caption] The phrase 'correctly identify the cue' should read 'correctly identified the cue'; the figure shows a single rehearsal trial, and one sentence on how representative that trace is across rehearsals would strengthen the claim.
- [Section 4] The phrase 'without significant injury, only taking two 15 minute breaks' uses 'only' as an editorial judgment, and it is unclear whether the robot continued to run during the dancer's breaks; stating the robot's behavior during those breaks would clarify the 'eight-hour' claim.
Circularity Check
No circularity: the motion-generation pipeline is data-driven fitting and system description, not a derivation that reduces to its inputs.
full rationale
The paper's central technical content is a robot-choreography pipeline: OpenPose extracts human joint angles, the angles are smoothed and low-pass filtered in the Fourier domain, sinusoids are fit or manually generated, and the resulting trajectories are replayed on a UR5e. This is an empirical fitting and engineering description, not a derivation whose output is equivalent to its input by construction. The FFT threshold of 20 and the force-cue threshold of 20 N are manually tuned operating parameters, and the paper does not present either as a predicted quantity; they do not make the central claim circular. The self-citations in the related-work and acknowledgement sections provide historical and artistic context and are not load-bearing for any technical result. The pseudocode discrepancy noted by a skeptical reader (θi = [θ0(t), Hi0, Hi1, Hi2, θ4(t), Hi3] versus the prose mapping H1→shoulder lift, H2→elbow, H3→wrist 1, H4→wrist 3) is a real reproducibility and correctness concern about whether the implemented mapping matches the described mapping, but it is not a circularity: the human-derived angles are inputs to the robot joint trajectories, and no output is reintroduced as an input to define the same quantity. The successful eight-hour performance is an external outcome, and the paper's claims are explicitly about having produced and staged that performance rather than about deriving it from first principles. No circular step, self-definitional reduction, fitted-input-as-prediction, or load-bearing self-citation chain is present.
Assumptions & free parameters
free parameters (4)
- FFT high-frequency threshold =
20
- Force threshold for cue detection =
20 N
- Smoothing kernel size =
15
- Sinusoid parameters (A, omega, phi, gamma) per motif =
manually tuned
assumptions (3)
- domain assumption OpenPose 2D keypoints accurately reflect human joint positions
- domain assumption Human arm joint angles computed via atan2 and differencing map meaningfully to UR5e joints
- domain assumption Simulation in urdfpy reliably predicts safety of the physical robot
Cite this review
Pith. "Pith review of Breathless: An 8-hour Performance Contrasting Human and Robot Expressiveness." pith.science (2026). https://pith.science/paper/KXRHNNHR
@misc{pith2026241112361,
author = {Pith},
title = {Pith review of: Breathless: An 8-hour Performance Contrasting Human and Robot Expressiveness},
year = {2026},
howpublished = {\url{https://pith.science/paper/KXRHNNHR}},
note = {Machine review of arXiv:2411.12361}
}
read the original abstract
This paper describes the robot technology behind an original performance that pairs a human dancer (Cuan) with an industrial robot arm for an eight-hour dance that unfolds over the timespan of an American workday. To control the robot arm, we combine a range of sinusoidal motions with varying amplitude, frequency and offset at each joint to evoke human motions common in physical labor such as stirring, digging, and stacking. More motions were developed using deep learning techniques for video-based human-pose tracking and extraction. We combine these pre-recorded motions with improvised robot motions created live by putting the robot into teach-mode and triggering force sensing from the robot joints onstage. All motions are combined with commercial and original music using a custom suite of python software with AppleScript, Keynote, and Zoom to facilitate on-stage communication with the dancer. The resulting performance contrasts the expressivity of the human body with the precision of robot machinery. Video, code and data are available on the project website: https://sites.google.com/playing.studio/breathless
Figures
Figures from the paper (6 more)
Reference graph
Works this paper leans on
-
[1]
J. Tinguely, M´eta-matic no. 10, 1959. [Online]. Available: https://www.youtube. com/watch?v=CIEYlUosKUY
work page 1959
- [2]
-
[3]
S. R. Labs, What is real? 1998. [Online]. Available: https://www.srl.org/shows/ web98/
work page 1998
-
[4]
Stelarc, Augmented and amplified body: Event for extended arm & sensor bracelet, 2023
work page 2023
-
[5]
Robot choreography: Moving in a new direction,
M. K. Apostolos, “Robot choreography: Moving in a new direction,”Leonardo, vol. 23, no. 1, pp. 25–29, 1990
work page 1990
-
[6]
W. Forsythe. “Black flags.” (2014), [Online]. Available: https://www.williamforsythe. com/installations.html
work page 2014
-
[7]
Forsythe, Choreographic objects, https://www.williamforsythe.com/essay
W. Forsythe, Choreographic objects, https://www.williamforsythe.com/essay. html, 2017. 14 Catie Cuan, Tianshuang Qiu, Shreya Ganti, Ken Goldberg
work page 2017
-
[8]
K. Ladenheim and A. LaViers, “Babyface,” inProceedings of the 7th International Conference on Movement and Computing, ser. MOCO ’20, Jersey City/Virtual, NJ, USA: Association for Computing Machinery, 2020, I S B N: 9781450375054. D O I: 10.1145/3401956.3404253. [Online]. Available: https://doi.org/10.1145/ 3401956.3404253
Show all 32 references
-
[9]
Style-based robotic motion in contem- porary dance performance,
A. LaViers, L. Teague, and M. Egerstedt, “Style-based robotic motion in contem- porary dance performance,” Controls and Art, pp. 205–229, 2014
2014
-
[10]
Choreographic and somatic approaches for the development of expressive robotic systems,
A. LaViers, C. Cuan, C. Maguire, et al., “Choreographic and somatic approaches for the development of expressive robotic systems,” inArts, MDPI, vol. 7, 2018, p. 11
2018
-
[11]
Do you love me?
B. Dynamics. “Do you love me?” Boston Dynamics. (2020), [Online]. Available: https://www.youtube.com/watch?v=fn3KWM1kuAw
2020
-
[12]
Choreography moves reference,
B. Dynamics. “Choreography moves reference,” Boston Dynamics. (2021), [On- line]. Available: https://dev.bostondynamics.com/docs/concepts/choreography/ move reference
2021
-
[13]
Uptown spot,
B. Dynamics. “Uptown spot,” Boston Dynamics. (2018), [Online]. Available: https://www.youtube.com/watch?v=kHBcVlqpvZ8
2018
-
[14]
Modulation of simple sinusoidal patterns by a coupled oscillator model for biped walking,
J. Morimoto, G. Endo, J. Nakanishi, et al., “Modulation of simple sinusoidal patterns by a coupled oscillator model for biped walking,” in Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006., 2006, pp. 1579–1584. D O I: 10.1109/ROBOT.200...
2006 arXiv
-
[15]
A biologically inspired biped locomotion strategy for humanoid robots: Modulation of sinusoidal patterns by a coupled oscillator model,
J. Morimoto, G. Endo, J. Nakanishi, and G. Cheng, “A biologically inspired biped locomotion strategy for humanoid robots: Modulation of sinusoidal patterns by a coupled oscillator model,” IEEE Transactions on Robotics, vol. 24, no. 1, pp. 185–191, 2008. D O I: 10.1109/TRO.2008.915457
2008
-
[16]
Nonholonomic motion planning: Steering using sinusoids,
R. M. Murray and S. S. Sastry, “Nonholonomic motion planning: Steering using sinusoids,” IEEE T. Automatic Control, vol. 38, no. 5, 1993
1993
-
[17]
Robotics, Ur5e robot arm, https://www.universal-robots.com/products/ur5- robot/, 2008
U. Robotics, Ur5e robot arm, https://www.universal-robots.com/products/ur5- robot/, 2008
2008
-
[18]
Interactive jamming with shimon: A social robotic musician,
G. Weinberg, A. Raman, and T. Mallikarjuna, “Interactive jamming with shimon: A social robotic musician,” in ACM/IEEE International Conference on Human- Robot Interaction, 2009, pp. 233–234
2009
-
[19]
Time to compile: An interactive art installa- tion,
C. Cuan, I. Pakrasi, and A. LaViers, “Time to compile: An interactive art installa- tion,” in Biennial Symposium on Arts & Technology, vol. 51, 2018, p. 19
2018
-
[20]
Live dance performance investigating the feminine cyborg metaphor with a motion-activatedwearable robot,
K. Ladenheim, R. McNish, W. Rizvi, and A. LaViers, “Live dance performance investigating the feminine cyborg metaphor with a motion-activatedwearable robot,” in ACM/IEEE International Conference on Human-Robot Interaction , 2020, pp. 243–251
2020
-
[21]
Beyonc ´e’s ‘renaissance’ tour has freaking robots,
J. Blistein. “Beyonc ´e’s ‘renaissance’ tour has freaking robots,” Rolling Stone. (2023), [Online]. Available: https://www.rollingstone.com/music/music-news/ beyonce-perform-cozy-robots-renaissance-tour-opener-1234733103/
2023
-
[22]
All together now
C. Hennick. “All together now.” (2021), [Online]. Available: https : / / www. bostondynamics.com/resources/blog/all-together-now
2021
-
[23]
When i’m gone,
K. Perry. “When i’m gone,” Katy Perry. (2022), [Online]. Available: https://www. youtube.com/watch?v=N-4YMlihRf4. Breathless: An 8-hour Human-Robot Dance Performance 15
2022
-
[24]
Designing robots with movement in mind,
G. Hoffman and W. Ju, “Designing robots with movement in mind,”Journal of Human-Robot Interaction, vol. 3, no. 1, pp. 91–122, 2014
2014
-
[25]
Curtain and time to compile: A demonstration of an experimental testbed for human-robot interaction,
C. Cuan, I. Pakrasi, E. Berl, and A. LaViers, “Curtain and time to compile: A demonstration of an experimental testbed for human-robot interaction,” in IEEE International Symposium on Robot and Human Interactive Communication, 2018, pp. 255–261
2018
-
[26]
Human-robot artistic co-creation: A study in improvised robot dance,
O. Th ¨orn, P. Knudsen, and A. Saffiotti, “Human-robot artistic co-creation: A study in improvised robot dance,” in 2020 29th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), IEEE, Aug. 2020, pp. 845–850
2020
-
[27]
Towards symbiotic creativity: A methodological approach to compare human and ai robotic dance creations,
A. De Filippo, L. Giuliani, E. Mancini, A. Borghesi, P. Mello, and M. Milano, “Towards symbiotic creativity: A methodological approach to compare human and ai robotic dance creations,” in IJCAI, Aug. 2023, pp. 5806–5814
2023
-
[28]
Dance teaching by a robot: Combining cognitive and physical human–robot inter- action for supporting the skill learning process,
D. F. P. Granados, B. A. Yamamoto, H. Kamide, J. Kinugawa, and K. Kosuge, “Dance teaching by a robot: Combining cognitive and physical human–robot inter- action for supporting the skill learning process,” IEEE Robotics and Automation Letters, vol. 2, no. 3, pp. 1452–1459, 2017
2017
-
[29]
Music mode: Transforming robot movement into music increases likability and perceived intelligence,
C. Cuan, E. Fisher, A. Okamura, and T. Engbersen, “Music mode: Transforming robot movement into music increases likability and perceived intelligence,” arXiv preprint arXiv:2306.02632, 2023
2023 arXiv
-
[30]
Realtime multi-person 2d pose estimation using part affinity fields,
Z. Cao, T. Simon, S. E. Wei, and Y . Sheikh, “Realtime multi-person 2d pose estimation using part affinity fields,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, IEEE, 2017, pp. 7291–7299
2017
-
[31]
[Online]
OpenRR, Urdfpy, Github repository. [Online]. Available: https://github.com/ openrr/urdfpy
-
[32]
Teaching machines to be human, and humans to live with machines,
B. Wolff. “Teaching machines to be human, and humans to live with machines,” Forbes Magazine. (2024), [Online]. Available: https://www.forbes.com/sites/ benjaminwolff/2024/03/24/teaching-machines-to-be-human-and-humans-to- live-with-machines/
2024
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.