Hand Shadow Art: A Differentiable Rendering Perspective

Aalok Gangopadhyay; Ashish Tiwari; Prajwal Singh; Shanmuganathan Raman

REVIEW 3 major objections 6 minor 8 references

Hand Shadow Art: A Differentiable Rendering Perspective

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that optimizing the joint angles and global placement of a parametric hand model with a differentiable silhouette renderer can make its cast shadow match a target image, and that the same optimization interpolates…

desk verdict A clean, honest application of differentiable shadow sculpting to parametric hands; genuinely new in two-hand coordination and failure transparency, but the value is as a proof of concept rather than a robust tool. read the letter →

arxiv 2505.21252 v1 pith:3DBM4PZ7 submitted 2025-05-27 cs.GR

Aalok Gangopadhyay , Prajwal Singh , Ashish Tiwari , Shanmuganathan Raman This is my paper

classification cs.GR

keywords handshadowartdifferentiablerenderingposeoptimizationMANOmodelsilhouettematchingpuppetryshapeinterpolationmeshintersectionpenalty

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that a desired shadow picture can be converted back into a hand pose, instead of leaving hand shadow art to manual trial and error. Starting from a parametric hand model with adjustable joints, the method renders its silhouette and uses the difference between that silhouette and a target image as a loss; because the renderer is differentiable, the loss tells the optimizer exactly how to bend each finger and move the hand. The authors demonstrate the idea on single-hand targets, on two-hand targets, and on interpolating between two target shadows, such as a rabbit morphing into a bird. A sympathetic reader would care because, if the approach works, anyone with a target silhouette and a light setup could in principle get a concrete, reproducible hand configuration that casts that silhouette.

What carries the argument

The central object is the differentiable silhouette renderer applied to a parametric hand model whose articulation is controlled by pose parameters: a shape code $\beta$ held fixed, per-joint angles $\theta$, a global rotation $Q$, and a translation $t$. Its role is to turn the objective $\|I - R(C, M_L, M_R)\|_2$ into gradients with respect to those parameters, so the optimizer can bend fingers in the direction that reduces the silhouette mismatch. The two supporting mechanisms are the intersection penalty, which keeps the fingers and the two hands from passing through each other, and the hard joint-angle limits, which keep intermediate poses anatomically plausible during interpolation.

What would settle it

Take a target silhouette that requires a large thumb rotation, initialize from several deliberately poor hand poses, and render each optimized result with an independent shadow renderer under the same light; if most runs fail to approach the target silhouette, the initialization dependence is confirmed, and if they succeed, the paper's stated limitation would be overturned.

Watch

Extended reading notes

Core claim

The paper's central claim is that hand shadow art can be generated by optimization rather than by hand-crafting poses: given a desired shadow image, the method adjusts the pose parameters of one or two parametric hand models until the rendered silhouette matches the target. Concretely, it minimizes the squared difference between the target image and the silhouette rendered by a differentiable renderer, with an added penalty that stops the fingers of a hand, or the two hands together, from intersecting, and with bounds that keep each of the 15 joints per hand within plausible angular ranges. With this objective, the paper reports reaching target shadows such as a rabbit and a bird, including shadows cast by two hands together, and it produces interpolation sequences that morph one target shadow into another through smooth pose changes. The shape parameters of the hand models are kept fixed; only joint angles, global rotations, and translations are optimized.

Load-bearing premise

The method assumes a good starting hand pose is available, because the paper shows that from a poor initial configuration the optimization cannot handle large thumb movements; without that starting point, the shadow match may never be reached.

Editorial extensions

If this is right

A target silhouette can be converted into a specific hand pose rather than a visual approximation, because the optimization directly adjusts pose parameters.
Two-hand shadow art is handled by optimizing both hand models together, with a penalty that keeps the hands from intersecting.
Given two target shadows, the optimizer produces a continuous sequence of hand poses whose rendered shadows animate from one target to the other.
The hand-crafted joint limits keep the optimized and intermediate poses anatomically plausible, which matters for transferring the results to real puppetry or character animation.
Because only pose parameters are optimized while shape stays fixed, the method isolates articulation as the source of the shadow match, making the result amenable to being replayed on other hand models.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Beyond the paper, a practical tool could start from an automatic coarse pose estimate, such as matching the blob positions of the target silhouette, so the optimization does not depend on a lucky random initialization; the paper's own failure example on thumb transitions suggests this is the main obstacle to deployment.
Beyond the paper, the same silhouette-matching objective with an intersection penalty could be applied to other articulated characters, not just hands, since nothing in the loss is hand-specific.
Beyond the paper, a single optimized pose could be evaluated under multiple lighting directions; the paper notes multi-view shadow art as future work, which would require adding a view-consistency term to the objective.
Beyond the paper, a natural test of the method's realism is to fabricate the optimized hand pose and photograph its shadow; a match under real light would show that the differentiable renderer's silhouette prediction transfers to physical shadow formation.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: this is a legitimate, clearly written extension of differentiable shadow sculpting to parametric hand models, with two-hand coordination and pose interpolation. It is not a big algorithmic leap, but it does what it claims for the examples shown, and it is honest about where it fails.

The genuinely new bits: nobody before has applied the Sadekar et al./Mitra–Pauly type optimization to MANO hands, so you get joint-angle limits, a penetration penalty for two hands, and shadow-to-shadow interpolation. The authors cite the right prior work and do not oversell novelty. The failure case in Figure 4 is a point in their favor—they show a thumb transition that breaks from a bad starting pose instead of hiding it.

The soft spots are the ones the authors acknowledge, plus a couple they do not. The evaluation is entirely qualitative. No silhouette IoU, no pixel error, no baseline comparison, no ablations of the loss weights or joint limits, and no code. The central claim—that you can match an arbitrary target shadow from a random initialization—is undermined by their own admission that convergence depends on a good starting pose. Section 3 says "We start with a random initial configuration" and then a few paragraphs later says a good initial configuration is needed and shows failures. That is an inconsistency in how the method is presented, but not a deep flaw; it just means the method is initialization-dependent, which is common in non-convex pose optimization. The interpolation section is thin: they show a rabbit-to-bird sequence but give no detail on how the intermediate poses are generated or whether the path is smooth in any controlled sense. That is a minor gap.

The citation pattern looks fine; the references are appropriate and not inflated. No invented constants; the free parameters are clearly listed joint limits, penetration weight, iterations.

Who gets value: people working on shadow art, digital puppetry, or authoring hand poses from silhouettes. It is also a decent teaching example of differentiable rendering for inverse problems, since the code path is simple and the limitations are visible. For a serious referee: I would send this to peer review, because it is a competent proof of concept with honest failure analysis, but I would require an initialization sensitivity study, quantitative metrics, and a code or data release before accepting. As presented, it is a 'conditional accept' work, not a fully supported tool.

Referee Report

3 major / 6 minor

Summary. The paper proposes a differentiable-rendering optimization framework for hand shadow art. Given a target silhouette image, the method optimizes MANO hand model parameters (joint angles, global rotation, translation) for one or two hands so that the rendered hand silhouette matches the target. It adds a penetration loss to avoid self- and cross-intersections and hand-crafted per-joint angular limits for plausibility. The authors also demonstrate interpolation between two target shadows and qualitatively show several examples. The paper is a short Pacific Graphics 2023 paper and reports no quantitative metrics, baselines, or ablations.

Significance. If the result holds, this would be a novel and useful extension of shadow art to articulated hand models, with potential applications in animation, entertainment, and puppetry. The idea of using differentiable silhouette rendering with physically motivated regularizers is sensible and follows standard inverse rendering practice. However, the validation is purely qualitative and the paper itself admits a strong sensitivity to initialization. The contribution is currently a proof-of-concept rather than a fully evaluated method; with additional quantitative evidence and a robust initialization strategy, it could become a solid graphics contribution.

major comments (3)

[§3, Figure 4] The paper claims in §3 that 'We start with a random initial configuration of hand models to reach the final pose,' but the same section concedes that 'a good initial hand configuration makes the convergence faster and easier, otherwise it leads to incorrect results,' and Figure 4 documents a failure on a large thumb transition. Since the optimization is non-convex over roughly 45 pose parameters per hand, the central claim that the method can produce a desired shadow from an arbitrary start is not established. Please either provide an automatic initialization or warm-start strategy, or report a quantitative success rate over multiple random initializations and characterize the basin of convergence. Without this, the method is demonstrated only on hand-picked starting poses.
[§2–3, evaluation] The paper reports no quantitative metrics for shadow fidelity. The objective is ||I − R(...)||_2, but no silhouette IoU, pixel error, or Chamfer distance is reported on a test set of target images; there is also no comparison to prior shadow-art methods [MP09, STR22] and no ablation of the loss weights, pen loss, or joint limits in Table 1. As a result, the reader cannot judge how well the optimization converges, how sensitive results are to the hand-crafted regularizers, or whether the visual examples are representative. Please add quantitative evaluation on multiple targets with baseline comparisons and ablations of the loss terms.
[§3, interpolation] The interpolation claim in §3 is under-specified. The paper states that a sequence (M^0_L, M^T_L) is found with endpoints matching IA and IB, but it does not describe the optimization for the intermediate frames: is it independent per-frame optimization, linear interpolation in parameter space, or an optimization with a temporal smoothness term? No equation or algorithm is given. This missing detail prevents reproduction and makes the interpolation result ambiguous. Please provide the full formulation for the sequence and, if applicable, the temporal regularization.

minor comments (6)

[§2] The notation is confusing: the paper writes "Let I* = [0,1]^{H×W} denote the space of all grayscale images" and then "Let I ∈ I* be a given target image," using the same symbol for the image space and a specific image. Rename the space or the target image to avoid ambiguity.
[Table 1] The table lists rotation angles as (θ,φ,ψ) but does not specify the units (degrees are presumably intended) or clarify whether the values are lower/upper bounds or offsets from a canonical pose.
[Figure 2] The caption mentions optimization over 5000 iterations, but no convergence curve or final loss value is shown; adding such a curve would help the reader assess whether the optimizations actually converge.
[References] The pen loss is cited to [Kar12], which is about BVH construction; a more direct reference for a surface penetration loss would be more appropriate, or the connection should be explained.
[Figure 3] The text says "Here is a link to a dynamic visualization" but provides no actual URL; this should be a working link or a reference to supplementary material.
[CCS Concepts] The CCS classification includes "Neural Nets," but the paper does not use neural networks; this category appears to be misapplied.

Circularity Check

0 steps flagged · score 1.0 of 10

No meaningful circularity; the target-shadow match is the optimization objective itself, and the one self-citation is not load-bearing.

full rationale

The paper's derivation chain is an inverse-rendering optimization. The objective in Section 2 is to minimize ||I - R(C, M_L, M_R)||^2 over MANO pose parameters, which is a fitting task rather than a prediction: the final shadow images are the optimized outputs, not independent claims. The success of a match is not forced by construction because the forward map from hand pose to silhouette is nontrivial; indeed, Section 3 and Figure 4 report failure cases from poor initialization, demonstrating that the optimization has genuine empirical content. The only self-citation is [STR22] (Shadow art revisited), used in the introduction to position the work and to point to prior differentiable-rendering shadow art; it is not load-bearing in the derivation of the pose optimization. The MANO hand model [RTB22], the PyTorch3D renderer [RRN*20], and the intersection penalty [TBS*16] are external, independently established components. No parameter is fitted to a subset of data and then reported as a prediction; no uniqueness theorem is imported; no ansatz is smuggled via citation; and no known result is renamed. Therefore no circular step reaches the threshold required by the review rules.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new physical entities. Its central claim rests on prior tools (MANO, PyTorch3D) and on hand-picked loss weights and joint limits, all of which are treated as given without independent validation.

free parameters (3)

Per-joint angular limits (Table 1) = Various (e.g., Right Hand Index J.1: (0.0, -15.0, -35.0) degrees)
These manually chosen bounds on joint rotations define the reachable pose space; they are ad hoc and not derived from data or anatomy, and they directly affect whether target shadows can be achieved.
Penetration loss weight = Not reported
The penalty for mesh intersections is added to the L2 silhouette loss, but its weighting is not specified, making a key hyperparameter a hand-chosen free parameter.
Optimization iterations = 5000
The number of gradient steps is set to 5000; this is a hand-chosen hyperparameter rather than a data-derived quantity.

assumptions (4)

domain assumption The MANO parametric hand model provides a sufficient and accurate representation of human hand geometry and articulation for shadow casting.
The paper fixes beta and optimizes only theta, Q, t; it does not validate that MANO's shape and pose space can express the hand configurations required to cast arbitrary target shadows (Section 2).
domain assumption Differentiable silhouette rendering in PyTorch3D gives gradients that lead to correct optimization towards shadow matches.
The method relies on Ravi et al. (2020) for differentiable rendering; this is a standard tool but its accuracy for thin structures like fingers is not discussed.
domain assumption Pixel-wise L2 distance between rendered silhouette and target image is an adequate perceptual measure of shadow similarity.
The objective is a squared L2 norm on images; the paper does not consider alternative shadow-matching metrics or the effect of soft shadows and lighting.
ad hoc to paper Pen loss and joint limits are sufficient regularizers to produce physically plausible, non-intersecting hand poses.
The joint limits are hand-picked (Table 1) and the pen loss weight is not reported; these choices are specific to this paper and not independently justified.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Hand Shadow Art: A Differentiable Rendering Perspective." pith.science (2026). https://pith.science/paper/3DBM4PZ7

@misc{pith2026250521252,
  author       = {Pith},
  title        = {Pith review of: Hand Shadow Art: A Differentiable Rendering Perspective},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3DBM4PZ7}},
  note         = {Machine review of arXiv:2505.21252}
}

read the original abstract

Shadow art is an exciting form of sculptural art that produces captivating artistic effects through the 2D shadows cast by 3D shapes. Hand shadows, also known as shadow puppetry or shadowgraphy, involve creating various shapes and figures using your hands and fingers to cast meaningful shadows on a wall. In this work, we propose a differentiable rendering-based approach to deform hand models such that they cast a shadow consistent with a desired target image and the associated lighting configuration. We showcase the results of shadows cast by a pair of two hands and the interpolation of hand poses between two desired shadow images. We believe that this work will be a useful tool for the graphics community.

Figures

Figures reproduced from arXiv: 2505.21252 by the authors.

**Figure 1.** Different hand configurations to render the target shadow image during the optimization process from one hand (first row) and two hands (second and third row). The images are rendered in 256×256 resolution. 2. Method The design of the proposed framework is described in [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗

**Figure 4.** Failure cases over large and abrupt transitions and sensitivity to the initial hand configuration. Joints Name/ Locations RH Index RH Middle RH Little RH Ring RH Thumb LH Index LH Middle LH Little LH Ring LH Thumb J.1 (θ,φ,ψ) 0.0, -15.0, -35.0 0.0, -25.0, -15.0 0.0, -40.0, -35.0 0.0, -15.0, -35.0 -50.0, -10.0, -30.0 0.0, 25.0, 75.0 0.0, 10.0, 75.0 0.0, 25.0, 75.0 0.0, 15.0, 75.0 20.0, 30.0, 40.0 J.2 (θ,φ,ψ) 0.0, 0.… view at source ↗

**Figure 3.** Different hand configurations (top: hand meshes, bottom: rendered image) while interpolating from a Rabbit to a Bird silhouette. Here is a link to a dynamic visualization. ters (βL and βR) fixed. The non-rigid nature of hands and fingers poses an additional challenge apart from rigid transformations i.e. rotations and translations and mesh intersections towards learning optimal hand configurations. To avoid self-in… view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

8 extracted references · 4 canonical work pages

[1]

write newline

" write newline "" before.all 'output.state := FUNCTION fin.entry add.period write newline FUNCTION new.block output.state before.all = 'skip after.block 'output.state := if FUNCTION new.sentence output.state after.block = 'skip output.state before.all = 'skip after.sentence 'output.state := if if FUNCTION not #0 #1 if FUNCTION and 'skip pop #0 if FUNCTIO...
[2]

write newline

" write newline "" before.all 'output.state := FUNCTION fin.entry.original add.period write newline FUNCTION new.block output.state before.all = 'skip after.block 'output.state := if FUNCTION new.sentence output.state after.block = 'skip output.state before.all = 'skip after.sentence 'output.state := if if FUNCTION not #0 #1 if FUNCTION and 'skip pop #0 i...
[3]

: Maximizing parallelism in the construction of bvhs, octrees, and k-d trees

Karras T. : Maximizing parallelism in the construction of bvhs, octrees, and k-d trees. In Proceedings of the Fourth ACM SIGGRAPH / Eurographics Conference on High-Performance Graphics (2012), Eurographics Association, pp. 33--37

work page 2012
[4]

J., Pauly M

Mitra N. J., Pauly M. : Shadow art. ACM Transactions on Graphics 28, CONF (2009), 156--1

work page 2009
[5]

: Accelerating 3d deep learning with pytorch3d

Ravi N., Reizenstein J., Novotny D., Gordon T., Lo W.-Y., Johnson J., Gkioxari G. : Accelerating 3d deep learning with pytorch3d. arXiv:2007.08501 (2020)

arXiv 2020
[6]

Romero J., Tzionas D., Black M. J. : Embodied hands: Modeling and capturing hands and bodies together. arXiv preprint arXiv:2201.02610 (2022)

arXiv 2022
[7]

: Shadow art revisited: a differentiable rendering based approach

Sadekar K., Tiwari A., Raman S. : Shadow art revisited: a differentiable rendering based approach. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (2022), pp. 29--37

work page 2022
[8]

: Capturing hands in action using discriminative salient points and physics simulation

Tzionas D., Ballan L., Srikantha A., Aponte P., Pollefeys M., Gall J. : Capturing hands in action using discriminative salient points and physics simulation. International Journal of Computer Vision (IJCV) 118, 2 (June 2016), 172--193

work page 2016

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

write newline

" write newline "" before.all 'output.state := FUNCTION fin.entry add.period write newline FUNCTION new.block output.state before.all = 'skip after.block 'output.state := if FUNCTION new.sentence output.state after.block = 'skip output.state before.all = 'skip after.sentence 'output.state := if if FUNCTION not #0 #1 if FUNCTION and 'skip pop #0 if FUNCTIO...

[2] [2]

write newline

" write newline "" before.all 'output.state := FUNCTION fin.entry.original add.period write newline FUNCTION new.block output.state before.all = 'skip after.block 'output.state := if FUNCTION new.sentence output.state after.block = 'skip output.state before.all = 'skip after.sentence 'output.state := if if FUNCTION not #0 #1 if FUNCTION and 'skip pop #0 i...

[3] [3]

: Maximizing parallelism in the construction of bvhs, octrees, and k-d trees

Karras T. : Maximizing parallelism in the construction of bvhs, octrees, and k-d trees. In Proceedings of the Fourth ACM SIGGRAPH / Eurographics Conference on High-Performance Graphics (2012), Eurographics Association, pp. 33--37

work page 2012

[4] [4]

J., Pauly M

Mitra N. J., Pauly M. : Shadow art. ACM Transactions on Graphics 28, CONF (2009), 156--1

work page 2009

[5] [5]

: Accelerating 3d deep learning with pytorch3d

Ravi N., Reizenstein J., Novotny D., Gordon T., Lo W.-Y., Johnson J., Gkioxari G. : Accelerating 3d deep learning with pytorch3d. arXiv:2007.08501 (2020)

arXiv 2020

[6] [6]

Romero J., Tzionas D., Black M. J. : Embodied hands: Modeling and capturing hands and bodies together. arXiv preprint arXiv:2201.02610 (2022)

arXiv 2022

[7] [7]

: Shadow art revisited: a differentiable rendering based approach

Sadekar K., Tiwari A., Raman S. : Shadow art revisited: a differentiable rendering based approach. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (2022), pp. 29--37

work page 2022

[8] [8]

: Capturing hands in action using discriminative salient points and physics simulation

Tzionas D., Ballan L., Srikantha A., Aponte P., Pollefeys M., Gall J. : Capturing hands in action using discriminative salient points and physics simulation. International Journal of Computer Vision (IJCV) 118, 2 (June 2016), 172--193

work page 2016