Pith. sign in

REVIEW 4 major objections 5 minor 57 references

FairHuman: Boosting Hand and Face Quality in Human Image Generation with Minimum Potential Delay Fairness in Diffusion Models

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read FairHuman claims a fairness rule for balancing diffusion losses improves hands and faces without hurting overall quality.

desk verdict Promising empirical recipe for improving hands/faces in human generation, but Eq. (8)/(10) make the MPD-optimality claims unsupported; fix the math and add error bars before trusting it. read the letter →

arxiv 2507.02714 v1 pith:S726PQSM submitted 2025-07-03 cs.CV cs.AI

Yuxuan Wang , Tianwei Cao , Huayu Zhang , Zhongjiang He , Kongming Liang , Zhanyu Ma This is my paper
classification cs.CVcs.AI
keywords diffusionmodelshumanimagegenerationminimumpotentialdelayfairnessmulti-objectiveoptimizationLoRAfine-tuningControlNetfaceandhandParetooptimality
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that the reason diffusion models render faces and hands poorly in full-scene human images is that the default mean-squared-error training loss weighs these small regions too lightly, and that this can be fixed during fine-tuning rather than by architecture changes. FairHuman formulates human image generation as a three-objective problem: global image quality, face quality, and hand quality, with face and hand losses computed by masking the same noise-prediction error. It then uses the Minimum Potential Delay (MPD) fairness criterion, borrowed from network bandwidth allocation, to choose each update step as a weighted combination of the three task gradients. The paper reports that fine-tuning SDXL LoRA or ControlNet modules this way improves hand and face detection confidence and image FID while keeping or slightly improving global preference scores. If correct, the contribution is a modular, task-agnostic recipe for balancing region-specific and global objectives in diffusion fine-tuning.

What carries the argument

The load-bearing object is the multi-objective update direction $d = \sum_i w_i \nabla l_i(\theta)$, with weights $w_i$ chosen by Minimum Potential Delay (MPD) fairness: minimize $F(d) = \sum_i 1/(\nabla l_i(\theta)^\top d)$, treating the shared update direction as a resource whose 'transmission rate' to objective $i$ is the projection of $d$ onto $\nabla l_i(\theta)$. The paper derives a closed-form weight vector $W = (G(\theta)^\top G(\theta))^{-2/3} \mathbf{1}$, where $G(\theta)$ stacks the three task gradients, and applies $\theta \leftarrow \theta - \eta d$ in each fine-tuning step. The face and hand masks that produce $l_{\text{face}}$ and $l_{\text{hand}}$ are obtained by reshaping pre-annotated bounding boxes into the latent-space shape, following the inpainting-mask convention. The MPD machinery does the work of dynamically shifting weight toward whichever objective is currently most delayed, which is how the method claims to protect global quality while forcing progress on the harder local regions.

What would settle it

Compute the Gram matrix $G(\theta)^\top G(\theta)$ during a real LoRA fine-tuning step and check its rank: if the global gradient lies in the span of the two masked gradients, the closed-form weight formula is not well-defined and MPD-fairness is not the operative mechanism. Separately, run the MPD-weighted and fixed-weight ablations on the same 5k prompts with several seeds and measure whether the reported hand and face confidence gains exceed seed-level variance; if they do not, the fairness weighting itself is not doing the work.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that insufficient training attention, not lack of model capacity, is what degrades local human details: SDXL already renders close-up faces and hands well, but in full-body images those regions contribute little to the global MSE and consequently receive too little gradient signal. FairHuman therefore replaces the single global diffusion loss with three losses---global, face-masked, and hand-masked, all computed on the same predicted-noise residual---and solves the resulting multi-objective problem with an MPD-fair gradient update. The claimed result is that this jointly fine-tuned model reaches a Pareto-optimal point where hand and face quality improve substantially (higher MediaPipe detection confidence, lower FID) while global quality, text alignment, and pose alignment are maintained or slightly improved, in both general text-to-image generation (LoRA on SDXL) and controllable generation (ControlNet).

Load-bearing premise

The method assumes that the three task gradients---global, face-masked, and hand-masked---are linearly independent and that the closed-form weight formula $W = (G(\theta)^\top G(\theta))^{-2/3} \mathbf{1}$ is a valid solution of the MPD optimality equations; since the global loss already includes the face and hand pixels, that independence is not automatic, and if it fails the claimed fairness and Pareto guarantees do not follow from the paper's derivation.

Editorial extensions

If this is right

  • Fine-tuning a single low-rank adapter with MPD-fair gradients gives better hand and face quality than a mixture-of-experts stack of separate hand and face adapters, while using roughly half the extra memory and about 1.4x instead of 3x inference time.
  • The same recipe transfers from LoRA to ControlNet, improving regional quality and pose alignment under control conditions.
  • The framework is extensible to additional human attributes such as feet and eyes by adding their masked losses as new objectives.
  • Because MPD weighting is a training-time gradient rule, it can be attached to existing diffusion backbones and parameter-efficient fine-tuning modules without changing the architecture or sampling procedure.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An untested extension of the same masked-loss-plus-MPD recipe is any small, semantically critical region in generated images, such as animal paws, text and logos, or small objects, where the region occupies few pixels and the global loss underweights it.
  • The paper's regional-mask construction may inherit limitations from the VAE latent resolution and from the detectors used to create the bounding-box priors; ablations varying mask dilation would isolate how much of the gain comes from the weighting rule versus the mask geometry, which the paper does not test.
  • Because the global loss already contains face and hand pixels, part of the reported local improvement could come from the masked losses reweighting those same pixels rather than from true gradient independence; a test with tighter segmentation masks would show whether the effect is genuinely localized.
  • The fairness criterion could be adapted to encode user priorities, such as weighting face quality higher for portrait prompts, with MPD supplying the balancing rule once those priorities are set; the paper only mentions adding preference feedback as future work.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. FairHuman proposes a multi-objective fine-tuning recipe for SDXL-based human image generation. Three losses are defined: a global diffusion MSE loss and two masked MSE losses for faces and hands. During LoRA or ControlNet fine-tuning, the three objective gradients are combined by weights derived, in the paper's claim, from a Minimum Potential Delay (MPD) fairness criterion. The authors assert that the resulting update direction is MPD-optimal and converges to a Pareto-optimal solution, and they report improved hand/face quality and comparable or better global quality on quantitative metrics (HPS, IR, FID, detection confidence) and in user studies.

Significance. The problem is well motivated: human image generation models often collapse local detail in face and hand regions, and existing solutions either use separate expert modules or post-processing. A single-module, multi-objective fine-tuning method that improves local details while preserving global quality would be practically useful for both LoRA and ControlNet deployments. The paper has strengths worth acknowledging: it is built on a plausible observation about regional supervision, it applies the method to two different fine-tuning schemes, it compares against appropriate baselines, it reports memory/inference overhead against MoLE, and it promises code release. If the theoretical claim were correct, the paper would offer a new principled way to balance multiple region-specific losses in diffusion models. As written, however, the central mathematical derivation does not support the MPD-optimality and Pareto-optimality claims, and the empirical gains are modest and reported without measures of variability.

major comments (4)
  1. [§3.3, Eqs. (6)–(10)] The derivation connecting Eq. (6) to Eq. (10) is not valid. The definition of G(θ) is ambiguous: if G(θ) is the 3×d matrix suggested by the displayed transpose notation, then G(θ)^T G(θ) is d×d and cannot multiply the 3-vector W in Eq. (8). If instead G is taken to be the d×3 matrix [∇l1, ∇l2, ∇l3], the substitution d = G W into Eq. (6) leads to the fixed-point relation (G^T G W)_i = W_i^{-1/2} only after identifying W_i with 1/(g_i^T d)^2; this identification itself requires an independent justification that the paper does not provide. The paper needs to define G with explicit dimensions and derive Eq. (8) from Eq. (6) step by step.
  2. [§3.3, Eq. (10)] Even if Eq. (8) is accepted as the intended fixed point M W = W^{-1/2}, the claimed closed-form solution W = M^{-2/3} 1 is not a solution for a non-diagonal positive-definite M. For example, in a two-objective analogue with M = [[3, 0.5], [0.5, 1]], setting W = M^{-2/3}1 gives M W ≈ (1.545, 1.097) while W^{-1/2} ≈ (1.662, 1.045), so the fixed-point equation is not satisfied. Since G(θ)^T G(θ) will generally be non-diagonal when the global loss and the masked losses overlap in pixels, the weights actually used in training are not the MPD-optimal weights asserted in the paper. The claim that the update (7) is MPD-optimal is therefore unsupported.
  3. [§3.3, last paragraph; §2 and §3.2] The invocation of Theorem 7.3 of [1] for Pareto optimality is not justified. The paper states that the objectives in Eq. (2) 'decouple regional and global optimization, satisfying MPD's prerequisite assumption of linearly independent task gradients,' but this is not true: l_global is an unweighted MSE over the full latent, while l_face and l_hand are masked versions of the same residual over subsets of the same pixels. The gradients are therefore overlapping and cannot be assumed linearly independent. Without a valid MPD-optimal weight solution and without verifying the theorem's conditions, the Pareto-optimality guarantee does not follow.
  4. [Tables 1–3 and §4.2] The empirical comparison is consistent but statistically thin. All quantitative results are single-run point estimates with no error bars, confidence intervals, or significance tests. The margins are small: for example, in Table 1 the HPS gap between Ours and MoLE is 0.86 (32.73 vs. 31.87), and in Table 2 the fairness-vs-no-fairness HPS gap is 0.13–0.16. Table 3 reports rank sensitivity without any variance estimate, and Table 1 averages over three test-time scales, which can hide sensitivity to that hyperparameter. The wording in the abstract, 'significant improvements,' is stronger than the evidence supports. The authors should report variance over seeds or prompt subsets and show the individual scale values, or soften the claim.
minor comments (5)
  1. [§3.3, Eq. (8)] Please define G(θ) explicitly as either a d×3 or 3×d matrix and state the orientation of W. The current notation '[∇l1(θ), ∇l2(θ), ∇l3(θ)]^T' is ambiguous and is directly related to the dimensional inconsistency noted above.
  2. [Table 2] The ablation row labels contain a typo: 'l2(θ)+l2(θ)+l3(θ)' should presumably read 'l1(θ)+l2(θ)+l3(θ)' in both the LoRA and ControlNet blocks.
  3. [§4.2, §4.5, §9.3] There are minor spelling and spacing issues: 'V olunteers' should be 'Volunteers', 'close-source methods' should be 'closed-source methods', and 'Dynamic Weigth Average' should be 'Dynamic Weight Average'.
  4. [§3.4, Eq. (12)] The argument order in the module notation is inconsistent: N_base is written as N_base(z; θ_base) while N_new is written as N_new(c, z; θ_update). Please settle on a single convention and state the conditioning inputs for the LoRA and ControlNet variants.
  5. [Supplementary §6.1] The derivation of Eq. (5) uses an incorrect expression for the projection magnitude: the projection of d onto ∇l_i(θ) should involve a factor 1/||∇l_i(θ)||, not 1/||d||. The upper-bound argument can likely be repaired, but as written it is not a valid proof of the claimed proportionality.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the MPD weighting is computed from current gradients rather than fitted to target metrics, and the empirical comparisons are external; the technical concern about Eq. (8)/(10) is a validity issue, not a circularity.

full rationale

The paper's fairness-aware weighting is not tuned to the evaluation metrics. In Sec. 3.3, the weights are derived from current gradients via the Minimum Potential Delay criterion, which is explicitly attributed to external networking literature [27,28] and to the algorithm of Ban & Ji [1], none of which are authored by the present paper's authors. No load-bearing step reduces to a self-citation or to a fitted parameter re-labeled as a prediction. The losses in Eq. (2) are masked MSE losses on predicted noise; the evaluation metrics in Tab. 1 (HPS, ImageReward, FID, detection confidence, pose mean difference) are independent of the training losses and are computed against external baselines with fixed seeds. The final weights W in Eq. (10) are computed per training step from the three task gradients, not fit to any held-out metric, so the reported improvements are not forced by construction. The genuine weakness is mathematical: Eq. (8) appears dimensionally inconsistent as written, Eq. (10) is not a valid closed-form solution of Eq. (8) for non-diagonal G^T G, and the linear-independence premise is questionable because the global loss includes face and hand pixels. These are correctness and validity concerns about the claimed MPD-optimality and Pareto-optimality guarantees, not circularity: the derivation does not encode the target metrics into its inputs. The mild concern that LoRA rank and control scales were selected using the same validation set used for evaluation may affect generalization claims, but it does not make any 'prediction' equivalent to its input by definition. Therefore the circularity score is 0.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on importing the MPD fairness framework and its convergence guarantees from [1], on the unverified assumption that the three task gradients are linearly independent, and on the assumption that masked MSE losses in latent space track perceived face and hand quality. The optimization also relies on an uncontrolled upper-bound proxy (Eq. 5) for the original MPD objective. No new physical or conceptual entities are introduced.

free parameters (3)
  • LoRA rank = 256
    Selected as the best among 64, 128, and 256 by evaluation on the validation set (Table 3). The main results use this rank, which constitutes hyperparameter tuning on the evaluation data.
  • Test-time LoRA scale = 0.3, 0.4, 0.5 (averaged)
    Main LoRA results are reported as the average over these three scales (Section 4.2), which is an evaluation-protocol choice that smooths over configuration sensitivity.
  • Test-time ControlNet scale = 0.4, 0.5, 0.6 (averaged)
    Main ControlNet results are reported as the average over these three scales (Section 4.2), affecting comparability with single-configuration baselines such as ControlNet-SDXL and ControlNet-Union.
assumptions (4)
  • domain assumption The MPD fairness criterion from [1] provides a valid and desirable objective for combining the three task gradients, and its convergence guarantee (Thm 7.3 in [1]) applies to this setting.
    Section 3.3 imports the MPD principle and its Pareto-optimality theorem from prior work without re-deriving or verifying its conditions beyond asserting linear independence.
  • domain assumption The three objective gradients, global, face-masked, and hand-masked, are linearly independent.
    Stated in Sec. 3.3: 'our constructed objective functions in Eq.(2) decouples regional and global optimization, satisfying MPD's prerequisite assumption of linearly independent task gradients.' Not proven; the global loss includes face/hand pixels, so coupling is plausible.
  • domain assumption Mask-based MSE losses defined on downsampled bounding-box masks in latent space are effective proxies for perceptual quality of faces and hands.
    The paper uses masks 'obtained by reshaping the mask image to the latent shape' (Sec. 3.2). It does not validate that these masked losses correlate with human judgment of face and hand realism.
  • ad hoc to paper The upper-bound proxy F'(d) in Eq. (5) is a faithful surrogate for the original MPD objective F(d) in Eq. (4).
    Sec. 6.1 only shows F(d) ≤ M F'(d) for a constant M, so minimizing F' minimizes an upper bound, but the gap between the two is uncontrolled and can be large when gradients have very different norms. This is a heuristic choice specific to this paper.

how reviews work

0 comments
Cite this review

Pith. "Pith review of FairHuman: Boosting Hand and Face Quality in Human Image Generation with Minimum Potential Delay Fairness in Diffusion Models." pith.science (2026). https://pith.science/paper/S726PQSM

@misc{pith2026250702714,
  author       = {Pith},
  title        = {Pith review of: FairHuman: Boosting Hand and Face Quality in Human Image Generation with Minimum Potential Delay Fairness in Diffusion Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/S726PQSM}},
  note         = {Machine review of arXiv:2507.02714}
}
read the original abstract

Image generation has achieved remarkable progress with the development of large-scale text-to-image models, especially diffusion-based models. However, generating human images with plausible details, such as faces or hands, remains challenging due to insufficient supervision of local regions during training. To address this issue, we propose FairHuman, a multi-objective fine-tuning approach designed to enhance both global and local generation quality fairly. Specifically, we first construct three learning objectives: a global objective derived from the default diffusion objective function and two local objectives for hands and faces based on pre-annotated positional priors. Subsequently, we derive the optimal parameter updating strategy under the guidance of the Minimum Potential Delay (MPD) criterion, thereby attaining fairness-ware optimization for this multi-objective problem. Based on this, our proposed method can achieve significant improvements in generating challenging local details while maintaining overall quality. Extensive experiments showcase the effectiveness of our method in improving the performance of human image generation under different scenarios.

Figures

Figures reproduced from arXiv: 2507.02714 by the authors.

Figure 1
Figure 1. Compare with existing methods under two scenarios: [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Examples of human-related images generated by SDXL. The first row shows the close-up view images with prompts like “a close-up human face” and “a detailed hand ges￾ture”. The second row shows the human images under different scenes. Zoom in for a better view of details. through iterative denoising processes. However, their ca￾pability of human image generation remains challenged by persistent artifacts in anatomical… view at source ↗
Figure 3
Figure 3. An overview of FairHuman framework. The entire pipeline is divided into three stages: 1) Construct multi objectives for human image generation; 2) Employ fairness-aware optimization strategy for parameter updates; 3) Apply the fine-tuned module into the backbone diffusion model. Here, we take LoRA[10] and ControlNet[50] as examples of different generation scenarios. the diffusion model; c represents the embedding of… view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: User study results in terms of overall quality and regional quality. We present the ratio of users’ preference for each method. their visual preferences based on overall and regional qual￾ity. The result is shown in Fig.4. It indicates that our method is more preferred…
Figure 5
Figure 5. Figure 5: Examples of the images generated by different set￾tings with the same prompt. First row: generated by LoRA. Sec￾ond row: generated by ControlNet. Zoom in for a better view of details [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Comparison with HanDiffuser [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]
Figure 7
Figure 7. Figure 7: Comparison with HyperHuman [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]
Figure 8
Figure 8. Figure 8: Comparison with general T2I methods [PITH_FULL_IMAGE:figures/full_fig_p016_8.png]
Figure 9
Figure 9. Figure 9: Comparison with controllable methods [PITH_FULL_IMAGE:figures/full_fig_p017_9.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

57 extracted references · 36 canonical work pages

  1. [1]

    Fair resource allocation in multi-task learning

    Hao Ban and Kaiyi Ji. Fair resource allocation in multi-task learning. arXiv preprint arXiv:2402.15638, 2024. 5, 6

    arXiv 2024

  2. [2]

    Improving image generation with better captions

    James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang, Linjie Li, Long Ouyang, Juntang Zhuang, Joyce Lee, Yufei Guo, et al. Improving image generation with better captions. Computer Science. https://cdn. openai. com/papers/dall-e-3. pdf, 2(3):8, 2023. 3

    2023

  3. [3]

    Adetailer

    Bing-su. Adetailer. https://github.com/Bing-su/ adetailer.git, 2024. 2

    work page 2024

  4. [4]

    Viton-hd: High-resolution virtual try-on via misalignment-aware normalization

    Seunghwan Choi, Sunghyun Park, Minsoo Lee, and Jaegul Choo. Viton-hd: High-resolution virtual try-on via misalignment-aware normalization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 14131–14140, 2021. 3

    work page 2021

  5. [5]

    Scaling recti- fied flow transformers for high-resolution image synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas M ¨uller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling recti- fied flow transformers for high-resolution image synthesis. In Forty-first International Conference on Machine Learn- ing, 2024. 3

    work page 2024

  6. [6]

    Stylegan-human: A data-centric odyssey of human genera- tion

    Jianglin Fu, Shikai Li, Yuming Jiang, Kwan-Yee Lin, Chen Qian, Chen Change Loy, Wayne Wu, and Ziwei Liu. Stylegan-human: A data-centric odyssey of human genera- tion. In European Conference on Computer Vision , pages 1–19. Springer, 2022. 3

    work page 2022

  7. [7]

    Mini-internvl: a flexible-transfer pocket multi-modal model with 5% parameters and 90% perfor- mance

    Zhangwei Gao, Zhe Chen, Erfei Cui, Yiming Ren, Weiyun Wang, Jinguo Zhu, Hao Tian, Shenglong Ye, Junjun He, Xizhou Zhu, et al. Mini-internvl: a flexible-transfer pocket multi-modal model with 5% parameters and 90% perfor- mance. Visual Intelligence, 2(1):1–17, 2024. 1

    work page 2024

  8. [8]

    Gans trained by a two time-scale update rule converge to a local nash equilib- rium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilib- rium. Advances in neural information processing systems , 30, 2017. 6

    2017

Show all 57 references
  1. [9]

    Denoising dif- fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020. 3

    2020

  2. [10]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021. 2, 3, 4, 6

    2021 arXiv

  3. [11]

    Minicpm: Unveiling the potential of small language models with scalable training strategies

    Shengding Hu, Yuge Tu, Xu Han, Chaoqun He, Ganqu Cui, Xiang Long, Zhi Zheng, Yewei Fang, Yuxiang Huang, Weilin Zhao, et al. Minicpm: Unveiling the potential of small language models with scalable training strategies. arXiv preprint arXiv:2404.06395, 2024. 1

    2024 arXiv

  4. [12]

    Catalin Ionescu, Dragos Papava, Vlad Olaru, and Cristian Sminchisescu. Human3. 6m: Large scale datasets and pre- dictive methods for 3d human sensing in natural environ- ments. IEEE transactions on pattern analysis and machine intelligence, 36(7):1325–1339, 2013. 3

    2013

  5. [13]

    Adaptive mixtures of local experts.Neu- ral computation, 3(1):79–87, 1991

    Robert A Jacobs, Michael I Jordan, Steven J Nowlan, and Geoffrey E Hinton. Adaptive mixtures of local experts.Neu- ral computation, 3(1):79–87, 1991. 3

    1991

  6. [14]

    Hu- man image generation: A comprehensive survey.ACM Com- puting Surveys, 56(11):1–39, 2024

    Zhen Jia, Zhang Zhang, Liang Wang, and Tieniu Tan. Hu- man image generation: A comprehensive survey.ACM Com- puting Surveys, 56(11):1–39, 2024. 3

    2024

  7. [15]

    Human-art: A versatile human-centric dataset bridg- ing natural and artificial scenes

    Xuan Ju, Ailing Zeng, Jianan Wang, Qiang Xu, and Lei Zhang. Human-art: A versatile human-centric dataset bridg- ing natural and artificial scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 618–629, 2023. 3

    2023

  8. [16]

    Humansd: A native skeleton-guided diffusion model for human image generation

    Xuan Ju, Ailing Zeng, Chenchen Zhao, Jianan Wang, Lei Zhang, and Qiang Xu. Humansd: A native skeleton-guided diffusion model for human image generation. InProceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 15988–15998, 2023. 2, 3

    2023

  9. [17]

    Multi-task learning using uncertainty to weigh losses for scene geome- try and semantics

    Alex Kendall, Yarin Gal, and Roberto Cipolla. Multi-task learning using uncertainty to weigh losses for scene geome- try and semantics. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7482–7491,

  10. [18]

    Auto-encoding variational bayes

    Diederik P Kingma. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013. 3

    2013 arXiv

  11. [19]

    Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation

    Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In Interna- tional conference on machine learning, pages 12888–12900. PMLR, 2022. 6

    2022

  12. [20]

    Con- trolnet++: Improving conditional controls with efficient consistency feedback: Project page: liming-ai

    Ming Li, Taojiannan Yang, Huafeng Kuang, Jie Wu, Zhaoning Wang, Xuefeng Xiao, and Chen Chen. Con- trolnet++: Improving conditional controls with efficient consistency feedback: Project page: liming-ai. github. io/controlnet plus plus. In European Conference on Com- puter Visio...

    2024

  13. [21]

    Cosmicman: A text-to-image foun- dation model for humans

    Shikai Li, Jianglin Fu, Kaiyuan Liu, Wentao Wang, Kwan- Yee Lin, and Wayne Wu. Cosmicman: A text-to-image foun- dation model for humans. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 6955–6965, 2024. 3, 6

    2024

  14. [22]

    Reasonable effectiveness of random weighting: A litmus test for multi-task learning

    Baijiong Lin, Feiyang Ye, Yu Zhang, and Ivor W Tsang. Reasonable effectiveness of random weighting: A litmus test for multi-task learning. arXiv preprint arXiv:2111.10603 ,

    arXiv

  15. [23]

    End- to-end multi-task learning with attention

    Shikun Liu, Edward Johns, and Andrew J Davison. End- to-end multi-task learning with attention. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1871–1880, 2019. 8, 3

    2019

  16. [24]

    Hyperhuman: Hyper-realistic human gener- ation with latent structural diffusion

    Xian Liu, Jian Ren, Aliaksandr Siarohin, Ivan Skorokhodov, Yanyu Li, Dahua Lin, Xihui Liu, Ziwei Liu, and Sergey Tulyakov. Hyperhuman: Hyper-realistic human gener- ation with latent structural diffusion. arXiv preprint arXiv:2310.08579, 2023. 2, 3, 8

    arXiv

  17. [25]

    Handrefiner: Refining malformed hands in generated images by diffusion-based conditional inpainting

    Wenquan Lu, Yufei Xu, Jing Zhang, Chaoyue Wang, and Dacheng Tao. Handrefiner: Refining malformed hands in generated images by diffusion-based conditional inpainting. In Proceedings of the 32nd ACM International Conference on Multimedia, pages 7085–7093, 2024. 3, 4

    2024

  18. [26]

    Medi- apipe: A framework for building perception pipelines

    Camillo Lugaresi, Jiuqiang Tang, Hadon Nash, Chris Mc- Clanahan, Esha Uboweja, Michael Hays, Fan Zhang, Chuo- Ling Chang, Ming Guang Yong, Juhyun Lee, et al. Medi- apipe: A framework for building perception pipelines. arXiv preprint arXiv:1906.08172, 2019. 6

    1906 arXiv

  19. [27]

    Bandwidth sharing: objectives and algorithms

    Laurent Massouli ´e and James Roberts. Bandwidth sharing: objectives and algorithms. In IEEE INFOCOM’99. Con- ference on Computer Communications. Proceedings. Eigh- teenth Annual Joint Conference of the IEEE Computer and Communications Societies. The Future is Now (Cat. No. 99...

    1999

  20. [28]

    Fair end-to-end window- based congestion control

    Jeonghoon Mo and Jean Walrand. Fair end-to-end window- based congestion control. IEEE/ACM Transactions on net- working, 8(5):556–567, 2000. 2, 5

    2000

  21. [29]

    Whose hands are these? hand detec- tion and hand-body association in the wild

    Supreeth Narasimhaswamy, Thanh Nguyen, Mingzhen Huang, and Minh Hoai. Whose hands are these? hand detec- tion and hand-body association in the wild. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4889–4899, 2022. 3, 6

    2022

  22. [30]

    Han- diffuser: Text-to-image generation with realistic hand ap- pearances

    Supreeth Narasimhaswamy, Uttaran Bhattacharya, Xiang Chen, Ishita Dasgupta, Saayan Mitra, and Minh Hoai. Han- diffuser: Text-to-image generation with realistic hand ap- pearances. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 2468...

    2024

  23. [31]

    Multi- task learning as a bargaining game

    Aviv Navon, Aviv Shamsian, Idan Achituve, Haggai Maron, Kenji Kawaguchi, Gal Chechik, and Ethan Fetaya. Multi- task learning as a bargaining game. arXiv preprint arXiv:2202.01017, 2022. 1, 3

    2022 arXiv

  24. [32]

    Pareto multi objective optimization

    Patrick Ngatchou, Anahita Zarei, and A El-Sharkawi. Pareto multi objective optimization. In Proceedings of the 13th in- ternational conference on, intelligent systems application to power systems, pages 84–91. IEEE, 2005. 5

    2005

  25. [33]

    Recon- structing hands in 3d with transformers

    Georgios Pavlakos, Dandan Shan, Ilija Radosavovic, Angjoo Kanazawa, David Fouhey, and Jitendra Malik. Recon- structing hands in 3d with transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9826–9836, 2024. 3, 1

    2024

  26. [34]

    Sdxl: Improving latent diffusion mod- els for high-resolution image synthesis

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas M ¨uller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion mod- els for high-resolution image synthesis. arXiv preprint arXiv:2307.01952, 2023. 1, 2, 3, 4, 6, 7

    2023 arXiv

  27. [35]

    From sparse to soft mixtures of experts

    Joan Puigcerver, Carlos Riquelme, Basil Mustafa, and Neil Houlsby. From sparse to soft mixtures of experts. arXiv preprint arXiv:2308.00951, 2023. 2

    2023 arXiv

  28. [36]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  29. [37]

    Hierarchical text-conditional image gener- ation with clip latents

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image gener- ation with clip latents. arXiv preprint arXiv:2204.06125, 1 (2):3, 2022. 3

    2022 arXiv

  30. [38]

    You only look once: Unified, real-time object de- tection

    J Redmon. You only look once: Unified, real-time object de- tection. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2016. 3

    2016

  31. [39]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 3

    2022

  32. [40]

    Stylegan-t: Unlocking the power of gans for fast large-scale text-to-image synthesis

    Axel Sauer, Tero Karras, Samuli Laine, Andreas Geiger, and Timo Aila. Stylegan-t: Unlocking the power of gans for fast large-scale text-to-image synthesis. In International con- ference on machine learning , pages 30105–30118. PMLR,

  33. [41]

    Multi-task learning as multi-objective optimization

    Ozan Sener and Vladlen Koltun. Multi-task learning as multi-objective optimization. Advances in neural informa- tion processing systems, 31, 2018. 5

    2018

  34. [42]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020. 3

    2010 arXiv

  35. [43]

    Mixture-of- hand-experts: Repainting the deformed hand images gener- ated by diffusion models

    Yuxuan Wang, Tianwei Cao, Kongming Liang, Zhongjiang He, Hao Sun, Yongxiang Li, and Zhanyu Ma. Mixture-of- hand-experts: Repainting the deformed hand images gener- ated by diffusion models. In Chinese Conference on Pattern Recognition and Computer Vision (PRCV) , pages 143–157...

    2024

  36. [44]

    Human preference score v2: A solid benchmark for evaluating human preferences of text-to-image synthesis

    Xiaoshi Wu, Yiming Hao, Keqiang Sun, Yixiong Chen, Feng Zhu, Rui Zhao, and Hongsheng Li. Human preference score v2: A solid benchmark for evaluating human preferences of text-to-image synthesis. arXiv preprint arXiv:2306.09341 ,

    arXiv

  37. [45]

    Imagere- ward: Learning and evaluating human preferences for text- to-image generation

    Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. Imagere- ward: Learning and evaluating human preferences for text- to-image generation. Advances in Neural Information Pro- cessing Systems, 36:15903–15935, 2023. 6

    2023

  38. [46]

    Effec- tive whole-body pose estimation with two-stages distillation

    Zhendong Yang, Ailing Zeng, Chun Yuan, and Yu Li. Effec- tive whole-body pose estimation with two-stages distillation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4210–4220, 2023. 3, 1

    2023

  39. [47]

    Gradient surgery for multi-task learning

    Tianhe Yu, Saurabh Kumar, Abhishek Gupta, Sergey Levine, Karol Hausman, and Chelsea Finn. Gradient surgery for multi-task learning. In Advances in Neural Information Pro- cessing Systems, pages 5824–5836, 2020. 5

    2020

  40. [48]

    Stack- gan: Text to photo-realistic image synthesis with stacked generative adversarial networks

    Han Zhang, Tao Xu, Hongsheng Li, Shaoting Zhang, Xiao- gang Wang, Xiaolei Huang, and Dimitris N Metaxas. Stack- gan: Text to photo-realistic image synthesis with stacked generative adversarial networks. In Proceedings of the IEEE international conference on computer vision , p...

    2017

  41. [49]

    Stack- gan++: Realistic image synthesis with stacked generative ad- versarial networks

    Han Zhang, Tao Xu, Hongsheng Li, Shaoting Zhang, Xiao- gang Wang, Xiaolei Huang, and Dimitris N Metaxas. Stack- gan++: Realistic image synthesis with stacked generative ad- versarial networks. IEEE transactions on pattern analysis and machine intelligence, 41(8):1947–1962, 2018. 3

    1947

  42. [50]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3836–3847, 2023. 1, 2, 3, 4, 6, 7

    2023

  43. [51]

    Scalable person re-identification: A benchmark

    Liang Zheng, Liyue Shen, Lu Tian, Shengjin Wang, Jing- dong Wang, and Qi Tian. Scalable person re-identification: A benchmark. In Proceedings of the IEEE international con- ference on computer vision, pages 1116–1124, 2015. 3

    2015

  44. [52]

    Mole: Enhancing human-centric text- to-image diffusion via mixture of low-rank experts

    Jie Zhu, Yixiong Chen, Mingyu Ding, Ping Luo, Leye Wang, and Jingdong Wang. Mole: Enhancing human-centric text- to-image diffusion via mixture of low-rank experts. arXiv preprint arXiv:2410.23332, 2024. 1, 2, 3, 6, 7 FairHuman: Boosting Hand and Face Quality in Human Image Gen...

    2024 arXiv

  45. [53]

    Proofs 6.1. Derivation of Eq.(5) Assuming there exists a constant M such that ∥∇li(θ)∥ ≤ M, we have: F (d) = 3X i=1 1 ∥proj∇li(θ)(d)∥ = 3X i=1 1 ∇li(θ)⊤d/∥d∥ = 3X i=1 ∥d∥ ∇li(θ)⊤d ≤ 3X i=1 M ∇li(θ)⊤d = M 3X i=1 1 ∇li(θ)⊤d Since F ′(d) =P3 i=1 1 ∇li(θ)⊤d, we can get that: F (d)...

  46. [54]

    Dataset Details 7.1. Construction of Training Set About the two public datasets we use, CosmicMan is specif- ically designed for generating highly realistic and photore- alistic human images, containing 6 million high-resolution real-world human images and detailed description...

  47. [55]

    Regarding the gen- eration of local masks for the faces and hands, we first gen- erate mask images using the pre-annotated bounding box information from the dataset

    Implementation Details We choose Stable Diffusion XL as the base model since it already possesses prior knowledge about human image generation through pre-training but has major limitations in generating local hand and face details. Regarding the gen- eration of local masks fo...

  48. [56]

    More Quantitative Comparisons In addition to comparing the image quality generated by the models, we also conduct experiments on memory usage and inference speed

    More Evaluation Details 9.1. More Quantitative Comparisons In addition to comparing the image quality generated by the models, we also conduct experiments on memory usage and inference speed. Specifically, both our LoRA-based method and MoLE[52] are deployed on SDXL[34] as the...

  49. [57]

    In addition to the baseline methods mentioned in our paper, we also com- pare two close-source methods for human image generation called HanDiffuser[30] and HyperHuman[24]

    More Visualizations More visualization results compared to existing methods are demonstrated in Fig.6, Fig.7, Fig.8, and Fig.9. In addition to the baseline methods mentioned in our paper, we also com- pare two close-source methods for human image generation called HanDiffuser[...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.