Pith. sign in

REVIEW 4 major objections 7 minor 25 references

Semantic Palette-Guided Color Propagation

T0 review · 4 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that local color edits made with sparse, non-fine-tuned strokes can be propagated to all semantically similar regions by extracting a semantic palette in a 6D color-semantic feature space and solving an edited palette via…

desk verdict The propagation term in Eq. (9) is backwards as written, which undermines the core method, though the semantic palette idea itself has some merit. read the letter →

arxiv 2506.01441 v1 pith:KP63CHN5 submitted 2025-06-02 cs.CV

Zi-Yu Zhang , Bing-Feng Seng , Ya-Feng Du , Kang Li , Zhe-Cheng Wang , Zheng-Jun Du This is my paper · ORCID
classification cs.CV
keywords colorpropagationsemanticpalettecontent-awareeditingpalette-basedrecoloringeditfeaturesenergyoptimizationuserstrokes
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes a two-step color-propagation method built around a semantic palette: first it builds a small set of palette entries in a six-dimensional space that mixes RGB color with three PCA-compressed semantic-feature dimensions, then it solves for an edited palette whose color shifts match sparse user strokes while a propagation term suppresses changes to dissimilar regions. The central claim is that local edits made with only a few rough strokes extend faithfully to every region that shares the same object or material, without the global color drift that palette recoloring often causes and without the dense stroke requirements of classic edit propagation. If the method works, image recoloring becomes a matter of marking one petal, one patch of wall, or one object and having the edit reach all its counterparts while leaving neighboring but different content untouched.

What carries the argument

The central object is the semantic palette, a set of k cluster centers in a 6D feature space where each pixel is represented as $(r, g, b, \kappa_1, \kappa_2, \kappa_3)$, with $\kappa$s being PCA-reduced semantic features from the soft-segmentation network. A modified k-means builds the palette without a preset k: sampling points come from SLIC superpixel centroids, initial centers are chosen greedily by importance weights updated with a repulsion factor $(1 - \exp(-d^2))$, and selection stops when the largest weight falls below threshold $t$. Pixel-to-palette similarity is computed by a radial basis function that factorizes color and semantic distances, and the coefficients in the similarity definition are recovered by solving the linear system used in palette-based recoloring. The edited palette is the minimizer of the energy $E = E_{\text{fidelity}} + E_{\text{propagation}}$, where $E_{\text{fidelity}}$ averages the $L^2$ error between transferred and target colors on stroke pixels and $E_{\text{propagation}}$ penalizes color shifts at 256 sampled pixels weighted by their similarity to the strokes. This palette-plus-energy machinery is what lets local strokes drive a global but content-aware recoloring.

What would settle it

Take a held-out image containing two visually distinct instances of the same semantic category (for example, two flowers under different lighting) next to a similarly colored but semantically different object, place a stroke on one flower, and run the method with the fixed parameters $w_c = 1$, $w_s = 3$, $t = 0.80$; the central claim is falsified if the stroke fails to reach the second flower, or reaches the similar-colored distractor.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that a palette extracted from a fused color-semantic feature space can act as a carrier for content-aware color propagation. Each palette entry is not merely a dominant color but a cluster center in the 6D space, so its similarity weights (computed with a radial basis function over color and semantic components and normalized over the palette) concentrate on pixels belonging to the same semantic regions. Given user strokes, the paper solves an edited palette by minimizing an energy function with two terms: a fidelity term that forces stroked pixels to their target colors and a propagation term that penalizes large color changes at sampled pixels dissimilar to the strokes. The solved palette is then applied through a standard palette color-transfer equation, giving pixel-level edits that extend to semantically similar but spatially distant regions. The paper reports that this requires only sparse, non-fine-tuned strokes and that on five test examples it yields lower MSE and higher PSNR and SSIM than the compared methods.

Load-bearing premise

The method assumes that the PCA-reduced semantic features reliably put a stroke on the intended semantic class and that the parameter values $w_c = 1$, $w_s = 3$, and $t = 0.80$, chosen on the evaluation examples, transfer to unseen images; if those features or settings fail, unrelated regions can join the same palette entry and the edit propagates incorrectly.

Editorial extensions

If this is right

  • A user can recolor one instance of an object and have the edit apply to all other instances of that object, even ones with different lighting or texture.
  • Palette editing and pixel-level edit propagation merge into one interaction: the palette is an intermediate, not the final editing surface, so users can stroke the image directly.
  • The method's reliance on continuous semantic features rather than a hard segmentation result means propagation does not inherit segmentation-boundary errors.
  • Because the edited palette is solved per image from sparse strokes, the same interaction can edit many semantically similar regions in a single stroke without global color shift.
  • On the paper's five test examples, the method reports lower MSE and higher PSNR and SSIM than the five compared methods.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The authors leave open whether the solved palette transfers across images: if the same semantic palette were applied to a second image of the same scene or object class, propagation might work with no new strokes, but the paper does not test this.
  • The 6D space separates color from semantics, so the same energy equation could be repurposed to propagate other attributes such as lightness, texture, or material by swapping the color component; this is an extension, not a claim of the paper.
  • The uniform 256-point sampling for the propagation term is a hidden capacity limit; on very large or detail-rich images, propagation quality may depend on whether those samples cover the semantically relevant regions, a sensitivity the paper does not report.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper proposes a semantic palette-guided color propagation method. In a first step, per-pixel semantic features from a neural network (Aksoy et al.) are reduced via PCA to three dimensions and concatenated with RGB color, yielding a 6D feature space. A modified k-means algorithm with SLIC-based sampling and an adaptive initial-center selection (Eqs. 1-2) extracts a semantic palette. In the second step, an edited palette is obtained by minimizing an energy function (Eq. 7) that combines a fidelity term (Eq. 8) and a propagation term (Eq. 9), and the edited palette is applied to all pixels via Eq. 6. Experiments on five images compare the method with five prior approaches, reporting lower MSE and higher PSNR/SSIM, along with qualitative comparisons and an ablation of the propagation term.

Significance. If the method works as described, the idea of jointly using color and semantic features in a palette-based editing framework is a reasonable and potentially useful contribution to interactive image editing. The paper clearly specifies the feature construction, the clustering procedure, and the energy function, which makes the approach reproducible in principle. The qualitative examples suggest that the method can achieve content-aware propagation when the energy is implemented consistently with the stated intent. However, the paper's experimental support is thin: only five images, no error bars or statistical tests, and the hyperparameters w_c, w_s, and t are tuned on the same examples used for evaluation. The most serious issue is that Eq. (9), as printed, appears to contradict the stated goal of propagation, which undermines confidence that the reported results follow from the described method.

major comments (4)
  1. [Section II-B, Eq. (9)] The propagation term as written is internally inconsistent with its stated purpose. In Eq. (9), E_propagation is a weighted sum of squared color deviations ||I_j.C - I'_j.C||^2 with weights α_j, where α_j is defined as "the similarity of I_j to the most similar pixel in user strokes." Thus α_j is largest for pixels most similar to the user strokes, and the term assigns the largest penalty to exactly those pixels whose colors should be propagated. The sentence following Eq. (9) says the term should "penalize those pixels [that] have larger color variations but not similar with user strokes," which would require weighting by a dissimilarity measure such as (1 - α_j), not by α_j. As printed, the objective suppresses color change in the semantic region where propagation is supposed to occur, and the solution of Eq. (7) cannot faithfully deliver the user's target color to non-stroke pixels in the target semantic region. Please correct the formula or carefully redefine α_j; if the implementation actually uses a different weight, provide the correct derivation and, ideally, code to verify the energy that is minimized.
  2. [Section III-A and III-C] The hyperparameters w_c, w_s, and t are evaluated and selected on the same five images that are later used for the quantitative comparison in Table I. For example, Section III-A states that "we find that setting w_c = 1, ws = 3 and t = 0.80 can generate palettes with an appropriate number of entries... So we use these values to extract semantic palettes for all examples," with the evaluation shown in Fig. 3 on the same examples. This is tuning on the evaluation set, which inflates the apparent performance and means the reported MSE/PSNR/SSIM values do not estimate performance on unseen images. Please provide a separate test set, cross-validation, or at least a sensitivity analysis that shows the results are stable over a range of parameters.
  3. [Section III-C, Fig. 5 and Table I] The ground truth column "GT" in Fig. 5 is not defined anywhere in the manuscript. It is unclear whether the ground-truth edited images were produced by a human expert, by another algorithm, or by a synthetic procedure (e.g., applying a known color change to a segmentation mask). Since Table I reports quantitative errors against GT, the meaning of those numbers depends entirely on how GT was created. Please specify the GT generation protocol, including the number of annotators (if any) and the exact editing instructions.
  4. [Section III-C] The quantitative evaluation is based on only five images (Man, Fox, Sky, Milk, Bear) with a single metric value per method per image and no error bars, confidence intervals, or significance tests. With n = 5 and hyperparameters tuned on the same images, the claim that the method is consistently better than the baselines is not statistically supported. Please expand the evaluation set and report variance or perform a paired test.
minor comments (7)
  1. [Section II-B, Eq. (9)] The normalization in Eq. (9) has an index inconsistency: the denominator is written as 1/∑_{i=1}^{|G|} α_j, mixing indices i and j. It should be 1/∑_{j=1}^{|G|} α_j.
  2. [Section II-A, Eq. (3)] The description of σ_c and σ_s is ambiguous: "both determined by averaging the color and semantic features across all palette entries" could mean the average of per-palette-entry standard deviations, the standard deviation of palette entries, or something else. Please clarify the exact formula.
  3. [Section II-A, Eq. (4)] The coefficients λ_{i,j} are said to be obtained by solving a linear system from [12], but the system is not stated and the role of the RBF expansion in Eq. (4) is not explained. A brief derivation or citation to the specific equations in [12] would improve reproducibility.
  4. [Section II-B, Eq. (7)] The energy is described as a "weighted sum" of a fidelity term and a propagation term, but no explicit weights appear in Eq. (7). If the two terms are meant to be equally weighted, please state that explicitly; otherwise, the balancing weights should be part of the equation.
  5. [Section II-B] The statement "we uniformly sample 256 pixels" should clarify whether these 256 pixels are sampled from the whole image, from all non-stroke pixels, or from the set of superpixel centroids. This affects the definition of the propagation term.
  6. [Section III-A] The text says that combining a smaller w_c and a larger w_s usually yields more palette entries, but the exact relationship and the impact on the final edit are only shown qualitatively in Fig. 3. A quantitative sensitivity plot would be more informative.
  7. [Abstract and Conclusion] The abstract and conclusion claim "Extensive experiments" and "efficient and accurate pixel-level color editing," but the experimental section uses only five images. Please either expand the experiments or soften the wording to match the actual scope.

Circularity Check

0 steps flagged · score 1.0 of 10

No definitional circularity; the derivation is self-contained. Minor concerns about hyperparameter tuning on evaluation examples and an inconsistency in Eq. (9) are correctness/fairness issues, not circularity.

full rationale

The paper's claimed derivation chain is self-contained. The semantic palette is extracted from external per-pixel semantic features (Aksoy et al.) combined with color, and does not presuppose the propagation result. The edited palette is solved by minimizing the energy function in Eq. (7), a fresh optimization whose output is not equivalent to any fitted input. No parameter is renamed as a prediction: the weights wc, ws and threshold t are hyperparameters chosen by visual inspection (Sec. III-A), and the reported quantitative results (Table I) are produced by the full pipeline on the same examples. This is a methodological data-leakage concern rather than a definitional circularity, because the algorithm's output is not statistically forced by the parameter choice; the success is selected, not derived. The self-citations (refs. [9], [20], [23]) are prior works used for comparison or background, and none is load-bearing for the central claim. A separate observation, not a circularity, is that Eq. (9) as printed weights the propagation penalty by alpha_j, the similarity to the stroke, which is opposite of its stated intent; this is a correctness/sign issue, not an equivalence between input and output. Therefore no significant circularity is found.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

All numeric free parameters are hand-tuned on the examples used for evaluation; the method depends on external semantic features and a similarity model borrowed from earlier palette work, together with a heuristic energy function and the COBYLA solver.

free parameters (4)
  • wc = 1
    Color distance weight in Eq.2; hand-tuned on Fig.3 examples.
  • ws = 3
    Semantic distance weight in Eq.2; hand-tuned on Fig.3 examples.
  • t = 0.80
    Threshold for stopping initial center selection; chosen on Fig.3 examples.
  • energy weights (fidelity vs. propagation) = 1 and 1
    No balancing weight in Eq.7; both terms have equal influence without justification.
assumptions (4)
  • domain assumption Aksoy et al. semantic features, reduced to 3D by PCA, reliably encode object/material identity for propagation.
    The entire palette extraction depends on these features; no validation across varied image types.
  • domain assumption The RBF similarity model of Eq.3, with lambda solved via a linear system from Chang et al., yields meaningful palette weights.
    Borrowed from prior palette recoloring; assumed to transfer to semantic space.
  • domain assumption COBYLA finds an edited palette that adequately minimizes the energy in Eq.7.
    No convergence analysis is provided; relies on solver behavior.
  • domain assumption Uniformly sampling 256 pixels is sufficient to measure global color variation in the propagation term.
    Stated as an acceleration choice without analysis.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Semantic Palette-Guided Color Propagation." pith.science (2026). https://pith.science/paper/KP63CHN5

@misc{pith2026250601441,
  author       = {Pith},
  title        = {Pith review of: Semantic Palette-Guided Color Propagation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KP63CHN5}},
  note         = {Machine review of arXiv:2506.01441}
}
read the original abstract

Color propagation aims to extend local color edits to similar regions across the input image. Conventional approaches often rely on low-level visual cues such as color, texture, or lightness to measure pixel similarity, making it difficult to achieve content-aware color propagation. While some recent approaches attempt to introduce semantic information into color editing, but often lead to unnatural, global color change in color adjustments. To overcome these limitations, we present a semantic palette-guided approach for color propagation. We first extract a semantic palette from an input image. Then, we solve an edited palette by minimizing a well-designed energy function based on user edits. Finally, local edits are accurately propagated to regions that share similar semantics via the solved palette. Our approach enables efficient yet accurate pixel-level color editing and ensures that local color changes are propagated in a content-aware manner. Extensive experiments demonstrated the effectiveness of our method.

Figures

Figures reproduced from arXiv: 2506.01441 by the authors.

Figure 1
Figure 1. Pipeline of our method. palette-based and tone curve editing. To enhance local control for palette-based image recoloring, Chao et al. [22] proposed “LoCoPalettes” that incorporates palette-based image editing with image-space constraints and semantic hierarchies. Du et al. [23] presented a palette-based approach for content-aware image recoloring, but this method neither generates palettes adaptively nor supports p… view at source ↗
Figure 2
Figure 2. Visualization of similarity weights. For this example, we provide the input image, similarity weights of all pixels to each semantic palette entry, and [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Parameter evaluation. The parameters wc and ws are evaluated in the first row, and t is evaluated in the second row. For each example, we provide the input image and user strokes, the generated palettes with different parameter settings, region assigned to each palette entry, and the edited images. III. EXPERIMENTS We performed all experiments on a laptop computer with AMD Ryzen 7 5800H 3.20 GHZ CPU and 16 GB RAM. W… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Ablation study. For each example, we provide the input image, user strokes, the generated palettes, the solved edited palettes, and the edited image. [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Qualitative comparison of our method with other existing methods. [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

25 extracted references · 25 canonical work pages

  1. [1]

    Colorization using optimization,

    Anat Levin, Dani Lischinski, and Yair Weiss, “Colorization using optimization,” in ACM SIGGRAPH 2004 Papers , pp. 689–694. 2004

    work page 2004

  2. [2]

    Interactive local adjustment of tonal values,

    Dani Lischinski, Zeev Farbman, Matt Uyttendaele, and Richard Szeliski, “Interactive local adjustment of tonal values,” ACM Transactions on Graphics (TOG), vol. 25, no. 3, pp. 646–653, 2006

    work page 2006

  3. [3]

    Appwand: Editing measured materials using appearance-driven optimization,

    Fabio Pellacini and Jason Lawrence, “Appwand: Editing measured materials using appearance-driven optimization,” in ACM SIGGRAPH 2007, 2007, p. 54–es

    work page 2007

  4. [4]

    Appprop: all-pairs appearance-space edit propagation,

    Xiaobo An and Fabio Pellacini, “Appprop: all-pairs appearance-space edit propagation,” in ACM SIGGRAPH 2008 papers , pp. 1–9. 2008

    work page 2008

  5. [5]

    Efficient affinity-based edit propagation using kd tree,

    Kun Xu, Yong Li, Tao Ju, Shi-Min Hu, and Tian-Qiang Liu, “Efficient affinity-based edit propagation using kd tree,” ACM Transactions on Graphics (ToG), vol. 28, no. 5, pp. 1–6, 2009

    work page 2009

  6. [6]

    Instant propagation of sparse edits on images and videos,

    Yong Li, Tao Ju, and Shi-Min Hu, “Instant propagation of sparse edits on images and videos,” in Computer Graphics F orum . Wiley Online Library, 2010, vol. 29, pp. 2049–2054

    work page 2010

  7. [7]

    Real time edit propagation by efficient sampling,

    Xiaohui Bie, Haoda Huang, and Wencheng Wang, “Real time edit propagation by efficient sampling,” in Computer Graphics F orum. Wiley Online Library, 2011, vol. 30, pp. 2041–2048

    work page 2011

  8. [8]

    Instant edit propagation on images based on bilateral grid.,

    Feng Li, Chaofeng Ou, Yan Gui, and Lingyun Xiang, “Instant edit propagation on images based on bilateral grid.,” Computers, Materials & Continua , vol. 61, no. 2, 2019

    work page 2019

Show all 25 references
  1. [9]

    Edit propagation via color palettes,

    Zi Xun Xia, Jian Yu Hao, Kang Li, Ao Xiang Tian, and Zheng Jun Du, “Edit propagation via color palettes,” Computers & graphics , , no. Apr., pp. 119, 2024

    2024

  2. [10]

    Deep- prop: Extracting deep features from a single image for edit propagation,

    Yuki Endo, Satoshi Iizuka, Yoshihiro Kanamori, and Jun Mitani, “Deep- prop: Extracting deep features from a single image for edit propagation,” in Computer Graphics F orum. Wiley Online Library, 2016, vol. 35, pp. 189–201. Gui et al. [11]Xia et al. [9] Chao et al. [22]Wang et ...

    2016

  3. [11]

    Joint learning of visual and spatial features for edit propagation from a single image,

    Yan Gui and Guang Zeng, “Joint learning of visual and spatial features for edit propagation from a single image,” The Visual Computer , vol. 36, no. 3, pp. 469–482, 2020

    2020

  4. [12]

    Palette-based photo recoloring.,

    Huiwen Chang, Ohad Fried, Yiming Liu, Stephen DiVerdi, and Adam Finkelstein, “Palette-based photo recoloring.,” ACM Trans. Graph., vol. 34, no. 4, pp. 139–1, 2015

    2015

  5. [13]

    Radial basis functions,

    Martin Dietrich Buhmann, “Radial basis functions,” Acta numerica, vol. 9, pp. 1–38, 2000

    2000

  6. [14]

    Palette- based image recoloring using color decomposition optimization,

    Qing Zhang, Chunxia Xiao, Hanqiu Sun, and Feng Tang, “Palette- based image recoloring using color decomposition optimization,” IEEE Transactions on Image Processing , vol. 26, no. 4, pp. 1952–1964, 2017

    1952

  7. [15]

    A blind color separation model for faithful palette-based image recoloring,

    Qing Zhang, Yongwei Nie, Lei Zhu, Chunxia Xiao, and Wei-Shi Zheng, “A blind color separation model for faithful palette-based image recoloring,” IEEE Transactions on Multimedia , vol. 24, pp. 1545–1557, 2021

    2021

  8. [16]

    Decomposing images into layers via rgb-space geometry,

    Jianchao Tan, Jyh-Ming Lien, and Yotam Gingold, “Decomposing images into layers via rgb-space geometry,” ACM Transactions on Graphics (TOG), vol. 36, no. 1, pp. 1–14, 2016

    2016

  9. [17]

    Efficient palette- based decomposition and recoloring of images via rgbxy-space geome- try,

    Jianchao Tan, Jose Echevarria, and Yotam Gingold, “Efficient palette- based decomposition and recoloring of images via rgbxy-space geome- try,” ACM Trans. Graph. , vol. 37, no. 6, pp. 1–10, 2018

    2018

  10. [18]

    An improved geometric approach for palette-based image decomposition and recoloring,

    Yili Wang, Yifan Liu, and Kun Xu, “An improved geometric approach for palette-based image decomposition and recoloring,” in Computer Graphics F orum. Wiley Online Library, 2019, vol. 38, pp. 11–22

    2019

  11. [19]

    Building coarse to fine convex hulls with auxiliary vertices for palette-based image recoloring,

    Qiwei Sun, Yongwei Nie, Qing Zhang, and Guiqing Li, “Building coarse to fine convex hulls with auxiliary vertices for palette-based image recoloring,” IEEE Transactions on Visualization and Computer Graphics, 2023

    2023

  12. [20]

    Video recoloring via spatial-temporal geometric palettes,

    Zheng-Jun Du, Kai-Xiang Lei, Kun Xu, Jianchao Tan, and Yotam Gingold, “Video recoloring via spatial-temporal geometric palettes,” ACM Transactions on Graphics (TOG) , vol. 40, no. 4, pp. 1–16, 2021

    2021

  13. [21]

    ColorfulCurves: Palette-aware lightness control and color editing via sparse optimization,

    Cheng-Kang Ted Chao, Jason Klein, Jianchao Tan, Jose Echevarria, and Yotam Gingold, “ColorfulCurves: Palette-aware lightness control and color editing via sparse optimization,” ACM Transactions on Graphics (TOG), vol. 42, no. 4, July 2023

    2023

  14. [22]

    LoCoPalettes: Local control for palette-based image editing,

    Cheng-Kang Ted Chao, Jason Klein, Jianchao Tan, Jose Echevarria, and Yotam Gingold, “LoCoPalettes: Local control for palette-based image editing,” Computer Graphics F orum (CGF) , vol. 42, no. 4, June 2023, Special issue for Eurographics Symposium on Rendering (EGSR)

    2023

  15. [23]

    Palette-based content-aware image recoloring,

    Zheng-Jun Du, Jia-Wei Zhou, Zi-Xun Xia, Bing-Feng Seng, and Kun Xu, “Palette-based content-aware image recoloring,” in International Conference on Computational Visual Media . Springer, 2024, pp. 240– 258

    2024

  16. [24]

    Unmixing-based soft color segmentation for image manipulation,

    Ya ˘giz Aksoy, Tunc ¸ Ozan Aydin, Aljo ˇsa Smoli ´c, and Marc Pollefeys, “Unmixing-based soft color segmentation for image manipulation,” ACM Transactions on Graphics (TOG) , vol. 36, no. 2, pp. 1–19, 2017

    2017

  17. [25]

    Slic superpixels compared to state- of-the-art superpixel methods,

    Radhakrishna Achanta, Appu Shaji, Kevin Smith, Aurelien Lucchi, Pascal Fua, and Sabine S ¨usstrunk, “Slic superpixels compared to state- of-the-art superpixel methods,” IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 34, no. 11, pp. 2274–2282, 2012

    2012

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.