Pith. sign in

REVIEW 3 major objections 6 minor 23 references

A Distraction Score for Watermarks

T0 review · 3 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that watermark distraction can be reduced to a single fitted score based on Gaussian-weighted pixel area, and that a hybrid segmenter-classifier makes this score precise enough for real-world image ranking.

desk verdict A practical watermark-detection plus scoring pipeline whose headline ranking accuracy is largely in-sample; the held-out ranking results are far weaker than advertised. read the letter →

arxiv 1908.03651 v1 pith:PIWNHA5W submitted 2019-08-09 cs.CV

Aurelia Guy , Sema Berkiten This is my paper · ORCID
classification cs.CV
keywords watermarkdetectionsemanticsegmentationimageclassificationperceptualscoringrankinghybridmodeldeeplearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to turn the messy visual nuisance of watermarks into a single number that tracks how much people are bothered by them. It argues that two ingredients are enough: reliable detection of watermark pixels across varied real-world images, and a scoring formula that weights a watermark's area by where it sits in the frame. The authors report that in a sparse real-world setting, where under 10% of images carry watermarks, the hybrid model would return correct detections about 79.5% of the time versus about 26.1% for segmentation alone. If the score holds, image-ranking systems could automatically demote heavily watermarked photos.

What carries the argument

The load-bearing object is the hybrid segmentation label $L(i,j) = w_I \cdot S(i,j)$, where $w_I$ is the binary image-level classification (thresholded) and $S(i,j)$ is the pixel-level segmentation. This masking suppresses false-positive pixels from text-like objects such as signs and menus that the segmentation tower alone would flag. The scoring engine is then the Gaussian-weighted pixel sum $G(\sigma,L) = \sum_{i,j} g(i,j) L(i,j)$ followed by the fitted sigmoid $\text{Score}(I) = 1/(1 + e^{-\lambda(G(\sigma,L) - \alpha)})$, with $\sigma$, $\lambda$, and $\alpha$ optimized by mean squared error against human scores. The Gaussian encodes the assumption that center watermarks distract more, and the sigmoid encodes saturating human response to growing watermark area.

What would settle it

Take pairs of images whose predicted scores are equal but whose watermarks differ in opacity or contrast, such as a faint large watermark versus a small solid one; if human raters systematically rank one type above the other, the area-only scoring assumption fails. Concretely, collect a few hundred such pairs and check whether the model's pairwise ranking accuracy drops toward chance.

Watch

Extended reading notes

Core claim

The central discovery is that a watermark's perceptual impact can be captured by a two-stage pipeline: a two-tower CNN (one semantic segmentation tower, one binary classification tower) whose outputs are combined by masking, followed by a fitted nonlinear scoring function. The scoring function computes a Gaussian-weighted sum of detected watermark pixels centered at the image center, then maps it through a sigmoid with parameters fit by least squares to human ratings. The fitted standard deviation and steepness produce a very steep response: once a watermark covers more than a small central region, the score jumps toward 1. Pairwise ranking accuracy reaches 95.36% for images with very distracting watermarks versus ignorable ones, and the hybrid model keeps pixel-level quality while sharply reducing image-level false positives.

Load-bearing premise

The whole score rests on the assumption that human distraction is fully determined by the area of detected watermark pixels weighted by one fixed center-biased Gaussian and passed through one global sigmoid, with opacity, content, contrast, and scene interaction playing no role.

Editorial extensions

If this is right

  • Image-ranking systems can use the score directly to demote images with large or central watermarks; the paper reports 95.36% pairwise ranking accuracy for very distracting versus ignorable watermarks.
  • Because the segmentation tower learns from positive examples and the classifier filters negatives, the hybrid model keeps pixel IOU near the segmentation model while raising estimated image precision from 26.13% to 79.51% in a dataset with 10% watermarked images.
  • The scoring function generalizes to unseen watermark varieties, including rotated text, logos, and both transparent and solid watermarks, as shown on public images outside the training set.
  • Watermarks covering most of the frame receive scores near 1, and small edge watermarks receive scores near 0, matching the intended ranking behavior for demotion.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If opacity were added as a third input to the scoring function, the fitted sigmoid's steepness suggests human ratings saturate quickly; opacity may separate mid-range scores better than area alone.
  • The same Gaussian-weighted-area score could apply to other localized visual obstructions such as subtitles, UI overlays, or timestamp labels, where center-bias and area plausibly dominate distraction.
  • A direct test is to compare human rankings of images with matching Gaussian-weighted area but different watermark opacity or contrast; any systematic disagreement would indicate the scoring function needs an opacity term.
  • The strong center bias implies ranking gains concentrate on center watermarks, so corner watermarks may need separate treatment in quality-sensitive ranking tasks.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes a two-tower CNN system for watermark detection, combining a DeepLab-based semantic segmentation model with a binary image classifier, and a nonlinear scoring function that maps detected watermark masks to a single 0-1 'distraction score' via a Gaussian-weighted pixel sum followed by a sigmoid. The authors report pixel- and image-level metrics for the hybrid detector, estimate image precision under a sparse 10% positive-example assumption, compare against a text detector, and evaluate the scoring function with pairwise ranking accuracies. The central claims are that the hybrid model substantially improves image precision on real-world sparse data and that the fitted distraction score is correlated with human perception and validated in an image ranking setup.

Significance. If the ranking claim were established, the distraction score would be a practically useful tool for downranking watermarked images in large-scale search and mapping applications, and the hybrid detection architecture offers a sensible way to trade recall for precision in sparse positive settings. The scoring function is explicitly fitted to human ratings and the paper reports an out-of-sample MSE on a 2k-image validation split, which is a strength. The hybrid detector's ePrecision improvement from 26.13% to 79.51% under the 10%-positive assumption is also a clear, falsifiable quantitative claim. However, the ranking validation is currently not convincing: the only clearly held-out full-distribution ranking table shows near-chance accuracy for the score-1 vs score-0 pair, and the main ranking table is ambiguous about whether training images are included. The paper is therefore a useful engineering contribution but needs a substantially clearer and more complete evaluation to support its headline claim.

major comments (3)
  1. [Section 5, Table 8] The pairwise ranking accuracies in Table 8 are captioned 'Evaluated on 10k images,' but Section 5 states that the scoring function was fit on an 8k subset and validated on the remaining 2k. As written, Table 8 appears to include the 8k training images, so the 92-97% accuracies may be in-sample and do not establish generalization. This is load-bearing for the abstract's claim that the method is 'validated in an image ranking setup.' Please either clarify explicitly that the 10k evaluation excludes the 8k fitting split, or report the pairwise accuracies on the 2k validation set only. The held-out numbers in Table 10 are much lower, so the distinction is material.
  2. [Section 5, Table 10] On the validation set, the score-1 vs score-0 pairwise accuracy is 57.95%, close to chance. This is the most important distinction for the stated application of demoting images with distracting watermarks, because score 0 means 'no watermark' and score 1 means 'small watermark near edges.' The paper does not discuss this near-chance result or explain why the fitted Gaussian-sigmoid scoring function fails to separate these classes. If Table 8 is in-sample, then the only out-of-sample evidence for the ranking claim is Table 10, which does not support the claim that the score is correlated with human perception in a ranking setup. Please analyze this failure and, if possible, revise the scoring function or the evaluation to address it.
  3. [Section 3.4] The scoring function maps watermarks to a score using only a Gaussian-weighted sum of the segmentation label and a single sigmoid. It does not account for watermark opacity, contrast, content, or interaction with the underlying scene, all of which contribute to perceptual obstructiveness. The paper itself lists opacity as future work, but this limitation directly affects the central claim that the score represents 'visual obstructiveness.' The held-out ranking results suggest that even for the limited size/location features, the score does not reliably separate small edge watermarks from no watermark. Please provide a quantitative analysis of where the scoring function fails on the validation set, and state clearly which perceptual factors are and are not captured by the current score.
minor comments (6)
  1. [Section 5, Table 10 caption] The caption says '1.7k randomly sampled images from the validation set,' but the validation set is described as 2k images. Please clarify why 1.7k images were used and whether the 300 excluded images were excluded before or after the random sample.
  2. [Section 3.3] The text states that for a scoring function based on normalized watermark area, maximizing pairwise ranking accuracy is equivalent to maximizing IOU and minimizing image false positives/negatives. However, the actual scoring function introduced in Section 3.4 is a Gaussian-weighted sum passed through a sigmoid, not a normalized area. The equivalence argument should be updated to match the final scoring function or explicitly framed as a motivating approximation.
  3. [Section 3.4 and Figure 5] The Gaussian function g(i,j) is not fully specified: it is not stated whether the Gaussian is normalized to have a peak value of 1 or unit integral. This matters because the fitted values of sigma and alpha depend on the normalization. Also, the caption says 'isometric' but the correct term is 'isotropic.'
  4. [Section 4.1] The dataset split is described as 80/10/10 of 200k images, but Section 4.3 says the semantic segmentation model is trained on 160k watermarked images. Please clarify whether the 160k training images are all watermarked, and how the image classification model's training set is constructed from the same 200k images.
  5. [Section 4.2 and Tables 4/6] The ePrecision formula says iTP and iFP are computed from a balanced dataset of 50% watermarked images, but the table captions say the metrics are computed on a dataset with 62.5% watermarked images. Please clarify which dataset was used; this affects the reproducibility of the estimated precision numbers.
  6. [Section 4.6 and Tables 11/12] The Flickr results are only qualitative. It would strengthen the cross-domain claim to report at least a small quantitative measure, such as detection or scoring accuracy on a labeled subset of Flickr images.

Circularity Check

1 steps flagged · score 6.0 of 10

Headline ranking validation is computed on the same 10k set that contains the scoring function's 8k training split, so the strong pairwise accuracies are largely in-sample.

  1. fitted input called prediction [Section 3.4 (scoring function fit) and Section 5 / Table 8 (ranking validation)]
    "We split our dataset into a training set of 8k to fit our scoring function and a validation set of 2k to compute the MSE of the function... In Table 8, we show the percentage of image pairs (A,B ) with true ranking A > B, whose predicted score ˆA > ˆB. ... Evaluated on 10k images."

    The scoring function's parameters α, σ, λ are fit by minimizing MSE against the human scores y on an 8k subset of the 10k scoring dataset (Section 3.4). Section 5 then presents pairwise ranking accuracy as the method's validation, but Table 8 is captioned 'Evaluated on 10k images' with no statement that the 8k fitting subset was excluded. Since those 10k images are the same set from which the 8k training split was drawn, the headline 92–97% pairwise accuracies include pairs whose ground-truth scores were used to fit the very function being ranked. The only clearly held-out ranking result, Table 10 on 1.7k validation images, drops to 57.95% for the score-1 vs score-0 pair, showing the out-of-sample signal is much weaker.

full rationale

The paper's watermark detection pipeline has independent content: the hybrid model is compared against a text-detector baseline and evaluated on held-out image sets, and the authors do not rely on self-citations or imported uniqueness theorems. The distraction scoring function itself is honestly fit to human labels, and a 2k held-out MSE (0.041) is reported, so the existence of a genuine out-of-sample component prevents a higher score. The circularity is concentrated in the ranking validation: Section 5 defines the 10k scoring dataset as the source of the 8k fitting split, yet Table 8 — the table used to support the claim that watermarked images are ranked correctly 95% of the time — is captioned 'Evaluated on 10k images' without excluding the training portion. Consequently, the strongest empirical support for the headline claim largely measures the fitted function's agreement with its own training labels. The fully held-out Table 10 is substantially weaker, confirming that the strong ranking claim is not established independently of the fit. No definitional, self-citation, ansatz-smuggling, or renaming circularity is present.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim depends on three fitted parameters (sigma, lambda, alpha) and two hand-chosen thresholds, plus the assumption that Gaussian-weighted watermark area is a sufficient statistic for human distraction. The human labels themselves are treated as ground truth without reliability analysis.

free parameters (5)
  • sigma (Gaussian width) = 0.44
    Controls the spatial weighting of watermark pixels by distance from image center; optimized with lambda and alpha against human scores (Section 5).
  • lambda (sigmoid steepness) = 78
    Controls how sharply the score jumps from 0 to 1 as the weighted watermark area crosses the threshold; optimized on the 8k training set (Section 5).
  • alpha (sigmoid bias) = not reported
    Shifts the sigmoid along the weighted-area axis; jointly optimized with sigma and lambda (Section 3.4).
  • pixel classification threshold = 0.75
    A hand-chosen threshold for classifying a pixel as watermark in the image classifier (Section 3.2).
  • binary mask pixel-count threshold = 'small fixed value based on image size'
    Used to convert the segmentation map into a binary image-level decision; value is not specified (Section 3.2).
assumptions (4)
  • domain assumption A watermark's perceptual distraction is a monotone function of its Gaussian-weighted pixel area, with the Gaussian centered on the image center.
    Stated in Section 3.4 when the scoring function is defined; it is not derived or justified from human perception models.
  • domain assumption The human-provided integer scores from 0 to 3, averaged over three raters, are a reliable ground truth for distraction.
    Used in Section 5 to fit and validate the scoring function; inter-rater reliability is not reported.
  • domain assumption The true fraction of watermarked images in the live database is about 10%.
    Assumed in Section 4.2 based on a small unspecified random sample; used in the ePrecision estimate.
  • standard math The sigmoid and Gaussian functions are standard mathematical tools.
    Used throughout Section 3.4 without proof, which is appropriate.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Distraction Score for Watermarks." pith.science (2026). https://pith.science/paper/PIWNHA5W

@misc{pith2026190803651,
  author       = {Pith},
  title        = {Pith review of: A Distraction Score for Watermarks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PIWNHA5W}},
  note         = {Machine review of arXiv:1908.03651}
}
read the original abstract

In this work we propose a novel technique to quantify how distracting watermarks are on an image. We begin with watermark detection using a two-tower CNN model composed of a binary classification task and a semantic segmentation prediction. With this model, we demonstrate significant improvement in image precision while maintaining per-pixel accuracy, especially for our real-world dataset with sparse positive examples. We fit a nonlinear function to represent detected watermarks by a single score correlated with human perception based on their size, location, and visual obstructiveness. Finally, we validate our method in an image ranking setup, which is the main application of our watermark scoring algorithm.

Figures

Figures reproduced from arXiv: 1908.03651 by the authors.

Figure 1
Figure 1. Diagram of the proposed hybrid model. A watermarked image is input to the image segmentation and classification models based on the DeepLab architecture [7]. binary image mask (w) is a black or white image based on the number of pixels is below or above (t). The final segmentation label (L) is obtained by taking inner product of the binary image mask w and segmentation map S where S is a binary image obtained by thr… view at source ↗
Figure 2
Figure 2. Semantic segmentation predictions for a small watermark in the corner of the image (left). This image demonstrates the hybrid model’s higher watermark pixel recall. Our strict image classification model’s low pixel recall results in the segmentation label (middle) with significantly fewer predicted watermark pixels than our hybrid model’s predicted segmentation label (right). 3.1. Semantic Image Segmentation In orde… view at source ↗
Figure 3
Figure 3. Example of an annotated image used as a segmentation label. Watermarks on the top right and left corners of the image are annotated with two polygons with an arbitrary numbers of vertices. where y is the watermark score obtained from user re￾sponses. To fit our scoring function, we obtained a ground￾truth dataset of 10k segmentation labels with corresponding watermark scores. Scores were obtained by displaying wa￾te… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Google Cloud API’s text detector [2] falsely predicts images to contain watermarks. Our hybrid model predicts these images to not contain watermarks. 4.6. Results on Images From a Different Domain We also run our algorithm on publicly available images containing waterm…
Figure 5
Figure 5. Figure 5: Visualization of the isometric 2D Gaussian (left) and the sig￾moid function (right) used in our watermark scoring function by fitting them on human evaluations of watermarked images. termark pixels in the center of the image a considerably larger weight than edge water…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

23 extracted references · 16 canonical work pages

  1. [1]

    http://www.flickr.com

    Flickr. http://www.flickr.com. 6, 8, 10

    work page

  2. [2]

    https://cloud

    Google cloud api text detector. https://cloud. google.com/vision/docs/detecting-text. 1, 6

    work page

  3. [3]

    M. Belmont. Fully convolutional watermark removal attack. https://github.com/marcbelmont/ cnn-watermark-removal/, 2018. 2

    work page 2018

  4. [4]

    Bertalmio, G

    M. Bertalmio, G. Sapiro, V . Caselles, and C. Ballester. Image inpainting. In Proceedings of the 27th annual conference on Computer graphics and interactive techniques , pages 417– 424, 2000. 2

    work page 2000

  5. [5]

    G. W. Braudaway, K. A. Magerlein, and F. C. Mintzer. Pro- tecting publicly available images with a visible image water- mark. In Electronic Imaging: Science & Technology , page 126133, 1996. 2

    work page 1996

  6. [6]

    L. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille. Semantic image segmentation with deep convolu- tional nets and fully connected crfs. CoRR, abs/1412.7062,

    arXiv

  7. [7]

    L. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. CoRR, abs/1606.00915, 2016. 1, 2, 3, 5

    arXiv 2016

  8. [8]

    L. Chen, G. Papandreou, F. Schroff, and H. Adam. Re- thinking atrous convolution for semantic image segmenta- tion. CoRR, abs/1706.05587, 2017. 1

    arXiv 2017

Show all 23 references
  1. [9]

    L. Chen, Y . Zhu, G. Papandreou, F. Schroff, and H. Adam. Encoder-decoder with atrous separable convolution for se- mantic image segmentation. CoRR, abs/1802.02611, 2018. 1

    2018 arXiv

  2. [10]

    J. Dai, K. He, and J. Sun. Instance-aware semantic segmentation via multi-task network cascades. CoRR, abs/1512.04412, 2015. 2

    2015 arXiv

  3. [11]

    Dashti, R

    M. Dashti, R. Safabakhsh, M. Pourfard, and M. Abdollahi- fard. Video logo removal using iterative subsequent match- ing. In AISP, 2015. 2

    2015

  4. [12]

    Dekel, M

    T. Dekel, M. Rubinstein, C. Liu, and W. T. Freeman. On the effectiveness of visible watermarks. In 2017 IEEE Confer- ence on Computer Vision and Pattern Recognition (CVPR) , pages 6864–6872, July 2017. 1

    2017

  5. [13]

    K. He, G. Gkioxari, P. Doll ´ar, and R. B. Girshick. Mask R-CNN. CoRR, abs/1703.06870, 2017. 2

    2017 arXiv

  6. [14]

    Huang and J.-L

    C.-H. Huang and J.-L. Wu. Attacking visible watermarking schemes. In Multimedia, IEEE Transactions, pages 16–30,

  7. [15]

    M. S. Kankanhalli and K. Ramakrishnan. Adaptive visible watermarking of images. In Multimedia Computing and Sys- tems, 1999. IEEE International Conference on , volume 1, pages 568–573, 1999. 2

    1999

  8. [16]

    Levin, D

    A. Levin, D. Lischinski, and Y . Weiss. Protecting pub- licly available images with a visible image watermark. In A closed-form solution to natural image matting. Pattern Anal- ysis and Machine Intelligence, pages 228–242, 2008. 1

    2008

  9. [17]

    Pei and Y .-C

    S.-C. Pei and Y .-C. Zeng. A novel image recovery algorithm for visible watermarked images. In Information Forensics and Security, IEEE Transactions, pages 543–550, 2016. 2

    2016

  10. [18]

    P. H. O. Pinheiro, R. Collobert, and P. Doll ´ar. Learning to segment object candidates. CoRR, abs/1506.06204, 2015. 2

    2015 arXiv

  11. [19]

    P. H. O. Pinheiro, T. Lin, R. Collobert, and P. Doll´ar. Learn- ing to refine object segments. CoRR, abs/1603.08695, 2016. 2

    2016 arXiv

  12. [20]

    Wang and M

    J. Wang and M. F. Cohen. Image and video matting: a survey. In Now Publishers Inc, 2008. 1

    2008

  13. [21]

    J. Wang, Q. Liu, H. L. L. Duan, and C. Xu. Automatic tv logo detection, tracking and removal in broadcast video. In ICMM, 2007. 2

    2007

  14. [22]

    W.-Q. Yan, J. Wang, and M. S. Kankanhalli. Automatic video logo detection and removal. In Multimedia Systems,

  15. [2005]

    From left to right: original image, detected watermark segmentation, and final watermark score between [0, 1]

    2 Table 12: More watermark detection results on Flickr images [1] that were not included in training or testing. From left to right: original image, detected watermark segmentation, and final watermark score between [0, 1]. Images are ordered by decreasing watermark scores. Ori...

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.