Pith. sign in

REVIEW 3 major objections 5 minor 26 references

Visual Coin-Tracking: Tracking of Planar Double-Sided Objects

T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Coin-tracking is a distinct, hard visual tracking problem, and a segmentation-plus-pose baseline beats state-of-the-art long-term trackers on a new 17-video dataset.

desk verdict A genuinely new benchmark problem with a solid baseline; the performance claims are plausible but the paper needs a mask-quality study and repeated-run statistics before you should fully trust the numbers. read the letter →

arxiv 1908.02664 v1 pith:6X3JFBIY submitted 2019-08-07 cs.CV

Jonáš Šerých , Jiří Matas This is my paper · ORCID
classification cs.CV
keywords cointrackingplanarobjectvideosegmentationlong-termvisualhomographyestimationbenchmarkonlineadaptationmasks
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces coin-tracking: the visual tracking of rigid, approximately planar objects—coins, cards, badges—when both sides are visible and the object rotates fast enough to blur and change aspect ratio. It argues this is a distinct problem, separate from standard tracking benchmarks, and supports that with the CTR dataset of 17 long sequences annotated with segmentation masks on every fifth frame. The paper's main empirical claim is that a baseline tracker combining CNN segmentation with explicit homography pose modeling, called CTR-Base, reaches a mean IoU of 0.70 on CTR, ahead of the state-of-the-art long-term trackers MBMD (0.47) and DaSiam LT (0.44), and even ahead of a bounding-box oracle (0.63). The result matters because it shows that bounding-box representations are insufficient for such objects and that explicit pose modeling plus conservative online adaptation is a workable recipe on long sequences.

What carries the argument

The core machinery is the pair (CTR dataset, CTR-Base tracker). Coin-tracking state is represented by a segmentation mask plus a homography—a projective mapping from a canonical side image of the object to the current frame—because symmetric coins often make the homography non-unique on its own. The tracker's objective is a product of four sub-scores: the fraction of the segmentation inside the hypothesized contour, the fraction of the contour covered by the segmentation, the overlap of the current and previous visibility masks (occlusion consistency), and a zero-normalized cross-correlation appearance match to the canonical template. This homography optimization, driven by simulated annealing with optical-flow initialization, both stabilizes online adaptation and enables a lost-state re-detection that decides when the tracker may safely add new object and background examples to the k-NN embedding index.

What would settle it

Re-annotate the CTR masks on the hardest frames (especially pingpong1 and the iccv handheld sequences) with multiple independent annotators and recompute the mean IoU; if the corrected masks shrink the gap between CTR-Base and the bounding-box oracle, or change the ranking against MBMD and DaSiam LT, the central performance claim fails.

Watch

Extended reading notes

Core claim

The central claim is that coin-tracking is a distinct tracking problem with its own failure modes—fast out-of-plane rotation, rapid illumination change, rotational blur, and strong aspect-ratio changes—and that existing datasets do not capture these modes. On the paper's own terms, the CTR dataset differs from VOT2016 and OTB in object textureness and in the magnitude and speed of aspect-ratio change, and the proposed CTR-Base method, which uses the fast-vos segmentation k-NN classifier and then optimizes a homography against a four-term product score, outperforms the VOT 2018 long-term tracking winners MBMD and DaSiam LT by mean IoU. The paper further reports a mean IoU of 0.88 on frames the tracker itself judges as confident, a generalization result of 0.81 mean IoU on the POT-210 planar-object dataset, and an explicit failure-detection mechanism that prevents online adaptation from drifting on long sequences.

Load-bearing premise

All dataset statistics and IoU scores rest on hand-annotated segmentation masks for the 17 sequences, with no inter-annotator agreement or independent verification reported for frames with fast motion blur, strong perspective change, or partial occlusion.

Editorial extensions

If this is right

  • Trackers should be evaluated on coin-tracking before claiming robustness to planar-object motion, because standard benchmarks miss the fastest out-of-plane rotations.
  • Bounding-box representations are a bottleneck even with perfect localization: the bounding-box oracle scores 0.63 mean IoU while the segmentation-based method reaches 0.70.
  • The confident-frame IoU of 0.88 implies that the remaining gap is failure detection and re-detection, not the segmentation quality itself.
  • The POT-210 generalization result suggests the approach transfers to planar objects seen from one side, so it may serve augmented-reality and robotic manipulation scenarios.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's mask-only state sidesteps the symmetric-ambiguity problem, but a natural extension it leaves implicit is predicting full 6D pose on sequences with known camera calibration, which robotic grasping would require.
  • Because the paper finds CTR objects far less textured than standard tracking sequences, keypoint-based homography estimators will likely fail on this dataset; a head-to-head stress test would make that boundary explicit.
  • The dataset could also be used for training rather than only evaluation, since its long sequences and rare object classes may improve robustness to rotational motion blur.
  • Re-annotating the masks on the hardest frames (e.g., pingpong1, where CTR-Base scores 0.33) with multiple annotators would test whether the reported performance gaps survive annotation noise.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces 'coin-tracking' as a new visual tracking task: tracking rigid, approximately planar double-sided objects whose two sides are alternately visible. It contributes the CTR dataset of 17 sequences with segmentation masks on every fifth frame, and reports dataset statistics (textureness, aspect-ratio change) intended to show that CTR differs from VOT/OTB. The proposed baseline CTR-Base combines a fast-vos-style k-NN segmentation network with explicit homography pose estimation via simulated annealing and conservative online adaptation. Experiments report IoU 0.70 on CTR, outperforming MBMD (0.47), DaSiam LT (0.44), and a bounding-box oracle (0.63), plus mIoU 0.81 on POT-210. The paper concludes that coin-tracking is an open and challenging problem.

Significance. If the results hold, the paper provides a useful new benchmark and a sensible baseline for a practically common but previously unbenchmarked tracking scenario. The dataset fills a gap: existing planar-object tracking benchmarks mostly show one side and are dominated by camera motion, while CTR has fast object-driven out-of-plane rotation, blur, and illumination change. The baseline is a reasonable combination of published components (fast-vos embeddings, DeepLabv3+, simulated annealing), and the positive result on the external POT-210 benchmark is a genuine generalization check. The conservative online adaptation principle—only adapt when a homography-based pose model agrees with segmentation—is a well-motivated idea that could inform video object segmentation on long sequences. The paper provides the dataset URL and detailed training hyperparameters, though no code for the baseline is released.

major comments (3)
  1. [Section 2 and Section 3.1, Eq. (5); Table 1] The central comparative claim depends entirely on the accuracy of the hand-annotated CTR masks, yet the paper reports no annotation protocol, no inter-annotator agreement, and no quality check. This is load-bearing because CTR-Base's objective in Eq. (5) uses the ground-truth mask as the template: sobj and scover reward agreement with that mask, and sappearance compares against the ground-truth frame. If annotations are systematically biased, for example tighter than the true boundary on motion-blurred or strongly rotated frames, CTR-Base is rewarded for reproducing the bias while bounding-box baselines are penalized for including the true object extent. The same masks drive the dataset-difference statistics in Section 2.1. A concrete check would be to measure inter-annotator IoU on a subset of challenging frames, for example from sequences 'iccv handheld' and 'pingpong1', where CTR-Base reports 0.50 and 0.33.
  2. [Section 4.1, Table 1; Section 3.1] CTR-Base is a stochastic method (random homography perturbations, 50 random flow-based initializations, and simulated annealing in Section 3.1), but the reported IoU comes from a single run. Without multiple seeds or error bars, the 0.70 versus 0.63 gap over the bounding-box oracle may not be statistically significant. In addition, the protocol for the long-term trackers is underspecified: MBMD and DaSiam LT output a confidence and may declare frames 'not confident,' but the paper does not state how such outputs are converted to a mask for IoU. If absent detections are scored as empty masks, the comparison is dominated by this mapping; if they are excluded, the evaluated frame set becomes tracker-dependent. Please specify the mapping and report variance across at least several runs.
  3. [Section 4.2, Table 2] The 'confident frames' IoU of 0.88 is computed only on the 47% of frames where the tracker itself decides it is in the tracking state, and that state is determined by the same score function used for pose estimation. The statement that the tracker 'is confident on the frames, where it performs well' is therefore partly self-referential. This experiment is useful as an indication of failure detection, but it does not support the overall superiority claim. The paper should either avoid framing this as a performance measure or compare failure-detection quality against an oracle, for example by reporting precision and recall of the confident state relative to frames where IoU exceeds a threshold.
minor comments (5)
  1. [Section 2, first paragraph after the problem definition] There is a typo: 'The are multiple levels of tracking' should be 'There are multiple levels of tracking.'
  2. [Section 2.2 and Table 1] The paper should state explicitly that IoU is computed only on annotated frames (every fifth frame). The text says frames with empty ground-truth segmentation do not contribute, but the caption 'Mean over all frames' is ambiguous.
  3. [Section 3.1, Eq. (3)] The ZNCC formula appears typeset incorrectly in the provided text; the denominator is missing the usual parentheses. Please verify that the equation matches the intended zero-normalized cross-correlation coefficient.
  4. [Section 4.1, POT-210 evaluation] For the POT-210 experiment, state whether frames with occlusion or out-of-view are included in the mIoU computation or excluded, since the ground-truth corners are converted to segmentation 'not modeling occlusions.'
  5. [Table 2] Format the 'frames in tracking state' column consistently; the coin3 row shows '02' without a percent sign, which is confusing.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper introduces a new dataset and a baseline tracker, evaluates against external and baseline methods, and does not reduce any prediction to its inputs.

full rationale

The paper's central claims are (i) that coin-tracking is a new problem with a new benchmark and (ii) that CTR-Base outperforms state-of-the-art long-term trackers on that benchmark. Neither claim is derived from its own inputs. The CTR dataset is a new annotated resource; its statistics (textureness, aspect ratio changes) are empirical measurements, not renamings of a known result. CTR-Base is built from published components (fast-vos, DeepLabv3+, MobileNet, ContinualFlow), and its homography objective in Eq. 5 uses the ground-truth frame as initialization and template, which is standard tracking practice rather than a fitted parameter. The per-frame evaluation masks are not used to fit any model parameter, and the tracker is additionally evaluated on the external POT-210 benchmark, where it reports mIoU 0.81. The absence of an inter-annotator agreement study is a dataset-quality limitation, not a circularity of the derivation. No equation reduces to another by construction, no fitted parameter is renamed as a prediction, and no load-bearing claim rests on a self-citation. The result is therefore self-contained for circularity purposes.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The central claims rest on the correctness of the hand-annotated CTR masks, the assumption that coin-like objects are rigid and planar with always-visible boundaries, and the standard homography and ZNCC models. The method introduces hand-tuned optimization parameters but no physics-like free constants.

free parameters (1)
  • Simulated annealing and lost-state hyperparameters = not reported
    CTR-Base's reported IoU depends on the initial temperature, perturbation sigma schedule, 350 iterations, and the threshold that triggers the lost state. These are chosen by hand and not released with exact values or sensitivity analysis.
assumptions (4)
  • domain assumption Tracked objects are rigid and approximately planar, so their state is a visible side plus a homography (or segmentation mask when symmetric).
    Section 2 defines coin-tracking this way; if a tracked object deforms or has significant thickness, the homography model breaks.
  • domain assumption The boundary between the two sides is always visible except for occlusion or out-of-view, so the object does not self-occlude.
    Section 2 states 'the boundary between their two sides is always visible, except for occlusions...'. This underpins the visibility mask and occlusion score soccl.
  • domain assumption Manual segmentation masks for the 17 CTR sequences are accurate enough for evaluation.
    All dataset statistics and IoU results in Section 4 use these masks; no inter-annotator agreement or validation is reported.
  • standard math Standard homography and ZNCC models map the canonical frame and compute appearance similarity.
    Section 3.1 uses homography mapping and Eq. 3; these are standard tools in the tracking literature.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Visual Coin-Tracking: Tracking of Planar Double-Sided Objects." pith.science (2026). https://pith.science/paper/6X3JFBIY

@misc{pith2026190802664,
  author       = {Pith},
  title        = {Pith review of: Visual Coin-Tracking: Tracking of Planar Double-Sided Objects},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6X3JFBIY}},
  note         = {Machine review of arXiv:1908.02664}
}
read the original abstract

We introduce a new video analysis problem -- tracking of rigid planar objects in sequences where both their sides are visible. Such coin-like objects often rotate fast with respect to an arbitrary axis producing unique challenges, such as fast incident light and aspect ratio change and rotational motion blur. Despite being common, neither tracking sequences containing coin-like objects nor suitable algorithm have been published. As a second contribution, we present a novel coin-tracking benchmark containing 17 video sequences annotated with object segmentation masks. Experiments show that the sequences differ significantly from the ones encountered in standard tracking datasets. We propose a baseline coin-tracking method based on convolutional neural network segmentation and explicit pose modeling. Its performance confirms that coin-tracking is an open and challenging problem.

Figures

Figures reproduced from arXiv: 1908.02664 by the authors.

Figure 1
Figure 1. Examples from the coin-tracking dataset (frame number in the top-right [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Comparison of object “textureness” in the proposed CTR and VOT 2016 [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Histogram of aspect ratio changes The maximum of the two ratios is chosen because only the magnitude of the aspect ratio change matters. Aspect ratio change relative to the first frame. We have computed aspect ra￾tio changes ∆r(R1, Rt) between the bounding rectangle on the first frame and each of the other annotated frames in the sequence. We then represent each tested dataset (VOT2016, OTB, CTR) by a histogram of t… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Homography score computation. Left: the segmentation mask split into [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

26 extracted references · 18 canonical work pages

  1. [1]

    arXiv preprint arXiv:1811.10302 (2018)

    Bai, S., He, Z., Xu, T.B., Zhu, Z., Dong, Y., Bai, H.: Multi-hierarchical independent correlation filters for visual tracking. arXiv preprint arXiv:1811.10302 (2018)

    arXiv 2018

  2. [2]

    In: Proceedings of the European Conference on Computer Vision (ECCV)

    Bhat, G., Johnander, J., Danelljan, M., Shahbaz Khan, F., Felsberg, M.: Unveil- ing the power of deep tracking. In: Proceedings of the European Conference on Computer Vision (ECCV). pp. 483–498 (2018) Visual Coin-Tracking: Tracking of Planar Double-Sided Objects 13

    work page 2018

  3. [3]

    In: Proceedings of the IEEE conference on computer vision and pattern recognition

    Caelles, S., Maninis, K.K., Pont-Tuset, J., Leal-Taix´ e, L., Cremers, D., Van Gool, L.: One-shot video object segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 221–230 (2017)

    work page 2017

  4. [4]

    In: Proceedings of the European conference on computer vision (ECCV)

    Chen, L.C., Zhu, Y., Papandreou, G., Schroff, F., Adam, H.: Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Proceedings of the European conference on computer vision (ECCV). pp. 801–818 (2018)

    work page 2018

  5. [5]

    Journal of Electronic Imaging 28(1), 1–16 (2019)

    Chen, L., Ling, H., Shen, Y., Zhou, F., Wang, P., Tian, X., Chen, Y.: Robust visual tracking for planar objects using gradient orientation pyramid. Journal of Electronic Imaging 28(1), 1–16 (2019)

    work page 2019

  6. [6]

    In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition

    Chen, Y., Pont-Tuset, J., Montes, A., Van Gool, L.: Blazingly fast video object seg- mentation with pixel-wise metric learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 1189–1198 (2018)

    work page 2018

  7. [7]

    In: 2009 IEEE Conference on Computer Vision and Pattern Recognition

    Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition. pp. 248–255 (June 2009)

    work page 2009

  8. [8]

    http://www.pascal-network.org/challenges/VOC/voc2012/workshop/index.html

    Everingham, M., Van Gool, L., Williams, C.K.I., Winn, J., Zisserman, A.: The PASCAL Visual Object Classes Challenge 2012 (VOC2012) Results. http://www.pascal-network.org/challenges/VOC/voc2012/workshop/index.html

    2012

Show all 26 references
  1. [9]

    In: International Conference on Computer Vision (ICCV) (2011)

    Hariharan, B., Arbelaez, P., Bourdev, L., Maji, S., Malik, J.: Semantic contours from inverse detectors. In: International Conference on Computer Vision (ICCV) (2011)

    2011

  2. [10]

    arXiv preprint arXiv:1704.04861 (2017)

    Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., An- dreetto, M., Adam, H.: Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861 (2017)

    2017 arXiv

  3. [11]

    IEEE Transactions on Big Data (2019)

    Johnson, J., Douze, M., J´ egou, H.: Billion-scale similarity search with gpus. IEEE Transactions on Big Data (2019)

    2019

  4. [12]

    In: The DAVIS Challenge on Video Object Segmentation (2017)

    Khoreva, A., Benenson, R., Ilg, E., Brox, T., Schiele, B.: Lucid data dreaming for object tracking. In: The DAVIS Challenge on Video Object Segmentation (2017)

    2017

  5. [13]

    In: Proceedings of the 3rd International Conference on Learning Representations (ICLR) (2014)

    Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. In: Proceedings of the 3rd International Conference on Learning Representations (ICLR) (2014)

    2014

  6. [14]

    Kristan, M., Leonardis, A., Matas, J., Felsberg, M., Pflugfelder, R., ˇCehovin, L., Voj´ ıˇ r, T., H¨ ager, G., Lukeˇ ziˇ c, A., et al.: The Visual Object Tracking VOT2016 Challenge Results, pp. 777–823. Springer International Publishing (2016)

    2016

  7. [15]

    In: Proceedings of the European Conference on Computer Vision (ECCV) (2018)

    Kristan, M., Leonardis, A., Matas, J., Felsberg, M., Pflugfelder, R., Cehovin Zajc, L., Vojir, T., Bhat, G., Lukezic, A., Eldesokey, A., et al.: The sixth visual object tracking vot2018 challenge results. In: Proceedings of the European Conference on Computer Vision (ECCV) (2018)

    2018

  8. [16]

    In: Proceedings of the IEEE international conference on computer vision workshops

    Kristan, M., Matas, J., Leonardis, A., Felsberg, M., Cehovin, L., Fern´ andez, G., Vo- jir, T., Hager, G., Nebehay, G., Pflugfelder, R.: The visual object tracking vot2015 challenge results. In: Proceedings of the IEEE international conference on computer vision workshops. pp. ...

    2015

  9. [17]

    IEEE Transactions on Pattern Analysis and Machine Intelligence 38(11), 2137–2155 (2016)

    Kristan, M., Matas, J., Leonardis, A., Tom´ aˇ s, V., Pflugfelder, R., Fernandez, G., Nebehay, G., Porikli, F., ˇCehovin, L.: A novel performance evaluation methodology for single-target trackers. IEEE Transactions on Pattern Analysis and Machine Intelligence 38(11), 2137–2155 (2016)

    2016

  10. [18]

    In: 2018 IEEE International Conference on Robotics and Automation (ICRA)

    Liang, P., Wu, Y., Lu, H., Wang, L., Liao, C., Ling, H.: Planar object tracking in the wild: A benchmark. In: 2018 IEEE International Conference on Robotics and Automation (ICRA). pp. 651–658. IEEE (2018)

    2018

  11. [19]

    In: Asian Conference on Computer Vision

    Neoral, M., ˇSochman, J., Matas, J.: Continual occlusion and optical flow estima- tion. In: Asian Conference on Computer Vision. pp. 159–174. Springer (2018) 14 Jon´ aˇ s ˇSer´ ych, Jiˇ r´ ı Matas

    2018

  12. [20]

    In: Computer Vision and Pattern Recognition (2016)

    Perazzi, F., Pont-Tuset, J., McWilliams, B., Van Gool, L., Gross, M., Sorkine- Hornung, A.: A benchmark dataset and evaluation methodology for video object segmentation. In: Computer Vision and Pattern Recognition (2016)

    2016

  13. [21]

    arXiv preprint arXiv:1704.00675v2 (2017)

    Pont-Tuset, J., Perazzi, F., Caelles, S., Arbel´ aez, P., Sorkine-Hornung, A., Van Gool, L.: The 2017 davis challenge on video object segmentation. arXiv preprint arXiv:1704.00675v2 (2017)

    2017 arXiv

  14. [22]

    British Machine Vision Conference (BMVC) (2017)

    Voigtlaender, P., Leibe, B.: Online adaptation of convolutional neural networks for video object segmentation. British Machine Vision Conference (BMVC) (2017)

    2017

  15. [23]

    IEEE Transactions on Pattern Analysis and Machine Intelligence 37(9), 1834–1848 (2015)

    Wu, Y., Lim, J., Yang, M.H.: Object tracking benchmark. IEEE Transactions on Pattern Analysis and Machine Intelligence 37(9), 1834–1848 (2015)

    2015

  16. [24]

    In: The European Conference on Computer Vision (ECCV) (2018)

    Xu, N., Yang, L., Fan, Y., Yang, J., Yue, D., Liang, Y., Price, B., Cohen, S., Huang, T.: Youtube-vos: Sequence-to-sequence video object segmentation. In: The European Conference on Computer Vision (ECCV) (2018)

    2018

  17. [25]

    arXiv preprint arXiv:1809.04320 (2018)

    Zhang, Y., Wang, D., Wang, L., Qi, J., Lu, H.: Learning regression and verification networks for long-term visual tracking. arXiv preprint arXiv:1809.04320 (2018)

    2018 arXiv

  18. [26]

    In: Proceedings of the European Conference on Computer Vision (ECCV)

    Zhu, Z., Wang, Q., Li, B., Wu, W., Yan, J., Hu, W.: Distractor-aware siamese networks for visual object tracking. In: Proceedings of the European Conference on Computer Vision (ECCV). pp. 101–117 (2018)

    2018

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.