{"id":"56c4cf27-c0a9-43aa-bcea-61263e0ce3b6","arxiv_id":"1908.02664","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"The paper defines coin-tracking, introduces the CTR dataset of 17 annotated sequences, and shows a segmentation-with-pose baseline (CTR-Base) outperforms existing long-term trackers on it.","lead":"This paper introduces a new video tracking task, coin-tracking, for rigid flat objects whose front and back can both appear, and releases a 17-sequence benchmark with pixel-level ground truth. It shows these sequences stress trackers in unusual ways and presents a baseline tracker built on segmentation plus explicit pose modeling.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Unverified ground-truth mask quality, coupled to CTR-Base's own score function, could inflate the reported performance gap over bounding-box baselines.","rationale":"The reader's weakest assumption identifies the unverified quality of hand-annotated segmentation masks, and I agree that this is the most load-bearing concern. The entire evaluation pipeline, including the dataset-difference statistics and all IoU numbers, depends on these masks. The additional specificity I add is the structural coupling between the masks and the proposed method: CTR-Base is initialized with ground-truth masks and its objective function directly optimizes agreement with those masks. This means annotation bias could differentially advantage CTR-Base relative to the bounding-box baselines, which do not have access to the mask during tracking. The concern is concrete and testable. I am not claiming the annotations are wrong, only that the paper provides no evidence they are correct, and the load-bearing claim depends on that evidence. There is no internal inconsistency in the paper, and the method itself is plausible; the dataset is released, which is a positive step. However, until mask quality is demonstrated, the quantitative headline results should be treated as provisional. Hence CONDITIONAL, matching the reader's verdict.","tokens_in":9560,"tokens_out":6072,"duration_ms":68432,"concrete_test":"Select a stratified sample of about 100 frames from the CTR dataset, oversampling challenging cases (e.g., from 'pingpong1', 'iccv bg handheld', and 'iccv handheld'). Have two independent annotators re-draw the object segmentation masks without seeing the original annotations, and compute the IoU between the original and each re-annotated mask. Then re-run the CTR-Base pose optimization and IoU evaluation on these frames using the re-annotated masks as ground truth, and compare with the bbox oracle on the same frames. If the inter-annotator IoU is below 0.85, or if CTR-Base's mean IoU shifts by more than 0.05 relative to the bbox oracle, the reported performance advantage is not robust to annotation uncertainty.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim that CTR-Base outperforms state-of-the-art long-term trackers on the CTR dataset rests entirely on IoU scores computed against hand-annotated segmentation masks. The paper provides no inter-annotator agreement study, no quality-control protocol, and no independent verification of masks on frames with motion blur, fast perspective change, or partial occlusion. This is not merely a generic dataset-quality caveat: the CTR-Base method is initialized with these very masks, and its homography objective in Section 3.1 uses the ground-truth mask as the canonical template. Specifically, Eq. 5's `sobj` and `scover` components reward a homography whose projected contour agrees with the ground-truth mask, and `sappearance` compares the current frame to the ground-truth template. If the annotations are systematically biased, for example consistently tighter than the true object boundary under motion blur, CTR-Base will be rewarded for reproducing that bias, while the bounding-box baselines and the bounding-box oracle are penalized for including the true object extent. Thus, even the mean IoU gap of 0.70 versus 0.63 for the bbox oracle could reflect annotation idiosyncrasies rather than genuine tracking superiority. The same masks underpin the dataset-difference claims (textureness and aspect-ratio statistics in Section 2.1), so the unsupported mask quality threatens both parts of the paper's central claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces 'coin-tracking' as a new visual tracking task: tracking rigid, approximately planar double-sided objects whose two sides are alternately visible. It contributes the CTR dataset of 17 sequences with segmentation masks on every fifth frame, and reports dataset statistics (textureness, aspect-ratio change) intended to show that CTR differs from VOT/OTB. The proposed baseline CTR-Base combines a fast-vos-style k-NN segmentation network with explicit homography pose estimation via simulated annealing and conservative online adaptation. Experiments report IoU 0.70 on CTR, outperforming MBMD (0.47), DaSiam LT (0.44), and a bounding-box oracle (0.63), plus mIoU 0.81 on POT-210. The paper concludes that coin-tracking is an open and challenging problem.","tokens_in":9757,"tokens_out":5245,"duration_ms":56419,"significance":"If the results hold, the paper provides a useful new benchmark and a sensible baseline for a practically common but previously unbenchmarked tracking scenario. The dataset fills a gap: existing planar-object tracking benchmarks mostly show one side and are dominated by camera motion, while CTR has fast object-driven out-of-plane rotation, blur, and illumination change. The baseline is a reasonable combination of published components (fast-vos embeddings, DeepLabv3+, simulated annealing), and the positive result on the external POT-210 benchmark is a genuine generalization check. The conservative online adaptation principle—only adapt when a homography-based pose model agrees with segmentation—is a well-motivated idea that could inform video object segmentation on long sequences. The paper provides the dataset URL and detailed training hyperparameters, though no code for the baseline is released.","major_comments":[{"comment":"The central comparative claim depends entirely on the accuracy of the hand-annotated CTR masks, yet the paper reports no annotation protocol, no inter-annotator agreement, and no quality check. This is load-bearing because CTR-Base's objective in Eq. (5) uses the ground-truth mask as the template: sobj and scover reward agreement with that mask, and sappearance compares against the ground-truth frame. If annotations are systematically biased, for example tighter than the true boundary on motion-blurred or strongly rotated frames, CTR-Base is rewarded for reproducing the bias while bounding-box baselines are penalized for including the true object extent. The same masks drive the dataset-difference statistics in Section 2.1. A concrete check would be to measure inter-annotator IoU on a subset of challenging frames, for example from sequences 'iccv handheld' and 'pingpong1', where CTR-Base reports 0.50 and 0.33.","section":"Section 2 and Section 3.1, Eq. (5); Table 1"},{"comment":"CTR-Base is a stochastic method (random homography perturbations, 50 random flow-based initializations, and simulated annealing in Section 3.1), but the reported IoU comes from a single run. Without multiple seeds or error bars, the 0.70 versus 0.63 gap over the bounding-box oracle may not be statistically significant. In addition, the protocol for the long-term trackers is underspecified: MBMD and DaSiam LT output a confidence and may declare frames 'not confident,' but the paper does not state how such outputs are converted to a mask for IoU. If absent detections are scored as empty masks, the comparison is dominated by this mapping; if they are excluded, the evaluated frame set becomes tracker-dependent. Please specify the mapping and report variance across at least several runs.","section":"Section 4.1, Table 1; Section 3.1"},{"comment":"The 'confident frames' IoU of 0.88 is computed only on the 47% of frames where the tracker itself decides it is in the tracking state, and that state is determined by the same score function used for pose estimation. The statement that the tracker 'is confident on the frames, where it performs well' is therefore partly self-referential. This experiment is useful as an indication of failure detection, but it does not support the overall superiority claim. The paper should either avoid framing this as a performance measure or compare failure-detection quality against an oracle, for example by reporting precision and recall of the confident state relative to frames where IoU exceeds a threshold.","section":"Section 4.2, Table 2"}],"minor_comments":[{"comment":"There is a typo: 'The are multiple levels of tracking' should be 'There are multiple levels of tracking.'","section":"Section 2, first paragraph after the problem definition"},{"comment":"The paper should state explicitly that IoU is computed only on annotated frames (every fifth frame). The text says frames with empty ground-truth segmentation do not contribute, but the caption 'Mean over all frames' is ambiguous.","section":"Section 2.2 and Table 1"},{"comment":"The ZNCC formula appears typeset incorrectly in the provided text; the denominator is missing the usual parentheses. Please verify that the equation matches the intended zero-normalized cross-correlation coefficient.","section":"Section 3.1, Eq. (3)"},{"comment":"For the POT-210 experiment, state whether frames with occlusion or out-of-view are included in the mIoU computation or excluded, since the ground-truth corners are converted to segmentation 'not modeling occlusions.'","section":"Section 4.1, POT-210 evaluation"},{"comment":"Format the 'frames in tracking state' column consistently; the coin3 row shows '02' without a percent sign, which is confusing.","section":"Table 2"}],"recommendation":"major_revision","confidential_remarks":"For the editor: this is a standard benchmark-plus-baseline paper. The main risk is that the CTR annotations are not validated and the reported gains could be artifacts of annotation bias or the stochastic evaluation protocol. I recommend requesting annotation-quality statistics, repeated runs with variance, and a precise protocol for how absent detections from long-term trackers are scored before accepting. The benchmark novelty is sufficient for the journal's scope, and the POT-210 generalization result is a useful positive check."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The coin-tracking paper is a genuine new benchmark contribution. It defines a new problem, releases a dataset of 17 sequences with segmentation masks, and shows convincingly that these sequences stress different dimensions (fast out-of-plane rotation, blur, aspect ratio change, low texture) than standard VOT/OTB. The dataset statistics in Section 2.1 are simple but effective. The baseline CTR-Base is a reasonable first attempt: fast-vos segmentation plus simulated-annealing homography search and a conservative adaptation rule, and it beats MBMD and DaSiam LT by a large margin. The POT-210 result (mIoU 0.81) shows the method generalizes beyond the new dataset.\n\nThe soft spots are real but not fatal. The IoU numbers come from a single run of a stochastic optimizer; simulated annealing is random and there are no error bars. For a paper whose headline is a performance gap, this is a gap. The code is not released, only the dataset, which makes reproduction harder. The evaluation masks have no inter-annotator agreement or quality control on blurry/rotated frames; since all IoU numbers and the dataset statistics come from those masks, a systematic annotation bias would affect both the dataset-difference claims and the reported gap. That said, the stress-test note overstates the coupling: CTR-Base uses the ground-truth mask only as the canonical template for initialization, not per-frame masks, so annotation bias would not directly reward the tracker's optimization. Still, a benchmark paper should demonstrate mask reliability. The \"confident frames\" analysis is interesting but covers only 47% of frames; it's more a failure detection demo than a full evaluation.\n\nThe paper is honest about its limitations, especially adaptation drift on long sequences. The problem formulation is new and the dataset is likely to be used by others. It deserves serious peer review; the right conditions would be a mask-quality analysis, multi-run statistics, and code release.","headline":"A genuinely new benchmark problem with a solid baseline; the performance claims are plausible but the paper needs a mask-quality study and repeated-run statistics before you should fully trust the numbers.","tokens_in":10338,"tokens_out":1622,"would_cite":true,"duration_ms":17188,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Coin-tracking is a distinct, hard visual tracking problem, and a segmentation-plus-pose baseline beats state-of-the-art long-term trackers on a new 17-video dataset.","keywords":["coin tracking","planar object tracking","video object segmentation","long-term visual tracking","homography estimation","tracking benchmark","online adaptation","segmentation masks"],"falsifier":"Re-annotate the CTR masks on the hardest frames (especially pingpong1 and the iccv handheld sequences) with multiple independent annotators and recompute the mean IoU; if the corrected masks shrink the gap between CTR-Base and the bounding-box oracle, or change the ranking against MBMD and DaSiam LT, the central performance claim fails.","tokens_in":9320,"feed_emoji":"🪙","tokens_out":8091,"duration_ms":72982,"temperature":0.7,"pith_summary":"The paper introduces coin-tracking: the visual tracking of rigid, approximately planar objects—coins, cards, badges—when both sides are visible and the object rotates fast enough to blur and change aspect ratio. It argues this is a distinct problem, separate from standard tracking benchmarks, and supports that with the CTR dataset of 17 long sequences annotated with segmentation masks on every fifth frame. The paper's main empirical claim is that a baseline tracker combining CNN segmentation with explicit homography pose modeling, called CTR-Base, reaches a mean IoU of 0.70 on CTR, ahead of the state-of-the-art long-term trackers MBMD (0.47) and DaSiam LT (0.44), and even ahead of a bounding-box oracle (0.63). The result matters because it shows that bounding-box representations are insufficient for such objects and that explicit pose modeling plus conservative online adaptation is a workable recipe on long sequences.","feed_headline":"Coin flips defeat standard trackers; a new baseline wins","feed_subtitle":"The new tracker scores 0.70 mean IoU on a 17-video benchmark; the best long-term baseline scores 0.47.","key_machinery":"The core machinery is the pair (CTR dataset, CTR-Base tracker). Coin-tracking state is represented by a segmentation mask plus a homography—a projective mapping from a canonical side image of the object to the current frame—because symmetric coins often make the homography non-unique on its own. The tracker's objective is a product of four sub-scores: the fraction of the segmentation inside the hypothesized contour, the fraction of the contour covered by the segmentation, the overlap of the current and previous visibility masks (occlusion consistency), and a zero-normalized cross-correlation appearance match to the canonical template. This homography optimization, driven by simulated annealing with optical-flow initialization, both stabilizes online adaptation and enables a lost-state re-detection that decides when the tracker may safely add new object and background examples to the k-NN embedding index.","core_discovery":"The central claim is that coin-tracking is a distinct tracking problem with its own failure modes—fast out-of-plane rotation, rapid illumination change, rotational blur, and strong aspect-ratio changes—and that existing datasets do not capture these modes. On the paper's own terms, the CTR dataset differs from VOT2016 and OTB in object textureness and in the magnitude and speed of aspect-ratio change, and the proposed CTR-Base method, which uses the fast-vos segmentation k-NN classifier and then optimizes a homography against a four-term product score, outperforms the VOT 2018 long-term tracking winners MBMD and DaSiam LT by mean IoU. The paper further reports a mean IoU of 0.88 on frames the tracker itself judges as confident, a generalization result of 0.81 mean IoU on the POT-210 planar-object dataset, and an explicit failure-detection mechanism that prevents online adaptation from drifting on long sequences.","pith_inferences":["The paper's mask-only state sidesteps the symmetric-ambiguity problem, but a natural extension it leaves implicit is predicting full 6D pose on sequences with known camera calibration, which robotic grasping would require.","Because the paper finds CTR objects far less textured than standard tracking sequences, keypoint-based homography estimators will likely fail on this dataset; a head-to-head stress test would make that boundary explicit.","The dataset could also be used for training rather than only evaluation, since its long sequences and rare object classes may improve robustness to rotational motion blur.","Re-annotating the masks on the hardest frames (e.g., pingpong1, where CTR-Base scores 0.33) with multiple annotators would test whether the reported performance gaps survive annotation noise."],"forward_implications":["Trackers should be evaluated on coin-tracking before claiming robustness to planar-object motion, because standard benchmarks miss the fastest out-of-plane rotations.","Bounding-box representations are a bottleneck even with perfect localization: the bounding-box oracle scores 0.63 mean IoU while the segmentation-based method reaches 0.70.","The confident-frame IoU of 0.88 implies that the remaining gap is failure detection and re-detection, not the segmentation quality itself.","The POT-210 generalization result suggests the approach transfers to planar objects seen from one side, so it may serve augmented-reality and robotic manipulation scenarios."],"supporting_citations":[{"why":"Supplies the k-NN embedding segmentation backbone that CTR-Base builds on.","marker":"[6]"},{"why":"Provides the DeepLabv3+ segmentation architecture used for the embedding network.","marker":"[4]"},{"why":"Defines the VOT long-term tracking evaluation protocol and the baselines' challenge context.","marker":"[15]"},{"why":"The planar-object dataset used to test generalization of CTR-Base.","marker":"[18]"},{"why":"A segmentation-annotated dataset used for comparison and for training/adaptation context.","marker":"[20]"},{"why":"Used to fine-tune the embedding network for segmentation on YouTube-VOS.","marker":"[24]"},{"why":"One of the state-of-the-art long-term trackers used as a baseline to beat on CTR.","marker":"[25]"},{"why":"Another long-term tracker baseline compared on CTR.","marker":"[26]"},{"why":"Optical flow used to initialize the homography optimization.","marker":"[19]"}],"fun_headline_variants":["Coin flips break standard trackers; new baseline leads","Coin-tracking: a new benchmark and baseline for double-sided objects","Fast-spinning coins expose tracking gaps; CTR-Base wins","Standard trackers fail on coin flips; new method wins IoU"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"All dataset statistics and IoU scores rest on hand-annotated segmentation masks for the 17 sequences, with no inter-annotator agreement or independent verification reported for frames with fast motion blur, strong perspective change, or partial occlusion.","fun_headline_variants_meta":{"raw":{"variants":["Coin flips break standard trackers; new baseline leads","Coin-tracking: a new benchmark and baseline for double-sided objects","Fast-spinning coins expose tracking gaps; CTR-Base wins","Standard trackers fail on coin flips; new method wins IoU"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000171,"raw_usage":{"total_tokens":1229,"prompt_tokens":863,"completion_tokens":366,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":479,"completion_tokens_details":{"reasoning_tokens":293}},"tokens_in":479,"tokens_out":366,"duration_ms":4023,"temperature":1.0,"reasoning_tokens":293,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T14:38:01.655079+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-annotate the CTR masks on the hardest frames (especially pingpong1 and the iccv handheld sequences) with multiple independent annotators and recompute the mean IoU; if the corrected masks shrink the gap between CTR-Base and the bounding-box oracle, or change the ranking against MBMD and DaSiam LT, the central performance claim fails.","supporting_citations":[{"cited_title":"In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","cited_arxiv_id":null,"evidence_quote":"Supplies the k-NN embedding segmentation backbone that CTR-Base builds on."},{"cited_title":"In: Proceedings of the European conference on computer vision (ECCV)","cited_arxiv_id":null,"evidence_quote":"Provides the DeepLabv3+ segmentation architecture used for the embedding network."},{"cited_title":"In: Proceedings of the European Conference on Computer Vision (ECCV) (2018)","cited_arxiv_id":null,"evidence_quote":"Defines the VOT long-term tracking evaluation protocol and the baselines' challenge context."},{"cited_title":"In: 2018 IEEE International Conference on Robotics and Automation (ICRA)","cited_arxiv_id":null,"evidence_quote":"The planar-object dataset used to test generalization of CTR-Base."},{"cited_title":"In: The European Conference on Computer Vision (ECCV) (2018)","cited_arxiv_id":null,"evidence_quote":"Used to fine-tune the embedding network for segmentation on YouTube-VOS."},{"cited_title":"In: Asian Conference on Computer Vision","cited_arxiv_id":null,"evidence_quote":"Optical flow used to initialize the homography optimization."}],"review_version":1}