{"id":"255931ec-b285-4973-be28-67a38d48e48c","arxiv_id":"2412.08313","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"A time-symmetric multi-object tracker, originally built for yeast cells, matches Tracktor's HOTA on pedestrian MOTS while scoring higher on association, though on a different test set.","lead":"This paper tests a tracking method that looks both forward and backward in video, originally designed for yeast cells, on synthetic objects and on pedestrian videos. It reports that the method matches an established pedestrian tracker on overall score while doing much better at keeping identities straight, but the comparison uses different test data.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The MOTS benchmark claim is uncontrolled: TS is scored on three training sequences with a 10-frame track filter, while Tracktor's 48.8 HOTA is the official test-set score, so the 'closely matches' comparison is unsupported.","rationale":"The reader's weakest assumption precisely identifies the uncontrolled benchmark comparison and the track-length filter's inflationary effect. The paper itself concedes the HOTA inflation in Sec. 6, and Sec. 5.3 explicitly substitutes training-set sequences for the official test set. This makes the strongest claim—that TS matches Tracktor and far exceeds its association accuracy—currently unverified. No ad hominem is intended: the concern is about the experimental comparison, not the authors' integrity. The synthetic experiments and attention analysis provide independent value, and the code is shared, so a conditional acceptance with a required same-protocol re-evaluation is appropriate. The concrete test would settle whether the headline MOTS claim holds when Tracktor is measured under identical conditions.","tokens_in":10674,"tokens_out":2340,"duration_ms":26334,"concrete_test":"Run the official Tracktor model (or a faithful reimplementation using the same Mask R-CNN detections) on the exact three MOTS training sequences used by the authors, apply the identical 10-frame track filter, and compute HOTA, DetA, and AssA with the authors' evaluation script. If Tracktor's filtered HOTA/AssA on those sequences is close to 48.56/82.39, the paper's claimed advantage collapses; if TS still shows a large AssA gain and comparable HOTA, the claim survives. Report per-sequence scores and variances, not just means.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim (Sec. 6.2) rests on comparing a mean HOTA of 48.56, computed on three MOTS training-set sequences (Sec. 5.3), against Tracktor's published HOTA of 48.8 from the official MOTS test set. These are different evaluation domains and different data, so the 'closely matches' assertion has no controlled basis. The AssA comparison (82.39 vs. 44.6) suffers from the same mismatch. Furthermore, Sec. 4 states that all tracks shorter than 10 frames were omitted, and Sec. 6 explicitly warns that this 'inflates DetA and HOTA scores.' This filter likely affects the TS result more than Tracktor's published numbers, which were not subjected to the same post-processing. Thus the main evidence for superior association and for the claim that 'detection is the limiting factor' is not established. The synthetic ablations and attention analysis are not affected by this concern, but the headline MOTS conclusion is.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper evaluates the time-symmetric tracking (TS) architecture introduced in [23] beyond its original yeast-cell setting. It presents an optimized implementation of the tracking pipeline, defines IoU-50 association metrics, and compares four trackers (Kalman filter, full TS, TS-L2, TS-Shape) on synthetic arrow and amoeboid scenarios designed around visual signaling and positional noise. It then reports a zero-shot transfer experiment from the MOTSynth-MOTS-CVPR22 synthetic dataset to three MOTS training-set sequences, together with an attention analysis of the local tracking model. The principal claim is that TS achieves a MOTS HOTA of 48.56 versus 48.8 for Tracktor, with a much higher AssA (82.39 vs. 44.6), implying superior association with detection as the limiting factor.","tokens_in":10909,"tokens_out":8984,"duration_ms":91565,"significance":"The synthetic ablations are thoughtfully designed and provide clean evidence that combining positional and morphological information helps in the tested regimes; the optimized implementation with memory and runtime comparisons is a useful engineering contribution; and the attention analysis is a reasonable first look at the model's temporal and spatial focus. The paper also makes code, data, and models available, which supports reproducibility. If the MOTS benchmark claim were placed on a controlled footing, the result would be of broad interest for offline and post-hoc tracking. As it stands, the headline benchmark is not established because the comparison uses different test sets and different post-processing, and the metric computation is not fully specified.","major_comments":[{"comment":"The headline comparison of TS with Tracktor is uncontrolled. The reported TS mean HOTA of 48.56 and AssA of 82.39 are computed on three sequences from the MOTS training set, while the quoted Tracktor values of 48.8 and 44.6 are the published official MOTSynth-MOTS-CVPR22 test-set scores. Different image sets, different difficulty levels, and different evaluation protocols make the statement that the two HOTA scores 'closely match' unsupported. The fix is to run Tracktor on exactly the same three sequences with the same track-filtering and scoring code and to report per-sequence numbers; otherwise the benchmark framing in §6.2 and the conclusion should be removed or explicitly downgraded to a non-comparative report.","section":"§5.3, §6.2, Fig. 6"},{"comment":"The 10-frame minimum track-length filter biases the comparison. The paper applies the filter to all four of its own models, but the quoted Tracktor scores were not produced under this filter. The paper itself states in §6 that excluding short tracks 'inflates DetA and HOTA scores.' This means the HOTA comparison with Tracktor cannot be attributed to tracking quality. Report all metrics both with and without the filter, give the number of removed tracks per model, and ensure that any external benchmark comparison uses identical postprocessing on both sides.","section":"§4, §6"},{"comment":"There is an inconsistency in the temporal ranges used in the ablation. Section 4 states that the Kalman filter was allowed a maximum re-interpolation distance of 8 frames to match a local-tracker TR value of 4, with maximum assignment distance 2TR = 8. Section 5.3 then says that for the MOTS experiment the TS model used a local tracking range of 2. If TR differs between the synthetic and MOTS experiments, the Kalman filter's maximum re-interpolation distance should be set to 2TR = 4 for the MOTS comparison; if it was not changed, the Kalman baseline is not matched to the TS model's temporal range. Please clarify and rerun the affected comparison.","section":"§4, §5.3"},{"comment":"The paper reports KDEs and 'mean values' for HOTA and AssA without defining the unit over which these quantities are computed. In the HOTA metric family, HOTA and AssA are globally defined per sequence, not per frame. If the KDEs are computed over frames, a per-frame adaptation of the metric must be defined; otherwise the reported mean HOTA of 48.56 is not the standard HOTA and cannot be compared with Tracktor's published value. Please specify the computation unit (frames, sequences, or sliding windows) and provide sequence-level HOTA and AssA values in addition to any frame-level distributions.","section":"§6, Fig. 6"},{"comment":"The assertion that 'detection is the limiting factor' is not directly supported. No detection-only evaluation of the Mask R-CNN stage on the same three sequences is reported, and no comparison of the detection quality of Tracktor and TS is given. The AssA gap alone cannot separate detection errors from association errors. Please report DetA and detection recall/precision per sequence for both Tracktor and TS, and ideally run a cross-pipeline experiment such as TS tracking applied to Tracktor detections or Tracktor applied to TS detections.","section":"§6.2"}],"minor_comments":[{"comment":"The attention analysis is based on one test sample of ten frames; the convergence claim is plausible, but a second sample or per-sample variance would make the conclusions about pretraining effects more robust.","section":"§6.3"},{"comment":"Please clarify the notation for the temporal window: '2TR + 1' appears as both an input/output channel count and a temporal range; state explicitly whether TR = 2 gives a 5-frame window for the MOTS model.","section":"§2"},{"comment":"The summation symbol is rendered as an italic 'X' in Eq. (3), and the definition of the intersection of IoU-50 indicators should be spelled out as a sum over matched pairs.","section":"§3.1, Eqs. (3)-(5)"},{"comment":"The phrase 'state-of-the-art overall segmentation and tracking performance on the MOTS personnel tracking dataset' should be softened until the controlled comparison requested above is available.","section":"Abstract, §8"},{"comment":"The sentence about the official test set is confusing because it first says the official test set has seven samples and then says the paper uses three training-set sequences; please state explicitly that the paper does not use the official test set and explain why.","section":"§5.3"},{"comment":"The code, data, and models link is a Google Drive folder; a versioned archive such as Zenodo or a GitHub release with a commit hash would improve reproducibility.","section":"Abstract"}],"recommendation":"major_revision","confidential_remarks":"To the editor: this is a useful but uneven evaluation study. The synthetic experiments are competently designed and the engineering improvements to the tracking pipeline are real. The main risk is the paper's most visible claim, the MOTS comparison with Tracktor, which is not a valid controlled benchmark as written. The issues are fixable by rerunning Tracktor under the same protocol, clarifying the HOTA computation unit, and aligning the temporal range settings. If those numbers cannot be supplied, the benchmark-based claims should be removed and the paper reframed as a qualitative zero-shot transfer study."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThis paper evaluates the authors' own time-symmetric (TS) tracker from cell imaging on pedestrian MOTS, plus synthetic scenarios and ablations. The synthetic part is genuinely useful: they compare TS against a Kalman filter and two stripped variants, and the results are internally consistent, showing that morphology plus position beats either alone, and that visual signaling helps. The attention analysis is minor but honest. The pipeline refactoring and shared code are also real contributions.\n\nThe problem is the MOTS benchmark claim. The authors compute TS scores on three MOTS training-set sequences, while the Tracktor baseline they compare against is the official test-set score. Those are different data with different difficulty and different evaluation protocols. On top of that, they drop all tracks shorter than 10 frames, and they themselves say this inflates DetA and HOTA. So the \"closely matches HOTA\" and \"much higher AssA\" statements in Sec. 6.2 have no controlled basis. The stress-test note is correct; this is not a minor caveat, it is the load-bearing claim of the paper. It is also the basis for the conclusion that detection, not tracking, is the limiting factor.\n\nThe paper is not sloppy in an incoherent way. The authors are candid about the track filter, and they clearly state why they chose the training sequences (outdated evaluation kit). But a candid explanation of why you couldn't do a controlled comparison does not make the comparison controlled. They need to run the same tracker and Tracktor on the same sequences, or at least re-run Tracktor on the three training sequences under the same 10-frame filter. Without that, the headline should be withdrawn or heavily softened.\n\nThe synthetic scenario results stand on their own. They are small but reproducible, and the code is shared. I would not cite the MOTS numbers, but I would consider the synthetic ablations if I were working on offline tracking.\n\nRecommendation: send to peer review, but condition acceptance on fixing the benchmark comparison. The paper has enough real content to deserve referee time, but as it stands it should not be accepted.","headline":"The synthetic ablations are solid, but the headline MOTS claim—TS matching Tracktor with far higher AssA—rests on comparing training-set scores with the official test-set score, so it collapses.","tokens_in":11379,"tokens_out":1852,"would_cite":false,"duration_ms":18601,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper shows that a time-symmetric tracker originally built for budding yeast cells transfers zero-shot to pedestrian multi-object tracking and segmentation, matching Tracktor's HOTA (48.56 vs 48.8) while nearly doubling its…","keywords":["time-symmetric tracking","MOTS","HOTA","instance segmentation","Kalman filter","zero-shot transfer","pedestrian tracking","saliency maps"],"falsifier":"Re-run the TS pipeline on the official seven-sequence MOTS test set with the standard evaluation kit and no 10-frame track cutoff; if HOTA falls well below Tracktor's 48.8, the parity claim fails. A second check is to recompute HOTA on the three training clips with all short tracks included, which would reveal how much of the score depends on the cutoff.","tokens_in":10471,"feed_emoji":"🎯","tokens_out":10535,"duration_ms":101062,"temperature":0.7,"pith_summary":"The paper sets out to show that a time-symmetric tracking architecture, designed for budding yeast cells, transfers zero-shot to pedestrian multi-object tracking and segmentation (MOTS) and that its identity-association strength is its main asset. On three MOTS training-set sequences, the TS architecture attains a mean HOTA of 48.56, closely matching the 48.8 published for the Tracktor benchmark, while its mean AssA of 82.39 far exceeds Tracktor's 44.6. The paper interprets this as evidence that the tracking stage is strong and that the Mask R-CNN detection step is what holds overall performance back. Through synthetic scenarios, it also argues that combining positional and morphological cues in the assignment step beats using either alone, and that future visual cues can inform predictions about past motion.","feed_headline":"Time-symmetric tracker matches Tracktor HOTA, doubles identity scores","feed_subtitle":"A yeast-cell tracker works zero-shot on pedestrians, nearly doubling Tracktor's identity association.","key_machinery":"The load-bearing mechanism is time-symmetric local tracking: a DeepLabV3+ semantic-segmentation model consumes a $2T_R+1$-frame window with a marker channel for the object being tracked, and outputs predicted masks for all frames in that window, both backward and forward. Matching uses mean IoU between local predictions at temporal offsets up to $2T_R$, so the same object can be re-identified after temporary loss; the Hungarian method then makes globally optimal assignments across the sequence, a depth-first search merges IDs to the minimal count, and missed instances are re-interpolated. The comparison machinery is the HOTA family (HOTA, DetA, AssA) plus IoU-50 association scores, evaluated against a Kalman filter baseline and two ablated TS variants: TS-L2, which ignores morphology, and TS-Shape, which ignores position.","core_discovery":"The paper's central discovery, stated on its own terms, is that the TS architecture's time-symmetric tracking mechanism generalizes far beyond videomicroscopy. With models trained on synthetic pedestrian data (MOTSynth) and evaluated zero-shot on real-world MOTS sequences, the TS tracker reaches overall HOTA parity with Tracktor (48.56 vs 48.8) while achieving a much higher association accuracy (AssA 82.39 vs 44.6). The authors conclude that the architecture's ability to compare local forward and backward predictions in the same state space, and to re-interpolate missed instances, gives it superior identity tracking, and that the remaining gap to a clearly better overall score is in detection and segmentation, not tracking.","pith_inferences":["Beyond the paper's comparison, the equal-HOTA / doubled-AssA pattern suggests a general recipe: take any forward tracker's boxes and run a time-symmetric re-association pass on the recorded sequence to recover identities, without retraining the detector.","The paper evaluates on three training clips; an obvious extension is to submit the same pipeline to the official MOTS test server with the current evaluation kit, which would turn the parity claim into a certified benchmark result.","The attention analysis's finding that pretrained temporal channels inherit RGB channel biases could be probed experimentally by fine-tuning with randomized channel orders; the paper itself reports no practical performance difference.","The 10-frame minimum-track cutoff inflates DetA and HOTA, so an independent re-evaluation that counts short tracks would clarify how much of the headline parity is tracking skill versus score curation."],"forward_implications":["If the central claim holds, the association half of MOTS can be decoupled from detection: pairing the TS tracker with a stronger detector should push overall HOTA well above the Tracktor benchmark.","The large AssA advantage implies the TS tracker maintains stable identities across missed detections and occlusions much better than a forward-only box tracker, which matters for surveillance and crowd analysis on recorded footage.","Because the local tracker sees future frames, visual cues such as turn signals can be used to explain or predict earlier motion, an advantage unique to offline, post-hoc processing.","The ablation results imply that assignment by both position and morphology is more than the sum of its parts; a tracker that keeps either alone will be substantially weaker in cluttered or morphologically informative scenes."],"supporting_citations":[{"why":"Introduces the TS architecture and its original yeast-cell experiments, providing the model and training setup that this paper extends.","marker":"[23]"},{"why":"Tracktor is the benchmark whose published HOTA and AssA on MOTS are compared with the TS scores.","marker":"[5]"},{"why":"MOTSynth-MOTS-CVPR22 is the synthetic pedestrian training set used to train the models before zero-shot evaluation on MOTS.","marker":"[8]"},{"why":"MOTS is the real-world pedestrian dataset on which the zero-shot transfer and benchmark comparison are evaluated.","marker":"[26]"},{"why":"Defines HOTA, DetA and AssA, the metrics used for the headline comparison.","marker":"[16]"},{"why":"Kalman filter is the classical forward-prediction baseline against which the TS architecture and variants are compared.","marker":"[14]"},{"why":"Mask R-CNN is the instance-segmentation detector inside the TS pipeline, identified as the limiting component.","marker":"[11]"},{"why":"DeepLabV3+ is the semantic segmentation model used by the local tracking module for time-symmetric mask prediction.","marker":"[6]"}],"fun_headline_variants":["Time-symmetric tracker doubles pedestrian identity scores","Zero-shot pedestrian tracking with time-symmetric model","Time-symmetric MOTS matches Tracktor, outperforms on identity","Yeast-cell tracker generalizes to pedestrians zero-shot","Post-hoc MOTS: Strong identity tracking across domains"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The headline parity claim rests on treating HOTA computed on three MOTS training-set clips, with a 10-frame minimum track cutoff, as directly comparable to Tracktor's published score on the official test set.","fun_headline_variants_meta":{"raw":{"variants":["Time-symmetric tracker doubles pedestrian identity scores","Zero-shot pedestrian tracking with time-symmetric model","Time-symmetric MOTS matches Tracktor, outperforms on identity","Yeast-cell tracker generalizes to pedestrians zero-shot","Post-hoc MOTS: Strong identity tracking across domains"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000243,"raw_usage":{"total_tokens":1473,"prompt_tokens":835,"completion_tokens":638,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":451,"completion_tokens_details":{"reasoning_tokens":562}},"tokens_in":451,"tokens_out":638,"duration_ms":6554,"temperature":1.0,"reasoning_tokens":562,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T17:57:38.010638+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the TS pipeline on the official seven-sequence MOTS test set with the standard evaluation kit and no 10-frame track cutoff; if HOTA falls well below Tracktor's 48.8, the parity claim fails. A second check is to recompute HOTA on the three training clips with all short tracks included, which would reveal how much of the score depends on the cutoff.","supporting_citations":[{"cited_title":"Enhancing Cell Tracking with a Time-Symmetric Deep Learning Approach","cited_arxiv_id":"2308.03887","evidence_quote":"Introduces the TS architecture and its original yeast-cell experiments, providing the model and training setup that this paper extends."},{"cited_title":"Tracking without bells and whistles","cited_arxiv_id":null,"evidence_quote":"Tracktor is the benchmark whose published HOTA and AssA on MOTS are compared with the TS scores."},{"cited_title":"Motsynth: How can synthetic data help pedestrian detection and tracking? In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 10849–10859, 2021","cited_arxiv_id":null,"evidence_quote":"MOTSynth-MOTS-CVPR22 is the synthetic pedestrian training set used to train the models before zero-shot evaluation on MOTS."},{"cited_title":"Mots: Multi-object tracking and segmen- tation","cited_arxiv_id":null,"evidence_quote":"MOTS is the real-world pedestrian dataset on which the zero-shot transfer and benchmark comparison are evaluated."},{"cited_title":"Hota: A higher order metric for evaluating multi-object tracking","cited_arxiv_id":null,"evidence_quote":"Defines HOTA, DetA and AssA, the metrics used for the headline comparison."},{"cited_title":"K ´alm´an","cited_arxiv_id":null,"evidence_quote":"Kalman filter is the classical forward-prediction baseline against which the TS architecture and variants are compared."},{"cited_title":"Encoder-decoder with atrous separable convolution for semantic image segmentation","cited_arxiv_id":null,"evidence_quote":"DeepLabV3+ is the semantic segmentation model used by the local tracking module for time-symmetric mask prediction."}],"review_version":1}