{"id":"4e128c97-5761-4588-86c6-c0f2299f5696","arxiv_id":"1908.07135","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"This paper introduces a trainable framework that combines ConvLSTM-based video text detection with GRU-based online tracking, achieving state-of-the-art F-scores on ICDAR2013 Video, Minetto, and YVT.","lead":"An end-to-end neural network jointly detects text in video frames and tracks text instances across frames using temporal memory and learned descriptors. On three public video text benchmarks it reports about 2% higher F-score than previous methods, at real-time speed.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"YVT SOTA depends on under-specified tracking thresholds: without online tracking the method is below prior SOTA there, so the 2 F-score gain may be a threshold artifact.","rationale":"I read the paper as claiming a learnable end-to-end detector+tracker whose joint training and inference-time trajectory generation outperform prior methods by about 2 F on three benchmarks. The architecture is internally coherent: ConvLSTM detection, descriptor generation, GRU prediction, contrastive tracking loss, and an inference algorithm. The paper reports some hyperparameters (α=5, β=0.1, m=1.0, clip length 24, ROI 8×64, descriptor size 136) and useful ablations (Tables I and II). However, the inference algorithm's parameters are the load-bearing joint: YVT is the only benchmark where the method needs tracking to beat the previous best, and the gain there is a 10.6-point precision jump with flat recall. The reader's verdict already flags the missing thresholds and asks for sensitivity analysis; I agree. The concern is not that the method is impossible, but that the empirical SOTA claim is underdetermined by the paper: without the θl/θm/θh/τ values and robustness data, the 2% advantage on YVT could be a favorable selection of post-processing thresholds. Because this can be settled by an ablation grid and is fixable by reporting parameters, the correct disposition remains CONDITIONAL rather than rejection. No code or checkpoints are provided, so an independent reviewer cannot currently run the check; this reinforces, rather than weakens, the need for the clarified revision.","tokens_in":8844,"tokens_out":7965,"duration_ms":83644,"concrete_test":"On the YVT test split, fix the end-to-end detector and sweep Algorithm 1's parameters: θl ∈ {0.1,0.3,0.5}, θm ∈ {0.2,...,0.8}, θh ∈ {0.2,...,0.8}, τ ∈ {0,0.01,0.05,0.2,1}, and report the P/R/F surface. Include the tracker-free setting (bypassing Algorithm 1 or setting τ=0) with the same θl grid. If any tracker-free setting reaches F≈77.00, or if F=79.05 occurs only in a narrow ridge of the grid while neighboring settings drop below Wu et al.'s 77.00, the YVT improvement is a threshold artifact. The authors should also report the exact settings used for Table III and a one-dimensional sensitivity plot for each parameter.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The strongest empirical claim is an ~2% F-score improvement over SOTA on three benchmarks. On YVT, the end-to-end detector alone reaches F=74.86 in Table III, below Wu et al.'s 77.00; the claimed 79.05 appears only after Algorithm 1's online trajectory generation. That algorithm contains at least five unreported settings: detection threshold θl (step 1), matching threshold θm (step 3), new-trajectory threshold θh (step 5), the reward scale τ in τ·ln(length(tracklet)) (step 5), and K, the number of proposals kept from NMS. No sensitivity analysis is provided. The YVT improvement is entirely in precision (78.51→89.12) while recall is flat (71.54→71.03), the profile expected if the confidence reward is moving a threshold rather than recovering true detections. If θh/τ were chosen to suppress low-confidence unmatched boxes, F would change substantially; a different grid could put the method below 77.00. Because tracking is what lifts the method above SOTA on YVT, and its parameters are undisclosed, the central claim is not yet attributable to the end-to-end architecture, ConvLSTM, AGD/GRU, or joint training.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes an end-to-end trainable framework for video text detection with an online tracking branch. The detection branch augments a ResNet50+U-Net backbone with a ConvLSTM block to propagate temporal context, and the tracking branch builds appearance-geometry descriptors (AGD) for detected text proposals, estimates their next-frame descriptors (EAGD) with a GRU, and associates instances across frames by metric learning. At inference, an online trajectory generation algorithm (Algorithm 1) refines detection confidence scores by rewarding long matched tracklets. The method is evaluated on ICDAR 2013 Video, Minetto, and YVT, reporting state-of-the-art F-measure gains of about 2 points on each dataset and a runtime of 24.36 fps on a TITAN Xp. The paper also includes ablations of the ConvLSTM versus single-frame and Conv3D features, and of descriptor variants in a tracking metric evaluation on Minetto.","tokens_in":9131,"tokens_out":3214,"duration_ms":34500,"significance":"If the reported results hold, the paper would demonstrate a useful integration of video text detection and online tracking into one trainable model, with shared features and a real-time rate, which is a practically relevant contribution. The comparison of ConvLSTM against Conv3D and single-frame detection, and the ablation showing the benefit of matching AGD with EAGD, are informative. The use of public benchmarks with standard train/test splits is a positive feature; the central result is not circular in the sense of fitting a quantity and then calling it a prediction. However, the significance is currently limited by the lack of specification of the inference-time trajectory generation thresholds and the under-specification of the contrastive tracking loss, both of which are necessary to attribute the reported improvements to the proposed architecture rather than to a favorable post-processing configuration.","major_comments":[{"comment":"The online trajectory generation algorithm introduces at least five unreported settings: the detection threshold θ_l (step 1), the matching threshold θ_m (step 3), the high-confidence threshold θ_h and the reward scale τ (step 5), and the number of proposals K (Section III-B). On YVT, the end-to-end detector without online tracking reaches F=74.86 (Table III), which is below the prior state of the art (Wu et al., F=77.00); the reported F=79.05 is achieved only after applying Algorithm 1. The precision jump from 78.51 to 89.12 with essentially flat recall (71.54 to 71.03) is the characteristic signature of threshold-based suppression of low-confidence boxes rather than recovery of new true positives. Without reporting the parameter values and a sensitivity analysis, the central claim that the method significantly outperforms prior methods cannot be attributed specifically to the end-to-end architecture, ConvLSTM, AGD/GRU, or joint training. Please provide the exact values for these parameters and an experiment that shows the F-measure as a function of each threshold on at least one dataset.","section":"III-D (Algorithm 1) and Table III"},{"comment":"The contrastive tracking loss is under-specified. The double sum over i and j with denominator K^2 implies that all pairs of the K proposals are used, but the construction of the label y (L^t_{i,j}) is not described: it is not stated how positive pairs are defined (e.g., by IoU with ground-truth trajectories), how negative pairs are sampled, whether self-pairs (i=j) are excluded, or how the number of valid proposals per frame is handled when it is less than K. Since the loss is central to the joint training claim, please specify the pair sampling procedure, the label assignment protocol, and any handling of class imbalance between positive and negative pairs. In particular, clarify whether the descriptor distance d in Eq. (5) is computed between AGD_t and EAGD_{t-1} or between raw AGDs, because Eq. (4) suggests that matching is done with the estimated descriptors, but the loss formula does not make this explicit.","section":"Eq. (5)"},{"comment":"The GRU-based descriptor estimation is not sufficiently specified to be reproducible. The notation (EAGD_t, h_t) = GRU(AGD_t, mask_t * h_{t-1}) uses a single hidden state h_{t-1}, yet in multi-target tracking there is one hidden state per text trajectory; it is unclear how hidden states are indexed, initialized, or updated for newly appearing and disappearing text instances. The mask_t is described as a scalar ('set to zero when the instance does not exist'), but the mask must be instance-specific and applied per trajectory. Without a precise description of the hidden-state bookkeeping, the trajectory generation in Algorithm 1 cannot be reimplemented. Please provide the exact state-transition equations and the procedure for creating and discarding hidden states.","section":"Eq. (4) and Section III-C"}],"minor_comments":[{"comment":"There are typographical errors, e.g., 'palys' in the first paragraph of the Introduction, and 'rame size' in the ICDAR 2013 dataset description of Section IV-A.","section":"Abstract and Introduction"},{"comment":"The caption of Table III says 'SEVERAL PUBIC DATASETS'; 'Moslen' in the table appears to be a misspelling of Mosleh (reference [9]). Please correct these.","section":"Table III caption and References"},{"comment":"The row 'Ours with optical flow' shows a large drop in precision (79.97 to 68.49) but an increase in recall (55.21 to 55.69) compared with 'Ours with ConvLSTM'. The paper does not explain why optical flow hurts precision so markedly; a sentence explaining this would aid the reader.","section":"Section IV-C, Table I"},{"comment":"The implementation details state 'Each frame contains 10 detection boxes, including positive samples and negative samples' but it is not clear how these 10 boxes are chosen during training, e.g., whether they are ground-truth boxes, predicted proposals, or a mixture. This affects the interpretation of the tracking loss.","section":"Section IV-B"}],"recommendation":"major_revision","confidential_remarks":"The missing parameter values in Algorithm 1 are the main obstacle to accepting the paper in its current form. If the authors supply the thresholds and a sensitivity analysis, and if the YVT gain remains robust, the paper may be acceptable. The under-specification of Eq. (5) also needs to be fixed. I would not reject the paper, as the core idea is sound and the evidence, while incomplete, is not contradictory."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a competent incremental paper that joins ConvLSTM-based video text detection with GRU-based descriptor tracking in one trainable network, and it includes a decent set of ablations. The main empirical claim — roughly 2% F-score gains over SOTA on three benchmarks — mostly holds up for ICDAR 2013 and Minetto, where the detector alone already beats prior SOTA. On YVT, though, the story is different, and the stress-test concern is on target: the end-to-end detector without tracking scores 74.86, below Wu et al.'s 77.00, and the claimed 79.05 appears only after Algorithm 1's online trajectory generation. That algorithm uses at least five unreported settings (θl, θm, θh, τ, K), and no sensitivity analysis is given. The large precision jump on YVT (78.51→89.12) with flat recall is the profile you'd expect from a confidence threshold tweak, not from genuinely new detections. So the YVT claim is not yet attributable to the architecture; it could be a favorable parameter choice.\n\nThe paper does other things well. The ablations are honest and informative: two-stage versus end-to-end, without/with tracking, ConvLSTM versus Conv3D versus optical flow, appearance versus geometry descriptors, and AGD versus EAGD matching. These support the claim that joint training and the GRU-based descriptor help. The speed number (24.36 fps on TITAN Xp) is useful. The self-citation pattern is benign.\n\nSoft spots, in order: first, the threshold issue above. Second, Eq. 5 is under-specified — the double sum over K×K pairs with a binary label y doesn't explain how positive pairs are sampled or weighted; most pairs will be negative, and the loss will be dominated by them unless there's an explicit sampling scheme. Third, there are no error bars and no code or model release, so independent verification is hard. Fourth, calling the whole thing 'end-to-end' is a stretch: the online trajectory generation is a hand-designed inference heuristic with reward parameters, not part of the trained model. The paper itself admits this in the conclusion, which I appreciate.\n\nNone of this is fatal. The architecture is sensible, the ablations are thoughtful, and the ICDAR gain is robust. But the YVT headline number needs the thresholds disclosed and a sensitivity analysis before I'd trust it. The right outcome is major revision: report the settings, run a small grid over θh/τ, show the F-score landscape, and clarify the contrastive loss sampling.","headline":"A competent incremental video-text detection paper whose claimed SOTA on YVT rests on an under-specified online tracking step; the rest of the ablations are solid but the thresholds need disclosure and sensitivity analysis.","tokens_in":9637,"tokens_out":3398,"would_cite":false,"duration_ms":33627,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that video text detection and online tracking can be trained as one end-to-end network, with ConvLSTM carrying motion memory and GRU-based descriptors associating text across frames.","keywords":["video text detection","online tracking","ConvLSTM","scene text","text instance association","appearance-geometry descriptor","multi-object tracking","end-to-end learning"],"falsifier":"Vary the matching threshold $\\theta_m$ and the high-confidence threshold $\\theta_h$ over a grid on the ICDAR 2013 Video test videos while keeping the trained network fixed. If the F-score moves by more than the claimed two-point margin across reasonable settings, the benchmark gain would undercut the claim that end-to-end training is the source of the improvement.","tokens_in":8677,"feed_emoji":"🎬","tokens_out":8978,"duration_ms":83852,"temperature":0.7,"pith_summary":"This paper claims that video text detection and online tracking belong in a single trainable network instead of separate stages. The detection branch uses a ConvLSTM block on shared backbone features to carry motion memory forward, while the tracking branch represents each detected text box with an appearance-geometry descriptor and uses a GRU to predict the descriptor in the next frame before matching instances into trajectories. Jointly training the two branches, the authors report F-scores about two points above the best prior numbers on ICDAR 2013 Video, Minetto, and YVT, at a real-time 24.36 frames per second on a TITAN Xp. A reader should care because it suggests temporal context and trajectory supervision help video text detection without paying for a separate tracking network.","feed_headline":"One end-to-end network lifts video text F-score by 2 points","feed_subtitle":"ConvLSTM motion memory plus GRU-based text descriptors beats two-stage methods on three benchmarks at 24 fps.","key_machinery":"The load-bearing object is the appearance-geometry descriptor (AGD) together with its GRU-estimated next-frame version (EAGD). Appearance comes from an ROI Transform on the shared feature map followed by convolution and global pooling, giving 128 numbers. Geometry comes from embedding the eight normalized quadrangle coordinates through two fully connected layers, giving 8 numbers, and concatenation yields a 136-dimensional descriptor. A GRU consumes previous-frame descriptors, with a mask that resets the hidden state when an instance is new, and outputs predicted current-frame descriptors. These are matched to current descriptors by a similarity matrix and Kuhn-Munkres assignment with thresholds, and a contrastive loss trains positive same-instance pairs to be close and negative pairs to be far. In parallel, a ConvLSTM block on the shared features gives the detection branch long-term motion memory. The claim is that this descriptor-plus-GRU mechanism is what converts tracking from a separate post-processing stage into a nearly cost-free, jointly trainable part of detection.","core_discovery":"The central claim is that an end-to-end video text detector with online tracking outperforms two-stage detect-then-track pipelines. In the detection branch, ConvLSTM propagates frame-level information through the shared ResNet50/U-Net features, and detection is done in EAST-like anchor-free per-pixel quadrangle regression. In the tracking branch, every text proposal is summarized by a 136-dimensional appearance-geometry descriptor, where 128 appearance dimensions come from an ROI-transformed shared feature map and 8 geometry dimensions come from embedded quadrangle coordinates. The descriptors from the previous frame pass through a GRU to produce estimated descriptors for the current frame, and a Kuhn-Munkres match between estimated and current descriptors builds trajectories. The two branches are trained together with a detection loss plus a contrastive tracking loss. With this joint model, the paper reports F-scores of 66.92 on ICDAR 2013 Video, 90.32 on Minetto, and 79.05 on YVT, exceeding their compared two-stage versions and prior methods on each benchmark.","pith_inferences":["The largest reported jump is in YVT precision, from 78.51 to 89.12 in end-to-end mode, while recall is nearly flat; a testable extension would be measuring whether trajectory rewards mainly suppress false positives such as background patches that look like text.","Since the paper notes trajectory generation is not part of training, adding a differentiable trajectory-level loss could widen the gap between the two-stage and end-to-end variants further.","The descriptor design only needs a region feature and a geometry vector, so the same framework should transfer to arbitrary-shape text by swapping quadrangle regression for a segmentation-based detector.","If the matching thresholds are disclosed and swept, a reader could separate the contribution of joint training from the contribution of inference-time trajectory reward."],"forward_implications":["If the central claim is correct, two-stage video text systems can be replaced by one network whose tracking branch reuses detection features, making tracking nearly free and enabling real-time operation.","The ConvLSTM ablation suggests temporal context is the main source of detection gain over single-frame detectors, so streams with persistent text should benefit most.","Joint training with the tracking loss improves detection itself, meaning trajectory supervision can serve as extra training signal for text detection.","Because trajectory generation is online, the method applies to streaming video without waiting for future frames."],"supporting_citations":[{"why":"Supplies the anchor-free per-pixel quadrangle regression head and the EAST baseline the model builds on.","marker":"[30]"},{"why":"Provides the ConvLSTM block used to propagate spatial-temporal memory in the detection branch.","marker":"[23]"},{"why":"Defines the ROI Transform used to extract per-instance appearance features from shared feature maps.","marker":"[15]"},{"why":"Supplies the CLEAR MOT metrics (MOTP, MOTA) used to evaluate the tracking branch.","marker":"[1]"},{"why":"Provides the ICDAR 2013 Video dataset used for training and testing.","marker":"[6]"},{"why":"Provides the Minetto outdoor-video benchmark and the SnooperTrack comparison method.","marker":"[8]"},{"why":"Provides the YVT web-video dataset and benchmark used for evaluation.","marker":"[10]"},{"why":"Supplies the strongest Minetto tracking baseline the paper compares against.","marker":"[11]"}],"fun_headline_variants":["End-to-end video text detector with online tracking lifts F1 by 2","Joint detect-tracking net for video text: +2 F1, real-time 24 fps","ConvLSTM + GRU descriptor: video text detection F1 up 2 points","One network for video text: detection and tracking in tandem, +2 F1","Video text: end-to-end model with tracking beats two-stage by ~2 F1"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported gains assume the trajectory-generation thresholds and tracklet-length reward were fixed without per-dataset tuning; the paper does not disclose their values or a sensitivity analysis, so threshold tuning remains a possible alternative explanation for the improvement.","fun_headline_variants_meta":{"raw":{"variants":["End-to-end video text detector with online tracking lifts F1 by 2","Joint detect-tracking net for video text: +2 F1, real-time 24 fps","ConvLSTM + GRU descriptor: video text detection F1 up 2 points","One network for video text: detection and tracking in tandem, +2 F1","Video text: end-to-end model with tracking beats two-stage by ~2 F1"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000145,"raw_usage":{"total_tokens":1197,"prompt_tokens":983,"completion_tokens":214,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":599,"completion_tokens_details":{"reasoning_tokens":104}},"tokens_in":599,"tokens_out":214,"duration_ms":2840,"temperature":1.0,"reasoning_tokens":104,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T12:24:33.728747+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Vary the matching threshold $\\theta_m$ and the high-confidence threshold $\\theta_h$ over a grid on the ICDAR 2013 Video test videos while keeping the trained network fixed. If the F-score moves by more than the claimed two-point margin across reasonable settings, the benchmark gain would undercut the claim that end-to-end training is the source of the improvement.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the anchor-free per-pixel quadrangle regression head and the EAST baseline the model builds on."},{"cited_title":"Xingjian, Z","cited_arxiv_id":null,"evidence_quote":"Provides the ConvLSTM block used to propagate spatial-temporal memory in the detection branch."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the ROI Transform used to extract per-instance appearance features from shared feature maps."},{"cited_title":"Bernardin and R","cited_arxiv_id":null,"evidence_quote":"Supplies the CLEAR MOT metrics (MOTP, MOTA) used to evaluate the tracking branch."},{"cited_title":"Karatzas, F","cited_arxiv_id":null,"evidence_quote":"Provides the ICDAR 2013 Video dataset used for training and testing."},{"cited_title":"Minetto, N","cited_arxiv_id":null,"evidence_quote":"Provides the Minetto outdoor-video benchmark and the SnooperTrack comparison method."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the YVT web-video dataset and benchmark used for evaluation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the strongest Minetto tracking baseline the paper compares against."}],"review_version":1}