{"id":"942814b1-c5b4-4b39-8d23-1658da48f2ad","arxiv_id":"2412.18505","paper_version":2,"verdict":"REJECT","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"An OCR pipeline extracts telemetry from FPV drone HUD video, and the authors claim 5-second sampling is the best accuracy-versus-efficiency trade-off while unprojected WGS84 coordinates badly underestimate distances and speeds.","lead":"VORTEX reads drone flight data such as position, speed, and altitude from the on-screen display of FPV drone videos using an OCR pipeline, then tests how often frames should be sampled and how coordinates should be processed. A smart generalist might read this because it proposes a cheap, open-source way to recover drone telemetry when proprietary flight logs are missing, though the validation rests on a single flight and no true ground truth.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Accuracy claims rest on unvalidated OCR baseline and are undermined by a factor-of-3.7 path-length contradiction; 5-second optimality is not supported.","rationale":"The reader's weakest assumption correctly identifies the core issue: the accuracy evaluation uses 1-second OCR output as the baseline, not ground-truth telemetry. My stress test confirms this is load-bearing and adds a concrete internal inconsistency that reinforces it. The 1-second sampling produces both a 2.23 km path length (from 82 points at 27.5 m spacing) and an 8.181 km UTM/Haversine distance in the same results section; these cannot both describe the same flight. The speed benchmark of roughly 67 km/h is consistent with the 2.23 km length over about 2 minutes, making 8.181 km an outlier that suggests a bug in distance computation, perhaps from using absolute positions rather than consecutive point-to-point distances. Because the 'within 4.2%' accuracy claim and the RMSE analysis are built on these distance and speed calculations, the central 5-second optimality conclusion has no sound empirical basis. Additionally, Figure 7's caption says raw WGS84 'overestimate' distances and speeds while the text says 'underestimate,' another sign the reported numbers are unreliable. The provided Google Colab links make the proposed ground-truth comparison and path-length recomputation feasible. The single-flight dataset and the already-known raw-WGS84 limitation are secondary; the reference-validity and internal-consistency problems are sufficient to reject the central claim. Since the reader already reached REJECT, my verdict is unchanged.","tokens_in":11417,"tokens_out":5219,"duration_ms":47277,"concrete_test":"Obtain the original flight-controller telemetry log for the Mount Etna flight (or re-fly the same route while logging GPS/barometer), then run VORTEX on the video and compute speed/distance errors of 1s, 5s, 10s, 15s, 20s OCR outputs against ground truth. Also recompute the 1s path length as the sum of consecutive Haversine distances between the 82 retained, spatially filtered points and compare it with both 2.23 km and 8.181 km. If the 1s OCR baseline deviates from ground truth by more than a few percent, or if the path-length contradiction persists, the 5s optimality claim is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central optimality claim is that 5-second sampling gives mean speed accuracy within 4.2% of the 1-second baseline (Section IV). The 1-second sampling is never validated against true flight-controller telemetry; it is just another OCR output. If MMOCR's errors are systematic, both 1s and 5s readings can carry the same bias, so the 4.2% figure measures agreement between two OCR runs, not accuracy. This concern is not abstract: Section IV first states that 1-second sampling yields 82 retained points with 27.5 m average spacing and a 2.23 km total path length, then later reports UTM/Haversine distance at 1-second as 8.181 km. The same flight cannot have both. A 67 km/h speed over 2.23 km implies about 2 minutes; over 8.181 km it implies about 7.3 minutes. The 122 raw frames at 1s sampling are consistent with ~2 minutes, so the 8.181 km value is likely an artifact of incorrect distance calculation. If the distance/speed numbers used for the accuracy comparison are wrong, the 'within 4.2%' and 80.5% overhead-reduction claims no longer support the 5-second optimum.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents VORTEX, an OCR-based pipeline for extracting drone telemetry (coordinates, speed, altitude) from FPV HUD video using MMOCR with CLAHE, adaptive thresholding, spatial filtering, and UTM/Haversine/WGS84 coordinate analysis. The authors compare temporal sampling rates (1, 5, 10, 15, and 20 seconds) and report that 5-second sampling is optimal, with 64% point retention, mean speed error within 4.2% of the 1-second baseline, and 80.5% computational overhead reduction. They also report that raw WGS84 coordinates underestimate distances and speeds relative to UTM and Haversine calculations. The manuscript includes open-source code links and claims to provide the first quantitative benchmarks for OCR-based drone telemetry extraction.","tokens_in":11715,"tokens_out":5263,"duration_ms":47153,"significance":"If the results were reliable, the paper would fill a genuine gap: an open-source, modular pipeline for recovering telemetry from FPV video when proprietary flight-controller data are unavailable. The systematic comparison of coordinate-processing methods and the release of code are useful strengths. However, the central accuracy and optimality claims are not supported as written. The evaluation baseline is the system's own 1-second OCR output rather than ground-truth telemetry, and key numerical results in Section IV are internally contradictory. These issues affect the paper's main conclusions, so the current significance is limited to the methodological proposal rather than the quantitative benchmarks claimed.","major_comments":[{"comment":"The Results section contains mutually inconsistent total path lengths. The first paragraph reports that 1-second sampling produced 122 raw points, 82 retained points, average spacing of 27.5 m, and a total path length of 2.23 km, while the later distance-comparison paragraph reports that UTM and Haversine methods both give 8.181 km at 1-second sampling. The 122 raw points at 1-second intervals imply a flight duration of about 122 seconds; a 2.23 km path at the reported mean speed of 67.03 km/h implies about 2 minutes, while 8.181 km at that speed implies about 7.3 minutes. The same contradiction appears at 5 seconds, where 16 clean points at 75 m spacing imply a path length of about 1.125 km, but the UTM/Haversine measurement is reported as 4.072 km, and at 15 seconds, where 9 points at 280 m spacing imply about 2.24 km, but the reported UTM/Haversine value is 1.690 km. These contradictions mean the distance and speed values used for the accuracy comparisons cannot all be correct, and the claimed 5-second optimality is therefore not supported by the data as presented.","section":"Section IV"},{"comment":"The accuracy evaluation is circular. All deviations, including the headline 'within 4.2% of the 1-second baseline,' compare OCR readings at one sampling rate against OCR readings at another sampling rate. The 1-second output is itself produced by the same MMOCR pipeline and is never validated against flight-controller telemetry, manual ground truth, or any independent source. If MMOCR makes systematic errors, those errors may be shared across sampling rates, so the RMSE and percentage-deviation figures measure internal consistency, not telemetry accuracy. This directly affects RQ2 and the conclusion that 5-second sampling is optimal, because the claim that 5-second speed is 'accurate' is meaningful only if the reference itself is accurate.","section":"Section IV and RQ2"},{"comment":"The headline numbers are internally inconsistent. The abstract states that 5-second sampling achieves 'mean speed accuracy within 4.2% of the 1-second baseline,' but Section IV reports that the 5-second mean speed (66.93 km/h) deviates from the 1-second value (67.03 km/h) by only 0.15%, while the 10-second sampling deviates by 4.4%. The abstract and conclusion also claim an 80.5% reduction in computational overhead, but the Results section derives this figure from a reduction in clean points (82 to 16), not from raw frames (122 to 25, which would be a 79.5% reduction) and not from actual processing time. The statement that 5-second sampling uses 4.07% of available frames is also unexplained, since 25 frames taken every 5 seconds from roughly 122 seconds of footage would be about 20% of the 1-second frame set. These discrepancies need to be reconciled before the claimed optimality can be assessed.","section":"Abstract vs. Section IV"},{"comment":"The caption of Figure 7 directly contradicts the text in Section IV. The caption states that raw WGS84 calculations 'consistently overestimate distances and speeds,' while the Results text repeatedly states that raw WGS84 coordinates underestimate distances and speeds (e.g., 'underestimated the total distance by 1.681 km (20.5% error)' and 'raw WGS84 calculations showed an average speed of 44.85 km/h (33.0% underestimation)'). This is not a minor wording issue because RQ3 and the coordinate-method comparison are load-bearing for the paper's conclusions; the reader cannot tell which direction the claimed bias actually takes.","section":"Figure 7 caption"},{"comment":"The spatial filtering stage relies on a fixed 2-km buffer around the LineString flight path to separate OCR outliers from valid positions. No sensitivity analysis is provided for this threshold, and the retention rates (67.2%, 64%, 76.9%, etc.) that feed into the sampling-rate comparison depend directly on this choice. Since the buffer size is an ad-hoc parameter rather than a derived quantity, the paper should either justify it empirically or show that the conclusions are robust to reasonable variations; otherwise the reported outlier-removal and retention-rate results are conditional on an unexamined assumption.","section":"Section III, spatial filtering"}],"minor_comments":[{"comment":"The index terms list 'Optical Computer Recognition' but the paper is about Optical Character Recognition; the expansion 'EXtraction' for VORTEX is also oddly capitalized.","section":"Abstract and Index Terms"},{"comment":"The methodology contains tense inconsistency, such as 'will be implemented' and 'The first ROI regions are the latitude and longitude regions,' which should be unified into a consistent past or present tense.","section":"Section III"},{"comment":"The Data Availability section says 'GutHub' instead of 'GitHub,' and the code links are given as bare URLs without version or access-date information.","section":"Data Availability"},{"comment":"Figure 6 panels are referenced in the text, but the text does not explain what is shown in each panel (for example, which panel contains the confidence intervals and which contains the speed distributions), making the figure harder to interpret.","section":"Section IV, Figure 6"}],"recommendation":"reject","confidential_remarks":"The paper addresses a worthwhile application area and provides a modular open-source pipeline, but the central results are internally inconsistent in multiple places (path lengths, percentage deviations, over/underestimation direction) and the accuracy evaluation is self-referential rather than validated against ground truth. Because the 5-second optimality claim depends on these numbers, the issues are load-bearing and cannot be fixed by minor edits. If the authors can reconcile the numerical contradictions and re-validate against true telemetry, a resubmission could be worth considering, but the current manuscript does not support its conclusions."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"VORTEX's real contribution is the pipeline itself: a modular, open-source way to pull HUD telemetry from FPV drone video with OCR, including CLAHE/threshold preprocessing, ROI labeling, UTM-based spatial filtering, and CSV/KMZ export. That specific combination is new in the literature, and the paper's review of adjacent OCR work (dashcams, drone license plates, industrial displays) is fair and correctly identifies the gap. The architecture is sensible and the code/data are promised on Colab/GitHub, though the links are not actually given in the manuscript text.\n\nWhat the paper does well: a clean four-stage design, systematic sampling-rate comparison, and an explicit comparison of UTM, Haversine, and raw WGS84 distance/speed calculations. The qualitative finding that raw WGS84 undercuts distances and speeds is mathematically expected, but it is a useful reminder for practitioners, and the reported UTM-vs-Haversine agreement (within 0.1%) is plausible.\n\nThe soft spots are serious. First, Section IV contains a factor-of-3.7 contradiction in the 1-second result: 82 clean points, 27.5 m average spacing, and 2.23 km total path length, then later 8.181 km for the same 1-second UTM/Haversine path. At an average speed of ~67 km/h, 2.23 km corresponds to roughly 2 minutes, which matches the 122 raw frames at 1-second sampling; 8.181 km implies 7.3 minutes. Both numbers cannot be right, and the distance figure feeds directly into the speed and error comparisons.\n\nSecond, the accuracy evaluation is self-referential. The 1-second sampling rate is treated as the baseline, but it is itself OCR output, never validated against true flight-controller telemetry. So 'within 4.2% of baseline' means two OCR runs agree, not that either is accurate. If MMOCR has systematic biases, both arms share them and the 5-second optimality claim collapses.\n\nThird, the single Mount Etna flight is too narrow to support a general optimal-sampling benchmark. The limitations section acknowledges this, but the abstract and conclusion state the 5-second result as a definitive benchmark. There is also a minor caption inconsistency in Figure 7 (text says underestimate, caption says overestimate).\n\nNet: the pipeline deserves referee time, but the paper is not close to acceptable as-is. A serious revision needs ground-truth validation, reconciliation of the distance numbers, and a much more hedged optimality claim. I would engage with it, not desk reject it.","headline":"Useful open-source pipeline for OCR-from-FPV-HUD telemetry, but the evaluation is self-referential and the reported 1-second path lengths are internally inconsistent; the 5-second optimality claim is not supported as written.","tokens_in":12214,"tokens_out":3632,"would_cite":false,"duration_ms":32829,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Five-second temporal sampling is the optimal trade-off for OCR-based drone telemetry extraction from FPV video, and raw WGS84 coordinates systematically understate distance and speed.","keywords":["optical character recognition","drone telemetry","FPV video","temporal sampling rate","coordinate reference systems","UTM projection","Haversine formula","spatial filtering"],"falsifier":"Run the same VORTEX pipeline on the same flight video and compare the OCR-extracted 5-second positions and speeds against the flight controller's internal telemetry log or a high-rate GPS truth track; if the mean speed discrepancy against the log exceeds the reported 4.2% (or the RMSE is much larger than 3.44 km/h), the optimality benchmark would need revision.","tokens_in":11224,"feed_emoji":"🛰️","tokens_out":11056,"duration_ms":90856,"temperature":0.7,"pith_summary":"VORTEX is a four-stage pipeline that reads drone telemetry—latitude, longitude, altitude, and speed—directly from the on-screen HUD text in first-person-view (FPV) flight video, using an open-source OCR engine and image preprocessing. The paper's central claim is that a 5-second temporal sampling rate is the best trade-off: it uses only about 4% of available frames, keeps 64% of extracted points after spatial filtering, stays within 4.2% of the 1-second baseline for mean speed, and reduces computational load by 80.5%. The second claim is that coordinate processing method is a separate, equally important lever: raw WGS84 coordinates underestimate distances by 15-30% and speeds by 20-35%, whereas UTM projection and Haversine calculations agree within 0.1%. Altitude measurements remain stable (2.1% variation) across all tested sampling rates. If these benchmarks are right, drone operators and tool builders can pick sampling intervals and coordinate methods from measured trade-offs instead of guesswork.","feed_headline":"Five-second sampling is the best balance for OCR drone telemetry","feed_subtitle":"New benchmarks: it cuts 80.5% of processing while staying within 4.2% of the 1-second baseline.","key_machinery":"The machinery is VORTEX, a four-stage pipeline: frame extraction with configurable temporal sampling, image preprocessing (CLAHE, Gaussian blur, adaptive thresholding, and Sobel), OCR through MMOCR's SATRN model on user-defined screen regions, and spatial filtering that removes outliers with a median-based baseline and a 2-km buffer around a UTM-projected flight path. The controlling object is the sampling interval, because it determines how many frames the OCR engine reads; the controlling comparison is among three coordinate-to-distance methods—UTM Zone 33N projection, Haversine on WGS84, and raw Euclidean distance on WGS84. The 5-second interval carries the optimality claim, and the UTM-versus-Haversine agreement carries the coordinate-processing claim.","core_discovery":"The central discovery, stated on the paper's own terms, is that the accuracy of OCR-derived drone telemetry depends jointly on how often frames are sampled and on how coordinates are turned into distances, and that a 5-second interval combined with UTM or Haversine processing is the practical optimum. At 5-second sampling the extracted mean airspeed (66.93 km/h) almost matches the 1-second baseline (67.03 km/h), with an RMSE of 3.44 km/h, while the number of frames OCR must read drops from 122 to 25, an 80.5% reduction in data volume. Across all sampling rates, UTM Zone 33N projection and the Haversine formula give essentially identical distances and speeds, while raw WGS84 Euclidean calculations shrink the measured flight, understating total distance by about 20% at 1-second sampling and average speed by about 33%. The paper also finds altitude is unusually robust to downsampling, varying only 2.1% across all intervals, and presents these numbers as the first quantitative benchmarks for this application category.","pith_inferences":["Editorial inference: the 5-second optimum is likely specific to this flight's dynamics; a slow, straight flight could tolerate 10-20 second sampling, while a fast, maneuver-heavy flight might need sub-5-second sampling, so the transferable result is the measured trade-off curve rather than the single number.","Editorial inference: the sharp RMSE spike at 15 seconds (24.46 km/h) followed by partial recovery at 20 seconds looks more like a few OCR misreads or route-segment effects than pure temporal aliasing; testing on multiple flights would show whether the spike is systematic.","Editorial inference: the near-identical UTM and Haversine results mean the two can be used interchangeably in practice; the important design rule is to avoid raw degree-based Euclidean distance, and that rule can be enforced as a simple automated check.","Editorial inference: because the baseline is denser OCR output rather than flight-controller ground truth, this paper establishes repeatability of the pipeline, not absolute accuracy; adding an external truth track would turn the benchmark into a calibration."],"forward_implications":["Telemetry extraction from FPV video becomes practical for long flights: sampling every 5 seconds cuts the number of frames OCR must process by roughly 80% while preserving the flight path's overall shape and mean speed.","Drone-telemetry software should compute distances with a projected or geodesic method; using raw latitude/longitude degrees will systematically under-report path length by 15-30% and speed by 20-35%.","Altitude readings can be trusted even at sparse sampling, so vertical profiles are cheap to produce and can be used as a stable reference when horizontal points fail.","Denser sampling does not automatically mean cleaner data: the 1-second interval needed the most aggressive spatial filtering (32.8% of points removed), so outlier removal is part of any accuracy budget.","Future OCR telemetry studies should report sampling rate and coordinate method together, since both materially change every distance and speed number."],"supporting_citations":[{"why":"Supplies the MMOCR OCR toolbox and SATRN model that perform the HUD text recognition.","marker":"[6]"},{"why":"Provides the image-preprocessing methods such as CLAHE and thresholding used to make HUD text readable.","marker":"[17]"},{"why":"Supplies the post-OCR processing context that motivates cleaning raw OCR output before spatial analysis.","marker":"[18]"},{"why":"Establishes the prior use case of reconstructing drone flight paths from GPS data that VORTEX extends to video-derived telemetry.","marker":"[4]"},{"why":"Shows a parallel telemetry-extraction approach from dashcam video, the closest existing application.","marker":"[12]"}],"fun_headline_variants":["5s sampling cuts OCR drone telemetry compute by 80.5%","First benchmarks: OCR drone telemetry at 5s sampling","UTM and Haversine beat raw WGS84 for drone distances","Altitude stable despite 20x fewer drone frames in OCR","VORTEX: 5s sampling trims 80% processing, keeps 4.2% accuracy"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The accuracy numbers are computed by comparing 5-second OCR output against 1-second OCR output, treating the denser OCR readings as ground truth rather than comparing them to the drone flight controller's own telemetry logs.","fun_headline_variants_meta":{"raw":{"variants":["5s sampling cuts OCR drone telemetry compute by 80.5%","First benchmarks: OCR drone telemetry at 5s sampling","UTM and Haversine beat raw WGS84 for drone distances","Altitude stable despite 20x fewer drone frames in OCR","VORTEX: 5s sampling trims 80% processing, keeps 4.2% accuracy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000199,"raw_usage":{"total_tokens":1431,"prompt_tokens":1061,"completion_tokens":370,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":677,"completion_tokens_details":{"reasoning_tokens":270}},"tokens_in":677,"tokens_out":370,"duration_ms":4067,"temperature":1.0,"reasoning_tokens":270,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T04:42:07.668693+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same VORTEX pipeline on the same flight video and compare the OCR-extracted 5-second positions and speeds against the flight controller's internal telemetry log or a high-rate GPS truth track; if the mean speed discrepancy against the log exceeds the reported 4.2% (or the RMSE is much larger than 3.44 km/h), the optimality benchmark would need revision.","supporting_citations":[{"cited_title":"OCR-based Solution for The Integration of Legacy And-Or Non-Electric Counters in Cloud Smart Grids,","cited_arxiv_id":null,"evidence_quote":"Provides the image-preprocessing methods such as CLAHE and thresholding used to make HUD text readable."},{"cited_title":"Drone GPS data analysis for flight path reconstruction: A study on DJI, Parrot & Yuneec make drones,","cited_arxiv_id":null,"evidence_quote":"Establishes the prior use case of reconstructing drone flight paths from GPS data that VORTEX extends to video-derived telemetry."},{"cited_title":"Extracting dashcam telemetry data for predicting energy use of electric vehicles,","cited_arxiv_id":null,"evidence_quote":"Shows a parallel telemetry-extraction approach from dashcam video, the closest existing application."}],"review_version":1}