{"id":"58029a85-f815-452f-b4b7-46533a8d7510","arxiv_id":"2504.19398","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A memory-based dynamic tracking system for arthroscopic ACL navigation runs at 25 FPS and reduces tracking error by roughly 20-58% relative to the authors' previous static system.","lead":"This paper reports a video tracking system that keeps a surgeon's view of the knee bone steady during ACL reconstruction surgery, even when tools block the camera. It borrows a memory-based computer vision approach and reports faster, more accurate tracking than the authors' earlier static system.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Unreported ground-truth annotation reliability leaves the central pixel-error improvement potentially within label noise; this is the load-bearing uncertainty.","rationale":"The reader's weakest assumption is exactly the ground-truth annotation reliability, and I agree it is the most load-bearing. The central claim is an empirical comparison of pixel errors, and every number in that comparison is only as meaningful as the reference labels. The paper provides no inter-annotator agreement, no count of annotated frames, and no operational definition of the annotated 'position,' so the reported 45% / 35% / 19% improvements could be within annotation noise. This is not merely an omission; it directly determines whether the main result is statistically meaningful. I considered whether the missing definition of the anisotropic similarity measure (Section 5.2) or the arithmetic inconsistency (5.3 vs 12.6 px implies 57.9% improvement, not 45%) is more load-bearing. Those are serious reporting flaws, but they do not independently invalidate the empirical comparison the way unreliable ground truth does: the similarity measure could be defined in a revision, and the arithmetic error could be a typo, while the annotation issue threatens every reported accuracy number. The absence of external baselines is also a concern, but the central claim as stated is against the authors' prior static system, so the static-vs-dynamic comparison is the primary evidential basis. Therefore, the recommended verdict remains CONDITIONAL: the paper is a plausible engineering contribution, but until the annotation reliability is quantified, the headline accuracy improvements cannot be taken at face value. The concrete test I propose—a fourth annotator and per-annotator error recomputation—is a feasible, decisive check that would settle whether this concern lands.","tokens_in":17869,"tokens_out":4768,"duration_ms":46896,"concrete_test":"Take a random subset of 50 frames from each of the 20 sequences (or at least several representative sequences). Have a fourth arthroscopic surgeon, blinded to the system outputs and to the three original annotators, annotate the same femoral condyle position using an explicitly defined protocol (e.g., mark the centroid of the lateral femoral condyle articular surface). Compute pairwise distances among the four annotators and the mean distance from each annotator to the consensus. If the mean inter-annotator distance is comparable to or larger than the reported improvements (7.3 px at 1000 frames, 3.3 px at 500 frames, 0.9 px at 100 frames), the central accuracy claim is not reliably above label noise.","verdict_should_be":"UNCHANGED","load_bearing_attack":"All quantitative evidence for the central claim (dynamic system substantially outperforms static system) rests on the pixel-error metric defined in Section 4.2, which compares tracked femoral condyle positions against 'reference positions independently annotated by three experienced arthroscopic surgeons.' The paper never reports: how many frames were annotated, how the three surgeons' annotations were combined (majority vote, average, consensus?), or any inter-annotator agreement statistic. Without this, the reported effect sizes (e.g., 5.3±1.5 vs 12.6±3.7 px at 1000 frames; 4.5±1.1 vs 7.8±2.3 px at 500 frames) are not interpretable. The improvement at 500 frames is 3.3 px and at 100 frames is 0.9 px; if inter-annotator variability is on the order of even 2-3 px, the superiority claim is not statistically secured above the noise floor. Additionally, 'femoral condyle positions' is ambiguous—centroid, boundary, or a specific anatomical landmark? The static system comparison uses the same reference, but no detail is given about whether identical annotated frames were used for both systems. This concern is load-bearing because if the labels are noisy or systematically biased, every reported improvement percentage and the central claim of 'multi-level memory architecture' gains could be artifacts of the evaluation rather than real tracking improvements.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This manuscript extends the authors' previously published static markerless navigation system for ACL reconstruction by adding a dynamic tracking component based on a three-level memory architecture inspired by XMem and the Atkinson-Shiffrin memory model. The system performs automatic foreground segmentation of the femoral condyle, virtual-camera registration, multi-level memory tracking, and projection of a Bernard-Hertel grid onto arthroscopic views. Experiments on 20 arthroscopic video sequences are reported to show lower pixel error than the static baseline (5.3 vs. 12.6 pixels at 1000 frames), real-time operation at 25.3 FPS with 39.5 ms latency, reduced GPU memory, and lower tracking failure rates under occlusion, illumination change, motion blur, and viewpoint changes. Ablation and robustness analyses are also included.","tokens_in":18182,"tokens_out":4598,"duration_ms":48148,"significance":"If the reported results are reliable and reproducible, the system would be a practically useful contribution to arthroscopic navigation: it is markerless, runs in real time on mainstream hardware, and explicitly targets a clinical problem that is not well served by existing navigation systems. The authors deserve credit for reporting failure-rate analysis, ablation studies, and a candid limitations section. However, the central quantitative claim rests entirely on a self-referential comparison with the authors' own prior static system, and the evaluation protocol omits the annotation-reliability information needed to interpret the pixel-error numbers. Moreover, the headline improvement percentages are arithmetically inconsistent with the reported means. For these reasons the paper is best viewed as a promising engineering report whose performance claims require correction and strengthening before publication.","major_comments":[{"comment":"The reported improvement percentages are inconsistent with the reported error means. For 1000-frame sequences, (12.6−5.3)/12.6 = 57.9%, not approximately 45%; for 500-frame sequences, (7.8−4.5)/7.8 = 42.3%, not approximately 35%; and for 100-frame sequences, (4.1−3.2)/4.1 = 22.0%, not approximately 19%. These are the paper's headline quantitative claims and must be corrected or the calculation explicitly explained.","section":"Abstract and §4.2"},{"comment":"The pixel-error evaluation is not interpretable without details of the manual reference annotations. The paper does not state how many frames were annotated, how the three surgeons' annotations were combined, what exactly is meant by 'femoral condyle position' (centroid, boundary, or anatomical landmark), or any inter-annotator agreement statistic. Because the improvements at 500 and 100 frames are only 3.3 px and 0.9 px, label noise of even a few pixels could change the conclusions. The authors should provide this information and confirm that identical reference frames and identical annotations were used for both the dynamic and static systems.","section":"§4.1–§4.2"},{"comment":"The ablation study is reported only in prose and in a figure, with no numeric table in the text. Claims such as '62% higher tracking error,' '133% higher memory usage,' and '29% increase' cannot be checked from the material provided. Please include a table with per-condition tracking error, memory usage, and frame rate, including standard deviations and the number of sequences used.","section":"§4.4 and Fig. 12"},{"comment":"The multi-level memory architecture is described at a level of abstraction that prevents independent implementation. The training procedure, loss function, training data, network architecture for the feature extractor, GRU, and decoder, and key hyperparameters (e.g., α, β, γ, δ, r, and T_max) are not specified, and no code or model is released. Given that Section 2.2 identifies XMem as the source of the three-level memory structure, the authors should state precisely which components are borrowed and which are new, and provide enough implementation detail to reproduce the claimed gains.","section":"§3.4 and §3.5"}],"minor_comments":[{"comment":"The heading '3.4' is used twice, once for 'Multi-level Memory Architecture' and once for 'Bernard & Hertel Grid Projection', although the system overview in §3.1 describes the latter as Section 3.5; renumbering is needed.","section":"Section numbering"},{"comment":"The sentence 'Visual evidence supporting our quantitative findings is presented in Figure 3' appears to refer to the tracking comparison shown in Figure 10, not to the segmentation framework in Figure 3.","section":"§4.2, figure reference"},{"comment":"The statement 'Inspired by XMem [4]' cites the wrong reference; XMem is reference [20] in the bibliography.","section":"§3.4.1, citation"},{"comment":"Several equations contain typographical artifacts; for example, Eq. (34) is missing terminal punctuation and Eq. (12) is difficult to read as rendered. Please ensure all mathematical notation is legible and consistent.","section":"Equations"},{"comment":"The Data Availability Statement says data are included in the article or supplementary material, but no supplementary file is provided; please clarify what data or code can be shared.","section":"Data Availability Statement"},{"comment":"The robustness section refers to 'moderate motion blur' when describing the condition with a 15.7% failure rate, while the preceding text calls the same condition 'severe motion blur'; the terminology should be made consistent.","section":"§4.5, terminology"}],"recommendation":"major_revision","confidential_remarks":"This is a plausible engineering contribution, but the evaluation is currently self-referential and the quantitative claims are not robustly supported. The arithmetic inconsistencies in the headline percentages, the missing annotation-reliability protocol, and the lack of reproducibility details are likely to be seen as central weaknesses by the readership. I would ask the authors to also consider a comparison with a published VOS or tracking method on the same arthroscopic data, even if only as an additional table, and to provide a clear data/code availability statement before resubmission."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a plausible engineering adaptation of XMem to arthroscopic ACL navigation, but the headline accuracy gains rest on an under-documented annotation protocol and the paper's own arithmetic doesn't line up.\n\nWhat's actually new: they take the Atkinson-Shiffrin three-level memory idea and adapt it to the arthroscopic setting, with viewpoint-conditioned working memory, anatomical identifiers in long-term memory, and phase-aware forgetting. That is a sensible domain adaptation, and the system does run in real time (25.3 FPS, 39.5 ms latency, 1.8 GB memory) on modest hardware. The qualitative figures show tracking surviving instrument occlusion and viewpoint changes. They are explicit about building on XMem, and the ablation study at least attempts to isolate the contributions of sensory, working, and long-term memory.\n\nThe soft spots are in the evaluation. The only baseline is their own previous static system; there is no comparison to XMem, TAM, or any generic VOS method. That would be fine if the gains were large and clean, but the ground truth is not characterized. 'Reference positions independently annotated by three experienced arthroscopic surgeons' — with no inter-annotator agreement, no statement of how the three sets were combined, and no count of how many frames were annotated. If label noise is on the order of 2-3 pixels, the 100-frame improvement (0.9 px) is below the noise floor, and the 500-frame improvement (3.3 px) is not clearly above it. This is the load-bearing uncertainty and it is unresolved.\n\nAlso, the paper's own percentages are wrong. From the reported means: 1000 frames is (12.6-5.3)/12.6 = 58%, not ~45%; 500 frames is (7.8-4.5)/7.8 = 42%, not ~35%; 100 frames is (4.1-3.2)/4.1 = 22%, not ~19%. Sloppy arithmetic in the headline numbers makes a reader wonder about the rest.\n\nThe 'anisotropic similarity measure' touted in Section 5.2 is never defined in Methods; the hyperparameters for HSV thresholds, memory consolidation weights, sampling interval, and forgetting decay are listed without values. No code or data is available. There's also a section-numbering glitch (two Section 3.4s). None of these kill the idea, but they make the current evidence conditional.\n\nThe citation pattern is fine; they cite the relevant VOS literature and their own prior static system appropriately.\n\nBottom line: the problem is real, the approach is plausible, and the system may well work. But the evaluation needs major revision: external baselines, a proper annotation-reliability analysis, corrected percentages, and a release of code/data or at least parameter settings. A serious referee should look at this; I'd recommend sending it out rather than desk-rejecting, with the expectation of a heavy revision.","headline":"Plausible XMem adaptation for arthroscopic ACL navigation, but the evaluation's ground-truth reliability and internal arithmetic need serious work before the accuracy claims hold.","tokens_in":18712,"tokens_out":3277,"would_cite":false,"duration_ms":29556,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A three-level memory architecture lets a markerless arthroscopic tracker follow the femoral condyle continuously, cutting pixel error by about 45 percent over 1000 frames while running at 25.3 FPS.","keywords":["ACL reconstruction","arthroscopic navigation","multi-level memory architecture","video object tracking","femoral condyle tracking","computer-assisted surgery","real-time segmentation","Atkinson-Shiffrin memory model"],"falsifier":"Recompute the pixel-error comparison on a set of frames independently annotated by at least two surgeons and report inter-annotator agreement, for example the mean distance between surgeon labels. If the average disagreement between annotators is comparable to or larger than the dynamic system's 5.3-pixel error, the claimed accuracy improvement over the static system is not established. A second check would be to run both systems on held-out sequences from a different hospital and verify that the roughly 45 percent gap persists.","tokens_in":1490,"feed_emoji":"🦵","tokens_out":2568,"duration_ms":66211,"temperature":0.7,"pith_summary":"The paper claims that adding a three-level memory architecture, adapted from the Atkinson-Shiffrin memory model, turns a static markerless arthroscopic navigation system into a dynamic one that tracks the femoral condyle continuously through ACL reconstruction video. On 20 surgical sequences, the dynamic system keeps tracking error near 5.3±1.5 pixels over 1000 frames while the static baseline drifts to 12.6±3.7 pixels, an improvement of roughly 45 percent, and it runs at 25.3 FPS with 39.5 ms latency. This matters because continuous, markerless navigation could give surgeons a stable anatomical reference during tunnel placement without extra hardware or repeated recalibration.","feed_headline":"Multi-level memory cuts ACL navigation error by 45 percent","feed_subtitle":"A markerless tracker follows the femoral condyle in real time, holding 5.3-pixel error over 1000 frames.","key_machinery":"The central mechanism is the three-level memory store built on the Atkinson-Shiffrin model: arthroscopic sensory memory, a GRU updated every frame and gated by an image-quality factor; arthroscopic working memory, which stores high-resolution features every r frames together with viewpoint parameters; and arthroscopic long-term memory, which compresses features into anatomical prototypes tagged with region identifiers such as intercondylar notch or lateral condyle. Reading fuses all three levels with current-frame features through attention and viewpoint similarity, while writing uses consolidation, potentiation, and a phase-aware forgetting rule. This machinery supplies temporal continuity, viewpoint-aware retrieval, and anatomical priors, which together are claimed to produce the accuracy and memory-efficiency gains.","core_discovery":"The paper's central claim is that a multi-level memory architecture, with sensory memory updated every frame, working memory holding viewpoint-tagged features, and long-term memory storing anatomical prototypes, is what lets the system track the femoral condyle across viewpoint changes, instrument occlusion, and tissue deformation. The authors argue that the static system fails during these events because it re-registers each frame independently, whereas the memory-based dynamic system reuses stored features and recovers tracking after disruptions. The quantitative evidence is the error comparison: 3.2 vs 4.1 pixels on 100-frame clips, 4.5 vs 7.8 on 500-frame clips, and 5.3 vs 12.6 on 1000-frame clips, with ablations showing that each memory level contributes and that removing long-term memory more than doubles memory use.","pith_inferences":["Beyond the paper's data, a direct comparison against a general video object segmentation model fine-tuned on arthroscopy would isolate whether the gains come from the three-level memory structure itself or from the arthroscopy-specific additions such as viewpoint parameters, anatomical identifiers, and quality gating.","Because the static baseline's error grows sharply with sequence length while the dynamic system's error grows slowly, the 45 percent improvement on 1000 frames likely understates the benefit on full 60-90 minute procedures, assuming the trend continues.","The authors list tibial plateau tracking as future work; if the memory architecture generalizes, the same system could plausibly track both femur and tibia without markers, giving complete ACL reconstruction navigation.","A practical deployment test on lower-resolution or compressed arthroscope feeds would be valuable, since motion blur is already the dominant failure mode and compression would add further degradation."],"forward_implications":["Surgeons could receive continuous navigation without recalibration during viewpoint changes, because the tracker reuses stored features instead of re-registering each frame.","The system meets real-time clinical demands, running at 25.3 FPS with 39.5 ms latency on a mainstream laptop GPU, comfortably above the 15 FPS threshold cited for arthroscopic surgery.","Memory efficiency improves with sequence length, with 1.8 GB used over 1000 frames versus 2.7 GB for the static baseline, making long procedures feasible on standard hardware.","Tracking failure rates drop under occlusion, illumination changes, and extreme viewpoint changes, with severe motion blur remaining the largest weakness at a 15.7 percent failure rate.","The anatomy-tagged long-term memory should allow the system to re-identify the femoral condyle after the arthroscope returns to a previously seen viewpoint."],"supporting_citations":[{"why":"Supplies the previous static markerless navigation system that serves as the baseline for all accuracy, speed, and memory comparisons.","marker":"[17]"},{"why":"Contributes the Atkinson-Shiffrin three-level memory architecture that the paper adapts into arthroscopic sensory, working, and long-term memory.","marker":"[20]"},{"why":"Provides the Semantic-SAM segmentation model used in the automatic foreground segmentation pipeline that initializes tracking.","marker":"[18]"},{"why":"Defines the general Track Anything model whose limitations in arthroscopic conditions motivate the domain-specific memory design.","marker":"[19]"}],"fun_headline_variants":["Memory-based tracking reduces ACL navigation error by 45%","ACL tracker with memory reduces error 45% over 1000 frames","Multi-level memory improves ACL navigation accuracy by 45%","Real-time ACL navigation with memory architecture reduces error 45%","Markerless ACL tracking gains 45% accuracy via memory architecture"],"cache_read_input_tokens":20736,"weakest_assumption_plain":"The reported pixel errors and improvement percentages treat the positions annotated by three surgeons as ground truth, yet the paper does not report how much the surgeons agreed with each other or how many frames were annotated; if those labels are noisy, the error numbers and the 45 percent improvement would not be trustworthy.","fun_headline_variants_meta":{"raw":{"variants":["Memory-based tracking reduces ACL navigation error by 45%","ACL tracker with memory reduces error 45% over 1000 frames","Multi-level memory improves ACL navigation accuracy by 45%","Real-time ACL navigation with memory architecture reduces error 45%","Markerless ACL tracking gains 45% accuracy via memory architecture"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000617,"raw_usage":{"total_tokens":2875,"prompt_tokens":964,"completion_tokens":1911,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":580,"completion_tokens_details":{"reasoning_tokens":1824}},"tokens_in":580,"tokens_out":1911,"duration_ms":12107,"temperature":1.0,"reasoning_tokens":1824,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T05:53:43.070681+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Recompute the pixel-error comparison on a set of frames independently annotated by at least two surgeons and report inter-annotator agreement, for example the mean distance between surgeon labels. If the average disagreement between annotators is comparable to or larger than the dynamic system's 5.3-pixel error, the claimed accuracy improvement over the static system is not established. A second check would be to run both systems on held-out sequences from a different hospital and verify that the roughly 45 percent gap persists.","supporting_citations":[{"cited_title":"Marker-Less Navigation System for Anterior Cruciate Ligament Reconstruction with 3D Femoral Analysis and Arthroscopic Guidance","cited_arxiv_id":null,"evidence_quote":"Supplies the previous static markerless navigation system that serves as the baseline for all accuracy, speed, and memory comparisons."},{"cited_title":"Xmem: Long-term video object segmentation with an atkinson-shiffrin memory model,","cited_arxiv_id":null,"evidence_quote":"Contributes the Atkinson-Shiffrin three-level memory architecture that the paper adapts into arthroscopic sensory, working, and long-term memory."}],"review_version":1}