{"id":"a3d55517-c452-48cb-a2c6-8aa0feca48a7","arxiv_id":"2505.05518","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":6,"one_line_summary":"A transformer-based model trained on hybrid synthetic and water-tank ICE data predicts catheter tip entry angle, rotation angle, and location, but the reported accuracy numbers conflict between the abstract and the results section.","lead":"This paper trains an AI model to estimate the angle and location of a medical catheter tip in intracardiac ultrasound images. The goal is to let a robot adjust the ultrasound probe automatically so the tip stays visible during heart procedures.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Abstract and Results report irreconcilable error numbers for the same evaluation, so the paper's headline accuracy claim is unsupported.","rationale":"I agree with the reader's REJECT verdict, but the load-bearing concern is the numerical contradiction in the central accuracy claim, not primarily the synthetic-to-real transfer. The reader's weakest_assumption is a valid external-validity limitation, yet the internal inconsistency is more fundamental: a paper cannot support its headline accuracy if the abstract and body report different results for the same evaluation. The proposed check would settle which number is correct; absent that, the paper is unverifiable. Lack of code, data, and baselines compounds the problem, but the immediate blocker is the unreconciled numbers. Therefore the reader's REJECT verdict stands unchanged; my partial agreement reflects that the reader's strongest_claim matches this concern while the weakest_assumption points to a different issue.","tokens_in":3252,"tokens_out":6220,"duration_ms":60197,"concrete_test":"Request the authors' evaluation code and saved predictions, or ask them to recompute the mean absolute entry-angle and rotation-angle errors over the same 250 test cases with the exact protocol used in Results, including per-sequence breakdowns. Then compare the reproduced numbers with the abstract's 3.32/12.76 and the Results' 13.76/22.99. If one set reproduces, the other section needs correction; if neither reproduces, the accuracy claim is not reproducible. This single check settles whether the headline accuracy claim is supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Abstract claims 3.32 deg entry angle error and 12.76 deg rotation angle error, but the Results section reports 13.76 +/- 9.58 deg and 22.99 +/- 15.12 deg for the same 250 test cases / 12 sequences. The two sets differ by factors of ~4 and ~1.8, and the manuscript offers no reconciliation (e.g., different metric, subset, or evaluation mode). The Discussion restates 'minimal errors' qualitatively without numbers, so the discrepancy is unresolved. Since these errors are the entire quantitative support for the central claim that the method can guide robotic ICE manipulation, the paper currently does not support its headline result. The synthetic-overlay evaluation is a further external-validity limitation, but it is secondary to the internal inconsistency: even before considering clinical transfer, the reported numbers are not self-consistent.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes a learning-based approach for estimating the incident entry angle, rotation angle, and passing point of a therapy device tip in intra-cardiac echocardiography (ICE) images, intended to support robot-assisted ICE catheter control. The method combines a pretrained ultrasound foundation model with a transformer encoder, and trains on a hybrid dataset of 5,698 image pairs built by overlaying extracted catheter-tip images onto real clinical ICE backgrounds, with ground-truth angles obtained from electromagnetic sensors. The authors report real-time inference at 25 Hz and evaluate on 250 synthetic test cases with 12 motion sequences, reporting an entry angle error and a rotation angle error, as well as an average IoU of 0.66.","tokens_in":3409,"tokens_out":5026,"duration_ms":45820,"significance":"The proposed hybrid dataset generation strategy is a useful methodological contribution, and the use of a pretrained US foundation model is timely; the authors are also transparent about the need for clinical validation. If the reported accuracy were consistent and clinically transferable, the framework could indeed facilitate real-time robotic ICE adjustments. However, the significance is currently limited by an unresolved internal contradiction between the abstract and the results, and by the fact that all quantitative evaluation is performed on synthetic overlays. The paper would be strengthened substantially by a reconciled set of error metrics, a comparison with simpler baselines, and at least a small set of real clinical sequences.","major_comments":[{"comment":"The abstract states that 'our method achieves 3.32 degree entry angle error, 12.76 degree rotation angle error,' whereas the Results section reports 'an average entry angle error of 13.76°±9.58° and a rotation angle error of 22.99°±15.12°' for the same test set of 250 cases across 12 sequences. These two sets of numbers differ by factors of approximately 4 and 1.8, respectively, and no explanation is given for the discrepancy (e.g., a different metric, a different test subset, or a different evaluation protocol). Because these error values are the sole quantitative evidence for the central claim that the model can support real-time robotic ICE manipulation, the paper must reconcile them: clearly define the reported metric (e.g., mean absolute error vs. root mean square error or median), specify whether the abstract numbers refer to a different experiment, or correct the erroneous figure. Without this reconciliation, the headline result is unsupported.","section":"Abstract and Results"},{"comment":"The quantitative evaluation is based entirely on synthetic test sequences created by overlaying tip images onto real ICE backgrounds, as described in the Dataset section. There is no validation on real clinical ICE data with either EM ground truth or manual annotation, and the Discussion acknowledges that 'expanding with more clinical cases is essential for better generalization.' As a result, the clinical utility of the method is not established; if the synthetic-to-real appearance and motion gap is large, the reported errors may not transfer to actual procedures. I would expect either a clinical feasibility study (even on a few sequences) or a clearly stated claim that the results are limited to synthetic evaluation and should not yet be interpreted as clinical performance.","section":"Dataset and Evaluation"},{"comment":"The paper does not include any baseline comparison or ablation study. The architecture combines a pretrained US foundation model, a transformer encoder, and historical information (B_{N-1}, A_{N-1}), but the Results only report the final model's performance. Without comparing to, e.g., a single-frame CNN, a model without the transformer, or a model without historical inputs, the contribution of each component (and the 'key innovation' of the hybrid dataset) is not verified. Adding at least one baseline and one ablation would substantiate the architectural choices and make the accuracy claim interpretable.","section":"Methods and Results"}],"minor_comments":[{"comment":"In Eq. (2), the text says 'where B_N and T_N are ground-truth values,' but the loss uses \\hat{B}, \\hat{A}, B_N, and A_N; T_N appears to be a typo for A_N.","section":"Equation (2)"},{"comment":"The caption of Figure 2 defines the entry angle as 'the angle at which the tip enters the US fan area,' while the Methods compute a_entry from the z-axis of E_tip_ice. It would be helpful to state explicitly whether the EM-based definition and the 2D visual definition coincide, or to provide a diagram aligning the two.","section":"Discussion and Figure 2"},{"comment":"The sentence 'One important thing to note is that there is no overlap between the real ICE images and tip sequence images in the training and test sets' is unclear: it should specify whether the same clinical background images could appear in both training and test sets with different overlays, and how the 12 test sequences were selected. Please clarify the split protocol to rule out information leakage.","section":"Data split"}],"recommendation":"major_revision","confidential_remarks":"The contradiction between the abstract and the body is the main obstacle; if the authors can provide the correct error numbers and a clear description of the evaluation metric, the paper could be publishable after major revision. I would encourage the editor to ask the authors to verify their reported results carefully before resubmission."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick read of arXiv:2505.05518. The thing you should know: the two headline numbers cannot both be right. The abstract says 3.32° entry angle error and 12.76° rotation angle error. The Results section reports 13.76°±9.58° and 22.99°±15.12° for the same 250 test cases and 12 sequences. That is a factor of ~4 on entry angle and ~1.8 on rotation. The manuscript offers no reconciliation, and the Discussion just restates 'minimal errors' without numbers. So the central quantitative claim is unsupported as written. If the abstract numbers are the real ones, the abstract omits essential context; if the Results numbers are real, the abstract is wrong. Either way, the paper needs a correction before any accuracy claim can be trusted.\n\nNow the credit. Predicting the device tip incident angle and passing point for ICE manipulation is a reasonable extension of the group's prior view-guidance work. The dataset construction is the most solid part: EM sensors on both the ICE catheter and the 9-Fr device tip in a water chamber, entry angle computed from the sensor transform, and tip images overlaid onto real clinical ICE backgrounds with motion continuity. That is a legitimate way to get labeled training data when clinical ground truth is expensive, and the no-overlap split between training and test images is a good detail. The architecture is sensible too: a pretrained US foundation model for features, a transformer over N=5 frames, separate heads for box and angles, and 25 Hz inference for real-time use.\n\nSoft spots beyond the contradiction. The evaluation is only on the synthetic-overlay test set. There is no baseline, so we do not know if the transformer and foundation model add anything over a per-frame detector or their earlier work. There is no ablation, either for the temporal history or the foundation model. The paper acknowledges the need for more clinical cases, which is honest, but the missing comparisons make it hard to assess the increment. I would not call the synthetic evaluation fatal on its own; the dataset design is careful and it is a reasonable first validation. The internal inconsistency is the load-bearing problem.\n\nBottom line: this is a genuine methods effort with a credible dataset setup, but the reported results are not self-consistent. If a quick correction resolves the discrepancy, the paper could be useful to people building real-time ICE guidance. I would not cite the accuracy numbers as they stand, though the dataset idea has merit. This deserves a serious referee: the flaw is fixable, not a dead end. I would send it to peer review with the requirement that the authors reconcile the numbers, add at least one baseline, and ideally an ablation.","headline":"The abstract and Results report incompatible error numbers for the same 250 test cases, so the paper's headline accuracy claim doesn't stand as written; the underlying dataset and real-time approach deserve a careful revision, not a desk reject.","tokens_in":3915,"tokens_out":3055,"would_cite":false,"duration_ms":25566,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"An AI model estimates the incident angle and passing point of a therapy-device tip in intra-cardiac ultrasound frames in real time, aiming to let a robot keep the tip continuously visible.","keywords":["intra-cardiac echocardiography","ICE","device tip tracking","incident angle estimation","ultrasound foundation model","transformer","synthetic data augmentation","robotic catheter control"],"falsifier":"Record real ICE sequences during a clinical ablation with EM sensors on both the ICE catheter and the therapy-device tip to obtain ground-truth entry and rotation angles, run this model on those sequences, and compare mean errors; if the entry-angle error exceeds the reported range or IoU drops far below 0.66, the synthetic-to-real transfer claim fails. Reproducing the reported experiment should also resolve whether the correct mean entry-angle error is the abstract's 3.32 degrees or the Results section's 13.76 degrees.","tokens_in":3001,"feed_emoji":"🩺","tokens_out":6791,"duration_ms":67761,"temperature":0.7,"pith_summary":"This paper is trying to establish that a neural network can watch a short sequence of intra-cardiac echocardiography (ICE) frames and continuously predict where an ablation catheter tip crosses the imaging plane and at what angle, so a robotic ICE catheter could be steered automatically to maintain visibility. The authors argue this capability is the missing piece for reducing manual ICE adjustments during electrophysiology and structural-heart procedures. They build a hybrid training set by overlaying synthetically positioned catheter-tip images on real clinical ICE backgrounds, which yields thousands of labeled examples without manual angle annotation. Using a pretrained ultrasound foundation model plus a transformer that remembers the previous tip position and angle, they report real-time prediction at 25 Hz and, in the Results section, average errors of 13.76 degrees in entry angle, 22.99 degrees in rotation angle, and an IoU of 0.66 on 250 synthetic test cases. If the approach transfers to real clinical imaging, it would give robotic systems the continuous tip-visibility signal they currently lack.","feed_headline":"AI predicts heart-device tip angle in ultrasound at 25 fps","feed_subtitle":"Trained on synthetic plus real ICE images, the transformer estimates tip entry and rotation to enable automatic probe re-aiming.","key_machinery":"The machinery is a sequence-to-state prediction pipeline: a pretrained ultrasound foundation model, trained on 37.4 million echocardiography images, extracts features from five 224-by-224 ICE frames; a linear layer projects the previous passing point and incident angle into the same feature space; a transformer with eight encoder layers and six attention heads fuses these features with a [CLS] token; and separate linear heads output the predicted bounding box and the two angular components. The hybrid dataset generation strategy is equally load-bearing: EM-sensor-equipped catheters in a water chamber provide precise ground truth, while extracted tip images overlaid on real clinical ICE backgrounds provide realistic anatomy, intensity variation, and motion continuity without manual annotation.","core_discovery":"The paper's central claim is that a device-tip orientation in an ICE image can be parameterized as a passing-point bounding box plus a two-component angle (entry angle and rotation angle), and that a transformer fed with features from a pretrained ultrasound foundation model can predict these quantities from five consecutive frames with enough accuracy and speed to guide robotic ICE catheter repositioning. The authors treat temporal history, namely the previous passing point and incident angle, as part of the input, so the network is predicting a continuous state rather than an isolated detection. The intended payoff is a closed loop: as the therapy device moves, the ICE catheter is re-aimed automatically so the tip does not leave the imaging plane, reducing operator workload during ablation and structural-heart device delivery.","pith_inferences":["The reported accuracy is measured only on synthetic composite test images made by overlaying tip images on real clinical backgrounds; the paper does not report a fully clinical evaluation, so the practical error on real ICE video could be different.","The abstract's error figures of 3.32 degrees and 12.76 degrees do not match the Results section's 13.76 degrees and 22.99 degrees, and the paper never addresses this discrepancy; reconciling it is a prerequisite for trusting either number.","A testable next step would be to train the same architecture on real clinical ICE clips with EM-sensor ground truth and check whether the synthetic-to-real transfer holds; if it does not, the hybrid dataset would need real images of the tip, not just real backgrounds."],"forward_implications":["At 25 Hz, the angle and position estimates are fast enough to update a robotic ICE catheter control loop in real time rather than only flagging views offline.","Because the previous passing point and angle are fed back into the transformer, the model is a temporal state estimator; it should stay stable across insertion, withdrawal, and speed changes instead of re-detecting from scratch each frame.","The synthetic overlay strategy removes the need for manual angular annotation at scale: EM-sensor water-chamber data provide labels, while clinical backgrounds provide realism and anatomical diversity.","If the model works in real clinical imaging, an operator would need fewer manual ICE re-aiming movements during ablation and structural-heart device delivery.","The same bounding-box-plus-angle formulation should extend to other therapy devices visualized by ICE, such as mitral clips and occluders, because they enter the imaging plane in the same geometric manner."],"supporting_citations":[{"why":"Shows an earlier automated tip-repositioning system for ICE, defining the robotic-control setting this model is built to feed.","marker":"[1]"},{"why":"Describes an earlier AI-driven view guidance system for ICE that this sequential angle estimator extends toward continuous tip visibility.","marker":"[2]"},{"why":"Reports earlier work toward automatic ICE catheter manipulation, motivating the need for real-time tip-direction estimates.","marker":"[3]"},{"why":"Supplies the pretrained ultrasound foundation model, trained on 37.4 million echocardiography images, used as the feature extractor.","marker":"[4]"}],"fun_headline_variants":["AI keeps therapy tip in view by predicting entry angle in ICE","Robotic ICE steering driven by AI tip-angle predictions","AI predicts tip angles to auto-aim ICE catheter at 25 fps","Transformer model tracks device tip for continuous ICE visibility"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The quantitative claims rest on a test set made by overlaying synthetic catheter-tip images onto real clinical ICE backgrounds, so the whole result depends on the assumption that those composites behave like genuine clinical ICE video of a moving therapy-device tip.","fun_headline_variants_meta":{"raw":{"variants":["AI keeps therapy tip in view by predicting entry angle in ICE","Robotic ICE steering driven by AI tip-angle predictions","AI predicts tip angles to auto-aim ICE catheter at 25 fps","Transformer model tracks device tip for continuous ICE visibility"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000996,"raw_usage":{"total_tokens":4243,"prompt_tokens":996,"completion_tokens":3247,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":612,"completion_tokens_details":{"reasoning_tokens":3179}},"tokens_in":612,"tokens_out":3247,"duration_ms":23016,"temperature":1.0,"reasoning_tokens":3179,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T23:18:10.922492+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Record real ICE sequences during a clinical ablation with EM sensors on both the ICE catheter and the therapy-device tip to obtain ground-truth entry and rotation angles, run this model on those sequences, and compare mean errors; if the entry-angle error exceeds the reported range or IoU drops far below 0.66, the synthetic-to-real transfer claim fails. Reproducing the reported experiment should also resolve whether the correct mean entry-angle error is the abstract's 3.32 degrees or the Results section's 13.76 degrees.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Shows an earlier automated tip-repositioning system for ICE, defining the robotic-control setting this model is built to feed."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Describes an earlier AI-driven view guidance system for ICE that this sequential angle estimator extends toward continuous tip visibility."},{"cited_title":"Towards Automatic Manipulation of Intra-cardiac Echocardiography Catheter","cited_arxiv_id":"2009.05859","evidence_quote":"Reports earlier work toward automatic ICE catheter manipulation, motivating the need for real-time tip-direction estimates."}],"review_version":1}