{"id":"764f73e6-81b3-4df0-a32a-12acd43a52ff","arxiv_id":"2501.05961","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A Swin Transformer-based model reconstructs labeled 3D bone segmentations directly from two synthetic orthopaedic X-ray views, reporting state-of-the-art results across four anatomies.","lead":"Swin-X2S is a deep learning system that turns two X-ray views into a labeled 3D map of bones. It reports top accuracy on nine public datasets across four body regions, but the X-rays are synthetic simulations rather than real clinical images.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"All benchmark numbers are on DRRs synthesized from the same CT volumes used as ground truth; real X-ray physics and unknown geometry are untested. A paired real-X-ray/CT evaluation is required before the clinical and 'directly from raw X-ray' claims can stand.","rationale":"The reader's weakest assumption and my load-bearing concern are the same: all training and evaluation happen on synthetic DRRs, so the clinical and raw-X-ray claims rest on an unverified transfer step. I agree with the conditional verdict. The paper has real strengths: nine public datasets, four anatomies, multiple ablations, released code, and internally consistent SOTA margins on the DRR benchmark. The overbroad 'first' claim and missing significance tests are secondary, both because the reported margins are large and because the core empirical contribution is the benchmark itself. The synthetic-to-real gap is the one issue that determines whether the motivating clinical scenario is actually supported. Keeping the reader's CONDITIONAL verdict is appropriate: accept the benchmark as reported, but require a real-X-ray validation before the clinical promise is asserted.","tokens_in":16924,"tokens_out":5257,"duration_ms":54479,"concrete_test":"Evaluate the released Swin-X2S-Base checkpoint, without fine-tuning, on a paired dataset of real biplanar X-rays and CT scans for one anatomy, such as 50 pelvis cases with calibrated AP/lateral radiographs and CT. Register CT to X-ray geometry using known or estimated projection parameters, compute the same Dice, HD, L-error, and ID-rate, and compare with Table 2. A clinically meaningful drop, including a Dice decrease greater than 5 points or an ID-rate below 90 percent, would confirm the synthetic-to-real transfer concern; if performance is comparable on real X-rays, the concern is settled.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that Swin-X2S directly reconstructs 3D segmentation and labeling from biplanar 2D X-ray images with state-of-the-art performance and clinical promise. Every reported number is obtained on DRRs generated from CT volumes by ray casting (Section 4.1), and the ground-truth masks are the same CT segmentations. Consequently, the metrics measure a synthetic-to-synthetic mapping under fixed, known projection geometry. Real clinical X-rays introduce scatter, beam hardening, noise, unknown magnification and pose, arbitrary FOV cropping, and imperfect orthogonality between views; Section 4.4's fixed resampling to 128x160 presupposes an alignment that real data will not provide. The paper itself concedes in Section 6 that DRRs are a compromise because of the lack of paired real X-ray/CT datasets. If the synthetic-to-real gap is large, the reported Dice, HD, L-error, and ID-rate will not transfer, and both the clinical promise and the 'directly from raw X-ray images' claim are unsupported. This is an external validity gap, not an internal inconsistency, but it is the load-bearing assumption for the paper's motivating application.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Swin-X2S, an end-to-end encoder-decoder that takes biplanar X-ray images as input and produces a 3D voxel segmentation with per-instance labels. The encoder is a 2D Swin Transformer, a dimension-expanding module converts 2D features to 3D, and a 3D U-shaped decoder with cross-view attention fuses the two viewpoints. The method is evaluated on nine DRR-based subsets derived from five public CT datasets, covering femur, pelvis, spine, and rib with 54 categories, and is compared against seven reimplemented baseline architectures. The main reported outcomes are Dice, HD, L-error, ID-rate, plus clinical morphometry and landmark metrics, together with ablations on the number of views, skip connections, cross loss, pretraining, and data augmentation.","tokens_in":17150,"tokens_out":6267,"duration_ms":62003,"significance":"If the reported DRR results transfer to real clinical X-rays, this would be a useful contribution to a clinically relevant inverse problem. The paper offers a broad evaluation across nine public datasets and four anatomies, releases code, and explicitly targets segmentation and labeling in one framework. The main strengths are the breadth of the benchmark, the inclusion of clinical morphometry metrics, and the ablation study. However, the current evidence is entirely synthetic: all inputs are digitally reconstructed radiographs rendered from the same CT volumes that provide the ground-truth segmentations. Since the motivating application is clinical, the external-validity gap is the decisive weakness. The architectural claims are internally consistent and plausible, but the clinical and 'directly from raw X-ray' claims are not supported by the present evaluation.","major_comments":[{"comment":"All training and evaluation use DRR images generated from CT volumes, with fixed known projection geometry and with ground-truth masks derived from the same CT scans. Section 6 concedes that DRR generation is 'a compromise solution due to the lack of datasets consisting of paired real X-ray images and aligned CT scans.' Because the abstract and introduction claim reconstruction 'directly from biplanar raw X-ray images' and clinical promise, this is a load-bearing external-validity gap. Please add an evaluation on real X-ray/CT pairs, even a small dataset, or substantially revise the claims to state that all results are on DRR-simulated X-rays and that transfer to real X-rays remains untested.","section":"Section 4.1 and Section 6"},{"comment":"The main quantitative results are reported as single numbers without error bars, confidence intervals, or significance tests, and it is not stated whether these come from one run or multiple seeds. Several conclusions are drawn from small differences, such as Swin-X2S-Large versus Swin-X2S-Base on CTSpine1K (83.41% versus 83.84% Dice), and the 'state-of-the-art' claim depends on these comparisons. Please report mean and standard deviation over at least three training runs and, where appropriate, pairwise significance tests.","section":"Tables 2 and 3"},{"comment":"The seven comparison methods are reimplemented by the authors from the settings of [5], with the regression head replaced by a classification head, but no evidence is given that these reimplementations reproduce the original methods' performance. This creates a risk of unfair comparison. Please provide official implementations or pretrained weights when available, or demonstrate that the reimplementations match the original reported numbers on a shared benchmark.","section":"Section 5.2 and Appendix A"},{"comment":"The DRR generation protocol is under-specified: the paper does not report source-to-detector distance, detector resolution, ray-casting integration scheme, attenuation model, intensity normalization, or any parameter values. Since every reported metric depends on these synthetic images, the benchmark is not reproducible from the text. Please release the generation script or provide the full protocol.","section":"Section 4.1"},{"comment":"The clinical-metric results are shown only as scatter plots with regression lines; the actual mean values per method for FHR, NSA, pelvic landmark distances, vertebral parameters, and rib nlDice/LSCDError are not given in the text. The statement that 'Swin-X2S achieves the best femoral head radius (FHR) and neck shaft angle (NSA)' is therefore not directly verifiable. Please add a table with these numerical values and per-sample variability.","section":"Section 5.3 and Figure 6"}],"minor_comments":[{"comment":"The phrase 'raw X-ray images' is used in several places, but all inputs are DRR-synthesized images. This wording should be qualified to avoid overstating the evaluation setting.","section":"Abstract and Section 1"},{"comment":"Table 1 lists the Tiny model's Image Size as [96,96,128], which is inconsistent with the stated resampling to 128x128x160 in Section 4.4.","section":"Table 1 and Section 4.4"},{"comment":"The notation Q,K,V in R^{H×D×N} is inaccurate for attention matrices; these should be token-count by dimension, not image spatial dimensions.","section":"Equation (5)"},{"comment":"TL-Embedding's L-error on the TotalSegmentator-All dataset is reported as '-' with no explanation, which is confusing in a quantitative comparison table.","section":"Table 3"},{"comment":"The table uses the name 'AttentionNet' while the text refers to 'AttentionUNet', and the citation for 'UNet' points to Kasten et al. rather than to the original U-Net paper.","section":"Appendix Table 6"},{"comment":"The text describes a 'slight improvement' with more DRR views, but N=2 yields a lower Dice (83.23%) than N=1 (83.84%). The non-monotonic pattern should be described accurately.","section":"Section 5.4.1"},{"comment":"There are numerous typos, including 'state of art', 'preformed', and 'ti access' in Section 5.4.2, which should be corrected.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The paper is a solid engineering contribution with a broad DRR-based evaluation, but the absence of real X-ray validation is a central limitation that currently prevents acceptance. The authors' own Section 6 acknowledges the gap. I would be willing to reconsider after either a real X-ray/CT evaluation or a careful scoping of the claims, plus the statistical and reproducibility improvements listed above. The 'first' claim for four anatomies is plausible but hard to verify from the text; the comparison with [5] should be handled very carefully given the reimplementation risk."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Let me give you the short version: this is a solid, well-executed deep learning paper that reconstructs 3D bone segmentations with per-instance labels from biplanar X-ray images. The catch is that all of those X-ray images are synthetic DRRs generated from the same CT volumes that provide the ground truth. Real X-ray physics is never tested. The paper admits this in Section 6, so it's not hiding anything, but it means the clinical promise is unverified.\n\nWhat's genuinely new here is the scope and the packaging. The authors combine a 2D Swin Transformer encoder, a 1x1x1 convolution dimension expansion, and a 3D U-Net decoder with cross-view attention. These are known components, but the specific combination produces a single end-to-end model that handles four anatomies and 54 categories from uncropped biplanar inputs. The closest prior work (the [5] benchmark) required binary masks and spine cropping; this paper gets per-instance labels directly. The evaluation is extensive: nine public datasets, multiple baselines, ablations, and clinical metrics. The code is released. That is real work and it will be useful to the 2D-3D reconstruction community.\n\nThe soft spots are the ones you'd expect. First and foremost, the DRR-only evaluation. The stress-test note is correct: scatter, beam hardening, noise, unknown magnification and pose, non-orthogonal views, and FOV cropping are all absent. The fixed resampling to 128x160 presupposes alignment that real data won't give. So the headline claim of 'directly reconstructing from raw X-ray images' is only supported for synthetic inputs. Second, Tables 2 and 3 have no error bars or significance tests. The baselines are reimplemented by the authors from the settings in [5], which is standard practice but still worth flagging. Third, the 'first' claim in Section 2 is a bit strong, but it's defensible if you accept the qualifiers about four anatomies and direct per-instance labeling without cropping. The harder cases—rib and spine—have low Dice (45-56% for ribs, 64-84% for spine), which the paper reports honestly but which makes clear this isn't close to clinical use yet.\n\nWho should read this? Anyone working on 2D-3D reconstruction from X-rays or DRR-based shape regression. It's a good reference point and a fair benchmark-level contribution. The external validity gap is real, but it's not a fatal flaw in the methods; it's a limitation that should be stated more prominently. My recommendation: send it to peer review, with the expectation that the authors either add real X-ray validation or explicitly reframe the claims as applying to DRR inputs only. If they add error bars and significance tests, even better.","headline":"Solid multi-anatomy DRR-to-3D reconstruction with honest limitations; clinical claims rest on untested real-X-ray generalization.","tokens_in":17746,"tokens_out":4598,"would_cite":true,"duration_ms":38522,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Swin-X2S turns two orthogonal X-ray views into labeled 3D bone segmentation in one pass.","keywords":["2D-to-3D reconstruction","biplanar X-ray","Swin Transformer","3D segmentation and labeling","digitally reconstructed radiographs","bone shape reconstruction","cross-attention","54 bone categories"],"falsifier":"Collect paired real X-ray and CT images for the same patients, run the released Swin-X2S model, and compare its Dice, HD, localization error, and identification rate against CT ground truth; a large drop relative to the reported DRR results would show that DRR training does not transfer to clinical X-rays, which is the premise on which the clinical promise rests.","tokens_in":16700,"feed_emoji":"🦴","tokens_out":9009,"duration_ms":77104,"temperature":0.7,"pith_summary":"This paper introduces Swin-X2S, an end-to-end deep network that takes two uncropped orthogonal X-ray images (coronal and sagittal) and directly outputs a 3D voxel segmentation with per-instance labels. The authors claim it is the first framework to cover four anatomies—femur, pelvis, spine, and rib—totaling 54 bone categories, without per-anatomy preprocessing such as cropping individual vertebrae. Across nine datasets derived from five public CT collections, they report that Swin-X2S achieves the best Dice, Hausdorff distance, localization error, and identification rate among the methods compared, and also the best femoral head radius and neck-shaft angle among tested models. The clinical motivation is that X-ray is cheap and low-dose, so reliable 2D-to-3D reconstruction could provide 3D bone geometry for diagnosis and surgical planning where CT is unavailable or undesirable, particularly for children and low-resource settings. All experiments use DRR-synthesized images rather than real X-rays, so the paper's clinical promise rests on how well DRR training transfers to real patients.","feed_headline":"Two X-ray views yield labeled 3D bone models in one pass","feed_subtitle":"A Swin-transformer network reconstructs 54 bone classes across four anatomies and reports top scores on nine DRR datasets.","key_machinery":"The load-bearing components are (1) a 2D Swin Transformer encoder with windowed and shifted-window self-attention that captures long-range context in each X-ray; (2) a dimension-expanding module that aligns the coronal and sagittal feature maps into a common 3D volume via 1x1x1 3D convolutions, bridging the 2D-pixel-to-3D-voxel gap; and (3) a 3D U-shaped convolution decoder whose bottleneck applies cross-attention between the two views, letting one view recover structure occluded in the other. The training loss combines per-view DiceCE supervision with a KL-divergence cross-view consistency term that encourages the two views to agree. This machinery is what lets a single model output voxel-level segmentation and per-instance labels for all 54 classes directly from uncropped X-rays.","core_discovery":"The central claim is that a single encoder-decoder architecture can convert a biplanar X-ray pair into a full 3D labeled segmentation without any structure-specific preprocessing. The 2D Swin Transformer encoder extracts features from each view; a dimension-expanding module bridges the 2D-to-3D dimension gap by stacking coronal and sagittal features into a common volume with 1x1x1 convolutions; and a 3D U-shaped decoder with cross-attention at the bottleneck fuses the two views. The paper reports Swin-X2S-Base reaches 89.11% Dice on CTPelvic1K, 83.84% on CTSpine1K, and 57.51% on TotalSegmentator-All, with the best labeling and clinical morphometry results among the compared methods. It further claims to outperform prior work without requiring known vertebra locations or input cropping, which earlier spine reconstruction methods needed.","pith_inferences":["A concrete extension would be to fine-tune the released model on a small set of real X-ray/CT pairs; if DRR pretraining transfers, such fine-tuning may close much of the domain gap without a large real dataset.","Because the cross-view KL loss needs no ground-truth label, it opens a route to semi-supervised training on unpaired real X-rays, which could mitigate the DRR-to-real mismatch.","The dimension-expanding module is a generic 2D-to-3D bridge and could be reused for other biplanar reconstruction problems, including metal implants, foreign objects, or non-osseous structures mentioned as future work.","A head-to-head spine comparison against BX2S-Net without any vertebra-location preprocessing would isolate whether the reported advantage comes from the architecture itself or from removing the cropping requirement."],"forward_implications":["A single trained model can segment and label 54 bone categories across four anatomies directly from two uncropped orthogonal X-rays, removing the per-vertebra cropping and localization steps required by earlier spine methods.","Varying the number of DRR views from 1 to 20 changes Dice by less than 1%, so the paper argues biplanar input is the preferred operating point given radiation dose and scan time.","Ablation results attribute large gains to transformer skip connections, the cross-view KL loss, ImageNet-22K pretraining, and data augmentation; removing convolution skip connections makes the model untrainable.","On ribs, the reported nlDice above 80% with larger centerline errors indicates the dominant error is axial segment localization rather than shape curvature, which is the clinically relevant direction for fracture assessment.","The reported results are on DRR-synthesized images, and the paper identifies paired real X-ray/CT datasets as the urgent next step for clinical validation."],"supporting_citations":[{"why":"Defines the biplanar X-ray-to-3D-bone-shape reconstruction benchmark and comparison settings that Swin-X2S extends and evaluates against.","marker":"[5]"},{"why":"First end-to-end CT reconstruction from two orthogonal X-rays; provides the MultiScale2DConcat baseline and frames the 2D-to-3D reconstruction problem.","marker":"[4]"},{"why":"BX2S-Net, the prior spine reconstruction method that requires known vertebra locations for cropping; the limitation Swin-X2S claims to overcome.","marker":"[31]"},{"why":"Swin Transformer with shifted-window self-attention; supplies the 2D encoder backbone of Swin-X2S.","marker":"[33]"},{"why":"Swin UNETR; serves as a comparison architecture and evidence for Swin-based 3D medical image segmentation.","marker":"[35]"},{"why":"Digitally reconstructed radiograph technique; generates the paired biplanar images and CT masks used for all training and evaluation.","marker":"[41]"},{"why":"TotalSegmentator; source of the femur, pelvis, spine, rib, and all-anatomy datasets.","marker":"[42]"},{"why":"CTSpine1K; provides one of the three spine datasets used for model comparison.","marker":"[43]"},{"why":"VerSe benchmark; supplies the spine dataset and the Dice, HD, localization error, and identification-rate metrics.","marker":"[44]"},{"why":"RibSeg v2; provides the rib dataset and the centerline extraction/evaluation pipeline used for clinical metrics.","marker":"[45]"}],"fun_headline_variants":["One X-ray pair yields labeled 3D bone models directly","Swin Transformer reconstructs 3D bones from two X-ray views","End-to-end 3D bone labeling from biplanar X-rays","Two orthogonal X-rays become a segmented 3D bone volume","No manual steps: X-ray pair to full 3D bone segmentation"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that DRR images generated by ray-casting CT volumes are faithful enough stand-ins for real clinical X-rays that a model trained on them will reconstruct real patient anatomy with the same accuracy.","fun_headline_variants_meta":{"raw":{"variants":["One X-ray pair yields labeled 3D bone models directly","Swin Transformer reconstructs 3D bones from two X-ray views","End-to-end 3D bone labeling from biplanar X-rays","Two orthogonal X-rays become a segmented 3D bone volume","No manual steps: X-ray pair to full 3D bone segmentation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000253,"raw_usage":{"total_tokens":1600,"prompt_tokens":1013,"completion_tokens":587,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":629,"completion_tokens_details":{"reasoning_tokens":495}},"tokens_in":629,"tokens_out":587,"duration_ms":5649,"temperature":1.0,"reasoning_tokens":495,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T21:06:18.533515+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Collect paired real X-ray and CT images for the same patients, run the released Swin-X2S model, and compare its Dice, HD, localization error, and identification rate against CT ground truth; a large drop relative to the reported DRR results would show that DRR training does not transfer to clinical X-rays, which is the premise on which the clinical promise rests.","supporting_citations":[{"cited_title":"Benchmarking encoder-decoder architectures for biplanar x-ray to 3d bone shape reconstruction","cited_arxiv_id":null,"evidence_quote":"Defines the biplanar X-ray-to-3D-bone-shape reconstruction benchmark and comparison settings that Swin-X2S extends and evaluates against."},{"cited_title":"X2ct-gan: reconstruct- ing ct from biplanar x-rays with generative adversar- ial networks","cited_arxiv_id":null,"evidence_quote":"First end-to-end CT reconstruction from two orthogonal X-rays; provides the MultiScale2DConcat baseline and frames the 2D-to-3D reconstruction problem."},{"cited_title":"Bx2s-net: Learning to reconstruct 3d spinal structures from bi- planar x-ray images","cited_arxiv_id":null,"evidence_quote":"BX2S-Net, the prior spine reconstruction method that requires known vertebra locations for cropping; the limitation Swin-X2S claims to overcome."},{"cited_title":"Swin unetr: Swin transformers for semantic segmentation of brain tumors in mri images","cited_arxiv_id":null,"evidence_quote":"Swin UNETR; serves as a comparison architecture and evidence for Swin-based 3D medical image segmentation."},{"cited_title":"Ct imaging based digi- tally reconstructed radiographs and their application in brachytherapy","cited_arxiv_id":null,"evidence_quote":"Digitally reconstructed radiograph technique; generates the paired biplanar images and CT masks used for all training and evaluation."},{"cited_title":"Totalsegmentator: Robust segmen- tation of 104 anatomic structures in ct images","cited_arxiv_id":null,"evidence_quote":"TotalSegmentator; source of the femur, pelvis, spine, rib, and all-anatomy datasets."},{"cited_title":"Verse: a vertebrae labelling and segmenta- tion benchmark for multi-detector ct images","cited_arxiv_id":null,"evidence_quote":"VerSe benchmark; supplies the spine dataset and the Dice, HD, localization error, and identification-rate metrics."},{"cited_title":"Ribseg v2: A large-scale benchmark for rib labeling and anatomical centerline extraction","cited_arxiv_id":null,"evidence_quote":"RibSeg v2; provides the rib dataset and the centerline extraction/evaluation pipeline used for clinical metrics."}],"review_version":1}