{"id":"7f2a0576-b97f-471c-8123-1e532d959c85","arxiv_id":"1908.08431","paper_version":2,"verdict":"REJECT","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"A multi-hypothesis CNN with a learned PET-residual loss produces pseudo-CTs that reduce PET reconstruction error at the cost of higher CT error.","lead":"This paper trains a neural network to synthesize pseudo-CT images for PET/MR attenuation correction by directly minimizing the downstream PET reconstruction error, not just the pixel error against the true CT. On a dataset of 400 MR/CT/PET slices, the method reports lower PET error (115 vs 140 a.u.) despite worse CT error (69.68 vs 66.25 HU).","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The pPET improvement is not established: the 70/10/20 split is applied to 400 slices from 20 patients, so the same patients appear in both training and test sets, and the paired t-test across slices does not demonstrate cross-patient generalization.","rationale":"The reader's weakest_assumption focuses on the unvalidated imitation network g_psi, which is a genuine and serious issue. I agree with that concern, but I would put the slice-level train/test split first because it directly invalidates the only statistical evidence for the headline result. Section 3 makes the data structure explicit (20 patients times 20 slices), and Section 2.4 states the split as 70/10/20 on images, with no indication of patient-level separation. Given the high within-patient correlation of adjacent central slices, the paired t-test is pseudoreplicated. The retraining with g_psi as a loss is also a real soft spot: the surrogate is trained on pCTs from the stage-1 network and then applied to the retrained network's outputs, with no validation of its predictions on that new distribution. If the patient-level re-test removes the improvement, the central claim fails regardless of whether g_psi is accurate; if the improvement survives, validating g_psi would be the next required check. The reader's REJECT verdict is therefore appropriate, and my stress test does not move it.","tokens_in":6009,"tokens_out":7572,"duration_ms":76001,"concrete_test":"Re-run the entire three-stage pipeline with a strict patient-level split: hold out 4 of the 20 patients (all 20 slices per patient, 80 slices) before any training, and use only the remaining 16 patients for training and validation. Recompute the pCT and pPET MAE on the 4 held-out patients and perform the paired comparison with patient as the unit of analysis (or using a mixed-effects model that accounts for within-patient correlation). If the imitation-learning pPET advantage over HighResNet does not survive this test, the reported p<10^-4 is attributable to slice-level leakage and the central claim fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central quantitative claim rests on the paired t-test (p<10^-4) comparing pPET MAE for imitation learning (115.41 +/- 78.72) vs HighResNet (140.76 +/- 91.87). Section 3 reports 20 patients with 20 central slices each, giving 400 images; Section 2.4 states a 70/10/20 split performed on 'whole images' — that is, on slices, not on patients. Under a random slice-level split, the 80 test slices will include slices from all 20 patients, and each patient's remaining slices are in training or validation. Adjacent central slices from the same patient are highly correlated in anatomy, CT Hounsfield units, and PET uptake, so the network can memorize patient-specific intensity patterns. The paired t-test then treats clustered slices as independent samples, a form of pseudoreplication that can produce a highly significant p-value without demonstrating generalization to new patients. A second, independent weakness is that the imitation network g_psi is trained on only 60 pCT/pPET pairs from 20 slices and is never validated; after the first network is retrained with g_psi as a loss term, g_psi is applied to pCTs outside its training distribution. Both issues bear on the central claim, but the slice-level split is the more immediate threat to the reported statistical evidence.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes a two-stage deep learning method for MR-to-CT synthesis in PET/MR attenuation correction. A multi-hypothesis HighResNet first generates M candidate pCTs; a second 'imitation' network learns to predict PET reconstruction residual maps from CT/pCT pairs; the first network is then retrained with a loss combining the CT L2 error and the imitation metric. The authors report on 400 slices from 20 patients that the proposed imitation learning yields a higher CT MAE (69.68 HU) than a baseline HighResNet (66.25 HU) but a significantly lower PET MAE (115.41 vs 140.76 arbitrary units; paired t-test p<10^-4). They also compare multi-hypothesis sampling with MC dropout for uncertainty estimation.","tokens_in":6285,"tokens_out":3655,"duration_ms":35621,"significance":"If the claimed downstream PET improvement were robustly demonstrated, the paper would make a useful contribution to MR-based attenuation correction by explicitly optimizing for PET-relevant error rather than CT fidelity, and by introducing a learned differentiable surrogate for the PET reconstruction process. The multi-hypothesis formulation and the comparison with MC dropout are also interesting. However, the current evidence is undermined by methodological concerns in the evaluation protocol and by the lack of validation of the surrogate metric network; the central claim that minimizing the learned metric improves final PET reconstruction remains to be established.","major_comments":[{"comment":"The 70/10/20 split is described as being performed on 'whole images', which in this paper are the 400 extracted 2D slices from 20 patients. The manuscript does not state that the split is patient-wise. Since the 20 central slices from a single patient are highly correlated in anatomy and intensity, a random slice-level split places slices from the same patient in both training and test sets. The reported paired t-test on the test slices then treats within-patient correlated observations as independent, which is pseudoreplication. Consequently, the central quantitative claim—that imitation learning significantly reduces pPET MAE (p<10^-4)—does not demonstrate cross-patient generalization. The authors should repeat the evaluation with a patient-level split or use a cluster-adjusted statistical test.","section":"Section 2.3 and Section 3"},{"comment":"The imitation network g_psi is trained on only 60 pCT/pPET pairs derived from 20 slices, and no validation of g_psi is reported (e.g., accuracy of its predicted PET residual maps on held-out patients). After the first network is retrained using g_psi as a loss term, the resulting pCTs are likely outside the distribution of the 60 training pairs. Without evidence that g_psi remains accurate on pCTs produced by the retrained network, the proposed mechanism for the PET improvement is unsupported. The authors should provide a quantitative evaluation of g_psi on an independent test set and demonstrate that minimizing the learned metric correlates with true PET residuals.","section":"Section 2.2"},{"comment":"The combined loss for the first network is written as L(x_i, y_i, z_i) = min_m [g_psi(f_phi(x_i), y_i), z_i]. This expression is not a well-defined scalar objective: g_psi outputs an error map, z_i is the ground-truth residual map, and the bracketed notation is ambiguous. The text later states that the first network is retrained using 'both the CT L2-loss and the metric loss in equal proportions', but the exact functional form and weighting are not specified. Since this loss is the core of the proposed method, the authors must define it precisely to make the method reproducible and the optimization objective unambiguous.","section":"Section 2.2"}],"minor_comments":[{"comment":"The description of data extraction states that '20 central slices per volume' were used, resulting in 400 images, but it is unclear how these slices were selected across patients (e.g., whether the same anatomical locations were used). This matters for interpreting the slice-level split and the extent of correlation in the test set.","section":"Section 3"},{"comment":"The PET MAE values are reported in arbitrary units ('a.u.'), but the manuscript does not explain how these units are defined or normalized. Please clarify the scale of the PET intensities and whether the MAE is computed on absolute activity values or on some standardized scale.","section":"Section 4"},{"comment":"Several implementation details are missing, including the number of hypotheses M, the exact weighting between the CT L2 loss and the metric loss during retraining, and the convergence criteria. Reporting these would improve reproducibility.","section":"Section 2.4"}],"recommendation":"major_revision","confidential_remarks":"The central idea is potentially valuable, but the evaluation is currently compromised by the likely slice-level data split and by the unvalidated imitation network. These are fixable with additional experiments (patient-level split, validation of g_psi, and a precisioned loss definition), so I recommend major revision rather than rejection. The paper's novelty is moderate, but the application to PET/MR attenuation correction is clinically relevant."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing you should know: this paper proposes something genuinely different for MR-to-CT synthesis in PET/MR. Instead of minimizing CT error directly, the authors generate multiple pCT hypotheses with a winner-takes-all loss, train a second CNN to imitate the PET reconstruction residual from a CT/pCT pair, then retrain the first network with that learned metric as part of its loss. The result they report—lower pPET MAE (115 vs 141) despite slightly higher CT MAE (70 vs 66)—is exactly what you'd want to see if the idea works. The authors earn credit for evaluating on the downstream endpoint and for illustrating that localized CT differences produce disproportionately large PET residuals.\n\nThe soft spots are not minor. Section 2.4 says the 70/10/20 split was applied to whole images—meaning slices, not patients. With 20 patients and 20 central slices each, the test set almost certainly contains slices from every patient, and adjacent central slices from the same patient are highly correlated. The paired t-test across slices is pseudoreplication; a patient-level split could easily erase the p<1e-4. That is load-bearing because the entire claim is cross-patient generalization.\n\nThe second issue is the imitation network. It was trained on 60 pairs from 20 slices and never validated. Once the first network is retrained using this network's outputs as a loss, the pCTs can move outside the distribution g_psi was trained on, and nothing in the paper checks whether g_psi is still accurate there. This matters because g_psi is the mechanism, not a detail.\n\nA smaller but real caveat: the PET data were simulated because raw PET data were unavailable. The authors are transparent about this, but the reported 18% improvement is in a simulation, not in real reconstructed PET.\n\nWho is this for: people working on PET/MR attenuation correction, and more broadly anyone designing task-specific losses for image translation. The idea is worth testing properly; the reported evidence is not. If I were editor, I would send it to review, but with the clear expectation that the authors redo the split at patient level, validate the surrogate metric, and temper the claims until those analyses exist. Desk rejection would be too harsh; acceptance without major revision would be unjustified.","headline":"A genuinely different MR-to-CT synthesis idea for PET/MR attenuation correction, but the slice-level split and unvalidated imitation network mean the headline PET improvement isn't established.","tokens_in":6849,"tokens_out":2864,"would_cite":true,"duration_ms":26969,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Optimizing PET residuals, not CT similarity, improves MR-to-CT synthesis for PET/MR attenuation correction.","keywords":["MR to CT synthesis","PET attenuation correction","imitation learning","multi-hypothesis learning","pseudo CT","PET/MRI","deep learning"],"falsifier":"Compute the imitation network's predicted PET residual maps on the retrained pCTs and compare them to the actual simulated PET residual maps; if the predictions are substantially inaccurate or biased, the improvement in pPET mean absolute error cannot be attributed to the learned metric.","tokens_in":5813,"feed_emoji":"🧠","tokens_out":6138,"duration_ms":57118,"temperature":0.7,"pith_summary":"This paper contends that the usual way of judging MR-to-CT synthesis—how closely the pseudo CT matches the true CT—is the wrong target when the pseudo CT is meant to be used as an attenuation map in PET/MR imaging. The authors build a two-stage framework: a network that predicts several plausible pseudo CTs, and a second 'imitation' network that learns to predict the PET reconstruction error that any given pCT would produce. The first network is then retrained using this learned error metric alongside the standard CT error. On a held-out test set drawn from 400 paired MR/CT/PET slices, the proposed pseudo CTs have a higher CT mean absolute error (69.68 HU) than the baseline (66.25 HU), yet yield a significantly lower PET error (115.41 versus 140.76, paired t-test p<$10^{-4}$). The take-away is that downstream task performance, not intermediate image fidelity, should drive synthesis.","feed_headline":"Worse CT matches, better PET reconstructions","feed_subtitle":"An imitation-learning loss that predicts PET residuals beats pixel-wise CT error for MR-to-CT synthesis.","key_machinery":"The central machinery is a three-phase training schedule. In phase one, a convolutional network is trained with a Winner-Takes-All loss to output several plausible pCTs from one MR, so each output head specialises in different anatomical features such as skull thickness or bone density. In phase two, a second network is trained, with the first network frozen, to take a CT and a pCT as input and predict the absolute residual between the true PET and the PET reconstructed using that pCT as the attenuation map; training data comes from a simulated PET acquisition and reconstruction pipeline. In phase three, the first network is retrained with a loss that is an equal combination of the CT L2 error and the learned PET-residual metric, which acts as a differentiable imitation of the reconstruction process. The multi-hypothesis Winner-Takes-All scheme is what allows the generator to maintain several plausible solutions rather than a single risk-minimising average.","core_discovery":"The paper's central discovery is that minimising the intensity-wise error between a synthetic CT and the true CT is not a good surrogate for minimising the error in the PET image that will be reconstructed from that synthetic CT. A small, localised difference in CT attenuation can produce a large error in the reconstructed PET. The authors therefore propose a differentiable proxy for PET reconstruction: a convolutional network trained on pairs of (CT, pCT) to predict the residual between the gold-standard PET and the pseudo-PET reconstructed with each pCT. When this proxy is used as a loss term in retraining the multi-hypothesis pCT generator, the generator learns pCTs that score worse on CT mean absolute error but better on PET mean absolute error by a statistically significant margin. The paper also reports that sampling multiple pCT hypotheses gives a more reliable estimate of PET uncertainty than Monte-Carlo dropout.","pith_inferences":["The same imitation-loss idea could transfer to radiotherapy planning, where synthetic CTs are used for dose calculation and the real metric is dose error, not CT intensity error.","Because the imitation network is trained on a specific PET reconstruction simulation, its benefit may shrink if the clinical reconstruction pipeline differs from the simulation; testing across reconstruction parameters would clarify this.","The multi-hypothesis outputs could serve as a built-in uncertainty estimate for the pCT itself, potentially flagging slices where the downstream PET is unreliable."],"forward_implications":["Evaluation of pCT synthesis should include downstream PET error, not just CT intensity metrics, because the two can disagree.","A learned differentiable proxy for PET reconstruction can be used to optimise the pCT generator without needing to reconstruct every training sample.","Multi-hypothesis sampling gives a wider and more reliable distribution of pPET intensities than Monte-Carlo dropout when estimating PET uncertainty.","The framework produces an average reduction in pPET mean absolute error from about 141 to about 115 arbitrary units on the held-out test data."],"supporting_citations":[{"why":"Introduces the multi-hypothesis Winner-Takes-All loss that lets the generator output several plausible pCTs.","marker":"[8]"},{"why":"Supplies the convolutional network architecture used for both the pCT generator and the imitation network.","marker":"[9]"},{"why":"Powers the simulated PET acquisition and reconstruction that creates the pPET training targets for the imitation network.","marker":"[12]"},{"why":"Provides the registered MR/CT brain data and the registration methodology used to build the training set.","marker":"[3]"}],"fun_headline_variants":["Imitation learning trades CT accuracy for PET quality","Synthetic CT that's worse is better for PET","Learning to fake CT for better PET scans","PET-first loss beats pixel-perfect CT synthesis","For PET/MR, perfect CT isn't the goal"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The imitation network, trained on only 60 pCT/pPET pairs from 20 slices, remains accurate on the pCTs produced after the first network is retrained, even though those retrained pCTs were not part of its training distribution.","fun_headline_variants_meta":{"raw":{"variants":["Imitation learning trades CT accuracy for PET quality","Synthetic CT that's worse is better for PET","Learning to fake CT for better PET scans","PET-first loss beats pixel-perfect CT synthesis","For PET/MR, perfect CT isn't the goal"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000539,"raw_usage":{"total_tokens":2600,"prompt_tokens":974,"completion_tokens":1626,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":590,"completion_tokens_details":{"reasoning_tokens":1553}},"tokens_in":590,"tokens_out":1626,"duration_ms":10840,"temperature":1.0,"reasoning_tokens":1553,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T11:54:54.169258+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute the imitation network's predicted PET residual maps on the retrained pCTs and compare them to the actual simulated PET residual maps; if the predictions are substantially inaccurate or biased, the improvement in pPET mean absolute error cannot be attributed to the learned metric.","supporting_citations":[{"cited_title":"In: Proceedings of the IEEE ICCV","cited_arxiv_id":null,"evidence_quote":"Introduces the multi-hypothesis Winner-Takes-All loss that lets the generator output several plausible pCTs."},{"cited_title":"In: IPMI, Springer (2017) 348–360","cited_arxiv_id":null,"evidence_quote":"Supplies the convolutional network architecture used for both the pCT generator and the imitation network."},{"cited_title":"Neuroinformatics 16(1) (2018) 95–115","cited_arxiv_id":null,"evidence_quote":"Powers the simulated PET acquisition and reconstruction that creates the pPET training targets for the imitation network."},{"cited_title":"IEEE TMI 33(12) (2014) 2332–2341","cited_arxiv_id":null,"evidence_quote":"Provides the registered MR/CT brain data and the registration methodology used to build the training set."}],"review_version":1}