{"id":"0fb760b7-584b-41af-931d-fcb65750b120","arxiv_id":"2412.11938","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"Rotated pathology image patches produce more similar latent representations in models that used rotation augmentation during self-supervised training.","lead":"This paper measures how much the internal representations of twelve pathology AI models change when their input images are rotated. Models trained with rotation augmentation stayed much more stable under rotation than models trained without it.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported rotation-invariance benefit may be confounded by unquantified corner artifacts: §2.3 does not specify rotation interpolation or border filling, and §3 attributes poor 45°/135°/225°/315° alignment to patch-corner differences, so the augmented-vs-non-augmented t-tests may compare artifact…","rationale":"The reader's weakest assumption identifies the same load-bearing concern: the rotation implementation is under-specified and the paper itself concedes that corner differences contaminate the rotated patches. I agree that this is the most direct threat to the central measured claim. If the rotated inputs contain variable border artifacts, then the significant group differences could arise from how models handle those artifacts rather than from rotation invariance of the underlying tissue representations. This is especially plausible because models trained with rotation augmentation may have been exposed to the same padding and interpolation artifacts during training, allowing them to appear more invariant on this specific benchmark even if their geometric invariance to clean rotations is no better than that of other models. The proposed center-crop or FFT-rotation check would settle this directly. The concern does not by itself overturn the empirical finding, but it does mean the claim should remain conditional on measurement details, matching the reader's conditional verdict. I would not move to reject or accept without the additional check, and the under-specified grouping and statistical unit could be addressed in the same revision, but the corner-artifact issue is the single most load-bearing concern for the central claim as stated.","tokens_in":4228,"tokens_out":6930,"duration_ms":67720,"concrete_test":"Re-run the full Section 3 analysis using a rotation method that preserves tissue content without introducing empty corners: for each 256×256 patch, rotate a larger canvas (e.g., 512×512) about its center using FFT-based or high-quality interpolation, then center-crop to 256×256; keep all model calls and metrics otherwise identical. If the augmented-vs-non-augmented t-tests for cosine distance and m-kNN remain significant, the corner-artifact objection is refuted; if they do not, the reported invariance benefit is an artifact of border handling rather than rotation invariance proper.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 2.3 states that rotations from 0° to 360° at 15° intervals were applied to WSI patches but gives no interpolation, resizing, or border-filling details. Section 3 then reports that alignment is poorest at 45°, 135°, 225°, and 315°, \"likely due to slight differences in the patch corners introduced by rotation\". This is an admission that the rotated inputs are not clean geometric rotations of the original tissue content; they contain non-tissue boundary pixels. Because each of the twelve foundation models has its own preprocessing (patch resizing, normalization, possibly its own augmentation-induced expectation of zero or reflected borders), models that saw rotation augmentation during training may have learned to accommodate exactly the corner artifacts produced by that pipeline. The central t-tests (t = −8.88, t = 6.91) are computed on these artifact-laden comparisons. If, for example, the non-augmented models attend more strongly to the empty or interpolated corners, the measured group difference could reflect border-handling behavior rather than invariance of tissue-feature representations. This concern is load-bearing because the paper's headline claim is specifically about rotation invariance, not robustness to input padding artifacts.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper evaluates the rotational invariance of latent representations from twelve pathology foundation models by comparing embeddings of original and rotated H&E patches from TCGA-KIRC WSIs. Invariance is quantified with mutual k-nearest neighbours (m-kNN) and cosine distance across rotations from 0° to 360° in 15° increments. The authors report that models trained with rotation augmentation achieve significantly better (i.e., more invariant) scores on both metrics, with t = −8.88 and t = 6.91, and conclude that rotation augmentation is necessary to learn rotation invariance in transformer-based pathology models. The paper includes a code repository and presents results as figures and heatmaps.","tokens_in":4482,"tokens_out":4348,"duration_ms":38910,"significance":"The question addressed is timely and practically relevant: foundation models for pathology are increasingly used in clinical pipelines, and understanding their invariance to geometric transformations is important for robustness. If the central claim holds, the paper provides actionable guidance for training future pathology models and contributes to the broader understanding of learned invariances in vision transformers. The study is empirical, the two metrics are clearly defined, and the code is released, which supports reproducibility. However, the result is not yet firmly established because of underspecified rotation details, missing statistical reporting, and confounded model grouping. The paper is a useful benchmark but currently falls short of a definitive causal conclusion.","major_comments":[{"comment":"The rotation implementation is not specified: the paper does not state the interpolation method, the border-filling policy (e.g., zero-fill, reflection, or crop-to-valid), or whether the rotated patch is resized back to the original dimensions. This is load-bearing because Section 3 attributes the poorest alignment at 45°, 135°, 225°, and 315° to 'slight differences in the patch corners introduced by rotation,' implying that the rotated inputs contain non-tissue boundary content. Without specifying and controlling the border handling, the central t-tests may partly measure how models respond to padding artifacts rather than to rotation of the tissue content itself. Please describe the exact rotation pipeline and include a robustness check with at least two border-filling strategies (e.g., zero-padding vs. reflected padding) to show that the group comparison is not an artifact of the chosen implementation.","section":"Section 2.3"},{"comment":"The t-test statistics are reported as t = −8.88 and t = 6.91 with p < 0.0001, but the degrees of freedom and the unit of analysis are not given. If the tests are computed by pooling all patch-level measurements across the twelve models, the effective sample size is the number of models (12), not the number of patches, and the independence assumption is violated because patches from the same model are correlated. Please report the unit of analysis and either perform a model-level analysis (e.g., comparing the six augmented vs. six non-augmented models) or use a mixed-effects model with the model as a random effect. The current presentation does not allow the reader to assess whether the significance claim is statistically valid.","section":"Section 3"},{"comment":"Figure 2 shows only mean values without any measure of dispersion. Because invariance scores vary across patches and WSIs, the reader cannot determine whether the group differences are large relative to the within-group variance. Add error bars or confidence intervals, and report the standard deviation or interquartile range for each model. This is important for interpreting the practical magnitude of the reported effects.","section":"Section 3, Figure 2"},{"comment":"The conclusion states that 'rotation augmentation is necessary to achieve learned invariance,' which is a causal claim. The evidence, however, is associative: the two groups of models differ not only in rotation augmentation but also in architecture details, pretraining data, objective functions, and training scale. A controlled comparison (e.g., the same architecture trained with and without rotation augmentation, or matching on other training factors) is required to attribute the observed differences specifically to rotation augmentation. The conclusion should be phrased as an association or hypothesis unless such a control is provided.","section":"Conclusion"}],"minor_comments":[{"comment":"The phrase 'rotations from 0° to 360° at 15° intervals' is ambiguous about whether 360° is included; clarify that 0° and 360° are the same rotation.","section":"Section 2.3"},{"comment":"The total number of patches extracted from the TCGA-KIRC WSIs is not reported; providing N would help the reader assess the statistical precision of the measurements.","section":"Section 2.1"},{"comment":"The text says 'Fig. 3 presents a heatmap of m-kNN and cosine distances,' but Figure 3 has two panels; specify in the caption which panel corresponds to which metric.","section":"Section 3, Figure 3"},{"comment":"The notation N_k(z'_i) is used in the equation but defined only in the preceding text; add a brief definition in or after the equation for readability.","section":"Equation (1)"},{"comment":"The code repository link is provided, but the paper would benefit from stating the software versions used for rotation, feature extraction, and the m-kNN computation.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The paper is within the scope of the journal and addresses a relevant and under-explored topic. The main uncertainties are experimental and statistical rather than conceptual. The underspecified rotation procedure and the pooling issue in the t-test are fixable with additional details and re-analysis. The causal interpretation in the conclusion should be softened or supported by a controlled experiment."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short empirical benchmark that makes one plausible point: pathology FMs trained with rotation augmentation show higher m-kNN and lower cosine distance under patch rotation. I think the measured effect is real on this dataset, but the paper under-specifies the rotation and the statistics enough that the headline should be treated as provisional.\n\nWhat is new: the comparison across twelve pathology foundation models is the first I know of that ties rotation invariance to the presence of rotation augmentation during pretraining. The two metrics are standard, and the rotation-agnostic idea appears in PathDino, but the benchmark itself is a useful addition. The paper is honest about its scope and ships code, which matters.\n\nThe soft spots are in the measurement details. Section 2.3 says rotations were applied but does not give interpolation, resizing, or border-filling. Section 3 then says alignment is poorest at 45, 135, 225, and 315 degrees “likely due to slight differences in the patch corners introduced by rotation.” That is the paper telling you the rotated patches are not clean rotations of the tissue content; they contain boundary artifacts. If augmented models learned to tolerate exactly those artifacts, the t-tests may be comparing artifact handling rather than invariance of tissue features. This concern is load-bearing for the specific claim, though not fatal: robustness to the full practical rotation including corners is still a useful property.\n\nThe statistics also need reporting. The paper gives t = -8.88 and t = 6.91 and p < 0.0001, but no degrees of freedom and no statement about the unit of analysis. With only twelve models, treating individual patches as independent observations would inflate significance; with model-level means, the test is appropriate but the df should be shown. There are no error bars in Fig. 2. The evaluation is on one dataset, TCGA-KIRC, so generalizability is unknown. The transformer inductive-bias hypothesis is reasonable but not tested, since no model architecture is varied while holding augmentation fixed.\n\nWho it is for: anyone building or selecting pathology foundation models and wondering whether rotation augmentation buys robustness. It deserves a serious referee. The fix is straightforward: document the rotation pipeline, report model-level values, and ideally repeat on a second dataset or add a control that removes corner artifacts. I would not desk-reject it.","headline":"Useful 12-model benchmark showing rotation-augmented pathology FMs align better under patch rotation, but the rotation pipeline and t-test details are under-specified enough that the headline should be read as provisional.","tokens_in":4949,"tokens_out":2711,"would_cite":true,"duration_ms":24855,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Pathology foundation models are not inherently rotation-invariant; training with rotation augmentation significantly improves their alignment on TCGA-KIRC.","keywords":["foundation models","digital pathology","rotation invariance","self-supervised learning","latent representations","computational pathology","histopathology","mutual k-nearest neighbours"],"falsifier":"Recompute the same m-kNN and cosine distance after cropping away the border regions created by rotation, or after padding and filling with tissue-like content, and see whether the large gap between rotation-augmented and non-augmented models persists; if it shrinks, the reported invariance difference is partly a corner-artifact effect.","tokens_in":4048,"feed_emoji":"🔄","tokens_out":8619,"duration_ms":67150,"temperature":0.7,"pith_summary":"The paper asks whether the latent representations that pathology foundation models produce are stable when a patch is rotated, and what determines that stability. It compares twelve self-supervised models on patches from TCGA-KIRC, measuring alignment between non-rotated and rotated representations with mutual k-nearest neighbours and cosine distance. The central empirical claim is that models trained with rotation augmentation are significantly more rotation-invariant: the augmented group beats the non-augmented group on both metrics with p < 0.0001. This matters because these encoders are used for tumour detection and grading on whole slide images, where tissue orientation is not controlled, and a representation that shifts with rotation can quietly degrade downstream decisions.","feed_headline":"Pathology AI models only resist rotation if trained on it","feed_subtitle":"Training with rotation augmentation, not model architecture, decides whether representations stay aligned after rotation.","key_machinery":"The central objects are two representational-alignment metrics applied between a patch and its rotated copies: mutual k-nearest neighbours (m-kNN), the fraction of a given patch's k=10 nearest neighbours, measured by Euclidean distance in latent space, that are shared between the non-rotated and rotated representation sets; and cosine distance, the average angular mismatch between a patch's latent vector and its rotated vector. Rotation augmentation during self-supervised training is the explanatory mechanism: the paper contrasts the twelve models by whether their training included rotated inputs. The underlying hypothesis is that the transformer architecture provides no rotational inductive bias, so stable representations must be learned rather than inherited.","core_discovery":"On 256x256 patches from TCGA-KIRC whole slide images, the paper finds that rotational invariance is not a shared property of pathology foundation models. Mean mutual k-nearest neighbour scores across rotation angles range from 0.53 for Virchow to 0.85 for PathDino, and mean cosine distance ranges from 0.016 for Hibou-L to 0.145 for Phikon2. When the twelve models are split by whether rotation augmentation was used during self-supervised training, the two groups separate cleanly: rotation-augmented models have significantly smaller cosine distance (t = -8.88, p < 0.0001) and significantly higher m-kNN (t = 6.91, p < 0.0001) than models trained without it. Alignment is poorest at 45, 135, 225 and 315 degrees, which the paper attributes to slight differences in the patch corners introduced by rotation. The authors propose that because transformers lack a rotational inductive bias, rotation augmentation is needed for the encoder to learn invariance.","pith_inferences":["The paper does not test whether the invariance gap survives when rotated patches are cropped to remove border effects; if the diagonal-angle dip is mostly corners, the ranking between augmented and non-augmented groups could shift under a border-free protocol.","Because the benchmark uses one dataset (TCGA-KIRC) and only five tissue contours per slide, the clean separation between groups should be replicated across cancer types and magnifications before it is treated as a general law of pathology foundation models.","The same alignment metrics could be run on other transformations, such as flips, small translations, or stain perturbations, to produce a cheap multi-axis robustness profile before clinical deployment.","The results suggest a practical selection rule: reject a pathology foundation model if its rotated-patch alignment is poor, even if its benchmark accuracy looks good."],"forward_implications":["Rotation augmentation should be treated as a necessary component of self-supervised pretraining for pathology foundation models, since the transformer architecture on its own does not confer rotational invariance.","The two-metric protocol (m-kNN at k=10 plus cosine distance across 15-degree rotations) gives practitioners a label-free way to check a model's orientation robustness before deploying it.","On this benchmark, rotation-augmented models such as PathDino and Hibou-L are the safer defaults for downstream tasks where slide orientation is uncontrolled.","Even the best models lose alignment at diagonal angles, so no evaluated model is fully rotation-invariant; residual corner artifacts remain a known source of variation."],"supporting_citations":[{"why":"supplies TCGA-KIRC, the whole-slide image collection from which all benchmark patches are drawn.","marker":"Akin et al., 2016"},{"why":"provides the survey of representational similarity measures from which the m-kNN and cosine distance choice is drawn.","marker":"Klabunde et al., 2024"},{"why":"provides the representational alignment perspective that motivates comparing latent directions under rotation.","marker":"Huh et al., 2024"},{"why":"introduces PathDino and its rotation-agnostic pretraining, supplying the most rotation-invariant model by m-kNN and a direct example of rotation augmentation.","marker":"Alfasly et al., 2024"},{"why":"introduces Hibou base and large; Hibou-L achieves the lowest mean cosine distance in the benchmark.","marker":"Nechaev et al., 2024"},{"why":"introduces Phikon, a non-rotation-augmented model in the comparison group.","marker":"Filiot et al., 2023"},{"why":"introduces Phikon2, the model with the highest mean cosine distance among the twelve, anchoring the non-augmented group.","marker":"Filiot et al., 2024"},{"why":"introduces Virchow, the model with the lowest mean m-kNN, anchoring the non-augmented group.","marker":"Vorontsov et al., 2024"}],"fun_headline_variants":["Pathology AI models: rotation invariance is learned, not inherent","Study: only rotation-augmented pathology models stay rotation-stable","Pathology foundation models don't resist rotation unless trained to","Rotation invariance in pathology AI is learned via augmentation"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The rankings depend on rotated patches being genuine rotations of the same tissue, but the paper does not say how border pixels were handled, and it attributes the worst scores at diagonal angles to corner differences, so empty-corner artifacts could be doing some of the work.","fun_headline_variants_meta":{"raw":{"variants":["Pathology AI models: rotation invariance is learned, not inherent","Study: only rotation-augmented pathology models stay rotation-stable","Pathology foundation models don't resist rotation unless trained to","Rotation invariance in pathology AI is learned via augmentation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000367,"raw_usage":{"total_tokens":1929,"prompt_tokens":861,"completion_tokens":1068,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":477,"completion_tokens_details":{"reasoning_tokens":1000}},"tokens_in":477,"tokens_out":1068,"duration_ms":9442,"temperature":1.0,"reasoning_tokens":1000,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T14:26:11.736737+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Recompute the same m-kNN and cosine distance after cropping away the border regions created by rotation, or after padding and filling with tissue-like content, and see whether the large gap between rotation-augmented and non-augmented models persists; if it shrinks, the reported invariance difference is partly a corner-artifact effect.","supporting_citations":[],"review_version":1}