Pith. sign in

REVIEW 2 major objections 5 minor 22 references

Ejection fraction's ratio nature makes a volume concept layer's absolute scale unidentifiable from an EF-only loss; predicted volumes collapse to 0.1 mL spread while EF accuracy is preserved.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Because ejection fraction is a ratio, an EF-only objective leaves the volume concept layer determined only up to rescaling, and the ungrounded layer collapses to near-zero volume spread despite decent EF accuracy.

T0 review reviewed 2026-08-01 challenge →

load-bearing objection A correct but elementary invariance argument, illustrated with a suggestive yet likely confounded empirical demo: weight decay, not the EF objective alone, probably drives the collapse to 0.1 mL. the 2 major comments →

arxiv 2607.25748 v1 pith:3E7U7RSQ submitted 2026-07-28 cs.AI

Loss Invariance Determines What Concept Layers Encode: Volume Grounding in Echocardiography

Hyunkyung Han , Min Jung Kim This is my paper
classification cs.AI
keywords concept bottleneck modelsejection fractionechocardiographyscale invarianceinvariance groupconcept groundinginterpretabilitymedical image analysis
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that judging a concept bottleneck by how accurately it predicts its concepts is not enough: the training objective's invariance structure can leave entire properties of the concept layer undetermined. For ejection fraction, which is a ratio of end-systolic and end-diastolic volumes, any common rescaling of the two volumes leaves the prediction unchanged, so an EF-only loss fixes the concept layer only up to scale. Empirically, a video transformer with a volume concept layer trained on EF alone predicts volumes with a spread of 0.1 mL against a reference spread of about 36 and 46 mL, while still giving competitive EF error. The paper proves that no invariance-preserving regularizer can recover the scale, and shows that supervising volumes in millilitres restores spread (to about 20 to 22 mL) at a cost of 0.4 in EF error. The broader consequence is that clinical concept bottlenecks should be validated against the symmetries of the loss, not only against concept accuracy.

Core claim

The paper's central claim is that the EF objective, because it is a ratio of two volumes, determines a volume concept layer only up to a one-parameter rescaling; every point in the same scale orbit yields identical loss. Formalized as Proposition 1, any loss invariant under a group acting on the concept space cannot distinguish representations within the same orbit, and Corollary 1 applies this to the scale group. The empirical demonstration is that the EF-only concept layer produces volume predictions with standard deviation 0.1 mL—about 0.3% of the reference spread—while achieving EF MAE of 6.89, and that grounding in millilitres breaks the invariance, reducing volume MAE from 89.8 to 25.8

What carries the argument

The central object is the scale group G = {c -> lambda c, lambda > 0} acting on the concept space of (ESV, EDV). The ejection fraction identity EF = 100 * (1 - ESV/EDV) is invariant under this group, so the training loss L_EF cannot tell apart volumes in millilitres, litres, or arbitrary units. Proposition 1 generalizes this: any loss invariant under a group G determines the concept layer only up to a G-orbit. The work this machinery does is to predict both the collapse of volume scale under an EF-only objective and the impossibility of recovering scale with a G-invariant regularizer, while pointing to supervision in physical units as the only way to break the orbit.

Load-bearing premise

The load-bearing premise is that the observed collapse to a 0.1 mL spread is the typical behavior of the invariant objective under this architecture, whereas the formal theorem only guarantees indeterminacy; the empirical demonstration relies on a single fine-tuning checkpoint and no reported seed-to-seed variability for the key quantity.

What would settle it

Train the EF-only concept bottleneck from ten random seeds or checkpoints and measure the standard deviation of predicted volumes on the test set; if a majority of runs retain a spread above, say, 20 mL, the collapse is an optimization artifact rather than the generic consequence of the scale-invariant objective, though the formal indeterminacy claim would remain.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • A concept bottleneck can be complete for the prediction task (the two volumes determine EF exactly) and still be physically meaningless; completeness is not the same as grounding.
  • Concept accuracy evaluated in normalized units cannot detect the scale loss; the authors' own ratio-regularized variant was a failure invisible to such evaluation.
  • Any regularizer that is itself invariant under the scale group, such as penalizing the volume ratio, is mathematically equivalent to the EF objective and cannot restore physical units.
  • Grounding volumes in absolute units cuts concept error by roughly two-thirds at a small cost in EF accuracy (0.4 MAE), but it does not fully restore the reference spread, leaving residual compression as an unexplained phenomenon.
  • The invariance argument transfers to any concept vocabulary defined up to scale, such as shape and margin descriptors, where the same hidden degeneracy is expected under objectives that do not reference absolute size.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • This result suggests a simple diagnostic for any clinical concept bottleneck: compare the standard deviation of predicted concepts to the reference distribution; if the predicted spread is orders of magnitude smaller, the concept layer has lost its physical scale even if concept error is low.
  • The same scale indeterminacy should affect other ratio-based cardiac metrics (e.g., fractional shortening or regurgitant fraction) whose concept layers are lengths or volumes, predicting similar collapses unless trained with absolute-unit supervision.
  • One counterintuitive consequence not developed in the paper: because the theory predicts indeterminacy rather than collapse, the specific direction of the collapse (toward small positive values) is an artifact of the softplus output head; a different positivity constraint might produce a different but equally scale-free behavior.
  • The framing invites a more general audit: for any task, enumerating the symmetry group of the target (rescaling, translation, rotation) could reveal hidden degeneracies in concept layers before deployment, making invariance analysis a standard part of concept-bottleneck validation.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper studies concept bottleneck models (CBMs) for ejection fraction (EF) estimation from echocardiographic video, with end-systolic and end-diastolic volumes as the intermediate concepts. The authors give a formal invariance argument: because EF is a ratio, any loss built on EF alone is invariant under a common rescaling of the two volumes, so the training objective can determine the concept layer only up to scale (Proposition 1, Corollary 1). They then report a CBM with a softplus volume head and no residual path, trained from a common checkpoint under an EF-only objective, a volume-grounded objective, and a normalized variant. The EF-only model attains EF error comparable to direct regression (6.891 vs. 7.129 MAE), but its predicted volume standard deviation collapses to 0.1 mL against reference SDs of 35.7 and 45.7 mL, while correlations are partly preserved. Grounding in millilitres restores spread at a small EF-accuracy cost. The paper concludes that concept accuracy alone can conceal a concept layer that carries no physical scale, and that invariance-breaking supervision is the appropriate remedy.

Significance. The point is important and timely: if a concept bottleneck passes standard accuracy checks while its concepts have lost their physical units, the interpretability contract is broken. The elementary invariance argument is clean and clearly positioned against prior work on concept leakage, completeness, and spatial grounding. The empirical design is generally careful—all conditions start from a shared checkpoint, the temporal max/min assumption is verified, bootstrap confidence intervals are reported for EF MAE, and the dataset is public. The authors also report their own failed regularizer and explicitly list limitations. The main weaknesses are that the headline collapse experiment is confounded by the optimizer regularizer and that seed-to-seed variability is not reported for the key quantity; these are fixable and do not invalidate the underlying corollary.

major comments (2)
  1. [IV-B / II-B / Eq. (2)] The central empirical demonstration—predicted volume SD collapsing to 0.1 mL in the EF-only condition—is presented as a consequence of Proposition 1, but the objective actually optimized is not G-invariant. Section II-B specifies AdamW with weight decay 0.05, while the loss in Eq. (2) omits the regularizer. Since L_EF is flat along the common rescaling direction, the weight-decay term is the only gradient component along that direction and, combined with the softplus head (Section II-C), drives the scale toward small positive values. Proposition 1 predicts indeterminacy, not collapse to zero. The observed collapse is therefore selected by an optimization artifact, not 'determined in advance' by the invariance. Provide control runs with zero or varied weight decay (and ideally varied initialization scales) to demonstrate the orbit degeneracy directly, or revise the causal claim to state t
  2. [Table II / V-C] The key quantity of interest, predicted SD of 0.1 mL, is reported for a single run. The paper acknowledges 2–3 seeds per configuration in Section V-C, but no seed-to-seed variability is given for the collapse, and all conditions start from one epoch-43 checkpoint. If another seed retained scale near the initialization, the empirical narrative would be substantially weakened. Add per-seed predicted SDs (or bootstrap intervals) for at least the EF-only and grounded conditions, and state explicitly how many seeds produced each number in Table II.
minor comments (5)
  1. [III / Eq. (1)-(3)] Eq. (3) mixes percentage and fractional representations. If EF is normally expressed as a percentage in Eq. (1), then '1 − EF' is dimensionally inconsistent unless EF is explicitly converted to a fraction. Please state the convention once and keep it throughout the derivation.
  2. [Table II] The methods say bootstrap 95% CIs are reported, but Table II has no intervals. Add CIs or seed ranges at least for the concept MAE and predicted SD columns, since those support the main claim.
  3. [II-B / II-C] It is not fully clear whether the same optimizer settings (AdamW, weight decay 0.05) were used for the concept-bottleneck training as for the initial EF fine-tuning. Specify the optimizer, weight decay, and learning-rate schedule used for all CBM conditions.
  4. [Fig. 1] Because the top row spans 0.6–2.0 mL while the bottom row spans up to 100 mL, shared axis limits make the top-row structure nearly invisible. Consider insets, separate limits, or annotated zoom panels.
  5. [V-B] The pixel-spacing hypothesis for residual compression is interesting but the measurement set is not described. Please state the number of studies, how calibration was recorded, and how the 32%/15% decomposition was computed.

Circularity Check

0 steps flagged

No circularity: the invariance result is an algebraic consequence of EF being a ratio, and the empirical volume collapse is an observed outcome rather than a fitted prediction.

full rationale

The paper's central claim is self-contained and not circular. Corollary 1 follows directly from the definition of EF as a ratio and from the training loss depending on EF alone: if both volumes are rescaled by a common positive factor, EF is unchanged, so the loss cannot distinguish members of the scale orbit. This is an elementary algebraic fact, which the paper explicitly acknowledges ("The proposition itself is elementary..."), and it is not obtained by fitting a parameter and then renaming the fit as a prediction. The empirical collapse to 0.1 mL predicted-volume SD is an observed experimental outcome, not a derived consequence of a fitted constant; the paper does not use that observation to infer the invariance. The only equation in the paper that reduces to an input is Eq. (3), where the regularizer |rho_hat - rho| is shown to equal |cEF_hat - EF|. The authors themselves flag this as a degenerate design ("The regularizer was therefore not merely similar to L_EF but equal to it"), so it is an explicit cautionary example, not a hidden circular step in the derivation. The self-citation to [18] supplies the encoder architecture and not the invariance argument, so it is not load-bearing for the central claim; the formal result would hold for any encoder. Potential concerns about the collapse direction being driven by weight decay or softplus are correctness/confounding risks, not circularity, and the paper's limitations section candidly reports residual unexplained compression and seed variability. No specific reduction of a prediction to an input or to a self-citation chain exists, so the appropriate score is 0.

Axiom & Free-Parameter Ledger

3 free parameters · 6 axioms · 0 invented entities

No new entities are postulated. The load-bearing inputs are the public dataset annotations, the standard group-invariance argument, and several domain assumptions about temporal phase alignment, preprocessing, and the softplus head. The main fitted/choice parameters are the grounding weight, the reduced learning rate, and the single shared checkpoint.

free parameters (3)
  • grounding weight µ = 3 and 10
    Chosen by hand to balance the EF loss and the volume loss; it is central to the reported 0.4 EF MAE cost and to the claim that a larger grounding weight buys no additional concept validity.
  • grounded learning rate 5×10^-5 = 5×10^-5
    Introduced ad hoc after two of three seeds diverged at 10^-4; needed for the grounded-condition results reported in Tables I and II.
  • fine-tuning checkpoint (epoch 43) = epoch 43
    Selected by validation loss from one 50-epoch run and used to initialize every condition, so all experimental comparisons inherit this single checkpoint.
axioms (6)
  • standard math A loss that is invariant under a group action cannot distinguish members of the same orbit (Proposition 1).
    Elementary group-invariance fact invoked in Section III to draw the corollary about ejection fraction and scale.
  • domain assumption EchoNet-Dynamic reference volumes (43.7±35.7 mL ESV, 91.6±45.7 mL EDV) are treated as ground truth for supervision and evaluation.
    Section II-A; all grounding and evaluation numbers depend on these annotations being correct.
  • domain assumption The temporal max/min of the per-token trace correspond to end-diastolic and end-systolic phases.
    Section II-E; the paper verifies this empirically (97.5%/99%/97.7%) but it remains a definitional assumption for what the concept layer outputs.
  • domain assumption Preprocessing resizes all inputs to a fixed grid, removing physical pixel spacing.
    Section V-B; this is the paper's hypothesis for why even grounded predictions remain compressed relative to reference spread.
  • domain assumption The softplus head constrains ungrounded predictions toward small positive values rather than arbitrary constants.
    Section II-C; this architectural choice is what makes the scale collapse appear as near-zero spread instead of an arbitrary constant.
  • domain assumption All conditions initialized from the same epoch-43 checkpoint, so differences are attributed to the objective rather than to initialization.
    Section II-B; the causal comparison between EF-only and grounded training relies on this assumption.

reviewed 2026-08-01 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Loss Invariance Determines What Concept Layers Encode: Volume Grounding in Echocardiography." pith.science (2026). https://pith.science/paper/3E7U7RSQ

@misc{pith2026260725748,
  author       = {Pith},
  title        = {Pith review of: Loss Invariance Determines What Concept Layers Encode: Volume Grounding in Echocardiography},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3E7U7RSQ}},
  note         = {Machine review of arXiv:2607.25748}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Objective: Concept bottleneck models route prediction through interpretable intermediate variables, and their validity is normally judged by how accurately those variables are predicted. We ask whether that judgement is sufficient, using left ventricular volumes as the concepts underlying ejection fraction estimation from echocardiographic video. Methods: A video transformer encoder was trained on a publicly available echocardiography dataset. End-systolic and end-diastolic volumes formed a concept layer from which ejection fraction was computed analytically, with no residual path to the output. We compared training under an ejection fraction objective alone against training with additional supervision of the volumes in millilitres, and evaluated both on 1276 held-out studies. Results: The concept bottleneck did not increase ejection fraction error relative to direct regression, at 6.89 against 7.13 mean absolute error. Without volume supervision, however, the spread of predicted volumes collapsed to 0.1 millilitres against reference spreads of 35.7 and 45.7 millilitres, while correlation was partly preserved. We show that this follows from an invariance property of the objective: ejection fraction is a ratio and is unchanged when both volumes are rescaled, so the loss determines the concept layer only up to scale. Supervision in absolute units reduced volume error from 89.8 to 25.8 millilitres at a cost of 0.4 in ejection fraction error. Conclusion: Concept accuracy alone can conceal a concept layer that carries no physical scale. Significance: Interpretable intermediate variables in clinical models should be validated against the invariance structure of the training objective, not only against prediction accuracy.

Figures

Figures reproduced from arXiv: 2607.25748 by Hyunkyung Han, Min Jung Kim.

Figure 1
Figure 1. Figure 1: Predicted against reference volumes on the test set (n = 1,276). Top: without volume supervision the concept layer produces almost constant volumes (predicted SD 0.1 mL against reference SD 35.7 and 45.7 mL; across all 1,276 studies the end-systolic predictions span 0.6– 1.4 mL and the end-diastolic predictions 1.6–2.0 mL), collapsing onto a narrow band along the horizontal axis; the softplus output constr… view at source ↗
Figure 2
Figure 2. Figure 2: Validation error during volume-grounded training. At a learning rate of 10−4 two of three seeds diverged and were terminated; halving the learning rate removed the instability without changing the final error materially. at 13.81 at epoch 0. Both were terminated once divergence was evident, at epochs 13 and 1 respectively; the surviving seed completed 100 epochs at a final validation MAE of 7.22. Halving t… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

22 extracted references · 4 linked inside Pith

  1. [1]

    We show empirically that a volume concept layer trained under an EF objective retains part of the ordering among studies while losing physical scale almost entirely, with predicted volume spread collapsing to well under one percent of the reference spread

  2. [2]

    We report a design of our own that fell into exactly this failure and was not detectable from concept accuracy

    We formalize the cause as an invariance property of the objective (Proposition 1), which predicts that no invariance-preserving regularizer can recover scale. We report a design of our own that fell into exactly this failure and was not detectable from concept accuracy

  3. [3]

    We verify the prescription that follows—supervision in absolute units—and quantify its cost, which is an increase of 0.4 in EF mean absolute error, together with an optimization instability it introduces and a practical remedy. The encoder used here was introduced in our earlier study of attribution faithfulness in echocardiographic video [18]; the resear...

  4. [4]

    A degenerate concept loss: An earlier version of this design regularized the concept layer toward the correct volume ratio rather than the correct volumes. Writing ˆρ = ˆVES/ ˆVED for the predicted ratio, that term penalized |ˆρ − ρ| with HAN AND KIM: LOSS INVARIANCE DETERMINES WHAT CONCEPT LAYERS ENCODE 3 the target ρ = 1 − EF, which is the identity obta...

  5. [5]

    Concept bottleneck models,

    P. W. Koh, T. Nguyen, Y . S. Tang, S. Mussmann, E. Pierson, B. Kim, and P. Liang, “Concept bottleneck models,” in Proc. 37th Int. Conf. Mach. Learn. (ICML) , PMLR, vol. 119, 2020, pp. 5338–5348

    2020

  6. [6]

    Video-based AI for beat-to-beat assessment of cardiac function,

    D. Ouyang, B. He, A. Ghorbani, N. Yuan, J. Ebinger, C. P. Langlotz, P. A. Heidenreich, R. A. Harrington, D. H. Liang, E. A. Ashley, and J. Y . Zou, “Video-based AI for beat-to-beat assessment of cardiac function,” Nature, vol. 580, no. 7802, pp. 252–256, Apr. 2020, doi: 10.1038/s41586-020-2145-8

    doi:10.1038/s41586-020-2145-8 2020

  7. [7]

    Promises and pitfalls of black-box concept learning models,

    A. Mahinpei, J. Clark, I. Lage, F. Doshi-Velez, and W. Pan, “Promises and pitfalls of black-box concept learning models,” arXiv:2106.13314, 2021

    Pith/arXiv arXiv 2021

  8. [8]

    Do concept bottleneck models learn as intended?

    A. Margeloiu, M. Ashman, U. Bhatt, Y . Chen, M. Jamnik, and A. Weller, “Do concept bottleneck models learn as intended?” in ICLR Workshop on Responsible AI , 2021, arXiv:2105.04289

    Pith/arXiv arXiv 2021

  9. [9]

    Addressing leakage in concept bottleneck models,

    M. Havasi, S. Parbhoo, and F. Doshi-Velez, “Addressing leakage in concept bottleneck models,” in Adv. Neural Inf. Process. Syst. (NeurIPS), vol. 35, 2022, pp. 23386–23397

    2022

  10. [10]

    Post-hoc concept bottle- neck models,

    M. Yuksekgonul, M. Wang, and J. Zou, “Post-hoc concept bottle- neck models,” in Proc. Int. Conf. Learn. Represent. (ICLR) , 2023, arXiv:2205.15480

    Pith/arXiv arXiv 2023

  11. [11]

    Challenging common assumptions in the unsupervised learning of disentangled representations,

    F. Locatello, S. Bauer, M. Lucic, G. R ¨atsch, S. Gelly, B. Sch ¨olkopf, and O. Bachem, “Challenging common assumptions in the unsupervised learning of disentangled representations,” in Proc. 36th Int. Conf. Mach. Learn. (ICML) , PMLR, vol. 97, 2019, pp. 4114–4124

    2019

  12. [12]

    Incremental residual concept bottleneck models,

    C. Shang, S. Zhou, H. Zhang, X. Ni, Y . Yang, and Y . Wang, “Incremental residual concept bottleneck models,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR) , 2024, pp. 11030–11040

    2024

  13. [13]

    On the concept trustworthiness in concept bottleneck models,

    Q. Huang, J. Song, J. Hu, H. Zhang, Y . Wang, and M. Song, “On the concept trustworthiness in concept bottleneck models,” in Proc. AAAI Conf. Artif. Intell. , vol. 38, no. 19, 2024, pp. 21161–21168

    2024

  14. [14]

    What’s in the bottle? A survey and roadmap of concept bottleneck models,

    P. Knab, D. Steinmann, C. Bartelt, K. Kersting, B. Schiele, T. Seidl, U. Schlegel, and W. Stammer, “What’s in the bottle? A survey and roadmap of concept bottleneck models,” Trans. Mach. Learn. Res. , 2026

    2026

  15. [15]

    Learning a clinically-relevant concept bottleneck for lesion detection in breast ultrasound,

    A. Bunnell, Y . Glaser, D. Valdez, T. Wolfgruber, A. Altamirano, C. Zamora Gonz ´alez, B. Y . Hernandez, P. Sadowski, and J. A. Shepherd, “Learning a clinically-relevant concept bottleneck for lesion detection in breast ultrasound,” in Proc. Med. Image Comput. Comput. Assist. Interv. (MICCAI), LNCS vol. 15003, 2024

    2024

  16. [16]

    Spatially grounded concept bottleneck models for trustworthy breast ultrasound diagnosis,

    M. R. Tonmoy, D. Che, H. Y . Adarbah, and A. Noore, “Spatially grounded concept bottleneck models for trustworthy breast ultrasound diagnosis,” arXiv:2607.20691, 2026

    Pith/arXiv arXiv 2026

  17. [17]

    Symbol grounding in neuro-symbolic AI: A gentle introduction to reasoning shortcuts,

    E. Marconato et al., “Symbol grounding in neuro-symbolic AI: A gentle introduction to reasoning shortcuts,” arXiv:2510.14538, 2025

    arXiv 2025

  18. [18]

    Shortcuts and identifiability in concept-based models from a neuro- symbolic lens,

    S. Bortolotti, E. Marconato, P. Morettin, A. Passerini, and S. Teso, “Shortcuts and identifiability in concept-based models from a neuro- symbolic lens,” in Proc. 39th Conf. Neural Inf. Process. Syst. (NeurIPS) , 2025, arXiv:2502.11245

    arXiv 2025

  19. [19]

    Towards robust metrics for concept represen- tation evaluation,

    M. Espinosa Zarlenga, P. Barbiero, Z. Shams, D. Kazhdan, U. Bhatt, A. Weller, and M. Jamnik, “Towards robust metrics for concept represen- tation evaluation,” in Proc. AAAI Conf. Artif. Intell. , vol. 37, 2023, pp. 11791–11799

    2023

  20. [20]

    There was never a bottleneck in concept bottleneck models,

    A. Almud ´evar, J. M. Hern ´andez-Lobato, and A. Ortega, “There was never a bottleneck in concept bottleneck models,” arXiv:2506.04877, 2025

    arXiv 2025

  21. [21]

    A review of methods for trustworthy AI in medical imaging: The FUTURE-AI guidelines,

    H. Kondylakis et al., “A review of methods for trustworthy AI in medical imaging: The FUTURE-AI guidelines,” IEEE J. Biomed. Health Inform. , vol. 30, no. 3, pp. 2299–2315, 2026

    2026

  22. [22]

    Anatomically faithful but temporally diffuse: Auditing attribution for left-ventricular ejection-fraction estimation from echocardiography,

    H. Han and M. J. Kim, “Anatomically faithful but temporally diffuse: Auditing attribution for left-ventricular ejection-fraction estimation from echocardiography,” 2026, under review

    2026

This paper was first reviewed by deepseek-v4-flash on August 1, 2026.