REVIEW 3 major objections 5 minor 25 references
SATAM derives cryptographic bills of materials from architecture analysis instead of tooling, preserving the security rationale behind every crypto choice.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-02 17:34 UTC pith:VLQU5476
load-bearing objection A plausible and well-structured method proposal for architecture-derived CBOMs, but the evidence is limited to a self-authored toy example and the key transformation steps are underspecified. the 3 major comments →
Architecture-Derived CBOMs for Cryptographic Migration: A Security-Aware Architecture Tradeoff Method
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The central claim is that cryptographic transparency should be treated as an architectural concern, not an inventory problem. SATAM extends the ATAM evaluation process with explicit security steps: STRIDE threats are refined into Security Quality Attribute Scenarios, cryptographic choices are recorded as Architecture Decision Records, and CARAF converts these artifacts into migration-relevant risk notes. The conceptual traceability model then consolidates these relationships into CBOM entries, each annotated with architectural context, security intent, threat references, decision rationale, and risk findings. The resulting CBOM is positioned as an artifact of the design process: it explains
What carries the argument
SATAM itself is the central mechanism: a scenario-based architecture evaluation method with four minimal requirements (a security-readable architecture baseline, explicit cryptographic assets and protected flows, a traceability chain linking threats and scenarios to decisions and assets, and a machine-readable CBOM that preserves these links). The specific techniques—arc42 for documentation, STRIDE for threat modeling, ADRs for decisions, CARAF for risk assessment—are modular and replaceable; what SATAM contributes is the consolidation of architecture-evaluation outputs into a traceability-preserving CBOM.
Load-bearing premise
The method assumes practitioners can systematically and reliably transform architectural threats (STRIDE results) into Security Quality Attribute Scenarios, Architecture Decision Records, and CBOM annotations; the paper provides no step-by-step procedure for that transformation, and the single proof-of-concept is handcrafted by the authors, so independent reproducibility is unverified.
What would settle it
Let several independent security architects apply SATAM to the same non-trivial architecture and compare the produced CBOMs; if they diverge substantially in which assets, threats, and decisions are recorded, the method's reliability is refuted.
If this is right
- A SATAM-derived CBOM lets migration teams see immediately which data flows, threats, and decisions are affected when a crypto asset is replaced.
- Replacing an algorithm like RSA-2048 with post-quantum signatures no longer requires manual reconstruction of architectural scope.
- Prioritizing upgrades under budget constraints can use traceability to security scenarios and CARAF-derived risk estimates.
- The CBOM remains consistent across human-readable and machine-readable representations because both reference the same identifiers, and it can feed CycloneDX-compatible tooling.
Where Pith is reading between the lines
- A natural next step would be a controlled consistency study: give the same architecture to several analysts, apply SATAM, and measure agreement on assets, threats, and ADRs; the transformation from STRIDE to Security QAS is currently underspecified, so inter-analyst reliability is an open question.
- The cost structure of CBOM generation shifts from automated scanning to structured architecture review; this trade may pay off for high-assurance systems but would need empirical cost-benefit data.
- With automation of ADR-to-CBOM annotation, SATAM could be embedded in continuous delivery pipelines, turning CBOMs into living artifacts that track crypto-related technical debt over time.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces SATAM, a method-level artifact that adapts ATAM-style scenario-based architecture evaluation to derive CBOMs enriched with architectural context, security intent, and migration-relevant metadata. SATAM integrates arc42, STRIDE, Security QAS, ADRs, and CARAF, and defines a traceability model linking cryptographic assets to architectural elements, threats, quality scenarios, decisions, and risk findings. The method is demonstrated on a compact example architecture, producing CBOM tables and a CycloneDX-compatible JSON excerpt. The evaluation is analytical and illustrative; the authors explicitly state that industrial empirical validation is future work.
Significance. If the method works as claimed, it addresses a real and timely gap: inventory-derived CBOMs lack architectural rationale and threat context, which hampers cryptographic migration planning. The integration of existing techniques is sensible and the traceability model is a useful conceptual contribution. The proof-of-concept shows end-to-end feasibility on a small example, and the CycloneDX-compatible annotation via component properties is a pragmatic way to extend existing tooling. However, the central claim that SATAM 'enables' practitioners to construct such CBOMs is not yet supported by the evidence: the transformation steps are only illustrated via a single handcrafted example, with no demonstrated reliability across independent analysts. The paper is honest about this limitation, but the abstract and conclusion overstate what the results show.
major comments (3)
- [§5 and §6.2–6.5] The transformation from STRIDE threats to Security QAS, and then to ADRs and CBOM annotations, is described only through the example. Section 5 defines traceability relationships but not how they are generated. Section 6.2 says threats are 'converted' into QAS with no conversion rules; Section 6.4 says decisions are 'captured' as ADRs without a decision procedure. Since the central claim is that SATAM 'enables' construction by practitioners, the method must specify these steps in a repeatable way, or at least demonstrate inter-analyst consistency. As written, the proof-of-concept shows only internal coherence of the authors' own reasoning.
- [Abstract and §7.2 / §8] The abstract says 'The results demonstrate that architecture-derived CBOMs capture migration-relevant context...' and the conclusion says 'The results indicate that SATAM enables...'. But the only results are an illustrative proof-of-concept with a handcrafted example and a hypothetical task-based comparison (Table 7). The paper itself acknowledges (Sections 3.2 and 8) that empirical evaluation is out of scope and that scalability/cost-benefit remain unvalidated. The wording should be softened to 'illustrate' or 'suggest', and the limitations should be made prominent in the abstract and conclusion to avoid overclaiming.
- [§6.4, Table 4] The CARAF example includes a specific risk figure ('Risk 600,000e') but no derivation is shown. The text says it is illustrative, but the notation 'Risk=T r×Cost' and the use of a concrete number could mislead readers into treating it as a calibrated estimate. Either remove the numeric example or provide a fully transparent calculation with explicit assumptions about the cost model and parameters.
minor comments (5)
- [§4, Table 1] The table caption says 'authoritative process overview'; consider rewording to avoid implying a formal standard. Also, the table mixes activities and artifacts; separating them would improve readability.
- [§6 and Appendix A.1] The JSON listing has unusual line breaks and quote styles. More importantly, the component 'type' is set to 'cryptographic-asset', which may not be a valid CycloneDX component type; verify against the CycloneDX CBOM specification and cite the exact extension mechanism.
- [§6.2, Figure 2] The figure labels (e.g., 'S,T', 'T,I,E') are small and may be hard to read in print. Enlarge or add a table listing the threats per flow.
- [Throughout] The paper alternates between 'Security QAS' and 'QAS'; define the abbreviation at first use and use consistently. Also, 'This results into' in Section 8 should be 'This results in'.
- [§7.3, Table 7] The task-based comparison is labeled as illustrative, which is good. Consider adding a sentence explicitly stating that the entries are the authors' qualitative reasoning, not measurements, and that equal analyst skill is assumed.
Circularity Check
No significant circularity: SATAM is an integrative method with no fitted parameters or predictions, and its proof-of-concept is explicitly acknowledged as illustrative rather than empirical validation.
full rationale
The paper makes no quantitative predictions and fits no parameters, so the classic circularity patterns (fitted input called prediction, self-definitional reduction, equation X = equation Y by construction) do not occur. The derivation chain is methodological: SATAM adapts ATAM, applies STRIDE to produce Security QAS, documents decisions as ADRs, uses CARAF as a risk lens, and consolidates these artifacts into an annotated CBOM. Each component rests on established, externally citable work rather than on the paper's own outputs. The traceability model in Section 5 defines relationships but does not smuggle the CBOM back into the method's inputs. The proof-of-concept is handcrafted by the authors, but the paper itself states that 'An empirical evaluation in an industrial setting is outside the scope of this paper' (Section 3.2) and that 'the practical effort, scalability, and cost-benefit characteristics of SATAM in real-world environments remain to be validated' (Section 8). This is a reproducibility and external-validity limitation, not a circular reduction: the authors do not claim the illustrative CBOM independently proves the method's effectiveness. The central claim—that SATAM enables construction of architecture-grounded, context-sensitive CBOMs—is supported by the worked example and traceability definitions, but it is not derived from the CBOM itself. No load-bearing self-citation or imported uniqueness theorem is present. Thus, under the stated hard rules, there is no significant circularity.
Axiom & Free-Parameter Ledger
axioms (4)
- domain assumption Scenario-based architecture evaluation (ATAM) provides a suitable foundation for security-oriented extensions that require architectural context and traceability.
- domain assumption STRIDE threats can be systematically transformed into Security QAS with measurable response measures.
- domain assumption CARAF provides a valid and useful framework for assessing cryptographic agility risk.
- domain assumption CycloneDX CBOM format can represent SATAM annotations without loss of traceability.
read the original abstract
Cryptographic migration driven by algorithm deprecation, regulatory change, and post-quantum readiness requires more than an inventory of cryptographic assets. Existing Cryptographic Bills of Materials (CBOMs) are typically tool- or inventory-derived. They lack architectural intent, rationale, and security context, limiting their usefulness for migration planning. This paper introduces Security-Aware Architecture Tradeoff Analysis Method (SATAM), a security-aware adaptation of scenario-based architecture evaluation that derives an architecture-grounded, context-sensitive CBOM. SATAM integrates established approaches: ATAM, arc42, STRIDE, ADR, and CARAF. These are included to identify and analyze security-relevant cryptographic decision points and document them as explicit architectural decisions. These artifacts are used to annotate CBOM entries with architectural context, security intent, and migration-critical metadata using CycloneDX-compatible extensions. Following a Design Science Research approach, the paper presents the method design, a conceptual traceability model, and an illustrative application. The results demonstrate that architecture-derived CBOMs capture migration-relevant context that is typically absent from inventory-based approaches. Thereby, SATAM improves availability of information required for informed cryptographic migration planning and long-term cryptographic agility.
Figures
Reference graph
Works this paper leans on
-
[1]
Alagic, G., Bros, M., Ciadoux, P., Cooper, D., Dang, Q., Dang, T., Kelsey, J., Lichtinger, J., Liu, Y.K., Miller, C., Moody, D., Peralta, R., Perlner, R., Robinson, A., Silberg, H., Waller, N., Smith-Tone, D.: Status Report on the Fourth Round of the NIST Post-Quantum Cryptography Standardization Process. Tech. Rep. NISTIR 8545, National Institute of Stan...
-
[2]
Applied Sciences 11(8), 3678 (2021)
Algarni, A.M., Thayananthan, V., Malaiya, Y.K.: Quantitative assessment of cy- bersecurity risks for mitigating data breaches in business systems. Applied Sciences 11(8), 3678 (2021). https://doi.org/10.3390/app11083678, defines the mathemat- ical ALE (Annual Loss Expectancy) model used to convert potential breach costs into a yearly financial risk value
-
[3]
Journal of Critical Infrastructure Policy3, 111–135 (2022)
Arora, A., Wright, V., Garman, C.: Strengthening the Security of Operational Technology: Understanding Contemporary Bill of Materials. Journal of Critical Infrastructure Policy3, 111–135 (2022). https://doi.org/10.18278/jcip.3.1.8
-
[4]
Addison- Wesley Professional, 4th edn
Bass, L., Clements, P., Kazman, R.: Software Architecture in Practice. Addison- Wesley Professional, 4th edn. (2021)
2021
-
[5]
Bundesamt für Sicherheit in der Informationstechnik (BSI): Migration to Post Quantum Cryptography. Tech. rep., BSI (May 2021), https://www.bsi.bund.de/ 18 E. Hirsch et al. SharedDocs/Downloads/EN/BSI/Crypto/Migration_to_Post_Quantum_Crypt ography.pdf
2021
-
[6]
Dang, Q., Kelsey, J., McKay, K., Bassham, L., Chang, S.J.: Considerations for Cryptographic Agility and Migration. Tech. Rep. NISTIR 8259, National Institute of Standards and Technology (2019). https://doi.org/10.6028/NIST.IR.8259
-
[7]
Hassouna, A.B.: Atraf-driven imrad methodology: Tradeoff and risk analysis of software architectures across abstraction levels. arXiv (2025). https://doi.org/10 .48550/arxiv.2505.03624
work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2505.03624 2025
-
[8]
Poster, ETSI/IQC QSC 2024 (2024), https://research.ibm.com/pub lications/standardization-of-cryptography-bill-of-materials-in-owasp-cyclonedx
Hess, B., Körtge, N.: Standardization of Cryptography Bill of Materials in OWASP CycloneDX. Poster, ETSI/IQC QSC 2024 (2024), https://research.ibm.com/pub lications/standardization-of-cryptography-bill-of-materials-in-owasp-cyclonedx
2024
-
[9]
MIS Quarterly28(1), 75–105 (2004)
Hevner, A.R., March, S.T., Park, J., Ram, S.: Design Science in Information Sys- tems Research. MIS Quarterly28(1), 75–105 (2004)
2004
-
[10]
IBM Security and Ponemon Institute: Cost of a data breach report 2025. Tech. rep., IBM Corporation (2025), https://www.ibm.com/reports/data-breach, provides the empirical basis for the 158 Euro per-record cost in the financial sector, accounting for detection, notification, and lost business
2025
-
[11]
Journal of Theoretical and Applied Information Technology (2015), https://www.jatit.org/volumes/Vol75No1/2Vol75No1.pdf
Ibnugraha, P.D., et al.: Evaluation of Security in Software Architecture Using Com- bination of ATAM and STRIDE. Journal of Theoretical and Applied Information Technology (2015), https://www.jatit.org/volumes/Vol75No1/2Vol75No1.pdf
2015
-
[12]
Kazman, R., Klein, M., Clements, P.: ATAM: Method for Architecture Evaluation. Tech. Rep. CMU/SEI-2000-TR-004, Software Engineering Institute, Carnegie Mel- lon University (Aug 2000), https://www.sei.cmu.edu/documents/629/2000_005 _001_13706.pdf
2000
-
[13]
Journal of Cybersecurity7(1), tyab013 (2021)
Ma, C., Colon, L., Dera, J., Rashidi, B., Garg, V.: Caraf: Crypto agility risk assessment framework. Journal of Cybersecurity7(1), tyab013 (2021). https: //doi.org/10.1093/cybsec/tyab013, https://doi.org/10.1093/cybsec/tyab013
-
[14]
Näther, C., Herzinger, D., Gazdag, S.L., Steghöfer, J.P., Daum, S., Loeben- berger, D.: Migrating Software Systems Towards Post-Quantum Cryptography – A Systematic Literature Review. IEEE Access pp. 132107–132127 (2024), https: //doi.org/10.1109/ACCESS.2024.3450306
arXiv 2024
-
[15]
National Telecommunications and Information Administration: Framing Software Component Transparency: Establishing a Common Software Bill of Materials (SBOM), Second Edition. Tech. rep., U.S. Department of Commerce (NTIA) (Oct 2021), https://www.ntia.gov/files/ntia/publications/ntia_sbom_framing_2nd _edition_20211021.pdf, defines an SBOM as a nested invent...
2021
-
[16]
Newhouse, W., et al.: NIST SP 1800-38B: Migration to Post-Quantum Cryptogra- phy: Cryptographic Discovery (Preliminary Draft). Tech. rep., NIST NCCoE (Dec 2023), https://www.nccoe.nist.gov/sites/default/files/2023-12/pqc-migration-nis t-sp-1800-38b-preliminary-draft.pdf
2023
-
[17]
Blog post (November 2011), ht tps://www.cognitect.com/blog/2011/11/15/documenting-architecture-decisions, the foundational article for Architecture Decision Records (ADRs)
Nygard, M.: Documenting Architecture Decisions. Blog post (November 2011), ht tps://www.cognitect.com/blog/2011/11/15/documenting-architecture-decisions, the foundational article for Architecture Decision Records (ADRs)
2011
-
[18]
OWASP CycloneDX Project: Authoritative guide to CBOM. Tech. rep., OWASP Foundation (Apr 2024), https://cyclonedx.org/guides/OWASP_CycloneDX-Aut horitative-Guide-to-CBOM-en.pdf, describes CBOM as a structured, comprehen- sive inventory of cryptographic assets and their properties
2024
-
[19]
Web page (2024), https://cyclonedx.org/capabilities/cbom/ Architecture-Derived CBOMs for Cryptographic Migration 19
OWASP CycloneDX Project: Cryptography bill of materials (CBOM). Web page (2024), https://cyclonedx.org/capabilities/cbom/ Architecture-Derived CBOMs for Cryptographic Migration 19
2024
-
[20]
In: Interna- tional Conference on Software Engineering (ICSE) (2024), https://ojcchar.github .io/files/27-icse24-sboms.pdf
Stalnaker, T., et al.: BOMs Away! Inside the Minds of Stakeholders. In: Interna- tional Conference on Software Engineering (ICSE) (2024), https://ojcchar.github .io/files/27-icse24-sboms.pdf
2024
-
[21]
https://arc42.org (2024), accessed: 2026-01-28
Starke, G., Hruschka, P.: arc42: The Template for Software Architecture Docu- mentation and Communication. https://arc42.org (2024), accessed: 2026-01-28
2024
-
[22]
Leanpub, 3rd edn
Starke, G., Simons, M., Zörner, S., Müller, R.D., Lösch, H.: arc42 by Example: Real software architectures, documented with the arc42 template. Leanpub, 3rd edn. (2023), https://leanpub.com/arc42byexample
2023
-
[23]
IEEE Software22(2), 19–27 (2005)
Tyree, J., Akerman, A.: Architecture Decisions: Demystifying Architecture. IEEE Software22(2), 19–27 (2005). https://doi.org/10.1109/MS.2005.27
-
[24]
arXiv (2023), https://arxiv.org/pdf/2301.05362
Xia, B., et al.: An Empirical Study on Software Bill of Materials. arXiv (2023), https://arxiv.org/pdf/2301.05362
Pith/arXiv arXiv 2023
-
[25]
c o m p o n e n t s
Zhou, L., et al.: A Reality Check on SBOM-based Vulnerability Management: An Empirical Study and A Path Forward. arXiv (2025), https://arxiv.org/pdf/2511 .20313 A Appendix This section provides additional material related to the illustrative example in Section 6.5. Listing A.1.Minimal CBOM JSON excerpt of components with SATAM-derived annotations 1" c o m...
2025
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.