pith. sign in

arxiv: 2604.17256 · v1 · submitted 2026-04-19 · 💻 cs.CR · cs.LG

A Unified Compliance Aggregator Framework for Automated Multi-Tool Security Assessment of Linux Systems

Sheldon Paul , Izzat Alsmadi This is my paper

Pith reviewed 2026-05-10 06:25 UTC · model grok-4.3

classification 💻 cs.CR cs.LG
keywords security assessmentcomplianceLinuxaggregationnormalizationfile integrityhardeningmulti-tool
0
0 comments X

The pith

The Unified Compliance Aggregator combines outputs from five security tools into a single 0-100 score for Linux systems.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper proposes the Unified Compliance Aggregator framework to integrate results from multiple open-source security tools into one composite score. The tools include Lynis, OpenSCAP with STIG and CIS profiles, AIDE, Tripwire, and Nmap NSE scripts. Outputs are normalized to a 0-100 scale and then aggregated using weights, with a logarithmic scoring method introduced for file integrity to improve on linear models. Tests across Ubuntu 22.04 systems with varying hardening show the score improves as systems become more secure, though compliance and integrity tools sometimes disagree. The framework offers a way to assess overall security posture without manually reconciling separate tool reports.

Core claim

The paper shows that combining the selected tools through a normalization process to a consistent 0 to 100 scale and weighted aggregation produces a composite security score. This score rises consistently when systems are hardened, as shown in experiments on Ubuntu 22.04 at different levels. A logarithmic model is used for file integrity to address limitations of linear approaches. Two case studies demonstrate application to a basic web server and a DVWA system.

What carries the argument

The normalization to a 0-100 scale and weighted aggregation of outputs from Lynis, OpenSCAP, AIDE, Tripwire, and Nmap, along with the logarithmic scoring for file integrity.

If this is right

  • The composite score tracks improvements from hardening efforts.
  • Compliance and file integrity tools can show contrasting behaviors.
  • The framework works on both standard and vulnerable web application setups.
  • A single numerical value summarizes multi-tool security assessments.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • This could enable continuous automated security monitoring with threshold-based alerts.
  • Weights might be customized for different environments or compliance standards.
  • The method could be tested on other Linux distributions or with additional tools.
  • Logarithmic scaling may prevent minor file changes from disproportionately affecting the score.

Load-bearing premise

Heterogeneous outputs from the security tools can be normalized to a shared 0-100 scale without significant loss of information and that the chosen weights reflect true overall security.

What would settle it

A test where the framework is applied to a system before and after applying full CIS hardening, checking if the composite score increases, or applying it to a system with known severe vulnerabilities to see if the score remains low.

Figures

Figures reproduced from arXiv: 2604.17256 by Izzat Alsmadi, Sheldon Paul.

Figure 1
Figure 1. Figure 1: UCA Framework Architecture. Six tools independently scan the target, outputs are [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: UCA weight distribution. Multi-domain tools (Lynis, OpenSCAP CIS) receive 0.20 [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: FABRIC testbed topology. Five nodes on a private L2 bridge (10.10.1.0/24): three [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Individual tool scores across three hardening levels. Compliance tools trend upward; [PITH_FULL_IMAGE:figures/full_fig_p009_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: UCA composite scores: 58.34 (baseline), 64.80 (partial), 68.17 (full), representing [PITH_FULL_IMAGE:figures/full_fig_p011_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Score trends revealing three patterns: compliance tools trending upward, file integrity [PITH_FULL_IMAGE:figures/full_fig_p012_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Grouped bar comparison of individual tool scores and composite UCA scores before [PITH_FULL_IMAGE:figures/full_fig_p012_7.png] view at source ↗
read the original abstract

Assessing the security posture of modern computing systems typically requires the use of multiple specialized tools. These tools focus on different aspects such as configuration compliance, file integrity, and vulnerability exposure, and their outputs are often difficult to interpret collectively. This paper introduces the Unified Compliance Aggregator (UCA), a framework that integrates several open-source security tools into a single composite score representing overall system security. The proposed framework combines outputs from Lynis, OpenSCAP (STIG and CIS profiles), AIDE, Tripwire, and Nmap NSE. A normalization process converts heterogeneous outputs into a consistent 0 to 100 scale, followed by weighted aggregation. We also introduce a logarithmic scoring model for file integrity measurements to address limitations observed in prior linear approaches. Experiments were conducted on Ubuntu 22.04 across different hardening levels and environments. Results show consistent improvement in composite scores as systems are hardened, while also revealing contrasting behavior between compliance and file integrity tools. Two case studies, a basic web server and a DVWA-based system illustrate how the framework can be applied in practical scenarios.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

4 major / 2 minor

Summary. The paper introduces the Unified Compliance Aggregator (UCA) framework that integrates outputs from Lynis, OpenSCAP (STIG and CIS profiles), AIDE, Tripwire, and Nmap NSE into a single composite security score on a 0-100 scale. It describes a normalization step for heterogeneous tool outputs followed by weighted aggregation, introduces a logarithmic scoring model for file integrity tools, reports experiments on Ubuntu 22.04 across hardening levels showing score improvements, and presents two case studies on a basic web server and a DVWA-based system.

Significance. If the normalization mappings, weight selection, and resulting composite score can be shown to preserve decision-relevant security information without arbitrary bias, the framework could serve as a practical engineering contribution for automated multi-tool assessment of Linux systems. The work currently functions as an integration prototype whose utility remains unproven due to missing validation.

major comments (4)
  1. [Framework description (normalization subsection)] The normalization process that converts heterogeneous outputs (e.g., Lynis pass/fail counts, AIDE/Tripwire file-change counts, Nmap port states) to a shared 0-100 scale is described only at a high level; no explicit mapping functions, scaling parameters, or handling of severity strings are supplied, which is load-bearing for the claim that the composite score meaningfully represents overall security posture.
  2. [Aggregation model] The weighted aggregation step relies on unspecified weights whose selection process, justification, or sensitivity analysis is absent; the free parameters noted in the axiom ledger (aggregation weights and logarithmic scaling parameters) are not constrained or validated against any external criterion.
  3. [Experiments section] The Ubuntu 22.04 experiments report consistent score improvement with hardening but supply neither the exact normalization formulas applied nor statistical validation, error bars, or correlation with independent metrics such as expert audit scores or red-team success rates, rendering the results tautological rather than confirmatory.
  4. [Logarithmic scoring model] The logarithmic scoring model for file integrity tools is introduced to address limitations of prior linear approaches, yet no derivation, comparison data, or quantitative improvement metric is provided to support its superiority.
minor comments (2)
  1. [Abstract and Discussion] The abstract states that results 'reveal contrasting behavior between compliance and file integrity tools' but the discussion does not elaborate on the security implications or recommended actions arising from this contrast.
  2. [Framework description] Notation for the composite score and individual tool contributions could be formalized with equations to improve reproducibility.

Simulated Author's Rebuttal

4 responses · 0 unresolved

We thank the referee for the constructive and detailed feedback on our manuscript. We address each major comment below with clarifications and commitments to strengthen the paper where the current description is insufficient.

read point-by-point responses

  1. Referee: [Framework description (normalization subsection)] The normalization process that converts heterogeneous outputs (e.g., Lynis pass/fail counts, AIDE/Tripwire file-change counts, Nmap port states) to a shared 0-100 scale is described only at a high level; no explicit mapping functions, scaling parameters, or handling of severity strings are supplied, which is load-bearing for the claim that the composite score meaningfully represents overall security posture.

    Authors: We agree that the normalization subsection is currently high-level and lacks the explicit details needed for full reproducibility. In the revised manuscript we will add the precise mapping functions for each tool (including Lynis counts, AIDE/Tripwire change counts, and Nmap states), the scaling parameters applied to reach the 0-100 range, and the rules for interpreting severity strings. These additions will directly support the claim that the composite score preserves decision-relevant information. revision: yes

  2. Referee: [Aggregation model] The weighted aggregation step relies on unspecified weights whose selection process, justification, or sensitivity analysis is absent; the free parameters noted in the axiom ledger (aggregation weights and logarithmic scaling parameters) are not constrained or validated against any external criterion.

    Authors: The weights were initially selected using a combination of tool documentation emphasis and iterative tuning on pilot systems to balance compliance, integrity, and exposure signals. We acknowledge that the manuscript does not document this process or include sensitivity analysis. The revision will add an explicit subsection describing the selection rationale, the constraints applied to the free parameters, and a sensitivity study showing how score variation remains limited across reasonable weight perturbations. revision: yes

  3. Referee: [Experiments section] The Ubuntu 22.04 experiments report consistent score improvement with hardening but supply neither the exact normalization formulas applied nor statistical validation, error bars, or correlation with independent metrics such as expert audit scores or red-team success rates, rendering the results tautological rather than confirmatory.

    Authors: The experiments were intended to demonstrate that the framework produces monotonic, expected behavior across controlled hardening levels, providing an initial sanity check on the aggregation logic. We accept that the current presentation lacks the requested statistical elements. The revised version will include the exact normalization formulas used, error bars on the reported scores, statistical significance tests, and discussion of correlation with available public benchmarks. Full red-team or expert-audit correlation is outside the scope of this work and would require separate controlled studies; we will note this limitation and cite related validation approaches in the literature. revision: partial

  4. Referee: [Logarithmic scoring model] The logarithmic scoring model for file integrity tools is introduced to address limitations of prior linear approaches, yet no derivation, comparison data, or quantitative improvement metric is provided to support its superiority.

    Authors: We will expand the relevant section to include the mathematical derivation of the logarithmic model, a side-by-side comparison against linear scoring on the same file-integrity data sets, and quantitative metrics (such as score variance and discrimination power) that illustrate the improvement. This will substantiate the claim that the logarithmic formulation better handles varying change counts. revision: yes

Circularity Check

0 steps flagged

No circularity: engineering framework with explicit construction, no derivations or predictions

full rationale

The paper presents UCA as an integration framework that normalizes outputs from Lynis, OpenSCAP, AIDE, Tripwire and Nmap into a 0-100 scale then applies weighted aggregation plus a logarithmic transform for integrity tools. No first-principles derivation, uniqueness theorem, or prediction is claimed; the composite score is defined by construction from the chosen tool outputs and design choices. Experiments only show the expected directional change when hardening is applied, which follows directly from the definition rather than constituting an independent result. No self-citations, fitted parameters renamed as predictions, or ansatzes smuggled via prior work appear in the described chain. The work is self-contained as an engineering artifact.

Axiom & Free-Parameter Ledger

2 free parameters · 1 axioms · 0 invented entities

The central claim rests on the assumption that tool outputs are commensurable after normalization and that weighted summation produces a meaningful security metric. No new physical or mathematical entities are introduced.

free parameters (2)
  • aggregation weights
    Weights applied to each tool's normalized score; values not specified in abstract but required for the composite.
  • logarithmic scaling parameters
    Parameters in the logarithmic model for file integrity scores introduced to address prior linear limitations.
axioms (1)
  • domain assumption Outputs from Lynis, OpenSCAP, AIDE, Tripwire, and Nmap can be normalized to a common 0-100 scale without loss of critical security information.
    Invoked in the normalization process described in the abstract.

pith-pipeline@v0.9.0 · 5487 in / 1366 out tokens · 33214 ms · 2026-05-10T06:25:56.374306+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

21 extracted references · 21 canonical work pages

  1. [1]

    Security and Privacy Controls for Information Systems and Organizations,

    NIST, “Security and Privacy Controls for Information Systems and Organizations,” SP 800- 53 Rev. 5, 2020

    work page 2020

  2. [2]

    Cost of a Data Breach Report 2023,

    IBM Security, “Cost of a Data Breach Report 2023,” IBM Corporation, 2023

    work page 2023

  3. [3]

    2023 Data Breach Investigations Report,

    Verizon, “2023 Data Breach Investigations Report,” Verizon Enterprise Solutions, 2023

    work page 2023

  4. [4]

    Security Hardening Using FABRIC: Implementing a Unified Com- pliance Aggregator,

    S. Paul and I. Alsmadi, “Security Hardening Using FABRIC: Implementing a Unified Com- pliance Aggregator,” arXiv:2601.00909, 2025. 14

    work page arXiv 2025

  5. [5]

    FABRIC: A National-Scale Programmable Experimental Network Infras- tructure,

    I. Baldin et al., “FABRIC: A National-Scale Programmable Experimental Network Infras- tructure,”IEEE Internet Computing, vol. 23, no. 6, pp. 38-47, 2019

    work page 2019

  6. [6]

    Lynis Security Auditing Tool,

    CISOfy, “Lynis Security Auditing Tool,”https://cisofy.com/lynis/

    work page

  7. [7]

    Technical Specification for SCAP,

    D. Waltermire et al., “Technical Specification for SCAP,” NIST SP 800-126 Rev. 3, 2018

    work page 2018

  8. [8]

    OpenSCAP NIST Certified Toolkit,

    OpenSCAP Project, “OpenSCAP NIST Certified Toolkit,”https://www.open-scap.org/

    work page

  9. [9]

    SCAP Security Guide,

    ComplianceAsCode, “SCAP Security Guide,”https://github.com/ComplianceAsCode/ content

    work page

  10. [10]

    Lehti and P

    R. Lehti and P. Virolainen, “AIDE,”https://aide.github.io/

    work page

  11. [11]

    Open Source Tripwire,

    Tripwire Inc., “Open Source Tripwire,”https://github.com/Tripwire/ tripwire-open-source

    work page

  12. [12]

    CIS Benchmarks,

    Center for Internet Security, “CIS Benchmarks,”https://www.cisecurity.org/ cis-benchmarks

    work page

  13. [13]

    Lyon,Nmap Network Scanning, Nmap Project, 2009

    G. Lyon,Nmap Network Scanning, Nmap Project, 2009

    work page 2009

  14. [14]

    OpenVAS,

    Greenbone Networks, “OpenVAS,”https://www.openvas.org/

    work page

  15. [15]

    OWASP Top Ten,

    OWASP Foundation, “OWASP Top Ten,”https://owasp.org/Top10/

    work page

  16. [16]

    Damn Vulnerable Web Application,

    DVWA Project, “Damn Vulnerable Web Application,”https://github.com/digininja/ DVWA

    work page

  17. [17]

    Chuvakin, K

    A. Chuvakin, K. Schmidt, and C. Phillips,Logging and Log Management, Syngress, 2012

    work page 2012

  18. [18]

    Market Guide for SOAR Solutions,

    Gartner, “Market Guide for SOAR Solutions,” 2019

    work page 2019

  19. [19]

    Taxonomy of ISRA,

    A. Shameli-Sendi et al., “Taxonomy of ISRA,”Computers & Security, vol. 57, pp. 14-30, 2016

    work page 2016

  20. [20]

    Automated Security Compliance Assessment,

    M. Schwartz et al., “Automated Security Compliance Assessment,”Bell Labs Technical Journal, vol. 12, no. 3, pp. 203-218, 2007

    work page 2007

  21. [21]

    A Survey on Vulnerability Assessment Tools,

    A. Alhomoud et al., “A Survey on Vulnerability Assessment Tools,”ICICS, pp. 1–6, 2011. 15

    work page 2011