{"id":"82b7cb16-9eab-4502-85bd-e2458d020867","arxiv_id":"2509.04191","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"KubeGuard generates and refines Kubernetes Roles, NetworkPolicies, and Deployments from aggregated audit, network, and provenance logs using prompt-chained LLMs, achieving F1 up to 0.96 with GPT-4o.","lead":"KubeGuard is a system that uses large language models to read Kubernetes logs and create or tighten security rules, granting only the permissions a workload actually used. It offers a practical path to least-privilege configurations without manually auditing every manifest.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Evaluation baselines are derived from the same runtime logs KubeGuard consumes, so reported F1 cannot detect omissions of legitimate operations absent from logs; log completeness is untested.","rationale":"I read the paper's central claim as: KubeGuard can take runtime logs plus manifests and produce least-privilege manifests that are both minimal and functionally complete. The evaluation provides strong evidence that KubeGuard can align manifests with the provided logs: ablations show the matching step and data sources matter, and comparisons to audit2rbac/KUBETEUS show it outperforms simple log-to-policy tools. However, the ground truth is not independent of the input. The paper's own definitions of TP/FP/FN (Sec. 4.3.1) tie correctness to what is 'evidenced in the logs.' That makes the evaluation circular with respect to the property that matters most: completeness of legitimate operations. This is precisely the failure mode that breaks least-privilege hardening in practice, and it is the assumption flagged by the reader. The sensitivity analysis (Sec. 5.8.2) is a step in the right direction but only shows internal stability, not external completeness. I therefore see no need to change the reader's CONDITIONAL verdict; my concern is essentially a sharper statement of the same weakest assumption. A single independent-ground-truth experiment would settle it.","tokens_in":31209,"tokens_out":3402,"duration_ms":33239,"concrete_test":"Independently build a ground truth for Online Boutique and AKS Store Demo: enumerate required RBAC verbs/resources and network flows from application source code, Kubernetes manifests, documentation, and targeted executions of rare paths (admin endpoints, scheduled jobs, failover, retries). Re-run all five prompt chains with GPT-4o and compare outputs to this independent ground truth using the paper's TP/FP/FN definitions (Sec. 4.3.1), without reference to the original log-derived baselines. If recall drops substantially (e.g., >0.1) or if specific legitimate permissions/connections are missing from KubeGuard's outputs, the completeness assumption is violated and the central claim must be weakened. Conversely, if F1 remains high against independent ground truth, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing concern is that the reported metrics are computed against ground-truth baselines 'derived from real application logs' (Sec. 4.2) — the same logs KubeGuard uses as input. A false negative is defined only as a required element 'despite being evidenced in the logs' (Sec. 4.3.1). Consequently, if a legitimate Role verb, network connection, or Deployment configuration is absent from the collected audit/Hubble/provenance data, it is also absent from the baseline and is never counted as a miss. The central claim that KubeGuard produces least-privilege manifests that maintain functionality therefore rests on an untested completeness assumption: the load-generated traces must contain every legitimate operation. The paper acknowledges this in Sec. 6 ('capturing sufficient log data requires representative traffic') and argues convergence in Sec. 5.8.2, but the similarity metrics there only show that aggregated token sets stabilize; they say nothing about whether the stabilized set is complete relative to the application's true permission/network requirements. Thus, the perfect F1 for Role Creation and high F1s for refinement may reflect faithful reproduction of the input logs rather than correct inference of complete least-privilege policies.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"KubeGuard is a log-driven Kubernetes hardening framework that aggregates audit logs, Hubble network flows, and SPADE provenance data into key-value summaries and feeds them, together with existing manifests, through modular LLM/SLM prompt chains. It supports two tasks: creating least-privilege Roles and NetworkPolicies from observed behavior, and refining existing Roles, NetworkPolicies, and Deployments. The paper evaluates KubeGuard on two microservice applications, using GPT-4o and several open-source models, and reports precision/recall/F1 against baseline manifests 'derived from real application logs' (Sec. 4.2), with additional ablations, sensitivity analyses, and comparisons to audit2rbac and KUBETEUS. The strongest reported results are perfect F1 for Role Creation and high F1 for the refinement tasks with GPT-4o.","tokens_in":31485,"tokens_out":3829,"duration_ms":41615,"significance":"If the evaluation supported the completeness of the generated policies, KubeGuard would be a valuable practical contribution: it combines multiple runtime data sources, uses a modular prompt-chain design, and demonstrates that local SLMs provide a usable privacy-preserving alternative. The paper includes a substantial set of experiments: prompt-strategy selection, model comparisons, iterative refinement, sensitivity to log duration and prompt order, ablation of matching steps and data sources, and two external baseline comparisons. These are real strengths and go beyond what is typical in this line of work. The main weakness is that the evaluation measures extraction fidelity from the input logs rather than the correctness of the hardening recommendation with respect to the application's true operational requirements. Because the ground truth is derived from the same logs KubeGuard consumes, false negatives are defined only relative to what is present in those logs. This limits the strength of the central claim that KubeGuard preserves functionality while reducing attack surface.","major_comments":[{"comment":"The ground-truth baselines are 'derived from real application logs' (Sec. 4.2), and a false negative is defined as a required element 'despite being evidenced in the logs' (Sec. 4.3.1). This makes the evaluation circular with respect to log completeness: if a legitimate Role verb, network flow, or Deployment setting never appears in the collected audit/Hubble/provenance data, it is also absent from the baseline and is never counted as a miss. The reported F1 scores therefore show how well KubeGuard recovers the evidence contained in the logs, not whether the resulting least-privilege manifest is complete for the application. This is a load-bearing gap for the paper's 'maintaining application functionality' claim. I ask the authors to provide a concrete test of completeness, for example by (a) constructing ground-truth policies from application semantics or documentation that include oper","section":"Sec. 5.8.2"},{"comment":"The convergence analysis uses Cosine, Overlap, and Dice similarity between successive cumulative aggregated-log segments (Fig. 8). This establishes that the token sets stabilize over time, but it does not establish that the stabilized set is complete relative to the application's true permission/network requirements. A stable set can still omit a rare but legitimate scheduled job, a failover path, or an administrative action. The paper's own limitation section (Sec. 6) correctly notes that 'capturing sufficient log data requires representative traffic,' and the convergence argument does not close that gap. I suggest adding an evaluation that varies the length of the log-collection window and measures the effect on the generated policy's recall against a ground truth that includes operations outside the observed window, or, at minimum, an analysis of which rare tokens are absent from the","section":"Sec. 5.8.2"},{"comment":"The perfect F1 of 1.00 for Role Creation is suspicious given the baseline construction. If the baseline Roles are a rule-based extraction of verbs/resources from the AALs, and KubeGuard's prompt chain is designed to translate AALs into Roles, then perfect agreement may reflect deterministic information transfer rather than the model learning to generalize to unseen permission requirements. The same concern applies to the high refinement scores, which may be inflated because the baseline contains only elements already present in the logs. Please clarify how the baselines were constructed (human review, rule-based extraction, or a combination), and report at least one metric that is defined against an independent source of required permissions/connections, not against the logs themselves.","section":"Table 3, Role Creation row"}],"minor_comments":[{"comment":"The text refers to 'similarity metrics (Sec. 5.9.1)', but the Cosine, Overlap, and Dice metrics are defined in Sec. 4.3.2. The cross-reference should be corrected.","section":"Sec. 5.8.2"},{"comment":"The text says Qwen2.5-7B's precision was 0.722, while Table 4 reports 0.772±0.122. One of these is a typo; please correct it.","section":"Sec. 5.3, Table 4"},{"comment":"Figure 7 shows iterative refinement results without the standard-deviation bars that accompany the other tables. Since the earlier tables report ± values, adding the same confidence information to the figure would help the reader judge the stability of the oscillation claim.","section":"Sec. 5.5, Fig. 7"},{"comment":"The paper repeatedly mentions 'postprocessing' as a mitigation for recall drops and underspecified rules, but never describes what postprocessing is or whether the reported numbers include it. Please clarify, since this is a free parameter in the evaluation. The axiom ledger in the review identifies postprocessing as a free parameter; the manuscript should state exactly how and when it was applied.","section":"Sec. 5.2 and Sec. 5.10"},{"comment":"The chain has both step 4 and step '4.A' (validation of log-to-policy matches), but the text does not explain what the validation consists of or how it differs from the matching step. A brief description or a pointer to Figure 13/14 would make the workflow easier to follow.","section":"Sec. 3.2.2, NetworkPolicy Refinement step 4"}],"recommendation":"major_revision","confidential_remarks":"The paper is a systems contribution with a thorough experimental apparatus, and I do not see grounds for rejection at this stage. The central issue is the circular evaluation design: the ground truth is derived from the same logs the framework consumes, so the reported F1 scores do not validate the completeness of the generated least-privilege policies. This can be addressed with additional experiments or with a clearly scoped restatement of the claim (e.g., 'correctly summarizes the evidence in the logs' rather than 'preserves application functionality'). I also note that while the paper compares against audit2rbac and KUBETEUS, those comparisons are relatively coarse; the audit2rbac comparison in particular mixes a unit of analysis difference (user-centric vs. pod-centric) with the performance difference. The authors should separate these factors in the revision. I would encourage the editor to let the authors respond to the completeness concern before a final decision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Bottom line: this is a solid, well-executed piece of systems work that does something genuinely new – it fuses audit, network, and provenance logs into staged LLM prompt chains to create and refine Kubernetes Roles, NetworkPolicies, and Deployments. The engineering is thoughtful: K-V aggregation cuts token counts by roughly 99.96%, the prompt chains are modular, and the evaluation includes ablations, prompt-order sensitivity, log-duration sensitivity, and head-to-head comparisons with audit2rbac and KUBETEUS. For GPT-4o the results are strong (F1 0.93-1.00 across tasks), and the Llama-3.1-8B results are honest and useful.\n\nThe main soft spot is the one the stress-test flags. The ground-truth baselines were derived from the same runtime logs KubeGuard consumes. So the reported F1 only measures how well the model reproduces what is in the logs; it cannot detect cases where a legitimate permission, connection, or config is missing from the logs entirely. The convergence analysis (Section 5.8.2) shows that aggregated token sets stabilize over time, but stability is not the same as completeness - a rare scheduled job or failover path that never runs during collection won't appear, and the baseline won't count it as a miss. The paper acknowledges this in Section 6, which is good, but it is a real gap, not a minor caveat, because the central claim is \"least-privilege while maintaining functionality.\" On the current evidence, KubeGuard is a high-fidelity log-to-manifest translator, and likely a useful one, but its guarantees are bounded by trace coverage.\n\nSecondary concerns: no code or data release, so independent check of the exact prompts and K-V aggregation is impossible; only two demo apps with load-generated traffic. These are common and don't sink the paper. I'd trust the reported numbers on the testbed; I just wouldn't generalize beyond it.\n\nWho should read it: anyone working on LLM-assisted infrastructure policy, and K8s security people thinking about least-privilege automation. It deserves a serious referee - the architecture and evaluation design are above the usual LLM-prompting paper - but the revision should release artifacts and confront the completeness assumption directly, either with a third-party ground truth or a fault-injection study that removes logs and shows where failures appear.","headline":"KubeGuard is a well-executed LLM+multi-log pipeline for Kubernetes least-privilege manifests; the evaluation is encouraging on its own terms, but the baseline's dependence on the same logs leaves a real completeness gap.","tokens_in":32002,"tokens_out":2175,"would_cite":true,"duration_ms":22238,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Kubernetes hardening can be driven entirely by runtime logs: LLM prompt chains turn audit, network, and provenance data into least-privilege Roles, NetworkPolicies, and Deployments, reaching a perfect F1 on Role creation.","keywords":["Kubernetes hardening","least-privilege configuration","LLM prompt chaining","RBAC Roles","NetworkPolicies","Deployment refinement","audit and provenance logs","log-driven security"],"falsifier":"Run KubeGuard on a workload with a documented rare-but-legal operation (e.g., a nightly batch job or a failover endpoint) that is excluded from the traffic fed into the logs; if the hardened Role or NetworkPolicy drops the permission or connection that operation needs and the workload breaks, the completeness assumption fails. A cheaper check: compare KubeGuard's refined manifests against the full documented API surface of a mature application and count pruned permissions that are later exercised in production.","tokens_in":31122,"feed_emoji":"🛡️","tokens_out":8569,"duration_ms":65838,"temperature":0.7,"pith_summary":"KubeGuard is a recommender framework that hardens Kubernetes clusters by converting runtime observability into least-privilege configuration. It collects three kinds of logs — API audit records, network flows, and OS-level provenance — compresses them into compact key-value summaries, and feeds them through LLM prompt chains (sequences of prompts where each output seeds the next) to create missing Roles and NetworkPolicies or refine overly permissive ones. The paper's central claim is that observed behavior, properly aggregated, is sufficient signal to generate accurate least-privilege manifests, and it reports strong results: with GPT-4o, Role creation scores a perfect F1 of 1.00, and Role, NetworkPolicy, and Deployment refinement reach 0.953, 0.961, and 0.929. The framework also works with open-source local models such as Llama-3.1-8B, making it usable in privacy-sensitive settings. If the claim holds, operators gain a practical path from detecting misconfigurations to adaptively generating the tighter configuration.","feed_headline":"KubeGuard derives least-privilege Kubernetes configs from live logs","feed_subtitle":"With GPT-4o, Role creation scores a perfect F1 and refinements beat audit2rbac and KUBETEUS head-to-head.","key_machinery":"Two mechanisms carry the argument. (1) K-V aggregation: raw audit logs, Hubble network flows, and SPADE provenance records are flattened into key-to-set-of-values maps grouped by microservice, compressing model input tokens by up to 99.96% while preserving the behavioral facts a manifest must encode. (2) Prompt chaining: each task is a fixed sequence of structured prompts — analyze logs, analyze manifest, match logs to manifest, recommend revisions, revise — where each prompt's output becomes the next prompt's input; the matching step is what ties declared permissions to observed activity. Together they turn the least-privilege principle into a concrete, log-derived editing operation on YAML","core_discovery":"The paper's central claim: multi-source runtime logs, compressed into key-value summaries, carry enough information for an LLM to derive Kubernetes manifests that grant only the permissions, connections, and settings an application actually uses. KubeGuard implements this as Resource Creation (generate Roles or NetworkPolicies from scratch) and Resource Refinement (strip unused verbs, redundant ports, and over-broad rules from existing manifests). The discovery is that prompt chaining — decompose each task into analyze, match, recommend, revise — is what makes this reliable, and that the log-to-manifest matching step is load-bearing: removing it drops refinement F1 across all resources (Netw","pith_inferences":["The completeness assumption cuts both ways: KubeGuard prunes what the logs do not show, so any legitimate operation absent from the observation window — nightly batch jobs, failover paths, manual debugging access — will be dropped unless an operator reviews the output; the paper's convergence result (audit logs stabilize after ~1.75 days) suggests collecting at least that long before trusting a Ro","If KubeGuard is right that aggregated observability is the right input signal, the same aggregation-plus-chaining recipe should transfer to other policy surfaces beyond Kubernetes, such as cloud IAM or service-mesh authorization, where declared configuration can be checked against observed calls.","A testable extension would be to pair KubeGuard's log-derived baseline with an explicit operator-supplied keep-list of rare-but-legitimate operations, turning the completeness assumption from an unstated gap into a controllable input.","The near-perfect Role-creation score may reflect how cleanly audit logs encode RBAC actions (each API request names verb, resource, and subject), whereas NetworkPolicy and Deployment refinement must reason from noisier flow and provenance data — so future work should expect harder ceilings for network- and container-level hardening."],"forward_implications":["Operators can replace static, rule-based hardening tools with adaptive, behavior-aware configuration: every generated or refined manifest is grounded in the cluster's own audit, network, and provenance records.","Kubernetes hardening becomes measurable: a refined manifest's residual attack surface is quantified by the permissions and rules left after matching against observed behavior, scored with precision, recall, and F1 against a log-derived baseline.","Privacy-constrained organizations can run the full workflow offline with an 8-billion-parameter local model (Llama-3.1-8B), accepting a moderate accuracy drop (e.g., Role refinement F1 0.808) in exchange for keeping cluster data on-premises.","Iterative refinement offers a cheap accuracy boost for weaker models: recursing the NetworkPolicy refinement chain lifts Llama-3.1-8B recall from 0.385 to 0.593 over three iterations, at the cost of oscillation and diminishing returns in later rounds.","The framework is model-agnostic and resource-agnostic, so the same creation-and-refinement pattern extends to additional Kubernetes resource types beyond Roles, NetworkPolicies, and Deployments."],"supporting_citations":[{"why":"Kubernetes auditing documentation; supplies the audit-log source whose recorded API actions define observed RBAC behavior.","marker":"[40]"},{"why":"Hubble network observability tool; its flow capture becomes the Aggregated Network Logs that drive NetworkPolicy creation and refinement.","marker":"[18]"},{"why":"SPADE provenance auditing; the collection and provenance-model support that feed the Aggregated Provenance Logs used in Deployment refinement.","marker":"[9, 25]"},{"why":"Prompt chaining technique; the prompting method selected over zero-shot and chain-of-thought, forming the backbone of all five workflows.","marker":"[5]"},{"why":"GPT-4o; the proprietary LLM selected after benchmarking, achieving the paper's best scores across all tasks.","marker":"[69]"},{"why":"Llama 3.1; the open-source SLM selected as the local privacy-preserving alternative and evaluated on all tasks.","marker":"[26]"},{"why":"audit2rbac; the baseline for Role creation that KubeGuard must beat in the head-to-head comparison (F1 0.343 vs. 1.00).","marker":"[55]"},{"why":"KUBETEUS; the baseline for NetworkPolicy creation, compared under the same small shared SLM (F1 0.275 vs. 0.332).","marker":"[36]"},{"why":"Online Boutique microservices demo; one of the two testbed applications whose workloads and logs ground the evaluation.","marker":"[72]"}],"fun_headline_variants":["KubeGuard: LLM turns runtime logs into least-privilege K8s configs","From live logs to locked-down K8s: KubeGuard's LLM prompt chains","LLM prompt-chaining hardens Kubernetes by mining runtime logs","KubeGuard refines K8s manifests from actual behavior, not static rules","Prompt-chained LLM derives least-privilege K8s configs from logs"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The framework assumes the logs collected during the observation window include every legitimate operation the application will ever need; any permission or connection absent from those logs is pruned as excessive, even if a rare scheduled job or failover path requires it.","fun_headline_variants_meta":{"raw":{"variants":["KubeGuard: LLM turns runtime logs into least-privilege K8s configs","From live logs to locked-down K8s: KubeGuard's LLM prompt chains","LLM prompt-chaining hardens Kubernetes by mining runtime logs","KubeGuard refines K8s manifests from actual behavior, not static rules","Prompt-chained LLM derives least-privilege K8s configs from logs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000178,"raw_usage":{"total_tokens":1158,"prompt_tokens":794,"completion_tokens":364,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":538,"completion_tokens_details":{"reasoning_tokens":264}},"tokens_in":538,"tokens_out":364,"duration_ms":3637,"temperature":1.0,"reasoning_tokens":264,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T10:18:11.412993+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run KubeGuard on a workload with a documented rare-but-legal operation (e.g., a nightly batch job or a failover endpoint) that is excluded from the traffic fed into the logs; if the hardened Role or NetworkPolicy drops the permission or connection that operation needs and the workload breaks, the completeness assumption fails. A cheaper check: compare KubeGuard's refined manifests against the full documented API surface of a mature application and count pruned permissions that are later exercised in production.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"GPT-4o; the proprietary LLM selected after benchmarking, achieving the paper's best scores across all tasks."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Online Boutique microservices demo; one of the two testbed applications whose workloads and logs ground the evaluation."}],"review_version":1}