REVIEW 11 cited by
Constrained Decoding for Secure Code Generation
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Constrained Decoding for Secure Code Generation
read the original abstract
Code Large Language Models (Code LLMs) have been increasingly used by developers to boost productivity, but they often generate vulnerable code. Thus, there is an urgent need to ensure that code generated by Code LLMs is correct and secure. Previous research has primarily focused on generating secure code, overlooking the fact that secure code also needs to be correct. This oversight can lead to a false sense of security. Currently, the community lacks a method to measure actual progress in this area, and we need solutions that address both security and correctness of code generation. This paper introduces a new benchmark, CodeGuard+, along with two new metrics, to measure Code LLMs' ability to generate both secure and correct code. Using our new evaluation methods, we show that the state-of-the-art defense technique, prefix tuning, may not be as strong as previously believed, since it generates secure code but sacrifices functional correctness. We also demonstrate that different decoding methods significantly affect the security of Code LLMs. Furthermore, we explore a new defense direction: constrained decoding for secure code generation. We propose new constrained decoding techniques to generate secure code. Our results reveal that constrained decoding is more effective than prefix tuning to improve the security of Code LLMs, without requiring a specialized training dataset. Moreover, our evaluations over eight state-of-the-art Code LLMs show that constrained decoding has strong performance to improve the security of Code LLMs, and our technique outperforms GPT-4.
Forward citations
Cited by 11 Pith papers
-
Correct Code, Vulnerable Dependencies: A Large Scale Measurement Study of LLM-Specified Library Versions
LLMs frequently specify library versions with known CVEs in generated code (36-56% of tasks), show low compatibility (20-63%), and converge on the same risky versions across models.
-
Themis: Training Robust Multilingual Code Reward Models for Flexible Multi-Criteria Scoring
Themis builds a multilingual benchmark and large preference dataset to train code reward models that score outputs on multiple criteria like correctness, efficiency, and style.
-
Themis: Training Robust Multilingual Code Reward Models for Flexible Multi-Criteria Scoring
Themis introduces the largest open code preference dataset with over 350k pairs and trains multilingual reward models from 600M to 32B parameters that support flexible multi-criteria scoring, with experiments showing ...
-
HardSecBench: Benchmarking the Security Awareness of LLMs for Hardware Code Generation
A 924-task benchmark reveals that LLMs generate functionally correct but security-incomplete hardware code, and that prompting with security hints substantially improves security pass rates.
-
Functional and Secure Code Generation with Task Vectors
LPO-derived Secure-Anchored task vectors raise simultaneous functional-and-secure code rates by 2.1–36 pp on six coding LLMs with near-zero inference overhead.
-
SoK: AI Secure Code Generation: Progress, Pitfalls, and Paths Forward
The paper introduces a three-level framework for AI secure code generation, finds that principle understanding statistically predicts code outcomes, but identifies persistent knowledge-actuation gaps across models and agents.
-
DeepGuard: Secure Code Generation via Multi-Layer Semantic Aggregation
DeepGuard aggregates multi-layer representations in code LLMs to raise the secure-and-correct generation rate by 11.9% on average over baselines like SVEN while preserving correctness and generalizing to new vulnerabi...
-
Beyond Final Code: A Process-Oriented Error Analysis of Software Development Agents in Real-World GitHub Scenarios
Empirical study of 3977 agent trajectories finds Python execution errors correlate with lower success rates on GitHub issues, flags challenging errors, and reports three confirmed bugs in the SWE-Bench platform.
-
CoGate: Confidence-Gated Co-Decoding for Secure Code Generation
Gating a security expert by its own confidence (max-prob or entropy) fixes inverted acceptance in co-decoding and improves joint functional-security code generation, especially on OOD CWEs.
-
An Empirical Analysis of Static Analysis Methods for Detection and Mitigation of Code Library Hallucinations
Static analysis tools detect 14-85% of library hallucinations in LLM code but are limited to at most 48.5-77% coverage even in ideal cases.
-
MMIR-TCM: Memory-Integrated Multimodal Inference and Retrieval for TCM Clinical Decision Support
MMIR-TCM is a multimodal framework using MLLM, memory-SAM, and RAG that claims to outperform GPT-4o and Gemini on TCM tongue diagnosis tasks via a new dataset and custom metric.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.