POSTCONDBENCH is a new multilingual benchmark that evaluates LLM postcondition generation on real code using defect discrimination to assess completeness beyond surface matching.
arXiv preprint arXiv:2206.08474 , year=
9 Pith papers cite this work, alongside 18 external citations. Polarity classification is still indexing.
9
Pith papers citing it
18
external citations · external index
citation-role summary
background 1
dataset 1
citation-polarity summary
representative citing papers
AutoVecCoder combines VecPrompt for automated intrinsic knowledge synthesis and VecRL for efficiency-aligned RL to train an 8B LLM that achieves SOTA on SimdBench SSE/AVX subsets and sometimes exceeds -O3 compiler results.
PseudoBridge uses LLM-synthesized pseudo-code to bridge NL semantics and PL logic plus logic-invariant style augmentation to boost robustness and generalization in code retrieval.
LiveCodeBench collects 400 recent contest problems to create a contamination-free benchmark evaluating LLMs on code generation and related capabilities like self-repair and execution.
UNICS pre-trains on a pseudocode dataset for cross-lingual logic then applies multi-task transfer learning with hard-positive mining and dynamic hard-negative sampling to reach claimed SOTA on multilingual code-search benchmarks.
Multisage uses multi-semantic augmentation and self-calibration to enhance LLM-based code translation, achieving up to 2.22x higher success rates on HumanEval-X.
Unifying AST labels across languages and encoding paired graphs with a Graph Matching Network creates a shared semantic vector space that places functionally equivalent code from different languages near each other.
A survey of methods, benchmarks, and open challenges for large language models in multilingual code generation and translation.
citing papers explorer
-
POSTCONDBENCH: Benchmarking Correctness and Completeness in Formal Postcondition Inference
POSTCONDBENCH is a new multilingual benchmark that evaluates LLM postcondition generation on real code using defect discrimination to assess completeness beyond surface matching.
-
AutoVecCoder: Teaching LLMs to Generate Explicitly Vectorized Code
AutoVecCoder combines VecPrompt for automated intrinsic knowledge synthesis and VecRL for efficiency-aligned RL to train an 8B LLM that achieves SOTA on SimdBench SSE/AVX subsets and sometimes exceeds -O3 compiler results.
-
PseudoBridge: Pseudo Code as the Bridge for Better Semantic and Logic Alignment in Code Retrieval
PseudoBridge uses LLM-synthesized pseudo-code to bridge NL semantics and PL logic plus logic-invariant style augmentation to boost robustness and generalization in code retrieval.
-
LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code
LiveCodeBench collects 400 recent contest problems to create a contamination-free benchmark evaluating LLMs on code generation and related capabilities like self-repair and execution.
-
UNICS: Multilingual Code Search via Unified Pseudocode and Contrastive Transfer Learning
UNICS pre-trains on a pseudocode dataset for cross-lingual logic then applies multi-task transfer learning with hard-positive mining and dynamic hard-negative sampling to reach claimed SOTA on multilingual code-search benchmarks.
-
Enhancing LLM-Based Code Translation with Verified Multi-Semantic Representations
Multisage uses multi-semantic augmentation and self-calibration to enhance LLM-based code translation, achieving up to 2.22x higher success rates on HumanEval-X.
-
Bridging the Programming Language Gap: Constructing a Multilingual Shared Semantic Space through AST Unification and Graph Matching
Unifying AST labels across languages and encoding paired graphs with a Graph Matching Network creates a shared semantic vector space that places functionally equivalent code from different languages near each other.
-
Large Language Models for Multilingual Code Intelligence: A Survey
A survey of methods, benchmarks, and open challenges for large language models in multilingual code generation and translation.
- Human agency in initial human-AI proof formalization workflows