CommitSuite is a large benchmark for commit classification and message generation that includes AST-level changes and LLM annotations, together with a reference-free evaluation framework achieving 0.849 Cohen's Kappa with humans.
Title resolution pending
4 Pith papers cite this work. Polarity classification is still indexing.
4
Pith papers citing it
citation-role summary
background 1
citation-polarity summary
fields
cs.SE 4years
2026 4roles
background 1polarities
background 1representative citing papers
A systematic mapping of 97 papers finds growing research on commit messages for bug analysis and fix identification, mainly combined with code diffs via repository mining and AI/ML, with developers as primary stakeholders, though messages frequently lack key details.
An independent large-scale replication confirms security commit messages are typically uninformative, with informativeness declining over time, varying by ecosystem, and lower for Conventional Commits Specification messages.
ARGUS extracts fragmented code change rationales from multiple documents using LLMs and generates summaries that developers rate as useful for review and maintenance.
citing papers explorer
-
CommitSuite: A Comprehensive Benchmark for Commit Classification and Message Generation
CommitSuite is a large benchmark for commit classification and message generation that includes AST-level changes and LLM annotations, together with a reference-free evaluation framework achieving 0.849 Cohen's Kappa with humans.
-
On the Use of Commit Messages for Corrective Software Maintenance: A Systematic Mapping Study
A systematic mapping of 97 papers finds growing research on commit messages for bug analysis and fix identification, mainly combined with code diffs via repository mining and AI/ML, with developers as primary stakeholders, though messages frequently lack key details.
-
On the Informativeness of Security Commit Messages: A Large-scale Replication Study
An independent large-scale replication confirms security commit messages are typically uninformative, with informativeness declining over time, varying by ecosystem, and lower for Conventional Commits Specification messages.
-
Fine-grained Multi-Document Extraction and Generation of Code Change Rationale
ARGUS extracts fragmented code change rationales from multiple documents using LLMs and generates summaries that developers rate as useful for review and maintenance.