REVIEW 13 cited by
RepoFusion: Training Code Models to Understand Your Repository
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
abstract
Despite the huge success of Large Language Models (LLMs) in coding assistants like GitHub Copilot, these models struggle to understand the context present in the repository (e.g., imports, parent classes, files with similar names, etc.), thereby producing inaccurate code completions. This effect is more pronounced when using these assistants for repositories that the model has not seen during training, such as proprietary software or work-in-progress code projects. Recent work has shown the promise of using context from the repository during inference. In this work, we extend this idea and propose RepoFusion, a framework to train models to incorporate relevant repository context. Experiments on single-line code completion show that our models trained with repository context significantly outperform much larger code models as CodeGen-16B-multi ($\sim73\times$ larger) and closely match the performance of the $\sim 70\times$ larger StarCoderBase model that was trained with the Fill-in-the-Middle objective. We find these results to be a novel and compelling demonstration of the gains that training with repository context can bring. We carry out extensive ablation studies to investigate the impact of design choices such as context type, number of contexts, context length, and initialization within our framework. Lastly, we release Stack-Repo, a dataset of 200 Java repositories with permissive licenses and near-deduplicated files that are augmented with three types of repository contexts. Additionally, we are making available the code and trained checkpoints for our work. Our released resources can be found at \url{https://huggingface.co/RepoFusion}.
Forward citations
Cited by 13 Pith papers
-
In Line with Context: Repository-Level Code Generation via Context Inlining
InlineCoder reframes repository-level code generation as function-level coding by using a draft anchor to inline the target function into its call graph for upstream usage and downstream dependency context.
-
SWE-QA: Can Language Models Answer Repository-level Code Questions?
SWE-QA creates a new repository-level code QA benchmark with 576 pairs and an agentic LLM framework, showing promise but open challenges for models handling complex codebases.
-
Code2LoRA: Hypernetwork-Generated Adapters for Code Language Models under Software Evolution
Code2LoRA generates repo-specific LoRA adapters via hypernetwork for code LMs, matching per-repo LoRA on static tasks and exceeding shared LoRA by 5.2 pp on evolving code in a 604-repo benchmark.
-
REAgent: Requirement-Driven LLM Agents for Software Issue Resolution
REAgent improves LLM patch generation for software issues by 17.4% on average through automated construction, quality checking, and iterative refinement of structured issue-oriented requirements.
-
Better Call Grep: Evaluating and Improving Grep-Like Lexical Retrieval for Repository-Level Code Completion
LLM-generated ripgrep queries plus BM25 re-ranking and line-interval de-duplication outperform graph- and RL-based retrievers for repository-level code completion on CrossCodeEval and RepoEval-Updated.
-
RepoDebug: Repository-Level Multi-Task and Multi-Language Debugging Evaluation of Large Language Models
RepoDebug is a new multi-language, multi-task benchmark for repository-level code debugging on which current LLMs, including the best model Claude 3.5 Sonnet, perform poorly.
-
Exploring the Challenges and Opportunities of AI-assisted Codebase Generation
Developers prompting codebase-level AI assistants are often dissatisfied with generated code, citing missing functionality, poor code quality, and communication gaps, despite varied prompting strategies.
-
Enhancing Project-Specific Code Completion by Inferring Internal API Information
A retrieval-augmented code completion method that infers project-internal APIs from a rough draft and a static knowledge base, beating existing repo-level baselines.
-
Optimization Hyper-parameter Laws for Large Language Models
Opt-Laws predicts LLM final training loss from LR schedules via SDE-derived convergence and escape features, with 94% Top-2 hit rate on held-out schedules and F1=0.92 for divergence detection.
-
Benchmarking LLM-Based Static Analysis for Secure Smart Contract Development: Reliability, Limitations, and Potential Hybrid Solutions
LLMs for smart contract security analysis show lexical bias from identifier names causing high false positives, with prompting creating precision-recall trade-offs, positioning them as complements rather than replacem...
-
GRACE: Graph-Guided Repository-Aware Code Completion through Hierarchical Code Fusion
GRACE combines a multi-level code graph, hybrid text-structure retrieval, and graph fusion to improve repository-level code completion over vanilla and graph-based RAG baselines.
-
Retrieval-Augmented Generation for AI-Generated Content: A Survey
A survey classifying RAG foundations for AIGC, summarizing enhancements, cross-modal applications, benchmarks, limitations, and future directions.
-
A Survey on Large Language Models for Code Generation
A systematic literature review that organizes recent work on LLMs for code generation into a taxonomy covering data curation, model advances, evaluations, ethics, environmental impact, and applications, with benchmark...
Discussion (0). Sign in to comment.