REVIEW 13 cited by
Towards Translating Real-World Code with LLMs: A Study of Translating to Rust
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Large language models (LLMs) show promise in code translation - the task of translating code written in one programming language to another language - due to their ability to write code in most programming languages. However, LLM's effectiveness on translating real-world code remains largely unstudied. In this work, we perform the first substantial study on LLM-based translation to Rust by assessing the ability of five state-of-the-art LLMs, GPT4, Claude 3, Claude 2.1, Gemini Pro, and Mixtral. We conduct our study on code extracted from real-world open source projects. To enable our study, we develop FLOURINE, an end-to-end code translation tool that uses differential fuzzing to check if a Rust translation is I/O equivalent to the original source program, eliminating the need for pre-existing test cases. As part of our investigation, we assess both the LLM's ability to produce an initially successful translation, as well as their capacity to fix a previously generated buggy one. If the original and the translated programs are not I/O equivalent, we apply a set of automated feedback strategies, including feedback to the LLM with counterexamples. Our results show that the most successful LLM can translate 47% of our benchmarks, and also provides insights into next steps for improvements.
Forward citations
Cited by 13 Pith papers
-
Syzygy: Dual Code-Test C to (safe) Rust Translation using LLMs and Dynamic Analysis
A dual code-and-test generation pipeline with dynamic-analysis specifications translates the 3,000-line Zopfli C library into safe Rust, though the top-level validation compares compression ratios rather than exact outputs.
-
Towards Reliable C-to-Rust Translation with Rule-Guided Reasoning and Reinforcement Learning
A rule-guided MCTS plus dual-reward reinforcement learning pipeline improves LLM-based C-to-Rust translation accuracy and cuts unsafe Rust output across three benchmarks.
-
His2Trans: A Knowledge-Guided Agentic Framework for Project-Level C-to-Rust Migration
Combining build-trace-derived skeletons with a knowledge base of historical translation pairs raises C-to-Rust compilation pass rates and lowers unsafe-code ratios, but on general benchmarks functional-correctness sco...
-
Vibration-Based Energy Metric for Restoring Needle Alignment in Autonomous Robotic Ultrasound
A vibration-based energy metric, obtained by periodically vibrating the needle, is used as an out-of-plane feedback signal to re-align the ultrasound probe with the needle plane, reporting 0.41 mm translation and 0.51...
-
Can LLMs Replace Humans During Code Chunking?
LLM-generated partitions of legacy code yield documentation that LLM judges rate as up to 20% more factual and up to 10% more useful than documentation based on human expert partitions.
-
CODEMENV: Benchmarking Large Language Models on Code Migration
CODEMENV provides 922 examples and three tasks for evaluating LLMs on cross-version code migration, finding models are much better at migrating old code to new versions (up to 43.84% pass@1) than the reverse.
-
Quantifying the benefits of code hints for refactoring deprecated Java APIs
Automated refactoring of deprecated Java APIs succeeds 71 to 82 percent of the time when Javadoc contains code hints, and at most 14 percent without them.
-
From C to Idiomatic Rust: A Ship-of-Theseus Agentic Translation
A C-to-Rust migration methodology that separates lossless c2rust transpilation from agentic per-function rewriting with compilation and behavioral gates is demonstrated on the 12.5 kSLOC iodine DNS tunnel.
-
CITER: Collaborative Inference for Efficient Large Language Model Decoding with Token-Level Routing
CITER trains a token-level router with preference optimization to route non-critical tokens to a small model and critical tokens to a large model, reducing inference cost on QA and math benchmarks.
-
Guided Debugging of Auto-Translated Code Using Differential Testing
tHinter uses coverage-guided fuzzing and differential testing to highlight lines in auto-translated code that likely contain translation errors, reducing debugging effort.
-
Autonomous Legacy Web Application Upgrades Using a Multi-Agent System
A multi-agent LLM system can update small legacy CakePHP files, but plain zero-shot and one-shot prompts are often as good or better.
-
How is Google using AI for internal code migrations?
An experience report shows Google using fine-tuned LLMs plus AST-based tooling and human review to accelerate large internal code migrations, with estimated time savings of roughly 50% to 89%.
-
Generating Move Smart Contracts based on Concepts
A concept-based knowledge graph and iterative planning, coding, and debugging agents improve LLM-generated Move smart contracts, with reported relative gains up to 47 percent over baselines.
Discussion (0). Continue with ORCID to comment.