ParEval-Repo: A Benchmark Suite for Evaluating LLMs with Repository-level HPC Translation Tasks

Abhinav Bhatele; Daniel Nichols; Ishan Khillan; Joshua H. Davis

REVIEW 3 major objections 7 minor 30 references

ParEval-Repo: A Benchmark Suite for Evaluating LLMs with Repository-level HPC Translation Tasks

T0 review · 3 major / 7 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read ParEval-Repo, a new benchmark, shows that current LLMs fail to translate complete HPC repositories beyond tiny micro-applications.

desk verdict Useful benchmark with a striking negative result, but the missing per-cell sample counts undercut the precision of the headline zero. read the letter →

arxiv 2506.20938 v2 pith:LKIMNLLV submitted 2025-06-26 cs.DC

Joshua H. Davis , Daniel Nichols , Ishan Khillan , Abhinav Bhatele This is my paper · ORCID

classification cs.DC

keywords LLMprogramtranslationrepository-levelGPGPUprogrammingmodelsOpenMPoffloadKokkosCUDAbenchmarksuiteexpectedtokencost

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

ParEval-Repo is a benchmark for testing whether large language models can translate entire HPC software repositories from one GPU programming model to another (CUDA to OpenMP offload, CUDA to Kokkos, and OpenMP threads to OpenMP offload). The paper evaluates five LLMs with three strategies—file-by-file translation, a top-down multi-agent pipeline, and SWE-agent—across six applications ranging from roughly 100 to 3,000 lines. Its central finding is negative: no LLM and translation technique combination achieved a pass@k above 0 for any application larger than the tiny microXOR, and the main failure points were generated build systems and inconsistent interfaces across files. The benchmark matters because it isolates whether LLM translation can scale from single kernels to whole codebases, which is the real porting burden in HPC.

What carries the argument

The load-bearing object is the ParEval-Repo task suite itself: six applications deliberately spanning size, file count, and dependency structure (nanoXOR, microXORh, microXOR, SimpleMOC-kernel, XSBench, llm.c) with the contamination guard that all but XSBench lack a public translation in the target programming model. Correctness is measured by build@1 and pass@1, extensions of the pass@k estimator from prior work: build@1 is the chance a single translation compiles, and pass@1 is the chance it compiles and passes the application's reference tests on an A100. The token-economy metric $E_\kappa$ is the expected number of inference tokens needed for one correct translation, computed as $(1/\mathrm{pass}@1)$ times the average tokens per generation, and it lets the authors compare cost across LLMs and techniques.

What would settle it

A single correct end-to-end translation of an application larger than microXOR—for example, XSBench from CUDA to Kokkos using an LLM-generated CMake build that passes XSBench's reference tests—would refute the paper's core negative result; alternatively, discovering a pre-cutoff public port of SimpleMOC-kernel or llm.c in the target programming model would undermine the contamination guard.

Watch

Extended reading notes

Core claim

The paper's claim is that repository-scale translation is not yet viable: across all tested LLMs (Gemini 1.5 Flash, GPT-4o mini, o4 mini, Llama 3.3 70B, QwQ-32B) and all three translation techniques, no configuration achieved a pass@k above 0 for any application larger than microXOR, and even for the micro-applications success was sporadic and model-dependent. Separating 'code-only' scores (using a hand-written ground-truth build system) from 'overall' scores (using the LLM-generated build system) shows that the gap is dominated by build-system generation: LLM-produced Makefiles and CMakeLists.txt fail with configuration errors, invalid compiler flags, and missing targets, while cross-file mismatches in function names and types account for many remaining failures. The paper also introduces $E_\kappa$, the expected token cost of a correct translation, and reports that non-agentic o4-mini is the cheapest commercial option and non-agentic Llama-3.3 the cheapest open-source option among configurations that succeeded at all.

Load-bearing premise

The paper's anti-contamination premise is that for every task except XSBench no public translation in the target programming model exists, so the LLM cannot simply recite memorized output; this cannot be verified for opaque commercial training corpora, and if a public port is actually in the training data, the reported scores would reflect memorization rather than translation ability.

Editorial extensions

If this is right

LLM-based automated porting is not yet a substitute for human effort on real HPC codebases; at best it works for single-file micro-applications.
Researchers should focus on build-system generation and cross-file interface repair rather than on kernel-level translation, since code-only scores are much higher than overall scores.
Benchmark users should report code-only and overall scores separately, because conflating kernel translation with build-system generation hides where the failure lies.
$E_\kappa$ provides a practical cost target: future methods must reduce expected tokens to a correct translation, not just raise pass@1, to be economically useful.
The contamination-aware task selection sets a template for future translation benchmarks, with XSBench as a built-in control for measuring how much public ports help.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The code-only versus overall gap suggests a directed remedy the paper leaves implicit: generate the source translation with the LLM and synthesize the build system with a separate, build-system-specialized tool, which could push larger tasks from 0 to positive pass@1.
Because commercial LLM training data is opaque, the benchmark's contamination guard is a design guarantee, not an empirical one; a version of ParEval-Repo that creates fresh private repositories with unique identifiers would make memorization impossible to hide.
$E_\kappa$ generalizes beyond HPC: any code-generation task billed per token could use expected-token-cost as the economic objective, especially in agentic settings where reasoning models emit large overhead.
The finding that SWE-agent cannot handle Makefiles suggests that agentic software-engineering tools tuned to Python ecosystems will need HPC-aware file handling before they can be applied to scientific codebases.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

First, the good news: this is the first benchmark I know of that targets repository-level translation between GPU programming models, and it includes build system generation, which prior work (RepoTransBench, LASSI) mostly ignores. The authors evaluate a reasonable spread of LLMs and three translation techniques, and they test actual compilation and execution rather than CodeBLEU similarity. The headline result—no LLM/technique combination gets a pass@k > 0 for any application larger than microXOR—is striking and, if even approximately true, useful for the community. The error clustering analysis is also valuable: it shows build systems and cross-file interfaces are the main bottlenecks, not kernel translation itself.

The main soft spot is the missing sample count. The paper defines pass@k with N samples per task but never reports N, k, or any confidence interval. A single cell showing 0 with N samples has an upper 95% bound of roughly 3/N; if N is 25—which the heatmap increments suggest—a true pass@1 as high as 0.12 is compatible with the observed zero. That directly weakens the universal zero claim in Section 8.2. The authors should report N per cell and, ideally, confidence intervals or at least a rule-of-three bound. This is an addressable revision, not a fatal flaw, because the qualitative pattern—larger applications fail—is robust across many cells and techniques.

The stress-test note also raises the anti-contamination assumption. I think that concern lands less well: contamination can only inflate pass@k, and the positive results are on custom micro-apps with no public translations. The deliberately contaminated XSBench case also yields zero, so leakage is not a plausible driver of the negative finding. So I'd not weight that heavily.

One more minor thing: the E_kappa metric is only computed where pass@1 > 0, so it doesn't apply to the large applications. That's fine, but it means the cost analysis is limited to the trivial cases.

Overall, this is a solid empirical paper. The missing N is the one thing I'd want fixed before relying on the exact zero. The benchmark itself is a real contribution and likely to be used. I'd take it for peer review and would cite it in work on LLM-based HPC translation. It's a good reading-group paper too, mostly for the negative result and the error taxonomy.

Referee Report

3 major / 7 minor

Summary. ParEval-Repo is a benchmark suite for evaluating LLM-based translation of entire HPC repositories across GPGPU programming models. The paper defines 16 translation tasks spanning six applications (nanoXOR, microXORh, microXOR, SimpleMOC-kernel, XSBench, llm.c) and three translation pairs (CUDA to OpenMP Offload, CUDA to Kokkos, OpenMP Threads to OpenMP Offload). It evaluates five LLMs under three techniques: a non-agentic file-by-file translator, a top-down agentic pipeline, and SWE-agent. Correctness is measured with pass@k and build@k, and the paper proposes an expected token cost metric E_kappa. The headline finding is that no combination of LLM and technique produces a correct translation for any application larger than microXOR, with build-system generation and cross-file consistency identified as the main obstacles. The paper also contributes a semi-automated error-clustering analysis of build and run failures.

Significance. If the headline negative result is statistically well supported, the paper makes a valuable and timely contribution: it provides a concrete, reproducible benchmark for repository-level HPC translation and suggests that current LLM-based approaches are not yet viable for non-trivial codebases. The benchmark design is thoughtful in several respects: tasks are selected to avoid public reference translations, correctness is checked with real test cases, and code-only versus overall scoring separates source-translation ability from build-system generation. The public release of the benchmark is a strength, and the E_kappa metric is a simple but useful cost-aware comparison tool. The error taxonomy, while secondary, gives actionable insight into recurring failures. However, the strength of the central zero-result claim currently exceeds the statistical evidence: the number of samples per task is never reported, many heatmap cells are empty, and the error classification relies on a manually adjusted clustering step.

major comments (3)

[§2.2, Eq. (1); §8.2, Fig. 2] The paper defines pass@k in Eq. (1) in terms of N and c_t but never reports N, k, or per-cell confidence intervals anywhere in the text or figures. This is load-bearing for the headline result in §8.2: 'no combination of translation technique and LLM achieves a pass@k above 0 for any application larger than microXOR.' An observed zero with N samples has an approximate 95% upper bound of about 3/N by the rule of three; if N=25, true pass@1 values up to 0.12 are compatible with the reported zeros, and if N=10 the bound exceeds 0.25. The heatmap granularity in Fig. 2 (values in increments such as 0.02, 0.04, and 0.76) suggests N may be 25 or 50, but this is not stated. Please report per-cell N (or raw counts c_t and b_t), state k explicitly, and provide confidence intervals for the zero cells and for nonzero point estimates, such as the 0.76 versus 0.2 comparison for Llama-3.3 on microXORh versus nanoXOR in the CUDA-to-OpenMP code-only panel. Without this information, the universal zero cannot be distinguished from a small positive rate missed by a small sample.
[§8.2; Fig. 2 and §7.2] The universal negative statement overstates the coverage of the completed experiments. The paper correctly distinguishes empty cells ('we do not run that case') from zero cells, but many cells for SimpleMOC-kernel, XSBench, and llm.c are empty because of context-window overflows, node-hour budgets, or API budgets. Examples stated in §8.2 include Gemini and GPT-4o for llm.c, Gemini for XSBench (CUDA to OpenMP Offload), QwQ for XSBench and llm.c across all translation pairs, Llama-3.3 for XSBench and llm.c (CUDA to Kokkos), and SWE-agent for XSBench and llm.c. The sentence 'no combination of translation technique and LLM achieves a pass@k above 0 for any application larger than microXOR' should therefore be scoped to the completed configurations, with the number of completed cells stated and the unrun cells explicitly excluded. As written, it reads as a claim about the full Cartesian product of techniques, LLMs, applications, and translation pairs, which the data do not cover.
[§6.3; Fig. 3] The error taxonomy that supports the conclusions about build-system and cross-file-dependency failures is produced by a semi-automated pipeline whose subjective components are not fully specified. The paper states that DBSCAN hyperparameters were tuned by manual inspection, that the algorithm produced many clusters, and that a manual pass merged clusters, reassigned samples, and assigned labels. The paper does not report the hyperparameter values, the number of clusters before and after merging, the corpus used to train the word2vec embeddings, or a release artifact containing the raw logs and the cluster-to-label mapping. Please provide these details, or make the raw build/run logs and the post-merge cluster assignments available, so that the counts in Fig. 3 and the qualitative findings in §8.3 can be independently audited. This issue is secondary to the pass@k claim, but the error analysis is a stated contribution of the paper.

minor comments (7)

[§2.2, Eq. (1)] In Eq. (1), the summation index p in T is inconsistent with the per-task counts c_t; please use c_p or make the dependence on the summed task explicit.
[Fig. 2] The overall caption of Fig. 2 says 'Correctness metrics for OpenMP Threads to OpenMP Offload tasks,' but subfigures (a)-(d) are CUDA to OpenMP Offload and CUDA to Kokkos; the caption should be corrected to reflect the three translation pairs.
[Abstract; §8.2] The abstract and the results text refer to pass@k, but the presented heatmaps show only pass@1 and build@1. Please clarify whether k=1 is the only reported value, and if larger k were computed, report them as well.
[§5.1] The claim that all but one task has no publicly available translation in the target programming model is difficult to verify for commercial training corpora; please state how this absence was checked (e.g., repository and web searches) and acknowledge the residual contamination risk.
[§8.2] The phrase 'consistently significantly lower' is used without statistical testing; either add appropriate tests or use a non-statistical comparator such as 'consistently lower in the measured cells.'
[§6.2 and §8.4] E_kappa is aggregated only over cases with pass@1 greater than 0; please state this limitation directly where the metric is introduced in §6.2 so that readers do not interpret empty cells in Fig. 5 as zero expected cost.
[§8.4; Fig. 3] Please fix the typo 'estiamtes' in §8.4 and add axis labels and a color-scale label to Fig. 3 for readability.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper's results are measured benchmark outcomes and derived cost statistics, not predictions reducible to fitted inputs.

full rationale

The paper's load-bearing claims are empirical measurements, not derivations from fitted parameters. pass@k and build@k are computed by generating translations, compiling them, and running each application's own test cases; no parameter is fitted to the outcome and then renamed as a prediction. The E_kappa metric (Eq. 2) is explicitly a definition: expected generations (1/pass@1) multiplied by average tokens per generation. Both factors are measured quantities, so E_kappa is a descriptive arithmetic combination, not a fitted predictor. The only self-citations are to prior work for the pass@k metric [18] and for the general observation that portable GPU programming models complicate compilation [6]; neither carries the central negative result, which is that no translation technique plus LLM exceeds pass@k = 0 for applications larger than microXOR. The anti-contamination design is an assumption about LLM training corpora, not a derivation step, and its possible failure could only inflate positive results, not explain the negative finding. The skeptic's concern about unreported sample counts (N) is a legitimate statistical-reporting limitation: with an observed zero, the true pass@1 could be small but nonzero. However, that concerns the strength and confidence of an empirical zero, not a reduction of the result to its own inputs, so it is a correctness/statistical issue rather than circularity.

Assumptions & free parameters 1 free parameters · 3 assumptions · 2 invented entities

The paper is an empirical benchmark study; its central claim rests on design assumptions rather than fitted parameters. The only hand-tuned numbers are the DBSCAN clustering hyperparameters used for the secondary error analysis. The primary pass@1 results are point estimates without stated sample counts, which is flagged in red_flags.

free parameters (1)

DBSCAN hyperparameters (eps, min_samples) = not reported
Hand-tuned in Sec 6.3 to cluster build and run logs; influences the error category counts in Fig 3, not the primary pass@1 results.

assumptions (3)

domain assumption For every ParEval-Repo task except XSBench, no public translation in the target programming model exists
Section 5.1 states this is the selection rule used to avoid data contamination; if it fails, LLMs may have memorized the expected output.
standard math The pass@k estimator of Eq. 1 correctly measures translation ability given the project's own test suite
Adopted from prior work [18] in Section 2.2; assumes correctness equals passing the provided tests.
domain assumption The test cases bundled with each application are complete enough to validate functional equivalence
Section 5 states the authors use the validation tests from the application developers; for the three custom micro-apps the authors wrote the tests themselves.

invented entities (2)

ParEval-Repo benchmark suite
purpose: Provides 16 translation tasks across six applications and three programming model pairs
Introduced by this paper; its usefulness is the claim under evaluation, not established elsewhere.
E_kappa (expected token cost) metric
purpose: Combines pass@1 and token cost to estimate expected tokens for a successful translation
New metric defined in Sec 6.2; no independent validation that it predicts real costs.

how reviews work

0 comments

Cite this review

Pith. "Pith review of ParEval-Repo: A Benchmark Suite for Evaluating LLMs with Repository-level HPC Translation Tasks." pith.science (2026). https://pith.science/paper/LKIMNLLV

@misc{pith2026250620938,
  author       = {Pith},
  title        = {Pith review of: ParEval-Repo: A Benchmark Suite for Evaluating LLMs with Repository-level HPC Translation Tasks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LKIMNLLV}},
  note         = {Machine review of arXiv:2506.20938}
}

read the original abstract

GPGPU architectures have become significantly more diverse in recent years, which has led to an emergence of a variety of specialized programming models and software stacks to support them. Portable programming models exist, but they require significant developer effort to port to and optimize for different hardware architectures. Large language models (LLMs) may help to reduce this programmer burden. In this paper, we present a novel benchmark and testing framework, ParEval-Repo, which can be used to evaluate the efficacy of LLM-based approaches in automatically translating entire codebases across GPGPU execution models. ParEval-Repo includes several scientific computing and AI mini-applications in a range of programming models and levels of repository complexity. We use ParEval-Repo to evaluate a range of state-of-the-art open-source and commercial LLMs, with both a non-agentic and a top-down agentic approach. We assess code generated by the LLMs and approaches in terms of compilability, functional correctness, categories of build errors, and the cost of translation in terms of the number of inference tokens. Our results demonstrate that LLM translation of scientific applications is feasible for small programs but difficulty with generating functional build systems and cross-file dependencies pose challenges in scaling to larger codebases.

Figures

Figures reproduced from arXiv: 2506.20938 by the authors.

**Figure 2.** Correctness metrics for OpenMP Threads to OpenMP Offload tasks. [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗

**Figure 3.** Count of categories of errors encountered when [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗

Figures from the paper (2 more)

**Figure 4.** Figure 4: Total inference tokens used in translation, aver [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]

**Figure 5.** Figure 5: Expected tokens needed for successful translation [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

30 extracted references · 20 canonical work pages

[1]

Llama 3.3 Model Card and Prompt Format

2024. Llama 3.3 Model Card and Prompt Format. https://www.llama.com/docs/ model-cards-and-prompt-formats/llama3_3/

work page 2024
[2]

QwQ: Reflect Deeply on the Boundaries of the Unknown

2024. QwQ: Reflect Deeply on the Boundaries of the Unknown. https://qwenlm. github.io/blog/qwq-32b-preview/

work page 2024
[3]

o3 and o4-mini System Card

2025. o3 and o4-mini System Card. https://cdn.openai.com/pdf/2221c875-02dc- 4789-800b-e7758f3722c1/o3-and-o4-mini-system-card.pdf

work page 2025
[4]

Waseem Ahmed, Mohsin Khan, Adeel Ahmed Khan, Rashid Mehmood, Abdullah Algarni, Aiiad Albeshri, and Iyad Katib. 2018. A Framework for Faster Porting of Scientific Applications Between Heterogeneous Clouds. InSmart Societies, Infras- tructure, Technologies and Applications, Rashid Mehmood, Budhendra Bhaduri, Iyad Katib, and Imrich Chlamtac (Eds.). Springer ...

work page 2018
[5]

Joshua H Davis, Justin Shafner, Daniel Nichols, Nathan Grube, Pino Martin, and Abhinav Bhatele. 2023. Porting a computational fluid dynamics code with amr to large-scale gpu platforms. In2023 IEEE International Parallel and Distributed Processing Symposium (IPDPS). IEEE, 602–612

work page 2023
[6]

Davis, Pranav Sivaraman, Joy Kitson, Konstantinos Parasyris, Harshitha Menon, Isaac Minn, Giorgis Georgakoudis, and Abhinav Bhatele

Joshua H. Davis, Pranav Sivaraman, Joy Kitson, Konstantinos Parasyris, Harshitha Menon, Isaac Minn, Giorgis Georgakoudis, and Abhinav Bhatele. 2025 (to ap- pear). Taking GPU Programming Models to Task for Performance Portability. In Proceedings of the International Conference on Supercomputing (ICS ’25)

work page 2025
[7]

Matthew T Dearing, Yiheng Tao, Xingfu Wu, Zhiling Lan, and Valerie Taylor. 2024. LASSI: An LLM-based Automated Self-Correcting Pipeline for Translating Parallel Scientific Codes. In2024 IEEE International Conference on Cluster Computing Workshops (CLUSTER Workshops). IEEE, 136–143

work page 2024
[8]

DeepSeek-AI. 2025. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. arXiv:2501.12948 [cs.CL] https://arxiv.org/abs/2501. 12948

arXiv 2025

Show all 30 references

[9]

Akash Dhruv and Anshu Dubey. 2025. Leveraging Large Language Mod- els for Code Translation and Software Development in Scientific Computing. arXiv:2410.24119 [cs.SE] https://arxiv.org/abs/2410.24119

2025
[10]

2024.The Llama 3 Herd of Models

Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024.The Llama 3 Herd of Models. Technical Report

2024
[11]

Martin Ester, Hans-Peter Kriegel, Jörg Sander, and Xiaowei Xu. 1996. A density- based algorithm for discovering clusters in large spatial databases with noise. In Proceedings of the Second International Conference on Knowledge Discovery and Data Mining(Portland, Oregon)(KDD’96...

1996
[12]

William F Godoy, Pedro Valero-Lara, Keita Teranishi, Prasanna Balaprakash, and Jeffrey S Vetter. 2024. Large language model evaluation for high-performance computing software development.Concurrency and Computation: Practice and Experience36, 26 (2024), e8269

2024
[13]

Ali Reza Ibrahimzada, Kaiyao Ke, Mrigank Pawagi, Muhammad Salman Abid, Rangeet Pan, Saurabh Sinha, and Reyhaneh Jabbarvand. 2024. Repository-level compositional code translation and validation. arXiv:2410.24117 [cs.SE] https: //arxiv.org/abs/2410.24117

2024 arXiv
[14]

Bin Lei, Caiwen Ding, Le Chen, Pei-Hung Lin, and Chunhua Liao. 2023. Creating a Dataset for High-Performance Computing Code Translation using LLMs: A Bridge Between OpenMP Fortran and C++. arXiv:2307.07686 [cs.SE] https: //arxiv.org/abs/2307.07686

2023 arXiv
[15]

Nicholas Malaya, Bronson Messer, Joseph Glenski, Antigoni Georgiadou, Justin Lietz, Kalyana Gottiparthi, Marc Day, Jackie Chen, Jon Rood, Lucas Esclapez, et al. 2023. Experiences readying applications for exascale. InProceedings of the International Conference for High Perform...

2023
[16]

Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient Estimation of Word Representations in Vector Space. arXiv:1301.3781 [cs.CL] https://arxiv.org/abs/1301.3781

2013 arXiv
[17]

Christian Munley, Aaron Jarmusch, and Sunita Chandrasekaran. 2023. LLM4VV: Developing LLM-Driven Testsuite for Compiler Validation. arXiv:2310.04963 [cs.AI]

2023 arXiv
[18]

Davis, Zhaojun Xie, Arjun Rajaram, and Abhinav Bhatele

Daniel Nichols, Joshua H. Davis, Zhaojun Xie, Arjun Rajaram, and Abhinav Bhatele. 2024. Can Large Language Models Write Parallel Code?. InProceedings of the 33rd International Symposium on High-Performance Parallel and Distributed Computing (HPDC ’24). Association for Computin...

2024
[19]

OpenAI, Aaron Hurst, and et al. 2024. GPT-4o System Card. arXiv:2410.21276 [cs.CL] https://arxiv.org/abs/2410.21276

2024 arXiv
[20]

OpenMP Application Program Interface

OpenMP4 2013. OpenMP Application Program Interface. Version 4.0. July 2013

2013
[21]

Gemini Team. 2023. Gemini: A Family of Highly Capable Multimodal Models. arXiv:2312.11805 [cs.CL]

2023 arXiv
[22]

Ali Tehrani, Arijit Bhattacharjee, Le Chen, Nesreen K Ahmed, Amir Yazdan- bakhsh, and Ali Jannesari. 2024. CodeRosetta: Pushing the Boundaries of Un- supervised Code Translation for Parallel Programming.Advances in Neural Information Processing Systems37 (2024), 100965–100999

2024
[23]

Tramm, Geoffrey Gunow, Tim He, Kord S

John R. Tramm, Geoffrey Gunow, Tim He, Kord S. Smith, Benoit Forget, and Andrew R. Siegel. 2016. A task-based parallelism and vectorized approach to 3D Method of Characteristics (MOC) reactor simulation for high performance computing architectures.Computer Physics Communicatio...

2016
[24]

John R Tramm, Andrew R Siegel, Tanzima Islam, and Martin Schulz. 2014. XSBench-the development and verification of a performance abstraction for Monte Carlo reactor analysis.The Role of Reactor Physics toward a Sustainable Future (PHYSOR)(2014)

2014
[25]

Trott, Damien Lebrun-Grandié, Daniel Arndt, Jan Ciesko, Vinh Dang, Nathan Ellingwood, Rahulkumar Gayatri, Evan Harvey, Daisy S

Christian R. Trott, Damien Lebrun-Grandié, Daniel Arndt, Jan Ciesko, Vinh Dang, Nathan Ellingwood, Rahulkumar Gayatri, Evan Harvey, Daisy S. Hollman, Dan Ibanez, Nevin Liber, Jonathan Madsen, Jeff Miles, David Poliakoff, Amy Powell, Sivasankaran Rajamanickam, Mikael Simberg, D...

2022
[26]

Pedro Valero-Lara, William F Godoy, Keita Teranishi, Prasanna Balaprakash, and Jeffrey S Vetter. 2024. ChatBLAS: The First AI-Generated and Portable BLAS Library. InSC24-W: Workshops of the International Conference for High Performance Computing, Networking, Storage and Analys...

2024
[27]

Ben Van Werkhoven and Pieter Hijma. 2015. An integrated approach to porting large scientific applications to GPUs. In2015 IEEE 11th International Conference on e-Science. IEEE, 57–66

2015
[28]

Yanli Wang, Yanlin Wang, Suiquan Wang, Daya Guo, Jiachi Chen, John Grundy, Xilin Liu, Yuchi Ma, Mingzhi Mao, Hongyu Zhang, and Zibin Zheng. 2024. Repo- TransBench: A Real-World Benchmark for Repository-Level Code Translation. arXiv:2412.17744 [cs.SE] https://arxiv.org/abs/2412.17744

2024
[29]

John Yang, Carlos Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press. 2024. Swe-agent: Agent-computer interfaces enable automated software engineering.Advances in Neural Information Processing Systems37 (2024), 50528–50652

2024
[150]

doi:10.1016/j.cpc.2016.01.007

2016 doi

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

Llama 3.3 Model Card and Prompt Format

2024. Llama 3.3 Model Card and Prompt Format. https://www.llama.com/docs/ model-cards-and-prompt-formats/llama3_3/

work page 2024

[2] [2]

QwQ: Reflect Deeply on the Boundaries of the Unknown

2024. QwQ: Reflect Deeply on the Boundaries of the Unknown. https://qwenlm. github.io/blog/qwq-32b-preview/

work page 2024

[3] [3]

o3 and o4-mini System Card

2025. o3 and o4-mini System Card. https://cdn.openai.com/pdf/2221c875-02dc- 4789-800b-e7758f3722c1/o3-and-o4-mini-system-card.pdf

work page 2025

[4] [4]

Waseem Ahmed, Mohsin Khan, Adeel Ahmed Khan, Rashid Mehmood, Abdullah Algarni, Aiiad Albeshri, and Iyad Katib. 2018. A Framework for Faster Porting of Scientific Applications Between Heterogeneous Clouds. InSmart Societies, Infras- tructure, Technologies and Applications, Rashid Mehmood, Budhendra Bhaduri, Iyad Katib, and Imrich Chlamtac (Eds.). Springer ...

work page 2018

[5] [5]

Joshua H Davis, Justin Shafner, Daniel Nichols, Nathan Grube, Pino Martin, and Abhinav Bhatele. 2023. Porting a computational fluid dynamics code with amr to large-scale gpu platforms. In2023 IEEE International Parallel and Distributed Processing Symposium (IPDPS). IEEE, 602–612

work page 2023

[6] [6]

Davis, Pranav Sivaraman, Joy Kitson, Konstantinos Parasyris, Harshitha Menon, Isaac Minn, Giorgis Georgakoudis, and Abhinav Bhatele

Joshua H. Davis, Pranav Sivaraman, Joy Kitson, Konstantinos Parasyris, Harshitha Menon, Isaac Minn, Giorgis Georgakoudis, and Abhinav Bhatele. 2025 (to ap- pear). Taking GPU Programming Models to Task for Performance Portability. In Proceedings of the International Conference on Supercomputing (ICS ’25)

work page 2025

[7] [7]

Matthew T Dearing, Yiheng Tao, Xingfu Wu, Zhiling Lan, and Valerie Taylor. 2024. LASSI: An LLM-based Automated Self-Correcting Pipeline for Translating Parallel Scientific Codes. In2024 IEEE International Conference on Cluster Computing Workshops (CLUSTER Workshops). IEEE, 136–143

work page 2024

[8] [8]

DeepSeek-AI. 2025. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. arXiv:2501.12948 [cs.CL] https://arxiv.org/abs/2501. 12948

arXiv 2025