An automated, execution-based benchmark of 1,325 Google Earth Engine unit tests shows 18 LLMs scoring between 31.40% and 71.55% pass@1, with parameter-knowledge errors the dominant failure mode.
A review on code generation with llms: Application and evaluation
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
citation-role summary
background 1
citation-polarity summary
fields
cs.SE 1years
2025 1verdicts
CONDITIONAL 1roles
background 1polarities
background 1representative citing papers
citing papers explorer
-
AutoGEEval: A Multimodal and Automated Framework for Geospatial Code Generation on GEE with Large Language Models
An automated, execution-based benchmark of 1,325 Google Earth Engine unit tests shows 18 LLMs scoring between 31.40% and 71.55% pass@1, with parameter-knowledge errors the dominant failure mode.