REVIEW 2 cited by
Cost-Effective Hyperparameter Optimization for Large Language Model Generation Inference
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Large Language Models (LLMs) have sparked significant interest in their generative capabilities, leading to the development of various commercial applications. The high cost of using the models drives application builders to maximize the value of generation under a limited inference budget. This paper presents a study of optimizing inference hyperparameters such as the number of responses, temperature and max tokens, which significantly affects the utility/cost of text generation. We design a framework named EcoOptiGen which leverages economical hyperparameter optimization and cost-based pruning. Experiments with the GPT-3.5/GPT-4 models on a variety of tasks verify its effectiveness. EcoOptiGen is implemented in the `autogen' package of the FLAML library: \url{https://aka.ms/autogen}.
Forward citations
Cited by 2 Pith papers
-
SuperCode: Sustainability PER AI-driven CO-DEsign
The paper proposes an AI-driven hardware-software-science co-design methodology for radio astronomy, using sustainability as the key performance indicator, with no empirical results yet.
-
From Text to Pose to Image: Improving Diffusion Model Control and Quality
A text-to-pose transformer and a face-and-hand-aware pose adapter form a text-to-pose-to-image pipeline for diffusion models, beating the prior adapter baseline on 70 to 78 percent of test cases.
Discussion (0). Continue with ORCID to comment.