REVIEW 6 cited by
Towards Making the Most of ChatGPT for Machine Translation
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
ChatGPT shows remarkable capabilities for machine translation (MT). Several prior studies have shown that it achieves comparable results to commercial systems for high-resource languages, but lags behind in complex tasks, e.g., low-resource and distant-language-pairs translation. However, they usually adopt simple prompts which can not fully elicit the capability of ChatGPT. In this paper, we aim to further mine ChatGPT's translation ability by revisiting several aspects: temperature, task information, and domain information, and correspondingly propose an optimal temperature setting and two (simple but effective) prompts: Task-Specific Prompts (TSP) and Domain-Specific Prompts (DSP). We show that: 1) The performance of ChatGPT depends largely on temperature, and a lower temperature usually can achieve better performance; 2) Emphasizing the task information can further improve ChatGPT's performance, particularly in complex MT tasks; 3) Introducing domain information can elicit ChatGPT's generalization ability and improve its performance in the specific domain; 4) ChatGPT tends to generate hallucinations for non-English-centric MT tasks, which can be partially addressed by our proposed prompts but still need to be highlighted for the MT/NLP community. We also explore the effects of advanced in-context learning strategies and find a (negative but interesting) observation: the powerful chain-of-thought prompt leads to word-by-word translation behavior, thus bringing significant translation degradation.
Forward citations
Cited by 6 Pith papers
-
BadMoE: Backdooring Mixture-of-Experts LLMs via Optimizing Routing Triggers and Infecting Dormant Experts
BadMoE implants backdoors into dormant experts of MoE LLMs and uses routing-trigger optimization to activate them, achieving high attack success while preserving normal accuracy.
-
Optimization is Better than Generation: Optimizing Commit Message Leveraging Human-written Commit Message
A commit-message optimization method that starts from human-written messages and uses GPT-4 plus automated evaluators outperforms message generation and completion methods on three of four quality metrics.
-
Optimising ChatGPT for creativity in literary translation: A case study from English into Dutch, Chinese, Catalan and Spanish
A simple 'translate creatively' prompt at temperature 1.0 yields the most creative ChatGPT translations in Dutch, Spanish and Chinese, but all ChatGPT outputs remain less creative and more error-prone than human translations.
-
Who Gets Recommended? Investigating Gender, Race, and Country Disparities in Paper Recommendations from Large Language Models
LLM recommendations of important AI research favor recent, well-cited, team-authored papers, but do not measurably over-represent male, white, or developed-country scholars relative to a human-curated benchmark.
-
Large Language Models for Security Operations Centers: A Comprehensive Survey
A systematic review of 138 papers classifying LLM applications in SOC workflows by phase, model family, datasets, and maturity.
-
Bangla-Bayanno: A 52K-Pair Bengali Visual Question Answering Dataset with LLM-Assisted Translation Refinement
A 52,650-pair Bengali VQA dataset built by translating VQA v2 with GPT-4, claimed as the largest open-source Bangla benchmark but weakly validated.
Discussion (0). Continue with ORCID to comment.