REVIEW 7 cited by
Natural Language Processing for Dialects of a Language: A Survey
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
State-of-the-art natural language processing (NLP) models are trained on massive training corpora, and report a superlative performance on evaluation datasets. This survey delves into an important attribute of these datasets: the dialect of a language. Motivated by the performance degradation of NLP models for dialectal datasets and its implications for the equity of language technologies, we survey past research in NLP for dialects in terms of datasets, and approaches. We describe a wide range of NLP tasks in terms of two categories: natural language understanding (NLU) (for tasks such as dialect classification, sentiment analysis, parsing, and NLU benchmarks) and natural language generation (NLG) (for summarisation, machine translation, and dialogue systems). The survey is also broad in its coverage of languages which include English, Arabic, German, among others. We observe that past work in NLP concerning dialects goes deeper than mere dialect classification, and extends to several NLU and NLG tasks. For these tasks, we describe classical machine learning using statistical models, along with the recent deep learning-based approaches based on pre-trained language models. We expect that this survey will be useful to NLP researchers interested in building equitable language technologies by rethinking LLM benchmarks and model architectures.
Forward citations
Cited by 7 Pith papers
-
Using Contextually Aligned Online Reviews to Measure LLMs' Performance Disparities Across Language Varieties
LLMs predict sentiment worse on Taiwan Mandarin than Mainland Mandarin reviews, using a new contextually paired dataset from Booking.com.
-
MassTool: A Multi-Task Search-Based Tool Retrieval Framework for Large Language Models
A multi-task retriever that combines tool-usage detection with query-centered graph and search-based modules improves tool retrieval accuracy over prior baselines.
-
Lost in Variation? Evaluating NLI Performance in Basque and Spanish Geographical Variants
Language models are significantly worse at natural language inference when sentences are written in Basque or Spanish regional dialects, especially for Basque.
-
AL-QASIDA: Analyzing LLM Quality and Accuracy Systematically in Dialectal Arabic
LLMs understand dialectal Arabic better than they generate it, and current post-training appears to bias them toward Modern Standard Arabic.
-
An End-to-End Real-World Camera Imaging Pipeline
A single end-to-end neural network performs RAW-to-RGB conversion and image compression jointly, reporting rate-distortion gains over separate ISP-plus-codec baselines.
-
Salamandra Technical Report
Salamandra is an open, from-scratch multilingual LLM family with 2B, 7B, and 40B checkpoints, instruction-tuned variants, a vision proof-of-concept, and detailed evaluations across Iberian and European languages.
-
Unification of Balti and trans-border sister dialects in the essence of LLMs and AI Technology
The paper argues that LLMs and ASR can document and standardize the endangered Balti language and its trans-border sister dialects, but it provides no computational evidence.
Discussion (0). Continue with ORCID to comment.