Learning Beyond Limits: Multitask Learning and Synthetic Data for Low-Resource Canonical Morpheme Segmentation

Changbing Yang; Garrett Nicolai

REVIEW 3 major objections 5 minor 15 references

Learning Beyond Limits: Multitask Learning and Synthetic Data for Low-Resource Canonical Morpheme Segmentation

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Multitask learning with glosses and LLM-generated training examples improves low-resource canonical morpheme segmentation by about 3.6 average points and cuts edit distance by roughly 30 percent.

desk verdict The multitask glossing result is real and worth refereeing; the LLM synthetic-data claim is compromised until leakage is ruled out. read the letter →

arxiv 2505.16800 v1 pith:3KOTTRBQ submitted 2025-05-22 cs.CL

Changbing Yang , Garrett Nicolai This is my paper · ORCID

classification cs.CL MSC 68T5068T07

keywords canonicalmorphemesegmentationlow-resourcelanguagesmultitasklearningsyntheticdatain-contextinterlinearglossedtexttransformermorphologicalgeneralization

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that two sources of already available information can improve canonical morpheme segmentation in low-resource languages: the gloss tier of interlinear glossed texts, used as an auxiliary training objective, and synthetic word examples generated by a large language model. Canonical segmentation restores underlying morphemes from surface forms, such as 'happiness' to 'happy-ness', and it is a costly step in language documentation when annotated data are scarce. Across seven languages from the SIGMORPHON 2023 shared task, the multitask model improves average word accuracy from 62.95 to 66.59 and morpheme F1 from 77.71 to 81.35, while reducing total edit distance by about 30 percent; adding LLM-generated examples at 75 percent of the gold data pushes the averages to 67.82 and 81.96. The result matters because it suggests documentation workflows can obtain better segmentation without additional human annotation.

What carries the argument

The load-bearing mechanism is a shared encoder with two task decoders: the input is the orthographic surface word, one decoder emits the canonical segmentation, and the other emits the morpheme glosses, trained with the joint loss $L_{\text{total}} = \lambda L_{\text{seg}} + (1-\lambda)L_{\text{gloss}}$, with $\lambda$ tuned near 0.9. A second mechanism is prompt-based synthetic-data generation: a large language model receives a stem, its meaning, example segmentations and glosses from the training set, and the list of grammatical morphemes, then outputs new triples that approximate interlinear glossed text. Together the two mechanisms add information without new human annotation, since glosses are already part of the documentary format and the synthetic examples expand coverage of rare or unseen morpheme combinations.

What would settle it

Have a linguist annotate a sample of the generated words for grammatical validity, or test the model on held-out words with unseen stems; if most generated forms are judged invalid or accuracy on unseen stems does not beat the baseline, the synthetic-data gains are pattern repetition rather than morphological learning.

Watch

Extended reading notes

Core claim

The paper's central claim is that a transformer sequence-to-sequence model with one shared encoder and two decoders, one emitting canonical segmentation and one emitting morpheme glosses, learns morphological structure better than a single-task segmenter, and that adding LLM-synthesized surface-form/segmentation/gloss triples as training data pushes accuracy further. The shared latent representation forces the model to encode information that predicts both the boundaries and the grammatical labels, which the authors observe reduces the unnecessary orthographic changes that a single-task baseline tends to make. On the low-resource languages, the multitask model reaches 66.59 average word accuracy and 81.35 morpheme F1 versus 62.95 and 77.71 for the baseline; adding synthetic examples at 75 percent of training size brings the averages to 67.82 and 81.96. The authors attribute the gains to better generalization from the glossing signal and to increased morphological coverage from the generated words, while noting that the synthetic gains may partly come from exposure to frequent patterns rather than genuine morphological generalization.

Load-bearing premise

The load-bearing premise is that the LLM-generated surface forms, segmentations, and glosses are linguistically faithful; if they are hallucinated or only repeat frequent patterns, the synthetic-data improvements will not transfer to real morphology.

Editorial extensions

If this is right

A system that sees only the orthographic word can be trained to predict both the segmentation and the gloss, because both tiers already exist in interlinear glossed texts and require no extra curation.
Multitask learning alone raises average word accuracy from 62.95 to 66.59 and morpheme F1 from 77.71 to 81.35, cutting total edit distance by about 30 percent.
Adding LLM-generated examples at 75 percent of the gold data brings average accuracy to 67.82 and F1 to 81.96, with the largest relative gains in the sparsest languages such as Gitksan.
The learning-curve result implies the benefit of multitask learning grows as more annotated data accumulates, so adopting the model early in a documentation project compounds with later data collection.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The paper's reported gains from synthetic data would be on firmer ground if a linguist's error analysis confirmed the generated forms; otherwise the improvements could come from exposure to frequent character patterns rather than true morphological generalization.
The same shared-encoder design could use other IGT tiers, such as the translation line or part-of-speech labels, as additional auxiliary tasks whenever those tiers are documented.
A sharper stress test would measure accuracy on held-out words whose stems never appear in the training set; if the gain disappears there, the synthetic data is not teaching new morphology.
Because the LLM prompt only recombines stems and affixes seen in training, this method will not create novel stems; pairing it with rule-based generation or human-in-the-loop correction would extend coverage to out-of-vocabulary forms.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The substantive contribution here is the multitask model: one encoder, two decoders, jointly predicting canonical segmentation and glosses. It beats the single-task Fairseq baseline on all seven languages on average, and it beats the reported TAMS numbers on the three languages TAMS covers. That is a clean, plausible result, and it does not depend on anything exotic. The learning-curve analysis is also a nice touch—shows the benefit grows with data, which is what you'd want if the mechanism is shared linguistic structure.

The synthetic-data half is where I part ways with the paper's own framing. The abstract says the approach 'significantly improves' segmentation, but Table 3 tells a messier story: several languages get worse at some augmentation ratios (Lezgi F1 drops at 0.5 and 0.75, Tsez accuracy drops, Nyangbo F1 drops at every ratio). The averages improve modestly, but the headline overstates it. More importantly, the stress-test note is correct: there is no reported filtering of LLM-generated surface forms against the dev or test splits. Given the prompt combines a real stem with 2–5 grammatical morphemes from the same dataset, and test sets are tiny (Gitksan has 109 words), exact overlap is not just possible but likely. If even a handful of test words appear in the augmented training set, the word-accuracy and edit-distance gains become artifacts. The Limitations section admits the generation process is limited to existing vocabulary but never mentions checking overlap. That is a load-bearing gap, and it is directly checkable once the code and data are released.

The other soft spots are minor by comparison: no error bars or multi-seed runs, the code link is a placeholder, and the splits are withheld. These are standard for a workshop-style submission but should be fixed in a journal version.

The multitask result alone is a solid empirical contribution for computational morphology and documentary linguistics. The synthetic-data contribution should be treated as unverified until the leakage check is done. I would send this to peer review, but I would ask the authors to release code, splits, and generated data, and to add a filtering step plus a leakage audit. If they can show the gains survive that, the paper becomes a useful reference. If not, the multitask half still stands on its own.

Referee Report

3 major / 5 minor

Summary. The paper proposes a transformer-based sequence-to-sequence model for canonical morpheme segmentation, augmented in two ways: a multitask objective that jointly predicts the gloss tier alongside the segmentation, and synthetic training triples generated by GPT-4o using in-context learning from training-set stems and grammatical morphemes. Experiments are reported on seven languages from the SIGMORPHON 2023 dataset, with comparisons to a single-task Fairseq baseline and to the previously published TAMS system on three shared languages. The multitask model outperforms the baseline on average across languages, and adding synthetic data at 25%, 50%, and 75% of the training size yields mixed but generally positive gains. The paper also reports learning-curve experiments and a single-task ablation for the synthetic-data component.

Significance. If the results hold, the multitask contribution is a clean and practically useful finding: using the gloss tier, which is already part of IGT documentation, as auxiliary supervision improves canonical segmentation without additional annotation, and the gains appear consistent across languages and data sizes. The synthetic-data contribution is potentially interesting but is currently under-supported; the generation procedure is novel in using LLMs for this task, yet the absence of a leakage check and the lack of statistical rigor leave the reported gains uncertain. The paper is commendably explicit about its limitations, particularly the risk of LLM hallucination and the restriction to existing vocabulary, which is a strength in transparency.

major comments (3)

[Section 2.2 / Section 3.3, Table 3] The manuscript never reports filtering LLM-generated synthetic examples against the dev or test splits. Because generation uses stems and grammatical morphemes extracted from the training split, the LLM can reproduce surface forms that occur in the held-out test portion of the same dataset; with small test sets (e.g., Gitksan has 109 test words), even a few exact matches could explain the observed gains (e.g., Gitksan ACC rising from 52.29 to 56.88 at 75% augmentation). The code and generated data are behind a placeholder link, so this cannot currently be checked. The authors should add an explicit overlap-removal step or provide an analysis showing that no generated surface form coincides with any dev/test form; without this, the synthetic-data claim is not distinguishable from leakage.
[Section 3, Tables 2, 3, and 5] All reported results are single runs with no error bars, seeds, or significance tests. The abstract and conclusions state that the approach 'significantly improves' performance, but the numerical differences are often small (e.g., average ACC 66.59 for the multitask model vs. 66.63 at 25% augmentation in Table 3), and some language-metric cells degrade with synthetic data (e.g., Lezgi F1 at 75% augmentation, Nyangbo F1 at 50% augmentation). Multi-seed experiments with standard deviations, or paired significance tests across the same train/dev/test splits, are needed to establish which gains are reliable.
[Section 2.1, Eq. (1), and Appendix A.2] The text states that the segmentation loss weight λ is tuned within the range 0.8 to 15, but λ is defined as the weight on L_seg with (1−λ) on L_gloss, so values above 1 make the gloss weight negative, and the appendix only reports λ from 0.5 to 1.0. This appears to be a typo, but since the multitask results depend directly on this weighting, the reported range should be corrected and the λ values used for each language should be reported alongside the final results.

minor comments (5)

[Section 3.3] The text says 'We report the results in Table 5,' but the relevant table is labeled Table 3; the appendix contains a separate Table 5, so the cross-reference should be fixed.
[Section 2.1] The word 'complemetary' should be spelled 'complementary'.
[Table 1] Number formatting is inconsistent: '3,558' appears with a comma while other numbers do not use thousands separators; please unify.
[Section 3.1] The row label 'TAMS-CLS' is not defined in the text; please clarify what this baseline is.
[Appendix A.4] The example prompt says 'You are a linguistics expert of Natügu' but the same template is presumably used for all languages; the paper should state explicitly how the prompt is adapted per language, including how the list of grammatical morphemes and stems is selected.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: reported gains are held-out SIGMORPHON 2023 results; the synthetic-data overlap concern is a leakage risk, not a by-construction reduction.

full rationale

The derivation chain is: train a transformer with segmentation and glossing decoders on gold IGT training splits; optionally append GPT-4o triples built from stems and grammatical morphemes extracted from the training split; evaluate on the held-out SIGMORPHON 2023 test split (Section 2, Table 1). No equation defines a predicted quantity in terms of the test target, and no fitted parameter is relabeled as a prediction: lambda in Eq. (1) is tuned on development accuracy (Appendix A.2), and the LLM examples are generated from the training split alone (Section 2.2). The only caveat found in the paper is the synthetic-data validity risk acknowledged in Section 5: 'Since our study does not include a detailed qualitative error analysis, it is difficult to determine whether the improvements stem from genuinely better morphological generalization or simply from increased exposure to frequent patterns.' This concedes that LLM outputs are not audited and that generation is limited to existing vocabulary; if a generated surface form coincided with a held-out test word, reported gains could be inflated. That is a data-leakage threat to experimental validity, not a circular derivation, because test labels do not enter the generation prompt, model objective, or hyperparameter selection. The one potentially overlapping citation (Ginn et al. 2023, co-authored by Nicolai) is the public dataset source and is not load-bearing. Verdict: no significant circularity; score 0.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The central claims rest on a small number of empirical assumptions: the usefulness of glossing as an auxiliary task, the linguistic validity of GPT-4o-generated forms, and the fairness of the custom splits. The only fitted hyperparameter that changes the headline numbers is lambda, chosen on Lezgi dev; the synthetic-data ratio is scanned rather than fit. No free constants or invented entities are introduced.

free parameters (3)

lambda, segmentation loss weight = 0.9 (selected from Lezgi dev sweep; Appendix A.2)
Appears in Equation (1) as the weight on segmentation loss. Section 2.1 says it is tuned and Appendix A.2 sweeps 0.5 to 1.0 on Lezgi, with 0.9 giving the best dev accuracy.
Synthetic data augmentation ratio = 0.25, 0.5, or 0.75 of gold training size
The paper reports results at three ratios and uses them to claim modest improvements; the choice of ratio changes whether a language improves, so the claim depends on this selection.
LLM prompt and sampling settings = not reported
GPT-4o version, temperature, number of generated examples, and filtering criteria are unspecified, yet they determine the synthetic training set.

assumptions (3)

domain assumption Glossing tier is already part of IGT and can be used as auxiliary supervision without extra curation.
Section 1 states glossing is already a component of IGT, so the multitask objective adds no annotation burden; the benefit of sharing an encoder between segmentation and glossing is assumed.
domain assumption GPT-4o in-context learning produces linguistically plausible canonical segmentations for unseen combinations of known stems and grammatical morphemes.
Section 2.2 relies on the LLM applying morphophonological alternations from examples; the Limitations section admits quality and hallucination risk are unexamined.
domain assumption Unique-word splits at 6:2:2 prevent train/test leakage and are comparable to TAMS splits.
Section 2 describes the split; splits are not released, so independence and comparability cannot be verified.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Learning Beyond Limits: Multitask Learning and Synthetic Data for Low-Resource Canonical Morpheme Segmentation." pith.science (2026). https://pith.science/paper/3KOTTRBQ

@misc{pith2026250516800,
  author       = {Pith},
  title        = {Pith review of: Learning Beyond Limits: Multitask Learning and Synthetic Data for Low-Resource Canonical Morpheme Segmentation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3KOTTRBQ}},
  note         = {Machine review of arXiv:2505.16800}
}

read the original abstract

We introduce a transformer-based morpheme segmentation system that augments a low-resource training signal through multitask learning and LLM-generated synthetic data. Our framework jointly predicts morphological segments and glosses from orthographic input, leveraging shared linguistic representations obtained through a common documentary process to enhance model generalization. To further address data scarcity, we integrate synthetic training data generated by large language models (LLMs) using in-context learning. Experimental results on the SIGMORPHON 2023 dataset show that our approach significantly improves word-level segmentation accuracy and morpheme-level F1-score across multiple low-resource languages.

Figures

Figures reproduced from arXiv: 2505.16800 by the authors.

**Figure 2.** The learning curves for the F1 (top) and Accuracy (bottom) metrics among all languages. [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

15 extracted references · 7 canonical work pages

[1]

Rich Caruana. 1997. Multitask learning. Machine learning, 28:41--75

1997
[2]

Michael Ginn, Sarah Moeller, Alexis Palmer, Anna Stacey, Garrett Nicolai, Mans Hulden, and Miikka Silfverberg. 2023. Findings of the SIGMORPHON 2023 shared task on interlinear glossing. In Proceedings of the 20th SIGMORPHON workshop on Computational Research in Phonetics, Phonology, and Morphology, pages 186--201

work page 2023
[3]

Le, Maxim Krikun, Yonghui Wu, Zhifeng Chen, Nikhil Thorat, Fernanda Vi \'e gas, Martin Wattenberg, Greg Corrado, Macduff Hughes, and Jeffrey Dean

Melvin Johnson, Mike Schuster, Quoc V. Le, Maxim Krikun, Yonghui Wu, Zhifeng Chen, Nikhil Thorat, Fernanda Vi \'e gas, Martin Wattenberg, Greg Corrado, Macduff Hughes, and Jeffrey Dean. 2017. https://doi.org/10.1162/tacl_a_00065 G oogle`s multilingual neural machine translation system: Enabling zero-shot translation . Transactions of the Association for C...

doi:10.1162/tacl_a_00065 2017
[4]

Katharina Kann, Ryan Cotterell, and Hinrich Sch \"u tze. 2016. https://doi.org/10.18653/v1/D16-1097 Neural morphological analysis: Encoding-decoding canonical segments . In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 961--967, Austin, Texas. Association for Computational Linguistics

work page doi:10.18653/v1/d16-1097 2016
[5]

Ying Lin, Shengqi Yang, Veselin Stoyanov, and Heng Ji. 2018. https://doi.org/10.18653/v1/P18-1074 A multi-lingual multi-task architecture for low-resource sequence labeling . In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 799--809, Melbourne, Australia. Association for Computationa...

work page doi:10.18653/v1/p18-1074 2018
[6]

Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli. 2019. https://doi.org/10.18653/v1/N19-4009 fairseq: A fast, extensible toolkit for sequence modeling . In Proceedings of the 2019 Conference of the North A merican Chapter of the Association for Computational Linguistics (Demonstrations) , pages 48...

doi:10.18653/v1/n19-4009 2019
[7]

Enora Rice, Ali Marashian, Luke Gessler, Alexis Palmer, and Katharina von der Wense. 2024. https://doi.org/10.18653/v1/2024.acl-long.366 TAMS : Translation-assisted morphological segmentation . In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6752--6765, Bangkok, Thailand. Associatio...

work page doi:10.18653/v1/2024.acl-long.366 2024
[8]

Tatyana Ruzsics and Tanja Samard z i \'c . 2017. https://doi.org/10.18653/v1/K17-1020 Neural sequence-to-sequence learning of internal word structure . In Proceedings of the 21st Conference on Computational Natural Language Learning ( C o NLL 2017) , pages 184--194, Vancouver, Canada. Association for Computational Linguistics

work page doi:10.18653/v1/k17-1020 2017

Show all 15 references

[9]

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems, 30

2017
[10]

Weihua Wang, Rashel Fam, Feilong Bao, Yves Lepage, and Guanglai Gao. 2019. Neural morphological segmentation model for mongolian. In 2019 International Joint Conference on Neural Networks (IJCNN), pages 1--7. IEEE

2019
[11]

Sam Wiseman and Alexander M Rush. 2016. Sequence-to-sequence learning as beam-search optimization. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1296--1306

2016
[12]

Shijie Wu, Ryan Cotterell, and Mans Hulden. 2021. https://doi.org/10.18653/v1/2021.eacl-main.163 Applying the transformer to character-level transduction . In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volu...

2021 doi
[13]

Shuyan Zhou, Xiangkai Zeng, Yingqi Zhou, Antonios Anastasopoulos, and Graham Neubig. 2019. https://doi.org/10.18653/v1/W19-5368 Improving robustness of neural machine translation with multi-task learning . In Proceedings of the Fourth Conference on Machine Translation (Volume ...

2019 doi
[14]

URL: " 'urlintro :=

ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before...
[15]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Rich Caruana. 1997. Multitask learning. Machine learning, 28:41--75

1997

[2] [2]

Michael Ginn, Sarah Moeller, Alexis Palmer, Anna Stacey, Garrett Nicolai, Mans Hulden, and Miikka Silfverberg. 2023. Findings of the SIGMORPHON 2023 shared task on interlinear glossing. In Proceedings of the 20th SIGMORPHON workshop on Computational Research in Phonetics, Phonology, and Morphology, pages 186--201

work page 2023

[3] [3]

Le, Maxim Krikun, Yonghui Wu, Zhifeng Chen, Nikhil Thorat, Fernanda Vi \'e gas, Martin Wattenberg, Greg Corrado, Macduff Hughes, and Jeffrey Dean

Melvin Johnson, Mike Schuster, Quoc V. Le, Maxim Krikun, Yonghui Wu, Zhifeng Chen, Nikhil Thorat, Fernanda Vi \'e gas, Martin Wattenberg, Greg Corrado, Macduff Hughes, and Jeffrey Dean. 2017. https://doi.org/10.1162/tacl_a_00065 G oogle`s multilingual neural machine translation system: Enabling zero-shot translation . Transactions of the Association for C...

doi:10.1162/tacl_a_00065 2017

[4] [4]

Katharina Kann, Ryan Cotterell, and Hinrich Sch \"u tze. 2016. https://doi.org/10.18653/v1/D16-1097 Neural morphological analysis: Encoding-decoding canonical segments . In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 961--967, Austin, Texas. Association for Computational Linguistics

work page doi:10.18653/v1/d16-1097 2016

[5] [5]

Ying Lin, Shengqi Yang, Veselin Stoyanov, and Heng Ji. 2018. https://doi.org/10.18653/v1/P18-1074 A multi-lingual multi-task architecture for low-resource sequence labeling . In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 799--809, Melbourne, Australia. Association for Computationa...

work page doi:10.18653/v1/p18-1074 2018

[6] [6]

Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli. 2019. https://doi.org/10.18653/v1/N19-4009 fairseq: A fast, extensible toolkit for sequence modeling . In Proceedings of the 2019 Conference of the North A merican Chapter of the Association for Computational Linguistics (Demonstrations) , pages 48...

doi:10.18653/v1/n19-4009 2019

[7] [7]

Enora Rice, Ali Marashian, Luke Gessler, Alexis Palmer, and Katharina von der Wense. 2024. https://doi.org/10.18653/v1/2024.acl-long.366 TAMS : Translation-assisted morphological segmentation . In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6752--6765, Bangkok, Thailand. Associatio...

work page doi:10.18653/v1/2024.acl-long.366 2024

[8] [8]

Tatyana Ruzsics and Tanja Samard z i \'c . 2017. https://doi.org/10.18653/v1/K17-1020 Neural sequence-to-sequence learning of internal word structure . In Proceedings of the 21st Conference on Computational Natural Language Learning ( C o NLL 2017) , pages 184--194, Vancouver, Canada. Association for Computational Linguistics

work page doi:10.18653/v1/k17-1020 2017