Pith. sign in

REVIEW 3 major objections 6 minor 48 references

Automatically Suggesting Diverse Example Sentences for L2 Japanese Learners Using Pre-Trained Language Models

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that for L2 Japanese learners, retrieving and ranking real corpus sentences with pre-trained language models produces better example sentences than asking LLMs to generate them from scratch.

desk verdict A transparent, well-scoped applied NLP paper whose retrieval pipeline beats LLM generation in a small human study, but the difficulty classifier's poor OOD accuracy and tiny evaluation leave the generality of the headline unproven. read the letter →

arxiv 2506.03580 v1 pith:5M2ZFSAM submitted 2025-06-04 cs.CL

Enrico Benedetti , Akiko Aizawa , Florian Boudin This is my paper
classification cs.CL
keywords examplesentencesuggestionJapaneseasasecondlanguageJLPTdifficultyclassificationpre-trainedmodelsword-in-contextdiversityretrievalversusgenerationGPT-4evaluation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that, for L2 Japanese learners, good example sentences for a target word can be produced more reliably by retrieving and ranking sentences from a large corpus with pre-trained language models than by asking a language model to generate them from scratch. The retrieval system scores each candidate sentence on JLPT difficulty match, sense similarity to a context sentence, and list-level syntactic and lexical diversity, while the baselines are zero-shot generation with LLM-jp and GPT-3.5. In an evaluation with human learners, native speakers, and GPT-4, the retrieval approach received the most first-place rankings across all rater groups, especially for beginner (N5) and advanced (N1) target levels. The paper also introduces WJTSentDiL, a 12.7-million-sentence Japanese corpus with automatically assigned JLPT labels, and reports that human raters disagree on most quality dimensions except difficulty.

What carries the argument

The load-bearing mechanism is the retrieval-scoring pipeline. A BERT-based JLPT difficulty classifier, fine-tuned on 5,000 distantly supervised sentences, assigns each candidate a difficulty label that feeds a linear score with penalty 0.2 (0.4 for sentences harder than the target). Sense similarity comes from MirrorWiC, an unsupervised fine-tuning method for word-in-context embeddings, whose cosine similarity to the query context scores whether the target word is used in the same sense. Syntactic diversity is measured with FastKASSIM, a tree-kernel similarity over dependency trees, combined with an n-gram uniqueness score, and the final list is built greedily from a 50-candidate window so that total list diversity is maximized. The corpus it draws on, WJTSentDiL, merges Wikipedia, JpWaC, and Tatoeba into 12.7 million filtered Japanese sentences.

What would settle it

Replace the distantly supervised difficulty classifier with one that reaches high accuracy on official JLPT exam sentences and rerun the same human ranking evaluation; if retrieval then no longer receives the most first-place votes, the paper's central claim is refuted.

Watch

Extended reading notes

Core claim

Given a target word, a context sentence, and a desired JLPT level, the paper's retrieval model returns five sentences that humans and GPT-4 rank above zero-shot generations from LLM-jp and GPT-3.5. The ranking combines a difficulty score with a heavier penalty for sentences harder than the target level, cosine similarity of MirrorWiC sense embeddings between the context sentence and each candidate, and a greedy selection step that maximizes average dependency-tree and n-gram diversity. The authors interpret this as evidence that human-authored corpus sentences, filtered by PLM-based quality signals, remain preferable to model-generated text for this task, and that PLMs are better used as scoring components than as direct generators.

Load-bearing premise

The whole pipeline depends on the JLPT difficulty classifier assigning accurate levels to arbitrary corpus sentences; the paper's own test on official JLPT exam sentences gives only 38% accuracy, so if the classifier mislabels sentences the retrieval ranking inherits the error.

Editorial extensions

If this is right

  • A practical system can suggest level-appropriate example sentences without supervised data for generation, using only corpus retrieval and PLM scoring.
  • Because retrieval won most first-place votes for N1 and N5 but not N3, the paper points to mid-range level-appropriateness as the remaining bottleneck.
  • The high agreement between GPT-4 and humans on JLPT difficulty suggests automated difficulty rating can be used to scale evaluation of learning materials.
  • The generative baselines' weaknesses—repetition, missing target words, incoherent text—can be patched by post-filtering, but still trailed retrieval in this setting.
  • The pipeline is language-agnostic in principle, requiring only a difficulty classifier, a dependency parser, and sense embeddings for the target language.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A likely next step the paper leaves implicit is retrieval-then-generation: use retrieved sentences as exemplars to condition a generator, combining corpus naturalness with the adaptability of generation.
  • The 38% out-of-distribution accuracy of the difficulty classifier suggests the retrieval advantage on level-appropriateness may be fragile; incorporating word-level JLPT or school-grade kanji features could close the gap.
  • The low inter-rater agreement on everything except difficulty points toward personalization: instead of one best list, systems could rank sentences differently per learner profile, with difficulty as a shared anchor.
  • GPT-4's consistent difficulty ratings could serve as a cheap first-pass filter for candidate sentences, reducing human annotation burden in future corpus construction.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper addresses the task of suggesting diverse, proficiency-appropriate example sentences for L2 Japanese learners. Given a target word, a context sentence, and a target JLPT level, a retrieval-based system scores candidate sentences from a newly built corpus (WJTSentDiL) according to three criteria: difficulty (a fine-tuned BERT classifier with labels assigned via Eq. 2), sense similarity (MirrorWiC embeddings), and diversity (dependency-tree kernel similarity plus n-gram overlap), followed by greedy selection. This retrieval approach is compared against zero-shot generation with LLM-jp and GPT-3.5. Evaluation uses five human raters (three native speakers, two learners) and GPT-4, who rate difficulty, sense similarity, rejection, syntactic diversity, and overall ranking. The main reported finding is that the retrieval approach receives the most first-place rankings across all evaluator groups, especially for N1 and N5 target levels, while the generative baselines receive lower scores.

Significance. If the difficulty-classifier component is reliable on the retrieval corpus, the paper makes a useful practical contribution: it introduces a sizable Japanese example-sentence corpus with JLPT labels, a modular retrieval pipeline that combines several PLM and NLP components, and a multi-rater evaluation protocol that includes GPT-4. The release of code, corpus, and model is a concrete asset for subsequent work on computer-assisted language learning. The paper is also honest about its limitations, including the small number of human raters and the acknowledged non-generalizability of the results. However, the central claim that retrieval is preferred because it produces level-appropriate sentences depends on a difficulty classifier whose out-of-distribution performance is only 38% accuracy on official JLPT exam sentences. Since that classifier directly drives the retrieval ranking (Eq. 2) and is also used to label the corpus, the level-appropriateness advantage is currently not supported by direct evidence on the retrieval corpus.

major comments (3)
  1. [§5.2.2, Appendix A (Table 7), Eq. (2)] The difficulty classifier is the component that enforces level-appropriateness in the retrieval ranking, but its out-of-distribution accuracy on official JLPT exam sentences is only 38% (macro F1 0.36, N1 recall 0.08), as reported in Table 7. This is only slightly above the 20% random baseline for five classes. The corpus WJTSentDiL, however, is 96.9% Wikipedia sentences with an average length of 26.39 tokens and 36.35% kanji, which is structurally far from the short, clean website sentences used for classifier training. The paper hypothesizes that long relative clauses cause the performance drop, but it never measures classifier accuracy on a sample of WJTSentDiL or on the candidate pools used for the ten evaluation queries. Since Eq. (2) directly translates the classifier label into a difficulty score that ranks candidates, the claimed advantage of retrieval in level-appropriateness (abstract, §7.2.1, Fig. 2) is currently unverified. The authors should report at least a human-validated accuracy estimate on a WJTSentDiL sample, and ideally an error-propagation analysis showing how classifier mistakes affect the final ranked lists.
  2. [§5.1, §5.2.2, §7.2.1] There is a circularity concern in the difficulty signal: the same BERT classifier is used both to assign JLPT labels to every sentence in WJTSentDiL and to score retrieval candidates in Eq. (2). This means the corpus itself and the retrieval ranking are jointly shaped by the same model's biases. The human difficulty ratings in Fig. 2 cover only the 150 final retrieved sentences, so they cannot reveal how much classifier error was filtered out by the greedy selection or propagated from the much larger candidate pool. An independent validation of the classifier on a sample of the corpus, or a comparison of classifier-assigned labels with human-assigned labels on a random corpus sample, would break this circularity and should be added.
  3. [§6.4, Table 4] The headline conclusion that "the retrieval approach was preferred by all evaluators" rests on only five human raters and ten evaluation queries, with three target levels and three systems. The authors acknowledge in the Limitations section that the results are not generalizable, which is appropriate, but the strength of the claim in the abstract and §7.2.5 is still larger than the evidence. Table 4 shows raw counts of first-place votes, but there is no statistical test, confidence interval, or analysis of per-query variance. Given that the human agreement on ranking is low (Table 2, ICC ≈ 0.22 among humans), a modest addition such as per-query breakdowns or a bootstrap estimate of the retrieval win rate would substantially strengthen the paper without requiring a new human evaluation.
minor comments (6)
  1. [§6.4] There is a typo: "Addionally" should be "Additionally".
  2. [Figure 2 and Figure 8] The header "GPT-3.5 T urbo" contains a stray space; it should read "GPT-3.5 Turbo".
  3. [Appendix C] The prompt templates use placeholders such as "k", "target word", and "context sentence" without explaining exactly how these are instantiated in the code, and it is unclear whether the final prompt sent to LLM-jp and GPT-3.5 is in English, Japanese, or both. A short example of a fully instantiated prompt would improve reproducibility.
  4. [§4.1.1, §4.1.4] The manually set parameters penaltydiff (0.2/0.4), the window size (50), and the equal weighting of difficulty and sense scores are introduced without sensitivity analysis. A brief discussion of how these choices affect outputs would be useful, even if only on a development set.
  5. [References] Several references contain spacing errors, e.g., "V olodina" and "Y" in the Koo and Li citation; these should be corrected.
  6. [§7.3] The sentence "In that case, GPT-3.5 produced a similar sentence as the context in which the usage was 'uncommon'" is ambiguous; it should clarify whether GPT-3.5 produced a sentence similar to the 1938 Dazai context or similar in the sense of using the same uncommon positive construction of 全然.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: retrieval difficulty is validated by independent human raters, not by the classifier used to rank.

full rationale

The retrieval pipeline does use the same BERT difficulty classifier (fine-tuned on 5,000 distantly supervised sentences, Section 5.2.2) both to label WJTSentDiL and to score candidates via Eq. (2). However, the paper's central claim, that retrieval is preferred and produces level-appropriate sentences, is tested against human JLPT ratings and GPT-4 ratings collected under the protocol of Section 6.4 ('We ask evaluators to rate: 1. Difficulty level, by rating the difficulty of each sentence on the JLPT scale'), not against the classifier's own labels. Figure 2 reports those human-assigned proportions. A self-consistency circularity would require the reported 'difficulty' outcome to be the classifier's di by construction; instead it is an independent rater judgment. The 38% accuracy on official JLPT exam sentences (Appendix A, Table 7) and the absence of accuracy figures on WJTSentDiL are real validity threats for the corpus labels and for retrieval ranking, but they are correctness and robustness concerns, not circularity: a noisy or biased classifier would weaken the system rather than force the human evaluation results. No load-bearing argument in the paper reduces to a self-citation or to an ansatz imported from the authors' own prior work; the cited MirrorWiC and BERT-difficulty baselines are external. Accordingly the score is 0.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The central claim does not rest on novel theoretical entities. It relies on the adequacy of the difficulty classifier, which is the most fragile component due to its poor out-of-distribution performance, and on several manually chosen scoring coefficients. The paper introduces no new forces, particles, or hypothetical constructs.

free parameters (4)
  • penaltydiff = 0.2 (0.4 for harder sentences)
    Manual coefficient in the difficulty score formula (Equation 2). The paper states 'We manually set the coefficient penaltydiff to 0.2.'
  • window size = 50
    The greedy selection window is set to 50 candidates for computational reasons, not derived from an objective. This affects which sentences are considered for final selection.
  • diversity weights = equal weights (0.5 each)
    The combined diversity score equally weights syntax and lexical diversity. No justification for equal weighting.
  • number of query repetitions for generation = 2 ('majority of cases')
    To get enough valid sentences, the authors prompt LLMs multiple times; the choice of when to stop is a practical heuristic.
assumptions (3)
  • domain assumption The JLPT difficulty classifier trained on distantly supervised website data is sufficiently accurate for ranking sentences by difficulty.
    The classifier shows 84% in-distribution accuracy but drops to 38% on official JLPT exam sentences. The paper nevertheless uses it to label the entire WJTSentDiL corpus and to score retrieval candidates.
  • domain assumption MirrorWiC fine-tuning on randomly selected corpus sentences yields representations that capture sense similarity for the target word.
    The paper uses MirrorWiC embeddings without a strong validation on the target task; the guidance is from two external WiC tasks, but the transfer to Japanese L2 example selection is assumed.
  • ad hoc to paper Greedy selection with a window of 50 is a good approximation to the diversity objective.
    The window size is chosen for computational efficiency, and the effect on final sentence quality is not analyzed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Automatically Suggesting Diverse Example Sentences for L2 Japanese Learners Using Pre-Trained Language Models." pith.science (2026). https://pith.science/paper/5M2ZFSAM

@misc{pith2026250603580,
  author       = {Pith},
  title        = {Pith review of: Automatically Suggesting Diverse Example Sentences for L2 Japanese Learners Using Pre-Trained Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5M2ZFSAM}},
  note         = {Machine review of arXiv:2506.03580}
}
read the original abstract

Providing example sentences that are diverse and aligned with learners' proficiency levels is essential for fostering effective language acquisition. This study examines the use of Pre-trained Language Models (PLMs) to produce example sentences targeting L2 Japanese learners. We utilize PLMs in two ways: as quality scoring components in a retrieval system that draws from a newly curated corpus of Japanese sentences, and as direct sentence generators using zero-shot learning. We evaluate the quality of sentences by considering multiple aspects such as difficulty, diversity, and naturalness, with a panel of raters consisting of learners of Japanese, native speakers -- and GPT-4. Our findings suggest that there is inherent disagreement among participants on the ratings of sentence qualities, except for difficulty. Despite that, the retrieval approach was preferred by all evaluators, especially for beginner and advanced target proficiency, while the generative approaches received lower scores on average. Even so, our experiments highlight the potential for using PLMs to enhance the adaptability of sentence suggestion systems and therefore improve the language learning journey.

Figures

Figures reproduced from arXiv: 2506.03580 by the authors.

Figure 1
Figure 1. Task overview. Given a word in context and [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Evaluators’ ratings on difficulty. Each row presents the proportions of JLPT labels assigned by humans [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Confusion matrix for the difficulty classifier, on sentences obtained in the same way as the training data [PITH_FULL_IMAGE:figures/full_fig_p014_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Confusion matrix for the difficulty classifier, on sentences obtained from a different source (i.e. past [PITH_FULL_IMAGE:figures/full_fig_p014_4.png]
Figure 5
Figure 5. Figure 5: Ratings on sense similarity of proposed sentences. [PITH_FULL_IMAGE:figures/full_fig_p017_5.png]
Figure 6
Figure 6. Figure 6: Proportion of rejected proposed sentences. [PITH_FULL_IMAGE:figures/full_fig_p017_6.png]
Figure 7
Figure 7. Figure 7: Ratings on syntax diversity of proposed sentences. [PITH_FULL_IMAGE:figures/full_fig_p018_7.png]
Figure 8
Figure 8. Figure 8: Rankings (first, second, third place) for each system. [PITH_FULL_IMAGE:figures/full_fig_p018_8.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

48 extracted references · 35 canonical work pages

  1. [1]

    Tim Andersson and Pablo Picazo-Sanchez. 2023. https://doi.org/10.3390/educsci13121203 Closing the gap: Automated distractor generation in japanese language testing . Education Sciences, 13(12)

    work page doi:10.3390/educsci13121203 2023

  2. [2]

    Zhang Baicheng. 2009. Do example sentences work in direct vocabulary learning? Issues in Educational Research, 19

    work page 2009

  3. [3]

    Edoardo Barba, Luigi Procopio, Caterina Lacerra, Tommaso Pasini, and Roberto Navigli. 2021. https://doi.org/10.24963/ijcai.2021/520 Exemplification Modeling : Can You Give Me an Example , Please ? In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence , pages 3779--3785, Montreal, Canada. International Joint Conferences ...

    work page doi:10.24963/ijcai.2021/520 2021

  4. [4]

    Reihane Boghrati, Joe Hoover, Kate M Johnson, Justin Garten, and Morteza Dehghani. 2018. Conversation level syntax similarity metric. Behavior research methods, 50(3):1055--1073

    2018

  5. [5]

    Maximillian Chen, Caitlyn Chen, Xiao Yu, and Zhou Yu. 2023 a . Fastkassim: A fast tree kernel-based syntactic similarity metric. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics

    work page 2023

  6. [6]

    Yi Chen, Rui Wang, Haiyun Jiang, Shuming Shi, and Ruifeng Xu. 2023 b . https://aclanthology.org/2023.findings-ijcnlp.32 Exploring the use of large language models for reference-free text quality evaluation: An empirical study . In Findings of the Association for Computational Linguistics: IJCNLP-AACL 2023 (Findings), pages 361--374, Nusa Dua, Bali. Associ...

    work page 2023

  7. [7]

    Shang-Chien Cheng, Jhih-Jie Chen, Chingyu Yang, and Jason Chang. 2018. https://aclanthology.org/C18-2022 LanguageNet : Learning to Find Sense Relevant Example Sentences . In Proceedings of the 27th International Conference on Computational Linguistics : System Demonstrations , pages 99--102, Santa Fe, New Mexico. Association for Computational Linguistics

    work page 2018

  8. [8]

    Kevyn Collins-Thompson. 2014. https://doi.org/https://doi.org/10.1075/itl.165.2.01col Computational assessment of text readability: A survey of current and future research . ITL - International Journal of Applied Linguistics, 165(2):97--135

    work page doi:10.1075/itl.165.2.01col 2014

Show all 48 references
  1. [9]

    Peng Cui and Mrinmaya Sachan. 2023. https://doi.org/10.3929/ethz-b-000653699 Adaptive and personalized exercise generation for online language learning . In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics, Volume 1: Long Papers, pages 10...

    2023 doi

  2. [10]

    Gerard de Melo and Gerhard Weikum. 2009. https://aclanthology.org/W09-4407 Extracting sense-disambiguated example sentences from parallel corpora . In Proceedings of the 1st Workshop on Definition Extraction, pages 40--46, Borovets, Bulgaria. Association for Computational Linguistics

    2009

  3. [11]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. https://doi.org/10.18653/v1/N19-1423 BERT : Pre-training of deep bidirectional transformers for language understanding . In Proceedings of the 2019 Conference of the North A merican Chapter of the Associat...

    2019 doi

  4. [12]

    Fromkin, R

    V. Fromkin, R. Rodman, and N. Hyams. 2013. https://books.google.co.jp/books?id=rM_ntI7923AC An Introduction to Language . Cengage Learning

    2013

  5. [13]

    Noah Gardner, Hafiz Khan, and Chih-Cheng Hung. 2022. https://api.semanticscholar.org/CorpusID:247933647 Definition modeling: literature review and dataset analysis . Applied Computing and Intelligence

    2022

  6. [14]

    Veronika Hackl, Alexandra Elena Müller, Michael Granitzer, and Maximilian Sailer. 2023. https://www.frontiersin.org/articles/10.3389/feduc.2023.1272229 Is GPT -4 a reliable rater? Evaluating consistency in GPT -4's text ratings . Frontiers in Education, 8

    2023

  7. [15]

    Yoo, and Eunseop Yoon

    John Harvill, Mark Hasegawa-Johnson, Hee Suk Yoon, Chang D. Yoo, and Eunseop Yoon. 2023. https://doi.org/10.18653/v1/2023.repl4nlp-1.25 One- Shot Exemplification Modeling via Latent Sense Representations . In Proceedings of the 8th Workshop on Representation Learning for NLP (...

    2023 doi

  8. [16]

    Gregory Hazelbeck and Hiroaki Saito. 2009. https://doi.org/10.5715/jnlp.16.4_3 A Corpus -based E -learning System for Japanese Vocabulary . Journal of Natural Language Processing, 16(4):3--27

    2009 doi

  9. [17]

    Xingwei He and Siu Ming Yiu. 2022. https://doi.org/10.18653/v1/2022.acl-long.46 Controllable Dictionary Example Generation : Generating Example Sentences for Specific Targeted Audiences . In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistic...

    2022 doi

  10. [18]

    Eriko Kanagawa and Takeshi Okadome. 2016. https://doi.org/10.1109/IALP.2016.7875934 Syntactic characteristics and similarities of japanese authors' writing styles: A kernel-based approach . In 2016 International Conference on Asian Language Processing (IALP), pages 59--62

    2016

  11. [19]

    Pulkit Kathuria and Kiyoaki Shirai. 2012. https://doi.org/10.1007/978-3-642-33983-7_21 Word Sense Disambiguation Based on Example Sentences in Dictionary and Automatically Acquired from Parallel Corpus . In Advances in Natural Language Processing , Lecture Notes in Computer Sc...

    2012 doi

  12. [20]

    Rychl \'y

    Adam Kilgarriff, Milos Hus \'a k, Katie McAdam, Michael Rundell, and P. Rychl \'y . 2008. https://api.semanticscholar.org/CorpusID:56231537 Gdex: Automatically finding good dictionary examples in a corpus . In Proceedings of the 13th EURALEX International Congress, Barcelona, ...

    2008

  13. [21]

    Koo and Mae Y

    Terry K. Koo and Mae Y. Li. 2016. https://doi.org/10.1016/j.jcm.2016.02.012 A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research . Journal of Chiropractic Medicine, 15(2):155--163

    2016 doi

  14. [22]

    Jun Liu, Fei Cheng, Yiran Wang, Hiroyuki Shindo, and Yuji Matsumoto. 2018 a . https://aclanthology.org/Y18-1046 Automatic Error Correction on Japanese Functional Expressions Using Character -based Neural Machine Translation . In Proceedings of the 32nd Pacific Asia Conference ...

    2018

  15. [23]

    Jun Liu and Yuji Matsumoto. 2016. https://aclanthology.org/W16-4901 Simplification of Example Sentences for Learners of Japanese Functional Expressions . In Proceedings of the 3rd Workshop on Natural Language Processing Techniques for Educational Applications ( NLPTEA2016 ) , ...

    2016

  16. [24]

    Jun Liu and Yuji Matsumoto. 2017. https://aclanthology.org/Y17-1040 Sentence complexity estimation for C hinese-speaking learners of J apanese . In Proceedings of the 31st Pacific Asia Conference on Language, Information and Computation, pages 296--302. The National University...

    2017

  17. [25]

    Jun Liu, Hiroyuki Shindo, and Yuji Matsumoto. 2018 b . https://doi.org/10.18653/v1/P18-4010 Sentence Suggestion of Japanese Functional Expressions for Chinese -speaking Learners . In Proceedings of ACL 2018, System Demonstrations , pages 56--61, Melbourne, Australia. Associati...

    2018 doi

  18. [26]

    Qianchu Liu, Fangyu Liu, Nigel Collier, Anna Korhonen, and Ivan Vulić. 2021 a . https://doi.org/10.18653/v1/2021.conll-1.44 MirrorWiC : On Eliciting Word -in- Context Representations from Pretrained Language Models . In Proceedings of the 25th Conference on Computational Natur...

    2021 doi

  19. [27]

    Qianchu Liu, Edoardo Maria Ponti, Diana McCarthy, Ivan Vuli \'c , and Anna Korhonen. 2021 b . https://doi.org/10.18653/v1/2021.emnlp-main.571 AM 2i C o: Evaluating word meaning in context across low-resource languages with adversarial examples . In Proceedings of the 2021 Conf...

    2021 doi

  20. [28]

    Alessandro Moschitti. 2006. Making tree kernels practical for natural language learning. In 11th conference of the European Chapter of the Association for Computational Linguistics, pages 113--120

    2006

  21. [29]

    Nakamachi, Toshinori, Nishiuchi, Masayu, and Oku. 2022. Estimation of japanese text difficulty based on the japanese language proficiency test. Online

    2022

  22. [30]

    Manabu Okumura, Kiyoaki Shirai, Kanako Komiya, and Hikaru Yokono. 2011. https://doi.org/10.5715/jnlp.18.293 On SemEval -2010 Japanese WSD Task . Journal of Natural Language Processing, 18(3):293--307

    2011 doi

  23. [31]

    OpenAI. 2023. http://arxiv.org/abs/2303.08774 Gpt-4 technical report

    2023 arXiv

  24. [32]

    Mohammad Taher Pilehvar and Jose Camacho-Collados. 2019. https://doi.org/10.18653/v1/N19-1128 W i C : the word-in-context dataset for evaluating context-sensitive meaning representations . In Proceedings of the 2019 Conference of the North A merican Chapter of the Association ...

    2019 doi

  25. [34]

    Ildikó Pilán, Elena Volodina, and Richard Johansson. 2013 b . https://doi.org/10.14705/rpnet.2013.000164 Automatic Selection of Suitable Sentences for Language Learning Exercises . In 20 Years of EUROCALL : Learning from the Past , Looking to the Future , pages 218--225. Resea...

    2013

  26. [35]

    Alessandro Raganato, Tommaso Pasini, Jose Camacho-Collados, and Mohammad Taher Pilehvar. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.584 XL - W i C : A multilingual benchmark for evaluating semantic contextualization . In Proceedings of the 2020 Conference on Empirical M...

    2020 doi

  27. [36]

    Erjavec, and Yoshiko Kawamura

    Kristina Hmeljak Sangawa, T. Erjavec, and Yoshiko Kawamura. 2010. https://api.semanticscholar.org/CorpusID:62012373 Automated collection of japanese word usage examples from a parallel and a monolingual corpus . In "Proceedings of eLexicography in the 21st century: New challen...

    2010

  28. [37]

    Osamu Sawada. 2007. https://api.semanticscholar.org/CorpusID:55947762 Two types of adverbial polarity items in japanese: absolute and relative . In Proceedings of the 10th Conference of the Pragmatics Society of Japan

    2007

  29. [38]

    Burr Settles, Chris Brust, Erin Gustafson, Masato Hagiwara, and Nitin Madnani. 2018. https://doi.org/10.18653/v1/W18-0506 Second language acquisition modeling . In Proceedings of the Thirteenth Workshop on Innovative Use of NLP for Building Educational Applications , pages 56-...

    2018 doi

  30. [39]

    Hiroyuki Shinnou and Minoru Sasaki. 2008. https://api.semanticscholar.org/CorpusID:1803542 Division of example sentences based on the meaning of a target word using semi-supervised clustering . In International Conference on Language Resources and Evaluation

    2008

  31. [40]

    Mitchell Shortt. 2021. https://doi.org/10.1007/s11423-020-09890-2 Synthesizing a Japanese -language functional expression learning system with Chinese -speaking learners’ cultural interests and backgrounds . Educational Technology Research and Development, 69(1):319--322

    2021 doi

  32. [41]

    Shrout and Joseph L

    Patrick E. Shrout and Joseph L. Fleiss. 1979. https://doi.org/10.1037/0033-2909.86.2.420 Intraclass correlations: Uses in assessing rater reliability . Psychological bulletin, 86(2):420--428

    1979 doi

  33. [42]

    Arseny Tolmachev and Sadao Kurohashi. 2017. https://doi.org/10.18653/v1/W17-5014 Automatic extraction of high-quality example sentences for word learning using a determinantal point process . In Proceedings of the 12th Workshop on Innovative Use of NLP for Building Educational...

    2017 doi

  34. [43]

    Arseny Tolmachev, Sadao Kurohashi, and Daisuke Kawahara. 2022. https://doi.org/10.2197/ipsjjip.30.315 Automatic japanese example extraction for flashcard-based foreign language learning . Journal of Information Processing, 30:315--330

    2022 doi

  35. [44]

    Haifeng Wang, Jiwei Li, Hua Wu, Eduard Hovy, and Yu Sun. 2023. https://doi.org/https://doi.org/10.1016/j.eng.2022.04.024 Pre-trained language models and their applications . Engineering, 25:51--65

    2023 doi

  36. [45]

    Monica Ward. 2017. https://doi.org/10.14705/rpnet.2017.eurocall2017.735 ICALL’s relevance to CALL , pages 328--332. Research-publishing.net

    2017 doi

  37. [46]

    Hanqing Zhang, Haolin Song, Shaoyu Li, Ming Zhou, and Dawei Song. 2023 a . https://doi.org/10.1145/3617680 A survey of controllable text generation using transformer-based pre-trained language models . ACM Comput. Surv., 56(3)

    2023 doi

  38. [47]

    Hengyuan Zhang, Dawei Li, Yanran Li, Chenming Shang, Chufan Shi, and Yong Jiang. 2023 b . https://doi.org/10.18653/v1/2023.bea-1.23 Assisting language learners: Automated trans-lingual definition generation via contrastive prompt learning . In Proceedings of the 18th Workshop ...

    2023 doi

  39. [48]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before...

  40. [49]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.