Pith. sign in

REVIEW 4 major objections 6 minor 40 references

Attention on Multiword Expressions: A Multilingual Study of BERT-based Models with Regard to Idiomaticity and Microsyntax

T0 review · 4 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Fine-tuning steers BERT's attention to multiword expressions: syntax tasks focus lower layers on microsyntactic units, semantic tasks spread idiom attention evenly across layers.

desk verdict A legitimate multilingual attention study whose headline claim outruns the evidence: the 'significantly' is not statistically backed, but the raw patterns and released resources are worth a serious look. read the letter →

arxiv 2505.06062 v1 pith:OL2FWTJ7 submitted 2025-05-09 cs.CL

Iuliia Zaitova , Vitalii Hirak , Badr M. Abdullah , Dietrich Klakow , Bernd Möbius , Tania Avgustinova This is my paper · ORCID
classification cs.CL
keywords BERTattentionmultiwordexpressionsidiomsmicrosyntacticunitsfine-tuninglayer-wisemultilingualNLPsyntacticversussemantictasks
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether fine-tuning a BERT-style model on a supervised task changes where the model's attention lands on multiword expressions, and whether the direction of the change depends on whether the task is syntactic or semantic. It claims yes: fine-tuning on syntactic tasks (dependency-relation classification and part-of-speech tagging) raises attention to microsyntactic units in the lower layers, while fine-tuning on semantic tasks (named-entity recognition and topic classification) spreads attention to idioms more evenly across layers. The pattern is tested in six Indo-European languages with monolingual 24-layer BERT-based encoders. If the claim holds, attention is not a fixed residue of pretraining but a task-adaptable resource, and layer-wise attention profiles become a diagnostic for which linguistic demands a model has learned to meet. Russian is a reported exception, with attention to both idioms and MSUs mostly decreasing after fine-tuning.

What carries the argument

The load-bearing object is the layer-wise attention profile: for each of the model's 24 layers, attention matrices are averaged across heads and reduced to two numbers, the mean attention that context tokens pay to MWE tokens and the mean attention MWE tokens pay to each other. Subword tokens belonging to an MWE are aggregated so that each expression acts as one unit. The category contrast between idioms and microsyntactic units carries the argument, because the two are chosen to isolate semantic non-compositionality and syntactic unpredictability respectively. The interpretation step relies on a previously established mapping, cited by the paper, that lower BERT layers encode syntactic information and higher layers encode semantic information; attention peaks in lower layers are therefore read as a sign of syntactic processing requirements.

What would settle it

Take one language and model, fine-tune several random seeds on a syntactic task and a semantic task, and measure attention to idioms and microsyntactic units layer by layer. The claim predicts that syntactic fine-tuning raises lower-layer attention to microsyntactic units and semantic fine-tuning flattens idiom attention; if either pattern fails to replicate across seeds, or if a second Russian-initialized model reproduces the Russian decrease rather than the cross-lingual pattern, the causal story about task type collapses.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that the division between semantic and syntactic fine-tuning leaves a measurable trace in BERT's attention. For idioms, semantic tasks make the model distribute attention across more layers instead of concentrating it, consistent with the idea that a non-compositional expression has to be assembled from distributed semantic cues. For microsyntactic units, syntactic tasks push attention toward the lower layers where the model is presumed to do syntactic work, consistent with the idea that these expressions demand non-standard grammatical processing. The authors report the same broad contrast in Germanic and Slavic languages, with Russian as a consistent exception where fine-tuning mostly lowers attention to both categories.

Load-bearing premise

The load-bearing premise is that lower BERT layers encode syntax and higher layers encode semantics; if that mapping does not hold for a given model or language, the interpretation of attention peaks as syntactic or semantic processing loses its footing even if the raw attention differences are real.

Editorial extensions

If this is right

  • Fine-tuning on a syntactic task can be used deliberately to sharpen a model's lower-layer attention to syntactically irregular expressions, and fine-tuning on a semantic task to broaden attention to non-compositional ones.
  • Layer-wise attention to multiword expressions can serve as a diagnostic for what a fine-tuned model has learned, alongside task accuracy.
  • The syntactic-versus-semantic distinction is a genuine axis of attention behavior, not just a surface difference between datasets.
  • Because Russian breaks the pattern, task type alone does not determine attention; language-specific models and datasets must be part of any account.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If lower-layer attention to microsyntactic units is genuinely tied to syntactic irregularity, then fine-tuning on dependency relations or part-of-speech tagging should measurably improve a downstream parser's accuracy on sentences containing these units; the paper does not test that, but it follows directly and is testable.
  • The Russian anomaly may have a model- or tokenizer-level cause rather than a language-level one; comparing a second Russian-initialized architecture on the same tasks would separate those possibilities.
  • An even layer distribution of idiom attention under semantic tasks could be an attention-entropy increase rather than targeted semantic integration; re-analyzing the same data with entropy or headwise measures would tell which.
  • The same method could be applied to other multiword-expression families, such as collocations or light-verb constructions, predicting an attention profile intermediate between idioms and microsyntactic units.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The manuscript examines whether fine-tuning BERT-based models on syntactic (DepRel, POS) and semantic (NER, Topic) tasks changes their attention to two types of multiword expressions across six Indo-European languages. Attention weights are extracted from 24-layer monolingual BERT models, averaged over heads, and summarized as layer-wise percentages of attention from context to MWEs and within MWEs. The authors report that fine-tuning significantly changes attention: semantic fine-tuning distributes idiom attention more evenly across layers, while syntactic fine-tuning increases attention to microsyntactic units in lower layers, with Russian identified as an exception. The datasets and fine-tuned models are released.

Significance. If the main findings were supported by quantitative evidence, the paper would provide a useful multilingual contribution to interpretability of MWE processing in transformers, extending Jang et al. (2024) to idioms versus microsyntactic units. The release of datasets and fine-tuned models, the use of six languages and two MWE types, and the distinction between context-to-MWE and within-MWE attention are strengths. However, the current evidence is primarily visual, the abstract's 'significantly' is not backed by statistical tests, the Russian results in Figure 3 and Table 3 already contradict the blanket claims, and the interpretation relies on an unverified mapping from attention peaks to syntactic processing. The contribution is therefore potentially valuable but not yet established.

major comments (4)
  1. [Abstract; Section 5; Figures 1–3] The central claim that fine-tuning 'significantly influences' attention to MWEs is supported only by visual inspection of layer-wise averages. No significance tests, confidence intervals, or variation across random seeds are reported, and with 24 layers, two MWE types, four fine-tuned tasks, and six languages the number of implicit comparisons is large. Differences of a few percentage points (e.g., Figure 3) could arise from sampling variation over 227 contexts. A quantitative analysis—for example, paired per-layer tests with multiple-comparison correction, bootstrap confidence intervals, and effect sizes—is needed before the headline claim can be accepted.
  2. [Section 5.2; Figure 3; Table 3(b); Conclusion] The abstract and Conclusion state that models fine-tuned on syntactic tasks show increased attention to MSUs in lower layers, but the paper's own Russian results contradict this. Section 5.2 reports that for Russian MSUs the POS task increases attention in middle and upper layers and that DepRel does not show the same pattern; Table 3(b) lists layers 11 and 12 among the top three for Russian MSUs under POS, and Figure 3 shows mostly decreased attention in Russian after fine-tuning. The claims should be revised to language- and task-specific statements, or the generality of the pattern should be demonstrated statistically.
  3. [Section 5.2; Section 2 (Tenney et al.)] The interpretation of lower-layer attention peaks as evidence of 'syntactic processing requirements' rests on the premise that lower BERT layers encode syntax and higher layers encode semantics, cited to Tenney et al. (2019). That work uses linear probes on contextual representations, not attention weights, and no analysis here links attention to representations or to task performance. Without such evidence, the layer-to-function mapping is an unsupported interpretive assumption; at minimum it should be labeled as a hypothesis, and ideally tested, e.g., by comparing layer-wise attention peaks with layer-wise probing accuracy for the same MWE instances.
  4. [Section 3.1; Section 5.3] The cross-linguistic comparisons in Section 5.3 are confounded with dataset source and model architecture. The MSU dataset covers only Slavic languages, and the idiom datasets come from different sources per language (ID10M for EN/DE/NL/PL, a Russian idiom corpus plus dictionary for RU, and ChatGPT-generated data for UK). Differences between Germanic and Slavic attention patterns could therefore reflect dataset composition, annotation guidelines, or model choice rather than language properties. The Limitations section acknowledges the sources differ, but Section 5.3 still attributes the observed differences to morphological complexity; this attribution should be removed or supported with a matched analysis.
minor comments (6)
  1. [Section 5.1] The sentence 'Fine-tuning on syntactic tasks (Topic and DepRel)...' misclassifies Topic as syntactic, although Section 3.3.2 defines Topic as a semantic task; this should be corrected to avoid confusing the reported pattern.
  2. [Table 3 caption] The color coding of lower and middle layers is not reproducible in plain text; consider replacing colors with an unambiguous notation such as superscripts or labels.
  3. [Section 4.2–4.3] The exact definition of 'attention percentage' is not fully specified: the denominator, the treatment of [CLS]/[SEP] tokens, and the aggregation of subword tokens should be stated explicitly so that the reported percentages are reproducible.
  4. [Table 3 caption; Section 5.2] The layer grouping thresholds 'lower 1–8, middle 9–16, upper 17–24' are introduced without justification and are used in the discussion; the authors should state whether the conclusions depend on this particular partition.
  5. [Figure 1 caption; Section 5.1] The Figure 1 caption says results are shown for English and Ukrainian, while Section 5.1 says the figure shows PL, UK, and EN; the coverage should be reconciled.
  6. [Section 3.1.2] The text says ID10M test sets were available for EN and DE, yet PL is used in the experiments; please clarify which PL idiom data were used and how they were validated.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the attention measurements are direct model outputs, and the only self-citation supplies input data rather than the derived conclusions.

full rationale

The paper does not derive its central claims from its inputs by construction. Fine-tuned and pre-trained models are run on idiom and MSU datasets, attention matrices are extracted, and layer-wise averages are reported. The full-text claims about attention distribution are empirical observations of model outputs, not quantities fitted from the data. The MSU dataset from Zaitova et al. (2023) is a prior self-citation, but it is used as a set of experimental stimuli; the attention scores are computed independently of how the dataset was constructed. The paper's interpretive dependence on Tenney et al.'s claim that lower layers encode syntax and higher layers encode semantics is a background assumption from external work, not a circular reduction, and the paper explicitly states this assumption rather than hiding it. The lack of significance tests and the visual nature of the comparisons are a robustness or statistical-support concern, not a form of circularity. Accordingly, no step in the paper reduces a prediction to its own inputs, and the correct circularity score is zero.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

No new physical or computational entities are introduced. The analysis depends on interpretive assumptions and a hand-chosen layer grouping rather than fitted parameters.

free parameters (1)
  • Layer grouping thresholds (lower 1-8, middle 9-16, upper 17-24) = 1-8, 9-16, 17-24
    Used in Table 3 to classify attention layers; chosen by hand, not derived from data. The paper's conclusions about 'lower' vs 'higher' layers depend on this grouping.
assumptions (3)
  • domain assumption Lower layers of BERT encode syntactic information and higher layers encode semantic information (from Tenney et al., 2019).
    Invoked in Section 5.2 to interpret attention peaks at layers 3-4 as syntactic processing and spread attention as semantic integration; not validated for each of the six models.
  • domain assumption Averaged attention scores across heads and subword tokens reflect the model's meaningful focus on MWEs.
    Sections 4.2 and 4.3 average attention matrices over heads and aggregate subword tokens; attention is treated as a direct measure of processing emphasis without validation against other interpretability methods.
  • domain assumption The MSU and idiom datasets are comparable across languages despite different sources, definitions, and language coverage.
    Section 3.1 constructs idioms from ID10M, manual sources, and GPT-4, while MSUs come from a Slavic-only corpus; the paper acknowledges in Limitations that dataset variability could affect generalizability.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Attention on Multiword Expressions: A Multilingual Study of BERT-based Models with Regard to Idiomaticity and Microsyntax." pith.science (2026). https://pith.science/paper/OL2FWTJ7

@misc{pith2026250506062,
  author       = {Pith},
  title        = {Pith review of: Attention on Multiword Expressions: A Multilingual Study of BERT-based Models with Regard to Idiomaticity and Microsyntax},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OL2FWTJ7}},
  note         = {Machine review of arXiv:2505.06062}
}
read the original abstract

This study analyzes the attention patterns of fine-tuned encoder-only models based on the BERT architecture (BERT-based models) towards two distinct types of Multiword Expressions (MWEs): idioms and microsyntactic units (MSUs). Idioms present challenges in semantic non-compositionality, whereas MSUs demonstrate unconventional syntactic behavior that does not conform to standard grammatical categorizations. We aim to understand whether fine-tuning BERT-based models on specific tasks influences their attention to MWEs, and how this attention differs between semantic and syntactic tasks. We examine attention scores to MWEs in both pre-trained and fine-tuned BERT-based models. We utilize monolingual models and datasets in six Indo-European languages - English, German, Dutch, Polish, Russian, and Ukrainian. Our results show that fine-tuning significantly influences how models allocate attention to MWEs. Specifically, models fine-tuned on semantic tasks tend to distribute attention to idiomatic expressions more evenly across layers. Models fine-tuned on syntactic tasks show an increase in attention to MSUs in the lower layers, corresponding with syntactic processing requirements.

Figures

Figures reproduced from arXiv: 2505.06062 by the authors.

Figure 1
Figure 1. Layer-wise attention distribution in BERT [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Layer-wise attention distribution within Multi [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Differences in Attention Patterns for Idioms and Microsyntactic Units (MSUs) in Russian. The bars show [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

40 extracted references · 19 canonical work pages

  1. [1]

    Russian N ational C orpus

    2003--2023. Russian N ational C orpus. Accessed 25.09.2024

    work page 2003

  2. [2]

    Alabi, Yanke Mao, Haonan Gao, and Annie En-Shiun Lee

    David Ifeoluwa Adelani, Hannah Liu, Xiaoyu Shen, Nikita Vassilyev, Jesujoba O. Alabi, Yanke Mao, Haonan Gao, and Annie En-Shiun Lee. 2023. https://arxiv.org/abs/2309.07445 Sib-200: A simple, inclusive, and big evaluation dataset for topic classification in 200+ languages and dialects . Preprint, arXiv:2309.07445

    arXiv 2023

  3. [3]

    Katsiaryna Aharodnik, Anna Feldman, and Jing Peng. 2018. https://api.semanticscholar.org/CorpusID:21722705 Designing a russian idiom-annotated corpus . In International Conference on Language Resources and Evaluation

    work page 2018

  4. [4]

    Tania Avgustinova and Leonid Iomdin. 2019. https://doi.org/10.1007/978-3-030-30135-4_2 Towards a Typology of Microsyntactic Constructions , volume 11755 of Lecture Notes in Computer Science. Springer, Cham., pages 15--30

    work page doi:10.1007/978-3-030-30135-4_2 2019

  5. [5]

    Andrei Avram, Verginica Barbu Mititelu, and Dumitru-Clementin Cercel. 2023. https://doi.org/10.18653/v1/2023.mwe-1.4 R omanian multiword expression detection using multilingual adversarial training and lateral inhibition . In Proceedings of the 19th Workshop on Multiword Expressions (MWE 2023), pages 7--13, Dubrovnik, Croatia. Association for Computationa...

    doi:10.18653/v1/2023.mwe-1.4 2023

  6. [6]

    Timothy Baldwin and Su Nam Kim. 2010. Multiword expressions. In Nitin Indurkhya and Fred J. Damerau, editors, Handbook of Natural Language Processing, pages 267--292. Chapman and Hall/CRC

    work page 2010

  7. [7]

    A Baranov and D Dobrovolsky, editors. 2015. Academic Dictionary of Russian Phraseology, 2 edition. LEKSRUS, Moscow

    work page 2015

  8. [8]

    Joanne Boisson, Jose Camacho-Collados, and Luis Espinosa-Anke. 2022. https://doi.org/10.18653/v1/2022.semeval-1.20 C ardiff NLP -metaphor at S em E val-2022 task 2: Targeted fine-tuning of transformer-based language models for idiomaticity detection . In Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 169--177, ...

    doi:10.18653/v1/2022.semeval-1.20 2022

Show all 40 references
  1. [9]

    Van-Tuan Bui and Agata Savary. 2024. https://aclanthology.org/2024.lrec-main.374 Cross-type F rench multiword expression identification with pre-trained masked language models . In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Re...

    2024

  2. [10]

    Branden Chan, Stefan Schweter, and Timo M \"o ller. 2020. https://doi.org/10.18653/v1/2020.coling-main.598 G erman ' s next language model . In Proceedings of the 28th International Conference on Computational Linguistics, pages 6788--6796, Barcelona, Spain (Online). Internati...

    2020 doi

  3. [11]

    Verna Dankers, Christopher Lucas, and Ivan Titov. 2022. https://doi.org/10.18653/v1/2022.acl-long.252 Can transformer be too compositional? analysing idiom processing in neural machine translation . In Proceedings of the 60th Annual Meeting of the Association for Computational...

    2022 doi

  4. [12]

    P Delobelle and F Remy. 2023. https://clin33.uantwerpen.be/abstract/robbert-2023-keeping-dutch-language-models-up-to-date-at-a-lower-cost-thanks-to-model-conversion/ Robbert-2023: Keeping dutch language models up-to-date at a lower cost thanks to model conversion

    2023

  5. [13]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. https://doi.org/10.18653/v1/N19-1423 BERT : Pre-training of deep bidirectional transformers for language understanding . In Proceedings of the 2019 Conference of the North A merican Chapter of the Associat...

    2019 doi

  6. [14]

    Marzieh Fadaee, Arianna Bisazza, and Christof Monz. 2018. Examining the tip of the iceberg: A data set for idiom translation. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation ( LREC 2018) , Miyazaki, Japan. European Language Resource...

    2018

  7. [15]

    Mykola Haltiuk and Aleksander Smywi \'n ski-Pohl. 2024. https://aclanthology.org/2024.unlp-1.14 L i BERT a: Advancing U krainian language modeling through pre-training from scratch . In Proceedings of the Third Ukrainian Natural Language Processing Workshop (UNLP) @ LREC-COLIN...

    2024

  8. [16]

    Leonid Iomdin. 2015. Microsyntactic constructions formed by the Russian word raz. SLAVIA cˇasopis pro slovanskou filologii, 84(3)

    2015

  9. [17]

    Leonid Iomdin. 2016. https://aclanthology.org/W16-3803 Microsyntactic phenomena as a computational linguistics issue . In Proceedings of the Workshop on Grammar and Lexicon: interactions and interfaces ( G ram L ex) , pages 8--17, Osaka, Japan. The COLING 2016 Organizing Committee

    2016

  10. [18]

    Dongjun Jang, Sungjoo Byun, and Hyopil Shin. 2024. https://aclanthology.org/2024.lrec-main.148 A study on how attention scores in the BERT model are aware of lexical categories in syntactic and semantic tasks on the GLUE benchmark . In Proceedings of the 2024 Joint Internation...

    2024

  11. [19]

    Murathan Kurfal . 2020. https://aclanthology.org/2020.mwe-1.18 TRAVIS at PARSEME shared task 2020: How good is (m) BERT at seeing the unseen? In Proceedings of the Joint Workshop on Multiword Expressions and Electronic Lexicons, pages 136--141, online. Association for Computat...

    2020

  12. [20]

    Tomáš Machálek. 2020. Kontext: Advanced and flexible corpus query interface. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 7003--7008, Marseille, France. European Language Resources Association

    2020

  13. [21]

    Francesca Masini. 2019. https://doi.org/10.1093/acrefore/9780199384655.013.611 Multi-word expressions and morphology . Oxford University Press, Oxford

    2019

  14. [22]

    Filip Mileti \'c and Sabine Schulte im Walde. 2024. https://doi.org/10.1162/tacl_a_00657 Semantics of multiword expressions in transformer-based models: A survey . Transactions of the Association for Computational Linguistics, 12:593--612

    2024 doi

  15. [23]

    Robert Mroczkowski, Piotr Rybak, Alina Wr \'o blewska, and Ireneusz Gawlik. 2021. https://www.aclweb.org/anthology/2021.bsnlp-1.1 H er BERT : Efficiently pretrained transformer-based language model for P olish . In Proceedings of the 8th Workshop on Balto-Slavic Natural Langua...

    2021

  16. [24]

    Manning, Sampo Pyysalo, Sebastian Schuster, Francis Tyers, and Daniel Zeman

    Joakim Nivre, Marie-Catherine de Marneffe, Filip Ginter, Jan Haji c , Christopher D. Manning, Sampo Pyysalo, Sebastian Schuster, Francis Tyers, and Daniel Zeman. 2020. https://aclanthology.org/2020.lrec-1.497 U niversal D ependencies v2: An evergrowing multilingual treebank co...

    2020

  17. [25]

    Xiaoman Pan, Boliang Zhang, Jonathan May, Joel Nothman, Kevin Knight, and Heng Ji. 2017. https://doi.org/10.18653/v1/P17-1178 Cross-lingual name tagging and linking for 282 languages . In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (...

    2017 doi

  18. [26]

    Giulia Rambelli, Emmanuele Chersoni, Marco S. G. Senaldi, Philippe Blache, and Alessandro Lenci. 2023. https://doi.org/10.18653/v1/2023.mwe-1.13 Are frequent phrases directly retrieved like idioms? an investigation with self-paced reading and language models . In Proceedings o...

    2023 doi

  19. [27]

    Vered Shwartz and Ido Dagan. 2019. https://doi.org/10.1162/tacl_a_00277 Still a pain in the neck: Evaluating text representations on lexical composition . Transactions of the Association for Computational Linguistics, 7:403--419

    2019 doi

  20. [28]

    Minghuan Tan and Jing Jiang. 2021. https://aclanthology.org/2021.ranlp-1.156/ Does BERT understand idioms? a probing-based empirical study of BERT encodings of idioms . In Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021...

    2021

  21. [29]

    Simone Tedeschi, Federico Martelli, and Roberto Navigli. 2022. https://doi.org/10.18653/v1/2022.findings-naacl.208 ID 10 M : Idiom identification in 10 languages . In Findings of the Association for Computational Linguistics: NAACL 2022, pages 2715--2726, Seattle, United State...

    2022 doi

  22. [30]

    Ian Tenney, Dipanjan Das, and Ellie Pavlick. 2019. https://arxiv.org/abs/1905.05950 Bert rediscovers the classical nlp pipeline . Preprint, arXiv:1905.05950

    2019 arXiv

  23. [31]

    Ye Tian, Isobel James, and Hye Son. 2023. https://doi.org/10.18653/v1/2023.starsem-1.16 How are idioms processed inside transformer language models? In Proceedings of the 12th Joint Conference on Lexical and Computational Semantics (*SEM 2023), pages 174--179, Toronto, Canada....

    2023 doi

  24. [32]

    Abigail Walsh, Teresa Lynn, and Jennifer Foster. 2022. https://aclanthology.org/2022.mwe-1.13 A BERT ' s eye view: Identification of I rish multiword expressions using pre-trained language models . In Proceedings of the 18th Workshop on Multiword Expressions @LREC2022, pages 8...

    2022

  25. [33]

    Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. 2018. https://doi.org/10.18653/v1/W18-5446 GLUE : A multi-task benchmark and analysis platform for natural language understanding . In Proceedings of the 2018 EMNLP Workshop B lackbox NLP : A...

    2018 doi

  26. [34]

    Beatrice Warren. 2005. A model of idiomaticity. Nordic Journal of English Studies, 4:35--54

    2005

  27. [35]

    Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mari...

    2020

  28. [36]

    Iuliia Zaitova, Irina Stenger, and Tania Avgustinova. 2023. https://aclanthology.org/2023.ranlp-1.134 Microsyntactic unit detection using word embedding models: Experiments on S lavic languages . In Proceedings of the 14th International Conference on Recent Advances in Natural...

    2023

  29. [37]

    Ziheng Zeng and Suma Bhat. 2021. https://doi.org/10.1162/tacl_a_00442 Idiomatic Expression Identification using Semantic Compatibility . Transactions of the Association for Computational Linguistics, 9:1546--1562

    2021 doi

  30. [38]

    Dmitry Zmitrovich, Alexander Abramov, Andrey Kalmykov, Maria Tikhonova, Ekaterina Taktasheva, Danil Astafurov, Mark Baushenko, Artem Snegirev, Tatiana Shavrina, Sergey Markov, Vladislav Mikhailov, and Alena Fenogenova. 2023. https://arxiv.org/abs/2309.10931 A family of pretrai...

    2023 arXiv

  31. [39]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  32. [40]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.