Pith. sign in

REVIEW 2 cited by

Are LLMs Models of Distributional Semantics? A Case Study on Quantifiers

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2410.13984 v1 pith:6UER3DOR submitted 2024-10-17 cs.CL

Zhang Enyan , Zewei Wang , Michael A. Lepori , Ellie Pavlick , Helena Aparicio This is my paper · ORCID
classification cs.CL
keywords distributionalmodelssemanticslanguagequantifiersvaguecapturecase
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Distributional semantics is the linguistic theory that a word's meaning can be derived from its distribution in natural language (i.e., its use). Language models are commonly viewed as an implementation of distributional semantics, as they are optimized to capture the statistical features of natural language. It is often argued that distributional semantics models should excel at capturing graded/vague meaning based on linguistic conventions, but struggle with truth-conditional reasoning and symbolic processing. We evaluate this claim with a case study on vague (e.g. "many") and exact (e.g. "more than half") quantifiers. Contrary to expectations, we find that, across a broad range of models of various types, LLMs align more closely with human judgements on exact quantifiers versus vague ones. These findings call for a re-evaluation of the assumptions underpinning what distributional semantics models are, as well as what they can capture.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Do Large Language Models Advocate for Inferentialism?

    cs.CL 2024-12 conditional novelty 6.0 of 10

    Large language models process language in ways that align with inferentialist, anti-representationalist semantics, and they may be understood through a consensus theory of truth grounded in human feedback.

  2. A Judge-free LLM Open-ended Generation Benchmark Based on the Distributional Hypothesis

    cs.CL 2025-02 conditional novelty 5.0 of 10

    A deterministic n-gram benchmark for Japanese open-ended QA, built from LLM-generated reference answer sets, that reports a 0.9896 correlation with GPT-4o judge scores.

Pith tools