REVIEW 9 cited by
Can Language Models Encode Perceptual Structure Without Grounding? A Case Study in Color
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Pretrained language models have been shown to encode relational information, such as the relations between entities or concepts in knowledge-bases -- (Paris, Capital, France). However, simple relations of this type can often be recovered heuristically and the extent to which models implicitly reflect topological structure that is grounded in world, such as perceptual structure, is unknown. To explore this question, we conduct a thorough case study on color. Namely, we employ a dataset of monolexemic color terms and color chips represented in CIELAB, a color space with a perceptually meaningful distance metric. Using two methods of evaluating the structural alignment of colors in this space with text-derived color term representations, we find significant correspondence. Analyzing the differences in alignment across the color spectrum, we find that warmer colors are, on average, better aligned to the perceptual color space than cooler ones, suggesting an intriguing connection to findings from recent work on efficient communication in color naming. Further analysis suggests that differences in alignment are, in part, mediated by collocationality and differences in syntactic usage, posing questions as to the relationship between color perception and usage and context.
Forward citations
Cited by 9 Pith papers
-
ICLR: In-Context Learning of Representations
As in-context examples grow, Llama-3.1-8B reorganizes its concept representations to mirror the connectivity structure of a graph defined entirely in context.
-
Text-to-Image Generation for Projector-Camera System Registration
Text-conditioned images with controlled ORB-feature distributions plus a distortion-trained registration network enable accurate, natural-image procam registration on smooth surfaces.
-
Tracking World States with Language Models: State-Based Evaluation Using Chess
A model-agnostic chess evaluation metric measures state-tracking fidelity by comparing legal-move sets of predicted and true positions, showing GPT-4o's reconstruction quality degrades over longer games.
-
What Does it Mean for a Neural Network to Learn a "World Model"?
Defines a world model as a simple commutative-diagram factorization through an intermediate representation, with conditions that the model be learned and emergent rather than inherited from input or output.
-
What Do Large Language Models Know? Tacit Knowledge as a Potential Causal-Explanatory Structure
Transformer LLMs can meet Davies's tacit knowledge conditions if the 'same syntax' requirement is replaced by semantic closeness in embedding space.
-
Haunted House: A text-based game for comparing the flexibility of mental models in humans and LLMs
A new text-based navigation game shows humans greatly outperform current LLMs at active, clue-driven spatial reasoning.
-
Harmonic Loss Trains Interpretable AI Models
Harmonic loss, which scores logits by inverse Euclidean distance to class prototypes with a scale-invariant normalization, yields more interpretable class centers, reduced grokking, and faster convergence across MLPs,...
-
Improving the Data-efficiency of Reinforcement Learning by Warm-starting with LLM
Warm-starting RL with a small LLM-generated buffer and then fine-tuning online (LORO) improves cumulative reward over pure RL in six Gym environments, though the theoretical guarantee is borrowed and the coverage assu...
-
Probing Audio-Generation Capabilities of Text-Based Language Models
Text-only LLMs can synthesize simple musical notes via generated Python code, but their environmental sound outputs score near chance and speech generation fails entirely.
Discussion (0). Continue with ORCID to comment.