BLaIR is a new benchmark and 570M-review dataset showing that LLM performance rankings on recommendation tasks have little correlation with rankings on general embedding benchmarks like MTEB.
Title resolution pending
76 Pith papers cite this work. Polarity classification is still indexing.
76
Pith papers citing it
citation-role summary
other 1
citation-polarity summary
roles
other 1polarities
unclear 1representative citing papers
Brain Score remains similar when language models are trained on diverse natural languages or on structured non-language data like DNA and code, indicating the metric tracks shared structural extraction but is not diagnostic of human-like language processing.
Merging fine-tuned models for multilingual translation fails because fine-tuning redistributes language-specific neurons rather than sharpening them, increasing representational divergence in output-generating layers.
Cascaded systems remain the most reliable for speech translation overall, but recent SpeechLLMs match or outperform them in many conditions while standalone speech models lag.
SciNLP is the first full-text entity and relation extraction benchmark for the NLP domain, built from 60 manually annotated publications and used to evaluate models and construct a domain knowledge graph.
Proposes a textbook-based true/false QA task where PTLMs score ~50% closed-book even after pre-training on the text and ~60% open-book with retrieval.
NarrativeTime is a timeline annotation framework achieving full TLink coverage, shown via re-annotation of TimeBankDense with comparable agreement and higher density plus a new TimeBankNT corpus.
FIESTA uses bandit algorithms to adaptively decide how many seeds and splits to run for each candidate model, focusing effort on promising ones while providing guarantees on selecting the optimal model.
Across 28 languages, an information-theoretic irregularity score derived from neural transduction models correlates positively with frequency, with stronger effects when aggregated over paradigms.
A fully differentiable parser that stochastically samples projective dependency trees using Gumbel perturbations and dynamic programming to boost downstream task performance without direct supervision.
A new dataset DDEP and reliability-weighted fusion model Rel-DDEP jointly detect deception, emotion, and personality from multimodal data, reporting F1 gains of 2.53%, 2.66%, and 9.30% over baselines.
SpreadsheetAgent uses incremental multi-format reading, structural sketching, and verification to raise spreadsheet benchmark accuracy from 35.27% to 38.16%.
EMERGE is a benchmark dataset of 233K Wikipedia passages paired with 1.45 million Wikidata edit operations across seven yearly snapshots from 2019 to 2025 for evaluating knowledge graph updates from emerging text.
MolReFlect introduces a teacher-student framework that automatically creates fine-grained molecule-text alignments to achieve SOTA results on molecule-caption translation.
The study filters non-English Wikipedia, reveals quality problems, proposes a 4-level ranking, and shows filtered data matches or beats raw data in language modeling with largest gains for lower-quality editions.
Systematic LLM evaluation for news framing detection reveals prompt sensitivity and emotional-language bias, introduces an out-of-domain headline dataset, and shows cross-model consensus aids annotation auditing.
TaDSE learns dialogue sentence embeddings via template-guided self-supervised contrastive learning plus synthetic slot-filling augmentation and reports gains on five downstream benchmarks.
Introduces the first interpersonal emotion dataset from congressional tweets and demonstrates that joint neural modeling of interpersonal group relationships and emotions yields performance gains on both.
GPT-NeoX-20B is a publicly released 20B parameter autoregressive language model trained on the Pile that shows strong gains in five-shot reasoning over similarly sized prior models.
Semantic constituency graphs outperform syntactic constituency and dependency structures from seven formalisms when added to a Transformer for language modeling.
Deduplicating training datasets reduces language model verbatim memorization by 10x, improves training efficiency, and enables more accurate evaluation by cutting train-test overlap.
Introduces a new English dataset from r/AskParents and r/needadvice annotated for advice sentences plus preliminary models showing pre-trained LMs outperform rule-based systems but the task remains challenging.
Gated lexical shortcut connections added to the transformer yield 0.9 BLEU average gains on five WMT directions while lowering the lexical content stored in hidden states.
The first shared task on MT robustness received 23 submissions showing up to +22.33 BLEU gains on noisy Reddit data, with strong human-BLEU correlation.
citing papers explorer
-
Bridging Language and Items for Retrieval and Recommendation: Benchmarking LLMs as Semantic Encoders
BLaIR is a new benchmark and 570M-review dataset showing that LLM performance rankings on recommendation tasks have little correlation with rankings on general embedding benchmarks like MTEB.
-
Brain Score Tracks Shared Properties of Languages: Evidence from Many Natural Languages and Structured Sequences
Brain Score remains similar when language models are trained on diverse natural languages or on structured non-language data like DNA and code, indicating the metric tracks shared structural extraction but is not diagnostic of human-like language processing.
-
One Model to Translate Them All? A Journey to Mount Doom for Multilingual Model Merging
Merging fine-tuned models for multilingual translation fails because fine-tuning redistributes language-specific neurons rather than sharpening them, increasing representational divergence in output-generating layers.
-
Hearing to Translate: The Effectiveness of Speech Modality Integration into LLMs
Cascaded systems remain the most reliable for speech translation overall, but recent SpeechLLMs match or outperform them in many conditions while standalone speech models lag.
-
SciNLP: A Domain-Specific Benchmark for Full-Text Scientific Entity and Relation Extraction in NLP
SciNLP is the first full-text entity and relation extraction benchmark for the NLP domain, built from 60 manually annotated publications and used to evaluate models and construct a domain knowledge graph.
-
Perhaps PTLMs Should Go to School -- A Task to Assess Open Book and Closed Book QA
Proposes a textbook-based true/false QA task where PTLMs score ~50% closed-book even after pre-training on the text and ~60% open-book with retrieval.
-
NarrativeTime: Dense Temporal Annotation on a Timeline
NarrativeTime is a timeline annotation framework achieving full TLink coverage, shown via re-annotation of TimeBankDense with comparable agreement and higher density plus a new TimeBankNT corpus.
-
FIESTA: Fast IdEntification of State-of-The-Art models using adaptive bandit algorithms
FIESTA uses bandit algorithms to adaptively decide how many seeds and splits to run for each candidate model, focusing effort on promising ones while providing guarantees on selecting the optimal model.
-
Morphological Irregularity Correlates with Frequency
Across 28 languages, an information-theoretic irregularity score derived from neural transduction models correlates positively with frequency, with stronger effects when aggregated over paradigms.
-
Learning Latent Trees with Stochastic Perturbations and Differentiable Dynamic Programming
A fully differentiable parser that stochastically samples projective dependency trees using Gumbel perturbations and dynamic programming to boost downstream task performance without direct supervision.
-
Dynamic Emotion and Personality Profiling for Multimodal Deception Detection
A new dataset DDEP and reliability-weighted fusion model Rel-DDEP jointly detect deception, emotion, and personality from multimodal data, reporting F1 gains of 2.53%, 2.66%, and 9.30% over baselines.
-
Towards Robust Real-World Spreadsheet Understanding with Multi-Agent Multi-Format Reasoning
SpreadsheetAgent uses incremental multi-format reading, structural sketching, and verification to raise spreadsheet benchmark accuracy from 35.27% to 38.16%.
-
EMERGE: A Benchmark for Updating Knowledge Graphs with Emerging Textual Knowledge
EMERGE is a benchmark dataset of 233K Wikipedia passages paired with 1.45 million Wikidata edit operations across seven yearly snapshots from 2019 to 2025 for evaluating knowledge graph updates from emerging text.
-
MolReFlect: Towards In-Context Fine-grained Alignments between Molecules and Texts
MolReFlect introduces a teacher-student framework that automatically creates fine-grained molecule-text alignments to achieve SOTA results on molecule-caption translation.
-
How Good is Your Wikipedia? Auditing Data Quality for Low-resource and Multilingual NLP
The study filters non-English Wikipedia, reveals quality problems, proposes a 4-level ranking, and shows filtered data matches or beats raw data in language modeling with largest gains for lower-quality editions.
-
Decoding News Narratives: A Critical Analysis of Large Language Models in Framing Detection
Systematic LLM evaluation for news framing detection reveals prompt sensitivity and emotional-language bias, introduces an out-of-domain headline dataset, and shows cross-model consensus aids annotation auditing.
-
Template-assisted Contrastive Learning of Task-oriented Dialogue Sentence Embeddings
TaDSE learns dialogue sentence embeddings via template-guided self-supervised contrastive learning plus synthetic slot-filling augmentation and reports gains on five downstream benchmarks.
-
How people talk about each other: Modeling Generalized Intergroup Bias and Emotion
Introduces the first interpersonal emotion dataset from congressional tweets and demonstrates that joint neural modeling of interpersonal group relationships and emotions yields performance gains on both.
-
GPT-NeoX-20B: An Open-Source Autoregressive Language Model
GPT-NeoX-20B is a publicly released 20B parameter autoregressive language model trained on the Pile that shows strong gains in five-shot reasoning over similarly sized prior models.
-
Linguistic Frameworks Go Toe-to-Toe at Neuro-Symbolic Language Modeling
Semantic constituency graphs outperform syntactic constituency and dependency structures from seven formalisms when added to a Transformer for language modeling.
-
Deduplicating Training Data Makes Language Models Better
Deduplicating training datasets reduces language model verbatim memorization by 10x, improves training efficiency, and enables more accurate evaluation by cutting train-test overlap.
-
Help! Need Advice on Identifying Advice
Introduces a new English dataset from r/AskParents and r/needadvice annotated for advice sentences plus preliminary models showing pre-trained LMs outperform rule-based systems but the task remains challenging.
-
Widening the Representation Bottleneck in Neural Machine Translation with Lexical Shortcuts
Gated lexical shortcut connections added to the transformer yield 0.9 BLEU average gains on five WMT directions while lowering the lexical content stored in hidden states.
-
Findings of the First Shared Task on Machine Translation Robustness
The first shared task on MT robustness received 23 submissions showing up to +22.33 BLEU gains on noisy Reddit data, with strong human-BLEU correlation.
-
Compositional Semantic Parsing Across Graphbanks
A compositional neural semantic parser achieves competitive accuracies across diverse graphbanks for the first time and sets new state-of-the-art results on DM, PAS, PSD, AMR 2015 and EDS when combined with BERT and multi-task learning.
-
Gated Embeddings in End-to-End Speech Recognition for Conversational-Context Fusion
Gated fusion of fastText and BERT embeddings into an end-to-end ASR model captures multi-sentence conversational context and lowers word error rate on the Switchboard corpus.
-
A Generative Model for Punctuation in Dependency Trees
A generative model of latent underlying punctuation in dependency trees, trained on incomplete data via local likelihood maximization, produces plausible reconstructions across languages and beats baselines on restoration.
-
Saliency-driven Word Alignment Interpretation for Neural Machine Translation
Saliency-driven interpretation methods reveal that NMT models learn word alignments of better quality than fast-align under force decoding and consistent with automatic tools under free decoding.
-
Good Secretaries, Bad Truck Drivers? Occupational Gender Stereotypes in Sentiment Analysis
Authors release a new 800-sentence gender-balanced profession dataset and use it to test occupational gender stereotypes in three sentiment analysis models.
-
Language Modelling Makes Sense: Propagating Representations through WordNet for Full-Coverage Word Sense Disambiguation
Contextual embeddings are propagated through WordNet to produce full-coverage sense representations that let a simple k-NN classifier outperform prior neural WSD models.
-
Retrieving Sequential Information for Non-Autoregressive Neural Machine Translation
Reinforce-NAT and FS-decoder retrieve target sequential information for non-autoregressive translation, yielding higher BLEU than baseline NAT while preserving fast decoding and approaching autoregressive quality.
-
Identification of Tasks, Datasets, Evaluation Metrics, and Numeric Scores for Scientific Leaderboards Construction
Creates datasets and an extraction model that identifies task-dataset-metric-score information in NLP papers to support automatic leaderboard construction.
-
Informative Image Captioning with External Sources of Information
A multimodal Transformer ingests image features plus multiple external entity label sources and learns to control their appearance in fluent output captions.
-
A Theory of Training Profit-Optimal LLMs
Develops an economic model combining scaling laws with microeconomics to derive profit-optimal LLM training expenditure and model size in compute-bound and data-bound regimes.
-
MIPIC: Matryoshka Representation Learning via Self-Distilled Intra-Relational and Progressive Information Chaining
MIPIC trains Matryoshka representations using self-distilled intra-relational alignment and progressive information chaining, yielding competitive results on STS, NLI, and classification tasks especially at low dimensions.
-
PatchRecall: Patch-Driven Retrieval for Automated Program Repair
PatchRecall combines codebase matching and history-based retrieval from past issues to achieve higher recall of relevant files for automated program repair while keeping the retrieved set concise.
-
Testing the Assumptions of Active Learning for Translation Tasks with Few Samples
Informativeness and diversity of samples selected by active learning show no correlation with test performance on translation tasks using few samples; ordering and pre-training effects dominate instead.
-
Infherno: End-to-end Agent-based FHIR Resource Synthesis from Free-form Clinical Notes
Infherno deploys LLM agents with code execution and terminology tools to synthesize FHIR resources from unstructured clinical notes, matching human baseline performance on synthetic and real datasets.
-
Is Human-Like Text Liked by Humans? Multilingual Human Detection and Preference Against AI
Humans detect AI-generated text at 87.6% accuracy across 9 languages and 9 domains, outperforming prior near-random results, and do not always prefer human-written text when the source is unclear.
-
AdaSwitch: Adaptive Switching between Small and Large Agents for Effective Cloud-Local Collaborative Learning
AdaSwitch improves small local LLM performance on reasoning tasks by adaptively switching to a large cloud LLM upon detected errors, sometimes matching cloud results with far less overhead.
-
Data-CUBE: Data Curriculum for Instruction-based Sentence Representation Learning
Data-CUBE applies a two-level curriculum (TSP-based task ordering via simulated annealing plus difficulty-sorted mini-batches) to multi-task instruction tuning and reports gains on MTEB sentence representation tasks.
-
Remember what you did so you know what to do next
GPT-J with full action history achieves 3.5x improvement over RL in ScienceWorld and matches a two-stage system using 29x larger models.
-
Gradient-Boosted Decision Tree for Listwise Context Model in Multimodal Review Helpfulness Prediction
Introduces listwise attention, listwise loss, and GBDT predictor to improve multimodal review helpfulness ranking over prior FCNN and pairwise approaches.
-
Enriching and Controlling Global Semantics for Text Summarization
A normalizing-flow neural topic model plus control mechanism are added to Transformer summarizers to supply and regulate global semantics, with reported gains over prior models on five benchmarks.
-
Inducing Syntactic Trees from BERT Representations
Word deletion impact on BERT embeddings is measured to estimate syntactic reducibility of words and n-grams, then applied to induce dependency trees.
-
Eliciting Knowledge from Experts:Automatic Transcript Parsing for Cognitive Task Analysis
Introduces a weakly-supervised framework partitioning CTA transcript parsing into sequence labeling and text span-pair relation extraction using distant supervision from protocols and neighbor sentences for long-range context.
-
Interpretable Question Answering on Knowledge Bases and Text
Compares LIME, input perturbation and attention for explaining QA on KB+text; proposes automatic evaluation paradigm and finds input perturbation superior in both automatic and human studies.
-
Multimodal and Multi-view Models for Emotion Recognition
Multimodal training with attention and contrastive multi-view learning improves both combined and acoustic-only emotion recognition on IEMOCAP over prior acoustic baselines.
-
Conversational Response Re-ranking Based on Event Causality and Role Factored Tensor Event Embedding
Re-ranking conversational responses with event causality and role-factored tensor embeddings improves coherency and dialogue continuity.
-
Investigating Biases in Textual Entailment Datasets
Hypothesis-only classification reaches 64% accuracy on SNLI, revealing dataset biases in SNLI and MultiNLI that the authors quantify and propose a simple mitigation for.