A streaming XML function-token interface with multi-channel scheduling lets robots execute concurrent speech and motion while the LLM is still generating, reportedly beating native function calling 15/15 vs 6/15 on concurrent tasks.
What is Multimodality?
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
The last years have shown rapid developments in the field of multimodal machine learning, combining e.g., vision, text or speech. In this position paper we explain how the field uses outdated definitions of multimodality that prove unfit for the machine learning era. We propose a new task-relative definition of (multi)modality in the context of multimodal machine learning that focuses on representations and information that are relevant for a given machine learning task. With our new definition of multimodality we aim to provide a missing foundation for multimodal research, an important component of language grounding and a crucial milestone towards NLU.
fields
cs.RO 1years
2025 1verdicts
UNVERDICTED 1representative citing papers
citing papers explorer
-
GhostShell: Streaming LLM Function Calls for Concurrent Embodied Programming
A streaming XML function-token interface with multi-channel scheduling lets robots execute concurrent speech and motion while the LLM is still generating, reportedly beating native function calling 15/15 vs 6/15 on concurrent tasks.