REVIEW 1 cited by
Behavioral Cloning via Search in Video PreTraining Latent Space
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Our aim is to build autonomous agents that can solve tasks in environments like Minecraft. To do so, we used an imitation learning-based approach. We formulate our control problem as a search problem over a dataset of experts' demonstrations, where the agent copies actions from a similar demonstration trajectory of image-action pairs. We perform a proximity search over the BASALT MineRL-dataset in the latent representation of a Video PreTraining model. The agent copies the actions from the expert trajectory as long as the distance between the state representations of the agent and the selected expert trajectory from the dataset do not diverge. Then the proximity search is repeated. Our approach can effectively recover meaningful demonstration trajectories and show human-like behavior of an agent in the Minecraft environment.
Forward citations
Cited by 1 Pith paper
-
STEVE-Audio: Expanding the Goal Conditioning Modalities of Embodied Agents in Minecraft
An audio-conditioned STEVE-1 agent, built with a new Minecraft audio-video CLIP model and a learned prior, matches or beats text- and video-conditioned versions on most short-horizon collection tasks.
Discussion (0). Continue with ORCID to comment.