robo-vln (Robotics Vision-and-Language Navigation)
The Robo-VLN dataset is a continuous control formulation of the VLN-CE dataset by Krantz et al ported over from Room-to-Room (R2R) dataset created by Anderson et al. The details regarding converting discrete VLN dataset into continuous control formulation can be found in our paper.
| Dataset | Path to extract | Size |
|---|---|---|
| robo_vln_v1.zip | data/datasets/robo_vln_v1 |
76.9 MB |
Robo-VLN Dataset
The dataset robo_vln_v1 contains the train, val_seen, and val_unseen splits.
- train: 7739 episodes
- val_seen: 570 episodes
- val_unseen: 1224 episodes
Format of {split}.json.gz
{
'episodes' = [
{
'episode_id': 4991,
'trajectory_id': 3279,
'scene_id': 'mp3d/JeFG25nYj2p/JeFG25nYj2p.glb',
'instruction': {
'instruction_text': 'Walk past the striped area rug...',
'instruction_tokens': [2384, 1589, 2202, 2118, 133, 1856, 9]
},
'start_position': [10.257800102233887, 0.09358400106430054, -2.379739999771118],
'start_rotation': [0, 0.3332950713608026, 0, 0.9428225683587541],
'goals': [
{
'position': [3.360340118408203, 0.09358400106430054, 3.07817006111145],
'radius': 3.0
}
],
'reference_path': [
[10.257800102233887, 0.09358400106430054, -2.379739999771118],
[9.434900283813477, 0.09358400106430054, -1.3061100244522095]
...
[3.360340118408203, 0.09358400106430054, 3.07817006111145],
],
'info': {'geodesic_distance': 9.65537166595459},
},
...
],
'instruction_vocab': [
'word_list': [..., 'orchids', 'order', 'orient', ...],
'word2idx_dict': {
...,
'orchids': 1505,
'order': 1506,
'orient': 1507,
...
},
'itos': [..., 'orchids', 'order', 'orient', ...],
'stoi': {
...,
'orchids': 1505,
'order': 1506,
'orient': 1507,
...
},
'num_vocab': 2504,
'UNK_INDEX': 1,
'PAD_INDEX': 0,
]
}
- Format of
{split}_gt.json.gz
{
'4991': {
'actions': [
...
[-0.999969482421875, 1.0],
[-0.9999847412109375, 0.15731772780418396],
...
],
'forward_steps': 325,
'locations': [
[10.257800102233887, 0.09358400106430054, -2.379739999771118],
[10.257800102233887, 0.09358400106430054, -2.379739999771118],
...
[-12.644463539123535, 0.1518409252166748, 4.2241311073303220]
]
}
...
}
Benchmarks
| Trend | Task | Dataset Variant | Best Model | Paper | Code |
|---|---|---|---|---|---|
|
robo-vln
|
Hierarchical Cross-Modal Agent
|
Papers
| Paper | Code | Results | Date | Stars |
|---|
Dataset Loaders
No data loaders found. You can
submit your data loader here.
No data loaders found. You can
submit your data loader here.
