vision-and-language-navigation-on-vln

Vision and Language Navigation

Results over time

Click legend items to toggle metrics. Hover points for model names.

Leaderboard

Paper	Code	success	length	error	oracle success	spl	ModelName	ReleaseDate
[]()		0.86	11.85	1.61	0.9	0.76	human
[]()		0.79	686.45	2.5	0.99	0.01	Lily
Airbert: In-domain Pretraining for Vision-and-Language Navigation	✓ Link	0.78	686.54	2.58	0.99	0.01	Airbert	2021-08-20
[]()		0.74	686.86	2.99	0.99	0.01	Global Normalization
[]()		0.74	625.27	3.55	0.99	0.01	explore@40 beam-search
Improving Vision-and-Language Navigation with Image-Text Pairs from the Web	✓ Link	0.73	686.62	3.09	0.99	0.01	VLN-Bert	2020-04-30
[]()		0.73	15.87	3.13	0.81	0.62	BEVBert
[]()		0.73	14.43	3.35	0.8	0.62	GMap
[]()		0.73	10.2	3.0	0.8	0.69	Gloabl Normalization pre-explore
[]()		0.72	1250.89	3.05	1.0	0.01	FOAM-Beam Search
[]()		0.72	16.14	3.44	0.79	0.6	Lily
[]()		0.72	15.24	3.33	0.78	0.61	ReadNet
[]()		0.71	176.22	3.07	0.94	0.05	Active Exploration (Beam Search)
Vision-Language Navigation with Self-Supervised Auxiliary Reasoning Tasks		0.71	40.85	3.24	0.81	0.21	Self-Supervised Auxiliary Reasoning Tasks (Beam Search)	2019-11-18
[]()		0.71	15.47	3.38	0.79	0.59	HOC
[]()		0.71	14.25	3.57	0.77	0.61	metaexplore
[]()		0.71	10.21	3.26	0.77	0.67	sponge
[]()		0.7	690.61	3.21	0.99	0.01	SERL (Beam_Search)
[]()		0.7	14.6	3.61	0.77	0.59	lxyict
[]()		0.7	14.39	3.55	0.76	0.6	DUET+PASTS
[]()		0.7	11.79	3.52	0.75	0.65	Single-run
[]()		0.7	9.85	3.3	0.77	0.68	Active Exploration (Pre-explore)
[]()		0.69	786.35	3.31	0.99	0.01	ADad
Learning to Navigate Unseen Environments: Back Translation with Environmental Dropout	✓ Link	0.69	686.82	3.26	0.99	0.01	null	2019-04-08
[]()		0.69	14.73	3.65	0.76	0.59	CVPR22
[]()		0.69	11.86	3.24	0.76	0.62	CMC-AAL2
[]()		0.68	11.9	3.59	0.73	0.64	EnvEdit+PT
Vision-Language Navigation with Self-Supervised Auxiliary Reasoning Tasks		0.68	10.43	3.69	0.75	0.65	Self-Supervised Auxiliary Reasoning Tasks (Pre-explore)	2019-11-18
[]()		0.67	13.55	3.74	0.73	0.61	DDL
[]()		0.67	12.07	3.41	0.76	0.6	CMG-AAL
[]()		0.66	19.62	3.85	0.74	0.55	VLN-TreeTrans
[]()		0.66	16.41	3.77	0.71	0.59	sliu_team
[]()		0.66	15.89	3.73	0.73	0.6	Single-Run, No Pre-Explore
[]()		0.66	14.78	3.68	0.72	0.6	TD-STP
[]()		0.66	13.07	3.67	0.73	0.6	VLN-BERT-Aug
[]()		0.66	12.75	3.86	0.72	0.6	Fortest
[]()		0.66	11.89	3.77	0.72	0.63	ESceme Single-run
[]()		0.65	15.9	3.78	0.71	0.59	WIN
[]()		0.65	15.9	3.78	0.71	0.59	WIN + RecVLN BERT
Vision-Language Navigation with Random Environmental Mixup	✓ Link	0.65	13.11	3.87	0.72	0.59	single-run	2021-06-15
[]()		0.65	12.71	3.82	0.72	0.6	Single-Run
[]()		0.65	12.27	3.93	0.72	0.6	HAMT
[]()		0.65	12.22	3.86	0.71	0.61	coefficient
[]()		0.65	11.91	4.0	0.7	0.6	bin
[]()		0.65	10.24	3.76	0.71	0.62	Greedy, No Pre-explore
[]()		0.64	13.75	3.97	0.71	0.58	DDL
[]()		0.64	12.84	3.9	0.7	0.58	clin
[]()		0.64	12.6	3.88	0.71	0.59	single-run
[]()		0.64	12.31	3.86	0.71	0.59	PANDA-TingLiu
[]()		0.64	12.3	3.94	0.71	0.59	single-run
[]()		0.64	9.79	3.97	0.7	0.61	Back Translation with Environmental Dropout (exploring unseen environments before testing)
[]()		0.63	357.62	4.03	0.96	0.02	Reinforced Cross-Modal Matching (optimized for SR; with beam search)
[]()		0.63	16.44	4.0	0.69	0.58	Hikari
[]()		0.63	13.57	4.02	0.71	0.57	SEvol_lzy
[]()		0.63	13.54	3.98	0.7	0.58	Colab_buaa
[]()		0.63	13.02	4.04	0.7	0.58	Geo
[]()		0.63	12.77	3.96	0.7	0.57	YBYB
[]()		0.63	12.62	3.99	0.71	0.58	hellohellohello
[]()		0.63	12.51	4.16	0.69	0.58	ed
[]()		0.63	12.35	4.09	0.7	0.57	Single-Run, No Pre-Explore
[]()		0.63	12.35	4.09	0.7	0.57	reg
[]()		0.63	12.35	4.09	0.7	0.57	ART
[]()		0.63	12.3	4.07	0.7	0.58	binbin
[]()		0.62	16.94	4.27	0.72	0.49	CCC(ssm)
[]()		0.62	10.22	4.18	0.67	0.58	MARVAL
[]()		0.61	373.09	4.48	0.97	0.02	Self-Aware Co-Grounded Model
Tactical Rewind: Self-Correction via Backtracking in Vision-and-Language Navigation	✓ Link	0.61	196.53	4.29	0.9	0.03	Tactical Rewind - long	2019-03-06
[]()		0.61	20.39	4.57	0.7	0.46	SSM
[]()		0.61	15.83	4.3	0.68	0.55	single-run
[]()		0.61	13.43	4.32	0.69	0.55	zq
[]()		0.61	13.2	4.22	0.69	0.56	GBSE
[]()		0.61	12.73	4.26	0.67	0.55	hellohello
[]()		0.61	12.13	4.27	0.67	0.56	homebody
[]()		0.61	10.66	4.02	0.72	0.55	GraphBert
[]()		0.6	21.03	4.34	0.71	0.43	Single Run
[]()		0.6	9.48	4.21	0.67	0.59	SIL-R2
[]()		0.59	14.29	4.26	0.66	0.55	Envdrop+SEVol+BT
[]()		0.59	13.07	4.53	0.66	0.53	a new baseline
[]()		0.59	12.22	4.49	0.67	0.54	trysth
[]()		0.59	10.31	4.71	0.64	0.55	SEA features + AuxRN (single-run)
[]()		0.59	10.21	4.52	0.64	0.56	PREVALENT
[]()		0.58	13.0	4.45	0.67	0.53	SQANv1, No Pre-explore
Neighbor-view Enhanced Model for Vision and Language Navigation	✓ Link	0.58	12.98	4.37	0.66	0.54	MM2021	2021-07-15
[]()		0.58	10.71	4.95	0.65	0.55	without pre-explore, beam-search
[]()		0.57	13.16	4.61	0.65	0.5	CMG-AAL-TCSVT
[]()		0.57	12.34	4.59	0.65	0.53	Single-Run, No Pre-Explore
[]()		0.57	10.99	4.57	0.65	0.5	test-sf
[]()		0.57	10.52	4.53	0.63	0.53	Greedy
[]()		0.56	1214.94	4.57	0.96	0.01	null
[]()		0.56	15.74	4.84	0.69	0.48	liuer
[]()		0.56	12.19	4.65	0.62	0.52	reward-vln
[]()		0.56	11.39	5.29	0.65	0.53	OAG(without pre-explore, beam-search)
[]()		0.56	10.58	5.17	0.63	0.52	jmebs
[]()		0.56	10.18	4.89	0.63	0.53	SEA features + Env-Dropout (single-run)
[]()		0.55	12.96	4.9	0.62	0.5	Single-Run
[]()		0.55	10.9	5.32	0.63	0.51	WQ_Pretrain
[]()		0.55	10.29	4.75	0.61	0.52	Lang-Vis-Entity VLN (Single-Run)
Tactical Rewind: Self-Correction via Backtracking in Vision-and-Language Navigation	✓ Link	0.54	22.08	5.14	0.64	0.41	Tactical Rewind - short	2019-03-06
[]()		0.54	14.31	5.24	0.64	0.46	single run
[]()		0.54	12.51	4.96	0.61	0.48	DCMT
[]()		0.54	11.74	5.35	0.64	0.5	map1
[]()		0.54	10.51	5.3	0.61	0.51	PREVALENT
[]()		0.54	10.06	5.11	0.62	0.52	27k
[]()		0.53	1257.38	4.87	0.96	0.01	Speaker-Follower
[]()		0.53	15.02	5.34	0.61	0.42	GVLN
[]()		0.53	12.13	5.63	0.61	0.49	SERL
[]()		0.53	10.4	5.3	0.61	0.5	2m-path
[]()		0.53	10.0	5.37	0.59	0.5	single-run
[]()		0.52	10.65	5.45	0.6	0.48	SYSU-ISE
[]()		0.51	13.05	5.14	0.6	0.45	licr19
Learning to Navigate Unseen Environments: Back Translation with Environmental Dropout	✓ Link	0.51	11.66	5.23	0.59	0.47	Back Translation with Environmental Dropout (no beam search)	2019-04-08
[]()		0.51	11.47	5.7	0.57	0.47	SERL (no_augmented)
[]()		0.51	11.15	5.45	0.57	0.47	Single-Run, No Pre-Explore
[]()		0.49	14.42	5.5	0.57	0.41	testliu
Self-Monitoring Navigation Agent via Auxiliary Progress Estimation	✓ Link	0.48	18.04	5.67	0.59	0.35	Self-Monitoring Navigation Agent (no beam search; Progress Inference)	2019-01-10
The Regretful Agent: Heuristic-Aided Navigation through Progress Estimation	✓ Link	0.48	13.69	5.69	0.56	0.4	The Regretful Agent (no beam search; greedy action selection)	2019-03-05
Transferable Representation Learning in Vision-and-Language Navigation		0.48	10.27	5.49	0.56	0.45	ALTR	2019-08-09
[]()		0.47	16.73	5.8	0.57	0.36	selfmoni0
[]()		0.47	16.03	5.56	0.57	0.35	DoubleAttn
[]()		0.47	14.07	5.42	0.55	0.4	SEA features + Speaker-Follower (single-run)
[]()		0.47	10.42	5.64	0.53	0.43	naive
Environment-agnostic Multitask Learning for Natural Language Grounded Navigation	✓ Link	0.45	13.35	6.03	0.56	0.4	Environment-Agnostic Multitask Learning	2020-03-01
[]()		0.45	10.47	6.01	0.53	0.4	speaker_follower_tesk2
[]()		0.43	11.97	6.12	0.5	0.38	Reinforced Cross-Modal Matching (single trajectory; NO beam search)
[]()		0.4	10.17	6.17	0.47	0.36	PTA
[]()		0.37	13.08	6.46	0.45	0.3	Khanh Nguyen
[]()		0.36	13.85	6.61	0.46	0.3	HAIL
[]()		0.36	10.83	6.67	0.44	0.33	rcm_test
[]()		0.36	10.44	6.95	0.43	0.31	ai like samurai with PNasNet5Large
[]()		0.35	9.81	6.55	0.45	0.31	Dynamic Convolutional Filters
[]()		0.34	8.32	6.89	0.41	0.32	AnonymousTeam
[]()		0.33	15.75	6.68	0.43	0.25	base_1
[]()		0.31	15.1	7.03	0.4	0.25	fuse_1
[]()		0.3	22.14	7.63	0.61	0.2	no
[]()		0.29	15.9	7.2	0.4	0.23	base_0
[]()		0.26	10.92	7.47	0.34	0.21	test_an
[]()		0.25	9.15	7.53	0.32	0.23	Look Before You Leap
[]()		0.24	9.48	8.56	0.32	0.22	X-Modal
[]()		0.2	8.26	7.99	0.26	0.18	zhangyong
[]()		0.2	8.13	7.85	0.27	0.18	Seq2Seq Baseline
[]()		0.18	9.56	8.82	0.24	0.16	zy123
[]()		0.14	9.91	9.8	0.19	0.12	zzzzzzzz55768
[]()		0.13	9.89	9.79	0.18	0.12	Random Agent
[]()		0.07	45.13	12.29	0.49	0.02	1111
[]()		0.0	0.0	9.93	0.0	0.0	15458

OpenCodePapers

vision-and-language-navigation-on-vln