Asynchronous Methods for Deep Reinforcement Learning

Open paper

Benchmark	Model	Rank	Results
atari-games-on-atari-2600-alien	A3C LSTM hs	#35	Score: 945.3
atari-games-on-atari-2600-alien	A3C FF hs	#41	Score: 518.4
atari-games-on-atari-2600-alien	A3C FF (1 day) hs	#43	Score: 182.1
atari-games-on-atari-2600-amidar	A3C FF (1 day) hs	#27	Score: 283.9
atari-games-on-atari-2600-amidar	A3C FF hs	#28	Score: 263.9
atari-games-on-atari-2600-amidar	A3C LSTM hs	#37	Score: 173.0
atari-games-on-atari-2600-assault	A3C LSTM hs	#11	Score: 14497.9
atari-games-on-atari-2600-assault	A3C FF hs	#22	Score: 5474.9
atari-games-on-atari-2600-assault	A3C FF (1 day) hs	#28	Score: 3746.1
atari-games-on-atari-2600-asterix	A3C FF hs	#22	Score: 22140.5
atari-games-on-atari-2600-asterix	A3C LSTM hs	#28	Score: 17244.5
atari-games-on-atari-2600-asterix	A3C FF (1 day) hs	#33	Score: 6723
atari-games-on-atari-2600-asteroids	A3C LSTM hs	#11	Score: 5093.1
atari-games-on-atari-2600-asteroids	A3C FF hs	#14	Score: 4474.5
atari-games-on-atari-2600-asteroids	A3C FF (1 day) hs	#16	Score: 3009.4
atari-games-on-atari-2600-atlantis	A3C FF hs	#17	Score: 911091.0
atari-games-on-atari-2600-atlantis	A3C LSTM hs	#18	Score: 875822.0
atari-games-on-atari-2600-atlantis	A3C FF (1 day) hs	#21	Score: 772392.0
atari-games-on-atari-2600-bank-heist	A3C FF hs	#23	Score: 970.1
atari-games-on-atari-2600-bank-heist	A3C FF (1 day) hs	#24	Score: 946.0
atari-games-on-atari-2600-bank-heist	A3C LSTM hs	#25	Score: 932.8
atari-games-on-atari-2600-battle-zone	A3C LSTM hs	#34	Score: 20760.0
atari-games-on-atari-2600-battle-zone	A3C FF hs	#39	Score: 12950.0
atari-games-on-atari-2600-battle-zone	A3C FF (1 day) hs	#40	Score: 11340.0
atari-games-on-atari-2600-beam-rider	A3C LSTM hs	#13	Score: 24622.2
atari-games-on-atari-2600-beam-rider	A3C FF hs	#17	Score: 22707.9
atari-games-on-atari-2600-beam-rider	A3C FF (1 day) hs	#25	Score: 13235.9
atari-games-on-atari-2600-berzerk	A3C FF (1 day) hs	#17	Score: 1433.4
atari-games-on-atari-2600-berzerk	A3C LSTM hs	#26	Score: 862.2
atari-games-on-atari-2600-berzerk	A3C FF hs	#27	Score: 817.9
atari-games-on-atari-2600-bowling	A3C LSTM hs	#32	Score: 41.8
atari-games-on-atari-2600-bowling	A3C FF (1 day) hs	#34	Score: 36.2
atari-games-on-atari-2600-bowling	A3C FF hs	#35	Score: 35.1
atari-games-on-atari-2600-boxing	A3C FF hs	#33	Score: 59.8
atari-games-on-atari-2600-boxing	A3C LSTM hs	#37	Score: 37.3
atari-games-on-atari-2600-boxing	A3C FF (1 day) hs	#38	Score: 33.7
atari-games-on-atari-2600-breakout	A3C LSTM hs	#7	Score: 766.8
atari-games-on-atari-2600-breakout	A3C FF hs	#12	Score: 681.9
atari-games-on-atari-2600-breakout	A3C FF (1 day) hs	#15	Score: 551.6
atari-games-on-atari-2600-centipede	A3C FF hs	#34	Score: 3755.8
atari-games-on-atari-2600-centipede	A3C FF (1 day) hs	#37	Score: 3306.5
atari-games-on-atari-2600-centipede	A3C LSTM hs	#38	Score: 1997.0
atari-games-on-atari-2600-chopper-command	A3C LSTM hs	#16	Score: 10150.0
atari-games-on-atari-2600-chopper-command	A3C FF hs	#19	Score: 7021.0
atari-games-on-atari-2600-chopper-command	A3C FF (1 day) hs	#29	Score: 4669.0
atari-games-on-atari-2600-crazy-climber	A3C LSTM hs	#16	Score: 138518.0
atari-games-on-atari-2600-crazy-climber	A3C FF hs	#30	Score: 112646.0
atari-games-on-atari-2600-crazy-climber	A3C FF (1 day) hs	#32	Score: 101624.0
atari-games-on-atari-2600-demon-attack	A3C LSTM hs	#11	Score: 115201.9
atari-games-on-atari-2600-demon-attack	A3C FF hs	#12	Score: 113308.4
atari-games-on-atari-2600-demon-attack	A3C FF (1 day) hs	#14	Score: 84997.5
atari-games-on-atari-2600-double-dunk	A3C FF (1 day) hs	#19	Score: 0.1
atari-games-on-atari-2600-double-dunk	A3C LSTM hs	#20	Score: 0.1
atari-games-on-atari-2600-double-dunk	A3C FF hs	#22	Score: -0.1
atari-games-on-atari-2600-enduro	A3C FF (1 day) hs	#40	Score: -82.2
atari-games-on-atari-2600-enduro	A3C FF hs	#41	Score: -82.5
atari-games-on-atari-2600-enduro	A3C LSTM hs	#42	Score: -82.5
atari-games-on-atari-2600-fishing-derby	A3C LSTM hs	#22	Score: 22.6
atari-games-on-atari-2600-fishing-derby	A3C FF hs	#24	Score: 18.8
atari-games-on-atari-2600-fishing-derby	A3C FF (1 day) hs	#27	Score: 13.6
atari-games-on-atari-2600-freeway	A3C FF (1 day) hs	#47	Score: 0.1
atari-games-on-atari-2600-freeway	A3C FF hs	#48	Score: 0.1
atari-games-on-atari-2600-freeway	A3C LSTM hs	#49	Score: 0.1
atari-games-on-atari-2600-frostbite	A3C LSTM hs	#43	Score: 197.6
atari-games-on-atari-2600-frostbite	A3C FF hs	#44	Score: 190.5
atari-games-on-atari-2600-frostbite	A3C FF (1 day) hs	#45	Score: 180.1
atari-games-on-atari-2600-gopher	A3C LSTM hs	#22	Score: 17106.8
atari-games-on-atari-2600-gopher	A3C FF hs	#28	Score: 10022.8
atari-games-on-atari-2600-gopher	A3C FF (1 day) hs	#31	Score: 8442.8
atari-games-on-atari-2600-gravitar	A3C LSTM hs	#36	Score: 320.0
atari-games-on-atari-2600-gravitar	A3C FF hs	#38	Score: 303.5
atari-games-on-atari-2600-gravitar	A3C FF (1 day) hs	#43	Score: 269.5
atari-games-on-atari-2600-hero	A3C FF hs	#8	Score: 32464.1
atari-games-on-atari-2600-hero	A3C LSTM hs	#12	Score: 28889.5
atari-games-on-atari-2600-hero	A3C FF (1 day) hs	#13	Score: 28765.8
atari-games-on-atari-2600-ice-hockey	A3C LSTM hs	#26	Score: -1.7
atari-games-on-atari-2600-ice-hockey	A3C FF hs	#32	Score: -2.8
atari-games-on-atari-2600-ice-hockey	A3C FF (1 day) hs	#37	Score: -4.7
atari-games-on-atari-2600-james-bond	A3C LSTM hs	#27	Score: 613.0
atari-games-on-atari-2600-james-bond	A3C FF hs	#32	Score: 541.0
atari-games-on-atari-2600-james-bond	A3C FF (1 day) hs	#35	Score: 351.5
atari-games-on-atari-2600-kangaroo	A3C LSTM hs	#39	Score: 125.0
atari-games-on-atari-2600-kangaroo	A3C FF (1 day) hs	#40	Score: 106.0
atari-games-on-atari-2600-kangaroo	A3C FF hs	#41	Score: 94.0
atari-games-on-atari-2600-krull	A3C FF (1 day) hs	#25	Score: 8066.6
atari-games-on-atari-2600-krull	A3C LSTM hs	#34	Score: 5911.4
atari-games-on-atari-2600-krull	A3C FF hs	#35	Score: 5560.0
atari-games-on-atari-2600-kung-fu-master	A3C LSTM hs	#18	Score: 40835.0
atari-games-on-atari-2600-kung-fu-master	A3C FF hs	#30	Score: 28819.0
atari-games-on-atari-2600-kung-fu-master	A3C FF (1 day) hs	#38	Score: 3046.0
atari-games-on-atari-2600-montezumas-revenge	A3C FF hs	#25	Score: 67
atari-games-on-atari-2600-montezumas-revenge	A3C FF (1 day) hs	#27	Score: 53
atari-games-on-atari-2600-montezumas-revenge	A3C LSTM hs	#31	Score: 41
atari-games-on-atari-2600-ms-pacman	A3C LSTM hs	#38	Score: 850.7
atari-games-on-atari-2600-ms-pacman	A3C FF hs	#40	Score: 653.7
atari-games-on-atari-2600-ms-pacman	A3C FF (1 day) hs	#41	Score: 594.4
atari-games-on-atari-2600-name-this-game	A3C LSTM hs	#19	Score: 12093.7
atari-games-on-atari-2600-name-this-game	A3C FF hs	#26	Score: 10476.1
atari-games-on-atari-2600-name-this-game	A3C FF (1 day) hs	#34	Score: 5614.0
atari-games-on-atari-2600-pong	A3C FF (1 day) hs	#39	Score: 11.4
atari-games-on-atari-2600-pong	A3C LSTM hs	#40	Score: 10.7
atari-games-on-atari-2600-pong	A3C FF hs	#42	Score: 5.6
atari-games-on-atari-2600-private-eye	A3C LSTM hs	#23	Score: 421.1
atari-games-on-atari-2600-private-eye	A3C FF hs	#29	Score: 206.9
atari-games-on-atari-2600-private-eye	A3C FF (1 day) hs	#34	Score: 194.4
atari-games-on-atari-2600-qbert	A3C LSTM hs	#15	Score: 21307.5
atari-games-on-atari-2600-qbert	A3C FF hs	#22	Score: 15148.8
atari-games-on-atari-2600-qbert	A3C FF (1 day) hs	#31	Score: 13752.3
atari-games-on-atari-2600-river-raid	A3C FF hs	#22	Score: 12201.8
atari-games-on-atari-2600-river-raid	A3C FF (1 day) hs	#26	Score: 10001.2
atari-games-on-atari-2600-river-raid	A3C LSTM hs	#30	Score: 6591.9
atari-games-on-atari-2600-road-runner	A3C LSTM hs	#8	Score: 73949.0
atari-games-on-atari-2600-road-runner	A3C FF hs	#31	Score: 34216.0
atari-games-on-atari-2600-road-runner	A3C FF (1 day) hs	#32	Score: 31769.0
atari-games-on-atari-2600-robotank	A3C FF hs	#27	Score: 32.8
atari-games-on-atari-2600-robotank	A3C LSTM hs	#36	Score: 2.6
atari-games-on-atari-2600-robotank	A3C FF (1 day) hs	#37	Score: 2.3
atari-games-on-atari-2600-seaquest	A3C FF hs	#32	Score: 2355.4
atari-games-on-atari-2600-seaquest	A3C FF (1 day) hs	#33	Score: 2300.2
atari-games-on-atari-2600-seaquest	A3C LSTM hs	#41	Score: 1326.1
atari-games-on-atari-2600-space-invaders	A3C LSTM hs	#8	Score: 23846.0
atari-games-on-atari-2600-space-invaders	A3C FF hs	#11	Score: 15730.5
atari-games-on-atari-2600-space-invaders	A3C FF (1 day) hs	#31	Score: 2214.7
atari-games-on-atari-2600-star-gunner	A3C LSTM hs	#6	Score: 164766.0
atari-games-on-atari-2600-star-gunner	A3C FF hs	#8	Score: 138218.0
atari-games-on-atari-2600-star-gunner	A3C FF (1 day) hs	#19	Score: 64393.0
atari-games-on-atari-2600-tennis	A3C FF hs	#30	Score: -6.3
atari-games-on-atari-2600-tennis	A3C LSTM hs	#31	Score: -6.4
atari-games-on-atari-2600-tennis	A3C FF (1 day) hs	#34	Score: -10.2
atari-games-on-atari-2600-time-pilot	A3C LSTM hs	#9	Score: 27202.0
atari-games-on-atari-2600-time-pilot	A3C FF hs	#14	Score: 12679.0
atari-games-on-atari-2600-time-pilot	A3C FF (1 day) hs	#32	Score: 5825.0
atari-games-on-atari-2600-tutankham	A3C FF hs	#26	Score: 156.3
atari-games-on-atari-2600-tutankham	A3C LSTM hs	#27	Score: 144.2
atari-games-on-atari-2600-tutankham	A3C FF (1 day) hs	#38	Score: 26.1
atari-games-on-atari-2600-up-and-down	A3C LSTM hs	#11	Score: 105728.7
atari-games-on-atari-2600-up-and-down	A3C FF hs	#13	Score: 74705.7
atari-games-on-atari-2600-up-and-down	A3C FF (1 day) hs	#18	Score: 54525.4
atari-games-on-atari-2600-venture	A3C LSTM hs	#38	Score: 25.0
atari-games-on-atari-2600-venture	A3C FF hs	#39	Score: 23.0
atari-games-on-atari-2600-venture	A3C FF (1 day) hs	#41	Score: 19.0
atari-games-on-atari-2600-video-pinball	A3C LSTM hs	#16	Score: 470310.5
atari-games-on-atari-2600-video-pinball	A3C FF hs	#20	Score: 331628.1
atari-games-on-atari-2600-video-pinball	A3C FF (1 day) hs	#26	Score: 185852.6
atari-games-on-atari-2600-wizard-of-wor	A3C LSTM hs	#12	Score: 18082.0
atari-games-on-atari-2600-wizard-of-wor	A3C FF hs	#13	Score: 17244.0
atari-games-on-atari-2600-wizard-of-wor	A3C FF (1 day) hs	#29	Score: 5278.0
atari-games-on-atari-2600-zaxxon	A3C FF hs	#8	Score: 24622.0
atari-games-on-atari-2600-zaxxon	A3C LSTM hs	#9	Score: 23519.0
atari-games-on-atari-2600-zaxxon	A3C FF (1 day) hs	#37	Score: 2659.0