FAdam: Adam is a natural gradient optimizer using diagonal empirical Fisher information

Benchmark Model Rank Results
speech-recognition-on-librispeech-test-cleanFAdam#21Word Error Rate (WER): 1.34
speech-recognition-on-librispeech-test-otherFAdam#8Word Error Rate (WER): 2.49