GPipe: Efficient Training of Giant Neural Networks using Pipeline Parallelism

Benchmark Model Rank Results
fine-grained-image-classification-on-birdsnapGPIPE#4Accuracy: 83.6%
fine-grained-image-classification-on-stanford-carsGPipe#26Accuracy: 94.6%
image-classification-on-cifar-10GPIPE + transfer learning#26Percentage correct: 99
image-classification-on-cifar-100GPIPE#29Percentage correct: 91.3
image-classification-on-imagenetGPIPE#342Top 1 Accuracy: 84.4%