GPipe: Efficient Training of Giant Neural Networks using Pipeline Parallelism

Benchmark Model Rank Results
fine-grained-image-classification-on-birdsnapGPIPE#4Accuracy: 83.6%
fine-grained-image-classification-on-stanfordGPipe#26Accuracy: 94.6%
image-classification-on-cifar-10GPIPE + transfer learning#21Percentage correct: 99
image-classification-on-cifar-100GPIPE#20Percentage correct: 91.3
image-classification-on-imagenetGPIPE#308Top 1 Accuracy: 84.4%