StereoSet: Measuring stereotypical bias in pretrained language models

Benchmark Model Rank Results
bias-detection-on-stereoset-1GPT-2 (small)#1ICAT Score: 72.97
bias-detection-on-stereoset-1XLNet (large)#2ICAT Score: 72.03
bias-detection-on-stereoset-1GPT-2 (medium)#3ICAT Score: 71.73
bias-detection-on-stereoset-1BERT (base)#4ICAT Score: 71.21
bias-detection-on-stereoset-1GPT-2 (large)#5ICAT Score: 70.54
bias-detection-on-stereoset-1BERT (large)#6ICAT Score: 69.89
bias-detection-on-stereoset-1RoBERTa (base)#7ICAT Score: 67.50
bias-detection-on-stereoset-1XLNet (base)#9ICAT Score: 62.10