LexGLUE: A Benchmark Dataset for Legal Language Understanding in English

Benchmark Model Rank Results
natural-language-understanding-on-lexglueBERT#1ECtHR Task A: 71.4 / 64.0ECtHR Task B: 87.6 / 77.8SCOTUS: 70.5 / 60.9
natural-language-understanding-on-lexglueLegal-BERT#2ECtHR Task A: 71.2 / 64.6ECtHR Task B: 88.0 / 77.2SCOTUS: 76.2 / 65.8
natural-language-understanding-on-lexglueCaseLaw-BERT#3ECtHR Task A: 71.2 / 64.2ECtHR Task B: 88.0 / 77.5SCOTUS: 76.4 / 66.2
natural-language-understanding-on-lexglueBigBird#4ECtHR Task A: 70.5 / 63.8ECtHR Task B: 88.1 / 76.6SCOTUS: 71.7 / 61.4
natural-language-understanding-on-lexglueLongformer#5ECtHR Task A: 69.6 / 62.4ECtHR Task B: 88.0 / 77.8SCOTUS: 72.2 / 62.5
natural-language-understanding-on-lexglueRoBERTa#6ECtHR Task A: 69.5 / 60.7ECtHR Task B: 87.2 / 77.3SCOTUS: 70.8 / 61.2
natural-language-understanding-on-lexglueDeBERTa#7ECtHR Task A: 69.1 / 61.2ECtHR Task B: 87.4 / 77.3SCOTUS: 70.0 / 60.0