GLUE (General Language Understanding Evaluation benchmark)

Introduced by Wang et al. in GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding

General Language Understanding Evaluation (GLUE) benchmark is a collection of nine natural language understanding tasks, including single-sentence tasks CoLA and SST-2, similarity and paraphrasing tasks MRPC, STS-B and QQP, and natural language inference tasks MNLI, QNLI, RTE and WNLI.

Source: Align, Mask and Select: A Simple Method for Incorporating Commonsense Knowledge into Language Representation Models

Homepage

Benchmarks

Add a new result Link an existing benchmark

Task	Dataset Variant	Best Model
Natural Language Inference	RTE	Vega v2 6B
Text Classification	GLUE	distilbert-base-uncased-finetuned-sst-2-english
Semantic Textual Similarity	MRPC	MT-DNN-SMART
Linguistic Acceptability	CoLA	En-BERT + TDA + PCA
Natural Language Inference	QNLI	ALBERT
Natural Language Inference	WNLI	Turing NLR v5 XXL 5.4B
Text Classification	GLUE COLA	TRANS-BLSTM
Text Classification	GLUE MRPC	TRANS-BLSTM
Text Classification	GLUE SST2	TRANS-BLSTM
Text Classification	GLUE RTE	TRANS-BLSTM
Text Classification	GLUE WNLI	fnet-base-finetuned-wnli
Text Classification	GLUE STSB	TRANS-BLSTM
Text Classification	GLUE QQP	bert-base-cased-finetuned-qqp
Text Classification	GLUE QNLI	bert-base-cased-finetuned-qnli
Text Classification	GLUE MNLI	bert-base-cased-finetuned-mnli
Natural Language Understanding	GLUE	MT-DNN-SMART
Text Classification	SST-2	DeBERTa
Model Compression	QNLI	MobileBERT + 2bit-1dim model compression using DKM
Few-Shot Learning	GLUE QQP	DART
Natural Language Inference	MRPC	DeBERTaV3large
Natural Language Inference	GLUE	Jiva/xlm-roberta-large-it-mnli
QQP	qqp	BiLSTM
Stochastic Optimization	CoLA	Bert
Semantic Textual Similarity within Bi-Encoder	MRPC	AugSBERT-BM25
Few-Shot Learning	MRPC	DART