ARC (AI2 Reasoning Challenge)
Introduced by Clark et al. in Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
The AI2’s Reasoning Challenge (ARC) dataset is a multiple-choice question-answering dataset, containing questions from science exams from grade 3 to grade 9. The dataset is split in two partitions: Easy and Challenge, where the latter partition contains the more difficult questions that require reasoning. Most of the questions have 4 answer choices, with <1% of all the questions having either 3 or 5 answer choices. ARC includes a supporting KB of 14.3M unstructured text passages.
Source: Quick and (not so) Dirty: Unsupervised Selection of Justification Sentences for Multi-hop Question AnsweringBenchmarks
| Trend | Task | Dataset Variant | Best Model | Paper | Code |
|---|---|---|---|---|---|
|
ARC (Challenge)
|
GPT-4
|
|||
|
ARC (Easy)
|
ST-MoE-32B 269B
|
|||
|
ARC (AI2 Reasoning Challenge)
|
TESTED
|
Papers
| Paper | Code | Results | Date | Stars |
|---|
