Transformers

Transformers are a type of neural network architecture that have several properties that make them effective for modeling data with long-range dependencies. They generally feature a combination of multi-headed attention mechanisms, residual connections, layer normalization, feedforward connections, and positional embeddings.

Subcategories

Method Year Papers
2017 9390
2023 5761
2018 5054
2020 1338
2019 747
2018 656
2019 553
2019 543
2019 448
2019 156
2019 153
2020 149
2019 126
2020 118
2022 111
2020 107
2020 80
2020 78
2020 72
2020 65
2019 61
2020 59
2019 55
2019 48
2020 48
2019 43
2019 35
2019 30
2020 29
2000 28
2019 21
2021 18
2018 17
2021 16
2020 15
2020 14
2020 13
2019 11
2021 10
2020 9
2021 7
2021 6
2020 6
2022 6
2020 5
2021 5
2021 5
2020 5
2021 3
2019 3
2021 3
2019 3
2021 3
2019 3
2020 3
2020 3
2020 3
2020 2
2019 2
2021 2
2020 2
2021 2
2020 2
2021 2
2022 2
2021 2
2020 1
2020 1
2021 1
2021 1
2019 1
2020 1
2020 1
2021 1
2020 1
2020 1
2020 1
2020 1
2018 1
2020 1
2020 1
2021 1
2021 1
2020 1
2019 1
2021 1
2020 1
2019 1