OpenELM: An Efficient Language Model Family with Open-source Training and Inference Framework

1 code implementation • 22 Apr 2024 • Sachin Mehta, Mohammad Hossein Sekhavat, Qingqing Cao, Maxwell Horton, Yanzi Jin, Chenfan Sun, Iman Mirzadeh, Mahyar Najibi, Dmitry Belenko, Peter Zatloukal, Mohammad Rastegari

To this end, we release OpenELM, a state-of-the-art open language model.

Language Modelling

LLM in a flash: Efficient Large Language Model Inference with Limited Memory

no code implementations • 12 Dec 2023 • Keivan Alizadeh, Iman Mirzadeh, Dmitry Belenko, Karen Khatamifard, Minsik Cho, Carlo C Del Mundo, Mohammad Rastegari, Mehrdad Farajtabar

These methods collectively enable running models up to twice the size of the available DRAM, with a 4-5x and 20-25x increase in inference speed compared to naive loading approaches in CPU and GPU, respectively.

Language Modelling Large Language Model +1

ReLU Strikes Back: Exploiting Activation Sparsity in Large Language Models

1 code implementation • 6 Oct 2023 • Iman Mirzadeh, Keivan Alizadeh, Sachin Mehta, Carlo C Del Mundo, Oncel Tuzel, Golnoosh Samei, Mohammad Rastegari, Mehrdad Farajtabar

Large Language Models (LLMs) with billions of parameters have drastically transformed AI applications.

ActiLabel: A Combinatorial Transfer Learning Framework for Activity Recognition

no code implementations • 16 Mar 2020 • Parastoo Alinia, Iman Mirzadeh, Hassan Ghasemzadeh

Sensor-based human activity recognition has become a critical component of many emerging applications ranging from behavioral medicine to gaming.

Human Activity Recognition Transfer Learning