Search Results for author:
Found 57 papers, 27 papers with code
CatLIP: CLIP-level Visual Recognition Accuracy with 2.7x Faster Pre-training on Web-scale Image-Text Data
Contrastive learning has emerged as a transformative method for learning effective visual representations through the alignment of image and text embeddings.
OpenELM: An Efficient Language Model Family with Open-source Training and Inference Framework
To this end, we release OpenELM, a state-of-the-art open language model.
Superposition Prompting: Improving and Accelerating Retrieval-Augmented Generation
Despite the successes of large language models (LLMs), they exhibit significant drawbacks, particularly when processing long contexts.
Speculative Streaming: Fast LLM Inference without Auxiliary Models
Speculative decoding is a prominent technique to speed up the inference of a large target language model based on predictions of an auxiliary draft model.
Weight subcloning: direct initialization of transformers using larger pretrained ones
The usual practice of transfer learning overcomes this challenge by initializing the model with weights of a pretrained model of the same size and specification to increase the convergence and training speed.
LLM in a flash: Efficient Large Language Model Inference with Limited Memory
These methods collectively enable running models up to twice the size of the available DRAM, with a 4-5x and 20-25x increase in inference speed compared to naive loading approaches in CPU and GPU, respectively.
Ranked #63 on
Sentence Completion
on HellaSwag
Knowledge Transfer from Vision Foundation Models for Efficient Training of Small Task-specific Models
Motivated by this, we ask the following important question, "How can we leverage the knowledge from a large VFM to train a small task-specific model for a new target task with limited labeled training data?
SAM-CLIP: Merging Vision Foundation Models towards Semantic and Spatial Understanding
By applying our method to SAM and CLIP, we obtain SAM-CLIP: a unified model that combines the capabilities of SAM and CLIP into a single vision transformer.
CLIP meets Model Zoo Experts: Pseudo-Supervision for Visual Enhancement
While CLIP is scalable, promptable, and robust to distribution shifts on image classification tasks, it lacks object localization capabilities.
ReLU Strikes Back: Exploiting Activation Sparsity in Large Language Models
Large Language Models (LLMs) with billions of parameters have drastically transformed AI applications.
Diffusion Models as Masked Audio-Video Learners
Over the past several years, the synchronization between audio and visual signals has been leveraged to learn richer audio-visual representations.
Do Compressed LLMs Forget Knowledge? An Experimental Study with Practical Implications
We start by proposing two conjectures on the nature of the damage: one is certain knowledge being forgotten (or erased) after LLM compression, hence necessitating the compressed model to (re)learn from data with additional parameters; the other presumes that knowledge is internally displaced and hence one requires merely "inference re-direction" with input-side augmentation such as prompting, to recover the knowledge-related performance.
On the Efficacy of Multi-scale Data Samplers for Vision Applications
Multi-scale resolution training has seen an increased adoption across multiple vision tasks, including classification and detection.
eDKM: An Efficient and Accurate Train-time Weight Clustering for Large Language Models
Since Large Language Models or LLMs have demonstrated high-quality performance on many complex language tasks, there is a great interest in bringing these LLMs to mobile devices for faster responses and better privacy protection.
Bytes Are All You Need: Transformers Operating Directly On File Bytes
Our model, \emph{ByteFormer}, achieves an ImageNet Top-1 classification accuracy of $77. 33\%$ when training and testing directly on TIFF file bytes using a transformer backbone with configuration similar to DeiT-Ti ($72. 2\%$ accuracy when operating on RGB images).
Reinforce Data, Multiply Impact: Improved Model Accuracy and Robustness with Dataset Reinforcement
Models pretrained on ImageNet+ and fine-tuned on CIFAR-100+, Flowers-102+, and Food-101+, reach up to 3. 4% improved accuracy.
RangeAugment: Efficient Online Augmentation with Range Learning
To answer the open question on the importance of magnitude ranges for each augmentation operation, we introduce RangeAugment that allows us to efficiently learn the range of magnitudes for individual as well as composite augmentation operations.
SPIN: An Empirical Evaluation on Sharing Parameters of Isotropic Networks
In this paper, we perform an empirical evaluation on methods for sharing parameters in isotropic networks (SPIN).
Separable Self-attention for Mobile Vision Transformers
The improved model, MobileViTv2, is state-of-the-art on several mobile vision tasks, including ImageNet object classification and MS-COCO object detection.
CVNets: High Performance Library for Computer Vision
We introduce CVNets, a high-performance open-source library for training deep neural networks for visual recognition tasks, including classification, detection, and segmentation.
Token Pooling in Vision Transformers
Token Pooling is a simple and effective operator that can benefit many architectures.
LCS: Learning Compressible Subspaces for Adaptive Network Compression at Inference Time
Our models require no retraining, thus our subspace of models can be deployed entirely on-device to allow adaptive network compression at inference time.
MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformer
Light-weight convolutional neural networks (CNNs) are the de-facto for mobile vision tasks.
Ranked #768 on
Image Classification
on ImageNet
DKM: Differentiable K-Means Clustering Layer for Neural Network Compression
For MobileNet-v1, which is a challenging DNN to compress, DKM delivers 63. 9% top-1 ImageNet1k accuracy with 0. 72 MB model size (22. 4x model compression factor).
Learning Neural Network Subspaces
Recent observations have advanced our understanding of the neural network optimization landscape, revealing the existence of (1) paths of high accuracy containing diverse solutions and (2) wider minima offering improved performance.
Layer-Wise Data-Free CNN Compression
We also show how to precondition the network to improve the accuracy of our layer-wise compression method.
Supermasks in Superposition
We present the Supermasks in Superposition (SupSup) model, capable of sequentially learning thousands of tasks without catastrophic forgetting.
What's Hidden in a Randomly Weighted Neural Network?
Training a neural network is synonymous with learning the values of the weights.
Ranked #915 on
Image Classification
on ImageNet
DeFINE: DEep Factorized INput Token Embeddings for Neural Sequence Modeling
For sequence models with large vocabularies, a majority of network parameters lie in the input and output layers.
Assisted Excitation of Activations: A Learning Technique to Improve Object Detectors
We present a simple and effective learning technique that significantly improves mAP of YOLO object detectors without compromising their speed.
DiCENet: Dimension-wise Convolutions for Efficient Networks
When DiCE units are stacked to build the DiCENet model, we observe significant improvements over state-of-the-art models across various computer vision tasks including image classification, object detection, and semantic segmentation.
Ranked #21 on
Semantic Segmentation
on PASCAL VOC 2012 val
Butterfly Transform: An Efficient FFT Based Neural Architecture Design
By replacing pointwise convolutions with BFT, we reduce the computational complexity of these layers from O(n^2) to O(n\log n) with respect to the number of channels.
Discovering Neural Wirings
In this work we propose a method for discovering neural wirings.
OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledge
In this paper, we address the task of knowledge-based visual question answering and provide a benchmark, called OK-VQA, where the image content is not sufficient to answer the questions, encouraging methods that rely on external knowledge resources.
Two Body Problem: Collaborative Visual Task Completion
Collaboration is a necessary skill to perform tasks that are beyond one agent's capabilities.
ELASTIC: Improving CNNs with Dynamic Scaling Policies
We formulate the scaling policy as a non-linear function inside the network's structure that (a) is learned from data, (b) is instance specific, (c) does not add extra computation, and (d) can be applied on any network architecture.
Learning to Learn How to Learn: Self-Adaptive Visual Navigation Using Meta-Learning
In this paper we study the problem of learning to learn at both training and test time in the context of visual navigation.
Ranked #2 on
Visual Navigation
on AI2-THOR
ESPNetv2: A Light-weight, Power Efficient, and General Purpose Convolutional Neural Network
Compared to YOLOv2 on the MS-COCO object detection, ESPNetv2 delivers 4. 4% higher accuracy with 6x fewer FLOPs.
Ranked #41 on
Semantic Segmentation
on PASCAL VOC 2012 test
Pyramidal Recurrent Unit for Language Modeling
We introduce the Pyramidal Recurrent Unit (PRU), which enables learning representations in high dimensional space with more generalization power and fewer parameters.
Label Refinery: Improving ImageNet Classification through Label Progression
Among the three main components (data, labels, and models) of any supervised learning system, data and models have been the main subjects of active research.
ESPNet: Efficient Spatial Pyramid of Dilated Convolutions for Semantic Segmentation
We introduce a fast and efficient convolutional neural network, ESPNet, for semantic segmentation of high resolution images under resource constraints.
Ranked #48 on
Semantic Segmentation
on PASCAL VOC 2012 test
IQA: Visual Question Answering in Interactive Environments
Our experiments show that our proposed model outperforms popular single controller based methods on IQUAD V1.
LCNN: Lookup-based Convolutional Neural Network
We introduce LCNN, a lookup-based convolutional neural network that encodes convolutions by few lookups to a dictionary that is trained to cover the space of weights in CNNs.
CNN-aware Binary Map for General Semantic Segmentation
To the best of our knowledge our method is the first attempt on general semantic image segmentation using CNN.
Newtonian Scene Understanding: Unfolding the Dynamics of Objects in Static Images
Direct and explicit estimation of the forces and the motion of objects from a single image is extremely challenging.
"What happens if..." Learning to Predict the Effect of Forces in Images
To build a dataset of forces in scenes, we reconstructed all images in SUN RGB-D dataset in a physics simulator to estimate the physical movements of objects caused by external forces applied to them.
XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks
We propose two efficient approximations to standard convolutional neural networks: Binary-Weight-Networks and XNOR-Networks.
Ranked #10 on
Classification with Binary Neural Network
on ImageNet
Action Recognition with Image Based CNN Features
Action recognition tasks usually relies on complex handcrafted structures as features to represent the human action model.
Newtonian Image Understanding: Unfolding the Dynamics of Objects in Static Images
Direct and explicit estimation of the forces and the motion of objects from a single image is extremely challenging.
On Large-Scale Retrieval: Binary or n-ary Coding?
To make large-scale search feasible, Distance Estimation and Subset Indexing are the main approaches.
Discriminative and Consistent Similarities in Instance-Level Multiple Instance Learning
In this paper we present a bottom-up method to instance-level Multiple Instance Learning (MIL) that learns to discover positive instances with globally constrained reasoning about local pairwise similarities.
Computationally Bounded Retrieval
We demonstrate this technique on large retrieval databases, specifically ImageNET, GIST1M and SUN-attribute for the task of nearest neighbor retrieval, and show that our method achieves a speed-up of up to a factor of 100 over state-of-the-art methods, while having on-par and in some cases even better accuracy.
Class Consistent Multi-Modal Fusion With Binary Features
Many existing recognition algorithms combine different modalities based on training accuracy but do not consider the possibility of noise at test time.
Comparing apples to apples in the evaluation of binary coding methods
We discuss methodological issues related to the evaluation of unsupervised binary code construction methods for nearest neighbor search.
Adding Unlabeled Samples to Categories by Learned Attributes
We propose a method to expand the visual coverage of training sets that consist of a small number of labeled examples using learned attributes.
Multi-attribute Queries: To Merge or Not to Merge?
We exploit a discriminative binary space to compute these geometric quantities efficiently.
