Search Results for author:
Found 39 papers, 24 papers with code
CatLIP: CLIP-level Visual Recognition Accuracy with 2.7x Faster Pre-training on Web-scale Image-Text Data
Contrastive learning has emerged as a transformative method for learning effective visual representations through the alignment of image and text embeddings.
OpenELM: An Efficient Language Model Family with Open-source Training and Inference Framework
To this end, we release OpenELM, a state-of-the-art open language model.
Weight subcloning: direct initialization of transformers using larger pretrained ones
The usual practice of transfer learning overcomes this challenge by initializing the model with weights of a pretrained model of the same size and specification to increase the convergence and training speed.
Knowledge Transfer from Vision Foundation Models for Efficient Training of Small Task-specific Models
Motivated by this, we ask the following important question, "How can we leverage the knowledge from a large VFM to train a small task-specific model for a new target task with limited labeled training data?
TiC-CLIP: Continual Training of CLIP Models
We introduce the first set of web-scale Time-Continual (TiC) benchmarks for training vision-language models: TiC-DataComp, TiC-YFCC, and TiC-Redcaps.
SAM-CLIP: Merging Vision Foundation Models towards Semantic and Spatial Understanding
By applying our method to SAM and CLIP, we obtain SAM-CLIP: a unified model that combines the capabilities of SAM and CLIP into a single vision transformer.
CLIP meets Model Zoo Experts: Pseudo-Supervision for Visual Enhancement
While CLIP is scalable, promptable, and robust to distribution shifts on image classification tasks, it lacks object localization capabilities.
SHARCS: Efficient Transformers through Routing with Dynamic Width Sub-networks
We introduce SHARCS for adaptive inference that takes into account the hardness of input samples.
ReLU Strikes Back: Exploiting Activation Sparsity in Large Language Models
Large Language Models (LLMs) with billions of parameters have drastically transformed AI applications.
Diffusion Models as Masked Audio-Video Learners
Over the past several years, the synchronization between audio and visual signals has been leveraged to learn richer audio-visual representations.
On the Efficacy of Multi-scale Data Samplers for Vision Applications
Multi-scale resolution training has seen an increased adoption across multiple vision tasks, including classification and detection.
Bytes Are All You Need: Transformers Operating Directly On File Bytes
Our model, \emph{ByteFormer}, achieves an ImageNet Top-1 classification accuracy of $77. 33\%$ when training and testing directly on TIFF file bytes using a transformer backbone with configuration similar to DeiT-Ti ($72. 2\%$ accuracy when operating on RGB images).
Reinforce Data, Multiply Impact: Improved Model Accuracy and Robustness with Dataset Reinforcement
Models pretrained on ImageNet+ and fine-tuned on CIFAR-100+, Flowers-102+, and Food-101+, reach up to 3. 4% improved accuracy.
APE: An Open and Shared Annotated Dataset for Learning Urban Pedestrian Path Networks
The process features the use of a multi-input segmentation network trained on our dataset to predict important classes in the pedestrian environment and then generate a connected pedestrian path network.
OASIS: Automated Assessment of Urban Pedestrian Paths at Scale
The inspection of the Public Right of Way (PROW) for accessibility barriers is necessary for monitoring and maintaining the built environment for communities' walkability, rollability, safety, active transportation, and sustainability.
RangeAugment: Efficient Online Augmentation with Range Learning
To answer the open question on the importance of magnitude ranges for each augmentation operation, we introduce RangeAugment that allows us to efficiently learn the range of magnitudes for individual as well as composite augmentation operations.
SPIN: An Empirical Evaluation on Sharing Parameters of Isotropic Networks
In this paper, we perform an empirical evaluation on methods for sharing parameters in isotropic networks (SPIN).
Separable Self-attention for Mobile Vision Transformers
The improved model, MobileViTv2, is state-of-the-art on several mobile vision tasks, including ImageNet object classification and MS-COCO object detection.
CVNets: High Performance Library for Computer Vision
We introduce CVNets, a high-performance open-source library for training deep neural networks for visual recognition tasks, including classification, detection, and segmentation.
Iconary: A Pictionary-Based Game for Testing Multimodal Communication with Drawings and Text
We investigate these challenges in the context of Iconary, a collaborative game of drawing and guessing based on Pictionary, that poses a novel challenge for the research community.
MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformer
Light-weight convolutional neural networks (CNNs) are the de-facto for mobile vision tasks.
Ranked #768 on
Image Classification
on ImageNet
Rethinking Semantic Segmentation Evaluation for Explainability and Model Selection
Semantic segmentation is a prerequisite for this task since it maps contiguous regions of the same class as single entities.
Classifying Breast Histopathology Images with a Ductal Instance-Oriented Pipeline
In this study, we propose the Ductal Instance-Oriented Pipeline (DIOP) that contains a duct-level instance segmentation model, a tissue-level semantic segmentation model, and three-levels of features for diagnostic classification.
MedICaT: A Dataset of Medical Images, Captions, and Textual References
To address challenges in figure retrieval and figure-to-text alignment, we introduce MedICaT, a dataset of medical images in context.
DeLighT: Deep and Light-weight Transformer
We introduce a deep and light-weight transformer, DeLighT, that delivers similar or better performance than standard transformer-based models with significantly fewer parameters.
Ranked #1 on
Machine Translation
on WMT2016 English-French
HATNet: An End-to-End Holistic Attention Network for Diagnosis of Breast Biopsy Images
HATNet extends the bag-of-words approach and uses self-attention to encode global information, allowing it to learn representations from clinically relevant tissue structures without any explicit supervision.
DeFINE: DEep Factorized INput Token Embeddings for Neural Sequence Modeling
For sequence models with large vocabularies, a majority of network parameters lie in the input and output layers.
DiCENet: Dimension-wise Convolutions for Efficient Networks
When DiCE units are stacked to build the DiCENet model, we observe significant improvements over state-of-the-art models across various computer vision tasks including image classification, object detection, and semantic segmentation.
Ranked #21 on
Semantic Segmentation
on PASCAL VOC 2012 val
A Facial Affect Analysis System for Autism Spectrum Disorder
In this paper, we introduce an end-to-end machine learning-based system for classifying autism spectrum disorder (ASD) using facial attributes such as expressions, action units, arousal, and valence.
ESPNetv2: A Light-weight, Power Efficient, and General Purpose Convolutional Neural Network
Compared to YOLOv2 on the MS-COCO object detection, ESPNetv2 delivers 4. 4% higher accuracy with 6x fewer FLOPs.
Ranked #41 on
Semantic Segmentation
on PASCAL VOC 2012 test
Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challenge
This study assesses the state-of-the-art machine learning (ML) methods used for brain tumor image analysis in mpMRI scans, during the last seven instances of the International Brain Tumor Segmentation (BraTS) challenge, i. e., 2012-2018.
Pyramidal Recurrent Unit for Language Modeling
We introduce the Pyramidal Recurrent Unit (PRU), which enables learning representations in high dimensional space with more generalization power and fewer parameters.
Y-Net: Joint Segmentation and Classification for Diagnosis of Breast Biopsy Images
In this paper, we introduce a conceptually simple network for generating discriminative tissue-level segmentation masks for the purpose of breast cancer diagnosis.
ESPNet: Efficient Spatial Pyramid of Dilated Convolutions for Semantic Segmentation
We introduce a fast and efficient convolutional neural network, ESPNet, for semantic segmentation of high resolution images under resource constraints.
Ranked #48 on
Semantic Segmentation
on PASCAL VOC 2012 test
Identifying Most Walkable Direction for Navigation in an Outdoor Environment
We present an approach for identifying the most walkable direction for navigation using a hand-held camera.
DeepSolarEye: Power Loss Prediction and Weakly Supervised Soiling Localization via Fully Convolutional Networks for Solar Panels
The impact of soiling on solar panels is an important and well-studied problem in renewable energy sector.
Learning to Segment Breast Biopsy Whole Slide Images
We trained and applied an encoder-decoder model to semantically segment breast biopsy images into biologically meaningful tissue labels.
Region Graph Based Method for Multi-Object Detection and Tracking using Depth Cameras
Experimental results show that the proposed method is able to suppress high magnitude noise in depth maps and detect/track the objects (with and without occlusion).
