Image-text matching
83 papers with code • 1 benchmarks • 1 datasets
Benchmarks
These leaderboards are used to track progress in Image-text matching
| Trend | Dataset | Best Model | Paper | Code | Compare |
|---|---|---|---|---|---|
|
Libraries
Use these libraries to find Image-text matching models and implementations
CommercialAdsDataset
Most implemented papers
AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networks
taoxugit/AttnGAN
•
•
CVPR 2018
In this paper, we propose an Attentional Generative Adversarial Network (AttnGAN) that allows attention-driven, multi-stage refinement for fine-grained text-to-image generation.
UNITER: UNiversal Image-TExt Representation Learning
ChenRocks/UNITER
•
•
ECCV 2020
Different from previous work that applies joint random masking to both modalities, we use conditional masking on pre-training tasks (i. e., masked language/region modeling is conditioned on full observation of image/text).
VinVL: Revisiting Visual Representations in Vision-Language Models
In our experiments we feed the visual features generated by the new object detection model into a Transformer-based VL fusion model \oscar \cite{li2020oscar}, and utilize an improved approach \short\ to pre-train the VL model and fine-tune it on a wide range of downstream VL tasks.
Stacked Cross Attention for Image-Text Matching
Prior work either simply aggregates the similarity of all possible pairs of regions and words without attending differentially to more and less important words or regions, or uses a multi-step attentional process to capture limited number of semantic alignments which is less interpretable.
BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation
salesforce/lavis
•
•
Furthermore, performance improvement has been largely achieved by scaling up the dataset with noisy image-text pairs collected from the web, which is a suboptimal source of supervision.
Align before Fuse: Vision and Language Representation Learning with Momentum Distillation
salesforce/lavis
•
•
NeurIPS 2021
Most existing methods employ a transformer-based multimodal encoder to jointly model visual tokens (region-based image features) and word tokens.
Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks
Large-scale pre-training methods of learning cross-modal representations on image-text pairs are becoming popular for vision-language tasks.
VL-BERT: Pre-training of Generic Visual-Linguistic Representations
jackroos/VL-BERT
•
•
ICLR 2020
We introduce a new pre-trainable generic representation for visual-linguistic tasks, called Visual-Linguistic BERT (VL-BERT for short).
Dual Attention Networks for Multimodal Reasoning and Matching
iammrhelo/pytorch-vqa-dan
•
•
CVPR 2017
We propose Dual Attention Networks (DANs) which jointly leverage visual and textual attention mechanisms to capture fine-grained interplay between vision and language.
Matching Images and Text with Multi-modal Tensor Fusion and Re-ranking
Wangt-CN/MTFN-RR-PyTorch-Code
•
•
We propose a novel framework that achieves remarkable matching performance with acceptable model complexity.
