NYUv2 (NYU-Depth V2)

Introduced by Nathan Silberman et al. in Indoor Segmentation and Support Inference from RGBD Images

The NYU-Depth V2 data set is comprised of video sequences from a variety of indoor scenes as recorded by both the RGB and Depth cameras from the Microsoft Kinect. It features:

1449 densely labeled pairs of aligned RGB and depth images
464 new scenes taken from 3 cities
407,024 new unlabeled frames
Each object is labeled with a class and an instance number. The dataset has several components:
Labeled: A subset of the video data accompanied by dense multi-class labels. This data has also been preprocessed to fill in missing depth labels.
Raw: The raw RGB, depth and accelerometer data as provided by the Kinect.
Toolbox: Useful functions for manipulating the data and labels.

Source: https://cs.nyu.edu/~silberman/datasets/nyu_depth_v2.html

Homepage

Benchmarks

Add a new result Link an existing benchmark

Task	Dataset Variant	Best Model
Semantic Segmentation	NYU Depth v2	OmniVec
Monocular Depth Estimation	NYU-Depth V2	Metric3Dv2
3D Semantic Scene Completion	NYUv2	SISNet
Depth Estimation	NYU-Depth V2	EVP
Real-Time Semantic Segmentation	NYU Depth v2	AsymFormer
Monocular Depth Estimation	NYU-Depth V2 self-supervised	IndoorDepth
3D Semantic Scene Completion from a single RGB image	NYUv2	NDC-Scene
Surface Normals Estimation	NYU Depth v2	Metric3Dv2
Depth Completion	NYU-Depth V2	NLSPN
Zero-shot Scene Classification (unified classes)	NYU Depth v2	LanguageBind
Plane Instance Segmentation	NYU Depth v2	PlaneNet
Surface Normal Estimation	NYU-Depth V2	Bae et al.
Multi-Task Learning	NYUv2	SwinMTL
3D Object Detection	NYU Depth v2	SGPN-CNN
Boundary Detection	NYU-Depth V2	InvPT
Panoptic Segmentation	NYU Depth v2	EMSANet
Scene Classification (unified classes)	NYU Depth v2	EMSANet
Instance Segmentation	NYU Depth v2	SGPN-CNN
Surface Normals Estimation	NYU-Depth V2 Surface Normals	DSN
Scene Segmentation	NYU Depth v2	Dilated FCN-2s RGB