Day: March 29, 2019

EngD Research Project 03: Introduction of Depth Sensors

Another update to this research project! Zubr VR is never late when it comes to the latest hardware. Depth sensors! What is it? It’s a camera that can detect depth, which means the camera can tell the distance to what it can see. In this blog we’ll take a look at the depth sensors that I have had a chance to test and talk about the advantages and disadvantages of each one.

First let’s look at the depth sensors. We shall take a look at five depth sensors. These are the Kinect for XBOX One (Kinect V2), Stereolabs ZED, Stereolabs ZED Mini, Intel RealSense D415 and Intel RealSense D435.

Kinect for XBOX One (Kinect V2)

This depth sensor was discontinued since April 2016. The bulkiest out of all the depth sensors, the V2 uses infrared to for creating depth maps. It is designed for human interaction such that it uses body tracking to estimate a person’s body structure. Microsoft are also currently developing a new Kinect in 2019. Although it is the oldest of the depth sensors, it is still being actively used for research.

Intel Realsense Series (D415 & D435)

The RealSense cameras are the smallest in size compared to all the other depth sensors but despite its size, there is no problem when it comes to obtaining depth data. D415 uses rolling shutters whereas D435 uses global shutters. The latter is better at capturing fast moving objects which may be essential with this research project. According to Intel, the D415 is better for more precise measurements whereas the D435 with its wider field of view are recommended for robotics and other computer vision research.

Stereolabs Cameras (ZED & ZED Mini)

The ZED camera was the camera used in creating Zubr’s real-time mixed reality application. ZED Mini is a smaller version of the ZED and can be attached on a VR headset for AR applications. The ZED Mini also has a built-in Inertial Measurement Unit (IMU) whereas the ZED does not.

Technology behind depth sensing

The Kinect V2 consists of three hardware components; RGB camera, infrared (IR) emitter, sensor. The RGB camera can be used on its own as a web camera if the depth is not required. To create depth, it uses Time of Flight. An IR light source is projected to the scene and the sensor reads the length of time for it to bounce back. The longer the time, the further the object is from the camera.

The Intel RealSense cameras consist of four components; an RGB camera, IR emitter and two sensors. Firstly, the RGB camera is used to create the 3 color channels of the scene. A depth is created by projecting an IR emitted pattern. The pattern allows the two sensors to select the same point from each view and the reconstruction method is used to measure distance. Stereolabs cameras on the other hand does not use infrared. It instead uses two RGB cameras and the reconstruction method is applied.