Floor Detection using Kinect

Kinect is primarily used in Body and Face tracking applications. Few people know, though, that Kinect is an amazing sensor for detecting the floor, too!

Vitruvius provides you with all the information you need to accurately recognize whether a particular plane in the 3D space is, actually, a floor. You can also acquire information such as the height the Kinect sensor is placed at and the tilt of its motor. You can even measure the distance between a point in the 3D space and the floor.

In today’s tutorial, I am going to show you the following:

• How to detect the floor clip plane.
• How to estimate the position (height) of the Kinect sensor.
• How to determine whether the field of view is crowded/limited.
• How to measure the distance between a point and the floor.

All this can have a ton of impact in modern Augmented Reality applications.

Prerequisites

To run the code and samples provided in this article, you’ll need the following:

• Kinect for XBOX v2 sensor with an adapter (or Kinect for Windows v2 sensor)
• Kinect for Windows v2 SDK
• Windows 8.1 or higher
• Visual Studio 2013 or higher
• A dedicated USB 3 port

Source code

The source code of the demo is included in the Academic, Premium, and Platinum versions of Vitruvius.

• Download Vitruvius

Detecting the Floor Clip Plane

Some floor-detection functionality is already built into the Kinect SDK. Please, meet the FloorClipPlane property. The FloorClipPlane is a member of the BodyFrame class. Very few people are aware of its existence, however, it’s one of the most useful components when developing motion apps.

This is how to access the FloorClipPlane property:

using (BodyFrame frame = e.FrameReference.AcquireFrame())
{
    if (frame != null)
    {
        Floor floor = frame.Floor();
    }
}

Here are some extremely useful Floor properties that will let you build motion apps that are aware of their environment!

Height

The Height property determines the vertical distance between the floor and the Kinect device. It is measured in meters. Using this property, you can provide feedback to your users:

float height = floor.Height;

if (height < 1f) // 1 meter
{
    Debug.WriteLine("The sensor is positioned too low!");
}

Field of View

The FieldOfView property determines whether the field of view is OK or the visibility is limited. This is a very easy way to understand whether there are any objects, such as tables, chairs, etc, that are blocking the field of view.

if (floor.FieldOfView == FieldOfView.Limited)
{
    Debug.WriteLine("The field of view is limited.");
}

Tilt angle

The Tilt property specifies the tilting angle of the device. All we need to do is use the orientation of the floor plane:

public double Tilt
{
    get
    {
        return Math.Atan(Z / Y) * (180.0 / Math.PI);
    }
}

Measuring distances

In one of my previous articles, I explained how to measure the distance between 2 points in the 3D space using simple Mathematical equations.

Now, we need to measure the distance between a point and a plane in the 3D space. This is trickier, but we can easily figure it out with the help of Geometry.

The detected floor is a plane with a known distance (W) and orientation (X, Y, Z). A point in the 3D space is a set of X, Y, and Z coordinates.

// Plane (X, Y, Z, W)
Floor floor = frame.Floor();

// Point (X, Y, Z)
CameraSpacePoint point = body.Joints[JointType.WristLeft].Position;

To measure the distance between a plane and a point, Vitruvius is using the Point-Plane Distance Formula:

This formula is implemented into the DistanceFrom method.

double distance = floor.DistanceFrom(point);

You can now get the 3D position of any point, pass it as a parameter to the above method, and find its distance from the floor.

Vitruvius also includes numerous extension methods that will let you draw the points of the floor.

‘Til the next time, keep Kinecting!