In this tutorial, I am going to help you get started with Vitruvius and Unity3D, step by step.
{ By the way, Vitruvius is featured in the official Microsoft Kinect Website and Channel9 }
Prerequisites
- Kinect for XBOX v2 sensor with an adapter (or Kinect for Windows v2 sensor)
- Kinect for Windows v2 SDK
- Unity3D 5.0 or higher
- Visual Studio 2013 or higher
- Windows 8.1 or higher
- A dedicated USB 3 port
Binaries
The following binaries are included:
LightBuzz.Vitruvius.dll
This is the most important file. Contains all of the magic sauce of Vitruvius.
Windows.Kinect.dll
Since Microsoft does not have an “official” Unity-compatible dll, we created for you. This dll should be always included in your projects.
In case you are already using the old Microsoft’s Unity package, you’ll need to delete the StandardAssets code and keep Windows.Kinect.dll.
Guide
Now, let’s create our first Kinect project using Unity and Vitruvius.
Step 0 – Download Unity & Visual Studio
First things first. To use the full power of Vitruvius, you need the Unity3D and Visual Studio (or MonoGame).
I recommend using the latest version of Unity, even though any version higher than 5.0 would do the job just fine. Unity is a powerful game engine based on DirectX. It gives you the ability to create amazing 3D and 2D applications easily, with performance in mind. Visual Studio is a state-of-the-art Integrated Development Environment that helps you build, debug, and deploy your apps.
Step 1 – Download Vitruvius
Unity binaries and samples are included in the Academic, Premium, and Platinum versions of Vitruvius. So, download the version of your choice and unzip the folder. The folder is named LightBuzz.Vitruvius.x.x.x.
The following files and folders are included:
Assemblies
Contains the .dll files for the supported platforms.
Documentation
Contains MSDN-like documentation files.
Samples
Contains a dozen demo projects to get started.
Changelog.txt
Includes the recent version changes and updates.
License.pdf
A dull, boring, legal agreement. In short: you can use the products you create with Vitruvius as you like. Just don’t re-sell the binaries.
ReadMe.pdf
Some basic information about the hardware and software requirements.
ThankYou.pdf
A personal message from Vangos Pterneas (Microsoft Kinect MVP) to you.
Step 2 – Open the Unity Package
We’ll get started immediately! Let’s open the one of the demos. Vitruvius comes with a handy .unitypackage file. If you are already familiar with Unity, you know that a .unitypackage is a set of code, assets, and dll files that can be integrated into your own projects. To open the .unitypackage file, navigate to the folder:
LightBuzz.Vitruvius.x.x.x ⇒ Samples ⇒ LightBuzz.Vitruvius.Samples.Unity
Double-click the .unitypackage and Unity Engine will launch automatically.
Step 3 – Create a new Project
When Unity launches, you have to either create a new project or choose an existing one. Vitruvius binaries will be installed automatically.
Let’s create a new project. Specify a name and a location for your project. Click “Create Project”.
Wait a few seconds while Unity is importing the packages. When finished, you should see a blank screen with all of the imported data.
Step 4 – Open a Sample Demo Scene
To reveal the sample demos, open the following folder:
Assets ⇒ LightBuzz.Vitruvius ⇒ Scenes
The following scenes are available:
Angle Sample
Displays angles between some specified human body joints, along with visual arcs.
Avateering Sample
Animates five 3D body models based on the movements of the human body.
Avateering Color Sample
Animates a 3D model on top of the Color stream.
Face Sample
Shows how easy it is to acquire High Definition Face points (eyes, nose, forehead, mouth, chin, jaw, chins, cheeks).
Fitting Room Sample
This is a virtual mirror (fitting room) demo with three clothes. Women will love it!
Frame Effect Sample
Shows the streams of every camera (Color, Depth, Infrared) and a stickman.
Gestures Sample
Demonstrates some common Kinect gestures (wave, swipe, zoom, start).
Green Screen Sample
The classic background removal application.
Green Screen Fitting Room Sample
This is a combination of the background removal demo and the fitting room demo: the app detects the body, removes the background, and puts on some clothes.
Recording Sample
Using this sample, you can record & playback the Color, Depth, Infrared, Body, and Face streams.
Selective Angle Sample
Shows you how to move only specific parts of a 3D avatar.
To open a scene, double-click the corresponding file. For example, I’m opening the AvateeringSample.
To edit the source code of each scene, navigate to the folder:
Assets ⇒ LightBuzz.Vitruvius ⇒ Scripts
The Scripts folder contains the code-behind C# files that are used to control the game objects and give interactivity to the apps.
Step 5 – Play!
To test a scene, click the Play button.
And… that’s it! You just saw how easy it is to get started with Kinect using Unity3D and Vitruvius.
So, go on! Modify the samples to create your own amazing apps. What are you planning to develop?
Till the next time… keep Kinecting!
I have a question, is it the angle sample is only for display purpose? what if I want to record and save the angle data in any file form so that I can analyse it? Thank you
Hello Syun. You can record everything with Vitruvius. Vitruvius is recording the body, face, and floor data, so you can then reproduce the exact same results during playback.
What about the dressing app, is it fully functional or just demo ?
The dressing app allows you to virtually try 10 3D models of clothes. You can customize the sample to fit your needs and UI.
Hi,
I am not able to find the orbbec astra pro camera in the sensor type. I can only find the kinect 2 and nuitrack. How can I add my camera? I have the vitruvious academic version and installed all the steps before.
Hello Dion. You need to select “Nuitrack” and Vitruvius will automatically detect the connected camera. Let me know if that worked for you!
Unfortunately not, usually when I use the camera I can see a red sign in the lens. This will not happen when I run the program. There have to be a mistake anywhere else but I cant find it.
This can be caused by either of these reasons:
Reason 1: there is no body-tracking module installed. Ensure you have installed Nuitrack.
Reason 2: you have not specified the correct USB port. If that’s the case, do this:
0) Close Unity3D.
1) Navigate to your Nuitrack installation folder (e.g. C:\Program Files\Nuitrack\nuitrack\data).
2) Open the nuitrack.config file using a text editor.
3) Find the following entry:
“AstraProPerseeDepthProvider”: {
“Windows”: {
“CameraID”: -1
},
“POSIX”: {
“PID”: “0x0501”,
“VID”: “0x2bc5”
}
}
4) Change the CameraID value to 0, 1, 2, or 3.
5) Save the file.
6) Run Unity3D again.
I tried but still no response from my Orbbec camera. The “nuitrack_c11_sample” does work and the body tracking aswell. Only not in Unity. Do I maybe have to add an extra asset or something in unity? I opened the vitruvius like described in this blog and tried to “play” the scene. I tried a lot but nothing is working. Do I need to follow an extra step as you describe above to open the vitruvius scenes?
In your Unity scene (e.g. the Avateering sample), you should also select the Nuitrack sensor from the drop-down menu.
Are you getting Color, Depth, or Skeleton data?
Yes I have select the Nuitrack sensor. I dont get any data from my camera. The camera is not responding on the unity software.
The problem is solved!
Only now when I use the AvateeringSample and the camera detects a body the program is going in the “break” mode. Is this a setting thing or just because of the computer can not handle the software?
Hello Dion. I’m glad that worked for you. Is Unity pausing or crashing? Are the skeletons displayed?
Hi Dion,
I have the same problem but my unit crassing
Can you help me how did you solve it?
He is pausing. When he detectes the skeleton and also displayed it he is getting in the pause mode. There isn’t an error or something.
Hello Dion. Is this occurring in every sample? Have you added any Debug.Break() commands in the C# code?
Hi, Not in every sample but just a few. Now I get an error that he is not able to detect a few bodypoints. Is that an reason he gets in the pausing mode? This happens in the first second so I am not able to step in the right position. I didn’t add any commands in the script.
I see. Consider using Unity 2017 LTS. This is the most stable version of Unity for Kinect applications.
This was not the solution. The avateering scene is still getting in the pausing mode and gives the following error:
transform.position assign attempt for ‘FootRight’ is not valid. Input position is { NaN, NaN, NaN }.
UnityEngine.Transform:set_position(Vector3)
ScreenViewStickman:UpdateStickman(SensorAdapter, Frame, Body, Transform, Visualization) (at Assets/LightBuzz.Vitruvius/Scripts/ScreenViewStickman.cs:78)
Sample_Avateering:Update() (at Assets/LightBuzz.Vitruvius/Scripts/Sample_Avateering.cs:118)
And this for every bodypoint.
Changing the Z scale and Z position of the ground gameobject works, so I do not have any error now. Just the pausing mode is still a problem. After every movement it is getting in that mode.
Is it still causing errors? Also, in your Unity Console window, you can click the “Error Pause” option. This way, Unity3D will not pause.
You are right, still getting this error.
transform.position assign attempt for ‘FootRight’ is not valid. Input position is { Infinity, NaN, Infinity }.
UnityEngine.Transform:set_position(Vector3)
ScreenViewStickman:UpdateStickman(SensorAdapter, Frame, Body, Transform, Visualization) (at Assets/LightBuzz.Vitruvius/Scripts/ScreenViewStickman.cs:78)
Sample_Avateering:Update() (at Assets/LightBuzz.Vitruvius/Scripts/Sample_Avateering.cs:118)
You can add the following check in the ScreenViewStickman.cs file:
if (!float.IsNaN(currPosition.X) && !float.IsNaN(currPosition.Y) && !float.IsNaN(currPosition.Z) &&
!float.IsInfinity(currPosition.X) && !float.IsInfinity(currPosition.Y) && !float.IsInfinity(currPosition.Z) &&
currPosition != Vector3D.Zero)
{
jointPoints[i].position = Vector3D.Lerp(jointPoints[i].position, currPosition, smoothness);
}