Avateering is the most exciting part of Kinect development and one of the most complex features to implement. Vitruvius encapsulates all of the hard work and avateering algorithms and exposes an easy-to-use API.

You only need a few rigid 3D models. Vitruvius comes with 9 models, which you can use for free in your personal or commercial applications. Also, Vitruvius includes numerous textures to choose from, so each model can appear with different clothing!

{ Reasonably, Vitruvius is featured in the official Microsoft Kinect Website and Channel9 }

If you need a different model and have a 3D artist, just provide them with the rigid skeleton and let them create a new 3D model on top of it. If you need different models and do not have a 3D artist, give us a shot and we can create a 3D model tailored to your needs.

In this tutorial, I am going to show you how you can integrate Vitruvius within your Unity project and animate the bundled male and female 3D models. Vitruvius allows a single player control a single avatar or multiple avatars. The videos below demonstrate the difference:

Prerequisites

Source Code

The source code, along with the required assemblies, is included in the following versions of Vitruvius:

Step 1: Create a new Unity project

Launch Unity and create a new project, or open an existing Kinect project. To get started, simply double-click the .unitypackage file from the fodler you downloaded.

You’ll also need to import Windows.Kinect.dll and LightBuzz.Vitruvius.dll. These are the required assemblies to use the complete power of Vitruvius.

Step 2: Import the 3D models

Open your scene file in the Editor and import the 3D models. Place them whenever you think it’s necessary for your game. I have imported the male and female 3D body models. They are named “male” and “female”, respectively.

This is how my editor looks like:

Avateering Unity Editor

Step 3: Action!

It’s now time to write your Kinect script. You’ll need to include the following namespaces:

using Windows.Kinect;
using LightBuzz.Vitruvius;

After importing the assemblies, we’ll create the required members to open the sensor, acquire the Body data and update the avatars accordingly.

The separatedPlayers variable indicates whether a single player will control one avatar or multiple avatars.

Members

// Kinect
BodyFrameReader bodyReader;
Body[] users;

// 3D models
Model[] models;

public FBX female;
public FBX male;

// Determines whether
// one player will control all avatars, or
// whether each player will control one avatar
public bool seperatedPlayers;

Start

The Start method initializes Kinect and Vitruvius. It’s important to open the sensor and enable avateering functionality to your 3D models. The code is really simple and self-explanatory:

void Start()
{
    // 1. Initialize Kinect
    sensor = KinectSensor.GetDefault();

    if (sensor != null)
    {
        bodyReader = sensor.BodyFrameSource.OpenReader();

        sensor.Open();
    }

    // 2. Enable Avateering
    Avateering.Enable();

    // 3. Specify the 3D models to animate.
    models = new Model[]
    {
        female,
        male
    };

    // 4. Initialize each 3D model.
    for (int i = 0; i < models.Length; i++)
    {
        models[i].Initialize();
    }
}

Update

Now, it’s time for the cool part! To see your avatars animated, you simply have to loop through the tracked bodies and update the corresponding avatar. Vitruvius does this with just one line of code.

void Update()
{
    if (bodyReader != null)
    {
        using (var frame = bodyReader.AcquireLatestFrame())
        {
            if (frame != null)
            {
                users = frame.Bodies().
                        Where(b => b.IsTracked).ToArray();

                if (seperatedPlayers)
                {
                    // Each user controls a different avatar.
                    for (int index = 0; index < users.Length; index++)
                    {
                       Model model = models[index];
                       Body user = users[index];

                       // Yes, this line does ALL of the hard work for you.
                       Avateering.Update(model, user);
                   }
                }
                else if (users.Length > 0)
                {
                    // A single user controls all of the avatars.
                    for (int index = 0; index < models.Length; index++)
                    {
                        Model model = models[index];
                        Body user = users[0];

                        // Yes, this line does ALL of the hard work for you.
                        Avateering.Update(model, user);
                    }
                }
            }
        }
    }
}

Dispose

Finally, do not forget to dispose Kinect and clean any resources.

public void Dispose()
{
    // 1. Dispose Kinect
    if (bodyReader != null)
    {
        bodyReader.Dispose();
        bodyReader = null;
    }

    if (sensor != null)
    {
        if (sensor.IsOpen)
        {
            sensor.Close();
        }

        sensor = null;
    }

    // 2. Dispose 3D models
    for (int i = 0; i < models.Length; i++)
    {
        models[i].Dispose();
    }

    // 3. Disable avateering
    Avateering.Disable();
}

This is it! You have just developed an avateering app using Kinect v2!

BONUS – New 3D models

Vitruvius is bundled with nine 3D models you can use in your commercial projects. We are adding new 3D models all the time. Our models are perfectly rigged and work with Kinect just fine. Each model is coming in multiple textures. to modify a texture, simply open the Samples/Models/FBX folder and pick the 3D models and textures of your choice.

The following models are included:

  • Female Casual – a beautiful female model with various hair colors and clothes.
  • Male Casual – a male model with various clothes.
  • Female Athletic – a female model with sports clothing.
  • Male Athletic – a male model with sports clothing.
  • Kid – a 3D model of a kid.
  • Skeleton – a fantastic model of a human skeleton with bones and skull!
  • Longsleeve – a shirt with long sleeves and various colors.
  • Tshirt – a simple t-shirt model with various colors.
  • Pants – a pair of trousers with various styles (jeans, black, formal).

Bodies

Vitruvius Kinect Models Bodies

Clothes

Vitruvius Kinect Models Clothes

This is a video of the skeleton model in action:

So, what will you do with Vitruvius? Let me know in the comments below!

Download Vitruvius

Author Vangos Pterneas

Vangos Pterneas is a Microsoft Most Valuable Professional in the Kinect technology. He helps companies from all over the world grow their revenue by creating profitable software products. Vangos is the owner of LightBuzz Software agency and author of The Dark Art Of Freelancing. Read more

More posts by Vangos Pterneas

Join the discussion 25 Comments

  • Tashi says:

    hi i was trying to add my own model for avateering. But when i run it my new model is not copying the users gestures. The error is it is saying the model is not initialized. Wait for ur help

  • Usman says:

    Hi Vangos,

    How can i port unity game which uses your libraries for windows store 8.1 desktop version ? it was showing some conflicts to kinect libraries

    Regards,
    Usman

  • rami says:

    hi vangos,
    i am working on fitting room project for retail shop.can you tell me how i can add clothes of different size???

    • Hello, Rami. Thank you for your message. If you already have a 3D model of a cloth (or if you are using one of the existing clothes), you can change the Color Scale Factor (or the Depth Scale Factor) and increase or decrease the size of the 3D model. This way, you’ll be able to have different sizes. The Scale Factor is a property of the AvatarCloth class.

  • bader says:

    dear
    i tried to import 3dmodel from 3dmax to unity vitruvius fitting room but i can not control it please let me know how to configer it

    • Hello, Bader. Thank you for your message. I assume you have already added your .fbx model in your Unity scene. In your Hierarchy window, select your model and click “Add Component”. Then, add the “AvatarCloth.cs” script.

      In your C# code, you need to call the following methods:


      void Start()
      {
      // Initialize the sensor, like the Samples.
      Avateering.Enable();
      model.Initialize();
      }

      void Update()
      {
      // Get the Body object, like the Samples.
      // Then, animate the 3D model.
      Avateering.Update(model, body);

      // Then, update the position of the 3D model in the 2D Color space:
      var position = body.Joints[model.Pivot].Position.ToPoint(Visualization.Color);
      if (!float.IsInfinity(position.x) && !float.IsInfinity(position.y))
      {
      frameView.SetPositionOnFrame(ref position);
      model.SetBonePosition(model.Pivot, position);
      var distance = model.JointInfos[(int)model.Pivot].RawPosition.z;
      if (distance != 0)
      {
      model.Body.transform.localScale = model.ScaleOrigin * (model.colorScaleFactor / distance) * frameView.ViewScale;
      }
      }
      }

      void OnApplicationQuit()
      {
      Avateering.Disable();
      model.Dispose();
      }

      Consider checking the FittingRoom sample, too. You could simply drag-and-drop your model in your scene.

      Feel free to contact us if you need more information or screenshots.

  • bader says:

    dear
    thank you for help but could you send to me how to control FBX model that i add it to FittingRoom sample because i still have problem with code
    is the code below correct
    using UnityEngine;
    using Windows.Kinect;
    using LightBuzz.Vitruvius;
    using LightBuzz.Vitruvius.Avateering;

    public class body111 : MonoBehaviour {

    // Use this for initialization
    void Start () {
    // Initialize the sensor, like the Samples.
    Avateering.Enable();
    model.Initialize();
    }

    // Update is called once per frame
    void Update () {
    // Get the Body object, like the Samples.
    // Then, animate the 3D model.
    Avateering.Update(model, body);

    // Then, update the position of the 3D model in the 2D Color space:
    var position = body.Joints[model.Pivot].Position.ToPoint(Visualization.Color);
    if (!float.IsInfinity(position.x) && !float.IsInfinity(position.y))
    {
    frameView.SetPositionOnFrame(ref position);
    model.SetBonePosition(model.Pivot, position);
    var distance = model.JointInfos[(int)model.Pivot].RawPosition.z;
    if (distance != 0)
    {
    model.Body.transform.localScale = model.ScaleOrigin * (model.colorScaleFactor / distance) * frameView.ViewScale;
    }
    }
    }

    void OnApplicationQuit()
    {
    Avateering.Disable();
    model.Dispose();
    }
    }

  • […] need to map specific body parts to these joint rotations. This post is using the male model from Vitruvius avateering tools, but you are welcome to use any properly rigged […]

  • […] need to map specific body parts to these joint rotations. This post is using the male model from Vitruvius avateering tools, but you are welcome to use any properly rigged […]

  • […] need to map specific body parts to these joint rotations. This post is using the male model from Vitruvius avateering tools, but you are welcome to use any properly rigged […]

  • Amin says:

    Hi Vangos.I want to add a virtual 3d model of hair. How can i do this? Is there any link to help me?

    • Hello Amin. You can add the 3D model of the hair as a child element of the Head of the avatar. This way, the hair will follow the position and rotation of the head. You can also set your own offset. Please, use our support page, so we can send you more details and screenshots.

  • Marko says:

    Hi,

    I’m working on my Master Thesis and would be great if I could use your models in my project.
    Is there any free student version of the product?

    Kind regards,
    Marko

  • Hi Vangos, our company offers Body tracking SDK for Android computer Orbbec Persee (with 3D sensor) http://www.3divi.com/products/nuitrack-sdk-for-persee and our clients ask -can your solution be ported to Orbbec Persee Android sensor? Looking forward for your reply. Thanks

    • Hello, Dmitry. Vitruvius is compatible with Kinect v2. Internally, our framework is capable of animating any 3D model, but you’ll need a custom version to support different sensors. Please, contact our team in case you would like to discuss about a custom version of Vitruvius.

  • Agustin says:

    Hi Vangos, the script avataringSample does’t recognise my 3d model and debug this error:
    NullReferenceException: Object reference not set to an instance of an object
    LightBuzz.Vitruvius.Avateering.Pose.ReadPose (LightBuzz.Vitruvius.Avateering.Model model)
    LightBuzz.Vitruvius.Avateering.FBX.InitializeBody ()
    LightBuzz.Vitruvius.Avateering.Model.Initialize ()
    AvateeringSample.Awake () (at Assets/LightBuzz.Vitruvius/Scripts/Samples/Avateering Sample/AvateeringSample.cs:35)

  • Agustin says:

    already try all that u.u, when i replace the models with mine it says error

Leave a Reply