We have just released a new demo and code sample for Unity. With this sample, you will learn how you can use Omnicept data to drive avatar gaze and blinks.
You can download this new sample at:
Unity project brief description
On the unity project, you will find "BlinksGazeVR" scene which contains the sample code and avatars used.
In this scene, you can notice that there are two cameras. One of the cameras will be the VR camera (located at the XR Rig) and the other one is rendering the avatars to a texture so later they can be displayed on a world space canvas in front of the user.
The code related to the avatar's gaze and blink can be found on the "BlinksGazeVRController" which has references to the avatars at the scene and their bones.
The controller also has a reference to the Glia Controller which holds the last eye-tracking and camera position. Once we have this data on the update the visible avatar is updated as follows:
private void Update()
{
UpdateEyeState();
UpdateHeadState();
}
private EyeTrackingData eyeData = new EyeTrackingData();
private void UpdateEyeState()
{
EyeTracking eyeTracking = gliaController.lastEyeTracking;
eyeData.From(eyeTracking);
SetAvatarEyeState(eyeData);
}
private void SetAvatarEyeState(EyeTrackingData state)
{
if (state == null) return;
SetBlinkState(state);
SetGazeState(state);
}
private void SetBlinkState(EyeTrackingData state)
{
float leftBlinkValue = state.left_openness == 0 ? 100f : 0f;
float rightBlinkValue = state.right_openness == 0 ? 100f : 0f;
// mirroring, so left is right and vice versa
currentAvatar.SetBlendShapeWeight("Eye_Blink_L", rightBlinkValue);
currentAvatar.SetBlendShapeWeight("Eye_Blink_R", leftBlinkValue);
}
private void SetGazeState(EyeTrackingData state)
{
Vector3 vLeftEye = new Vector3(state.left_gaze_y * eyeModifier.x, 0f, state.left_gaze_x * eyeModifier.z);
Vector3 vRightEye = new Vector3(state.right_gaze_y * eyeModifier.x, 0f, state.right_gaze_x * eyeModifier.z);
// mirroring, so left is right and vice versa
leftEye.transform.localRotation = leftEyeStartRotation * Quaternion.Euler(vRightEye * 40f);
rightEye.transform.localRotation = rightEyeStartRotation * Quaternion.Euler(vLeftEye * 40f);
Vector3 vTestEye = new Vector3(0f, state.left_gaze_x, state.left_gaze_y);
testEye.transform.localRotation = testEyeStartRotation * Quaternion.Euler(vTestEye * 40f);
}As you can see on the above code the eye rotation is set with transform rotations and the blink is achieved with blend shapes. With this base knowledge, you are ready to open the unity project and start discovering more details.
Stay tuned for more updates and check out the demo here!