On this page:

• Subscription List
• Sensors data:
  • Heart Rate
  • Eye Tracking
  • Camera Image
  • IMU
  • Cognitive Load
  • Heart Rate Variability
• Special Access data:
  • PPG

 

Accessing sensor data on Unreal can be achieved by two methods:

• Delegates: Handling the delegates for the data messages.
• Polling: Getting the sensor data on demand. 

If you want to understand more about the sensors and it's outputs read the fundamentals page.

 

Subscription List

By default, omnicept plugin subscribes to all available message types. A user application may also choose to subscribe to only specific message types. A blueprint function, Set Subscription is provided to customize message type subscription after the client program has been successfully connected to the Glia runtime. 

Set Subscription function takes a list of Subscription entity that contains the subscription information of a message type respectively. A complete Subscription entity contains the following information:

• Message Type: A mandatory field to specify the target message type of the subscription.
• Version: An optional field to specify the target version of the message of the subscription. If the Version field is empty, only the latest available version will be automatically assigned by Omnicept runtime.
• Sensor Info: An optional field to specify the target sensor that generates the message of the message type. If Sensor Info filed is empty, ANY sensor generating the message of the message type will be delivered.
• Sender: An optional field to specify the target sender that sends the message of the message type. If the Sender field is empty, ANY sensor device generating the message of the message type will be delivered. 

A subscription list can be made and set as follows:

 

And handle the result:

Upon receiving the subscription request from the client, Omnicept runtime will process and reply to a SubscriptionResultList message to either accept or reject the subscription of each message type individually. Subscriptions to individual messages may be accepted or rejected due to such factors as the availability of the version of the message type or the licensing privilege of the client, as communicated via the ‘Result Error’ blueprint field.

 

Sensor data

Heart Rate

PPG (photoplethysmogram) sensor detects the blood volume changes in the microvascular bed of tissue using light signals that reflect onto the skin. The PPG sensor is located on the forehead, represented by two green LEDs.   HP does not provide access to the raw PPG data.  Why?

Sensor data [FGliaHeartRateEvent]

• [int] heartRate: Beats Per Minute (BPM)

 

Eye Tracking

Cameras pointing at the user's left and right eye. The eye camera and eye-tracking on the HP Reverb G2 Omnicept edition is powered by Tobii.

Sensor data [FGliaEyeTrackingEvent]

• [FEyeTracking]  eyeTracking

  • [FVector] CombinedGaze
  • [float] CombinedGazeConfidence
  • [FVector] LeftGaze
  • [float] LeftGazeConfidence
  • [FVector] RightGaze
  • [float] RightGazeConfidence
  • [FVector] LeftPupilPosition
  • [float] LeftPupilPositionConfidence
  • [FVector] RightPupilPosition
  • [float] RightPupilPositionConfidence
  • [float] LeftPupilDilation
  • [float] LeftPupilDilationConfidence
  • [float] RightPupilDilation
  • [float] RightPupilDilationConfidence
  • [float] LeftEyeOpenness
  • [float] LeftEyeOpennessConfidence
  • [float] RightEyeOpenness
  • [float] RightEyeOpennessConfidence

 

Camera Image

A camera located on the bottom of the headset pointing through the mouth of the user.

Sensor data [FGliaCameraImageEvent]

• [FCameraImage] cameraImage
  • [EFormatType] format
  • [TArray<uint8>] imageData: Raw image data.
  • [int32] width
  • [int32] height
  • [int32] frameNumber
  • [float] fps

How to use camera data.

IMU

An inertial measurement unit (IMU) is used for the sensors. This IMU is not the one used for tracking on the HMD.

Sensor data [FGliaIMUArrayEvent]

• [FIMUArray] imuArray
  • [TArray <FIMU>] Imu
    • [FVector] Accelerometer: X,Y,Z values for IMU acceleration
    • [FVector] Gyroscope: X,Y,Z values for IMU gyro

 

Cognitive Load

A user’s cognitive load is the mental effort associated with performing a task or engaging in a learning process. Real-time biometric indicators of cognitive load are sourced from Omnicept sensor data for analysis, yielding a numerical estimate of cognitive load as a continuous value ranging from 0.0 to 1.0.

Cognitive Load starts reporting ~12-24 seconds from the first time Omnicept receives eye-tracking information.  After 24 seconds, cognitive load is reported once per second.    Prior to the 24 seconds, no value is sent from the cognitive load.   Eye-tracking and heart rate sensors must be enabled to get Cognitive Load output.

Sensor data [FGliaCogLoadEvent]

• [FCognitiveLoad] cogLoad
  • [float] CognitiveLoad
  • [float] StandardDeviation
  • [FString] DataState
  • [FDateTime] StartDataCollectTime
  • [FDateTime] EndDataCollectTime

 

Message versions: 

• Cognitive Load 1.1.0 (Latest) - Eye Tracking and PPG sensor inputs are both incorporated into the model results. During time periods when PPG signal quality is poor or the PPG sensor is unavailable, the Eye Tracking sensor data alone will be used to estimate cognitive load.

• Cognitive Load 1.0.0 – Legacy support for 4/30/2021 Omnicept 1.0 release. Uses Eye Tracking as the only sensor input. Use this version only if your application must match Omnicept 1.0.

Learn more on Cognitive load.

Heart Rate Variability

Derived from the PPG sensor.