Accessing sensor data

On this page:

 

Accessing sensor data on Unity can be achieved by two methods:

 

All sensors include the following data:

  • SeensorInfo
    • SensorDeviceId
      • [string] ID: Device ID
      • [string] SubID: Device SubID
    • [string] Location: Sensor Location
  • Timestamp
    • [long] SystemTimeMicroSeconds
    • [long] HardwareTimeMicroSeconds
    • [long] OmniceptTimeMicroSeconds
  • [unit] ABIIType: unit indicating the internal message type

 

If you want to understand more about the sensors and it's outputs read the fundamentals page.

 

Sensor data

Heart Rate

PPG (photoplethysmogram) sensor detects the blood volume changes in the microvascular bed of tissue using light signals that reflect onto the skin. The PPG sensor is located on the forehead, represented by two green LEDs.   HP does not provide access to the raw PPG data.  Why?

Sensor data [HeartRate]

  • [unit] Rate: Beats Per Minute (BPM)

 

Eye Tracking

Cameras pointing at the user's left and right eye. The eye camera and eye-tracking on the HP Reverb G2 Omnicept edition is powered by Tobii.

Sensor data [EyeTracking]

  • [Eye]  LeftEye / RightEye
    • [EyeGaze] Gaze: x,y,z eye gaze.
    • [PupilPosition] PupilPosition: x,y position.
    • [float] PupilDilation
    • [float] PupilDilationConfidence
    • [float] Openness

    • [float] OpennessConfidence

  • [EyeGaze] CombinedGaze: x,y,z eye gaze.

 

Camera Image

A camera located on the bottom of the headset pointing through the mouth of the user.

Sensor data [CameraImage]

  • [FormatT] Format: Image format
  • [byte[]] ImageData: Raw image data.
  • [unit] Width
  • [unit] Height
  • [ulong] FrameNumber
  • [float] FramesPerSecond

 

IMU

An inertial measurement unit (IMU) is used for the sensors. This IMU is not the one used for tracking on the HMD.

Sensor data [IMUFrame]

  • [List<IMU>] Data
    • [IMU3AxisValues] Acc: X,Y,Z values for IMU acceleration
    • [IMU3AxisValues] Gyro: X,Y,Z values for IMU gyro

 

Cognitive Load

A user’s cognitive load is the mental effort associated with performing a task or engaging in a learning process. Real-time biometric indicators of cognitive load are sourced from Omnicept sensor data for analysis, yielding a numerical estimate of cognitive load as a continuous value ranging from 0.0 to 1.0.

Cognitive Load starts reporting ~12-24 seconds from the first time Omnicept receives eye-tracking information.  After 24 seconds, cognitive load is reported once per second.    Prior to the 24 seconds, no value is sent from the cognitive load.   Eye-tracking and heart rate sensors must be enabled to get Cognitive Load output.

Sensor data [CognitiveLoad]

  • [float] CognitiveLoadValue
  • [float] StandardDeviation
  • [string] DataState

Message versions: 

  • Cognitive Load 1.1.0 (Latest) - Eye Tracking and PPG sensor inputs are both incorporated into the model results. During time periods when PPG signal quality is poor or the PPG sensor is unavailable, the Eye Tracking sensor data alone will be used to estimate cognitive load.
  • Cognitive Load 1.0.0 – Legacy support for 4/30/2021 Omnicept 1.0 release. Uses Eye Tracking as the only sensor input. Use this version only if your application must match Omnicept 1.0.

Learn more on Cognitive load.

Heart Rate Variability

Derived from the PPG sensor.

Sensor data [HeartRateVariability]

  • [float] Sdnn: Standard deviation between two normal heartbeats in milliseconds
  • [floa] Rmssd: Root-mean square of successive differences in milliseconds

 

Special access data

This data can only be accessed with special permission and legal agreement per use case.

PPG

Sensor data [PPGFrame]

  • [List<PPG>] Data
    • ​[List<unit>] PpgValues
  • [List<unit>] LEDCurrents