Skip to content

Configuring Handtracking#

Hand tracking technology allows users to interact with virtual environments using their bare hands, without the need for physical controllers. This technology is now integrated into next-generation VR headsets such as the Meta Quest 3 and the HTC Vive Focus 3, leveraging onboard cameras and sophisticated gesture recognition algorithms. Hand tracking can also be achieved through motion capture gloves (Manus Metagloves). XR Twin is compatible most hand tracking solutions on the market, as long as they comply with the OpenXR standard.

Why hand tracking ?#

Advantages Drawbacks
Enhanced Immersion: Using natural hand movements creates a more intuitive and realistic VR experience. Lower Precision: Compared to physical controllers, hand tracking can be less accurate, particularly for complex or fast actions. In addition, hands are only tracked within the VR headset field of view.
Improved Accessibility: Ideal for new users or individuals with physical limitations, as no controllers are required. Lighting Sensitivity: Performance can be affected by poor lighting conditions or overly bright environments.
Ease of Use: Quick to set up—simply put on the headset and start interacting. Latency Issues: Some delay may be noticeable, especially in high-speed or precision-based applications.
Greater Freedom: Less hardware to manage, which improves comfort and mobility. Lack of Haptic Feedback: Without physical feedback, interactions can feel less tactile or responsive.

Hand tracking represents a significant step toward more natural and user-friendly virtual experiences. While it's not yet a full replacement for controllers in all use cases—particularly where precision and haptics are crucial—it’s a powerful feature that continues to evolve and expand the possibilities of immersive technology.

Configure hand tracking in XR Twin#

  1. In the Working Tree, click on the OpenIcon User pop-up button next to the user object.

  2. In the Add User pop-up:

    • Ensure the display device is set to VR.

    • Click on silhouette's hand and select Hands instead of VR Controllers to enable hand tracking.

    • Click AddUserToSceneButton Add user to scene button.

  3. Click on StartButton Start button to start the VR simulation in XR Twin.

    1. Put on your headset but keep your controllers down and still (on the table).
    2. Make sure your hands are in sight of the VR headset sensors.
    3. Your virtual hands should move accordingly. You are now able to interact with physicalized objects as usual.

Users0

Users HandTracking_hand

How can I teleport or interact with the VR menu?

If you want to move around or interact with menus, you will need your controllers. Take your controllers in hand and use teleport or any other VR tool available. When you are done, place your controllers back down and still again. A few seconds later, hand tracking resumes.

Setup hand tracking with Vive Focus headsets#

From the Focus 3 headset#

In the headset, see the Head Up Display menus (HUD).

  • Hand tracking feature is active by default, but can be enabled/disabled.

    • From the VIVE Lobby menu, go to Settings > Inputs
    • Enable Hand tracking.

  • Hand tracking feature can be inactive in VIVE Streaming app.

    • With your head in the headset, double-click the Menu button on the left controller to display the VIVE Streaming Settings menu. Click on the Input tab. Hand tracking option must be checked.

In VIVE Streaming Settings software#

  • As an alternative to the HUD method described above, from the VIVE Streaming app, click the gear button to open the Settings pop-up, then select Input. Hand tracking option must be checked.

In SteamVR software#

  • Click the SteamVR SteamVR_15 burger button to open the menu.
  • Select Settings and navigate to the OpenXR panel.
  • Top right, SteamVR must be the the Current OpenXR Runtime. See image below.
  • Click MANAGE OPENXR API LAYERS button.
  • Toggle On the "Vive OpenXR Hand Tracking" API layer. See image below.
  • Disable Off all other OpenXR runtimes to ensure that only the Vive layer is the only one active.

SteamVR_5
SteamVR_5

Troubleshooting#

If you encounter any problems with Focus hand tracking.

  • Sometimes, when the Focus 3 is in standby mode, hand tracking does not reactivate after Start Simulation mode. In this case, click Stop, wake up the helmet and then click Start again.
  • If the hands are not tracked, from XR Twin Edition mode, verify that the User is VR+Hands.
  • If the headset does not "move" the view (no tracking), verify and/or try the following:
    • From SteamVR, SteamVR is the Current OpenXR Runtime. If not, set it so.
    • From the headset's Library menu, VIVE Business Streaming is launched (connection is effective). If not, launch again.
    • Reopen VIVE Business Streaming. Sometimes you'll need to restart XR Twin in this case.

Setup hand tracking with Quest headset#

Enable hand tracking feature in the Quest headset :

  • Press imgMeta button on your controller to open the universal menu.
  • Select the clock on the left side of the universal menu to open Quick settings.
  • Select icon_settings Settings in the top right, then scroll down and select icon_tracking Movement tracking.
  • Select the toggle next to Hand and body Tracking to turn this feature on (off to desable).
  • Put down your controllers (you can pick them up again at any time, see info below).

Setup hand tracking with Leap Motion#

Make sure Ultraleap SDK is properly installed (see Leap Motion Setup).

  • Click the SteamVR SteamVR_15 burger button to open the menu.
  • Select Settings and navigate to the OpenXR panel.
  • Top right, SteamVR must be the the Current OpenXR Runtime. See image below.
  • Click MANAGE OPENXR API LAYERS button.
  • Toggle On the "Ultraleap OpenXR Hand Tracking" API layer. See image below.
  • Disable Off all other OpenXR runtimes to ensure that only the Ultraleap one is active.

SteamVR_5
SteamVR_5

Setup hand tracking with Manus Metagloves#

Make sure Manus Core 2.0 is properly installed (see Manus Setup).

Connect your headset and Vive trackers#

  • Plug both VIVE dongles to your PC via USB (use a USB hub if you don't have enough ports),
  • Turn on both VIVE Trackers by pressing the power button. A solid green light indicates the trackers are on and paired. Icons are added in SteamVR window. If they aren't recognized: Navigate to Steam VR > Devices > Pair Controllers. Follow the instructions to pair VIVE Trackers.

SteamVR_14-1

Compatibility

Only HTC VIVE headsets PRO series are fully compatible with Manus gloves and VIVE trackers, using precise infrared laser based positioning.

Only HTC VIVE headsets Focus series are fully compatible with Manus gloves and VIVE Ultimate Trackers, using optical cameras and environment scanned maps positioning.

Setup your Manus gloves#

  • Plug the Manus Metagloves dongle into your PC via USB.
  • Insert a charged battery into both gloves.
  • Turn on both gloves using the power button and verify they are connected. A solid blue light without blinking indicates a successful connection.
  • Launch Manus Core. Manus Dashboard is launched automatically (see image below).
  • Click gear icon to open the Settings. Scroll to Tracking Systems and check OpenVR (SteamVR) checkbox.
    manus_10
  • If prompted with "Do you want to calibrate?" (or click Calibrate at the top right of the screen), select both. Follow the on-screen instructions to complete the calibration process. If both does not appear, restart Manus Core and the devices.

manus_8

MANUS Core 2.5.1 Dashboard. Two Metagloves and two trackers are detected and associated.

Configure Steam VR Settings#

  • Click the SteamVR SteamVR_15 burger button to open the menu.
  • Select Settings and navigate to the OpenXR panel.
  • Top right, SteamVR must be the Current OpenXR Runtime (see image below).
  • Click MANAGE OPENXR API LAYERS button.
  • Toggle On the OpenXR Manus Hand tracking API layer API layer. See image below.
  • Disable Off all other OpenXR runtimes to ensure that only the Manus one is active.

SteamVR_5
SteamVR_5

Troubleshooting hand tracking#

If two or more hand tracking openxr extensions are running then no hand data will be sent. Unfortunately, checking the list of active OpenXR extensions is not possible with a lot of OpenXR runtimes.

You can check which extensions are enabled with the open source tool: OpenXR Explorer. If more than one extension related to hand tracking is shown under API layers then hand tracking will not work.

openxr_explorer_ext

Congratulations, you are ready to use your hands to manipulate physicalized objects with XR Twin!