Skip to content

Handtracking

XR Twin is compatible with OpenXR. Many OpenXR standard hand tracking devices are now supported by XR Twin. To now more about Open XR.

Below are three examples of how to set up your devices to manipulate virtual objects directly with your hands :

Quest headset for XR Twin#

In this example, we use the Quest 3 headset to demonstrate the setup for optimal hand tracking performance in XR Twin.

Setup#

1. Requirements#
  • Download Meta and install Meta Quest Link on the computer. Update for last drivers if asked.
  • Setup the Quest 3 headset. Update for last drivers if asked. You need a Link cable type USB-C 3.2 (5 GBs mini). See Meta.
  • Launch Meta Quest Link and change the settings:
    • In Settings>General, search for OpenXR Runtime. Right, click Set Meta Quest Link as active... button. This set Meta Quest Link as the active OpenXR Runtime. See image below.
    • in Settings>Beta, search for Developer Runtime Features. Right, switch it On. See image below.

Meta_7
Meta_7

2. Connect the headset#

  • Plug your Link cable into a USB 3.0 (or USB-C) port on your PC, then plug the other end (USB-C) into your headset.

  • Put on your headset and take the controllers:

    1. Press imgMeta button on your (right) controller to open the universal menu.

    2. Select the clock on the left side of the universal menu to open Quick settings.

    3. Select img Link, then select the computer your headset is plugged into.

    4. Select Launch.

  • You can return in the universal menu and select icon_settings Settings in the top right at any time to toggle next to Link to enable or disable Link.

3. Enable hand tracking#
  • Enable hand tracking feature:
    1. Press imgMeta button on your controller to open the universal menu.
    2. Select the clock on the left side of the universal menu to open Quick settings.
    3. Select icon_settings Settings in the top right, then scroll down and select icon_tracking Movement tracking.
    4. Select the toggle next to Hand and body Tracking to turn this feature on (off to desable).
  • Put down your controllers (you can pick them up again at any time, see info below).

To continue with XR Twin#

1. Setup XR Twin for Quest#

  1. Launch XR Twin software on your PC.

  2. Select and open the project you want to explore.

  3. In the Working Tree, click on the OpenIcon User popup button.

  4. In the Add User popup:

    • Ensure the display device is set to VR.

    • Click on silhouette's hand and select Hands instead of VR Controllers to enable hand tracking.

    • Click AddUserToSceneButton Add user to scene button.

Users0

Users HandTracking_hand

2. Start VR Simulation#
  • Click on StartButton Start button to begin the VR simulation in XR Twin.

Congratulations, you are ready to use your hands to manipulate physicalized objects with XR Twin!

Info

To move in the scene or interact with the VR menu, take your controllers back for a moment and use them as usual for a teleport (click with a joystick), to open (A or X button) and interact with the VR menu, or using an other tool (its easier with a controller).

Leap Motion for XR Twin#

In this example, we use the Leap Motion Controller and the VIVE Pro 2 headset to demonstrate the setup for optimal hand tracking performance in XR Twin.

leapMotion_1

Setup#

To ensure optimal performance and accurate hand tracking in XR Twin using the Leap Motion Controller, follow these steps:

1. Install Ultraleap driver#
  • Download and install the recommended Ultraleap Gemini (or later) software from the Ultraleap website (account needed).
2. Connect your devices#
  • Plug the Leap Motion Controller via USB.
  • Connect your VR headset to your PC as well. Ensure your VR headset is properly connected and configured to work with XR Twin.
  • Fix the Leap Motion Controller in front of your headset. See image above.
3. Configure SteamVR settings#

Ensure you have Steam VR version 2.5.4 or higher installed on your PC.

  1. Click the SteamVR SteamVR_15 burger button to open the menu.
  2. Select Settings and navigate to the OpenXR panel.
  3. Top right, SteamVR must be the the Current OpenXR Runtime. See image below.
  4. Click MANAGE OPENXR API LAYERS button.
  5. Toggle On the "Ultraleap OpenXR Hand Tracking" API layer. See image below.
  6. Disable Off all other OpenXR runtimes to ensure that only the Ultraleap one is active.

SteamVR_5
SteamVR_5

4. Setup Ultraleap Control Panel#
  • Open the Ultraleap Control Panel.

  • Verify that the Leap Motion Controller is functioning correctly and tracking hand movements accurately.

  • Ensure the Tracking>Tracking Mode is set to Head Mounted.

  • Ensure the Settings>Settings>OpenXR Support is activated (toggle is green).

leapMotion_3

leapMotion_4

To continue with XR Twin#

1. Setup XR Twin for Leapmotion#
  • Launch XR Twin software on your PC.
  • Select and open the project you want to explore with Leap Motion.
  • From the Working tree, right the User, click on the OpenIcon Open popup button to open the Add User pop-up.
  • Ensure the display device is set to VR mode.
  • Click on silhouette's hands and select Hands instead of Controllers for hand tracking device.
  • Click AddUserToSceneButton Add user to scene button.

Users0

Users HandTracking_hand

2. Start VR Simulation#
  • Click on StartButton Start button to begin the simulation in XR Twin.

Congratulations, you are ready to use your Leap Motion with XR Twin!

Compatibility: All VR headsets are compatible with the Leap Motion Controller.

Manus Metagloves for XR Twin#

In this example, we set up Manus Quantum Mocap Metagloves equipped with VIVE Trackers 3.0 together with the VIVE Pro 2 headset to enjoy precise hand tracking within XR Twin.

Setup#

1. Requirements#

Ensure you have this hardware:

  • A complete VIVE tracking system: 2 or more VIVE Base Stations, VIVE Link Box, 2 VIVE Controllers, the VIVE Pro or VIVE Pro 2 headset.
    vive__19
  • A Manus set: 2 Quantum Mocap Metagloves, 1 Manus dongle, 2 adapters for trackers.
    manus_7manus_5
  • A VIVE Tracker set: 2 VIVE Trackers 3.0, 2 VIVE dongles.
    vive__29vive__29vive__14vive__14
  • A USB hub if you're short of USB ports on your PC.
    vive__30

Ensure you have this software installed on your PC:

2. Setup your headset#
  • VIVE tracking system (Base Stations) are powered.
  • Turn on the computer. Launch SteamVR.
  • VIVE Pro 2 headset and controllers are detected by SteamVR.

SteamVR_6

3. Setup your VIVE trackers#
  1. Plug both VIVE dongles to your PC via USB (use a USB hub if you don't have enough ports),
  2. Turn on both VIVE Trackers by pressing the power button. A solid green light indicates the trackers are on and paired. Icons are added in SteamVR window. If they aren't recognized: Navigate to Steam VR > Devices > Pair Controllers. Follow the instructions to pair VIVE Trackers.

SteamVR_14-1

4. Setup your Manus gloves#
  1. Plug the Manus Metagloves dongle into your PC via USB.
  2. Insert a charged battery into both gloves.
  3. Turn on both gloves using the power button and verify they are connected. A solid blue light without blinking indicates a successful connection.
  4. Launch Manus Core. Manus Dashboard is launched automatically (see image below).
  5. Click gear icon to open the Settings. Scroll to Tracking Systems and check OpenVR (SteamVR) checkbox.
    manus_10
  6. If prompted with "Do you want to calibrate?" (or click Calibrate at the top right of the screen), select both. Follow the on-screen instructions to complete the calibration process. If both does not appear, restart Manus Core and the devices.

manus_8

MANUS Core 2.5.1 Dashboard. Two Metagloves and two trackers are detected and associated.

5. Configure Steam VR Settings#
  1. Click the SteamVR SteamVR_15 burger button to open the menu.
  2. Select Settings and navigate to the OpenXR panel.
  3. Top right, SteamVR must be the Current OpenXR Runtime (see image below).
  4. Click MANAGE OPENXR API LAYERS button.
  5. Toggle On the OpenXR Manus Hand tracking API layer API layer. See image below.
  6. Disable Off all other OpenXR runtimes to ensure that only the Manus one is active.

SteamVR_5
SteamVR_5

To continue with XR Twin#

1. Setup XR Twin for Manus hand tracking#
  • Open XR Twin and open the project you want to explore with Manus gloves,
  • From the Working Tree, next the User, click the open icon OpenIcon,
  • In the Add User popup,
    • Ensure the User is set to VR mode,
    • Click on silhouette's hands and select Hands instead of Controllers for hand tracking,
    • Click AddUserToSceneButton Add user to scene button.

Users0

Users HandTracking_hand

2. Start VR Simulation#
  • Click on StartButton Start button to begin the VR simulation in XR Twin.
  • Move your hands.

Congratulations, you are ready to use your Manus Quantum Mocap Metagloves to manipulate physicalized objects with XR Twin!

Navigation and VR menu

When you are in an XR Twin VR simulation and you want to teleport/move your avatar in space or interact with the VR menu, take your controllers back for a moment and use them as usual to: teleport (click pad forward), walk (push pad forward/backward) open (Menu button) and interact with the VR menu, or use a tool (its easier with a controller).

Compatibility

Only HTC VIVE headsets PRO series are fully compatible with Manus gloves and VIVE trackers, using precise infrared laser based positioning.

Only HTC VIVE headsets Focus series are fully compatible with Manus gloves and VIVE Ultimate Trackers, using optical cameras and environment scanned maps positioning.