Tech & Innovation in Healthcare

Learn how bridge platforms convince you to feel virtual objects.

Physicians can use virtual reality (VR) to train for complicated surgeries, but something has been missing from the experience — the sense of touch. Innovators are developing haptic gloves and tools that fully immerse the physician in the operating room (OR) when they put on the VR headset.

Grasp how haptics technology is creating a more complete VR experience in healthcare.

Trick the Brain With the Sense of Touch

VR use by healthcare professionals has grown in popularity over recent years, as the technology delivers vivid, high-definition images with minimal lag time. The striking visuals help immerse you in the virtual environment, but you can’t physically feel anything — up until a couple of years ago when developers discovered a way.

Several companies have developed haptic feedback devices that you can wear or hold during a VR session. The haptic devices help stimulate your touch and motion senses to fully engage you in the environment. “I call them bridge platforms because they bridge the human senses and the VR for the digital world,” says Rafael Grossmann, MD, FACS, general and trauma surgeon at Portsmouth Regional Hospital in Portsmouth, New Hampshire.

Essentially, instead of holding the VR system’s controllers, you slip on a pair of gloves or grab a 3D-printed tool replica that communicates with the VR software. In turn, you’ll feel like you’re holding a scalpel, forceps, or scissors in the virtual world when you may not be holding anything at all in the real world.

Understand How You Can Feel Objects in VR

When you put on a VR headset, you can step into the OR to perform a delicate or complicated surgical procedure without having to scrub in. As you pick up your surgical tools in the VR session, how do the haptic devices deliver a realistic feeling? That realistic feeling is a combination of the device’s mechanical components and your mind playing a trick on you. “The brain is tricked very quickly to feel that you’re seeing just your hands and you’re feeling objects in VR,” Dr. Grossmann says.

Some of the mechanical components of haptic gloves that provide a realistic feel include:

• Tactile actuators: Create the feeling of clicking buttons, vibrations, and impacts
• Motion-tracking sensors: Convey an object’s size and shape in the virtual space
• Vibrotactile feedback: Allows you to feel different surface textures

One company that has developed haptic gloves is SenseGlove. SenseGlove’s model uses positional data of the user’s hand and fingers to create an accurate representation of the user’s hand in the virtual space. When the user grabs an object in the VR simulation, the technology registers contact between the user’s virtual hand and the virtual object. Then, haptic actuators activate in the glove to match the interaction.

The gloves feature force feedback cables that run from the hand to the fingertips. “Whenever we detect contact between the virtual hand and an object, we can apply forces to the cable to apply pressure on the finger. Because this pressure matches up exactly with the interaction of the user, they perceive they are grabbing the virtual object for real,” says Bryan Zaaijer, creative director for SenseGlove in Delft, the Netherlands.

If a physician uses the gloves, they’ll feel the tools and objects they pick up virtually in their real-world hands. However, since the force feedback of the gloves is limited to the fingers, the gloves cannot simulate the pressure exerted on the virtual tools, but the vibrotactile actuators will activate with certain events, such as making an incision or puncturing the skin. “If well implemented in the simulation, this can provide an intuitive and natural interaction,” Zaaijer says.

Another company that has developed haptic gloves is HaptX. Their device is powered by a backpack-worn compressed air system that activates the actuators. With a backpack, hoses, and gloves, the system may feel bulky. However, despite the multiple real-world components, “the experience is amazing. You can feel things even with a fine touch in virtual reality,” says Dr. Grossmann.

Perform Delicate Procedures in a Virtual Space

Haptic gloves aren’t the only way for you to enhance your VR healthcare experience. You can also hold tool replicas in the real world and see their movements transferred to the virtual space. FundamentalVR is a company that has developed a platform for training purposes using 3D-printed tools. With FundamentalVR’s platform, you insert a 3D-printed tool into an interface and then put on the VR headset. As you move the tool and apply pressure with it in the real world, you can see the same movements in the virtual environment.

Several specialties, including neurosurgery, orthopedic surgery, and ophthalmology, are using this modality to train their surgeons for complicated procedures. For example, ophthalmologists can recreate the view from a microscope, so they can safely and efficiently practice an operation in a patient’s cornea, iris, or lens with the VR headset and 3D-printed tools. “By torquing the movement and applying different pressures, you can actually feel the haptic sensation working with the microscope and tissue as so delicate as the eye,” Dr. Grossmann says.

Train the Next Generation of Surgeons

Instead of practicing procedures on plastic boxes, mannequins, or cadavers, will the next generation’s surgeons put on a VR headset and haptic gloves? While Dr. Grossmann thinks VR training won’t fully substitute for surgery in healthcare, he does think VR can improve the training experience rather than just using the traditional methods.

“You can certainly train for surgery … and that’s something that, with the power of technology and connectivity, you can do the same thing in a high-end simulation. You can do that at home with the headset, computer, or laptop, and you can have many people working on the same model, practicing, guiding, mentoring, and learning or teaching in a virtual environment,” Dr. Grossmann says.

When you add in the haptic gloves and the fact that you can feel yourself performing the procedure accurately via the bridge platform, “It’s almost like being teleported to the actual training site. You can go without geography being a barrier,” he adds.