VR Pilot Training Now Comes With a Sense of Touch

For training pilots, the more realistic the simulator, the better. So it's about time virtual reality-based systems came with a sense of touch.

FlyInside

Aviation simulators—the most valuable training tool pilots have—have to get things right. The instrument panel. The wind and the rain. The response of the aircraft when you flip a switch or pull on the yoke. It all must be as high fidelity, as true to life, as possible. Otherwise, pilots risk uncertainty or disorientation when transferring their simulated experience to the real world.
With the rise of virtual reality-based simulation, in which users wear headsets instead of sitting in a cockpit where everything is real but the view out the windshield, the challenge of maintaining that verisimilitude has really taken off. These systems cost just a few thousand dollars, instead of the tens or hundreds of thousands you pay for a full-size cockpit mockup. They’re smaller and more portable too, a plus for clients like militaries who like the option of training pilots in remote locations.

The downside is that in today’s systems, beside the joystick, rudder pedals, and maybe a throttle lever, all the controls are digital renderings. You “activate” the switches and dials by poking and jabbing into thin air. That amplifies the challenge of VR-based training, where the nuances of touch and movement are essential to programming the pilot’s brain.

One solution—long pursued across many virtual-reality applications, from gaming and design to sex—is haptic feedback. Mechanical actuators placed in contact with different areas of the user’s body, most notably the hands and fingertips, add the sensation of touch to these computer-generated worlds. Now, a French company called Go Touch VR is putting it into action.

Working with US virtual-reality simulation software developer FlyInside, Go Touch VR has adapted its fingertip-mounted technology for aviation. The goal is to give pilots using virtual-reality flight simulators that touch-based confirmation with every switch and dial used on their flights, just as they would experience in the kind of full-sized cockpit mockups found in large, commercial multimillion-dollar motion simulators.

“You should only have to give a glance to button that you need to press during an operation, while all the rest of the action is confirmed by the touch sensation—the ‘click’ that you have from the virtual switch,” says Eric Vezzoli, Go Touch’s co-founder and CEO. “Without that fundamental confirmation, you must look back and check if the action was performed, and spend precious time and attention that you need to dedicate to flying operations.”

In Go Touch VR’s new system, derived from its engineers’ expertise in haptic feedback, the user wears three sensors on each hand, which resemble the things look like the blood-pressure sensors doctors place on your fingertips. By applying pressure to your fingertips, the actuators can replicate object stiffness, coarse textures, and the sensation of holding physical objects in your hands. The devices contain numerous actuators beneath a flexible rubber cover, and they can be individually controlled and varied in pressure to simulate light touches up through more pronounced contact. Though clunky in appearance, they’re lightweight and designed not to interfere with natural hand and finger movements. (The company is working to miniaturize them further before starting production.)

The view from the headset omits the attachments from its representation of the user’s hands, so they’re easy to forget about. All the pilot in training knows is that when she flips her finger, she can feel the switch move as well as see it. Part of the effectiveness, the company explains, comes from the user’s brain amplifying the sensors’ work by merely anticipating and recognizing the physical contact.

The trick is fine-tuning the subtle mechanical interactions—what Vezzoli calls the “cutaneous force feedback through skin indentation”—so they feel natural. “The technology reproduces the exact skin stimulation that you perceive when you are interacting with real objects,” he says. “We are concentrating in the area that you use to interact most, the fingertips. When we couple it with a visual rendering in virtual or augmented reality, you reach out your hand toward an object, activating the skin pressure, the brain ‘clicks’ and let you perceive the virtual object in front of your eyes as real, because it is feeling a sensation that it is expecting.”

The system, and has potential far beyond aviation. According to the company, the technology can improve a wide variety of VR interactions, including, for example, catching and throwing balls with greater accuracy than other control systems.

The company exhibited the product, which is still in the development kit phase, at the European defense and security conference Eurosatory in mid-June. It says pilots and engineers who tried it out affirmed its effectiveness, and that some noted the portability benefits for military personnel and others the ease of use.

In addition, the technology has potential benefits beyond aviation, including retail contexts, allowing consumers to “touch” products remotely before buying them, and manufacturing training roles, where manual skills need to be taught and practiced before being applied in the real world. Beyond that, the sky is pretty clearly the limit.
SOURCE