AR / VR

Overview

Virtual and Augmented Reality is the new and upcoming technology that has nearly revolutionized the way gaming and simulations are conducted. Virtual reality allows for an immersive experience where the user can fully invest their attention into the virtual world. Augmented reality, provides the user the ability to overlay information, holograms, and data into the real world which can only be seen through an augmented reality headset or application. Each reality provides the user with an experience that can be taken anywhere or changed to fit their needs. Virtual and Augmented reality is still relatively new technology that is rapidly developing. At the GRILL, we strive to push excellence using this up and coming technology by creating software that can utilize that technology. To see some of the things we have done to live up to that statement, view our projects page.

Hardware

Headsets

The virtual reality headsets that are the most widespread are the headset produced by Oculus and HTC. The most recent headsets from Oculus are the Rift S and the Oculus Quest. HTC’s most recent is the Vive Pro and the Vive Cosmos. The most notable augmented reality headset is the Microsoft Hololens. The Rift S and Vive Pro are both headsets that must be connected to a computer. The Oculus Quest, Rift S, and Vive Cosmos all offer inside out tracking. This means that the headset is tracked through cameras located on the headset that track the user relative to the surrounding environment. The Microsoft Hololens also implements inside out tracking to tracks the user in the environment and how they interact with the applications. The Vive Pro uses outside-in tracking which means there are sensors around the player to track the headset.

Gloves

Haptic feedback is an assisting force behind virtual and augmented reality. It gives the user touch feedback to support the visual feedback, so the user physically feels the virtual object they’re interacting with. Haptic feedback gives the user more to process to make the simulations more immersive. The Haptx gloves provide an extra layer to the brain that the headsets can’t provide alone. Combining the gloves with the headset could be the next step in realism that would improve simulations beyond what they currently are.

Integration

Other hardware may be needed for different projects. One piece of hardware that the GRILL has had to use is the ZED camera. The ZED camera was attached to a VR headset that allowed the user to see the real world as well as the virtual world. Other possible integration includes a different controller for the user, meaning that the user would be using some other form of technology outside of the intended controllers.

Software

Before the hardware can be used for training, the software must be completed. There are many available game engines and development environments to create virtual and augmented reality projects. The game engines primarily used at the GRILL are Unity and Unreal Engine 4. Both engines have built-in support for virtual and augmented reality. Virtual and Augmented reality simulations can be made for either headsets or mobile devices. In most cases, starting in virtual reality is a good place to begin development of an application since virtual reality applications have the potential to be converted directly to augmented reality applications.

Here are a few video examples of an AR project and a VR project.

AR Project

VR Project

Developer’s Guide

New developers would benefit from first familiarizing themselves with a gaming engine or an app development IDE (Android Studio or iOS) for the platform of choice. Links to some tutorials have been provided below. Some further choices for development environments include Godot and CryEngine for AR and VR development. Android Studio, google ARCore, and Vuforia are good platforms for AR development. After the tutorials, integrating virtual or augmented reality may be simple depending on the platform. For Unreal Engine there is built in support for AR / VR. Unity has AR/ VR packages that can be integrated. Any further information besides basic learning of the development platform can be found online.

New developers would benefit from first familiarizing themselves with a gaming engine or an app development IDE (Android Studio or iOS) for the platform of choice. Links to some tutorials have been provided below. Once familiar with the engine, the next step would be integrating AR / VR into the project. Some engines make it as simple as turning on a setting or starting a project that already integrates virtual reality. Others may require a little more work. Specific aspects of projects can be found through online searches.

Resources

Unreal Engine

Unity

Android Studio

iOS