AR / VR

Overview

Virtual and Augmented Reality is the new and upcoming technology that has nearly revolutionized the way gaming and simulations are conducted. Virtual reality allows for an immersive experience where the user can fully invest their attention into the virtual world. Augmented reality, provides the user the ability to overlay information, holograms, and data into the real world which can only be seen through an augmented reality headset or application. Each reality provides the user with an experience that can be taken anywhere or changed to fit their needs. Virtual and Augmented reality is still relatively new technology that is rapidly developing. At the GRILL, we strive to push excellence using this up and coming technology by creating software that can utilize that technology. To see some of the things we have done to live up to that statement, view our projects page.

Hardware

Headsets

Headsets are the most common form of technology used for virtual and augmented reality. Virtual reality headsets are normally used for video games, but the GRILL team uses headsets for simulations and training programs. When headsets are connected to computers there needs to be a way to track where the user is. One way of doing this is outside in tracking where external sensors are situated around the headset to track the user’s movements. Another way of tracking the headset is through inside out tracking. Inside out tracking involves placing cameras inside of the headset to track the user’s movements based on how the environment around them moves in the camera.

Gloves

Haptic feedback is an assisting force behind virtual and augmented reality. It gives the user touch feedback to support the visual feedback, so the user physically feels the virtual object they’re interacting with. Haptic feedback gives the user more to process to make the simulations more immersive. The Haptx gloves provide an extra layer to the brain that the headsets can’t provide alone. Combining the gloves with the headset could be the next step in realism that would improve simulations beyond what they currently are.

Integration

Other hardware may be needed for different projects. One piece of hardware that the GRILL has had to use is the ZED camera. The ZED camera was attached to a VR headset that allowed the user to see the real world as well as the virtual world. Other possible integration includes a different controller for the user, meaning that the user would be using some other form of technology outside of the intended controllers.

Software

Before the hardware can be used for training, the software must be completed. There are many available game engines and development environments to create virtual and augmented reality projects. The game engines primarily used at the GRILL are Unity and Unreal Engine 4. Both engines have built-in support for virtual and augmented reality. Virtual and Augmented reality simulations can be made for either headsets or mobile devices. In most cases, starting in virtual reality is a good place to begin development of an application since virtual reality applications have the potential to be converted directly to augmented reality applications.

Here are a few video examples of an AR project and a VR project.

AR Project

VR Project

Developer’s Guide

New developers would benefit from first familiarizing themselves with a gaming engine or an app development IDE (Android Studio or iOS) for the platform of choice. Links to some tutorials have been provided below. Some further choices for development environments include Godot and CryEngine for AR and VR development. Android Studio, google ARCore, and Vuforia are good platforms for AR development. After the tutorials, integrating virtual or augmented reality may be simple depending on the platform. For Unreal Engine there is built in support for AR / VR. Unity has AR/ VR packages that can be integrated. Any further information besides basic learning of the development platform can be found online.

New developers would benefit from first familiarizing themselves with a gaming engine or an app development IDE (Android Studio or iOS) for the platform of choice. Links to some tutorials have been provided below. Once familiar with the engine, the next step would be integrating AR / VR into the project. Some engines make it as simple as turning on a setting or starting a project that already integrates virtual reality. Others may require a little more work. Specific aspects of projects can be found through online searches.

Resources

Unreal Engine

Unity

Android Studio

iOS