DART
Overview
Research Goals
Can we connect physiological sensors to rapid game-development software programs?
Can we dynamically update the environment or tasks based on the user state?
Task
Participants must perform cognitive memory tasks while driving. Tasks allowed for controlled and applied research which included:
Experimental Control: N-back
Applied Mission: Respond to system alerts and radio calls
Test
Researchers and developers changed the difficulty of the course and tasks based on driving, task, and physio performance.
For example, roads become more or less curvy or provides multi-modal cueing to assist user.
This provided researchers with an opportunity to explore adaptation approaches to integrate into training.
Result
The developers and team are better able to:
Connect with and log physio data with sim events
Allow for time-series data analysis syncing events to cognitive workload and stress data
Support sensors like Polar H10 and NIRsense.
Demonstration
The Driving-based Adaptive Research Testbed (DART) allows for live connection with various physiological sensors including heart rate and fNIRS. DART supports a variety of driving scenarios on straight and curvy roads. The straight road mode can be used as a physiological baseline for comparison. Auditory N-Backs integrated in DART act as secondary cognitive tasks. During each test, the users must maintain their driving speed and control of the vehicle.
Roads adapt in difficulty based on performance and physiological metrics. Users respond to cues in the environment by responding using a paddle on the wheel, which are logged with physiological data for performance analyses. Other cognitive tasks include cue discrimination tasks. Participants respond to mission-relevant cues using the paddles. As an adaptive training support, visual indicators are added or removed based on performance and physiological metrics. Survey metrics can also be integrated in the absence of sensor data such as the NASA-TLX after each trial provides subjective feedback for comparison. DART can also be used to explore adaptation strategies to operational relevant tasking such as responding to their call sign when radio commands are received. Users must execute the specific command given to their callsign by responding using buttons on the wheel. DART serves as a research testbed for game-based adaptive training to explore adaptive strategy, sensor effectiveness, and effective inputs for adaptation logic.
Using the DART simulator, the GRILL recently conducted a research study investigating the effects of adaptive training during auditory n-backs as well as applied system cue and radio communication missions while driving. Participants (N=28) wore sensors measuring their neural activity (functional near infrared spectroscopy, fNIRS) and heart rate (electrocardiogram, ECG).
We then used these physiological signals, in addition to their behavioral responses, to develop a rule-based augmentation protocol that adapted task difficulty in real time, based upon auditory task performance, driving task performance, and physiological stress states, in an effort to facilitate and enhance their cognitive performance. Preliminary analyses show differences in neural activity between the easy (green) and hard (red) difficulty levels during the driving simulation task, including a 20% decrease in signal variability (standard error) after the first level of adaptation for the hard trials-indicating a promising ability to place trainees into appropriate levels of difficulty based on performance and cognitive state.
This design could lead to flow-based training which can consistently challenge trainees, expedite training timelines, and improve training.
Publications
Rebensky, S., Stalker, W., Knight, R., & Perry, S. (in development). Using physiological and performance metrics for adaptive game-based simulation: Approaches for military research.
Rebensky, S., Perry, S., & Bennett, W. (2022). How, when, and what to adapt: Effective adaptive training through game-based development technology. 2022 Interservice/Industry Training, Simulation, and Education Conference. Education Subcommittee Best Paper Award.