ASR for Training and Simulation

Speech recognition has been in use in training and simulation for over three decades, but it was not until 2001 that the application of ASR in the training domain saw its first widespread success. It was at this time that the technology had reached sufficient maturity that it was a mandatory requirement for The United States Air Force contract, for the purchase of up to one hundred air traffic controller training simulators.

An air traffic control training simulator generates simulated aircraft that are displayed on various devices such as radar screens and for an airport simulator, as seen in the picture below,  they are displayed on an out of the window visual scene. Student air traffic controllers issue instructions to the aircraft on the appropriate radio frequency and traditionally, the control instructions are translated into computer inputs by role-players, which for ATC simulation are called pseudo pilots. The computer inputs are in turn used to direct the actions of the simulated aircraft. An example of an ATC simulator in use can be seen here.

Finding the personnel to serve as pseudo pilots can be a challenge to a large training organization and is accompanied in many situations by considerable expense. The USAF determined that as a result of supporting numerous overseas deployments, they would be unable to utilize human pseudo pilots and such an approach, would not be viable for the widespread distribution of simulation systems. They declared that the winning system must include a speech recognition capability that would automate the pseudo-pilot role.

There are numerous challenges to be overcome when introducing speech recognition to human in the loop simulation, but if you overcome those challenges, the benefits to the training providers and the trainees are clear. In the next article, we will discuss those challenges.