The relationship between the cost and usability of speech recognition, often these are the factors that drive the reluctance to use speech recognition for simulation training. Furthermore, usability and expense are linked when deciding the acquisition of speech technologies. It costs a lot of money to buy automatic speech recognition (ASR). Will it work for my situation?

Is it Worth the Expense?

Will speech recognition work is a fundamental question. I can use this question first to address the issue of expense. If the ASR works as intended, is the acquisition worth the cost? One of the main reasons to use speech recognition is to save on the expense of role players. If I can reduce the number of staff and the associated costs, how much will that save?

Understanding my annual savings, how many years will it take to recoup the ASR investment? Is this a price I am willing to pay?

Of course, there are other reasons for adopting speech in the training program. The additional benefits should also be part of the decision process. For example, added availability of simulators and other training resources. it results in easier planning of support resources. ASR enables self-paced learning for parts of the course.

The organization can eliminate the need to find and train pseudo pilots. And, ASR provides consistency of simulator behavior for all students. The calculation of tangible financial metrics is less precise in these circumstances. Still, they will provide added value, and an estimate of that value is worthwhile for return on investment deliberations.

If, after deliberation of the cost you believe the return on investment calculation makes sense, we can address the connected issue, will it work?

Will it Work?

To be blunt, that is entirely dependent on the buyer. It requires a disciplined approach to defining the requirements for your ASR component. Unfortunately, defining requirements is complicated. It is common, if not routine, to find vendor loopholes to be spread liberally across a requirements document. If you are evaluating vendors primarily on price, they have no other option but to price as cheaply as possible. Vendors will make maximum use of loopholes to do so.

The United States Air Force in 2001 specified that; the ASR should support phrases in FAA 7110.65 and locally adopted terminology. The system shall be 98% accurate. Quite simple for a component that was so critical to the success of the program. And wholly insufficient. Put aside the meaningless accuracy statement (and for more information see Speech Recognition Accuracy Doesn’t Matter!); the terminology requirement added little value or assurance to the buyer.

Writing Requirements

When comparing vendor offerings, it is paramount that the requirements allow you to compare like for like. Locally adopted terminology is not quantifiable. Terminology support is the most significant cost driver. Consequently, if the vendor wants to win on price, they have no option but to limit the number of supported phrases while reporting compliant with qualifying statements. Given how vague this statement is, it is not unreasonable to believe the buyer does not understand the problem with its inclusion.

As mentioned in other articles, this terminology requirement, as written, led to high additional cost to the buyer. The vendor estimated the inclusion of support for 120 phrases. The system in 2020 has support for well over 100,000 phrases.

System Comparison

When buying an ASR equipped system, you must be able to compare offerings on a like for like basis and be able to empirically test the winning system against the defined requirements and vendor response.

A comparison allows a better assessment of price. Buying the cheapest system can be disastrous if you find the winner has done a better job of exploiting loopholes.

System Testing

Testing ensures that you are getting what you thought you were getting. As a result, testing simplifies the process of assigning blame (and cost) for any shortfalls in system capability.

Preparation

Success in the comparison and testing of a system requires preparation. It will be hard work to write requirements that protect you as a vendor. The outcome of the weak definition of system requirements is the failure of project execution and unusable technology.

So, Speech Recognition in Simulation – Is it Worth the Expense?

Speech recognition is expensive. The expense can be justified, and if the system works as expected, it can quickly pay for itself. The onus for success initially lies with the buyer. If you plan to buy on price, you may want to consider changing that to value. Using value as the deciding criteria means that you get the features you want and need for the best price. You cannot do that if your requirements do not permit the comparison of systems and the methodological testing of what you bought.

So what is the answer to the question, will it work? It depends on the answer to the questions. Have you done sufficient work on your documentation? And do you understand how to select and test a system?

If you do not know where to start or if you wish a little more guidance, check out  Writing Requirements for Speech Recognition.