With a design environment in place, the next issue becomes that of deploying a solution. The fielded strategy is, after all, the final product. The quality of the strategy is paramount, since logic errors or other shortcomings can cause enormous impacts on maintenance costs. As such, quality control over the strategy is a key concern that is typically addressed through fault injection. Test strategy fault injection can occur by using any of the following sources of information:- Conditions of Test Success and Failure
- Actual Hardware / Testing Resources
- Math Models
Because using either the actual hardware or a math model prohibits validation during early phases, knowing the conditions that provide a pass or fail response to each test is the only way to validate tests at a point in the process where design decisions can still be influenced. Test conclusion information can be represented or obtained in the following ways:
- Manual
- Spreadsheets
- Diagnostic Models
Manual response to tests is clearly unacceptable because it is not repeatable, nor is the information captured for future engineers. Using spreadsheets is at least electronic, but is in a poor form. Spreadsheets invariably become unwieldly and difficult to manipulate for non-trival designs, partly due to their non-hierarchical approach. This leaves us with Diagnostic Models as the richest source of information from which to drive test strategy validation.
In addition to immediate benefits, the coupling of a diagnostic model with test strategy validation provides many long-term benefits. Furthermore, as new or legacy information is brought into such an environment, there are many additional validation benefits that occur as part of the process instead of remaining hidden until later test strategy validation. The benefits of the modeling environment include:
- Identification of Hidden Failure Sources
- Analysis of Existing Test Coverage
- Identification of Existing Logic Errors
Since the model allows a faster turn-around, the validation of the test strategy can be done more often and to a higher resolution. In fact, since the design provides early validation of the test strategy, it helps remove any ambiguity between the hardware and the test strategy. The following benefits can be realized through use of the design environment:
- Maintain single source of Strategy Logic
- Document Assumptions for Future Engineers
- Reduce Cost of Future Design Changes
- Reverification of Baseline Strategy Performance