The test plan defines the testing necessary to establish quality for the system. If the system passes all tests in the test plan, then it is declared to be complete. If the system does pass all test then it is considered to be of high quality. The more complete the coverage of the system, the higher is the confidence in the system: hence the system's quality rises.
The test plan could be considered as part of the design, which is the position taken here, or it could be considered as the first accomplishment in the testing phase. One of the goals of the design phase, is to establish a plan to complete the system, thus it is very natural to include the test plan. Also the trade-offs between alternative architectures can be influenced by differences in their test plans.
One single test will exercise only a portion of the system. The coverage of the test is the percentage of the system exercised by the test. The coverage of a suite of tests is the union of the coverage of each individual test in the suite.
Ideally, 100 percent test coverage of the entire system would be nice, but this is seldom achieved. Creating a test suite that covers 90 percent of the entire system is usually simple. Getting the last 10 percent requires significant amount of development time.
For an example, consider the Basic Input/Output System (BIOS) built by IBM in the early 1980s as the foundation of the Disk Operating System (DOS) built by Microsoft. For performance reasons, the BIOS needed to be placed in a Read Only Memory (ROM) chip. Because the BIOS would be placed on a ROM, error patches would be nearly impossible. Thus 100% test coverage of the BIOS was dictated. The BIOS code itself was small, only a few thousand lines. Because the BIOS is asynchronous in nature, creating a test would first require an asynchronous environment to bring the system to a desired state, and then an event would be needed to trigger a single test. Quickly, the test suite grew much larger than the BIOS. This introduced the problem of doing quality assurance on the test suite itself. Eventually, 100% coverage was reached but at a high cost. A more cost effective approach would be to place the BIOS on a Electronic Programmable ROM (EPROM) and ROM combination. Most of the BIOS would be on the ROM with error patches being placed on the EPROM. This is the approach that Apple took on the Macintosh.
Usually, it is sufficient that the test suite includes all of the typical and atypical scenarios and need not cover the entire system. This gives reasonable quality for the investment of resources. All the typical and atypical scenarios need to be covered, but in doing so, not all threads of execution within the system may be covered. The system may contain internal branches, errors, or interrupts that will lead to untested threads of execution. Tools exists to measure code coverage.
Systems are full of undiscovered bugs. The customer becomes a logical member of the testing team and bug fixes are pushed off to the next release.