For non-executable deliverables, the intent of this task is to conduct appropriate reviews. For executable deliverables, the intent of this task is to design and execute tests with the intend of uncovering errors. Subtasks for executable deliverables are considered below.
III.20.1 Revise existing test plan and procedure to accommodate the enhancement.
Intent: The intent of this task is to revise the plan for unit, integration and validation testing with an emphasis on software that has been modified for the enhancement.
Mechanics: The test plan and procedure, created in Task II.12.1, is revised to reflect the modified design and source code for the enhancement. A revised testing strategy is developed.
Application of Formal Methods: none
Application of CASE Tools: t.b.d
SQA Checklist:
1. Does the revised test plan focus on software modified to implement the enhancement?
2. Have existing test cases been reviewed for use as possible regression tests?
3. Does the revised test plan define an adequate number of milestones that will enable the testing activity to be tracked?
4. Has time been scheduled for debugging the errors that are uncovered during testing?
5. Has provision been made for change control as the software is modified during testing?
6. Are distinct test case design methods to be used to generate test cases as part of the test strategy?
7. Have provisions been made for recording metrics on errors?
Do's & Don'ts
Do: Take the project schedule into account. There is no point in defining a testing strategy that call for the integration of a module that has not yet been designed.
Do: Conduct regression tests.
Don't: Assume that every module that "enters" the test strategy will be perfect. Be sure to define a unit testing approach as well as integration and validation.
Deliverables: Test Plan and Strategy
III.20.2 Design white-box and black-box tests with product enhancement refinements.
Intent: The intent of this task is to design white-box test cases that will fully exercise all new program components and to design modified white box tests to exercise modified components.
Mechanics: Existing test case are examine for applicability to the enhancement and new test cases are derived.
Application of Formal Methods: white-box test case design methods
Application of CASE Tools: t.b.d.
SQA Checklist:
1. Have test cases been generated for all new program modules?
2. Will the test cases result in path coverage?
3. Have program loops been explicitly exercised?
4. Are all conditions exercised on their true and false side?
5. Have regression tests for modified components been acquired? Have they been scheduled for execution?
Do's & Don'ts
Do: Recall that white-box tests are best applied to unit level testing.
Do: Select critical modules from white-box testing, if resources are limited. Critical modules are those that are most error prone or those that address multiple requirements.
Do: Be sure that every statement in each program component is executed at least one time during testing.
Don't: Rely on black-box tests only. Certain classes of errors will go undetected!
Deliverables: white-box test cases and expected results
III.20.3 Design black-box tests for new and revised program components and for the integrated system.
Intent: The intent of this task is to design black-box test cases that will fully exercise the input and output domain of the software. Special attention should be paid to test that uncover errors due to enhancement side effects.
Mechanics: Black box test cases used for the existing application are reviewed for applicability. New test cases are added to address any new functionality or behavior attributable to the enhancement.
Application of Formal Methods: black-box test case design methods
Application of CASE Tools: t.b.d.
SQA Checklist:
1. Have test cases been organized by input class?
2. Have tests been designed to exercise the program at it boundaries?
3. Have unusual and erroneous input conditions been reflected in test case designs?
4. Have all expected results been documented?
5. Have test cases been tracked to requirements to ensure that all requirements have been tested?
Do's & Don'ts
Do: Recall that black-box tests are best applied to throughout testing and are used heavily for regression testing.
Do: Use test case design methods (e.g., equivalence partitioning) that result in the discovery of classes of errors.
Don't: Rely on black-box tests only. Certain classes of errors will go undetected. white-box tests should also be conducted.
Deliverables: black-box tests and expected results
III.20.4 Conduct white-box and black-box tests with product enhancement refinements.
Intent: The intent of this task is to execute white-box and black-box tests by following the testing strategy defined in Task III.20.1.
Mechanics: Use information in the test plan to determine the sequence of white-box and black-box tests. As errors are found, conduct necessary debugging.
Application of Formal Methods: none
Application of CASE Tools: t.b.d.
SQA Checklist:
1. Have the results of each test been saved?
2. Have all test cases been executed?
3. Have all errors uncovered been corrected?
4. Have expected results and actual results been compared for all tests?
Do's & Don'ts
Do: Keep careful records. The more information you collect on errors, the better your ability to improve the software engineering process.
Do: Be certain to archive all test cases and the results obtained from them.
Deliverables: Test results and error records
III.20.5 Conduct regression tests.
Intent: The intent of this task is to conduct regression tests in an attempt to uncover and correct any inadvertent side effects introduced during the modification activity. Regression tests are tests executed during a previous iteration that are re-executed to ensure that changes have not spawned errors
Mechanics: All previous white-box and black-box test cases are reviewed for their use as regression tests. The existing test cases that have the potential for uncovering errors due the modifications are re-executed. New results are compared to previous results.
Application of Formal Methods: none
Application of CASE Tools: t.b.d.
Do's & Don'ts
Don't: Skimp in your use of regression tests. This activity takes time, but it is one of the most effective ways of uncovering certain classes of side effects that are inherent in an enhancement activity.
Deliverables: regression tests and results
III.20.6 Prepare all product enhancement documentation.
Intent: The intent of this task is be certain that all software documentation (both technical documents for internal use and customer documentation) reflect the changes that have been made as a consequence of testing as well as the changes made to accommodate the enhancement. All documents are prepared for release to the customer.
Mechanics: The log of all modifications to the software made as a consequence of testing is reviewed to ensure that documentation reflects all changes. Documents are reviewed and modified as required.
Application of Formal Methods: none
Application of CASE Tools: t.b.d.
SQA Checklist:
1. Has the documentation been revised to reflect the enhancement?
2. Has the enhancement been traced to all levels of documentation?
3. Has documentation been revised to reflect changes made during debugging?
4. Have all errors been recorded and categorized?
5. Does the documentation accurately reflect user interact and system behavior?
Do's & Don'ts
Do: Be certain to use configuration management and control tasks (see Chapter 5, Umbrella Activities) as changes are made to software documentation.
Deliverables: Revised/updated software documentation
III.20.7 Develop a strategy for re-introduction of the software.
Intent: The intent of this task is to develop a strategy for re-introducing the software after the enhancement has been successfully implemented.
Mechanics: The strategy includes system integration testing, alpha and beta testing and special provisions for software that must be rolled-out to multiple customer sites.
Application of Formal Methods: none
Application of CASE Tools: t.b.d.
III.20.8 Create an revised Software Release Package.
Intent: The intent of this task is to construct an Software Release Package that contains all information necessary for internal support and system integration of the software.
Mechanics: All programs, documents and data are assembled and put under control. Appropriate release procedures and final SQA procedures are initiated.
Application of Formal Methods: none
Application of CASE Tools: t.b.d.
Do's & Don'ts
Do: Be certain that all documents, programs and data have been placed under configuration management and control (see Chapter 5, Umbrella Activities).
Do: "Test" the deliverables that are about to be released to the customer. Are they complete? comprehensible? useful? correct?
Don't: Assume that you or someone else on the project team will be around to answer questions after-the-fact. Be certain that the Software Release Package can stand on its own.
Helpful Hints
The Software Release Package should be defined as a checklist that your group or project team creates. Every item on the checklist should be considered as the package is assembled.
Deliverables: Software Release Package
III.20.9 Roll-out to each customer.
Intent: The intent of this task is to effect the release of the software to the user community.
Mechanics: All programs, documents and data are packaged and released to the end-user(s).
Application of Formal Methods: none
Application of CASE Tools: t.b.d.
Do's & Don'ts
Do: Be certain that you beta test the release procedure before rolling out to multiple sites.
Do: Conduct a final review of the deliverables that are about to be released to the customer. Are they complete? comprehensible? useful? correct?
Don't: Assume that installation is easy for you that it will be easy for another site.
Deliverables: Software Release Package
Use Browser "back" arrow or return to APM Process Design Language Description
