Systematic testing

  For Developers

NOTE: The contents of this page are not set in stone, and are subject to change! This page is a draft in active flux ... Please leave suggestions on the talk page.

Purpose

The purpose of this systematic testing effort is to achieve test coverage sufficient to make good ship/no-ship decisions about and to write accurate and timely release notes for the 8.2.0 release.

Battle Plan

1. Convert one of our lists of features into a priority-tree of of "we-must", "we-hope", and "we-wish" test cases.
   • Michael's original list of features
   • Joe's ordered list of features
   • Greg's release criteria wishlist
2. Write the "must"-priority test cases and finalize the exact procedures for recording test results.
   • When documented, the procedures for recording test results should be cited above.
3. Modify our call for volunteer test labor to request help writing and executing our test cases.
4. Draft other people as needed in order to reach our desired confidence in our information.
5. When feasible, contribute patches to the Tinderbox!

Bibliography

Never having performed a large-scale systematic software test myself, I went looking for reference works and found:

• ISO/IEC 12207.0,
• IEEE STD 829-1998,
• MIL-STD-498, and
• ISO/IEC TR 19759:2005(E) (which has a nice bibliography)