Systematic testing: Difference between revisions
Jump to navigation
Jump to search
See Friends in testing for detailed instructions on how to prepare for exploratory or systematic testing.
mNo edit summary |
|||
| (28 intermediate revisions by 5 users not shown) | |||
| Line 1: | Line 1: | ||
{{developers}} |
<noinclude>{{developers}} |
||
{{ |
{{testers}} |
||
<center>See [[Friends in testing]] for detailed instructions on how to prepare for [[exploratory testing|exploratory]] or systematic testing.</center> |
|||
</noinclude> |
|||
== Systematic Testing == |
|||
In addition to our freeform [[exploratory testing]] effort, we also engage in [[systematic testing]] efforts designed to achieve the test coverage necessary to make release decisions around [[future releases]], much like [[release notes|past releases]]. |
|||
== Purpose == |
|||
Systematic testing consists of several basic tasks: |
|||
The purpose of this systematic testing effort is to achieve [[Test cases 8.2.0|test coverage]] sufficient to make good ship/no-ship decisions about and to write accurate and timely release notes for the [[8.2.0]] release. |
|||
* Reviewing a '''[[Test_cases_8.2.1|test plan]]'''. |
|||
== Battle Plan == |
|||
* Writing '''[[Test cases 8.2.1#Create New Test Cases|the test cases]]''', if needed. |
|||
* Executing the '''[[Test_cases_8.2.1#Test_cases|test cases]]''' and recording test results. |
|||
** ''To record results on a particular test case page,'' click on the '''"Edit with form"''' button in the top of the page, and use the '''"Add Another" button''' just above the '''"Add New Results" section''' (i.e. at the bottom of the page). |
|||
* Contributing patches to the '''[[Tinderbox]]''' or '''[[Sugarbot]]''' automated testing platforms. |
|||
=== Test 1: Multiple Key Support === |
|||
# Convert one of our lists of features into a priority-tree of of "we-must", "we-hope", and "we-wish" test cases. |
|||
#* [[OLPC 8.2.0 Features|Michael's original list of features]] |
|||
#* [[TestPlan_8.2.0|Joe's ordered list of features]] |
|||
#* [[8.2.0#Possible_Release_Criteria_Wish_List|Greg's release criteria wishlist]] |
|||
# '''[[Test cases 8.2.0#Create New Test Cases|Write]]''' the "must"-priority test cases and finalize the exact procedures for recording test results. |
|||
#* ''When documented, the procedures for recording test results should be cited above.'' |
|||
# Modify our call for volunteer test labor to request help writing and executing our test cases. |
|||
# Draft [[Profiles|other people]] as needed in order to reach our desired confidence in our information. |
|||
# When feasible, contribute patches to the [[Tinderbox]]! |
|||
We are particularly interested, this week, in systematically testing [[Firmware|OFW]]'s new [[Firmware_security#Multiple-Key_Support|multiple key support]]. |
|||
== Bibliography == |
|||
Never having performed a large-scale systematic software test myself, I went looking for reference works and found: |
|||
* ISO/IEC 12207.0, |
|||
* IEEE STD 829-1998, |
|||
* MIL-STD-498, and |
|||
* ISO/IEC TR 19759:2005(E) (which has a nice bibliography) |
|||
[[Category:Testing]] |
|||
Latest revision as of 15:58, 10 December 2010
For Developers
For Testers
Systematic Testing
In addition to our freeform exploratory testing effort, we also engage in systematic testing efforts designed to achieve the test coverage necessary to make release decisions around future releases, much like past releases.
Systematic testing consists of several basic tasks:
- Reviewing a test plan.
- Writing the test cases, if needed.
- Executing the test cases and recording test results.
- To record results on a particular test case page, click on the "Edit with form" button in the top of the page, and use the "Add Another" button just above the "Add New Results" section (i.e. at the bottom of the page).
- Contributing patches to the Tinderbox or Sugarbot automated testing platforms.
Test 1: Multiple Key Support
We are particularly interested, this week, in systematically testing OFW's new multiple key support.