Test manager: Difference between revisions
m (New page: == Role == The role of a test manager is to handle QA and testing for the release they are responsible for. A test manager is responsible for being able to give the release manager answe...) |
mNo edit summary |
||
| Line 1: | Line 1: | ||
== |
== Responsibilities == |
||
The role of a test manager is to handle QA and testing for the release they are responsible for. |
The role of a test manager is to handle QA and testing for the release they are responsible for. |
||
| Line 7: | Line 7: | ||
* '''Are we done yet?''' How close are we to the "finish line" for the release? (Often this will be phrased as "What percentage of blocking tickets/bugs and regressions have been verified closed in the release candidate?") |
* '''Are we done yet?''' How close are we to the "finish line" for the release? (Often this will be phrased as "What percentage of blocking tickets/bugs and regressions have been verified closed in the release candidate?") |
||
* '''What are the risks?''' If we were to release our current build today, what do we know (confirmed open bugs) and what do we know we don't know (areas of potential regression/risk that haven't been fully explored)? In particular, have there been any regressions? |
* '''What are the risks?''' If we were to release our current build today, what do we know (confirmed open bugs) and what do we know we don't know (areas of potential regression/risk that haven't been fully explored)? In particular, have there been any regressions? |
||
== On transparency == |
|||
One strategy that a test manager can employ for getting a build well-tested is to encourage community participation. ''This is not the only release testing strategy.'' Given sufficient internal resources for thorough testing and the need to keep tight, localized control over QA for a particular item, it may make perfect sense to keep the workings of QA private for an organization - this is what the majority of technology companies do. |
|||
In order to do so, testing for that release needs to be as public as possible. (This also has the nice side effect of documenting the test management process so that the next person in that role can more ably faciliate testing for the next release.) |
|||
* The test plan for a release should be on the public wiki and available for open comment/editing and results reporting. For an example, see [[Test cases 8.2.1]]. |
|||
* Test plans and test cases should be written so that anyone can "just sit down and run them," whenever possible. They shoul |
|||
* Discussions on tests, announcements of new builds to test, etc. should happen on [http://lists.laptop.org/listinfo/testing the testing mailing list]. |
|||
* The IRC channel used by testers is currently #olpc on irc.freenode.net (with #olpc-meeting for weekly meetings), but we may create #olpc-qa if traffic grows unmanageable. |
|||
'''The only exception to this is security bugs.''' The test manager should encourage people to report security bugs privately to them and the release manager, and discussion about these bugs should be held via private email and on internal wikis until the bug is patched - at which point in time the conversations surrounding them can (and should) be made public. |
|||
== Test managers == |
== Test managers == |
||
Revision as of 16:16, 24 December 2008
Responsibilities
The role of a test manager is to handle QA and testing for the release they are responsible for.
A test manager is responsible for being able to give the release manager answers, given the input of a build to test and a set of "finish line" criteria for what the release should look like when it's done:
- Are we done yet? How close are we to the "finish line" for the release? (Often this will be phrased as "What percentage of blocking tickets/bugs and regressions have been verified closed in the release candidate?")
- What are the risks? If we were to release our current build today, what do we know (confirmed open bugs) and what do we know we don't know (areas of potential regression/risk that haven't been fully explored)? In particular, have there been any regressions?
On transparency
One strategy that a test manager can employ for getting a build well-tested is to encourage community participation. This is not the only release testing strategy. Given sufficient internal resources for thorough testing and the need to keep tight, localized control over QA for a particular item, it may make perfect sense to keep the workings of QA private for an organization - this is what the majority of technology companies do.
In order to do so, testing for that release needs to be as public as possible. (This also has the nice side effect of documenting the test management process so that the next person in that role can more ably faciliate testing for the next release.)
- The test plan for a release should be on the public wiki and available for open comment/editing and results reporting. For an example, see Test cases 8.2.1.
- Test plans and test cases should be written so that anyone can "just sit down and run them," whenever possible. They shoul
- Discussions on tests, announcements of new builds to test, etc. should happen on the testing mailing list.
- The IRC channel used by testers is currently #olpc on irc.freenode.net (with #olpc-meeting for weekly meetings), but we may create #olpc-qa if traffic grows unmanageable.
The only exception to this is security bugs. The test manager should encourage people to report security bugs privately to them and the release manager, and discussion about these bugs should be held via private email and on internal wikis until the bug is patched - at which point in time the conversations surrounding them can (and should) be made public.
Test managers
So far, test managers have been OLPC staff internally appointed to the position.
Present
- 9.1.0 - Joe Feinstein
- 8.2.1 - Mel Chua, in tandem with Brian Pepple who is running Friends in Testing.
Past
- 8.2.0 - Joe Feinstein