User:Mchua/Braindumps/learning testing

Ongoing

• Testing volunteers: learning how other large open-source projects get their testing to happen. The idea (strawman) is to look at and volunteer for a different open source project's test squad every month, and then find ways to bring best-practices back to OLPC.
• Testing Tools: an exploration of tools that seem to be used a lot in testing, that we already use in testing, etc. and how to use, configure/modify, and (most importantly) get others to consistently adopt them. Pick a tool or type of tool each month to learn about and (if we deem it useful) attempt to adopt it on an experimental basis; if it works, we'll se how we can spread it out.
• New words and concepts: Keep a log of terms I haven't heard before (hey, "waterfall" used to only refer to actual bodies of falling H2O for me). This will be useful to new testers in our community who may come from nontech backgrounds and are unfamiliar with the terms.
• Feedback loops: Everything we are doing should get a quick sanity check on who it's benefiting - the test team? the devels? our product manager? our volunteer dev and/or test community? And then actually make sure those people benefit; ask them what they want and how the current fix is doing.
• Testing books: One book per month.

Suggested resources

Groups to learn from

• http://fedoraproject.org (coordinating volunteer efforts with commercial releases)
• http://ubuntu.org (dealing with upstream)
• http://gnome.org (bug triage days, getting newbies started)
• http://openplans.org (session-based exploratory testing)
• http://mozilla.org (automated testing)

Toolsets to look at

• Trac
• tinderbox
• buildbot
• sugarbot

Topics to cover

• exploratory testing
• making metrics for how "good" your testing is

October

• Testing volunteers: Fedora. It is enlightening to watch how gregdek and others organize Fedora-on-XO testing.
• Testing tools: Ticketing systems (for OLPC, this means Trac).
• Testing book: Testing Computer Software, 2nd Edition

November

• Testing volunteers: Ubuntu. This is immediately after the release of Intrepid and the beginning of their 6-month release cycle; it will be enlightening to watch how they kick off a cycle in the testing world.
• Testing tools: Build tools, both automated and nonautomated. (Make it easy to get set up for testing.)
• Testing book: Lessons Learned in Software Testing

December

• Testing volunteers: GNOME. This is in the middle of their release cycle when there's nothing scheduled and the testers will have more time - I've admired their getting-newbies-started approach with bug triage days for a while, and want to learn how they manage to keep things friendly while turning nontechnical people into great testers.
• Testing tools: GUI testing tools. Things like X Window System event scripting, etc...
• Testing book: From here on out it's blank - I'd love to get some hardware/RF testing stuff in here, but will play it by ear.

January

• Testing volunteers: Ubuntu. This is immediately after the release of Intrepid and the beginning of their 6-month release cycle; I'm going to watch how they begin each round.
• Testing tools: Volunteer management frameworks. For Ubuntu, Launchpad. For us, perhaps the wiki? Semantic Mediawiki is one possible thing to focus on this month. Wiki/IRC bots to translate info between various frameworks may also be helpful here; automated scripts for testing volunteer web frameworks (twill, ClientCookie, etc.) can also be used for bots.

March

• Testing volunteers: OpenPlans? Tim has been doing great stuff with exploratory testing.
• Testing tools: Memory leak / performance testing. (This will fit in towards the end of OLPC's release cycle when we'll need to test many such bugs.) We do these manually right now.

April

• Testing volunteers: SugarLabs.
• Testing tools: This space intentionally left blank. Ideas?