TestMtg, June 11, 2007

Minutes from June 11, 2007 - there is a test meeting every Mon 1pm EDT

Attending: Zack, Cameron, Chris Blizzard, Jim, Kim

Ship meeting highlights: The milestones for Trial-2 were outlined at the ship meeting:

• Feature Freeze, 6/26
• Release Candidate 1, 6/29
• Release Candidate 2, 7/6
• Release Candidate 3, 7/16 -- also code freeze
• Release, 7/23

We discussed why we are calling this a 'release' candidate. Since there are many test organizations helping out (Redhat, Quanta, olpc people, and beta sites with students), we need some documentation with these releases. We will request from John (J5) a diff between releases for all builds and we can add specific work arounds or information from testing as we get it.

Process - builds: We discussed some process issues to help with consistency across the testing. We would like is to limit builds as we get closer to final release so the test group has time to test deeper. It is expected that feature freeze will probably be a time where there are a number of big blocking bugs, so it would be good to get a committment from developers to help attack the blockers as high priority right after feature freeze.

• Between Feature Freeze and RC1 we expect to get new builds pretty much daily.
• Between RC1 and RC2, we would like to have builds approximately every other day.
• Between RC2 and RC3, we would like only 2 builds.
• Once we get to Code freeze, RC3, we need to hold all checkins for review by the triage team.

Process - Info by developers as part of checkin. We would like to have information in the checkin that is useful for testing. Ideally all code (at least after feature freeze) would be checked in against a bug, so we can easily understand what was fixed. If not, the description at check in is key and can be used to automatically determine the diff between builds.

Process - Triage. All bugs should go into the system against 'untriaged' release. The triage team (minimum Chris Blizzard, Jim Gettys, Kim Quirk) will start reviewing on a daily basis by the end of this week. The triage team will review the priority set by the bug finder, what release it belongs in, and possibly re-assign it if it has the wrong owner. We will set up a time of day and conf bridge for people who want to call in.

We discussed which code base we should all be using -- we should all test from the head, not the 406.x branch. This branch was built to get a stable (but somewhat buggy) release for Quanta's build. At this point we should all be looking at the new releases.

Please remember that the latest release today is in a very unstable state. After we get to feature freeze, then we can start putting bugs in Trac. If you add bugs before feature freeze you are just making a lot of work for the triage team. Also, please try to do some searching when you want to add a bug... to see if this bug already exists. Between now and feature freeze, we should be focused more on test case creation rather than on test execution.

ACTIONS:

• Chris Blizzard with go over the request for code to be checked in with information and/or trac number whenever possible.
• Chris will work with John to see what kind of automated information we can get from each new build (especially important after we get to feature freeze)
• Zack will create a use case as a template for the test cases we can all write. He will send out email to the testing@laptop.org group for review before next Monday's meeting. We can review it at next week's meeting