TestMtg, 2008-10-10

Action items

Please mark here when done.

• Frances will be at the Review squad sprint at the MIT Museum tomorrow afternoon - results to be reported to testing@lists, sugar@lists afterwards
• Joe will email 1cc mass collaboration test results to testing@lists - even if most people can't replicate the test themselves (too many XOs needed) knowing the results is still useful!
• Joe will investigate VNC for automated testing
• Mel to send out notes from this meeting
• Mel will close out 8.2.0 bugs; any that are left open should be moved to 8.2.1.
• Mel will follow up on Ed/Marco/Cjb's suggestions and make pointers to "how to submit a good bug report" documents for novices to refer to; the SugarLabs and GNOME docs were suggested as good starters. defer - did not do yet
• Mel will spend a few minutes tracking down existing automated sugar/X testing efforts from the past
• Mel to schedule Make Tinderbox More Useful discussion defer - need tinderbox 101 first
• Reuben will work on configuration scripts

Exciting things this week

• We now have 62 laptops talking on 1 AP, hurrah!
• 1cc mass laptop testing: Joe went around the office this afternoon and got a lot of the OLPC staff to help out with testing collaboration on the 62 laptops mentioned above. It was a blast - we had people posting pictures of whales, hilarious political debates on Chat... Joe will post notes and results to this list.
• We're running into limitations on our AP hardware - 62 XOs over the entire space of OLPC's office is pushing it for an AP, and more distant XOs were definitely dropping off the network more often. It was suggested that we make an Activity that will do RF Surveys. (no follow up just yet, but anyone who wants to take this on is encouraged to holler to this list.)

8.2.1 and onwards

The QA team was mentioned as a Happy Thing To Have during the post-partum 8.2.0 meeting today - this is possible thanks to a ton of support from the developers in helping us ramp up. Now that we have our feet under us, our next release round will focus on test automation tools (which should in turn make it easier for volunteers to contribute) - more on this later.

There are 6 bugs for 8.2.0 that were not closed, some of them aren't really testable (memory leaks, for example, can't really be verified if we can't reliably reproduce the thing that leaks.) Mel will close these out and move the truly open bugs to 8.2.1.

Speaking of that: 8.2.1 is the queue we should be keeping our eye on now (9.1 is far off on the horizon, and we'll come back to it later). If you want to help us test, look at

• http://tinyurl.com/8-2-1-testinbuild
• http://tinyurl.com/8-2-1-testinrelease
• http://tinyurl.com/8-2-1-qasignoff

(If you want to help test but aren't sure where to get started, reply to this email and we'll set up a time for you to pair with someone and go through some tickets.)

Jabber

Reuben has a new server up this week and reports that 400 users registered within the first 24 hours. (Reuben: "In fact, there are something like 100 on right now.")

Background: Jabber is used for collaboration. During the first G1G1 each XO initially was set up to connect to the same collaboration server, but right before release, OLPC realized that having tens of thousands of laptops simultaneously trying to connect to that was a Bad Idea. So right now you have to know how to type in the right stuff to connect to Jabber. Getting it so that we can do more widespread testing for the next round would be great; Reuben to look into that.

Automated testing

This is QA's New Big Thing for the next release round (6 months). Things we are looking into so far:

• automated sugar/X testing efforts already underway (scripts, master-slave, etc... Mel to chase down)
• trac plugins, workflows (to be discussed with Ed, deferred to next week's meeting)
  • Make Tinderbox More Useful (Mel to schedule discussion; joint ownership of dev and qa person?)
• configuration scripts (Reuben to look at this)
• VNC (Joe to investigate)

We need to have a larger discussion/brainstorm on how we can automate our testing setup; deferred to next meeting.

Howto submit a bug

(earlier, from IRC post-partum 8.2.0 meeting)

<edmcnierney> (Random thought) It would be helpful to have pointers to "how to submit a good bug report" documents for novices to refer to. I was a little frustrated near the end at the effort required to decipher verbose but cryptic bug reports.

<marcopg> mchua: gnome has some guidelines, erikos stealed some of them

<cjb> oh, yes, we should definitely use those GNOME pages are guides next time

Review squad

It lives! See http://wiki.laptop.org/go/Talk:Review_squad#Current_draft for what Frances will be doing at the MIT Museum this weekend.

Next week's agenda

4pm EST in #olpc-meeting; Mel will be the IRC/Kim's-office transcribing-relay-person. Please join us - or at least add suggested items to the agenda!

• Review of action items
• community testing on Saturday - how did it go, what should we do? (invite learning team, greg?)
• larger discussion/brainstorm on how we can automate our testing setup. More ideas:
  • memory, networking, battery test scripts + script-analyzers
  • script for uploading test scripts (meta-test-script)
  • XS test scripts, tools, cases (we don't have many yet)
  • test results/stats reporting tools
• syncing up OLPC testing with Sugar, Fedora, etc. testing
  • bug tracker bifurcation
  • how do we work with upstream?
  • how do we work with volunteers, how much do we want to? test-gang ramp-up timeline.
• RF test warehouse facility - are we going to get one?
• Trac workflow - when/where/how to talk with devels about this?
• Budgets / equipment
• Other community efforts
• Mel's trip to google