Community testing meetings/2008-11-13

  This page is part of the OLPC Community testing Project. How to test an Activity | Reporting test results | Meetings

This is a Community testing meeting. This meeting is over. Location is in #olpc-meeting on Start date::November 13, 2008 22:00 (time is in UTC, click here to find the meeting time for your time zone.)

Meeting logs are here.

Previous meeting's action items

See Community testing meetings/2008-11-06#Action items - all done!

G1G1: What can we do to help?

Pre-meeting

• Caryl's Test Jam idea

During meeting

• Our first short-term goal (proposed thanks to Caryl): All G1G1-2008 Activities tested by Dec. 25, 2008. Definition of "tested" well on its way to being solid - this time next week we should have *metrics!* and be posting *how completed* we are! Oh boy! (Hey, I'm an engineer. I get excited about measuring stuff. :) These are listed on http://wiki.laptop.org/go/Activities/G1G1; there are 28 in total. See http://wiki.laptop.org/go/Community_testing for our current measurements of progress (as of this email writing, it's a simple list. This will change rapidly.)

• Longer-term plan: this will enable us to stress-test a good community testing participation framework so that we can recruit Lots Of New Volunteers (G1G1 donors?) starting in January - "please help!" pages are only useful inasmuch as there are people, groups, and tasks standing by to welcome newcomers and help them get started.

Activity testing instructions

Pre-meeting

Feedback on How to test an Activity? Volunteers to help build/test/maintain?

Excellent point brought up by Caryl and Greg via email: We are building a community of testers who are dedicated to keeping their individual Activities well-tested and properly functioning, and one part of this is making sure that the documentation matches with the product under test, and that they're both ridiculously good. ("High-quality" might be the, ah, more proper term.) The product under test includes documentation.

During meeting

• Tabitha and Alastair from the NZ Wellington OLPC community test group introduced themselves and their work - they have great procedures, months of experience testing Activities and are going to lead the charge in putting up oracles and reaching out to Activity developers. Their group meets again tomorrow. A warm welcome to all our NZ testers - and thank you to Martin for the introduction!
• Agreed to continue work on How to test an Activity - moving forward, things look good so far

Test result reporting instructions

Pre-meeting

Feedback on Reporting test results? Volunteers to help build/test/maintain?

During meeting

• Merged with #Activity testing instructions in discussion
• We need to make new forms for our G1G1 testing goal

Activity testing automation

Pre-meeting

See Activity testing automation

During meeting

• Many thanks to Anna from the Birmingham deployment for her work on http://wiki.laptop.org/go/Dsh on the XS - looking forward to hearing more about her + Kevix's experiments with the school server!

• Ben, Gary, Kevix, and Mel have drafted an http://wiki.laptop.org/go/Activity_testing_automation design, and experimental implementation has begin in parallel with a design review. Gary got in touch with Zach, developer of Sugarbot.

Mission/scope/focus of Community testing

Pre-meeting

What are our goals? See Community Testing#Introduction for the current strawman. After we agree on goals, we can figure out how to measure whether we're meeting them. We're already working towards these goals (or a very similar set), and should continue to - it's just sometimes useful to state these things explicitly.

During meeting

• We have a short-term goal for G1G1-goodness: (Test all G1G1-2008-shipped Activities by 2008-12-25.)
• More participants in meeting leading to more exciting - but also noisier, faster, and more chaotic - meetings. Still ok, but maybe time to think of ways to scale?

Action items

• mchua to talk with sj, kimquirk, and others about what kinds of brib^H^H^H^H resources we can offer to motivate and reward testers, both tangible and intangible. (Deferred from last week on account of SJ and Kim's G1G1 workload; deferred again to after Thanksgiving because of... G1G1 workload. Perhaps we should find another solution.)
• bjordan, gregdek, Caryl: Continue focusing on making http://wiki.laptop.org/go/How_to_test_an_Activity a good set of instructions through testing it with your Activities (Speaak, Paint, and TamTamEdit respectively) and giving feedback - Caryl with a focus on newbie testers, Greg with a focus on metrics. Done for the week, ongoing

• garycmartin, kevix, gregdek, mchua: 1-2 hour sprint to implement a first draft of a subjective 'complete' metric (and a 'participation' metric) to show the group next week, along with the "by our metrics, we are X% done with our G1G1-yay! goal" (Mel to see if skierpage would like to join us.) Semi-deferred: see this for Skierpage's thoughts on the matter, but we need to discuss this further.

• martin_xsa, marcopg: choose date for a testing meeting to devote mainly to developer-tester relationships, probably after Sugarcamp (Done, tentative - Dec. 4)

• tabitha_, aly: pick some Activities for the Wellington test group to tackle, go through the http://wiki.laptop.org/go/How_to_test_Activities "oracle-making" section for them (optional: go through your 2 previous test notes and put those in the oracles of the Activities you tested in the past).

• tabitha_, aly: with the Wellington test group, explore how we can build excellent test/developer relationships. This is a very open-ended charter - come back next week and tell us what we should do / what you're doing / what the plan is for this. (Might want to sync up with martin_xsa and marcopg on having that conversation with developers.)

• garycmartin: continue working on sugarbot with zach, post updates to the testing mailing list (side note: marcopg and mchua can be dragged in to join the sugarbot party when sugarcamp/G1G1 passes a bit.)

• anna_bham: Documenting dsh/ssh-to-remote-XO work - mchua to help figure out how to fit this into http://wiki.laptop.org/go/Testing_tools. Started work on this - need to discuss how to port info from internal blog. Ongoing.

• anna_bham, mchua: Gather to brainstorm out a plan for XS community testing; we should see if we can make it fit in with our G1G1 Activity testing goal somehow (maybe to set up large #s of XOs for test?) deferred.