Community testing meetings/2008-12-04

  This page is part of the OLPC Community testing Project. How to test an Activity | Reporting test results | Meetings

This is a Community testing meeting. This meeting is over. Location was in #olpc-meeting on Start date::December 04, 2008 22:00 (time is in UTC, click here to find the meeting time for your time zone.)

You can read the meeting logs here.

Previous meeting's action items

See Community testing meetings/2008-11-20. Note that we're trying an experiment and not reviewing action items this meeting round. Read the meeting announcement email for an explanation why.

Law of two feet

Everyone should read The Law Of Two Feet.

Recent events

Work continues within the community testing team on infrastructure/tools, with Adric setting up a Litmus demo and Carl crafting a next-gen smoke test reporting to be trialed for the G1G1 Activity test sprint.

We also released a first (under-construction) draft of progress towards their goal to test all G1G1 Activities by December 25; a special thanks goes out to Skierpage, Gary C. Martin, Marco Pesenti Gritti, and Caryl Bigenho for their work on http://wiki.laptop.org/go/G1G1_Activity_testing.

We're actually ahead of our ambitious schedule - the focus has been on capacity building to allow parallel and distributed test sprinting over the next two weeks - thanks to Tabitha Roder, Alastair Munro, and the Wellington test group*, who've been drawing on their Saturday workshop experiences to coach our group on coaching new workshop runners - and turning out some great Activity testing results we need to find a way to capture!

If you have access to at least 1 XO and are interested in hosting an Activity testing party (3+ people, 3+ hours) in your area, please email Mel Chua at mel at laptop.org so we can help you host the party that *you* want. (Really. OLPC will help you host a party.) Prior Activity testing experience is not required; if you can devote 12 hours to planning and execution between now and December 25, we will give you the resources, knowledge, and support you need to be successful.

Discussion: Developer/tester relationships

Open discussion time - bring your questions! What do you want to learn about developer/tester relationships or how to make them better? What are you thinking? Merged with the next topic, brainstorming.

Brainstorm: What are the blockers on Activity testing?

See the Rules of brainstorming before we start.

Focus question: Assuming that we want the G1G1 Activities to look like this before December 25, 2008, what are the things that are (or might) prevent us from getting there?

What we'll be doing:

• Sanity-check the focus question
• Generate as many possible solutions to that question as we can
• Agree on a way to identify the most pressing blockers to fix (by default: excellence voting)
• Identify the most pressing blockers to fix.
• Make sure our top N (for some value of N) blockers will have good problem statements within 24 hours of the meeting end time.

Note: this may be fast - if it is, we'll do some quick lightning brainstorm rounds to generate possible solutions to our most pressing blockers/problems. If not, I'm happy leaving with a good understanding of what needs to be fixed and why.

Raw ideas generated

• developers have too much information coming in from all the mailing lists and feeds - it's hard to spot bugs in the noise.
• it's not clear what priority/ranking bugs have.
• activity developers don't get e-mail when someone files a Trac bug against that activity's component
• testers often rediscover bugs that have been filed months ago and not fixed; this is demotivating
• when a ticket is filed, an active maintainer may not be notified.
• activities might not have components at dev.sl.o
• activity may not have a trac component (and thus their maintainers can't be notified on the filing of a bug)
• bug reporters don't know where to file bugs
• bug reporters don't know where to hunt to see if a bug has already been filed
• activity maintainers might not know they have a bug
• overload...too much to do
• activity maintainers might not know they have a bug because there's too much noise-to-signal
• trac is not audited for maitainers' names (to make sure there is a maintainer)
• maintainers became not interested, and they're not around at all any more
• maintainers might not know we care about them fixing bugs :)
• there is no maintainer for an activity; it was abandoned
• it might not be clear what a bug is - maybe the bug is hard to understand or read
• maybe they can't reproduce the bug
• it might not be clear how important a bug is (priority is almost ever not correctly set)
• it might not be clear why a bug is important to fix; i might think it's vital to the future of humanity, but the maintainer might see it as a feature
• maybe the bug is intermittent and really hard to reproduce
• maybe the bug needs special equipment to reproduce it, and the developer doesn't have it
• maybe the person who found the bug doesn't speak the developer's language
• maybe it's not clear how to contact someone about fixing a bug for an activity - testers not having someone to go to
• nobody brave enough to kill and remove an already dead activity that no one wants.
• maybe the developer doesn't know how to fix the bug, may not know where to look in the code, what tools to use, etc. (especially if they didn't write the original program, or if they did it as a learning exercise... imagine a new middle-school coder being hit by memory leak issues)

Action items

No new ones this week, other than Mel following up on last week's action items.