Community testing meetings/2008-11-06

This meeting is over.The meeting was in #olpc-meeting on Start date::November 11, 2008 17:00 (Eastern standard time in the USA). Please see the meeting log for more details.

Meeting log

See http://meeting.laptop.org/olpc-meeting.log.20081106_1658.html.

Meeting highlights

• We have a (very ugly, please help fix!) portal page, http://wiki.laptop.org/go/Community_testing, and a meetings portal page, http://wiki.laptop.org/go/Community_testing_meetings.

• Greg and Brian are trying out (on Speak and Paint respectively) our http://wiki.laptop.org/go/How_to_test_an_Activity instructions - which are under severe amounts of construction... are there any volunteers here with exploratory test experience that can help write up this guide?

• How can we encourage smart testers to take ownership of specific Activities and do "deep" testing on them, and come up with their own criteria for and metrics of quality? (As opposed to having community testers spending their time being/recruiting drones to run through scripts.) Greg quoted Patton: "Don't tell people how to do things. Tell them what needs to be done, and let the surprise you with their ingenuity." (Seriously, this was a great discussion - please do read the logs to get the full blast - it starts around 17:28:29.)

• It was generally agreed that automation was a good idea for reducing drone-ness, which is boring. Ben and Mel will be dreaming up designs over the next week, and welcome help. Basically, "I'm a tester. I want to automate this boring thing. What is my ideal interface to do so / the most beautiful tool I could imagine for it?" One possible source of inspiration: Sugarbot.

• Everybody likes Joe's design which was the basis of our current semantic-mediawiki-based test case management system (http://wiki.laptop.org/go/Test_cases_8.2.0).

• Marco brought up some great questions about the tester/developer relationship. We need to make sure that good bugs get filed when a test fails, and that developers know about the testing going on for the things they're working on (actually, this should be a "people are using my work, yay!" motivation.)

• Suggestions for rewarding community testers: cookies, whippings, whipped creme cookies, beer, wine, vodka, *Russian* vodka (it's special!), Caribbean vacations... we'll see what we can do. ;)

We also agreed to move two conversations to email - prioritizing Activities to test, and then displaying testing metrics in motivating ways. So as to not flood everyone's inboxes, I'll ask Greg and Chris to bring these conversations up over the next few days. :)

------- Everything below this line is from the pre-meeting agenda. See the meeting logs to see what was discussed. --------

Previous meeting's action items

See Testing meetings/20081030#Action items.

Community testing portal page (lfaraone, mchua)

We have a Community testing page! It needs a lot of help...

Current Activity prep-for-test procedure feedback (mchua)

Inspired by Gregdek: How to test an Activity - can we try this out?

Kimquirk requests that we spend some time on quality, metrics, and how people can take ownership of that for specific activities or features of Sugar.

Status of current Activities being tested (mchua, frandog)

To cover:

• Analyze (Mchua and Frances looking for someone to hand this off to)
• Speak (Gregdek)
• Paint? (Bjordan?)

Deferred to email

Prioritizing Activities to test (gregdek)

What options/tools are out there?

Pre-meeting notes on the topic are at /Prioritizing activities to test, add your own here!

Displaying testing metrics in motivating ways (cjl, from notes)

What options/tools are out there?

Pre-meeting notes on the topic are at /Displaying testing metrics in motivating ways, add your own here!

Action items

• mchua to post/mail-to-list these meeting minutes DONE
• mchua to follow-up with outstanding action items from last week DONE
• mchua Post an example on How to test an Activity of how a casual tester can record their test results. (done, Reporting test results)
• adricnet, mchua Dream up automation designs to present to group next week. "I'm a tester. I want to automate this boring thing. What is my ideal interface to do so / the most beautiful tool I could imagine for it?" DONE - thanks to garycmartin, kevix, and marcopg for joining!
• gregdek, bjordan try out (on Speak and Paint respectively) our How to test an Activity instructions. DONE for this round - again in the next!
• mchua to talk with sj, kimquirk, and others about what kinds of brib^H^H^H^H resources we can offer to motivate and reward testers, both tangible and intangible. (Deferred to next week in consideration of Kim and SJ being very busy with G1G1 right now.)
• gregdek, cjl to bring up items that were deferred to email in conversation in the testing mailing list over the next week. DONE

Help needed

• How to test an Activity instructions are under severe amounts of construction... are there any volunteers here with exploratory test experience that can help write up this guide?