Community testing meetings/2008-11-06: Difference between revisions

From OLPC
Jump to navigation Jump to search
 
(19 intermediate revisions by the same user not shown)
Line 1: Line 1:
{{TOCright}}
{{TOCright}}


'''''This is a pre-meeting agenda.'''The meeting is in #olpc-meeting on [[Start date::November 11, 2008 17:00]] (Eastern standard time in the USA).''
'''''This meeting is over.'''The meeting was in #olpc-meeting on [[Start date::November 11, 2008 17:00]] (Eastern standard time in the USA).'' Please see the [http://meeting.laptop.org/olpc-meeting.log.20081106_1658.html meeting log] for more details.


⚫
== Meeting log ==
⚫
== Previous meeting's action items ==


See http://meeting.laptop.org/olpc-meeting.log.20081106_1658.html.
⚫
See [[Testing meetings/20081030#Action items]].


== Activity testing ==
== Meeting highlights ==


* We have a (very ugly, please help fix!) portal page, http://wiki.laptop.org/go/Community_testing, and a meetings portal page, http://wiki.laptop.org/go/Community_testing_meetings.
⚫
=== Status of Analyze testing (mchua, frandog) ===


* Greg and Brian are trying out (on Speak and Paint respectively) our http://wiki.laptop.org/go/How_to_test_an_Activity instructions - which are under severe amounts of construction... are there any volunteers here with exploratory test experience that can help write up this guide?
[[Analyze]] is the first Activity going through community test. ([[Paint]] will probably be the second, if [[User:Bjordan]] or someone else is willing to lead.)


* How can we encourage smart testers to take ownership of specific Activities and do "deep" testing on them, and come up with their own criteria for and metrics of quality? (As opposed to having community testers spending their time being/recruiting drones to run through scripts.) Greg quoted Patton: "Don't tell people how to do things. Tell them what needs to be done, and let the surprise you with their ingenuity." (Seriously, this was a great discussion - please do read the logs to get the full blast - it starts around 17:28:29.)
⚫
=== Prioritizing Activities to test (gregdek) ===


* It was generally agreed that automation was a good idea for reducing drone-ness, which is boring. Ben and Mel will be dreaming up designs over the next week, and welcome help. Basically, "I'm a tester. I want to automate this boring thing. What is my ideal interface to do so / the most beautiful tool I could imagine for it?" One possible source of inspiration: Sugarbot.
⚫
=== Displaying testing metrics in motivating ways (cjl, from notes) ===


* Everybody likes Joe's design which was the basis of our current semantic-mediawiki-based test case management system (http://wiki.laptop.org/go/Test_cases_8.2.0).
⚫
=== Current Activity prep-for-test procedure feedback (lfaraone, mchua) ===


* Marco brought up some great questions about the tester/developer relationship. We need to make sure that good bugs get filed when a test fails, and that developers know about the testing going on for the things they're working on (actually, this should be a "people are using my work, yay!" motivation.)
⚫
== Test tools ==


* Suggestions for rewarding community testers: cookies, whippings, whipped creme cookies, beer, wine, vodka, *Russian* vodka (it's special!), Caribbean vacations... we'll see what we can do. ;)
We've got some neat ones that could use some work/help. Some details below are definitely fuzzy; please ask questions on things that aren't clear and I'll do my best to fill the details in before the meeting.


We also agreed to move two conversations to email - prioritizing Activities to test, and then displaying testing metrics in motivating ways. So as to not flood everyone's inboxes, I'll ask Greg and Chris to bring these conversations up over the next few days. :)
* [[Scripts for testing multiple XOs]]
* [[XO Monitor]]
* Cjb's screencast Activity (no wiki page yet)
* a Test Activity - this is still a vague idea, but some sort of "submit a bug report!" helper
* Finding a way to have XOs more easily (...automatically?) upload test logs to Some Central Server That Is As Of Yet Undefined
* Reuben's "configuration magic" scripts (soon to be in git)
* Others?


(Note that we might discuss this in a future meeting if we run out of time)


'''------- Everything below this line is from the pre-meeting agenda. See the meeting logs to see what was discussed. --------'''
[[User:Mchua|Mchua]] 01:43, 28 October 2008 (UTC)


⚫
== Previous meeting's action items ==
== Test automation ==


⚫
See [[Testing meetings/20081030#Action items]].
* This is a ''big'' deal. We need a lot of work/help here.
* Ideas? Suggestions?


== Community testing portal page (lfaraone, mchua) ==
(Note that we might discuss this in a future meeting if we run out of time)


We have a [[Community testing]] page! It needs a lot of help...
[[User:Mchua|Mchua]] 01:43, 28 October 2008 (UTC)


⚫
== Current Activity prep-for-test procedure feedback (mchua) ==
== Tests that are hard for volunteers to replicate ==


Inspired by [[User:Gregdek|Gregdek]]: [[How to test an Activity]] - can we try this out?
* Volunteers who don't have XOs
* Volunteers who don't have many XOs


[[User:Kimquirk|Kimquirk]] requests that we spend some time on quality, metrics, and how people can take ownership of that for specific activities or features of Sugar.
Ok, so things like "50 XO collaboration" are probably not testable by many people... how can (should?) we still keep you up to date on what is going on with testing in the office?


⚫
== Status of current Activities being tested (mchua, frandog) ==
What can you still test without "equipment?"


To cover:
What tools/equipment/infrastructure/support/encouragement/resources can we provide - both from OLPC, and from volunteers to each other - to help get testing done? LiveCDs and emulators? What do we think is worth our effort to create/maintain, given that everyone has limited resources and time?


* [[Analyze]] ([[User:Mchua|Mchua]] and [[User:Frances|Frances]] looking for someone to hand this off to)
(Note that we might discuss this in a future meeting if we run out of time)
* [[Speak]] ([[User:Gregdek|Gregdek]])
* [[Paint]]? ([[User:Bjordan|Bjordan]]?)


== Deferred to email ==
[[User:Mchua|Mchua]] 01:43, 28 October 2008 (UTC)


⚫
=== Prioritizing Activities to test (gregdek) ===
== Learning and teaching testing ==


What options/tools are out there?
Some people have expressed a desire to use this group as a way/motivation to study testing, as a way of building skill - I'd like to get out some ideas for how we might do that; we've had some very generous volunteers who've offered to help teach/mentor new testers in the community.

Pre-meeting notes on the topic are at [[/Prioritizing activities to test]], add your own here!

⚫
=== Displaying testing metrics in motivating ways (cjl, from notes) ===


What options/tools are out there?
[[User:Mchua|Mchua]] 01:43, 28 October 2008 (UTC)


Pre-meeting notes on the topic are at [[/Displaying testing metrics in motivating ways]], add your own here!
== Some notes on infrastructure ==


== Action items ==
Think of these all as strawmen to be tried out now, revised next week if needed. [[User:Mchua|Mchua]]


* '''mchua''' <s>to post/mail-to-list these meeting minutes</s> '''DONE'''
* Meeting time/place/format?
* '''mchua''' <s>to follow-up with outstanding action items from last week</s> '''DONE'''
⚫
* Meeting ground rules?
* '''mchua''' <s>Post an example on [[How to test an Activity]] of how a casual tester can record their test results.</s> '''(done, [[Reporting test results]])'''
* '''adricnet, mchua''' <s>Dream up automation designs to present to group next week. "I'm a tester. I want to automate this boring thing. What is my ideal interface to do so / the most beautiful tool I could imagine for it?"</s> '''DONE - thanks to garycmartin, kevix, and marcopg for joining!'''
* '''gregdek, bjordan''' <s>try out (on Speak and Paint respectively) our [[How to test an Activity]] instructions.</s> '''DONE for this round - again in the next!'''
* '''mchua''' <s>to talk with sj, kimquirk, and others about what kinds of brib^H^H^H^H resources we can offer to motivate and reward testers, both tangible and intangible.</s> '''(Deferred to next week in consideration of Kim and SJ being very busy with G1G1 right now.)'''
* '''gregdek, cjl''' <s>to bring up items that were [[Testing_meetings/2008-11-06#Deferred_to_email|deferred to email]] in conversation in the testing mailing list over the next week.</s> '''DONE'''


⚫
== Help needed ==
== Purpose/charter/mission of community test ==


* [[How to test an Activity]] instructions are under severe amounts of construction... are there any volunteers here with exploratory test experience that can help write up this guide?
We won't finish this discussion on the first meeting. We'll barely even start it. But we can get some thoughts on on the table. I'd like to spend no more than 10 minutes just hearing initial thoughts on this - no judgement, no decisions, just ideas - and send the conversation to the mailing list to get thoughts going in the background as we continue to test. I think this conversation will make much more sense in several weeks once we've gotten a working idea of how we, as a community test group, function.


[[User:Mchua|Mchua]]
[[Category:Test group meetings]]
[[Category:Test group meetings]]
[[Category:Community testing meetings]]

Latest revision as of 06:31, 14 November 2008

This meeting is over.The meeting was in #olpc-meeting on Start date::November 11, 2008 17:00 (Eastern standard time in the USA). Please see the meeting log for more details.

Meeting log

See http://meeting.laptop.org/olpc-meeting.log.20081106_1658.html.

Meeting highlights

  • Greg and Brian are trying out (on Speak and Paint respectively) our http://wiki.laptop.org/go/How_to_test_an_Activity instructions - which are under severe amounts of construction... are there any volunteers here with exploratory test experience that can help write up this guide?
  • How can we encourage smart testers to take ownership of specific Activities and do "deep" testing on them, and come up with their own criteria for and metrics of quality? (As opposed to having community testers spending their time being/recruiting drones to run through scripts.) Greg quoted Patton: "Don't tell people how to do things. Tell them what needs to be done, and let the surprise you with their ingenuity." (Seriously, this was a great discussion - please do read the logs to get the full blast - it starts around 17:28:29.)
  • It was generally agreed that automation was a good idea for reducing drone-ness, which is boring. Ben and Mel will be dreaming up designs over the next week, and welcome help. Basically, "I'm a tester. I want to automate this boring thing. What is my ideal interface to do so / the most beautiful tool I could imagine for it?" One possible source of inspiration: Sugarbot.
  • Marco brought up some great questions about the tester/developer relationship. We need to make sure that good bugs get filed when a test fails, and that developers know about the testing going on for the things they're working on (actually, this should be a "people are using my work, yay!" motivation.)
  • Suggestions for rewarding community testers: cookies, whippings, whipped creme cookies, beer, wine, vodka, *Russian* vodka (it's special!), Caribbean vacations... we'll see what we can do. ;)

We also agreed to move two conversations to email - prioritizing Activities to test, and then displaying testing metrics in motivating ways. So as to not flood everyone's inboxes, I'll ask Greg and Chris to bring these conversations up over the next few days. :)


------- Everything below this line is from the pre-meeting agenda. See the meeting logs to see what was discussed. --------

Previous meeting's action items

See Testing meetings/20081030#Action items.

Community testing portal page (lfaraone, mchua)

We have a Community testing page! It needs a lot of help...

Current Activity prep-for-test procedure feedback (mchua)

Inspired by Gregdek: How to test an Activity - can we try this out?

Kimquirk requests that we spend some time on quality, metrics, and how people can take ownership of that for specific activities or features of Sugar.

Status of current Activities being tested (mchua, frandog)

To cover:

Deferred to email

Prioritizing Activities to test (gregdek)

What options/tools are out there?

Pre-meeting notes on the topic are at /Prioritizing activities to test, add your own here!

Displaying testing metrics in motivating ways (cjl, from notes)

What options/tools are out there?

Pre-meeting notes on the topic are at /Displaying testing metrics in motivating ways, add your own here!

Action items

  • mchua to post/mail-to-list these meeting minutes DONE
  • mchua to follow-up with outstanding action items from last week DONE
  • mchua Post an example on How to test an Activity of how a casual tester can record their test results. (done, Reporting test results)
  • adricnet, mchua Dream up automation designs to present to group next week. "I'm a tester. I want to automate this boring thing. What is my ideal interface to do so / the most beautiful tool I could imagine for it?" DONE - thanks to garycmartin, kevix, and marcopg for joining!
  • gregdek, bjordan try out (on Speak and Paint respectively) our How to test an Activity instructions. DONE for this round - again in the next!
  • mchua to talk with sj, kimquirk, and others about what kinds of brib^H^H^H^H resources we can offer to motivate and reward testers, both tangible and intangible. (Deferred to next week in consideration of Kim and SJ being very busy with G1G1 right now.)
  • gregdek, cjl to bring up items that were deferred to email in conversation in the testing mailing list over the next week. DONE

Help needed

  • How to test an Activity instructions are under severe amounts of construction... are there any volunteers here with exploratory test experience that can help write up this guide?
Retrieved from "http://wiki.laptop.org/mediawiki/index.php?title=Community_testing_meetings/2008-11-06&oldid=180109"