Community testing meetings/2008-11-06: Difference between revisions

From OLPC
Jump to navigation Jump to search
mNo edit summary
 
(21 intermediate revisions by the same user not shown)
Line 1: Line 1:
{{TOCright}}
{{TOCright}}


'''''This is a pre-meeting agenda.'''The meeting is in #olpc-meeting on [[Start date::November 11, 2008 17:00]] (Eastern standard time in the USA).''
'''''This meeting is over.'''The meeting was in #olpc-meeting on [[Start date::November 11, 2008 17:00]] (Eastern standard time in the USA).'' Please see the [http://meeting.laptop.org/olpc-meeting.log.20081106_1658.html meeting log] for more details.


⚫
== Meeting log ==
⚫
== Previous meeting's action items ==


See http://meeting.laptop.org/olpc-meeting.log.20081106_1658.html.
⚫
See [[Testing meetings/2008-10-30#Action items]].


== Activity testing ==
== Meeting highlights ==


* We have a (very ugly, please help fix!) portal page, http://wiki.laptop.org/go/Community_testing, and a meetings portal page, http://wiki.laptop.org/go/Community_testing_meetings.
⚫
== Test tools ==


* Greg and Brian are trying out (on Speak and Paint respectively) our http://wiki.laptop.org/go/How_to_test_an_Activity instructions - which are under severe amounts of construction... are there any volunteers here with exploratory test experience that can help write up this guide?
We've got some neat ones that could use some work/help. Some details below are definitely fuzzy; please ask questions on things that aren't clear and I'll do my best to fill the details in before the meeting.


* How can we encourage smart testers to take ownership of specific Activities and do "deep" testing on them, and come up with their own criteria for and metrics of quality? (As opposed to having community testers spending their time being/recruiting drones to run through scripts.) Greg quoted Patton: "Don't tell people how to do things. Tell them what needs to be done, and let the surprise you with their ingenuity." (Seriously, this was a great discussion - please do read the logs to get the full blast - it starts around 17:28:29.)
* [[Scripts for testing multiple XOs]]
* [[XO Monitor]]
* Cjb's screencast Activity (no wiki page yet)
* a Test Activity - this is still a vague idea, but some sort of "submit a bug report!" helper
* Finding a way to have XOs more easily (...automatically?) upload test logs to Some Central Server That Is As Of Yet Undefined
* Reuben's "configuration magic" scripts (soon to be in git)
* Others?


* It was generally agreed that automation was a good idea for reducing drone-ness, which is boring. Ben and Mel will be dreaming up designs over the next week, and welcome help. Basically, "I'm a tester. I want to automate this boring thing. What is my ideal interface to do so / the most beautiful tool I could imagine for it?" One possible source of inspiration: Sugarbot.
(Note that we might discuss this in a future meeting if we run out of time)

* Everybody likes Joe's design which was the basis of our current semantic-mediawiki-based test case management system (http://wiki.laptop.org/go/Test_cases_8.2.0).

* Marco brought up some great questions about the tester/developer relationship. We need to make sure that good bugs get filed when a test fails, and that developers know about the testing going on for the things they're working on (actually, this should be a "people are using my work, yay!" motivation.)

* Suggestions for rewarding community testers: cookies, whippings, whipped creme cookies, beer, wine, vodka, *Russian* vodka (it's special!), Caribbean vacations... we'll see what we can do. ;)

We also agreed to move two conversations to email - prioritizing Activities to test, and then displaying testing metrics in motivating ways. So as to not flood everyone's inboxes, I'll ask Greg and Chris to bring these conversations up over the next few days. :)


'''------- Everything below this line is from the pre-meeting agenda. See the meeting logs to see what was discussed. --------'''

⚫
== Previous meeting's action items ==


⚫
See [[Testing meetings/20081030#Action items]].
[[User:Mchua|Mchua]] 01:43, 28 October 2008 (UTC)


== Community testing portal page (lfaraone, mchua) ==
== Test automation ==


* This is a ''big'' deal. We need a lot of work/help here.
We have a [[Community testing]] page! It needs a lot of help...
* Ideas? Suggestions?


== Current Activity prep-for-test procedure feedback (mchua) ==
(Note that we might discuss this in a future meeting if we run out of time)


Inspired by [[User:Gregdek|Gregdek]]: [[How to test an Activity]] - can we try this out?
[[User:Mchua|Mchua]] 01:43, 28 October 2008 (UTC)


[[User:Kimquirk|Kimquirk]] requests that we spend some time on quality, metrics, and how people can take ownership of that for specific activities or features of Sugar.
== Tests that are hard for volunteers to replicate ==


== Status of current Activities being tested (mchua, frandog) ==
* Volunteers who don't have XOs
* Volunteers who don't have many XOs


To cover:
Ok, so things like "50 XO collaboration" are probably not testable by many people... how can (should?) we still keep you up to date on what is going on with testing in the office?


* [[Analyze]] ([[User:Mchua|Mchua]] and [[User:Frances|Frances]] looking for someone to hand this off to)
What can you still test without "equipment?"
* [[Speak]] ([[User:Gregdek|Gregdek]])
* [[Paint]]? ([[User:Bjordan|Bjordan]]?)


== Deferred to email ==
What tools/equipment/infrastructure/support/encouragement/resources can we provide - both from OLPC, and from volunteers to each other - to help get testing done? LiveCDs and emulators? What do we think is worth our effort to create/maintain, given that everyone has limited resources and time?


=== Prioritizing Activities to test (gregdek) ===
(Note that we might discuss this in a future meeting if we run out of time)


What options/tools are out there?
[[User:Mchua|Mchua]] 01:43, 28 October 2008 (UTC)


Pre-meeting notes on the topic are at [[/Prioritizing activities to test]], add your own here!
== Learning and teaching testing ==


=== Displaying testing metrics in motivating ways (cjl, from notes) ===
Some people have expressed a desire to use this group as a way/motivation to study testing, as a way of building skill - I'd like to get out some ideas for how we might do that; we've had some very generous volunteers who've offered to help teach/mentor new testers in the community.


What options/tools are out there?
[[User:Mchua|Mchua]] 01:43, 28 October 2008 (UTC)


Pre-meeting notes on the topic are at [[/Displaying testing metrics in motivating ways]], add your own here!
== Some notes on infrastructure ==


== Action items ==
Think of these all as strawmen to be tried out now, revised next week if needed. [[User:Mchua|Mchua]]


* '''mchua''' <s>to post/mail-to-list these meeting minutes</s> '''DONE'''
* Meeting time/place/format?
* '''mchua''' <s>to follow-up with outstanding action items from last week</s> '''DONE'''
⚫
* Meeting ground rules?
* '''mchua''' <s>Post an example on [[How to test an Activity]] of how a casual tester can record their test results.</s> '''(done, [[Reporting test results]])'''
* '''adricnet, mchua''' <s>Dream up automation designs to present to group next week. "I'm a tester. I want to automate this boring thing. What is my ideal interface to do so / the most beautiful tool I could imagine for it?"</s> '''DONE - thanks to garycmartin, kevix, and marcopg for joining!'''
* '''gregdek, bjordan''' <s>try out (on Speak and Paint respectively) our [[How to test an Activity]] instructions.</s> '''DONE for this round - again in the next!'''
* '''mchua''' <s>to talk with sj, kimquirk, and others about what kinds of brib^H^H^H^H resources we can offer to motivate and reward testers, both tangible and intangible.</s> '''(Deferred to next week in consideration of Kim and SJ being very busy with G1G1 right now.)'''
* '''gregdek, cjl''' <s>to bring up items that were [[Testing_meetings/2008-11-06#Deferred_to_email|deferred to email]] in conversation in the testing mailing list over the next week.</s> '''DONE'''


⚫
== Help needed ==
== Purpose/charter/mission of community test ==


* [[How to test an Activity]] instructions are under severe amounts of construction... are there any volunteers here with exploratory test experience that can help write up this guide?
We won't finish this discussion on the first meeting. We'll barely even start it. But we can get some thoughts on on the table. I'd like to spend no more than 10 minutes just hearing initial thoughts on this - no judgement, no decisions, just ideas - and send the conversation to the mailing list to get thoughts going in the background as we continue to test. I think this conversation will make much more sense in several weeks once we've gotten a working idea of how we, as a community test group, function.


[[User:Mchua|Mchua]]
[[Category:Test group meetings]]
[[Category:Test group meetings]]
[[Category:Community testing meetings]]

Latest revision as of 06:31, 14 November 2008

This meeting is over.The meeting was in #olpc-meeting on Start date::November 11, 2008 17:00 (Eastern standard time in the USA). Please see the meeting log for more details.

Meeting log

See http://meeting.laptop.org/olpc-meeting.log.20081106_1658.html.

Meeting highlights

  • Greg and Brian are trying out (on Speak and Paint respectively) our http://wiki.laptop.org/go/How_to_test_an_Activity instructions - which are under severe amounts of construction... are there any volunteers here with exploratory test experience that can help write up this guide?
  • How can we encourage smart testers to take ownership of specific Activities and do "deep" testing on them, and come up with their own criteria for and metrics of quality? (As opposed to having community testers spending their time being/recruiting drones to run through scripts.) Greg quoted Patton: "Don't tell people how to do things. Tell them what needs to be done, and let the surprise you with their ingenuity." (Seriously, this was a great discussion - please do read the logs to get the full blast - it starts around 17:28:29.)
  • It was generally agreed that automation was a good idea for reducing drone-ness, which is boring. Ben and Mel will be dreaming up designs over the next week, and welcome help. Basically, "I'm a tester. I want to automate this boring thing. What is my ideal interface to do so / the most beautiful tool I could imagine for it?" One possible source of inspiration: Sugarbot.
  • Marco brought up some great questions about the tester/developer relationship. We need to make sure that good bugs get filed when a test fails, and that developers know about the testing going on for the things they're working on (actually, this should be a "people are using my work, yay!" motivation.)
  • Suggestions for rewarding community testers: cookies, whippings, whipped creme cookies, beer, wine, vodka, *Russian* vodka (it's special!), Caribbean vacations... we'll see what we can do. ;)

We also agreed to move two conversations to email - prioritizing Activities to test, and then displaying testing metrics in motivating ways. So as to not flood everyone's inboxes, I'll ask Greg and Chris to bring these conversations up over the next few days. :)


------- Everything below this line is from the pre-meeting agenda. See the meeting logs to see what was discussed. --------

Previous meeting's action items

See Testing meetings/20081030#Action items.

Community testing portal page (lfaraone, mchua)

We have a Community testing page! It needs a lot of help...

Current Activity prep-for-test procedure feedback (mchua)

Inspired by Gregdek: How to test an Activity - can we try this out?

Kimquirk requests that we spend some time on quality, metrics, and how people can take ownership of that for specific activities or features of Sugar.

Status of current Activities being tested (mchua, frandog)

To cover:

Deferred to email

Prioritizing Activities to test (gregdek)

What options/tools are out there?

Pre-meeting notes on the topic are at /Prioritizing activities to test, add your own here!

Displaying testing metrics in motivating ways (cjl, from notes)

What options/tools are out there?

Pre-meeting notes on the topic are at /Displaying testing metrics in motivating ways, add your own here!

Action items

  • mchua to post/mail-to-list these meeting minutes DONE
  • mchua to follow-up with outstanding action items from last week DONE
  • mchua Post an example on How to test an Activity of how a casual tester can record their test results. (done, Reporting test results)
  • adricnet, mchua Dream up automation designs to present to group next week. "I'm a tester. I want to automate this boring thing. What is my ideal interface to do so / the most beautiful tool I could imagine for it?" DONE - thanks to garycmartin, kevix, and marcopg for joining!
  • gregdek, bjordan try out (on Speak and Paint respectively) our How to test an Activity instructions. DONE for this round - again in the next!
  • mchua to talk with sj, kimquirk, and others about what kinds of brib^H^H^H^H resources we can offer to motivate and reward testers, both tangible and intangible. (Deferred to next week in consideration of Kim and SJ being very busy with G1G1 right now.)
  • gregdek, cjl to bring up items that were deferred to email in conversation in the testing mailing list over the next week. DONE

Help needed

  • How to test an Activity instructions are under severe amounts of construction... are there any volunteers here with exploratory test experience that can help write up this guide?
Retrieved from "http://wiki.laptop.org/mediawiki/index.php?title=Community_testing_meetings/2008-11-06&oldid=180109"