Community testing meetings/2008-11-06: Difference between revisions

From OLPC
Jump to navigation Jump to search
 
(11 intermediate revisions by the same user not shown)
Line 1: Line 1:
{{TOCright}}
{{TOCright}}


'''''This is a pre-meeting agenda.'''The meeting is in #olpc-meeting on [[Start date::November 11, 2008 17:00]] (Eastern standard time in the USA).''
'''''This meeting is over.'''The meeting was in #olpc-meeting on [[Start date::November 11, 2008 17:00]] (Eastern standard time in the USA).'' Please see the [http://meeting.laptop.org/olpc-meeting.log.20081106_1658.html meeting log] for more details.

== Meeting log ==

See http://meeting.laptop.org/olpc-meeting.log.20081106_1658.html.

== Meeting highlights ==

* We have a (very ugly, please help fix!) portal page, http://wiki.laptop.org/go/Community_testing, and a meetings portal page, http://wiki.laptop.org/go/Community_testing_meetings.

* Greg and Brian are trying out (on Speak and Paint respectively) our http://wiki.laptop.org/go/How_to_test_an_Activity instructions - which are under severe amounts of construction... are there any volunteers here with exploratory test experience that can help write up this guide?

* How can we encourage smart testers to take ownership of specific Activities and do "deep" testing on them, and come up with their own criteria for and metrics of quality? (As opposed to having community testers spending their time being/recruiting drones to run through scripts.) Greg quoted Patton: "Don't tell people how to do things. Tell them what needs to be done, and let the surprise you with their ingenuity." (Seriously, this was a great discussion - please do read the logs to get the full blast - it starts around 17:28:29.)

* It was generally agreed that automation was a good idea for reducing drone-ness, which is boring. Ben and Mel will be dreaming up designs over the next week, and welcome help. Basically, "I'm a tester. I want to automate this boring thing. What is my ideal interface to do so / the most beautiful tool I could imagine for it?" One possible source of inspiration: Sugarbot.

* Everybody likes Joe's design which was the basis of our current semantic-mediawiki-based test case management system (http://wiki.laptop.org/go/Test_cases_8.2.0).

* Marco brought up some great questions about the tester/developer relationship. We need to make sure that good bugs get filed when a test fails, and that developers know about the testing going on for the things they're working on (actually, this should be a "people are using my work, yay!" motivation.)

* Suggestions for rewarding community testers: cookies, whippings, whipped creme cookies, beer, wine, vodka, *Russian* vodka (it's special!), Caribbean vacations... we'll see what we can do. ;)

We also agreed to move two conversations to email - prioritizing Activities to test, and then displaying testing metrics in motivating ways. So as to not flood everyone's inboxes, I'll ask Greg and Chris to bring these conversations up over the next few days. :)


'''------- Everything below this line is from the pre-meeting agenda. See the meeting logs to see what was discussed. --------'''


== Previous meeting's action items ==
== Previous meeting's action items ==
Line 7: Line 32:
See [[Testing meetings/20081030#Action items]].
See [[Testing meetings/20081030#Action items]].


== Activity testing ==
== Community testing portal page (lfaraone, mchua) ==

=== Community testing portal page (lfaraone, mchua) ===


We have a [[Community testing]] page! It needs a lot of help...
We have a [[Community testing]] page! It needs a lot of help...


=== Current Activity prep-for-test procedure feedback (mchua) ===
== Current Activity prep-for-test procedure feedback (mchua) ==


Inspired by [[User:Gregdek|Gregdek]]: [[How to test an Activity]] - can we try this out?
Inspired by [[User:Gregdek|Gregdek]]: [[How to test an Activity]] - can we try this out?
Line 19: Line 42:
[[User:Kimquirk|Kimquirk]] requests that we spend some time on quality, metrics, and how people can take ownership of that for specific activities or features of Sugar.
[[User:Kimquirk|Kimquirk]] requests that we spend some time on quality, metrics, and how people can take ownership of that for specific activities or features of Sugar.


=== Status of current Activities being tested (mchua, frandog) ===
== Status of current Activities being tested (mchua, frandog) ==


To cover:
To cover:
Line 26: Line 49:
* [[Speak]] ([[User:Gregdek|Gregdek]])
* [[Speak]] ([[User:Gregdek|Gregdek]])
* [[Paint]]? ([[User:Bjordan|Bjordan]]?)
* [[Paint]]? ([[User:Bjordan|Bjordan]]?)

== Deferred to email ==


=== Prioritizing Activities to test (gregdek) ===
=== Prioritizing Activities to test (gregdek) ===
Line 31: Line 56:
What options/tools are out there?
What options/tools are out there?


Pre-meeting notes on the topic are at [[/Prioritizing activities to test]], add your own here!
''The below is from [[User:Cjl|Cjl]], via email (posted with permission)''

I'm not 100% sure that a whole lot of time thinking it through will yield significantly better results than taking list of activities in images used for large deployments (including both countries and G1G1) and ordering those by which activity author steps up first. Ideally all activities hosted on dev get tested at some point, I don't know that there is a lot of meaningful distinction in which order they get their "first testing".

After a first round, events that would merit a second round of testing would include significant version change (e.g. rebasing on a changed upstream element abiword, xulrunner, etc.), significant build change or new inclusion by a significant deployment e.g. (Conozco uruguay). Ideally community testing is something tha a new user can just jump into solo (after the hard work of building test cases and infrastructure is done), but there are distinct advantages of scheduled joint cooperative activity on that inital round of test case creation and testing. Activity author involvement seems critical to me during this phase.

If there is indeed some interest in gathering real data about usage from users, I do not think any covert means could or should be employed, on the otherhand, there is theoretically a "Poll Builder" activity that could possibly be leveraged to this end. A "customer survey" using it would also serve the purpose of providing a demonstrable use case for this activity to users.
There is some distinction between building the tools for enabling meaningful community testing and efforts to build *community* for community testing, but they are inter-related efforts and done well, these are mutually reinforcing.


=== Displaying testing metrics in motivating ways (cjl, from notes) ===
=== Displaying testing metrics in motivating ways (cjl, from notes) ===
Line 45: Line 62:
What options/tools are out there?
What options/tools are out there?


Pre-meeting notes on the topic are at [[/Displaying testing metrics in motivating ways]], add your own here!
== Test Activity brainstorming (cjb) ==

== Action items ==


* '''mchua''' <s>to post/mail-to-list these meeting minutes</s> '''DONE'''
What kind(s) of Sugar Activities could we make for community testing?
* '''mchua''' <s>to follow-up with outstanding action items from last week</s> '''DONE'''
* '''mchua''' <s>Post an example on [[How to test an Activity]] of how a casual tester can record their test results.</s> '''(done, [[Reporting test results]])'''
* '''adricnet, mchua''' <s>Dream up automation designs to present to group next week. "I'm a tester. I want to automate this boring thing. What is my ideal interface to do so / the most beautiful tool I could imagine for it?"</s> '''DONE - thanks to garycmartin, kevix, and marcopg for joining!'''
* '''gregdek, bjordan''' <s>try out (on Speak and Paint respectively) our [[How to test an Activity]] instructions.</s> '''DONE for this round - again in the next!'''
* '''mchua''' <s>to talk with sj, kimquirk, and others about what kinds of brib^H^H^H^H resources we can offer to motivate and reward testers, both tangible and intangible.</s> '''(Deferred to next week in consideration of Kim and SJ being very busy with G1G1 right now.)'''
* '''gregdek, cjl''' <s>to bring up items that were [[Testing_meetings/2008-11-06#Deferred_to_email|deferred to email]] in conversation in the testing mailing list over the next week.</s> '''DONE'''


== Some notes on infrastructure ==
== Help needed ==


* [[How to test an Activity]] instructions are under severe amounts of construction... are there any volunteers here with exploratory test experience that can help write up this guide?
* How are these meetings working out for people?
* Feedback on meeting time/place/format?
* Are there any ground rules/procedures we should obviously set? (Answer will probably be no this time - that's great, we'll keep an eye out and bring this up again in a few weeks)


[[Category:Test group meetings]]
[[Category:Test group meetings]]

Latest revision as of 06:31, 14 November 2008

This meeting is over.The meeting was in #olpc-meeting on Start date::November 11, 2008 17:00 (Eastern standard time in the USA). Please see the meeting log for more details.

Meeting log

See http://meeting.laptop.org/olpc-meeting.log.20081106_1658.html.

Meeting highlights

  • Greg and Brian are trying out (on Speak and Paint respectively) our http://wiki.laptop.org/go/How_to_test_an_Activity instructions - which are under severe amounts of construction... are there any volunteers here with exploratory test experience that can help write up this guide?
  • How can we encourage smart testers to take ownership of specific Activities and do "deep" testing on them, and come up with their own criteria for and metrics of quality? (As opposed to having community testers spending their time being/recruiting drones to run through scripts.) Greg quoted Patton: "Don't tell people how to do things. Tell them what needs to be done, and let the surprise you with their ingenuity." (Seriously, this was a great discussion - please do read the logs to get the full blast - it starts around 17:28:29.)
  • It was generally agreed that automation was a good idea for reducing drone-ness, which is boring. Ben and Mel will be dreaming up designs over the next week, and welcome help. Basically, "I'm a tester. I want to automate this boring thing. What is my ideal interface to do so / the most beautiful tool I could imagine for it?" One possible source of inspiration: Sugarbot.
  • Marco brought up some great questions about the tester/developer relationship. We need to make sure that good bugs get filed when a test fails, and that developers know about the testing going on for the things they're working on (actually, this should be a "people are using my work, yay!" motivation.)
  • Suggestions for rewarding community testers: cookies, whippings, whipped creme cookies, beer, wine, vodka, *Russian* vodka (it's special!), Caribbean vacations... we'll see what we can do. ;)

We also agreed to move two conversations to email - prioritizing Activities to test, and then displaying testing metrics in motivating ways. So as to not flood everyone's inboxes, I'll ask Greg and Chris to bring these conversations up over the next few days. :)


------- Everything below this line is from the pre-meeting agenda. See the meeting logs to see what was discussed. --------

Previous meeting's action items

See Testing meetings/20081030#Action items.

Community testing portal page (lfaraone, mchua)

We have a Community testing page! It needs a lot of help...

Current Activity prep-for-test procedure feedback (mchua)

Inspired by Gregdek: How to test an Activity - can we try this out?

Kimquirk requests that we spend some time on quality, metrics, and how people can take ownership of that for specific activities or features of Sugar.

Status of current Activities being tested (mchua, frandog)

To cover:

Deferred to email

Prioritizing Activities to test (gregdek)

What options/tools are out there?

Pre-meeting notes on the topic are at /Prioritizing activities to test, add your own here!

Displaying testing metrics in motivating ways (cjl, from notes)

What options/tools are out there?

Pre-meeting notes on the topic are at /Displaying testing metrics in motivating ways, add your own here!

Action items

  • mchua to post/mail-to-list these meeting minutes DONE
  • mchua to follow-up with outstanding action items from last week DONE
  • mchua Post an example on How to test an Activity of how a casual tester can record their test results. (done, Reporting test results)
  • adricnet, mchua Dream up automation designs to present to group next week. "I'm a tester. I want to automate this boring thing. What is my ideal interface to do so / the most beautiful tool I could imagine for it?" DONE - thanks to garycmartin, kevix, and marcopg for joining!
  • gregdek, bjordan try out (on Speak and Paint respectively) our How to test an Activity instructions. DONE for this round - again in the next!
  • mchua to talk with sj, kimquirk, and others about what kinds of brib^H^H^H^H resources we can offer to motivate and reward testers, both tangible and intangible. (Deferred to next week in consideration of Kim and SJ being very busy with G1G1 right now.)
  • gregdek, cjl to bring up items that were deferred to email in conversation in the testing mailing list over the next week. DONE

Help needed

  • How to test an Activity instructions are under severe amounts of construction... are there any volunteers here with exploratory test experience that can help write up this guide?
Retrieved from "http://wiki.laptop.org/mediawiki/index.php?title=Community_testing_meetings/2008-11-06&oldid=180109"