Community testing meetings/2008-12-11

  This page is part of the OLPC Community testing Project. How to test an Activity | Reporting test results | Meetings

This is a Community testing meeting. Location is in #olpc-meeting on Start date::December 1, 2008 23:00 (time is in UTC, click here to find the meeting time for your time zone.)

Previous meeting's action items

See Community testing meetings/2008-12-04

Activity test case reporting framework

We have one. Let's make sure everyone knows how to use it / is scheduled to learn, and that it's up and documented appropriately on the wiki.

Testing Jams

Testing Jam ready to go? What do people need?

Mchua notes she has groundwork to do on this before the meeting starts

Joyride/8.2.1 Testing: The Challenge

<mchua> we're coming up on our interim release, 8.2.1 (and longer-term,
what Gregoriov2 was talking about, 9.1)
<mchua> and joyride needs some testing love; it's been ignored
<mchua> (joyride being the bleeding-edge build, I think the Fedora equiv
is rawhide?)
<mchua> we have (warning, rough page):
http://wiki.laptop.org/go/Friends_in_testing
<mchua> which, thanks to the heroic efforts of m_stone and joef, was
glorious for 8.1.0 testing
<mchua> it's meant to be an "you have an XO, and some time this week? help
us hammer on our new build and Break It!" thing
<mchua> done in weekly cycles, no commitment beyond a single week unless
you want it
<cjb> (and it's particularly interesting at the moment, because we just
moved from F9 to F10)
* mchua found the email from cjb on this
<mchua> "We aren't testing the joyride builds, which contain the F10
rebase work we've done so far.  This means that this work will take
longer, because bugs will take longer to be reported, and might also
mean that some bugs
<mchua> will go unfixed because we'll run out of time to fix them at the
end. It would be much better to have testing happen while the development
is ongoing, not all in a bunch near the end of the release."
<cjb> In particular, it occured to us that #fedora-olpc folk probably
know much more about "stuff that breaks when you move your distro to
F10 from F9" than we do :)
<gregdek> mchua: So I know we can try to leverage the F10 Testers we
recruited by sending them XOs.  Shall I prepare that list for recruitment?
<mchua> That would be wonderful.
<gregdek> It's about 70 folks.  I'll do my best rounding them up.
<mchua> The other thing is that Michael and I are looking for someone
to run Friends in Testing.
<mchua> At least for the 8.2.1 cycle.
<mchua> So, for 8.2.1, we're doing an experiment (this is very recent,
as of yesterday)
<mchua> The question is: "For the given expenditure of OLPC's resources
vs the quality of output received, does community testing do better than
internal QA testing, or vice versa?"
<mchua> I'm biased; my bets are on community.
<mchua> But we Don't Know Yet.
<mchua> We'll set up the same test cases, same reporting structure,
same metrics-of-goodness, etc. for both, and run all of the same tests
(including Friends In Testing joyride tests)
<mchua> and clock internal-QA-hours-spent on both facilitating community
testing and doing internal testing ourselves
<mchua> and see.
<gregdek> Pepsi Challenge!
<mchua> exxxxactly.
<mchua> (in practice, I think most of the "facilitating community testing"
hours from internal QA are going to come from me, but... yeah.)
<mchua> So you can see why I'm beating the "can someone do Friends in
Testing for 8.2.1 please please please that isn't an OLPC employee?" drum.
<gregdek> To kick things off: mchua, please draft a letter to our 70
testers, and I will review and send along.
<cjb> mchua: does it have to be a single person?
<mchua> cjb: strictly speaking, no; it'll make it easier for me / take
less time for me to coordinate with them, though.
<mchua> (that's all I had)