XOCamp 2/QA

Jan 12 15:37:31 <mchua> Folks watching via camera,
I had to move the laptop to get at the cable - right now
http://wiki.laptop.org/go/XOCamp_2#QA is being displayed (and that's
what will be projected throughout). Let me know if you're okay.
Jan 12 15:37:33 <mungewell2>	OK must have been finger/brain
trouble. I'm in and listening to music waiting for chairperson.
Jan 12 15:37:48 *	bertf has quit (Connection timed out)
Jan 12 15:38:06 *	mchua is no longer on this channel, must turn
attention into being physically present at the conversation; will get
someone to transcribe.
Jan 12 15:40:13 *	kevix (n=kevin@68.237.104.10) has joined #olpc
Jan 12 15:41:06 <mchua> bjordan and edcherlin are on transcription -
if you are in this channel and having trouble keeping up, please yell
at them. :)
Jan 12 15:41:08 <Mokurai>	I'm physically present, and also on
this channel
Jan 12 15:41:13 <mchua> Mokurai == edcherlin
Jan 12 15:41:16 *	mchua out
Jan 12 15:41:18 <Mokurai>	Rigt
Jan 12 15:41:19 <olpc_wad>	This is strangely recursive.  I'm watching
a video of my typing....
Jan 12 15:41:20 *	mchua sets mode -e mchua
Jan 12 15:41:20 *	You are now known as mchua|in-real-li
Jan 12 15:41:25 *	You are now known as mchua|irl
Jan 12 15:41:35 <prakhar>	:P
Jan 12 15:43:02 <bjordan_>	can people hear?
Jan 12 15:43:06 <pgf>	olpc_wad: turns out teapot's ubuntu + firefox +
flash is fast enough on the XO.  until it crashes due to lack of memory.
:-)
Jan 12 15:43:08 <unmadindu>	yes
Jan 12 15:43:10 <prakhar>	bjordan_: yup
Jan 12 15:43:18 <unmadindu>	hello :)
Jan 12 15:43:19 <cjb>	can you hear really well, or kind of okay?
Jan 12 15:43:24 <olpc_wad>	Excellent!
Jan 12 15:43:32 <olpc_wad>	(in reply to pgf)
Jan 12 15:43:38 <unmadindu>	it's pretty well for me
Jan 12 15:43:43 <olpc_wad>	audio is low, but ok
Jan 12 15:43:51 <prakhar>	yeah, low, but ok
Jan 12 15:44:11 <pgf>	i think everyone is whispering.  really.
Jan 12 15:44:18 <olpc_wad>	cjb nostril cam
Jan 12 15:44:26 <prakhar>	lol
Jan 12 15:44:44 <mungewell2>	phone conference just went live...
Jan 12 15:44:54 <cjb>	I think I made the audio louder, but maybe not
Jan 12 15:45:03 <_sj_>	well, we have 20 people watching us on justin.tv
Jan 12 15:45:08 <_sj_>	and hopefully some of you on the conference line
Jan 12 15:45:10 <bjordan_>	Mel's hoping to get a lot of feedback
today because she's humble ;)
Jan 12 15:45:14 <_sj_>	though I've just been geetting echoes
Jan 12 15:45:26 <mungewell2>	a bit choppy, maybe my voip connection.
Jan 12 15:45:27 <bjordan_>	http://wiki.laptop.org/go/XOCamp_2#QA
Jan 12 15:45:29 <_sj_>	 mungewell2, around?
Jan 12 15:45:36 <_sj_>	good
Jan 12 15:45:49 <_sj_>	mel (speaker) is at the end of the room away
from the conf line mic
Jan 12 15:45:57 <Culseg>	move mic closer to Me's voice
Jan 12 15:46:21 <bjordan_>	mchua wants to make sure testing will
happen for 8.2.1, and figure out what resources OLPC, Sugarlabs and
Fedora have for testing
Jan 12 15:46:40 *	BryanWB (n=BryanWB@dhcp-49-146.media.mit.edu)
has joined #olpc
Jan 12 15:47:02 <pgf>	Culseg: moved.	out of cord.
Jan 12 15:47:11 <bjordan_>	how much QA needs to be done? how much
can OLPC do? (none) how much can the community do?
Jan 12 15:47:24 <cjb>	hasn't been a lot of XO-specific testing done
outside of OLPC
Jan 12 15:47:31 <buendia>	a-b=c
Jan 12 15:47:38 <cjb>	trying to migrate that, hand-drawn diagram on
the XOCamp_2 wiki page
Jan 12 15:47:58 <bjordan_>	3 kinds of testing (1) integration
(fedora + SL), (2) testbeds (OLPC), 3() acceptance (deployments) [see
wiki page diagram]
Jan 12 15:48:35 <bjordan_>	system level testing  -- you guys should
run the release on your XOs on your own at home
Jan 12 15:48:57 <bjordan_>	testing touchpads
Jan 12 15:49:24 <bjordan_>	(3) testing within the deployment
process... bringing that feedback to developers. making sure we have the
capacity to deploy and use whatever is being released during deployment
Jan 12 15:49:38 <bjordan_>	these are the things QA should be doing
as a community -- ed?
Jan 12 15:50:03 <bjordan_>	ed: one of the things we should do is
become more agnostic about the distros used on the XO
Jan 12 15:50:25 <bjordan_>	for the forseeable future, we have the
unique situation that OLPC does govern what software is released *from
the factory*
Jan 12 15:50:42 <bjordan_>	we have some level of responsibility
regarding what is released from the factory
Jan 12 15:51:03 <bjordan_>	testbed situation -- while OLPC is not
unique, OLPC needs to provide resources to the broader community where
that makes sense
Jan 12 15:51:19 <bjordan_>	one of the things that is difficult
for many of hte people who want to help us test is creating a 50 laptop
testbed (something most of us can't do)
Jan 12 15:51:45 <bjordan_>	this is somehting that we should continue
to do
Jan 12 15:51:58 *	behdad has quit ("Later")
Jan 12 15:52:13 <bjordan_>	we want to push more acceptance testing
into deployments
Jan 12 15:52:35 <bjordan_>	"you're responsible for the things on
your machine, use what you're comfortable with using"
Jan 12 15:52:50 <Orange-Green-98>	can people hear?
Jan 12 15:53:15 <bjordan_>	ed: i've been asked why we ship anyhting
on the machine.. why not just firmware
Jan 12 15:53:49 <bjordan_>	we need to show something runs on the
machine... more and more it will be the case that most deployments will
wnat to customize their machines... we're moving towards that deployments
want to reflash their machines and customize them
Jan 12 15:54:15 <bjordan_>	cjb: none of our deployments use the
same software as any of the other deployments
Jan 12 15:54:47 <bjordan_>	cjb: shows that open source is good in
that people aren't content with uncustomized software and can change that
Jan 12 15:55:02 <olpc_wad>	Quanta insists on shipping software on
the laptop.
Jan 12 15:55:04 <bjordan_>	cjb: costs a lot of money to reflash
100k machines
Jan 12 15:55:24 <bjordan_>	ed: for large deployments, we're moving
more towards them giving us an image to put on (in the factory)
Jan 12 15:55:40 <bjordan_>	ed: you have to move to tens or hundreds
of thousands of machines for that to happen
Jan 12 15:56:19 *	aa (n=olpc@dhcp-49-144.media.mit.edu) has
joined #olpc
Jan 12 15:56:28 <bjordan_>	marco: e.g. we ship fedora... instead of
doing testing irectly at OLPC, can we get fedora community doing testing?
Jan 12 15:56:46 <bjordan_>	mel: yes, please
Jan 12 15:57:01 <bjordan_>	ed: we sent laptops to europe to test
f10 on XOs
Jan 12 15:57:31 <bjordan_>	ed: we have a large inventory of XOs
available for testing
Jan 12 15:58:17 <bjordan_>	mchua: history -- we have a test
community. things we're doing aren't necessarily the things we should
be doing, some of those should be pushed upstream
Jan 12 15:58:23 <bjordan_>	refocus that effort, or send it to
other places
Jan 12 15:58:40 <bjordan_>	8.2.1 - 17 tickets still left
Jan 12 15:58:43 <bjordan_>	5 in a stage where we can test them
Jan 12 15:58:52 <bjordan_>	bpepple will be helping a lot with that
Jan 12 15:59:12 <bjordan_>	we need to come up with a procedure of
how we can have a community tested release
Jan 12 15:59:55 <bjordan_>	the big question: who do you need to
sign off on stuff being tested?
Jan 12 16:00:11 <bjordan_>	ed: for 8.2.1 release, the answer will
be me
Jan 12 16:00:19 <bjordan_>	marco: what do you mean by signed off
Jan 12 16:00:26 <bjordan_>	ed: that the bug will be closed, and
we'll ship this
Jan 12 16:01:01 <mungewell2>	Orange-Green-98 : phone conference is OK,
a bit choppy by I think that's my connection more than anything.
Jan 12 16:01:18 <bjordan_>	ed: 8.2.1, a lot of those fixes are
for Uruguay
Jan 12 16:01:29 <bjordan_>	reuben will help with coordinating that
Jan 12 16:01:36 <Orange-Green-98>	it is slightly better now
Jan 12 16:01:46 <Orange-Green-98>	I can even hear mel if I strain
Jan 12 16:02:08 <bjordan_>	who is speaking?: in Uruguay, there
would be a lot of people who are eager to do testing
Jan 12 16:02:38 <aa>	bjordan_: Luis Michelena
Jan 12 16:02:47 <bjordan_>	it would be good for htem, they would
have a chance to test, a way to make QA a big project
Jan 12 16:03:35 <bjordan_>	there would be a lot of things that
Fedora can test, but there will be a need for tests from deployments
Jan 12 16:04:01 *	appletizer has quit ()
Jan 12 16:04:09 <bjordan_>	and that will fulfill OLPC's need for
feedback... and provide a different viewpoint for testing
Jan 12 16:04:40 <bjordan_>	who to contact re: testing?
Jan 12 16:05:20 <olpc_wad>	Ed McNierney is in charge of testing
Jan 12 16:05:23 <bjordan_>	mel: my proposal is that OLPC spend
money for testing/beds
Jan 12 16:05:34 <bjordan_>	olpc_wad: sorry, meant in Uruguay
Jan 12 16:05:53 <olpc_wad>	In LATU on in their large volunteer
community ?
Jan 12 16:06:01 <olpc_wad>	s/on/or/
Jan 12 16:06:04 <bjordan_>	Bryan: ejabberd is the #1 thing we have
that needs testing
Jan 12 16:06:10 <bjordan_>	but it doesn't work reliably
Jan 12 16:06:17 <olpc_wad>	And Uruguay isn't using it yet.
Jan 12 16:06:28 <olpc_wad>	They are starting to play with it,
however.
Jan 12 16:06:31 <bjordan_>	edmcnierney: OLPC is not going to fix
those bugs
Jan 12 16:06:32 <aa>	olpc_wad: uruguayan community
Jan 12 16:07:12 <bjordan_>	ed: ther eis testing and fixing that
goes on that fedora resolves
Jan 12 16:07:33 <bjordan_>	Bryan: deployments can put hteir own
resources into testing
Jan 12 16:07:42 <bjordan_>	pay sugarlabs to fix bugs
Jan 12 16:08:13 <bjordan_>	Bryan: is OLPC ok telling people that?
Jan 12 16:08:36 <bjordan_>	it's important that OLPC says, go to
these guys, they are worht the money
Jan 12 16:08:50 <bjordan_>	as a deployment, want someone to
solve bugs
Jan 12 16:09:09 <bjordan_>	mel: want to hear of other organisations
that you cna go to
Jan 12 16:09:28 <anna_bham>	I can tell you right now Birmingham
doesn't have the money for that
Jan 12 16:09:40 <bjordan_>	marco: how are volunteers/deployments
going to interact as a community re: testing
Jan 12 16:10:04 <bjordan_>	mel: that's still an open question,
but it's a question that is able to be solved right now... what things
do we need to do to allow these open questions to be solved?
Jan 12 16:10:55 <_sj_>	mel: we have a list of tests that can be run
and a place for people to go...
Jan 12 16:11:04 <cjb>	(Test_cases_8.2.1 on wiki)
Jan 12 16:12:01 <_sj_>	cjb: mel can't solve all of these problems;
marco: it's fine to set up olpc testing labs, but have to direct the
bugs to the right project
Jan 12 16:12:20 <_sj_>	the fixing is by the maintaining
project... [testers may come to help test <foo> on OLPC]
Jan 12 16:12:55 <_sj_>	see more notes directly on the [[XOCamp 2]] page
Jan 12 16:14:26 <_sj_>	jonas s : by helping testing... you are the
only ones who know what is specific to olpc, what is specific to linux.
if you can both say thanks for telling me! that helps testers
Jan 12 16:14:38 <_sj_>	then send people directly to another group,
if they want more than thanks / want more control and followup.
Jan 12 16:15:10 <_sj_>	ed: a small but helpful ex was f10 on xo.
a fedora community tested, reported, fixed, and updated; making f10 on
xo work
Jan 12 16:15:22 *	prospekt_
(n=prospekt@pool-70-110-238-118.phil.east.verizon.net) has joined #olpc
Jan 12 16:15:28 <_sj_>	with little olpc direction (including the
thaking/redirecting).
Jan 12 16:15:51 <_sj_>	welcome prospekt_, kevix, bwy
Jan 12 16:16:26 *	prospekt_ has quit (Client Quit)
Jan 12 16:18:06 <bjordan_>	_sj_: we gave latpops to wellington,
moodle in europe, tests in specific countries before, shortest timeframe
was f10, aggressively soliciting 70 people and sending them laptops
instantly for testing
Jan 12 16:18:28 <bjordan_>	if we're promoting doing something
regularly, that's easy to do, we've been doing that for six months
Jan 12 16:18:41 <bjordan_>	q: were people given tests to run?
Jan 12 16:18:53 <bjordan_>	sj: f10 was the first time we had a list
of tests to run
Jan 12 16:19:16 <bjordan_>	q: what is being tested? (activity,
sugar, hardware level?)
Jan 12 16:19:38 <bjordan_>	sj: we found groups that had speicfic
testing goals... e.g. moodle/XS testing (they set up their own rtest
plans)
Jan 12 16:19:46 <bjordan_>	sj: the f10 release was the first time
we tried to impose a testing regimen
Jan 12 16:20:32 *	reubencaron (n=reubenkc@dhcp-49-31.media.mit.edu)
has joined #olpc
Jan 12 16:21:41 <bjordan_>	ed has to go, we have another talk
Jan 12 16:21:54 *	edmcnierney is now known as edmcnierney_away
Jan 12 16:23:36 bjordan_ buendia bdale` befana bemasc benzea bertf_
bigeasy birunko bjordan Blaketh bluefoxicy BryanWB bwy
Jan 12 16:23:38 <mchua|irl>	thank you bjordan_!
Jan 12 16:23:39 *	You are now known as mchua
Jan 12 16:23:39 *	services. sets mode +e mchua
Jan 12 16:24:21 <bjordan_>	thank YOU mchua
Jan 12 16:25:28 *	seumassoltysik has quit ()
Jan 12 16:26:21 *	mchua breathes
Jan 12 16:26:37 <mchua> ok - some things that might not have come
through in transcription (sorry about the mic problem, btw, didn't know
about that!)
Jan 12 16:26:57 <mchua> (1) I was deliberately avoiding addressing a
lot of huge, important problems in QA we have not solved yet
Jan 12 16:27:01 <olpc_wad>	The audio on the video was marginally
better than the audio teleconference
Jan 12 16:27:15 <mchua> like "how do we get reported bugs fixed by
$somebody" and "how do testers know what to test?"
Jan 12 16:27:27 <mchua> because those are, yes, problems that we need
to solve, and that we have not yet solved well
Jan 12 16:27:29 <olpc_wad>	I think backing up and rethinking the
purpose of QA is essential.
Jan 12 16:27:36 <mchua> but they are problems that we *can* solve with
the current situation
Jan 12 16:27:38 <mchua> olpc_wad +1
Jan 12 16:27:54 <mchua> I was attempting to address what I saw as the
problems that we needed to solve that were *insoluble* given the current
situation
Jan 12 16:28:21 <mchua> (i.e. "testing needs to be
done with large groups ~50 XOs" - "but we don't have any
readily-accessible-by-anyone-in-the-community 50 XO testbeds outside
of 1cc."
Jan 12 16:28:24 <mchua> )
Jan 12 16:28:46 <mchua> (while such testbeds exist, there are no good
interfaces for Random Volunteers to get good work in a known manner out
of them.)
Jan 12 16:28:48 <anna_bham>	I can ask if a group of older children
here might be willing to do something like that
Jan 12 16:28:59 *	Quozl
(n=james@CPE-124-186-170-71.qld.bigpond.net.au) has joined #olpc
Jan 12 16:29:03 <mchua> anna_bham: do you folks have enough equipment
to test on for that? if so, that would be awesome.
Jan 12 16:29:10 <anna_bham>	But that would have to be testing from
an end user perspective
Jan 12 16:29:15 <mchua> anna_bham: My dream is testbeds staffed by high
school and college students, actually.
Jan 12 16:29:20 <anna_bham>	There are 15K XOs here.  I think there
are enough
Jan 12 16:29:26 <mchua> anna_bham: *grin* true, that.
Jan 12 16:29:35 <bemasc>	I don't think you need to ask kids
to break the computer.	They'll happily do that whether you create a
structure or not.
Jan 12 16:29:52 <bemasc>	anna_bham: oh, wow!  I'm glad that's
happening!
Jan 12 16:29:54 <anna_bham>	mchua, I'd prefer older teenagers,
but if we're onsite at a school, there are security issues with that
Jan 12 16:29:55 <mchua> anna_bham: hm - I think that high school and
college kids are more than capable of doing - and running and designing
and managing, eventually - quite advanced technical/structured testing.
Jan 12 16:30:39 *	dirakx has quit (Remote closed the connection)
Jan 12 16:30:40 <mchua> anna_bham: do you have 50 XOs you might be able
to allocate to a local university for a testbed hosted there and formally
run by a group of their students/profs?
Jan 12 16:30:49 <_sj_>	I'm taking notes now;
Jan 12 16:30:53 <anna_bham>	mchua, No, there aren't 50 extra
Jan 12 16:31:01 <mchua> anna_bham: ok, we should figure something out
for you then
Jan 12 16:31:05 <mchua> I'l be back in ~30min, sugar talk starting
Jan 12 16:31:10 <_sj_>	on he whiteboard.  marco and bernie are talking
about time-based releases for sugar
Jan 12 16:31:18 <_sj_>	and implications for olpc builds, 0.82 and 0.84
Jan 12 16:31:32 <prakhar>	"I void Warranties" :)
Jan 12 16:31:40 <cjb>	how's the audio now?
Jan 12 16:31:53 <cjb>	I think justin.tv must be doing most of the
audio crappiness
Jan 12 16:31:54 <cjl>	_sj_ who is black shirt, who is sweater, who is
blue shirt?
Jan 12 16:31:58 <bjordan_>	justin.tv takes a while to generate
clips :-/
Jan 12 16:32:09 <bjordan_>	cjb: definitely
Jan 12 16:32:15 <BryanWB>	cjl, sj is blue shirt, sweater is marcopg
Jan 12 16:32:19 <bemasc>	anna_bham: In my view, there's not
much use in testing software versions that have already been released.
It's too late to solve those problems.	The interesting thing is to test
pre-release/release candidate software.
Jan 12 16:32:26 <BryanWB>	cjl, blk shirt is bernie
Jan 12 16:32:39 <cjl>	BryanWB: thanks,
Jan 12 16:33:01 <bemasc>	so the question is: could you find a
group of (e.g.) 7th-graders who are willing to run new, exciting, but
potentially buggy software, and report back when they hit a problem?
Jan 12 16:33:04 <_sj_>	(this is mchua typing on sj's keyboard, trying
to transcribe)
Jan 12 16:33:08 <prakhar>	BryanWB: thamks!
Jan 12 16:33:19 <BryanWB>	bemasc, there is a use for deployments
to test released software so they know what problems they will encounter