Community testing meetings/2009-02-05
This page is part of the OLPC Community testing Project. How to test an Activity | Reporting test results | Meetings
This was not a formal community testing meeting, as none was called for this particular date. However, logs of the conversation in #olpc-meeting have been published for posterity.
<mchua> as i dropped the ball this week and didn't preannounce a testing meeting, there isn't a formal one (not one hosted by me, in any case) - however, I'm around in case people have things they need done. <mchua> m_stone: any new testing requests for this week for 8.2.1? <m_stone> mchua: yes. <m_stone> mchua: dsd turned up some specific wifi regressions, which he documented in #.... <m_stone> something. * rgs_ has quit (Remote closed the connection) <mchua> Ok. I haven't quite caught up with all my travel backlog, so that one might have escaped me - was it on the testing list? <cjb> http://dev.laptop.org/ticket/9235 <cjb> mchua: we should have a proper real candidate image soon, like tomorrow <mchua> cjb, m_stone: cool, thanks! and it'll be about a week between candidate images again, so we have a week to attack testing on it? * pgf_away is now known as pgf <cjb> mchua: yup <cjb> probably the next week's image will have some wireless changes reversed, so we can ask people which build works better for them <mchua> cjb, m_stone: ok - 2 last questions: (1) can people with individual XOs test, or will the results really be not-so-useful to you unless they have multiple XOs? and (2) is firmware still the Area Of Big Attackingness, or is there Something New (like verifying dsd's bug / trying it with different AP hw/ etc)/ <cjb> so a release schedule might look like: <m_stone> hello again. <m_stone> distracted in #olpc-devel <cjb> Fri 6: first candidate <cjb> Fri 13: revert wifi changes, second candidate <cjb> Fri 20: decide which of first/second/other wins, release final candidate <mchua> m_stone: np, I see you're rather busy there <cjb> Fri 27: begin final release <m_stone> cjb: here's a slightly different proposal: <m_stone> (see what you think) <cjb> that's the first time I've put a schedule together for it, anyway :) <m_stone> this week, we need to track down the root cause of the wifi regressions. <m_stone> a good way to start doing that is to do a downgrade test on each of the kernel and the firmwar.e <m_stone> /hopefully/ it will be clearly one or the other. <m_stone> if it's not, then you can do a more fine-grained test by bisecting the kernel commits and testing each bisection choice both with and without the new firmware. <m_stone> so that's test problem #1. <m_stone> test problem #2 is etoys. <m_stone> if you want to ship a new etoys, then someone needs to do at least basic sanity testing. <cjb> yes. I extracted such an offer from Bert. * SMParrish has quit (Remote closed the connection) <m_stone> good, but lets involve more such people. <m_stone> bert knows what he's looking at. <m_stone> it's good to have people who don't as well. <m_stone> test problem #3 is general smoke testing. <cjb> ok. this all sounds good so far. <m_stone> which I think will go fine, but which needs to be continued and which should be done from the final build stream that you intend to release. <m_stone> just to deal with "did pilgrim put the right strings in the right places..." ? <cjb> yeah. I need to set up the 8.2.1 stream, pulling from the staging repo. <m_stone> and I propose that #4 should be continued firmware testing. <cjb> Are there tools to do that, or does one copy the repo directory into place? <m_stone> the 8.2.1 stream already exists. <cjb> right, but it has the wrong bits in. <m_stone> but rather than making it pull from the staging repo, you should copy the RPMs from the staging repo into the 8.2.1 mock branches. <cjb> right <m_stone> (personally, I don't think we actually know enough to set a release date yet.) <cjb> oh, yes, I wasn't attempting to announce one. <m_stone> but, as you recall, I am notoriously shy about doing so. <mchua> ok - in terms of getting people to do all this, #1 sounds like a "one person / small group takes this and just does it" job, not sure who that should be (if dsd can do it, if cjb would be fastest, if m_stone or I or someone else should attack this at 1cc or otherwise monday or tuesday or somesuch) <mchua> #2 can probably be sent to the welly testers as their sole "please get this done tomorrow" task <m_stone> cjb: test problem #4, by the way, is continued security-related testing of OFW. <m_stone> and the irfs changes dsd has proposed. <mchua> #3 would be a good intro for the sri lankan test team that CanoeBerry is talking to tomorrow morning - they might be able to come up with a better smoke test procedure (and way to log results) than our current 4-hour mess <m_stone> mchua: agreed. #1 would go a lot faster if you could get someone like dsd or dsaxena to help. <mchua> and #4 can be a general "plz halp" call, I think <cjb> I think dsd's expressed some interest in helping <cjb> (which would be fabulous) <m_stone> mchua: #3 should just be the "four hour mess". <cjb> since he's already found one disaster bug in there. <m_stone> mchua: the reason is that we have data from previous images for comparison. <m_stone> mchua: changing the test procedure just means that you can't get compare your results properly with previous work. <mchua> ok, so let's ask dsd if he can tackle #1; does he already know what needs to be done on it (since's he's already found bugs and the like) or is he/whoever going to have to catch up with someone on this before they start (and if so who?) <cjb> I think he knows more about the problem than any of us, at this pount <cjb> *point <m_stone> cjb: certainly, he does. on the other hand, he does have some other commitments. <m_stone> so we should be prepared to look for other sources of help. <mchua> m_stone: what I was going to suggest is that the sri lankan team (they're QA professionals, and create test procedures for a living) smoke-test 8.2.0 and the 8.2.1 candidate in parallel <m_stone> mchua: that sounds fine, but we /still/ need to do an actual "one our smoke test" on whatever we intend to ship. <m_stone> period. <m_stone> (in my opinion, anyway) <mchua> Right, and we can do that on a later image that we actually think that we might ship. <m_stone> mchua: the sri-lankan team can do whatever they're best at. <m_stone> we're not doing our jobs if we aren't smoke-testing the images we're producing in release-stream builds. <m_stone> and asking for others to do the same. <m_stone> (that's how I feel, anyway; but feel free to disagree) <m_stone> (it's not that the tests can't change; it's just that we have to demonstrate to our mutual satisfaction that we /actually tested/ the thing. and saying "I did a smoke test" is a really good mental indicator, for all of us, of what was tested and what wasn't.) <m_stone> since we have lots of experience with what that phrase means. <m_stone> so who writes the mail? <mchua> asking the sri lankan group to do/improve smoke testing in parallel doesn't rule out someone else doing the old n-hour smoke test. it can still be on the todo list. <m_stone> mchua: yes; I'm simply claiming that it /must/ be on the todo list. <mchua> m_stone: sure. did you want to send out the to-do list, then? ;) prioritization of #1-#4 would be a good thing to have, as my best guess is 1, 4, 3, 2 but you would have a better handle on what that ordering should be. <m_stone> as you wish. <m_stone> is staging-25 still the correct image for testing? <cjb> staging-26 <cjb> but tomorrow we get a candidate build <cjb> so people should wait and test that <m_stone> cjb: in that case, we should wait to announce the testing request. <m_stone> (I want to update [[FiT]] <m_stone> ) <cjb> yes, I agree <cjb> you could consider updating the <cjb> page to reflect candidate-foo, and sending out the mail tomorrow when we know the build details <m_stone> I'll just send a mail today saying that we'll send more mail tomorrow <cjb> I think I don't like that idea so much <cjb> although this doesn't mean you shouldn't do it <mchua> I'd post this log to the testing list now for those who might be curious what happened (since we usually send out testing mtg minutes thursday evenings) <cjb> I guess there are two schools of thought; I tend to think we get limited opportunities to impact people's consciousness, and we shouldn't waste them with an e-mail that says we'll e-mail them tomorrow <cjb> oh, yes, that's a good point <cjb> mail to the testing list is fine <mchua> (and at the top have the "here are #1-#4 things to work on, more detail tomorrow and you can safely ignore this email 'till the meantime.) <cjb> it's the more widely sent mail that should be delayed <mchua> +1 <cjb> (devel, support-gang, etc)
