Talk:Friends in testing
Recursing participation
Josh Gay recently gave a great talk at the grassroots bootcamp about on designing for participation in communities, in which he described how to make participant gathering recursive. We should include on this wiki page and in all form emails an action step of "Have any friends with G1G1 or developer XOs? Forward them this (wiki page, email, etc)."
I've added that to the wiki page.
Remembering the Goal
Please keep in mind that the goal of this testing is to determine if the software on the XO is complete and flexible enough to connect to a wide variety of wireless access points, or alternately, to gather information on the range of wireless access points to which the XO cannot easily connect.
The end user may not have the option to modify the configuration of a given wireless access point, may not be the owner of a particular wireless access point, and may not even know the model or brand details.
As a standard for declaring test success or failure, I'd cite the OLPC Connecting guide:
http://laptop.org/en/laptop/start/connecting.shtml
If it's possible to connect to a wireless access point with a certain configuration using only the information on this page, then the XO software can be declared adequate for that brand/model/configuration. If additional steps (for example, using a HEX-conversion of the passphrase, or running an accessory script like Wpa.sh) is required, that information is useful and should still be recorded, but the connectivity test must be declared a failure.
Log capturing
Hi Greg - how about this? annegentle 02:29, 7 October 2008 (UTC)
Here's how you capture a log file and attach it to a trac ticket from an XO laptop.
- Open the Terminal Activity.
- Type "olpc-log" and press enter. Sugar creates a file starting with log.
- In the Terminal Activity, type copy-to-journal -m application/x-bzip-compressed-tar log-filename and press enter.
- Open the Browse Activity, navigate to a trac ticket that needs a log file attached, and attach the log-filename file using the Journal window that opens when you click the Attach button in the Browse Activity.
Hi Joe et al,
It would help to put a link to the log capturing instructions on this page. I'll put one here when I find it....
Thanks,
Greg S
Software Update / Activities not working in candidate-765?
Clean Installed candidate-765 last night (9/29) after successfully using joyride 2456; went to Software Update to install Activities, but dialog just says "cancel"; no change after 5 minutes. Found this [|dev ticket] which looks active, and says that Activities folder is being deleted. Can't tell (I'm not good at Linux. Sigh.), but I know that this didn't go right. Not sure if current separate install of G1G1 Activity Bundle is advisable or possible. I've stopped for now and will check back about this build later.
Control Panel > About My XO > Build 765; Sugar: 0.82.1; Firmware: Q2E18
--- Liz @ G1G1 @ PDX
General notes for testing early releases
Caveats
This procedure asks you to install alpha software. You may lose all your data. Please back up personal files to another source if you would like to keep them.
Preparation
- Get a developer key for your XO laptop
- Backup your XO (optional -- you decide).
- Clean-install or upgrade your XO to the test build.
- If you clean-install, you will lose all your data.
- Install activities with your favorite activity installation method.
- Help test the OS and activities as described below.
- Should you need them, recovery instructions are available.
Several other test preparation instructions are available, for example in OS images and in Emulating the XO, and other pages in Category:Preparing for testing.
Exploratory Testing
- Known issues are recorded in the release notes for a release; check both the current stable release's notes and/or the in-progress release notes for the development release you're testing.
- Sometimes people enumerate problems in Test group release notes.
- People report problems on the testing mailing list
You should report new issues in our bug-tracking system. You can also send any issues or comments to the development list, devel at lists.laptop.org.
Test 1: Wifi Testing
Can Current test image: 11.3.1 associate with your (encrypted?) access point?
If not, let us know!
Test 2: Explore the Smoke Test
Try out parts of the release's smoke test that appeal to you.
(Older tests are available.)
Systematic Testing
In addition to our freeform exploratory testing effort, we also engage in systematic testing efforts designed to achieve the test coverage necessary to make release decisions around future releases, much like past releases.
Systematic testing consists of several basic tasks:
- Reviewing a test plan.
- Writing the test cases, if needed.
- Executing the test cases and recording test results.
- To record results on a particular test case page, click on the "Edit with form" button in the top of the page, and use the "Add Another" button just above the "Add New Results" section (i.e. at the bottom of the page).
- Contributing patches to the Tinderbox or Sugarbot automated testing platforms.
Test 1: Multiple Key Support
We are particularly interested, this week, in systematically testing OFW's new multiple key support.
Recovery
- If your machine becomes unbootable after a clean-install, you can recover by clean-installing an official build.
- If your machine becomes unbootable after an upgrade, you can recover by holding the 'O' gamepad key during your next boot.
Alternately, you could help test an even newer Joyride build!
Greg Smith's notes
Note that not every smoke test sub test has to pass to let it out for broader testing; maybe one round of exchange on results of smoketest is in order before deciding if its good to go on.
When we're in full release mode we do a daily bug triage and look at all incoming issues. Some get comments, some get deferred, some documented in release notes and some assigned for resolution. It's helpful to have good bugs which don't need a lot of back and forth to understand but its not critical
What does "test build" mean? The main point is to know if something that fails is a new bug or just a known issue. 8.2.1 test plan may help.
8.2.1 is targeted for mid-January release, which is day the release notes get flagged as "final" and code goes to manufacturing (if needed, not sure yet if it will). We won't have a precise date until next week, minimum. 8.2.1 is a focused bug fixes only release. It does have some changes in kernel and wireless so it would benefit from some regression testing (aka open source teams gang up on it) but hopefully we can bound the changes. (This is like practice for 9.1.0.)
Release notes will be posted here: http://wiki.laptop.org/go/Release_Notes/8.2.1. You can look at that link to see the 8.2.1 target bugs and consider how we would verify each as its marked resolved.