Nepal: XO Testing Plan

From OLPC
Jump to: navigation, search

This page is out of date. See Nepal:_Testing_and_QA for a the full testing, QA, and config document. Berrybw 23:08, 15 April 2008 (EDT)


The OLE Nepal team will use this document as the basis for acceptance testing for new XO's. This plan will test for hardware and software problems. Berrybw 06:55, 14 March 2008 (EDT)

We are using this document on a daily basis. If you have a tried-and-true solution you are welcome to post it here. Please keep suggestions or points for discussion onto the Talk Page that will make this document more easy to use.

This plan is overkill if overkill if you are testing only a few XO's but you may find it useful if you are testing 100's.

The new XO's will be kept in the OLE Nepal storage room and not moved until testing is complete. Otherwise they will mingle w/ our other XO's and everything will be confused. We will test them in batches of 24 XO's at a time. (Why 24? because they are stacked 6 high.)

We will test the XO's over the period of a week and we will actually spend most of this time testing the Update.1 XO Build with customized activities for Nepal.


Stage 1 - Receipt of XO's

Stage 1 will be done by Upaya, Bryan, and Sulochan. Covers receipt and basic testing

  1. At airport, count # of XO's if individually packed
  2. Upon arrival at office, recount XO's and note any discrepancies
  3. Inventory the XO's in a spreadsheet. The spreadsheet should record the XO's serial #, whether it shows any physical damage, or won't boot. Here is a inventory template spreadsheet you can use File:XO-Inventory-template.xls
    • If there is physical damage, write "physical damage" on the box w/ a red marker. If it won't boot, write "won't boot" with red marker.
    • In case of damage or not booting, put in the damaged stack
  4. Attach a label to the XO that has the Inventory Label as the identifier
  5. Write the Serial Number on the bottom of the laptop using a black permanent marker
  6. Type the Serial Number in the speadsheet next to the Inventory Number
  7. Insert USB key, containing runos.zip and actos.zip, see Procedure for Collecting UUID's and Serial Numbers
    • You should see a message like "SHFxxxxxxxx nnnnnnnnnnnnnnn; Laptop data recorded successfully"
    • The XO will then reboot, remove the USB key and turn it on again
  8. Send file to OLPC containing the information needed to get developer keys for each XO to help@laptop.org
  9. Once you have done this procedure for half of your XO's backup the laptops.dat and inventory spreadsheet w/ timestamps in the filenames (yes, I am paranoid)

How to check you have Serial Numbers and UUID's for each laptop 

 uniq laptops.dat > laptops-tmp.dat
 wc -l laptops-tmp.dat

This should match the exact number of XO's that you have.

Notes: It is very hard to attach any kind of sticker to the XO because of its nobby exterior. We placed the inventory label on the smooth green plastic above the screen. We used inventory labels only to keep track of the XO's while they are in our possession, part of a regulatory requirement. We will take off the inventory labels before we give the XO's to kids.

Stage 2 - Serious Testing

Stage 2 will be done by Bryan, Upaya, Rajeev, Sulochan, Rabi, and Saurav

What you Need

For Stage you need a USB key with the following contents:

  • os699.img and os699.crc,
  • /security/develop.sig -- developer key specific to the XO
  • /bundles/ containing library and activity bundles for our customized build
  • The Customization_key unpacked into the root directory


Print out a copy of this File:XO-Stage2 testplan.xls spreadsheet for every XO you will test (note: may be easier to record data on regular laptops.)

Firmware Testing

We won't be able to test the firmware until we get the developer keys for all XO's

  1. Write the Serial # for the XO on Testing Plan
  2. Place develop.sig specific to the XO you will test onto the USB key
  3. Hit the "X" Escape key in the upper left-hand corner of the keyboard
  4. At the OK prompt, type disable-security
  5. The XO will reboot. Type disable-security again
  6. Type test-all at the OK prompt
    • Note down any error messages you see during this process, basically any individual test that does not produce an OK result
    • You need to put pressure on the touchpad during the touchpad test, then hit the space bar
    • Don't hit the keys during the keyboard test, for some reason that causes a problem.
    • If you do get an error message, write Firmware -- Component xxx failed w/ a red marker on the XO's box
  7. after test-all completes, it is time to change the locale to Nepali from within the firmware (note: the XO's we received in Nepal had US as the locale)
    • At the OK prompt, type change-tag LO ne_NP.UTF-8 -- Note LO not LA
    • Turn off the XO and remove USB
  8. If no problems, mark the XO's label w/ a blue checkmark, if there is a problem check the label w/ a red marker.

TODO: I want to test the batteries for each XO but am not sure how to do this w/out spending massive amounts of time

Custom Build Testing

It may not be necessary to test the custom build for all of our XO's. We may be satisfied if we test 25-30 XO's without getting any errors.

Note: We had to make a special change to our build. We changed the default Jabber server to our local school server. Do this by changing the value of DEFAULT_JABBER_SERVER in /usr/lib/python2.5/site-packages/sugar/profile.py

USB Stick should contain:

  • os699.img and os699.crc,
  • /security/develop.sig -- developer key specific to the XO
  • /bundles/ containing library and activity bundles for our customized build
  • The Customization_key unpacked into the root directory

Steps:

  1. Insert the USB key
  2. Hit the "X" Escape key in the upper left-hand corner
  3. At the OK prompt, type copy-nand u:\os699.img
  4. The XO should reboot on its own. Leave the USB key in.
  5. Type in the XO's serial number as the user name
  6. Check that the default interface is in Nepali
  7. Use the camera activity to take a picture
  8. Connect to the Access Point 'Access-point Name' and connect to www.google.com
  9. Open the E-Paati activity
  10. Turn off the XO and put it back in the box
  11. If everything tested fine, write Software OK in blue marker on the box. Write Software Failed in red marker if otherwise.

Stage 3 - Load Testing

We are using build 703 and firmware Q2D14 for the pilot

Customizations to Build 703

  • Flash plugin 9 installed
  • Change Jabber Server to schoolserver2 in /usr/lib/python2.5/site-packages/sugar/profile.py
  • Installed latest Nepali language pack

Activities Pre-Installed

After these activities are loaded roughly 400 MB of SSD are left. EPaati is the largest activity by far, using more than 151 MB of space unpacked.

Overview of Test Plans

  1. Test 35 XO's with Access Point but without XS -- sniff network w/ wireshark and save summary
  2. Test 35 XO's w/ Access Point and with XS for Bashuki -- sniff network w/ wireshark, compare to w/out XS
  3. Test 35 XO's w/ Access Point and with XS for Bishwamitra
  4. Have Kamana, Saurav, and Bipul test all activities
  5. Test 70 XO's w/ the AP's we will use at the schools


Timeline

  1. Sunday - Test XO's w/ AP's and w/ XS
  2. Monday - Same as Sunday
  3. Tuesday - Test together with E-Pustakalaya, Test w/ AP we will use at the pilot schools, Complete final customized build
  4. Wednesday - Test at Bashuki and Bishwamitra

Test Plans

Basic

  1. connect to AP
  2. initiate Chat
  3. ping AP
  4. Share a document in Read, other can view the document
  5. Access E-Pustakalaya, download a book

Testing the XS?

Retrieved from "http://wiki.laptop.org/mediawiki/index.php?title=Nepal:_XO_Testing_Plan&oldid=185430"