Testing checklist: Difference between revisions
(..) |
|||
| (9 intermediate revisions by 5 users not shown) | |||
| Line 1: | Line 1: | ||
{{archive}} |
|||
=Overview= |
=Overview= |
||
| Line 15: | Line 18: | ||
* Basic functionality |
* Basic functionality |
||
** Seeing if the BIOS started |
** Seeing if the '''BIOS''' started |
||
** Running built in tests |
** Running built in tests |
||
* Networking: |
* Networking: |
||
** Getting an IP address on ethernet |
** Getting an '''IP''' address on ethernet |
||
** Copying an image from ethernet and checksumming it, before installing into NAND flash |
** Copying an image from ethernet and checksumming it, before installing into '''NAND''' flash |
||
** Rebooting into NAND Flash |
** Rebooting into '''NAND''' Flash |
||
** Rebooting into an image on disk |
** Rebooting into an image on disk |
||
* Reinstallation of the base software image |
* Reinstallation of the base software image |
||
* Wireless gets an IP address |
* Wireless gets an '''IP''' address |
||
** Copying 10 meg from wireless, and checksumming it |
** Copying 10 meg from wireless, and checksumming it |
||
* Running of CPU burn for a period |
* Running of '''CPU''' burn for a period |
||
* Running a video player and [http://samba.org/ftp/tridge/dbench/README dbench] on NAND and SD (coming soon). |
* Running a video player and [http://samba.org/ftp/tridge/dbench/README dbench] on '''NAND''' and '''SD''' (coming soon). |
||
** Getting an address on wireless |
** Getting an address on wireless |
||
** |
** Copying 10 meg of data, and checksumming the result |
||
* Installation and testing of the BIOS/OFW by |
* Installation and testing of the '''BIOS/OFW''' by |
||
* Rebooting, and repeating the whole sequence |
* Rebooting, and repeating the whole sequence |
||
| Line 36: | Line 39: | ||
==Manual Tests== |
==Manual Tests== |
||
Not all tests can be automated, unfortunately, |
|||
| ⚫ | |||
# DCON works in DCON mode (cold machine test - to test margins) |
|||
For that tests you can see [[BoardTestPlan|Board Test Plan]] |
|||
# X is running |
|||
# The camera application is able |
|||
==Software Tests== |
|||
##*to get video |
|||
##*display it on the screen |
|||
| ⚫ | |||
##*The controls work (e.g. contrast, chroma, flip) |
|||
# Touchpad |
|||
## Test that the capacitive touchpad sensor is working well in relative mode. |
|||
## Test the resistive sensor, and check the default calibration (details???) |
|||
# USB |
|||
## insert mouse and test |
|||
## insert keyboard and test |
|||
## insert USB devices and check |
|||
## remember, try all devices in all USB ports! |
|||
# SD |
|||
## insert SD card assortment and check, from smallest to > 2 gig cards |
|||
## test that driver works in high speed mode |
|||
## insert MMC card and verify the system will not read it gracefully |
|||
# Audio |
|||
## check that sound comes out of the speakers |
|||
## check that the built in microphone works |
|||
## check that mic-in works |
|||
## check that analog in works (to be re-implemented) |
|||
# DCON |
|||
## DCON mode test (warm machine testing) |
|||
# Battery |
|||
## check that the battery shows charging or charged |
|||
## unplug the machine |
|||
##* check that the battery shows present and |
|||
##* check that the machine does not show it is plugged in |
|||
## plug in the machine |
|||
##* check that the battery shows charging |
|||
## unplug the battery |
|||
##* see that the battery shows not present |
|||
## reinsert the battery |
|||
##* check that the battery shows present and charging again |
|||
# Sugar |
|||
[[Category:Hardware]] |
[[Category:Hardware]] |
||
Latest revision as of 08:11, 8 December 2009
Overview
We are a big believer in automated testing, to the extent that testing can be automated. Computers don't get bored, where as people do (and tend to skip tests). So making it as easy as possible to test our systems has a high priority.
Tinderbox is one such automation tool. In the OLPC tinderbox we can run multiple systems through many, but not all tests, while stressing the system. We reboot the systems, reinstall the image to be tested from scratch, run a set of tests (while loading the system), and repeat. You can see immediately if something has failed, and see where in the testing cycle the failure occurred. Some tinderboxes (e.g. mozilla's) are even able to identify the likely checkin that broke a build being tinderboxed, though that level of automation is currently out of our reach.
Equally important is that tinderboxes can automate performance tests, and arrange to systematically keep track of key performance and/or memory metrics over time, allowing quick identification of performance regressions.
Chris Ball heads OLPC's performance testing work: we're sure he can use help if you want to help in this area; there is always more that can be done! Not all the automatable tests have yet been implemented.
Automated(able) Tests
- Basic functionality
- Seeing if the BIOS started
- Running built in tests
- Networking:
- Getting an IP address on ethernet
- Copying an image from ethernet and checksumming it, before installing into NAND flash
- Rebooting into NAND Flash
- Rebooting into an image on disk
- Reinstallation of the base software image
- Wireless gets an IP address
- Copying 10 meg from wireless, and checksumming it
- Running of CPU burn for a period
- Running a video player and dbench on NAND and SD (coming soon).
- Getting an address on wireless
- Copying 10 meg of data, and checksumming the result
- Installation and testing of the BIOS/OFW by
- Rebooting, and repeating the whole sequence
We need to acquire a programmable power switch, to allow cold reboot tests; right now, we can only perform warm reboots. So far, however, most problems have occurred during warm reboots.
Manual Tests
Not all tests can be automated, unfortunately,
For that tests you can see Board Test Plan
Software Tests
We are not yet set up with Sugar to automate any tests. The testing of user interface software is notoriously hard to automatically test and notoriously sensitive to the details of the programs being tested, though some testing can be done.