TestPlan 8.2.0: Difference between revisions

From OLPC
Jump to navigation Jump to search
(New page: ===OLPC Test Plan/Release Criteria for 8.2.0=== This is a work-in-progress... Major test areas/features to be covered are: ==== Hardware/Firmware==== Robust behavior, ability to operate...)
 
m (Fix Etoys spelling)
 
(47 intermediate revisions by 4 users not shown)
Line 1: Line 1:
===OLPC Test Plan/Release Criteria for 8.2.0===
===OLPC Test Areas/Features for 8.2.0===


This is a work-in-progress...
This is a draft.


Major test areas/features to be covered are:
Below is the major test areas/features list that will be used to create [[Test cases 8.2.0|8.2.0 test cases]]:


==== Hardware/Firmware====
==== Network Connectivity/Collaboration====
Robust behavior, ability to operate with user-oriented (touch-pad, keyboard, etc.) and external devices (mouse, etc.)


Connectivity and ability to browse the web from WEP and WPA access points.
==== Power Management (Power Control & Conservation)====


Sharing between XOs on a simple mesh.
When the lid is closed or when the power button is pressed while Sugar is running, suspend in order to conserve power.


Between XOs with a school server with an 802.11 b/g access point (with or without WEP and WPA/WPA2 encryption). Access point connectivity is currently a priority.
Resume from suspend in response to lid opening or power-button presses


Active Antenna (low priority for 8.2)
Cut power by holding the power button for several seconds


In a mixed environment (when various wireless points of contact are present in the area), allow switching between different modes of connectivity.
Power off or reset the machine via the Central XO-person's Palette


Please see use cases that describe connectivity testing in more detail: [[Use_Cases]]
Control screen brightness and suspend state from the root command-line


==== School Server Backup Feature====
Control screen brightness with the brightness keys in Sugar


Register and automatically save files to school server


Restore files from school server to XO.
==== USB/SD Peripherals====


====Journal/Datastore ====
Reliably store user data to and retrieve user data from FAT16/VFAT formatted USB Mass Storage devices.


Reliably store user data to and retrieve user data from FAT16/VFAT formatted SD/SDHC devices.
Be able to save and retrieve activities and files from the Journal.


List, search, and filter records of actions on the system by time, title, or type.
USB ethernet adapters, keyboards, and mice should function similarly to their normal function on other operating systems.


Be able to display and edit text (rtf, pdf, x-pdf, doc, html), image (png, jpeg, gif), audio (ogg, x-wav, wav), video (ogg, mpeg, avi) formats.
Login via serial consoles. (very low priority)


==== Activity Updater ====


This is located in the sugar control panel, but is being identified here as it needs to be get specific test cases.
==== Firmware====


Be able to upgrade Activities based on the default location of Activites.
Developer keys should grant access to OFW.


Be able to upgrade Activities based on the G1G1/8.2 version of activities.
Should be able to flash any build from OFW.


Be able to upgrade Activities based on the Peru version, 8.2. (Lower priority as this needs to work for 8.2.1).
Should be able to rewrite manufacturing data (e.g. language, write-protect tags) from OFW (Very low priority, not a public feature)


Should respond to gamepad [go/Cheat_codes Cheat codes]


==== Power Management (Power Control & Conservation)====


When the lid is closed or when the power button is pressed quickly while Sugar is running, suspend the machine in order to conserve power.
==== Installation of Software (operating system, Sugar, activities) and firmware. Field upgrades.====


Resume from suspend in response to lid opening or quickly pressing the power-button.
Must be able to cleaninstall from a production image to a laptop. All data is lost. [[Tests/Boot/Cleaninstall]]


Power off a machine by holding the power button for several seconds.
W/ internet or preparation: install new activities and library content on a raw OS via the internet or via USB keys.


Power off or reset the machine via the Central XO-person's Palette.
W/ internet: olpc-update to new versions of the operating system via the internet. Data in /home should be PRESERVED.


Control screen brightness with the brightness keys in Sugar.
W/ preparation: olpc-update to new versions of the operating system via USB. Data in /home should be PRESERVED.


==== Installation/Upgrade ====
Access the root account via the virtual terminals or via the Terminal activity (if present.)


Original software must be installable at the factory. It can be upgraded via an USB stick (clean install) or over the network (upgrade). Clean install will result in deleting all previously existed user data. When upgraded over the network, user data collected while running previously installed software must be preserved and appear existed after the upgrade.


For testing, these are the features that need to be addressed:
Access to the firmware "ok" prompt w/ developer key, requestable through Browse and installable through the Terminal.
# Start with major release currently in the field; upgrade via olpc-upgrade to new release
# Start with minor release(s) currently in the field; upgrade via olpc-upgrade to new release
# Use USB upgrade
# Use cleaninstall procedure


In the cases of upgrades (rather than cleaninstall), the data and Journal entries from before the upgrade should remain intact.
====Sugar GUI====


==== Firmware====
TBD


Developer key should grant access to OFW.


It should be able to reinstall firmware If needed) without reinstalling the whole build.
====Datastore====


It should be possible to rewrite manufacturing data (e.g. language, write-protect tags) from OFW (?).
Be able to display and edit DOC, ODT, JPEG, PNG, GIF, WAV, OGG, HTML, JS, some SWF, and several other less-well known formats.


====Security====
We expect NOT to be able to display [go/Restricted_Formats restricted formats] including MP3, MPEG, most AVI, RM, FLV, etc.


Laptops should travel through the delivery chain unactivated.


Laptops should not boot until they are activated.
====Activities====


Laptops can be activated locally (via USB) or wirelessly (with a school server) (?).
Interact with one activity or "location" at a time


====Localization====
Switch between the current activity and other locations with the view keys or the frame


'''Keyboards'''
Switch between activities with Alt-Tab or the activity ring
Must support all keyboards that we developed. "Support" is defined as:
# There is a keyboard silkscreen, approved and able to be produced at Quanta
# The keys on the keyboard map to the appropriate characters on the display
# The text on the screen renders legibly and is directionally correct (left to right, right to left)


The tests to determine items 2 and 3 only include text that is typed in the Sugar UI, Write and Browse. See the list of keyboards we would like to support from the release notes: [[Release_Notes/8.2.0#New_Keyboard_Support]]
Start new activities and stop running ones with the frame, ring, or activity toolbar.


'''Translations'''
Pippy, Record, and TamTam will be able to play and record sounds.
Translations need to be developed and approved by the country as OLPC does not have language expertise.


====User documentation====
Pippy and Record will be able to take photographs.


Inside XO and online (wiki) available
Record will be able to record short videos.


Help Activity for G1G1
Measure will be able to measure signals with or without DC voltage bias.


==== USB/SD Peripherals====
Volume keys on the keyboard will adjust the speaker volume while Sugar is running.


Store and retrieve user data from FAT16/VFAT formatted USB Mass Storage and SD/SDHC devices.
Control audio volume with the volume keys in sugar


USB ethernet adapters, keyboards, and mice should function similarly to their normal function in other operating systems.
Pictures can be composed with text in a system editor such as Write. (We actually expect [go/Write Write] to do a lot of other things.)


Use SD cards for normal operation and be able to boot off of SD.
Fancier multimedia like video, audio, and programs can be manipulated in an environment like [go/EToys EToys].


Login via serial consoles. (low priority)
Copy-and-paste media clips between activities


====Sugar GUI====
Download media clips into the Journal


New Home View
Pictures can be composed with text in a system editor such as Write. (We actually expect [go/Write Write] to do a lot of other things.)


Frame features
Fancier multimedia like video, audio, and programs can be manipulated in an environment like [go/EToys EToys].


Activity launching
Copy-and-paste media clips between activities


Graphical Sugar Control Panel
Download media clips into the Journal


Remove network configuration settings
<div>The section below has to be edited (in progress)…


====Activities====
Installing or removing activities should never change the ability of the system to run.</div>


1 - Start activities at the same speed (including but not limited to all
Installing or removing activities should not affect the ability of the system to run.


Switch between activities with Alt-Tab or with the means provided in the Home view.
fructose activities)


Start new activities and stop running ones with the frame, ring, or activity toolbar.
2 - Boot to home at the same speed.


Browse must render at least the web sites listed at
3 - Cursor control the same, including number of XOs where it moves


http://dev.laptop.org/ticket/7421
without input or moves without correlation to finger on touchpad


Pippy, Record, and TamTam will be able to play and record sounds.
3 - Connect to the same APs in the same time and with the same user


Pippy and Record will be able to take photographs.
feedback. (e.g. blinks then shows white circle).


Record will be able to record short videos (but not reliably transfer them between laptops).
4 - Connect to active antennas in the same time and with the same user


Measure will be able to measure signals with or without DC voltage bias.
feedback (e.g. blinks then shows white circle).


Volume keys on the keyboard will adjust the speaker volume while Sugar is running.
5 - Run all the same activities (can require activity upgrade within


Should be able to control audio volume with the volume keys in Sugar.
documented bounds). Includes allowing activities to mesh the same way.


6 - Connect to mesh with the same number of XOs in the same time with
Pictures can be composed with text in the Write.


Fancier multimedia like video, audio, and programs can be manipulated in an environment like [[Etoys]].
the same failure rate.


7 - Save and open files for all activities. Fructose activities at a
Copy-and-paste media clips between activities.


Download media clips into the Journal.
minimum otherwise all activities per 5 above. Including when journal has


Fancier multimedia like video, audio, and programs can be manipulated in an environment like [[Etoys]].
thousands of entries.


Copy-and-paste media clips between activities
8 - Use the same NAND space.

9 - Always boots up, especially when there is no space on NAND

10 - Open and close all activities per 5 above.

11 - Must support all languages and keyboards previously supported.

"support" means all previously translated strings still work in

activities and sugar. All language types work the same (e.g. RTL)

12 - No new cases where the XO crashes and needs hard reboot.

13 - No new cases where the cursor stops responding (AKA hangs) for more

than 30 seconds.

14 - Sound and video recording and playback must record at the same

rate. Must playback all the same kinds of files. Must playback with the

same quality (assuming all other variables constant, e.g. network,

RAN=M, disk space, etc.)

15 - Browse must render the same web sites as before. See also

[http://dev.laptop.org/ticket/7421 http://dev.laptop.org/ticket/7421]

16 - USB read, write, mount and unmount must work the same and for all

the same devices.

17 - Mouse and keyboard must connect and work the same

18 - Journal has the same capabilities.

19 - Must use the same or less power for the same user interaction

20 - Must have the same security mechanisms and be as secure as previous

releases.

21 - Must be installable in the factory and upgradeable from 656, 703

and 708 via USB or network.



====Collaboration/Connectivity====

Between XOs with simple mesh (associate with other XO or Active-Antenna 802.11s meshes).

Between XOs with a school server with an active antenna and an access point (associate with 802.11 b/g access points with or without WEP and WPA1 encryption).

In a mixed environment.



====Security====

Laptops should travel through the delivery chain unactivated.

Laptops should not boot until they are activated.

Can be activated wirelessly (with a school server) ''(not recently tested)''

Can be activated via USB.


Download media clips into the Journal


====Performance/scalability====
====Performance/scalability====


TBD.


====Localization====
==== Hardware====
Support robust behavior, ability to operate with internal (touch-pad, keyboard, etc.) and external devices (mouse, etc.)

Keymaps are correct or are very close to correct for US/International, Amharic, Spanish, Portuguese, and Kreyól keyboards.

Translations

====User documentation====

Inside XO and online (wiki) available

TBD


====Journal ====

List, search, and filter records of actions on the system by time


[[Category:Test plans]]
filter records of actions by title

Latest revision as of 20:46, 9 December 2008

OLPC Test Areas/Features for 8.2.0

This is a draft.

Below is the major test areas/features list that will be used to create 8.2.0 test cases:

Network Connectivity/Collaboration

Connectivity and ability to browse the web from WEP and WPA access points.

Sharing between XOs on a simple mesh.

Between XOs with a school server with an 802.11 b/g access point (with or without WEP and WPA/WPA2 encryption). Access point connectivity is currently a priority.

Active Antenna (low priority for 8.2)

In a mixed environment (when various wireless points of contact are present in the area), allow switching between different modes of connectivity.

Please see use cases that describe connectivity testing in more detail: Use_Cases

School Server Backup Feature

Register and automatically save files to school server

Restore files from school server to XO.

Journal/Datastore

Be able to save and retrieve activities and files from the Journal.

List, search, and filter records of actions on the system by time, title, or type.

Be able to display and edit text (rtf, pdf, x-pdf, doc, html), image (png, jpeg, gif), audio (ogg, x-wav, wav), video (ogg, mpeg, avi) formats.

Activity Updater

This is located in the sugar control panel, but is being identified here as it needs to be get specific test cases.

Be able to upgrade Activities based on the default location of Activites.

Be able to upgrade Activities based on the G1G1/8.2 version of activities.

Be able to upgrade Activities based on the Peru version, 8.2. (Lower priority as this needs to work for 8.2.1).


Power Management (Power Control & Conservation)

When the lid is closed or when the power button is pressed quickly while Sugar is running, suspend the machine in order to conserve power.

Resume from suspend in response to lid opening or quickly pressing the power-button.

Power off a machine by holding the power button for several seconds.

Power off or reset the machine via the Central XO-person's Palette.

Control screen brightness with the brightness keys in Sugar.

Installation/Upgrade

Original software must be installable at the factory. It can be upgraded via an USB stick (clean install) or over the network (upgrade). Clean install will result in deleting all previously existed user data. When upgraded over the network, user data collected while running previously installed software must be preserved and appear existed after the upgrade.

For testing, these are the features that need to be addressed:

  1. Start with major release currently in the field; upgrade via olpc-upgrade to new release
  2. Start with minor release(s) currently in the field; upgrade via olpc-upgrade to new release
  3. Use USB upgrade
  4. Use cleaninstall procedure

In the cases of upgrades (rather than cleaninstall), the data and Journal entries from before the upgrade should remain intact.

Firmware

Developer key should grant access to OFW.

It should be able to reinstall firmware If needed) without reinstalling the whole build.

It should be possible to rewrite manufacturing data (e.g. language, write-protect tags) from OFW (?).

Security

Laptops should travel through the delivery chain unactivated.

Laptops should not boot until they are activated.

Laptops can be activated locally (via USB) or wirelessly (with a school server) (?).

Localization

Keyboards Must support all keyboards that we developed. "Support" is defined as:

  1. There is a keyboard silkscreen, approved and able to be produced at Quanta
  2. The keys on the keyboard map to the appropriate characters on the display
  3. The text on the screen renders legibly and is directionally correct (left to right, right to left)

The tests to determine items 2 and 3 only include text that is typed in the Sugar UI, Write and Browse. See the list of keyboards we would like to support from the release notes: Release_Notes/8.2.0#New_Keyboard_Support

Translations Translations need to be developed and approved by the country as OLPC does not have language expertise.

User documentation

Inside XO and online (wiki) available

Help Activity for G1G1

USB/SD Peripherals

Store and retrieve user data from FAT16/VFAT formatted USB Mass Storage and SD/SDHC devices.

USB ethernet adapters, keyboards, and mice should function similarly to their normal function in other operating systems.

Use SD cards for normal operation and be able to boot off of SD.

Login via serial consoles. (low priority)

Sugar GUI

New Home View

Frame features

Activity launching

Graphical Sugar Control Panel

Remove network configuration settings

Activities

Installing or removing activities should not affect the ability of the system to run.

Switch between activities with Alt-Tab or with the means provided in the Home view.

Start new activities and stop running ones with the frame, ring, or activity toolbar.

Browse must render at least the web sites listed at

http://dev.laptop.org/ticket/7421

Pippy, Record, and TamTam will be able to play and record sounds.

Pippy and Record will be able to take photographs.

Record will be able to record short videos (but not reliably transfer them between laptops).

Measure will be able to measure signals with or without DC voltage bias.

Volume keys on the keyboard will adjust the speaker volume while Sugar is running.

Should be able to control audio volume with the volume keys in Sugar.

Pictures can be composed with text in the Write.

Fancier multimedia like video, audio, and programs can be manipulated in an environment like Etoys.

Copy-and-paste media clips between activities.

Download media clips into the Journal.

Fancier multimedia like video, audio, and programs can be manipulated in an environment like Etoys.

Copy-and-paste media clips between activities

Download media clips into the Journal

Performance/scalability

TBD.

Hardware

Support robust behavior, ability to operate with internal (touch-pad, keyboard, etc.) and external devices (mouse, etc.)

Retrieved from "http://wiki.laptop.org/mediawiki/index.php?title=TestPlan_8.2.0&oldid=185407"