Testing: Difference between revisions

From OLPC
Jump to navigation Jump to search
(→‎Notes and minutes: They seem to be on Thursdays, not Mondays)
No edit summary
 
(33 intermediate revisions by 9 users not shown)
Line 1: Line 1:
{{outdated}}
{{OLPC}}
{{OLPC}}
The community and core team work through a slate of '''test issues''' for each major build and release. This page highlights the current tests being run, the testing process in general, and how to contribute your own results or bugs.
The community and core team work through a slate of '''test issues''' for each major build and release. This page highlights the current tests being run, the testing process in general, and how to contribute your own results or bugs.


{{Latest Releases | firmware = inline | devel = inline | rc = inline | livecd = inline |server = inline }}
For the latest tests, see [[Update.1 tests]].
== Release info ==
=== Testing for release 11.3.1 ===
Testers are reporting test results on the testing [[Mailing lists|mailing list]] or by [[reporting bugs]].


They are ''not'' using [[Test group release notes]].
=== Testing for release 11.2.0 ===
See [[Test_cases/11.2]] for information about testing the [[11.2.0]] release.


=== Release info ===
=== Testing for the 10.1.x releases ===
* [[10.1.3/Testing]] for [[:Category:Community testing]] of the [[Release notes/10.1.3|10.1.3 release]]
* [[Smoke_test/10.1.x/1_hour_smoke_test]]


''' Major Release 8.2.0 '''
=== Testing for major release 8.2.0 ===
* [[Release_notes/8.2.0#New_Features_in_8.2.0|8.2.0 Features]], on top of [[XO_Base_Features]] of 8.1.1
* [[Release_notes/8.2.0#New_features_in_8.2.0|New features in 8.2.0]], on top of [[XO_Base_Features]] of 8.1.1
* [[TestPlan_8.2.0]]
* [[TestPlan 8.2.0]]
* [[Test cases 8.2.0]]
* [[Test cases 8.2.0]], a set of test cases developed in the wiki
* [[TestResults_8.2.0]]
* [[TestResults 8.2.0]]
* [[Release_notes/8.2.0#New_Features_in_8.2.0|New features in 8.2.0]]


''' Minor releases 8.1.x '''
=== Testing for minor releases 8.1.x ===
* Release 8.1.3, [[OLPC_SW-ECO_7]], More Amharic support
* Release 8.1.3, [[OLPC SW-ECO 7]], More Amharic support
* Release 8.1.2, [[OLPC_SW-ECO_6]], Nand-full, SD card corruption, support of C3 production boards
* Release 8.1.2, [[OLPC SW-ECO 6]], Nand-full, SD card corruption, support of C3 production boards
* Release 8.1.1, [[OLPC_SW-ECO_5]], Amharic and Kreyol support
* Release 8.1.1, [[OLPC SW-ECO 5]], Amharic and Kreyol support
** [[Release_notes/8.1.1|Release notes for 8.1.1]]
** [[Release_notes/8.1.1|Release notes for 8.1.1]]


==Community Testing==
=== Testing for release 8.1.0 ===
[[Release notes/8.1.0|Release 8.1.0]] was also known as "Update.1", see [[Update.1 tests]] for tests and [[Testing Update.1 Results]].
* [[Community testing]] (main page)

* [[Friends in testing]]
==Community testing==
People interested in testing should join the [http://lists.laptop.org/listinfo/testing Testing mailing list], which is also mirrored on an external [http://olpc.396210.n2.nabble.com/Testing-f1469263.html forum].

The awesome New Zealand testers use a very simple [[Activity testing template]].

* [[Systematic testing]]
* [[Systematic testing]]


Line 30: Line 43:
* [[Ejabberd resource tests/try_5]] school server jabber testing
* [[Ejabberd resource tests/try_5]] school server jabber testing


''Older'' obsolete community efforts:

* [[Community testing]]
''Older'' community efforts (should be updated to the new test template):
* [[Activity Testing Project]]
* [[Friends in testing]]
* [[Activity Testing Matrix]]
* [[Activity testing project]]
* [[Activity Testing Template]]
* [[Testing matrix]]
* [[Activity testing template]]


== Links of Interest to Testing ==
== Links of Interest to Testing ==
* [[Creating new test cases]]
* [[Test Config Notes]] - Links to configuration info that are useful in testing
* [[Test Config Notes]] - Links to configuration info that are useful in testing
* [[Test Group Release Notes]] - Quick info/smoke test on the latest builds. Read this before you bother to load a build
* [[Test Group Release Notes]] - Quick info/smoke test on the latest builds. Read this before you bother to load a build
Line 46: Line 61:
* School Server general info: [[XS]]
* School Server general info: [[XS]]
** [[Schoolserver Testing]]
** [[Schoolserver Testing]]
* [[Update.1 tests]] - test process for the 8.1.0/Update.1 builds (builds 691...703)
** [[Release Notes/8.1.0]] - release notes for 8.1.0 also known as "Update.1"
** [[Testing_Update.1_Results]] - some test results for Update.1 (in progress...)
** [[OLPC Ship.2 Software Release Notes]] - release notes for Ship2, build 650
* 100 Laptop Testbed:
* 100 Laptop Testbed:
** [[Collaboration_Network_Testbed]]
** [[Collaboration_Network_Testbed]]
Line 62: Line 73:
* [[XS_Server_Specification#XSX_Specifications|Server HW Specs]]
* [[XS_Server_Specification#XSX_Specifications|Server HW Specs]]


== Builds of Interest:==
== Builds of interest==
See [[OS images]] for build streams, release candidates, joyride builds, etc.
* [http://download.laptop.org/ OLPC Archives]
* [http://pilgrim.laptop.org/~pilgrim/olpc/streams/8.2/ "8.2" builds]
* [http://download.laptop.org/xo-1/os/official/ Recent stable builds]
* [http://pilgrim.laptop.org/~pilgrim/olpc/streams/update.1/ "Update.1" builds]
* [http://xs-dev.laptop.org/~cscott/olpc/streams/ "Bleeding-edge" builds]
** [http://xs-dev.laptop.org/~cscott/olpc/streams/joyride "Joyride" builds]
** [http://xs-dev.laptop.org/~cscott/olpc/streams/faster "Faster" builds]
** [http://xs-dev.laptop.org/~cscott/olpc/streams/meshtest "Meshtest" builds]
** [http://xs-dev.laptop.org/~cscott/olpc/streams/ship.2 "Ship.2" builds]
* [http://download.laptop.org/xo-1/os/candidate/ "Candidate" builds] before they go public


For latest activities, run Software update in [[Sugar Control Panel]], or download individual [[Activities]].
Activities script/latest activities:

* From a laptop that is connected to the internet, start a Terminal Activity and type: '''wget http://dev.laptop.org/~erikos/sucrose-activities.py'''
* Then type: '''python sucrose-activities.py'''
===Older builds and activities ===

* [http://dev.laptop.org/~erikos/bundles/0.81.4/ Link to Latest Activities]
* [http://pilgrim.laptop.org/~pilgrim/olpc/streams/update.1/ "Update.1" builds]
* [http://xs-dev.laptop.org/~cscott/olpc/streams/ship.2 "Ship.2" builds]


Activities script/latest activities for 8.1:
* http://dev.laptop.org/~erikos/sucrose-activities.py'''
* [http://dev.laptop.org/~erikos/bundles/0.81.4/ Latest 0.81.4 activities] (July 2008)


== Notes and minutes ==
== Notes and minutes ==
Line 91: Line 98:
{{Community testing}}
{{Community testing}}


<br clear="all" />
The navigation template {{tl|Test-nav}} is shown below, it should be updated and many related pages need to be merged:
The navigation template {{tl|Test-nav}} is shown below, it should be updated and many related pages need to be merged:
{{Test-nav}}
{{Test-nav}}


== Test cases and results ==
== Test cases and test results ==
Mid-2008 users including Femslade, Charlie, and Kimquirk created test cases for [[8.2.0]] that can be created and modified using a form.
Mid-2008 users including Femslade, Charlie, and Kimquirk created test cases for [[8.2.0]] that can be created and modified using a form.
Testers can also use the same form to add testing results.
Testers can also use the same form to add testing results, see [[Reporting test results]].

See, e.g. [[Tests/Activity/Write/Public_sharing]].
See, e.g. [[Tests/Activity/Write/Public_sharing]] as a sample.

All these test cases are in [[:Category:Test cases]] and most are listed by the queries in [[Test cases 8.2.0]].
All these test cases are in [[:Category:Test cases]] and most are listed by the queries in [[Test cases 8.2.0]].


In 2007 and early 2008, tests were just wiki pages, e.g. [[[[Tests/Browse]]. Those older test cases are mostly in [[:Category:Test Plans]], also [[:Category:Tests]].
In 2007 and early 2008, tests were just wiki pages, e.g. [[Tests/Browse]]. Those older test cases are mostly in [[:Category:Test Plans]] and/or [[:Category:Tests]].

=== Test case management ===

To add test categories to the pull down menu in [[Form:Test case]], modify
[[Property:Test category]]

To add stream checkboxes to [[Form:Test case]], modify
[[Property:Build stream]]

To build queries that list test cases, edit [[TestCase Queries]] and copy the <tt><nowiki>{{#ask:&nbsp;...&nbsp;}}</nowiki></tt> markup.
For other examples of queries, see [[Testcase Query Examples]]

Comments at [[Semantic MediaWiki#Issues for Test cases]].

== Testing areas ==
:''These "User stories" tests date from 2007-2008''
<!--
These are tests for the builds, materials on them, networks, school servers, and any global synchronization. There is a [[Test activity]] being designed to make testing on an XO simple, and independent of the details of the backend that tracks test results.

Any volunteer who is willing to test should be directed to one single portal: [[Test_issues|Testing]], and s/he can choose which tasks s/he would like to execute. --Chih-yu, Jan 2008
-->
High level topics for "User stories testing" can be divided into a few major categories:

=== One Hour Smoke Test ===

# [[Smoke test]] links to the 1-hour smoke test for different releases, and other smoke tests.

=== Networks ===
# [[Tests/100 laptops]]

=== Views and visualizations ===


#[[Tests/Home view|Home view]]
=== An old test case format ===
#[[Tests/Groups view|Groups view]]
'''This is temporary place for a sample of the test case format.'''
#[[Tests/Neighborhood view|Neighborhood view]]
Test area: [[Sugar Control Panel]]
#[[Tests/Frame|Frame]]


<!--
{| cellspacing="0" cellpadding="0" border="1"
{|
| '''Number/Title'''
| 1/Change Timezone
|
|-
|-
! Tests !! Results !!
| '''Objective'''
| colspan="2" | Verify that the user can change their XO's timezone from the default UTC setting.
|-
|-
| [[Tests/Mesh view]] || [[Results/Mesh view]]
| '''Feature'''
| colspan="2" | Users are able to change their XO's timezone to match the local time without having to go into a terminal to do so.
|-
|-
| [[Tests/Neighborhood view]] || [[Results/Neighborhood view]]
| '''Source'''
| colspan="2" | http://wiki.laptop.org/go/Sugar_Control_Panel#Date_.26_Time]
|-
|-
| '''Approach'''
| colspan="2" | Act as an end-user trying to change the XO's timezone to match the local time.
|-
| '''Test Tools'''
| colspan="2" | No test tools required.
|-
| '''Test Setup'''
| colspan="2" | One XO laptop running an 8.2.0 (pre-)release build.
|-
| '''Test Procedure'''
| colspan="2" | 1. Open the graphical Sugar Control Panel from the Home view.<br>
2. Click on the "Date & Time" panel.<br>
3. Find the desired timezone in the list and click on it.<br>
4. Click the check-mark on the top-right. Select either "Later" or "Restart now".<br>
4a. If you picked "Later", restart the laptop after a minute or two.<br>
5. After the XO reboots, check that the timezone changed by opening a terminal and typing 'date'.<br>
|-
| '''Expected Results & Pass Criteria'''
| colspan="2" | The timezone for the laptop has been changed correctly and the timezone is represented properly by the 'date' command.
|-
| '''Comments'''
| colspan="2" |
|}
|}
-->


=== Mime Types ===

# [http://www.yolinux.com/TUTORIALS/LinuxTutorialMimeTypesAndApplications.html Mime Types test] from ''yolinux.com''
:*[[What should go into Mime Type tests]]

===Battery Life===
Base line test for battery life. Boot up the machines, fully charge, remove the adapters and see how long the battery lasts.
* [[Battery Results]]
**[[Image:Baseline_battery_test.pdf | Battery Life Test 1 Results]] <br>

=== Regular processes ===

# [[Tests/Start up and Activation]]
# [[Test Basic Operation]]
# [[Tests/Connectivity and Collaboration]]
# [[Tests/Archive, Backup, Content]]
# [[Tests/Invite]]

=== International Laptops ===
# [[Tests/Keyboard mappings]]
# [[Tests/Language settings]]

=== Upgrading ===
# [[From last stable build]]
# [[Tests/Backward_Comp]]

=== Hardware and firmware ===

# [[Tests/SD card]]
# [[Tests/SD card acting as a usb drive in card reader]]
# [[Tests/USB stick]]
# [[Tests/Manufacturing Data]]
# [[Tests/Suspend_Resume|Tests/Suspend and resume]]

=== Network/Time Sync ===

# [[Tests/Time sync]]
# [[Tests/Network mgr]]

=== SchoolServer ===


== Testing areas ==


===In-house testing (from 2007?) ===
===Items to test===
''Old''?
''Old''?


Line 171: Line 226:
=== Other test plans ===
=== Other test plans ===


*[[802.11s Connectivity Test Plan]] - Connectivity, wifi protocol
*[[802.11s Connectivity Test Plan]] - Connectivity, wifi protocol from 2007
*[[Image:B3-B4 OLPC Test Plan.xls]] - Low Level Test Plan
*[[Image:B3-B4 OLPC Test Plan.xls]] - Low level test plan from 2007
*<strike>[[Tinderbox Testing]] - automated testing for each build</strike> (outdated) --[[User:Chihyu|Chihyu]] 13:58, 29 January 2008 (EST)
*<strike>[[Tinderbox Testing]] - automated testing for each build</strike> (outdated) --[[User:Chihyu|Chihyu]] 13:58, 29 January 2008 (EST)
*[[XMPP collaboration and scalability testing]] - XMPP collaboration and scalability testing
*[[XMPP collaboration and scalability testing]] - XMPP collaboration and scalability testing
* see [[:Category:Test Plans]]
* see [[:Category:Tests]] and [[:Category:Test Plans]] ''TODO: These older test plans should be marked obsolete or merged into test cases.''
TODO: These older test plans should be marked obsolete or turned into test cases.

== Test Case Management System ==

To add categories to the Test Case pull down menu, click here: <br>
[[Property:Test_category]]

To add streams to the Test Case form, click here: <br>
[[Property:Build_stream]]

How to build queries: [[TestCase_Queries]]

For other examples of queries, see [[Testcase Query Examples]]

Comments at [[Semantic MediaWiki#Suggestions_for_test_cases]].


==Schedule==
==Schedule==
Line 199: Line 239:


This section covers generic ways to set up before running different kinds of tests. (These are not specific test cases, but can be used as shorthand references to common procedures that are used across many test cases.
This section covers generic ways to set up before running different kinds of tests. (These are not specific test cases, but can be used as shorthand references to common procedures that are used across many test cases.
* [[Test Config Notes]]

* [[Memory leak testing]]
* [[Memory leak testing]]
* [[Olpc-update]]
* [[Olpc-update]]
* [[Clean-install procedure]]
* [[Clean-install procedure]]


== Also See ==
== See also ==

[http://code.google.com/soc/2008/psf/appinfo.html?csaid=F89A4969030B59E6 GSOC08 OLPC Project Automated Testing]


==== Older notes ====
* [[Test process sprint]]- Wed, Oct 17th, 2007

=== Reporting test results ===


* [http://code.google.com/soc/2008/psf/appinfo.html?csaid=F89A4969030B59E6 GSOC08 OLPC Project Automated Testing]
See [[Reporting test results]].
* [[Test process sprint for 2007 first software release]]- Wed, Oct 17th, 2007


[[Category:Developers]]
[[Category:Developers]]

Latest revision as of 05:17, 6 August 2013

The contents of this page are considered outdated and some of the information may be stale. Please use information here with caution, or update it.
  This page is monitored by the OLPC team.

The community and core team work through a slate of test issues for each major build and release. This page highlights the current tests being run, the testing process in general, and how to contribute your own results or bugs.


 + Software 
XO
13.2.11 2020-01-29
XS
xs-0.7 2012-02-20


Release info

Testing for release 11.3.1

Testers are reporting test results on the testing mailing list or by reporting bugs.

They are not using Test group release notes.

Testing for release 11.2.0

See Test_cases/11.2 for information about testing the 11.2.0 release.

Testing for the 10.1.x releases

Testing for major release 8.2.0

Testing for minor releases 8.1.x

Testing for release 8.1.0

Release 8.1.0 was also known as "Update.1", see Update.1 tests for tests and Testing Update.1 Results.

Community testing

People interested in testing should join the Testing mailing list, which is also mirrored on an external forum.

The awesome New Zealand testers use a very simple Activity testing template.

Test beds:

Older obsolete community efforts:

Links of Interest to Testing

Equipment specs:

  • Infrastructure_AP: Zoom 4400 AP+2 Wireless-G Access Point (Approx. $50 USD)
  • Long range Wireless: Wavion’s WS410 Spatially Adaptive Wi-Fi Access Point (Approx. $1600 USD)
  • Server HW Specs

Builds of interest

See OS images for build streams, release candidates, joyride builds, etc.

For latest activities, run Software update in Sugar Control Panel, or download individual Activities.

Older builds and activities

Activities script/latest activities for 8.1:

Notes and minutes

General navigation

Test cases 8.2.0 lists the test cases created for the recent release 8.2.0.

Test results are in TestResults 8.2.0

The {{Community testing}} navigation template is shown below:

XO Checkbox


The navigation template {{Test-nav}} is shown below, it should be updated and many related pages need to be merged:

Test cases and test results

Mid-2008 users including Femslade, Charlie, and Kimquirk created test cases for 8.2.0 that can be created and modified using a form. Testers can also use the same form to add testing results, see Reporting test results.

See, e.g. Tests/Activity/Write/Public_sharing as a sample.

All these test cases are in Category:Test cases and most are listed by the queries in Test cases 8.2.0.

In 2007 and early 2008, tests were just wiki pages, e.g. Tests/Browse. Those older test cases are mostly in Category:Test Plans and/or Category:Tests.

Test case management

To add test categories to the pull down menu in Form:Test case, modify Property:Test category

To add stream checkboxes to Form:Test case, modify Property:Build stream

To build queries that list test cases, edit TestCase Queries and copy the {{#ask: ... }} markup. For other examples of queries, see Testcase Query Examples

Comments at Semantic MediaWiki#Issues for Test cases.

Testing areas

These "User stories" tests date from 2007-2008

High level topics for "User stories testing" can be divided into a few major categories:

One Hour Smoke Test

  1. Smoke test links to the 1-hour smoke test for different releases, and other smoke tests.

Networks

  1. Tests/100 laptops

Views and visualizations

  1. Home view
  2. Groups view
  3. Neighborhood view
  4. Frame


Mime Types

  1. Mime Types test from yolinux.com

Battery Life

Base line test for battery life. Boot up the machines, fully charge, remove the adapters and see how long the battery lasts.

Regular processes

  1. Tests/Start up and Activation
  2. Test Basic Operation
  3. Tests/Connectivity and Collaboration
  4. Tests/Archive, Backup, Content
  5. Tests/Invite

International Laptops

  1. Tests/Keyboard mappings
  2. Tests/Language settings

Upgrading

  1. From last stable build
  2. Tests/Backward_Comp

Hardware and firmware

  1. Tests/SD card
  2. Tests/SD card acting as a usb drive in card reader
  3. Tests/USB stick
  4. Tests/Manufacturing Data
  5. Tests/Suspend and resume

Network/Time Sync

  1. Tests/Time sync
  2. Tests/Network mgr

SchoolServer

In-house testing (from 2007?)

Old?

Areas that we need to test in house (due to the use of special equipment and quantities of laptops) include:

  • All keyboards. - Test: Keyboard Settings
    • Keys match the layouts designed
    • Language, keyboard, keymappings match the settings
    • All language and keyboard settings are preserved across updates
  • Languages - Test: Language Settings
  • Firmware
    • WPA - Richardo/Michailis
    • WEP - Richardo/Michailis
    • Mesh - upgrade under loaded network - Scott
    • Suspend/resume - Wad - Test: Suspend/resume
  • Scaling/performance
    • Jabber server scaling - OLPC/Collabora
    • Link local presence (Salut) - OLPC/Collabora
    • Journal/datastore
  • Upgrades /backward compatibility
  • Security

Other test plans

Schedule

Please see the official roadmap (from the trac main page) for schedule dates and features for each release.

Test setups

This section covers generic ways to set up before running different kinds of tests. (These are not specific test cases, but can be used as shorthand references to common procedures that are used across many test cases.

See also

Retrieved from "http://wiki.laptop.org/mediawiki/index.php?title=Testing&oldid=290836"