Update.1 tests: Difference between revisions

From OLPC
Jump to navigation Jump to search
 
(70 intermediate revisions by 14 users not shown)
Line 1: Line 1:
'''End Users''': ''see [[Release notes/8.1.0]] for more relevant information.''

==Schedule & Features==
==Schedule & Features==


Line 12: Line 14:
==Test Plans==
==Test Plans==


An [[1_Hour_Smoke_Test|1hr smoke test]] should be performed with each candidate, and the smoke test results should be reported in [[Test_Group_Release_Notes#Update.1_Builds|Test Group Release Notes]].
A [[smoke test|1hr smoke test]] should be performed with each candidate, and the smoke test results should be reported in [[Test_Group_Release_Notes#Update.1_Builds|Test Group Release Notes]].


===Major Foci===
===Major Foci===
Line 19: Line 21:


# Suspend/resume (does not apply to B4 machines)
# Suspend/resume (does not apply to B4 machines)
#* [[Tests/Suspend_Resume|Generic test cases]]
#* [[Tests/Suspend_Resume|Suspend-Resume test cases]]
<!--#* [http://dev.laptop.org/ticket/5400 #5400] covered in generic test cases-->
<!--#* [http://dev.laptop.org/ticket/5400 #5400] covered in generic test cases-->
<!--#* [http://dev.laptop.org/ticket/4843 #4843] covered in generic test cases-->
<!--#* [http://dev.laptop.org/ticket/4843 #4843] covered in generic test cases-->
Line 42: Line 44:
# <tt>olpc-update --usb</tt> '''without''' [[Olpc-update#Workaround:_updating_olpc-update|manually upgrading olpc-update]]. ([http://dev.laptop.org/ticket/3881 #3881])
# <tt>olpc-update --usb</tt> '''without''' [[Olpc-update#Workaround:_updating_olpc-update|manually upgrading olpc-update]]. ([http://dev.laptop.org/ticket/3881 #3881])
#* Type <tt>olpc-update --version</tt> in the console. Verify that the version is 2.0 or newer.
#* Type <tt>olpc-update --version</tt> in the console. Verify that the version is 2.0 or newer.
#* [[Tests/Backward_Comp|Test: Backward Compatibility]]
# Basic activity isolation ([http://dev.laptop.org/ticket/5033 #5033], [http://dev.laptop.org/ticket/5346 #5346])
# Basic activity isolation ([http://dev.laptop.org/ticket/5033 #5033], [http://dev.laptop.org/ticket/5346 #5346])
#* need help confirm that [[Tests/Isolation#Activity_Isolation|Tests: Isolation]] is testing [http://dev.laptop.org/ticket/5033 the correct issues] --[[User:Chihyu|Chihyu]] 11:46, 22 January 2008 (EST)
#* need help confirm that [[Tests/Isolation#Activity_Isolation|Tests: Isolation]] is testing [http://dev.laptop.org/ticket/5033 the correct issues] --[[User:Chihyu|Chihyu]] 11:46, 22 January 2008 (EST)
Line 51: Line 54:
#* cannot create test cases due to the disappearance of original bug description --[[User:Chihyu|Chihyu]] 12:29, 22 January 2008 (EST)
#* cannot create test cases due to the disappearance of original bug description --[[User:Chihyu|Chihyu]] 12:29, 22 January 2008 (EST)
# Mesh - upgrade under loaded network - Scott
# Mesh - upgrade under loaded network - Scott
#* [[Tests/Backward_Comp|Test: Backward Compatibility]]


====Wireless/USB/Network Manager====
====Wireless/USB/Network Manager====


[[Tests/Wireless| Wireless Test Cases]] - Yani, Ricardo
# [[Tests/Wireless_Test-scp_loop|Wireless Test Cases]], issues to address:
#* Intermittent connection to jabber service, [http://dev.laptop.org/ticket/4193 #4193]

# Intermittent connection to jabber service, [http://dev.laptop.org/ticket/4193 #4193]
#* Network sniffing script, [http://dev.laptop.org/ticket/5153 #5153]
# Network sniffing script, [http://dev.laptop.org/ticket/5153 #5153]
#* WPA working from sugar, [http://dev.laptop.org/ticket/5258 #5258]
# WPA working from sugar, [http://dev.laptop.org/ticket/5258 #5258]
#* WEP working from sugar, [http://dev.laptop.org/ticket/5485 #5485]
# WEP working from sugar, [http://dev.laptop.org/ticket/5485 #5485]


====Tubes/Presence (Scaling/Performance)====
====Tubes/Presence (Scaling/Performance)====
Line 77: Line 78:
# Scaling? [http://dev.laptop.org/ticket/5817 #5817]
# Scaling? [http://dev.laptop.org/ticket/5817 #5817]
# Link local presence (Salut) - OLPC/Collabora
# Link local presence (Salut) - OLPC/Collabora
#* need help with main issues and test cases --[[User:Chihyu|Chihyu]] 18:21, 24 January 2008 (EST)
# Journal/datastore
#* need help with main issues and test cases --[[User:Chihyu|Chihyu]] 18:21, 24 January 2008 (EST)
#* need help with main issues and test cases --[[User:Chihyu|Chihyu]] 18:21, 24 January 2008 (EST)


Line 84: Line 83:


# [[Tests/Browse|Browse]]
# [[Tests/Browse|Browse]]
# [[Tests/Write|Write]]
# Please follow the test cases in [[User_Stories_Testing|User-Focused Test Plans]]. Some test cases should already be covered in the previous section ([[#Major Foci]]).
# [[Tests/Record|Record]]
# [[Tests/Paint|Paint]]
# [[Tests/Chat|Chat]]
# [[Testing_matrix|Other activities]]
# [[User_Stories_Testing|User-Focused Test Plans]]. (Some test cases should already be covered in the previous sections).


===Incomplete Test Cases===
===Incomplete Test Cases===


Please help revise/create test cases in [[:Category:Test_Plans|Test Plans]]
These tests need to be done, but I have yet to create test cases that are brief enough for community testing. --[[User:Chihyu|Chihyu]] 14:27, 18 January 2008 (EST)


====Others====
===Others===


*[[802.11s Connectivity Test Plan]] - Connectivity, wifi protocol
# [[802.11s Connectivity Test Plan]] - Connectivity, wifi protocol
<!--*[[Image:B3-B4 OLPC Test Plan.xls]] - Low Level Test Plan-->
<!--*[[Image:B3-B4 OLPC Test Plan.xls]] - Low Level Test Plan-->
*[[Tinderbox Testing]] - automated testing for each build
# [[Tinderbox Testing]] - automated testing for each build
# Journal/datastore

#* need help with main issues and test cases --[[User:Chihyu|Chihyu]] 18:21, 24 January 2008 (EST)
===Testing in Australia===
<!--
* upgrade to update.1 rc1
* network melting down before hearing a school server (N machines in salut wake up from suspend all at once)
** N machines unsuspend in presence of school server -> network not melt
** switch from local link (salut) to using gabble
-->
====Boot up and Register====
<!--registration with school server-->
[note] Re-registering always increases the entries in the jabber roster.

Setup:
# Every XO has to run the same build ([http://pilgrim.laptop.org/~pilgrim/olpc/streams/update.1/ Update.1 RC1]) and firmware ([http://wiki.laptop.org/go/OLPC_Firmware_q2d09 q2d09]).
# Remove network manager and config file:
#* In the Terminal activity, type
#*: <tt>cd .sugar/default/</tt>
#*: <tt>rm nm/networks.cfg</tt>
#*: <tt>rm config</tt>
# Reboot the XO.
# Make sure the school server is running.

Actions:
# Reboot the XO again (in the presence of the school server).
# Click on the "Register" option when mouse over the XO guy in the center of Home View.
# Reboot the XO (in the presence of the school server).

Verify:
* All XOs are connected to the school server.
** In Home View, the connection indicator next to the battery icon should read "Connected to a School Mesh Portal".
** Alternatively, type <tt>olpc-netstatus</tt> in the Terminal activity (might have to become root), one should see
Telepathy : gabble
Jabber : schoolserver.laptop.org
* All XOs register properly with the school server.
** Mouse over the XO guy in the center of Home View, and the "Register" option should no longer be available.


===Scaling Tests===
====Network Melt Down in Gabble(?)====


[[Tests/Conference_scaling|Australia/conference tests]]
Setup:

# Turn off the school server.
# Remove network manager:
#* In the Terminal activity, type
#*: <tt>rm .sugar/default/nm/networks.cfg</tt>
# Boot up the XO, and it should connect to Simple Mesh 1
#* In the Terminal activity, type <tt>olpc-netstatus</tt> (might have to become root) to ensure every XO shows <tt>Telepathy : salut</tt>
# Suspend the machine by pressing the power button
#* The power indicator LED should be blinking in suspend mode.

Actions:
# Turn on school server, and ensure it's running.
# Unsuspend the XO by pressing the power button again.

Verify:
* Within 10 minutes or so, the XO should connect to the School Server Mesh.
** In Home View, the connection indicator next to the battery icon should read "Connected to a School Mesh Portal".

====RF Saturation (salut/local link scaling test?)====
This is designed to understand whether the XOs affect other regular laptops connected to wireless network.

Setup:

# On each XO, remove network manager:
#* In the Terminal activity, type
#*: <tt>rm .sugar/default/nm/networks.cfg</tt>
# Shut down the school server.

Actions:
# Turn on school server.
# On a regular laptop, connect to infrastructure network (i.e. regular wireless access point).
# Connect multiple XOs to the school server simultaneously.

Verify:
* The wireless network connection of the regular laptops are not affected by the XOs.


==Test Results==
==Test Results==
See also [[Testing Update.1 Results]].


=== Template ===
=== Template ===

When you sign under each reported result, simply type 4 tildes: <nowiki>~~~~</nowiki> instead of typing your username, date, and time.
# If the table has not been linked to any test cases, either the test cases have not been created, or please help us locate the test cases!
# When you sign under each reported result, simply type 4 tildes: <nowiki>~~~~</nowiki> instead of typing your username, date, and time.
# Values to fill in:
#* '''no''': not tested
#* '''good''': tested and passed
#* If a defect is found, please file a trac ticket and fill in the ticket number in the table. For example, [http://dev.laptop.org/ticket/5969 5969].


{| class="wikitable"
{| class="wikitable"
Line 181: Line 124:
|-
|-
! rowspan="2"| 684
! rowspan="2"| 684
! keyboard
! [[Tests/Keyboard_mappings|keyboard]]
| good
| good
[[User:Chihyu|Chihyu]] 13:56, 17 January 2008 (EST)
[[User:Chihyu|Chihyu]] 13:56, 17 January 2008 (EST)
|-
|-
! language
! [[Tests/Language_settings|language]]
| [http://dev.laptop.org/ticket/5969 5969]
| [http://dev.laptop.org/ticket/5969 5969]
[[User:Chihyu|Chihyu]] 17:47, 28 January 2008 (EST)
[[User:Chihyu|Chihyu]] 17:47, 28 January 2008 (EST)
|-
|-
! rowspan="2"| future<br>candidate
! rowspan="2"| future<br>candidate
! keyboard
! [[Tests/Keyboard_mappings|keyboard]]
| no
| no
|-
|-
! language
! [[Tests/Language_settings|language]]
| no
| no
|}
|}




=== Suspend/resume ===
=== Suspend/resume ===


{| class="wikitable"
* '''no''': not tested
! Build
* '''good''': tested and passed
! [[Tests/Suspend_Resume#Suspend_Mode|Suspend Mode]]
* If a defect is found, please file a trac ticket and fill in the ticket number in the table. For example, [http://dev.laptop.org/ticket/5969 5969].
! [[Tests/Suspend_Resume#Sleep_Mode_.28power_button.29|Sleep Mode<br>(power button)]]
! [[Tests/Suspend_Resume#Sleep_Mode_.28lid_closing.29|Sleep Mode<br>(lid closing)]]
! [[Tests/Suspend_Resume#Ebook_Mode|Ebook Mode]]
! [[Tests/Suspend_Resume#Restart_Sugar|Restarting Sugar]]
! [[Tests/Suspend_Resume#Shut_Down_in_Battery_Mode|Battery Mode]]
! [[Tests/Suspend_Resume#Turning_off_suspend.2Fresume|Disable<br>Suspend/Resume]]
|-
!|RC1-690
| no
| no
| no
| no
| no
| no
| no
|-
!|RC2-691
| no
| no
| no
| no
| no
| no
| no
|-
|}

=== Security/Upgrades ===


{| class="wikitable"
{| class="wikitable"
! Build
! Build
! [[Tests/Isolation|Isolation]]
! Suspend Mode
! [[From_last_stable_build|Upgrade]]
! Sleep Mode<br>(power button)
! [[Tests/Backward_Comp|Backward Comp]]
! Sleep Mode<br>(lid closing)
! Ebook Mode
! Restarting Sugar
! Battery Mode
! Disable<br>Suspend/Resume
|-
|-
!|690
!|RC1-690
| no
| no
| no
|-
!|RC2-691
| no
| no
| no
|}

=== Wireless/USB/Network Manager ===

{| class="wikitable"
! Build
! [[Tests/Wireless_Test-scp_loop|Wireless]]
! [http://dev.laptop.org/ticket/4193 Salut Issue]
! [http://dev.laptop.org/ticket/5258 WPA Bug]
! [http://dev.laptop.org/ticket/5485 WEP Bug]
! [[Tests/USB_stick|USB]]
! [[Tests/Time_sync|Time Sync]]
! [[Tests/Network_mgr#Network_mgr_SEGV|Network mgr]]
|-
!|RC1-690
| no
| no
| no
| no
Line 222: Line 214:
| no
| no
|-
|-
!|RC2-691
!|future<br>candidate
| no
| no
| no
| no
Line 233: Line 225:
|}
|}


=== Tubes/Presence (Scaling/Performance) ===


{| class="wikitable"
===Keyboard/Languages===
! Build
! [http://dev.laptop.org/ticket/5485 Friends]
! [[Tests/Conference_scaling|Conf_Scale]]
|-
!|RC1-690
| no
| no
|-
!|RC2-691
| no
| no
|-
|}

=== Core Activities ===

Please add the testing results for [[Testing_matrix|other core activities]] in this section!

{| class="wikitable"
! Build
! [[Tests/Browse|Browse]]
! [[Tests/Chat|Chat]]
! [[Tests/Paint|Paint]]
! [[Tests/Record|Record]]
! [[Tests/Write|Write]]
|-
!|RC1-690
| no
| no
| no
| no
| no
|-
!|RC2-691
| no
| no
| no
| no
| no
|-
|}

=== User-Focused Test Plans ===

{| class="wikitable"
! Build
! [[Tests/SD_card|SD Card]]
! [[Tests/SD_card_acting_as_a_usb_drive_in_card_reader|SD Card as USB drive]]
|-
!|RC1-690
| no
| no
|-
!|RC2-691
| no
| no
|-
|}


=== Keyboard/Languages ===
* '''no''': not tested
* '''good''': tested and passed
* If a defect is found, please file a trac ticket and fill in the ticket number in the table. For example, [http://dev.laptop.org/ticket/5969 5969].


{| class="wikitable"
{| class="wikitable"
Line 256: Line 305:
! Portuguese
! Portuguese
|-
|-
! rowspan="2"| future<br>candidate 1
! rowspan="2"| RC1-690
! keyboard
! [[Tests/Keyboard_mappings|keyboard]]
| no
| no
| no
| no
Line 271: Line 320:
| no
| no
|-
|-
! language
! [[Tests/Language_settings|language]]
| no
| no
| no
| no
Line 285: Line 334:
| no
| no
|-
|-
! rowspan="2"| future<br>candidate 2
! rowspan="2"| RC2-691
! keyboard
! [[Tests/Keyboard_mappings|keyboard]]
| no
| no
| no
| no
Line 300: Line 349:
| no
| no
|-
|-
! language
! [[Tests/Language_settings|language]]
| no
| no
| no
| no
Line 316: Line 365:




[[Category:Test plans]]
[[Category:Test Plans]]
[[Category:Tests]]
[[Category:Test results]]

Latest revision as of 03:36, 27 January 2011

End Users: see Release notes/8.1.0 for more relevant information.

Schedule & Features

The focus of this build is to

  1. fix bugs
  2. get the suspend/resume working for better battery life
  3. get the first level activity isolation for security
  4. backup of laptops to the school server
  5. there is a new wireless driver - some rewriting

Please also refer to the milestone and the official roadmap for schedule dates and features for each release.

Test Plans

A 1hr smoke test should be performed with each candidate, and the smoke test results should be reported in Test Group Release Notes.

Major Foci

Power Management

  1. Suspend/resume (does not apply to B4 machines)
  2. Network mgr SEGV, #4960
  3. ohm logfile #5457, optional
    • The code for this doesn't exist yet, so the testcase doesn't either.  :) Cjb 14:26, 23 January 2008 (EST)

Security/Upgrades

(does not apply to B4 machines)

  1. Activation w/USB, w/school server
    • does "activation" mean upgrade to another build? --Chihyu 11:13, 22 January 2008 (EST)
  2. olpc-update --usb without manually upgrading olpc-update. (#3881)
  3. Basic activity isolation (#5033, #5346)
  4. Automatic Time sync (#3359)
    • how to test this without using more than one machine? --Chihyu 12:08, 22 January 2008 (EST)
  5. orphaned files, #5637
    • cannot create test cases due to the disappearance of original bug description --Chihyu 12:29, 22 January 2008 (EST)
  6. Mesh - upgrade under loaded network - Scott

Wireless/USB/Network Manager

  1. Wireless Test Cases, issues to address:
    • Intermittent connection to jabber service, #4193
    • Network sniffing script, #5153
    • WPA working from sugar, #5258
    • WEP working from sugar, #5485

Tubes/Presence (Scaling/Performance)

  1. Remove N squared scaling issue in groups (#4965): test this once #5311 is fixed. --Chihyu 17:05, 24 January 2008 (EST)
    • add other XOs as friends
    • confirm that friends are recorded in /home/olpc/.sugar/default/friends
    • confirm that the school server also keeps the same record
      • has a tool been developed to test this? --Chihyu 18:21, 24 January 2008 (EST)
  2. eJabberd server issues/bugs (#5934, #5313, #5315, #5311 - cut traffic, #5310 - olpc rpm)
    • waiting for the fixes from Process One --Chihyu 17:28, 24 January 2008 (EST)
  3. Looking into Openfire as backup (#5223, #5224)
    • this is a task, not a bug to be tested --Chihyu 18:21, 24 January 2008 (EST)
  4. Point Update1 to different jabber server, #5933
    • this is a task, not a bug to be tested --Chihyu 18:21, 24 January 2008 (EST)
  5. Scaling? #5817
  6. Link local presence (Salut) - OLPC/Collabora
    • need help with main issues and test cases --Chihyu 18:21, 24 January 2008 (EST)

Completed Test Cases

  1. Browse
  2. Write
  3. Record
  4. Paint
  5. Chat
  6. Other activities
  7. User-Focused Test Plans. (Some test cases should already be covered in the previous sections).

Incomplete Test Cases

Please help revise/create test cases in Test Plans

Others

  1. 802.11s Connectivity Test Plan - Connectivity, wifi protocol
  2. Tinderbox Testing - automated testing for each build
  3. Journal/datastore
    • need help with main issues and test cases --Chihyu 18:21, 24 January 2008 (EST)

Scaling Tests

Australia/conference tests

Test Results

See also Testing Update.1 Results.

Template

  1. If the table has not been linked to any test cases, either the test cases have not been created, or please help us locate the test cases!
  2. When you sign under each reported result, simply type 4 tildes: ~~~~ instead of typing your username, date, and time.
  3. Values to fill in:
    • no: not tested
    • good: tested and passed
    • If a defect is found, please file a trac ticket and fill in the ticket number in the table. For example, 5969.
Build Test Item English
684 keyboard good

Chihyu 13:56, 17 January 2008 (EST)

language 5969

Chihyu 17:47, 28 January 2008 (EST)

future
candidate 		keyboard no
language no


Suspend/resume

Build Suspend Mode Sleep Mode
(power button) 		Sleep Mode
(lid closing) 		Ebook Mode Restarting Sugar Battery Mode Disable
Suspend/Resume
RC1-690 no no no no no no no
RC2-691 no no no no no no no

Security/Upgrades

Build Isolation Upgrade Backward Comp
RC1-690 no no no
RC2-691 no no no

Wireless/USB/Network Manager

Build Wireless Salut Issue WPA Bug WEP Bug USB Time Sync Network mgr
RC1-690 no no no no no no no
RC2-691 no no no no no no no

Tubes/Presence (Scaling/Performance)

Build Friends Conf_Scale
RC1-690 no no
RC2-691 no no

Core Activities

Please add the testing results for other core activities in this section!

Build Browse Chat Paint Record Write
RC1-690 no no no no no
RC2-691 no no no no no

User-Focused Test Plans

Build SD Card SD Card as USB drive
RC1-690 no no
RC2-691 no no

Keyboard/Languages

Build Test Item English Spanish Arabic Mongolian Devanagari Thai Nigerian Amharic Urdu Nepali Pashto Portuguese
RC1-690 keyboard no no no no no no no no no no no no
language no no no no no no no no no no no no
RC2-691 keyboard no no no no no no no no no no no no
language no no no no no no no no no no no no
Retrieved from "http://wiki.laptop.org/mediawiki/index.php?title=Update.1_tests&oldid=252056"