XO-1.5 Wireless Test Report

From OLPC
Revision as of 15:55, 9 May 2011 by Skierpage (talk | contribs) (Category:Test results)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

This is the test report for the thin firmware and libertastf driver for the XO-1.5.

Introduction and Methodology

OLPC explicitly limited the testing in the contract to reduce cost. Therefore, cozybit was unable to develop an extensive battery of automated tests and only the barest tests necessary to validate the operation of the sd8686 thin firmware and drivers. All testing has been done using basic commands, scripts of commands, or programs that can be downloaded and built on the XO, as well as an external sniffer. All validation utilizes manual verification of program output.

Environment Setup

There are a number of things that need to be done to the XO-1.5's default setup to successfully develop and test this software on it. For convenience, we changed the global setup of the XO instead of including these adjustments in the test scripts.

1. Disable power save mode.
2. Disable NetworkManager
# /sbin/chkconfig NetworkManager off
3. Add external USB Ethernet port. This is necessary to run test scripts and see results. While it is possible to use the terminal directly on the XO, or use the serial port terminal, using an external Ethernet port and sshing into the device is easier. To setup the port on boot, you can add the following script as /etc/init.d/ethup:
#!/bin/bash
ifconfig eth1 up
dhclient eth1
Substitute the appropriate device name. After adding the script, add a link in rc5.d so it gets called on startup:
ln -s ../init.d/ethup /etc/rc5.d/S50ethup
4. Install the thinfirmware to /lib/firmware/sd8686tf.bin
5. Install the new kernel with the libertas_tf and libertas_tf_sdio drivers as appropriate.
6. Install all necessary programs and commands with yum. In particular, the following programs and commands are needed to perform these tests or build the test programs: make, gcc, git, iw, iwconfig, libpcap, tcpdump, iperf.

Tests

Validate basic STA mode

This test checks for the most basic operation of an 802.11 STA: can it connect to an AP and transmit and recieve packets.

criteria

The XO must be able to use the thin firmware and libertas_tf_sdio driver to probe, authenticate and associate to an AP. Data packets must be both transmitted and received.

test

Use iwconfig and ifconfig to setup the interface and associate to an AP. Ping the AP from the XO.
# iwconfig wlan0 channel 6
# iwconfig wlan0 essid stevetest
# ifconfig wlan0 up 192.168.77.10
# ping 192.168.77.1

result - PASSED

The XO1.5 with thin firmware was able to authenticate and associate to the AP. Pings were sent and received.
All management and data frames were seen with the packet sniffer.
[root@localhost ~]# ping -c 5 192.168.77.1
PING 192.168.77.1 (192.168.77.1) 56(84) bytes of data.
64 bytes from 192.168.77.1: icmp_seq=1 ttl=64 time=3.24 ms
64 bytes from 192.168.77.1: icmp_seq=2 ttl=64 time=2.68 ms
64 bytes from 192.168.77.1: icmp_seq=3 ttl=64 time=2.95 ms
64 bytes from 192.168.77.1: icmp_seq=4 ttl=64 time=2.71 ms
64 bytes from 192.168.77.1: icmp_seq=5 ttl=64 time=2.81 ms

--- 192.168.77.1 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4009ms
rtt min/avg/max/mdev = 2.684/2.882/3.241/0.211 ms

Leak testing

An XO-1.5 with the thin firmware must be able to run over a long term without permanently losing buffer space.

criteria

iperf must run through at least 12 hours and continue to send packets for throughput measuring.

test

1. Start iperf on a different computer connected to the same AP
# iperf -s
2. Setup and connect the XO-1.5 to the same AP as the iperf server.
# iwconfig wlan0 channel 6
# iwconfig wlan0 essid stevetest
# ifconfig wlan0 up 192.168.77.10
3. Run iperf overnight with periodic outputs
iperf -c 192.168.77.20 -t 43200 -i 1200
The above sets iperf to transmit for 12 hours and print reports every 20 minutes

result - PASSED

iperf was successfully transmitting 12 hours later, with the server receiving packets.

Rate testing

An XO-1.5 with the thin firmware must have TX feedback enabled and exhibit the following:
  1. TX failure and retry feedback from the firmware to the driver.
  2. Use of mac80211 rate adaptation algorithm.

criteria

  1. Using dmesg output, the driver prints differing feedback messages from the firmware.
  2. Inspection of radiotap headers of packets captured by wireshark show multiple rates above 1.0 mbps.

test

1. The XO-1.5 needs to be connected to an AP as in prior tests.
2. Setup debugging output from libertas_tf:
# echo 0x01000000 > /sys/module/libertas_tf/parameters/libertas_tf_debug
3. Run iperf server on a different host, as well as iperf client on the XO-1.5
# iperf -c 192.168.77.100
4. After a few moments, unplug the AP the XO is connected to.
5. Examine both Wireshark output and dmesg logs.

result - PASSED

1. dmesg output reflects differing retry counts and failures:
[ 1213.331537] lbtf_sdio statistics: Got feedback event. retry: 10, failure: 0
[ 1213.331547] libertas_tf statistics: skb is ok
[ 1213.373193] lbtf_sdio statistics: **EVENT** 0x80000
[ 1213.373204] lbtf_sdio statistics: Got feedback event. retry: 8, failure: 0
[ 1213.373214] libertas_tf statistics: skb is ok
[ 1213.399584] lbtf_sdio statistics: **EVENT** 0x90000
[ 1213.399594] lbtf_sdio statistics: Got feedback event. retry: 9, failure: 0
[ 1213.399604] libertas_tf statistics: skb is ok
[ 1213.413286] lbtf_sdio statistics: **EVENT** 0xA0000
[ 1213.413296] lbtf_sdio statistics: Got feedback event. retry: 10, failure: 0
[ 1213.413306] libertas_tf statistics: skb is ok
[ 1213.426304] lbtf_sdio statistics: **EVENT** 0xA0000
[ 1213.426314] lbtf_sdio statistics: Got feedback event. retry: 10, failure: 0
[ 1213.426323] libertas_tf statistics: skb is ok
[ 1213.428776] lbtf_sdio statistics: **EVENT** 0xA0000
[ 1213.428785] lbtf_sdio statistics: Got feedback event. retry: 10, failure: 0
[ 1213.428794] libertas_tf statistics: skb is ok
[ 1213.440281] lbtf_sdio statistics: **EVENT** 0xA0000
[ 1213.440291] lbtf_sdio statistics: Got feedback event. retry: 10, failure: 0
[ 1213.440301] libertas_tf statistics: skb is ok
[ 1213.459788] lbtf_sdio statistics: **EVENT** 0xA0000
[ 1213.459799] lbtf_sdio statistics: Got feedback event. retry: 10, failure: 0
[ 1213.459809] libertas_tf statistics: skb is ok
[ 1230.014608] lbtf_sdio statistics: **EVENT** 0x1000000
[ 1230.014621] lbtf_sdio statistics: Got feedback event. retry: 0, failure: 1
[ 1230.014632] libertas_tf statistics: skb is ok
[ 1230.511866] lbtf_sdio statistics: **EVENT** 0x1000000
[ 1230.511877] lbtf_sdio statistics: Got feedback event. retry: 0, failure: 1
[ 1230.511887] libertas_tf statistics: skb is ok
[ 1231.012125] lbtf_sdio statistics: **EVENT** 0x1000000
[ 1231.012137] lbtf_sdio statistics: Got feedback event. retry: 0, failure: 1
2. Wireshark radiotap headers showed rate values from 1.0 Mbps to 54 Mbps.

Basic throughput

Test the throughput of an XO-1.5 running the thin firmware

criteria

There is no target performance value. This test is informational only.

test

1. Start iperf on a different computer connected to the same AP:
# iperf -s
2. Setup and connect the XO-1.5 to the same AP as the iperf server.
# iwconfig wlan0 channel 6
# iwconfig wlan0 essid stevetest
# ifconfig wlan0 up 192.168.77.10
3. Run iperf
iperf -c 192.168.77.100 -t 120

result - 0.8 to 1.1 Mbps

We have seen results varying between 0.8 and 1.1 Mbps. Below is one iperf run: 

------------------------------------------------------------
Client connecting to 192.168.77.100, TCP port 5001
TCP window size: 16.0 KByte (default)
------------------------------------------------------------
[  3] local 192.168.77.10 port 53242 connected with 192.168.77.100 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-120.3 sec  12.1 MBytes    845 Kbits/sec

Verify management frames TX

An XO-1.5 with the thin firmware must be able to TX 802.11 management frames.

criteria

Send management frames and see those on the air using Wireshark

test

Certain management frames can triggered by the simple test of connecting to an AP. These include Authentication and Association frames. This test uses Packetvector to construct other management frames, including Null, Beacon, Probe Request and Probe Response frames. Packetvector can be retrieved from git://github.com/derosier/packetvector.git
The commit used in these tests is e937b380dfeb0a820363
1. Create a monitor interface for Packetvector:
iw dev wlan0 interface add mon0 type monitor
iwconfig wlan0 channel 6
iwconfig mon0 channel 6
ifconfig mon0 up
2. Use Packetvector to send frames:
# ./packetvector mon0

result - PASSED

Packetvector's frames were seen in Wireshark. Other management frames, including Authentication and Association frames were also seen.

Verify monitor interface

An XO-1.5 with the thin firmware must be able to do the following with a monitor interface:
  1. Use a monitor interface to inject traffic including management frames.
  2. Receive packets in promiscuous mode, including management frames.

criteria

Management and data frames for various destinations are both transmitted and received. Transmitted frames can be viewed in Wireshark. Received frames can be captured in tcpdump.

test

Test part A, TX - see the previous test Test Report#Verify_management_frames_TX
Test part B, RX:
1. Create a monitor interface:
iw dev wlan0 interface add mon0 type monitor
iwconfig wlan0 channel 6
iwconfig mon0 channel 6
ifconfig mon0 up
2. Sniff with tcpdump:
# tcpdump -i mon0

result - PASSED

Test part A, TX: Transmitted management packets were received.
Test part B, RX: tcpdump was able to see and decode packets received, including management packets and packets not intended for the XO:
16:31:47.437204 1.0 Mb/s 2437 MHz (0x00a0) antenna 112 [0x0000000e] Probe Response (stevetest) [1.0* 2.0* 5.5* 11.0* 6.0 9.0 12.0 18.0 Mbit] CH: 6
16:31:47.479353 1.0 Mb/s 2437 MHz (0x00a0) antenna 56 [0x0000000e] Beacon () [1.0* 2.0* 5.5* 11.0* 6.0 12.0 24.0 36.0 Mbit] ESS CH: 6, PRIVACY
16:31:47.481073 1.0 Mb/s 2437 MHz (0x00a0) antenna 112 [0x0000000e] Data IV:b73999 Pad 20 KeyID 1
16:31:47.483933 1.0 Mb/s 2437 MHz (0x00a0) antenna 56 [0x0000000e] Data IV:b93999 Pad 20 KeyID 1
16:31:47.535390 1.0 Mb/s 2437 MHz (0x00a0) antenna 168 [0x0000000e] Beacon (stevetest) [1.0* 2.0* 5.5* 11.0* 6.0 9.0 12.0 18.0 Mbit] ESS CH: 6
16:31:47.538268 1.0 Mb/s 2437 MHz (0x00a0) antenna 56 [0x0000000e] Beacon (cmr dsl) [1.0* 2.0* 5.5* 11.0* 6.0 Mbit] ESS CH: 6, PRIVACY
16:31:47.579993 1.0 Mb/s 2437 MHz (0x00a0) antenna 240 [0x0000000e] Beacon () [1.0* 2.0* 5.5* 11.0* 6.0 12.0 24.0 36.0 Mbit] ESS CH: 6, PRIVACY
16:31:47.596084 1.0 Mb/s 2437 MHz (0x00a0) antenna 128 [0x0000000e] Beacon (7434 4740) [1.0* 2.0* 5.5* 11.0* 22.0 Mbit] ESS CH: 6, PRIVACY
16:31:47.633599 1.0 Mb/s 2437 MHz (0x00a0) antenna 120 [0x0000000e] Beacon (stevetest) [1.0* 2.0* 5.5* 11.0* 6.0 9.0 12.0 18.0 Mbit] ESS CH: 6
16:31:47.699067 1.0 Mb/s 2437 MHz (0x00a0) antenna 208 [0x0000000e] Beacon (7434 4740) [1.0* 2.0* 5.5* 11.0* 22.0 Mbit] ESS CH: 6, PRIVACY

Verify AP mode

This test utilizes hostapd to turn the XO-1.5 into a software AP. The firmware has special commands added to facilitate the use of a software AP.

criteria

Can a different STA authenticate and associate to an XO-1.5 acting in AP mode. Can that STA then ping the host AP XO-1.5?

test

hostapd must be used for this test. Instructions on using the XO as a soft AP via hostapd are already on the wiki on the XO as AP page. The version of hostapd this test was performed with is commit: 41e650ae5c90a83a0c22989d43527901347dacc0
1. Follow the instruction on the XO as AP page to build and configure hostapd. A .config file for building this version is here File:Dot config hostapd.txt, save it and rename to .config. A configuration file for running hostapd is here File:Hostapd.conf.
2. Assign an IP address of 192.168.78.1 to wlan0:
 # ifconfig wlan0 192.168.78.1
2. Connect to the SSID set on your hostapd configuration from a second computer.
3. Assign the IP 192.168.78.20 to the second computer:
 # ifconfig wlan0 192.168.78.20
4. From the second computer, ping the XO:
# ping -c 5 192.168.78.1

result - PASSED

The second computer was able to ping the XO AP:
[root@localhost ~]# ping -c 5 192.168.78.1
PING 192.168.78.1 (192.168.78.1) 56(84) bytes of data.
64 bytes from 192.168.78.1: icmp_seq=1 ttl=64 time=3.24 ms
64 bytes from 192.168.78.1: icmp_seq=2 ttl=64 time=2.68 ms
64 bytes from 192.168.78.1: icmp_seq=3 ttl=64 time=2.95 ms
64 bytes from 192.168.78.1: icmp_seq=4 ttl=64 time=2.71 ms
64 bytes from 192.168.78.1: icmp_seq=5 ttl=64 time=2.81 ms

--- 192.168.77.1 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4009ms
rtt min/avg/max/mdev = 2.684/2.882/3.241/0.211 ms

Verify mesh

Check that the XO will be able to use the mesh stack in mac80211 by verifying that the mesh IEs appear in the management frames.

criteria

An XO-1.5 with the thin firmware must transmit beacon frames with the mesh information elements.

test

1. Create a mesh interface:
# iw dev wlan0 interface add mesh0 type mesh mesh_id XOmesh
2. Use Wireshark to verify the mesh elements in the beacon.

result - PASSED

The mesh elements make it out in the beacon.

note

Mesh is currently broken in mac80211. Due to this, we were unable to connect with another mesh node. Both the libertas_tf driver and the thin firmware properly support mesh and when mac80211 is fixed, this support will become available once again.

Verify software encryption

criteria

An XO-1.5 with the thin firmware must authenticate and associate to a WPA secured AP via the wpa_supplicant.

test

1. Encode your preshared key:
# wpa_passphrase mywireless "secretpassphrase"
2. Configure wpa_supplicant by creating a configuration file /etc/wpa_supplicant/wpa-supplicant.conf as follows, with the information from the wpa_passphrase program:
ctrl_interface=/var/run/wpa_supplicant
ctrl_interface_group=wheel
network={
        ssid="WPA_AP"
        scan_ssid=1
        proto=WPA
        key_mgmt=WPA-PSK
        psk=...
}
3. Start wpa_supplicant:
# ifconfig wlan0 up
# wpa_supplicant -Dwext -i wlan0 -c /etc/wpa_supplicant/wpa_supplicant.conf
4. Get an ip address:
# dhclient wlan0
5. Ping the AP:
# ping -c 5 192.168.1.1

result - PASSED

The XO was able to authenticate and associate to the WPA protected AP.
# wpa_supplicant -Dwext -i wlan0 -c /etc/wpa_supplicant/wpa_supplicant.conf
CTRL-EVENT-DISCONNECTED - Disconnect event - remove keys
ioctl[SIOCGIWSCAN]: Resource temporarily unavailable
ioctl[SIOCSIWSCAN]: Device or resource busy
Failed to initiate AP scan.
Trying to associate with 00:13:10:fc:57:ff (SSID='cozybit' freq=2412 MHz)
Associated with 00:13:10:fc:57:ff
WPA: Key negotiation completed with 00:13:10:fc:57:ff [PTK=CCMP GTK=TKIP]
CTRL-EVENT-CONNECTED - Connection to 00:13:10:fc:57:ff completed (auth) [id=0 id_str=]
^z

# bg
# ping -c 5 192.168.1.1
PING 192.168.1.1 (192.168.1.1) 56(84) bytes of data.
64 bytes from 192.168.1.1: icmp_seq=1 ttl=64 time=5.95 ms
64 bytes from 192.168.1.1: icmp_seq=2 ttl=64 time=1.16 ms
64 bytes from 192.168.1.1: icmp_seq=3 ttl=64 time=1.21 ms
64 bytes from 192.168.1.1: icmp_seq=4 ttl=64 time=1.18 ms
64 bytes from 192.168.1.1: icmp_seq=5 ttl=64 time=1.29 ms

--- 192.168.1.1 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4006ms
rtt min/avg/max/mdev = 1.166/2.163/5.951/1.894 ms


Known Issues and Limitations

  1. Mesh is broken in mac80211. While the firmware and driver have been shown to support mesh, until mac80211 is fixed, mesh will not work properly.
  2. A number of issues were found during development that are beyond scope of this project. See: Steve's Thinfirm 1.5 developer notes
Retrieved from "http://wiki.laptop.org/mediawiki/index.php?title=XO-1.5_Wireless_Test_Report&oldid=255926"