802.11s Connectivity Test Plan

Test plan for the OLPC 802.11s implementation

This is a first attempt to systematically describe testing procedures for the Marvell/OLPC 802.11s implementation on the XO laptop.

[Note: this document is based mostly on tests that have been already performed by many people in different environments. ]

The main concept behind our implementation is that of functional splitting of the networking stack between the WiFi radio and the Host CPU. The ARM processor driving the 88W8388 radio processes Layer-2 frames, while the Host CPU processes packets (IP or any other Layer-3 protocol).

This architecture allows for power savings (since only the Marvell SoC radio is necessary for frame forwarding) and flexibility with upper layer protocols (since it is not depending on any specific Layer-3 protocol - automatically supporting IPv6 for example). It also presents testing challenges in the laptop context since there is no direct user interaction with the ARM processor of the radio.

Testing should concentrate on the following areas:

• Path discovery and management
• Communications integrity
• Performance testing
• Firmware reliability and memory resources management

Given that the implementation is completely distributed, all of the above parameters are stressed as the number of nodes increases. This mandates that test suites are run multiple times with an increasing number of nodes every time.

Path discovery and management

The main purpose of this testing is to ensure proper operation of the core frame forwarding functionality. Assuming a set of nodes with no disconnected members every node should be able to communicate with every other node.

This test can be implemented via a series of any-to-all IP ping sessions with the ping originator mode changing after each set of pings so that all the nodes have the opportunity to originate pings. It is important to have multi-hop topologies during this test so that path discovery and broadcast propagation are exercised. One way to achieve that is to limit the TX power of the radios. Given that the purpose here is to verify frame forwarding, it is desirable that this test is performed in a radio-quiet space so that frame losses due to external radio traffic are minimized. In such an environment, scalability should be tested be gradually increasing the number of nodes (>50).

Communications integrity

Given the complexity of interaction between the firmware on the wireless radio, the driver and the upper networking stack, it is important to test that data streams between applications do not exhibit any corruption. Our experience here indicates that a good way to test that using existing tools is via the use of the SCP command (copy over an SSH encrypted channel) since any bit flipping will trigger checksum errors in the protocol.

SCP implementations exist for all major platforms, so the test is not limited to a Linux host by any means.

Performance testing

With the core functionality established, testing should extend to performance. The first scenario to be tested is the multi-hop chain’s performance characterization.

The test should be performed using the Iperf tool with both TCP and UDP streams between the end nodes of the chain and should be repeated with an increasing number of nodes (1<n<10).

There are two ways to perform this test:

• “lab” environment, where all nodes are close to each other and the topology is forced upon them by means of selective blinding. This is useful to establish upper performance limit expectations for the real world test as well as uncover node-specific issues (driver performance etc.)
• “real” environment where the topology is actually enforced by the radio link range between nodes and where performance will be significantly lower due to the fact the interference range of any node is larger than its communications range (besides the fact that radio links will not be able to operate at maximum speeds).

A more complex topology will allow stressing intermediate nodes by having them forward traffic for more than one communication sessions.

In this example concurrent iperf sessions between nodes 1 and 4, 3 and 8, 2 and 10 will stress node 9. It is also important to use this topology to measure broadcast/multicast frame loss rates by means of having every node transmit a set of multicast frames at various rates while their reception is recorded on every other node.

Another important test topology is the dense deployment scenario where, for example, up to 60 laptops in a classroom are trying to access the internet via a single Internet gateway.

In a scenario like this, besides ensuring that laptops can indeed access the internet, what also needs to be tested are potential benefits of multi-hop vs single-hop paths to the Internet gateway given the capability for faster link speeds as well as lower TX power in the multi-hop case.

Firmware reliability and memory resources management

Given the limited amount of memory resources on the radio SoC, it is important to establish that continuous operation for several days in a row is possible. Cycling through the test suite described above for a week at a time will ensure that.

File:802.11s Connectivity Test Plan.doc - Original document

Test cases from Marvell for the OLPC XOs

1 Driver / Firmware version ethtool -i eth0