How laptop delivery breaks

Revision as of 15:41, 15 November 2008 by Aaronmk (talk | contribs)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search
Stop hand.png WARNING:
The content of this section is considered
It is preserved for historical or documenting reasons.

This page is an analysis by outside volunteers of current issues that are known, or believed to be, the major causes of XO Laptop non-delivery.

For an explanation of how laptop delivery was hypothetically supposed to work, see How laptop delivery works.

Note from the Author(s)

This page contains both incomplete and inaccurate information. Heroic attempts have been made to verify what's here and fill in the gaps, but in many cases the information should be considered little more than educated guesswork. (For an example of misuse of the information given here - a lazy journalist taking speculation and supposition here and reporting it as if it were verified fact - see articles such as this one at [Ars Technica]. )

In particular, note that the main author of the current page, User:Thatseattleguy, has no official contact or sources of information within OLPC or any of its contractors. In light of this, nothing here should be taken to represent the official position of OLPC, nor should the lack of response from OLPC to anything here be taken to mean they believe it to be accurate. Keep this in mind as you read.

Please make sure edits and additions you have to offer confirm to the topic at hand: the current OLPC laptop delivery systems, how they may go awry, and general responses of Give One Get One donors to these problems. Venting, discussion of individual circumstances, or instructions to OLPC or its contractors belong in the community forums or in other wiki pages.

Finally, note that OLPC itself is hosting this page and making the user-editable wiki it is part of freely available to all. In doing so, it permits content both here and elsewhere that is critical of itself and its decisions. The author believes the willingness of the organization to provide such a forum should be taken into account in any discussion of its openness, transparency, and commitment to community.

Potential causes of shipping delays and "lost" orders

Some PayPal orders received duplicate confirmation numbers

As an affected customer, I noticed this when using the OLPC tracking page at <>

If I entered my eMail address it reported my shipment was delayed or "in the queue". If I entered the confirmation number from my OLPC confirmation eMail, it produced a FedEx tracking number showing "my" system delivered hundreds of miles from here, months ago.

According to the Donor Services Hotline at 800-201-7144, a number of PayPal orders were discovered with duplicate confirmation numbers, and the second digit was changed to a '1' to create a new unique number - apparently without notifying the recipient. If I use "710004...." instead of "700004...." on the tracking page, the response agrees with that for my eMail address. It remains to be seen if an XO is really in the queue for me...

It arrived! By USPS PriorityMail, with Delivery Confirmation, from Brightstar, with "PO: 710004....". The OLPC tracking site says, "Your order has been shipped. A tracking number is not available at this time." Apparently they don't get the USPS confirmation numbers.

The shipping address contained more than one address line

A severe bug in order fulfillment software written for the G1G1 program by OLPC (or its contractors) caused the shipping addresses of many Give One Get One donors who paid via Paypal and other indirect means to be truncated in the customer service and shipping databases. This happened if the street address had more than one line - as, for example, anyone who tried to ship to their work address "care of" their company's name. This apparently affected a huge number of donor's orders.

For example, a hypothetical G1G1 Donor "Nick" might have had his normal work address transmitted from Paypal to OLPC as:

Nick Blancoputty
c/o Laptops Aren't Us, Inc
1234 Scrod Lane
Boston, MA 10506

But in OLPC database, as originally sent to PatriotLLC, their order processing and customer service contractor, and therefore in the actual database that Brightstar is attempting to ship from - it would have been recorded only as:

Nick Blancoputty
c/o Laptops Aren't Us, Inc
Boston, MA 10506

...which obviously is not a valid shipping address. The problem can also manifest in other ways, more subtle. For example, an original address like:

Maria Lee Jetson
4567 Pixel Place
Building 4, Apartment 123C
Chicago, IL 60609

...would have been recorded as simply::

Maria Lee Jetson
4567 Pixel Place
Chicago, IL 60609

While this last address likely will pass validation as "shippable", it's missing vital information (building, apartment, or suite number; or corporate mail stop number). This may cause the package to be lost, or rejected, as the simple street address alone could reference be a high rise apartment or office building with hundreds of tenants.

Donors affected by this issue have little recourse other than to contact PatriotLLC to attempt to get a shippable address in the system. There have been reports that this process has been successful for some, but not for the majority, of those in this category.

Internal OLPC and contractor database errors may have lost order updates

The databases at one or more of OLPC itself, OLPC customer service (Patriot) and/or the shipper (Brighstar's) databases are known to have internal integrity issues, such that order changes - most importantly, corrected shipping addresses - may be overwritten or dropped by subsequent updates or syncronization activities. (The databases are reportedly based on Microsoft SQL Server, but would be controlled and populated with custom programming by OLPC, Patriot and Brightstar).

Many donors affected by issues the two issues above have called or emailed customer service at Patriot numerous times to provide corrected information - such providing as the lines that were dropped from their shipping address - but the original data would later mysteriously reappear in the database, with the later changes lost. Patriot representatives have insisted they are following correct procedures to update customer information; if this is true, then it may be the fault of the custom programming or the way the database is managed. For example, an operational or programming procedure that would cause partial or complete overwriting of records in one database (e.g., Patriot's) with older or simply different information from the other database (e.g., OLPC's or Brightstar's) would explain these observations and continuing issues.

Note that PatriotLLC representatives have claimed they are only using data provided by OLPC and are not responsible for the database errors; similarly, Brightstar claims they can only ship units based on confirmed and shippable provided by OLPC and Patriot. There is, unfortunately, no sure way to know from the outside whether these claims accurately describe the true situation between the three entities. It is clear, however, that whatever the accuracy of the original data sent by OLPC to PatriotLLC, later database corrections taken by Patriot customer service staff by phone and email from donors have gone effectively unrecorded on a large and continuing basis.

Donors affected by these issues may have no recourse but to keep contacting Patriot by phone (reportedly more effective than email, which may or may not be acknowledged or processed by them) in the hope that one of the changes will eventually "take" and the order will be sent to shipping.

Order changes may be lost between the fulfillment contractors

Changes and corrections to orders taken by customer service (Patriot) may not have been received or processed by shipping (Brightstar). OLPC's customer service contractor, PatriotLLC, and their shipping contractor, Brightstar Inc, reportedly do not share a common database or have access to a common order fulfillment system, as would be the industry norm. Order changes, such as corrections to shipping addresses, taken by Patriot have been reported by some to be provided to Brightstar by the sending of single email messages, one email per change. There are obviously a large number of places such a system could fail and the corrected address not be processed by Brightstar, similarly to the issues described above.

Donors affected by this issue may have no recourse but to keep contacting Patriot by phone (reportedly more effective than email, which is rarely answered or acknowledged by them) in the hope that one of the changes will eventually "take".

The FedEx address verifier may have wrongly flagged addresses as unshippable

There have been many reports that the FedEx-supplied address-verification system being used is: (1) much too sensitive to small, ultimately inconsequential variations in addresses, (2) prone to having a high "false-negative" rate (i.e., flagging an address as unshippable when it's actually fine), and (3) only intended by FedEx to be used as a tool for address correction, not as a bright-line test for whether an address can truly be shipped to by FedEx or not. Others have suggested that it's possible OLPC or Patriot program that interprets the output from this tool may not recognize that certain error conditions do not actually mean that the address is unshippable. (Reportedly, for example, the tool presents an error message if any apartment number is present in the address, but of course that does not mean that FedEx will be unable to deliver to an apartment.) All of these reports are only speculation, but if true, the use of this system for other than its originally intended purpose, or in a way that misuses or misinterprets its output, could be explained by OLPC's and Brightstar's unfamiliarity and relative inexperience with delivery of products directly to single end-user addresses via common carrier, rather than to large distribution facilities.

Donors affected by this issue may be able to resolve their problems by specifying an alternate shipping address that the verification tool approves of, even if it's not their preferred shipping address.

The online version of the FedEx Address Checker can be found at:

As a customer affected by the FedEx address verification failure, I was told by the Donor Services Hotline on 19 March 2008 that "confirmed" addresses (presumably those verified at the Address Correction number 800-883-8102 or via Donor Services or eMail or volunteers) are no longer checked against the FedEx database. Shipment will be to the confirmed address, and it is up to the recipient to work out any problems with FedEx once they receive their tracking number. [Later... My XO arrived via USPS Priority Mail from Brightstar, FedEx was not involved and there was never a tracking number from either carrier. I fail USPS address verification, too, so I'm not sure why they chose USPS over FedEx.]

FedEx delivery may have failed without the donor being aware

If the laptop is returned by FedEx as "unable to deliver" (for any reason, which may or may not have to do with the donor-provided shipping address), or it is being returned under an RMA (Return Material Authorization - a defective unit replacement request), it is shipped directly to the Brightstar warehouse. Unfortunately, Brightstar reportedly doesn't have any donor email addresses, nor the ability to send out messages to one or all donors. Notifications about the returned shipment must go back to PatriotLLC, hopefully then being sent out to the donor in question. This takes extra time, with obvious and numerous places where the information could get lost before the donor is informed.

Status emails may have been intercepted by donor spam filters

There are numerous reports that mails originating from OLPC and/or PatriotLLC relating to order status or problems appear abnormally "spammy" to many ISP junk mail filters, for unknown reasons, possibly the specific nature of the text or HTML markup used in them. Affected mail handlers have even included Gmail - often considered the "gold standard" of error-free spam filtering - which definitely marked some of the 21 January 2008 emails as junk/spam, and reportedly a large percentage of OLPC-sent email from other times as well.

Donors who have not received emails relating to their unshipped orders would be advised to check their "spam" or "junk" folders to see if any lost OLPC mails are there, and also to whitelist all senders with addresses at "" or "".

OLPC may have had inaccurate inventory data

There simply may not have been enough XO laptops currently built and delivered to the US to satisfy all donors' orders at various points in the Give One Get One fulfillment cycle. For example, a posting in the 26 January 2008 blog of Walter Bender, President of OLPC Foundation, indicated a shortage of US-labeled keyboards and power supplies would delay delivery of most remaining Give One Get One (G1G1) orders until "early March".

It's not clear from this entry why OLPC thought a week earlier that nearly all of the 85,000 G1G1 donor systems had been delivered, but then found there were not enough machines on hand to satisfy those same orders. Presumably OLPC would be expected to know with a high degree of precision how many laptops have been built, shipped, and available in their US distribution center. For many donors, the sudden discovery of a significant gap between the available machines and the total number of orders in the G1G1 program (which had ended several weeks earlier, on 12/31/07) was not easy to understand.

The initial shipping address was a PO Box

Brightstar's contract with FedEx does not provide for shipping to PO Boxes. When it was realized by OLPC that Paypal orders could legitimately have PO Boxes as their shipping addresses, and that many Give One Get One donors had used these, the response was that Patriot call center staff (already busy with normal customer service activities) should call or email the affected donors to get a non-PO-Box address. Needless to say, many of these donors were never contacted, and those who were may not have had their corrected addresses updated successfully and permanently in the Brightstar shipping database (see above).

The second, and apparently current, solution was for Brightstar to open a special account with UPS (some reports say USPS), which will be able deliver to PO Boxes. Shipping the remain PO Box orders via UPS/USPS will presumably fix the problem for these donors.

Emails sent to OLPC by donors may have been intercepted by OLPC's spam filters

According to one report, OLPC's internet provider ( put an aggressive spam filter in place that served to rejected many inbound email message sent to addresses at "". If true, this left some donors unable to contact OLPC easily via email to provide new information or to request order status.

Emails from OLPC volunteers ( may have been caught by spam filters

There are several known instances of donors having contacted OLPC through their help ticketing system and then believing the organization to be unresponsive because of getting no reply when the problem was actually that their emails were blocking those replies (sent from help at laptop dot org). While this typically does not directly affect the speed at which a donor's laptop reaches them, it may affect their ability to contact OLPC about their lost laptop and get and update information about their shipment. In some cases where the shipment is stuck because of missing information, this removes one more possible route to escalation.

Dedicated customer service phone for resolution of address issues

OLPC or PatriotLLC sent some donors an email message on Friday, 19 January 2008, saying "We are in the process of setting up a 24 hour dedicated phone line for donors". The purpose of this dedicated number is apparently limited only to correction of address-failure issues. The number given is 800-883-8102; this was operational as of Monday 21 January 2008. Donors are also invited to email their shipping information to if they prefer. The G1G1 Fulfillment Information page in this wiki suggests contacting help @ .

Donors who called the number reported it appeared only to be a secondary means of getting updated address information into Patriot, and that no help, new information, investigation, or order status updates are available to callers using it.

The same email message included an offer for a partial refund (the $199 + shipping "Get One" portion of their donation) if the donor prefers this over continuing to await shipment of their XO.

Resulting criticism of OLPC by donors

Criticism of OLPC

There has been much comment online concerning the above order fulfillment problems, especially among the donors whose shipments were or still are affected by them. It is commonly held that a majority of issues were "self-inflicted" by OLPC Foundation and especially, by its order fulfillment contractors. Many donors believe these problems were foreseeable, easily preventable, or at the least could have been corrected many weeks earlier in the shipment process.

An officially reported 10%, or about 8500, of US donors' orders were unshipped as of 1/19/2008, but independent estimates of these numbers (derived from automatic scripts that test all possible confirmation numbers on the shipment tracking page) have put the potential percentage higher, perhaps as much as 20%, or 17,000 orders. (However, this higher number would include undelivered Canadian orders as well, which OLPC has consistently told donors will ship later than US ones.) In either case, the magnitude of the systems failure is so great that OLPC's claims to not have known order fulfillment problems existed until very recently is understandably difficult for many donors to fathom. This has added fuel to the already extensive online commentary among those same donors that OLPC has been almost willfully oblivious to the problems: far too slow to recognize that they even existed, unwilling to expend internal resources to resolving them quickly, too trusting of their third-party order-fulfillment contractors (PatriotLLC and Brightstar) to deal with them properly, and ultimately, unwilling or unable to exert oversight and pressure on these same contractors to resolve the issues. Some have speculated that internal politics have played a role in creating this situation, as these same contractors are also members of OLPC Board and are providing at least some of their services on a pro bono basis.

Because of inconsistent or absent communication by OLPC concerning order fulfillment issues, and responses seen as failing to acknowledge problems or "blaming the victim", many unshipped-to donors currently perceive OLPC as being aloof, uncaring, and defensive. Many have also said they would be greatly more patient with the absence of their laptop if the organization had been truthful and forthcoming in its responses and willing to take responsibility for the mistakes it and its contractors had made.

Criticism of third-party contractors

Compounding these perceptions were numerous, well-documented missteps by the outside order processing, and customer service contractor retained by OLPC to manage the order fulfillment process. The contractor's design and implementation of their order processing, support, and communications infrastructure was poor by almost any modern standard, and particularly so for a high-visibility program involving tens of thousands of online donors. One early example of these were the order confirmation email notices sent to many early donors in early December 2007, which did not include the internal order/confirmation number from the OLPC system. This left donors in this group that had "lost" or "stuck" orders with no viable means to track them or reference them with the contractor's customer service representatives.

Other examples include the contractor's installation of a touchtone phone call routing system that was not tested and did not actually accept any touchtones for over a week after it was announced and publicized, or the "track your shipment" web page, which only returned useful information well after the donor's order had shipped, and otherwise returned an "invalid reference number" message. This of course which did not assist the donors that had "stuck" orders which were not considered shippable by their or the shipper's database. Others have criticized the customer service representatives hired by this contractor, noting many broken promises of returned calls (promised to be made by the representative themselves or "a supervisor") and the wildly varying order status and shipping information given out to the same donor by different representatives on different days - or sometimes even during the same day.

(An example list of some of these issues as discussed at length by affected donors, many weeks before first acknowledged by OLPC, can be found on community forums such as this.)

Effects on donors' perception of OLPC and its mission

Numerous donors affected by the order fulfillment problems have complained in online forums concerning them, and OLPC's handling of same. The general perception is that such fundamental failures and missteps, such as not being able to correctly handle an order with a two-line address:

  • should not have existed in any reasonably-designed order-fulfillment system in the first place;
  • should have been automatically corrected by OLPC, and its outside contractors much more quickly after they were found (e.g., by going back to the source Paypal confirmation data to retrieve the missing address fields from the orders);
  • if not correctable, should have been proactively reported to each affected donor immediately;
  • should not have been blamed on the donors themselves (e.g., by claiming all address issues were caused by the use of PO Boxes);
  • should not have required repeated calls and emails to customer service representatives to correct problems and to receive accurate order status; and
  • should have been quickly and officially acknowledged by OLPC, or by OLPC and its contractors together.

Because of the problems and a perceived near-total lack of communication by the organization (as donors were rarely contacted to explain and correct the issues with their orders), many were left wondering and waiting, with no idea anything was wrong with their order except that their XO laptop had not yet arrived. In fact, the lack of accurate, timely, and transparent information from OLPC is often cited as the greatest source of frustration for the affected donors - many of whom where the greatest proponents of OLPC's mission and goals, and were among the first to donate via the G1G1 program.

Reportedly the G1G1 program delivery failures have cost OLPC potential new sponsorships and orders; for an anecdotal example, see forum postings like this. In the end, a number of donors have questioned the ability of OLPC to carry out its primary mission -- the shipment of millions of laptops to children in developing countries -- needs to be reexamined in light of the logistical failures seen in the much-smaller North American distribution program.

Other donors regret the trouble that this extra program has apparently caused in implementation and do not believe it reflects on OLPC's larger mission. Rather, some hope that despite the challenges, OLPC will offer G1G1 or similar opportunities to purchase and sponsor these laptops in the future.