Developers/Issues: Difference between revisions

From OLPC
Jump to navigation Jump to search
(Arranging and updating the text.)
(Finish basic integration and updating of the page.)
Line 1: Line 1:
{{Developers}}
{{Developers}}
[[Developers/Stack|Previous]] [[Developers/Projects|Next]]
[[Developers/Stack|Previous]] [[Developers/Projects|Next]]

This page tries to point out the major differences between coding for the OLPC project, and particularly targeting the [[Hardware|OLPC-XO]], and coding for traditional desktops. Most of the advice here is useful even for other systems, but on the XO, resource limitations can make "useful" turn "critical".


= Localization =
= Localization =


The OLPC project is deploying in a large number of countries, each of which has unique cultures and languages. To be accessible to the children in these countries we have to internationalize and localize our software for each country. Regardless of which Activity stack you are working on, your project will need to be localized.
The OLPC project is deploying in a large number of countries, each of which has unique cultures and languages. To be accessible to the children in these countries we have to internationalize and localize our software for each country. Regardless of which Activity stack you are working on, your project will need to be localized into potentially ''hundreds'' of languages.


The quick summary of how to make your project internationalizable is to:
The quick summary of how to prepare your project for internationalization is to:


* always remember that you will need to internationalize (so avoid building assumptions of culture into the activity)
* always remember that you will need to internationalize (so avoid building assumptions of culture into the activity)
* use gettext for all user-facing strings, text, icons and the like
* use [Gettext|gettext] for all user-facing strings, text, icons and the like
* use unicode text for rendering and GUI elements


If you do that the job of internationalizing won't be finished, but it will be reasonably straightforward and won't require a lot of retrofitting.
If you do that as you code, the job of internationalizing won't be finished, but it should be reasonably straightforward and won't require a lot of retrofitting.


See: [[Localization]] for extensive documentation on how to Localize your Activity
See: [[Localization]] for extensive documentation on how to Localize your Activity
See: [http://downloads.egenix.com/python/LSM2005-Developing-Unicode-aware-applications-in-Python.pdf Developing Unicode Aware Applications in Python] (PDF), 31-slide presentation on how to work with Unicode
See: [[Gettext]] and it's [http://www.gnu.org/software/gettext documentation]
Tip: You can create a fake translation with lengthy text containing non-ASCII characters. It is common to do this with Cyrillic (Russian) and Greek characters that are shaped similar to English characters. Does the text get rendered? Are any graphical elements too small? East-Asian characters may need extra height, along with some of the double-accented characters used in Eastern Europe. Right-to-left scripts like Arabic should also be considered.


= Processor =
= Hardware Constraints =


While Sugar can run on almost any computer, the target of the OLPC project is to run on extremely inexpensive machines, and particularly the [[Hardware|OLPC-XO]]. As such, you need to target your projects to the constraints of the hardware.
While Sugar can run on almost any computer, the target of the OLPC project is to run on extremely inexpensive machines, and particularly the [[Hardware|OLPC-XO]]. As such, you need to target your projects to the constraints of the hardware. The CPU in the machines is the rough equivalent of a 400MHz PIII processor from the turn of the century.


= Efficiency =
== Modularity is Good ==

Many OLPC-XOs will have to be hand-cranked or will need to last long periods between recharges. Every erg of energy is precious.

Our power-saving approach requires the ability to put the entire machine to sleep when it's not doing anything. If your activity is constantly doing silly things the machine may not be able to go to sleep at all.

Disable your activity's operations as soon as possible (i.e. as soon as you determine that you are not needing to do any more calculation). Avoid GUI operations that are "just for show" and don't improve the educational experience. Try only to calculate or render in response to events, don't render 30fps if your data only changes once every 10-20 minutes.

Try not to have your activity running constantly or waking frequently, the laptop is intended to "go to sleep" as quickly and often as possible, and your activity's trivial loop may prevent that sleeping. Use asynchronous interfaces such as "[http://docs.python.org/lib/module-select.html select]" rather than "polling" constantly for changes.

See: Dave Jones' paper [http://lwn.net/Articles/192214/ "Why Userspace Sucks"] Note that most of the particular problems identified are already being worked on, we want to avoid making similar mistakes!

Consider rewriting key sections of slow code in your high-level languages in lower-level and more efficient code. Improving the performance of your code by a factor of ten may let the machine go to sleep 3 or 4 times as often.

= Modularity is Good =


Breaking out the functionality of your application into separately-loadable plugins or modules can allow for substantial memory and/or storage savings.
Breaking out the functionality of your application into separately-loadable plugins or modules can allow for substantial memory and/or storage savings.
Line 29: Line 49:
If possible, implement optional components as a separate Python module and delete the .pyc file when a user chooses not to use that option. Remove objects when they are no longer needed using del.
If possible, implement optional components as a separate Python module and delete the .pyc file when a user chooses not to use that option. Remove objects when they are no longer needed using del.


== Memory leaks ==
= Memory leaks =

An OLPC-XO only has 256MB of memory.


Normally when programming in high-level garbage-collected languages such as Python, Javascript or Smalltalk you normally do not need to worry about garbage collection. In lower-level language development, including extensions for high-level languages you need to be more on your guard.
Normally when programming in high-level garbage-collected languages such as Python, Javascript or Smalltalk you normally do not need to worry about garbage collection. In lower-level language development, including extensions for high-level languages you need to be more on your guard.
Line 37: Line 59:
Remember that every leak, however small, counts!
Remember that every leak, however small, counts!


== Storage ==
= Storage =


An OLPC-XO has 1GB of storage using a [[JFFS2]] compressing, wear-leveling file-system. JFFS2 compression tends to provide about 2:1 compression for text and typical data, with no extra compression on already compressed data.
An OLPC-XO has 1GB of storage using a [[JFFS2]] compressing, wear-leveling file-system. JFFS2 compression tends to provide about 2:1 compression for text and typical data, with no extra compression on already compressed data.
Line 43: Line 65:
It is possible to install JFFS2 on a Linux machine should you feel you need particularly precise measurements of your storage usage. This likely isn't practically necessary, try to keep the size down and use the zip-compressed size as an approximate estimate of the JFFS2 storage requirement.
It is possible to install JFFS2 on a Linux machine should you feel you need particularly precise measurements of your storage usage. This likely isn't practically necessary, try to keep the size down and use the zip-compressed size as an approximate estimate of the JFFS2 storage requirement.


On flash file-systems write performance is generally slow, while random access is actually very good. Performance is glacial if the file system is low on space and has to continually erase freed blocks before writing (JFFS2 attempts to do this in the background, but if it can't....).
== Power Usage ==

Programs that continually write to the file system without need are anti-social; wear levelling helps flash longevity, but it has limits, and writing burns power.

Writable file mappings (via the mmap system call) may not be supported.

= Display =

The OLPC-XO has a dual-mode screen that has a high-resolution black and white mode. Every application should be coded to be usable in black and white graphics mode, mostly this is a matter of choosing display colours with different luminance values and high contrast.

The effective resolution in color mode is somewhat lower than in grayscale, even though the frame buffer will always be in high resolution. If your application does not honor the font resolution change-on-the-fly mechanisms, or hardwires fonts by pixel size, your application won't be able to switch modes gracefully and may require manual intervention to be usable. Please fix.

The OLPC-XO hardware does support alpha blending (Porter-Duff compositing), so we expect to eventually support this well, but at the moment we do '''not'' support it.

The OLPC-XO does not have any 3D hardware support. OpenGL can be used, but it is not included by default (so will bloat your activity size considerably) and will not have any hardware support. In theory you could use the MMX and Enhanced 3DNow! engine to provide reasonably 3D rendering support, but this is still just a theory.

= Networks =


You should not rely on having an Internet connection, let alone a fast or reliable one. Latency on satellite-uplink connections can be extremely bad, so lots of round-trip messages to a server may make your activity unusable for children in the field, even though it seems to work perfectly everywhere else.


The OLPC wireless network allows peer-to-peer networking with no Internet access. Normally this would be via a mesh network for OLPC-XOs away from school, but disconnected access-point configurations (i.e. a school with no active Internet uplink) will work in the same way.


[[Avahi]] and [[Telepathy]]-salut provide local-network discovery of applications even when there is no Internet uplink available. Try to make your activity usable on the local link.
* '''Ability to operate applications when running in black/white mode'''
:Judicious use of contrasting colors will mean that your program automatically works in B&W mode.
* '''The effective resolution''' in color mode is somewhat lower in color than grayscale: some applications will find this hard, even though the frame buffer will always be in high resolution. If your application does not honor the font resolution change-on-the-fly mechanisms, or hardwires fonts by pixel size, your application won't be able to switch modes gracefully and may require manual intervention to be usable. Please fix.
* '''The hardware''' does support alpha blending (Porter-Duff compositing), so we expect to eventually support this well
* '''No 3D hardware:''' we don't have shaded triangles or a geometry pipe; so any OpenGL used will be based on Mesa, and without hardware, will be very slow
:Ideally, don't use OpenGL at all except for specialty applications which absolutely must and which can accept the slow rendering. For instance a design program could use OpenGL to provide a rendered view because most of the time the user will not be using OpenGL.
:Remember that the [[Geode]] supports MMX and Enhanced 3DNow!, which can speed up some applications, such as 3D rendering, and should be appropriately used.
* '''Choice of libraries and required applications:''' you may not have the dependencies you might need, or those dependencies might come at too high a memory cost. We will inventory what you can "count on" in the basic system as it becomes clear.
: In the meantime, make sure you know your application's dependencies. For compiled code, run it with "ldd" to get a list of all the libraries being loaded. For Python code, run it with "python -v" to get a list of all modules imported.
* '''Graphical activity that does not turn itself off quickly;''' we want to save power
:This means avoid cute animated GIFs that do not add value to the educational experience. (That is one example of not letting the CPU be idle) Your users may have to exert themselves physically to provide the electricity to run their OLPC. Respect them and their effort.
* '''CPU performance:''' the system is slow relative to current desktops, though fast relative to desktops at the turn of the millenium.
* '''Power consumption:''' if your application is CPU bound for long periods (not letting the CPU be idle), or routinely requires itself to be run (can't be suspended well), this isn't good
* While we are a great fan of '''interpreted languages''', key CPU bound kernels had better be in compiled code, or your performance and power consumption will be poor.
:Python is an interpreted language, similar in operation to perl. It lacks a JIT, though it does use an intermediate code form rather than strictly interpreting the raw text as a shell script or makefile would. It is thus inherently slower than C# or Java, but not as bad as it could be. A more serious problem for Python is often memory consumption.
* Remember that your '''slow code''' has a direct impact on power consumption (far more than on a desktop), and its usability. Applications that run slowly or don't let the processor idle will be very, very unpopular on battery powered machines that may be powered by children having to run a generator.
* '''The file system''' is a flash file system, so its write performance is slow, while random access is actually very good. The performance is glacial if the file system is low on space and has to continually erase freed blocks before writing ([[JFFS2]] attempts to do this in the background, but if it can't....). Programs that continually write to the file system without need are anti-social; wear leveling helps flash longevity, but it certainly doesn't help, and burns power. Writable file mappings (via the mmap system call) may not be supported.
* '''Looping waiting for events (or other busy-waiting) eats power;''' don't do it. Poll and select with timeouts are your friends. Don't gratuitously wake up at frequent intervals just to test if something has happened; design your hardware and software to be completely idle between events they have to respond to.
:Right now this is a nice summary page. But items like this deserve some more explanation along with some sample code that uses select/poll/inotify/signals instead of busy-waiting.
* '''Keep in mind that the OLPC wireless network is peer-to-peer.''' Design applications accordingly. We can presume at least some technology like mDNS (e.g. Avahi) is available for discovery.
* '''Applications should be localizable.'''
: If you will be using Python, start by reading these 31 slides about how to use Unicode in Python: http://downloads.egenix.com/python/LSM2005-Developing-Unicode-aware-applications-in-Python.pdf
: Next, read the [[Gettext|page on gettext]] and it's [http://www.gnu.org/software/gettext documentation]. Become familiar with GTK+ and Pango features for I18N and L10N.
: You can create a fake translation with lengthy text containing non-ASCII characters. It is common to do this with Cyrillic (Russian) and Greek characters that are shaped similar to English characters. Does the text get rendered? Are any graphical elements too small? East-Asian characters may need extra height, along with some of the double-accented characters used in Eastern Europe. Right-to-left scripts like Arabic should also be considered.
* '''Read and comprehend Dave Jones' paper on "Why Userspace Sucks";''' a summary and pointers to the paper can be found at [http://lwn.net/Articles/192214/ LWN]. Note that most of those problems are now being worked on; please do not make similar mistakes!
* '''What kind of applications are needed?'''
:have a look at [[Sample Applications]] for some ideas. Or, release an early prototype of your own app as a sample application to help others learn the ropes of developping for this innovative, i.e. non-standard laptop.


[[Developers/Stack|Previous]] [[Developers/Projects|Next]]
[[Developers/Stack|Previous]] [[Developers/Projects|Next]]

Revision as of 01:46, 16 December 2007

Previous Next

This page tries to point out the major differences between coding for the OLPC project, and particularly targeting the OLPC-XO, and coding for traditional desktops. Most of the advice here is useful even for other systems, but on the XO, resource limitations can make "useful" turn "critical".

Localization

The OLPC project is deploying in a large number of countries, each of which has unique cultures and languages. To be accessible to the children in these countries we have to internationalize and localize our software for each country. Regardless of which Activity stack you are working on, your project will need to be localized into potentially hundreds of languages.

The quick summary of how to prepare your project for internationalization is to:

  • always remember that you will need to internationalize (so avoid building assumptions of culture into the activity)
  • use [Gettext|gettext] for all user-facing strings, text, icons and the like
  • use unicode text for rendering and GUI elements

If you do that as you code, the job of internationalizing won't be finished, but it should be reasonably straightforward and won't require a lot of retrofitting.

See: Localization for extensive documentation on how to Localize your Activity See: Developing Unicode Aware Applications in Python (PDF), 31-slide presentation on how to work with Unicode See: Gettext and it's documentation Tip: You can create a fake translation with lengthy text containing non-ASCII characters. It is common to do this with Cyrillic (Russian) and Greek characters that are shaped similar to English characters. Does the text get rendered? Are any graphical elements too small? East-Asian characters may need extra height, along with some of the double-accented characters used in Eastern Europe. Right-to-left scripts like Arabic should also be considered.

Processor

While Sugar can run on almost any computer, the target of the OLPC project is to run on extremely inexpensive machines, and particularly the OLPC-XO. As such, you need to target your projects to the constraints of the hardware. The CPU in the machines is the rough equivalent of a 400MHz PIII processor from the turn of the century.

Efficiency

Many OLPC-XOs will have to be hand-cranked or will need to last long periods between recharges. Every erg of energy is precious.

Our power-saving approach requires the ability to put the entire machine to sleep when it's not doing anything. If your activity is constantly doing silly things the machine may not be able to go to sleep at all.

Disable your activity's operations as soon as possible (i.e. as soon as you determine that you are not needing to do any more calculation). Avoid GUI operations that are "just for show" and don't improve the educational experience. Try only to calculate or render in response to events, don't render 30fps if your data only changes once every 10-20 minutes.

Try not to have your activity running constantly or waking frequently, the laptop is intended to "go to sleep" as quickly and often as possible, and your activity's trivial loop may prevent that sleeping. Use asynchronous interfaces such as "select" rather than "polling" constantly for changes.

See: Dave Jones' paper "Why Userspace Sucks" Note that most of the particular problems identified are already being worked on, we want to avoid making similar mistakes!

Consider rewriting key sections of slow code in your high-level languages in lower-level and more efficient code. Improving the performance of your code by a factor of ten may let the machine go to sleep 3 or 4 times as often.

Modularity is Good

Breaking out the functionality of your application into separately-loadable plugins or modules can allow for substantial memory and/or storage savings.

For example, GAIM supports just about every IM protocol in existence, but we can only anticipate 3 being common. Leaving off the unneeded plugins saves both memory and flash-storage footprint.

For Python development, don't import modules at the top. Instead, import them just where they are needed in the code and then unload the module. This will require some work with ihooks and imp to support unload() but for now, just use a dummy function in your code.

If possible, implement optional components as a separate Python module and delete the .pyc file when a user chooses not to use that option. Remove objects when they are no longer needed using del.

Memory leaks

An OLPC-XO only has 256MB of memory.

Normally when programming in high-level garbage-collected languages such as Python, Javascript or Smalltalk you normally do not need to worry about garbage collection. In lower-level language development, including extensions for high-level languages you need to be more on your guard.

It is a good idea to check for memory leaks at least a few times during the development process. Sample the memory size every minute and graph it. If there is a slowly increasing size, then you have a leak. A normal program will increase rapidly at the beginning and then remain perfectly flat or shrink and grow repeatedly.

Remember that every leak, however small, counts!

Storage

An OLPC-XO has 1GB of storage using a JFFS2 compressing, wear-leveling file-system. JFFS2 compression tends to provide about 2:1 compression for text and typical data, with no extra compression on already compressed data.

It is possible to install JFFS2 on a Linux machine should you feel you need particularly precise measurements of your storage usage. This likely isn't practically necessary, try to keep the size down and use the zip-compressed size as an approximate estimate of the JFFS2 storage requirement.

On flash file-systems write performance is generally slow, while random access is actually very good. Performance is glacial if the file system is low on space and has to continually erase freed blocks before writing (JFFS2 attempts to do this in the background, but if it can't....).

Programs that continually write to the file system without need are anti-social; wear levelling helps flash longevity, but it has limits, and writing burns power.

Writable file mappings (via the mmap system call) may not be supported.

Display

The OLPC-XO has a dual-mode screen that has a high-resolution black and white mode. Every application should be coded to be usable in black and white graphics mode, mostly this is a matter of choosing display colours with different luminance values and high contrast.

The effective resolution in color mode is somewhat lower than in grayscale, even though the frame buffer will always be in high resolution. If your application does not honor the font resolution change-on-the-fly mechanisms, or hardwires fonts by pixel size, your application won't be able to switch modes gracefully and may require manual intervention to be usable. Please fix.

The OLPC-XO hardware does support alpha blending (Porter-Duff compositing), so we expect to eventually support this well, but at the moment we do 'not support it.

The OLPC-XO does not have any 3D hardware support. OpenGL can be used, but it is not included by default (so will bloat your activity size considerably) and will not have any hardware support. In theory you could use the MMX and Enhanced 3DNow! engine to provide reasonably 3D rendering support, but this is still just a theory.

Networks

You should not rely on having an Internet connection, let alone a fast or reliable one. Latency on satellite-uplink connections can be extremely bad, so lots of round-trip messages to a server may make your activity unusable for children in the field, even though it seems to work perfectly everywhere else.

The OLPC wireless network allows peer-to-peer networking with no Internet access. Normally this would be via a mesh network for OLPC-XOs away from school, but disconnected access-point configurations (i.e. a school with no active Internet uplink) will work in the same way.

Avahi and Telepathy-salut provide local-network discovery of applications even when there is no Internet uplink available. Try to make your activity usable on the local link.

Previous Next