Developers/Issues: Difference between revisions

From OLPC
Jump to navigation Jump to search
(Move the localisation background text from developer's program over)
(Undo revision 256658 by JZA (Talk))
 
(9 intermediate revisions by 6 users not shown)
Line 2: Line 2:
[[Developers/Stack|Previous]] [[Developers/Projects|Next]]
[[Developers/Stack|Previous]] [[Developers/Projects|Next]]


This page tries to point out the major differences between coding for the OLPC project, and particularly targeting the [[Hardware|OLPC-XO]], and coding for traditional desktops. Most of the advice here is useful even for other systems, but on the XO, resource limitations can make "useful" turn "critical".
This page tries to point out the major differences between coding for the [[Hardware|OLPC-XO]], and coding for traditional desktops. Most of the advice here is useful even for other systems, but on the XO, resource limitations and diverse deployment can make "useful" turn "critical".


= Memory and Processor Usage =
There is a major difference between the OLPC system and a conventional laptop of approximately five years ago; that is, our base system software is much more capable than it was then. The Linux environment now supports internationalization capability for scripts that were out of our reach then, and much higher quality rendering and, best of all, a much wider range of applications.


Fundamentally, the OLPC-XO hardware which is our primary target platform has only 1024 MB of storage (Flash), and 256 MB of RAM on a single-core processor. This requires developers to pay much more attention to the memory "footprint" of applications. Developers may need to adjust attitudes and make tough choices to keep the system within this footprint.
This has come at a cost, however: Moore's "Law" has allowed us to become sloppy, both in memory usage and CPU usage; this tends to force us to make some tough choices to keep the "footprint" of the software acceptable.


While laptops increase in power and size along [http://en.wikipedia.org/wiki/Moore%27s_law Moore's Law], the OLPC project will opt for less expensive hardware and wider distribution. The Linux community has become much more sensitive to issues of "bloat", has provided internationalization capabilities for scripts, higher quality rendering and, best of all, a much wider range of applications.
The OLPC-XO hardware which is our primary target platform has only 1024 MB of storage (Flash), and 256 MB of RAM on a single-core processor. Over the last year or so, the wider Linux community has become much more sensitive to these issues, and work is well underway toward reigning in this "bloat" and performance work in general. Small is beautiful (and usually faster) for everyone.

Small is beautiful (and usually faster) for everyone.


= Localization =
= Localization =
Line 20: Line 22:
The quick summary of how to prepare your project for internationalization is to:
The quick summary of how to prepare your project for internationalization is to:


* always remember that you will need to internationalize (so avoid building assumptions of culture into the activity)
* always remember that you will need to internationalize, so avoid building assumptions of culture into the activity
* use [Gettext|gettext] for all user-facing strings, text, icons and the like
* use [[Gettext|gettext]] for all user-facing strings, text, icons and the like
* use unicode text for rendering and GUI elements
* use Unicode text for rendering and GUI elements


If you do that as you code, the job of internationalizing won't be finished, but it should be reasonably straightforward and won't require a lot of retrofitting.
If you do that as you code, the job of internationalizing won't be finished, but it should be reasonably straightforward and won't require a lot of retrofitting.
Line 33: Line 35:
= Processor =
= Processor =


While Sugar can run on almost any computer, the target of the OLPC project is to run on extremely inexpensive machines, and particularly the [[Hardware|OLPC-XO]]. As such, you need to target your projects to the constraints of the hardware. The CPU in the machines is the rough equivalent of a 400MHz PIII processor from the turn of the century.
While Sugar can run on almost any computer, the target of the OLPC project is to run on extremely inexpensive machines, and particularly the [[Hardware|OLPC-XO]]. As such, you need to target your projects to the constraints of the hardware. The CPU in the machines is the rough equivalent of a 500MHz Athlon processor from the turn of the century. This includes instruction set and performance.


= Efficiency =
= Efficiency =
Line 65: Line 67:
Normally when programming in high-level garbage-collected languages such as Python, Javascript or Smalltalk you normally do not need to worry about garbage collection. In lower-level language development, including extensions for high-level languages you need to be more on your guard.
Normally when programming in high-level garbage-collected languages such as Python, Javascript or Smalltalk you normally do not need to worry about garbage collection. In lower-level language development, including extensions for high-level languages you need to be more on your guard.


It is a good idea to check for memory leaks at least a few times during the development process. Sample the memory size every minute and graph it. If there is a slowly increasing size, then you have a leak. A normal program will increase rapidly at the beginning and then remain perfectly flat or shrink and grow repeatedly.
It is a good idea to check for memory leaks at least a few times during the development process. Sample the memory size (see [[Memory leak testing]] for instructions) every minute and graph it. If there is a slowly increasing size, then you have a leak. A normal program will increase rapidly at the beginning and then remain perfectly flat or shrink and grow repeatedly.


Remember that every leak, however small, counts!
Remember that every leak, however small, counts!
Line 87: Line 89:
The effective resolution in color mode is somewhat lower than in grayscale, even though the frame buffer will always be in high resolution. If your application does not honor the font resolution change-on-the-fly mechanisms, or hardwires fonts by pixel size, your application won't be able to switch modes gracefully and may require manual intervention to be usable. Please fix.
The effective resolution in color mode is somewhat lower than in grayscale, even though the frame buffer will always be in high resolution. If your application does not honor the font resolution change-on-the-fly mechanisms, or hardwires fonts by pixel size, your application won't be able to switch modes gracefully and may require manual intervention to be usable. Please fix.


The OLPC-XO hardware does support alpha blending (Porter-Duff compositing), so we expect to eventually support this well, but at the moment we do '''not'' support it.
The OLPC-XO hardware does support alpha blending (Porter-Duff compositing), so we expect to eventually support this well, but at the moment we do ''not'' support it.


The OLPC-XO does not have any 3D hardware support. OpenGL can be used, but it is not included by default (so will bloat your activity size considerably) and will not have any hardware support. In theory you could use the MMX and Enhanced 3DNow! engine to provide reasonably 3D rendering support, but this is still just a theory.
The OLPC-XO does not have any 3D hardware support. OpenGL can be used, but it is not included by default (so will bloat your activity size considerably) and will not have any hardware support. In theory you could use the MMX and Enhanced 3DNow! engine to provide reasonably 3D rendering support, but this is still just a theory.
Line 101: Line 103:
= Security =
= Security =


Your activities will have to work within the [[OLPC Bitfrost]] security system. Bitfrost is a rather intrusive approach to security from the developer's perspective (while to the user it is quite transparent). This is by design. To work within Bitfrost you will often need to consider the security ramifications of what you are doing.
Your code will have to work within the [[OLPC Bitfrost]] security system. While to the user it is quite transparent, from the developer's perspective Bitfrost is a rather intrusive approach to security. This is by design. To work within Bitfrost you will often need to consider the security ramifications of what you are doing.


Consider carefully whether you need certain types of operation, if you do you will need to flag your activity with a special security "bit". Some such permissions are, by default, mutually exclusive. If you need those mutually exclusive functions, you will require either an officially signed activity, or an explicit authorization by each child to be run on their security-enabled laptop.
Consider carefully whether you need certain types of operation, if you do you will need to flag your activity with a special security "bit". Some such permissions are, by default, mutually exclusive. If you need those mutually exclusive functions, you will require either an officially signed activity, or an explicit authorization by each child to be run on their security-enabled laptop.
Line 117: Line 119:
** store all user data within the [[Journal]]
** store all user data within the [[Journal]]
** unrestricted file access for a file-type -- by default not available if network access is available (to prevent automated uploading of e.g. all of a child's documents to an attacker), if you need both, you need to be signed/explicitly-enabled
** unrestricted file access for a file-type -- by default not available if network access is available (to prevent automated uploading of e.g. all of a child's documents to an attacker), if you need both, you need to be signed/explicitly-enabled
** constrain all file-writing to the ${SUGAR_ACTIVITY_ROOT}
** constrain all file-writing to be only within the ${SUGAR_ACTIVITY_ROOT} (available as an environmental variable passed to your Activity instance)
** file writes -- rate-limited by default to prevent flash storage exhaustion (if you are planning to stream video to flash storage, you likely need to consider this!)
** file writes -- rate-limited by default to prevent flash storage exhaustion (if you are planning to stream video to flash storage, you likely need to consider this!)
* camera/microphone access -- special flag to be allowed to ''ask'' for access, 30 minute timed access after which re-request required
* camera/microphone access -- special flag to be allowed to ''ask'' for access, 30 minute timed access after which re-request required

Latest revision as of 02:37, 31 May 2011

Previous Next

This page tries to point out the major differences between coding for the OLPC-XO, and coding for traditional desktops. Most of the advice here is useful even for other systems, but on the XO, resource limitations and diverse deployment can make "useful" turn "critical".

Memory and Processor Usage

Fundamentally, the OLPC-XO hardware which is our primary target platform has only 1024 MB of storage (Flash), and 256 MB of RAM on a single-core processor. This requires developers to pay much more attention to the memory "footprint" of applications. Developers may need to adjust attitudes and make tough choices to keep the system within this footprint.

While laptops increase in power and size along Moore's Law, the OLPC project will opt for less expensive hardware and wider distribution. The Linux community has become much more sensitive to issues of "bloat", has provided internationalization capabilities for scripts, higher quality rendering and, best of all, a much wider range of applications.

Small is beautiful (and usually faster) for everyone.

Localization

The OLPC project is deploying in a large number of countries, each of which has unique cultures and languages. To be accessible to the children in these countries we have to internationalize and localize our software for each country. Regardless of which Activity stack you are working on, your project will need to be localized into potentially hundreds of languages.

Our base technology choices have been predicated on the ability of the software to achieve the best overall worldwide "user experience". This requirement drove our choice of GTK+ and Pango (with Cairo as the graphical underpinnings) as our primary GUI system. Pango's abilities in complex scripts are currently the most advanced of free software technologies.

Other development toolkits can be used: but they come at a cost in memory and flash footprint, and today, in the ability of software based on them to be localized to many of the scripts we face immediately, which include both Thai and Arabic. Including other toolkits as a standard part of our base system is therefore problematic, and experience on embedded systems show that including multiple toolkits would almost certainly cause the overall experience to suffer.

The quick summary of how to prepare your project for internationalization is to:

  • always remember that you will need to internationalize, so avoid building assumptions of culture into the activity
  • use gettext for all user-facing strings, text, icons and the like
  • use Unicode text for rendering and GUI elements

If you do that as you code, the job of internationalizing won't be finished, but it should be reasonably straightforward and won't require a lot of retrofitting.

See: Localization for extensive documentation on how to Localize your Activity See: Developing Unicode Aware Applications in Python (PDF), 31-slide presentation on how to work with Unicode See: Gettext and it's documentation Tip: You can create a fake translation with lengthy text containing non-ASCII characters. It is common to do this with Cyrillic (Russian) and Greek characters that are shaped similar to English characters. Does the text get rendered? Are any graphical elements too small? East-Asian characters may need extra height, along with some of the double-accented characters used in Eastern Europe. Right-to-left scripts like Arabic should also be considered.

Processor

While Sugar can run on almost any computer, the target of the OLPC project is to run on extremely inexpensive machines, and particularly the OLPC-XO. As such, you need to target your projects to the constraints of the hardware. The CPU in the machines is the rough equivalent of a 500MHz Athlon processor from the turn of the century. This includes instruction set and performance.

Efficiency

Many OLPC-XOs will have to be hand-cranked or will need to last long periods between recharges. Every erg of energy is precious.

Our power-saving approach requires the ability to put the entire machine to sleep when it's not doing anything. If your activity is constantly doing silly things the machine may not be able to go to sleep at all.

Disable your activity's operations as soon as possible (i.e. as soon as you determine that you are not needing to do any more calculation). Avoid GUI operations that are "just for show"; those which don't improve the educational experience. Try only to calculate or render in response to events, don't render 30fps if your data only changes once every 10-20 minutes.

Try not to have your activity running constantly or waking frequently, the laptop is intended to "go to sleep" as quickly and often as possible, and your activity's trivial loop may prevent that sleeping. Use asynchronous interfaces such as "select" or "INotify" rather than "polling" constantly for changes.

See: Dave Jones' paper "Why Userspace Sucks" Note that most of the particular problems identified are already being worked on, we want to avoid making similar mistakes!

Consider rewriting key sections of slow code in your high-level languages in lower-level and more efficient code. Improving the performance of your code by a factor of ten may let the machine go to sleep 3 or 4 times as often.

Modularity is Good

Breaking out the functionality of your application into separately-loadable plugins or modules can allow for substantial memory and/or storage savings.

For example, GAIM supports just about every IM protocol in existence, but we can only anticipate 3 being common. Leaving off the unneeded plugins saves both memory and flash-storage footprint.

For Python development, don't import modules at the top. Instead, import them just where they are needed in the code and then unload the module. This will require some work with ihooks and imp to support unload() but for now, just use a dummy function in your code.

If possible, implement optional components as a separate Python module and delete the .pyc file when a user chooses not to use that option. Remove objects when they are no longer needed using del.

Memory leaks

An OLPC-XO only has 256MB of memory.

Normally when programming in high-level garbage-collected languages such as Python, Javascript or Smalltalk you normally do not need to worry about garbage collection. In lower-level language development, including extensions for high-level languages you need to be more on your guard.

It is a good idea to check for memory leaks at least a few times during the development process. Sample the memory size (see Memory leak testing for instructions) every minute and graph it. If there is a slowly increasing size, then you have a leak. A normal program will increase rapidly at the beginning and then remain perfectly flat or shrink and grow repeatedly.

Remember that every leak, however small, counts!

Storage

An OLPC-XO has 1GB of storage using a JFFS2 compressing, wear-leveling file-system. JFFS2 compression tends to provide about 2:1 compression for text and typical data, with no extra compression on already compressed data.

It is possible to install JFFS2 on a Linux machine should you feel you need particularly precise measurements of your storage usage. This likely isn't practically necessary, try to keep the size down and use the zip-compressed size as an approximate estimate of the JFFS2 storage requirement.

On flash file-systems write performance is generally slow, while random access is actually very good. Performance is glacial if the file system is low on space and has to continually erase freed blocks before writing (JFFS2 attempts to do this in the background, but if it can't....).

Programs that continually write to the file system without need are anti-social; wear levelling helps flash longevity, but it has limits, and writing burns power.

Writable file mappings (via the mmap system call) may not be supported.

Display

The OLPC-XO has a dual-mode screen that has a high-resolution black and white mode. Every application should be coded to be usable in black and white graphics mode, mostly this is a matter of choosing display colours with different luminance values and high contrast.

The effective resolution in color mode is somewhat lower than in grayscale, even though the frame buffer will always be in high resolution. If your application does not honor the font resolution change-on-the-fly mechanisms, or hardwires fonts by pixel size, your application won't be able to switch modes gracefully and may require manual intervention to be usable. Please fix.

The OLPC-XO hardware does support alpha blending (Porter-Duff compositing), so we expect to eventually support this well, but at the moment we do not support it.

The OLPC-XO does not have any 3D hardware support. OpenGL can be used, but it is not included by default (so will bloat your activity size considerably) and will not have any hardware support. In theory you could use the MMX and Enhanced 3DNow! engine to provide reasonably 3D rendering support, but this is still just a theory.

Networks

You should not rely on having an Internet connection, let alone a fast or reliable one. Latency on satellite-uplink connections can be extremely bad, so lots of round-trip messages to a server may make your activity unusable for children in the field, even though it seems to work perfectly everywhere else.

The OLPC wireless network allows peer-to-peer networking with no Internet access. Normally this would be via a mesh network for OLPC-XOs away from school, but disconnected access-point configurations (i.e. a school with no active Internet uplink) will work in the same way.

Avahi and Telepathy-salut provide local-network discovery of applications even when there is no Internet uplink available. Try to make your activity usable on the local link.

Security

Your code will have to work within the OLPC Bitfrost security system. While to the user it is quite transparent, from the developer's perspective Bitfrost is a rather intrusive approach to security. This is by design. To work within Bitfrost you will often need to consider the security ramifications of what you are doing.

Consider carefully whether you need certain types of operation, if you do you will need to flag your activity with a special security "bit". Some such permissions are, by default, mutually exclusive. If you need those mutually exclusive functions, you will require either an officially signed activity, or an explicit authorization by each child to be run on their security-enabled laptop.

You are encouraged to read OLPC Bitfrost to fully understand the underlying principles and approaches.

Restriction Summary

The major restrictions for working within Bitfrost are as follows:

  • network access -- boolean control flag bit, if not set, no network access is possible
    • rate-limited by default
    • filtering restrictions available to protect against corruption (i.e. you can firewall your activity, in a sense)
  • file access -- by default you will have no data-file access without explicit selection by the user
    • store all user data within the Journal
    • unrestricted file access for a file-type -- by default not available if network access is available (to prevent automated uploading of e.g. all of a child's documents to an attacker), if you need both, you need to be signed/explicitly-enabled
    • constrain all file-writing to be only within the ${SUGAR_ACTIVITY_ROOT} (available as an environmental variable passed to your Activity instance)
    • file writes -- rate-limited by default to prevent flash storage exhaustion (if you are planning to stream video to flash storage, you likely need to consider this!)
  • camera/microphone access -- special flag to be allowed to ask for access, 30 minute timed access after which re-request required
  • background sound -- flag, allows for playing sounds when not the foreground activity
  • background CPU -- flag, allows for using more than a small fixed percentage of CPU while not foreground
  • synthetic X events -- flag, allows you to generate synthetic X events (mouse and keyboard, for instance) and send them to other Activities
    • if you are considering tutorials or similar tools that "run" another activity for the user, you need to consider this.

Specifying Flags:

Specifying Network Filters:

Activity Signing Procedure

Previous Next