System software: Difference between revisions

From OLPC
Jump to navigation Jump to search
No edit summary
 
(81 intermediate revisions by 41 users not shown)
Line 1: Line 1:
[[Category:Developers]]
{{RightTOC}}
== Software Ideas ==
== Software Ideas ==


Line 7: Line 9:
=== System Software ===
=== System Software ===


A version of Touch Typing software to teach these kids to touch type, the faster you can work with a keyboard whatever age you are then the faster you can get on with solving the worlds problems and letting the world know about your solutions... ' eg unjustified government spending on military budgets that will eventully only lead to one thing, more War to justify more spending etc..."
A version of Touch Typing software to teach these kids to touch type, the faster you can work with a keyboard whatever age you are then the faster you can get on with solving the world's problems and letting the world know about your solutions...


And does someone not need to make clearer in your marketing for support of this project that it does not have to be the same person who turns the crank as types at the keyboard, that there is a shortage of electricity in developing nations not hands to turn cranks?
And does someone not need to make clearer in your marketing for support of this project that it does not have to be the same person who turns the crank as types at the keyboard, that there is a shortage of electricity in developing nations not hands to turn cranks?


==== Operating System Selection ====
=== Backup Software ===


I think an automated online backup solution would be important since laptops may be swapped, stolen or otherwise data lost on a child's system. I think that meshed with the suggestion below about a built in "hard reset" capability would allow a child to perform essentially a complete restore of the OS and then their lost files. My company, [http://www.filebanc.com FileBanc], provides a solution that could be a good start; it runs equally well in a shell environment, web-based console and as a GUI under X. Scheduling happens with cron, and even if a system is lost, one can use another, log into the web console and restore the files through the web browser without having to install anything. We could probably do something like this for $1 per year per laptop. Thoughts? - Sean Stoner
I would like to seriously question using Linux as the basis of the laptop. It's a path of constant tweaking and tuning. Linux is also further away from the "pure" microkernel architectures than the BSDs. In short Linux is like a F1 tuned for racing - but not as reliable. When working on a familiar hardware platform the BSDs become very robust and nice platform to work with. And more importantly without the horrible quality problems of the Linux kernel. FreeBSD kernel is also magnitudes more secure - it just is cleaner and better thought and developed with the focus first on stability and reliablity. Unlike Linux.. The bottom line is that the project has got the possibility to choose from others than just Linux as well. I'd go for Frisbee or mentioned reasons but there are others too.


:Each school will have a server. It would be easy to set up a [[Subversion]] repository on the server, with a graphical client integrated with the file browser on each laptop. Then a student could periodically check in everything changed since the last checkpoint, with a single mouse operation.--[[User:Mokurai|Mokurai]] 06:24, 5 December 2006 (EST)
* (While I agree that *BSD might be a better option -- perhaps OpenBSD in particular for reasons of stability, et cetera -- I think the disparaging remarks about Linux-based operating systems were poorly conceived, exaggerated, and unnecessary as presented. - [http://sob.apotheon.org Chad Perrin])


:The [[School server]] won't use subversion to implement the [[XS_Server_Services#Backup|backup]], but it will be part of the solution. We propose multi-tier backups. The first tier is a backup of the laptop to the school server. This extends the limited storage resources of the laptop to provide a better user experience. But a school server is both frail and itself has limited storage capabilities. A second tier of backup would take student documents from the school server and archive them using a general Internet service (such as that offered above.) An active area of discussion is how to automatically winnow the files in this second step of archiving.
* Indeed: BSD is a good alternative. Another alternative may be an older Linux kernel. The 2.4 kernel line is pretty stable and good also. The current Linux kernel version is 2.6


:FYI, I just recently started to suggest to some local Non-Profit Organizations to start looking at [http://mozy.com/home Mozy Home and Pro]. One organization is using it to backup their server, and seem to be very pleased with it. Looking at the options and features, I'll be recommending that each office workstation backup to the server on a regular basis... then the server is backed up continuous (i.e. once every hour). Fully featured restore options via local and web interface. Looks like a winner to me. Maybe a partnership with Mozy to expand/localize the server to XS servers and XO Laptops would serve both businesses well? Other than that, I vision that Subversion/SVN could be an easy combination after a few adjustments for XO Laptops. --[[User:ixo|ixo]] 02:21, 6 January 2008 (EST)
* ''(User-contributed)'' Yes, the older the Linux kernel is, the more stable it becomes. For example, the current 1.08 version of [[PuppyLinux]] is based on the 2.4 kernel, and can confidently deliver satisfactory GUI experience despite the resource constraints in the OLPC machine.


A simple backup script, activated by cron or at startup, to sense a USB or SD disk and then backup all the users files would be very helpful. In situations where there is no school server set up, it would be an easy way to save the users content. At the moment, I believe it's a manual process. - Thanks, Lewie K
<br>'''Why UNIX(like) at all?'''<br>
Recently Negroponte stated that a "slimmer Linux" is needed for the $100 laptop. Like others here, I question why it has to be Linux. But I'll take that notion even further: Why does the OS even need to be Unix or a Unix work alike? Why is Negroponte so bent on Linux? There are other FOSS options that are much more focused on the desktop, yet still have Posix compliance.


:A solution designed around [http://www.nongnu.org/rdiff-backup/ rdiff-backup] might be a good choice. This uses the rsync protocol for efficient transfer and storage of incremental backups (which are automatic), and the most-current versions can be accessed simply by copying files. And it's already packaged for Fedora. [[User:Mattdm|Mattdm]] 18:31, 5 October 2008 (UTC)
* [http://haiku-os.org Haiku-OS] is an FOSS implementation of the BeOS tool kits with a FOSS kernel. It maintains binary and source level compatibility with BeOS R5, so all pre-existing free R5 software will work on it. (There is quite a bit)


=== Peer To Peer Distribution, for Electronic Text, Software, Email ===
* [http://www.syllable.org Syllable OS] Is a lot like BeOS. (But it's not BeOS) It's getting pretty mature.

Both these options are FOSS. If you really want the Unixy goodness, both also happen to have Posix compliant shells, so much of the Unix CLI stuff is portable. Both have fast native GUI systems and very modern design behind the internals. Both are extremely light weight compared to BSD or Linux + X + desktop env. And if one was really concerned with the stability of the kernel, it might be possible to run one of these alternative environments on top of a BSD or Linux kernel instead of their native kernels.

It might also be possible to buy out the BeOS R5 codebase from Palm Inc. or have them donate it to the project. They don't seem interested in doing anything with it. Then the OLPC project could release the the source under GPL and get help from the Haiku and [http://www.yellowtab.com Yellow Tab] guys to whip it into shape for the $100 laptop. I really can't convey how great BeOS or Haiku would be project. It's a perfect match for a system with the $100 laptops specs.

<br>'''The need for a distributed OS'''

The deployment environment is a school, not a business. So it needs cheap, easy system administration above all, as well as painless system integration. It would also be good if it distributed the file system so the spare storage on the laptops can be aggregated.

The logical choice is [http://en.wikipedia.org/wiki/Plan_9_from_Bell_Labs Plan 9].
Plan 9 is open-sourced, designed for graphics, lightweight, network-centric, has a UNIX api (glued on the side, but it's there),
and is designed to integrate and implement distributed file systems.
It should be easy to aggregate the file systems of many laptops just by merging their name spaces.

==== Peer To Peer Distribution, for Electronic Text, Software, Email ====


Extending the original idea from below... this is more general then just about electronic text, though. In lack of a better term, let me call it "built-in support for non-real-time Internet connectivity", provided as shared service and usable by apps.
Extending the original idea from below... this is more general then just about electronic text, though. In lack of a better term, let me call it "built-in support for non-real-time Internet connectivity", provided as shared service and usable by apps.


For example, I myself often read some web pages that I had downloaded while on the network at home while traveling, disconnected from a network, and of course when clicking on a link you get some stupid technical error message. Why can't the thing remember I want to read the linked page later and "queue" it somewhere? This idea is probably more much more relevant in some OLPC scenarios than it is for myself; what if you are connected to the "Internet by Motorbike" say only once every two weeks, as in the Motoman project in Cambodia?
For example, I myself often read some web pages that I had downloaded while on the network at home while traveling, disconnected from a network, and of course when clicking on a link you get some stupid technical error message. Why can't the thing remember I want to read the linked page later and "queue" it somewhere?
(The [http://www.gedanken.demon.co.uk/wwwoffle/ wwwoffle proxy] can do just that.)
This idea is probably more much more relevant in some OLPC scenarios than it is for myself; what if you are connected to the "Internet by Motorbike" say only once every two weeks, as in the [[Motoman]] project in Cambodia?


This applies to many forms of data, from electronic content be it a complete ebook, HTML page, Email or some software to download - or publishing of content such as homepage or blog updates, etc. (I think [http://www.oneworld.net/ OneWorld] has an XML-based publishing along those lines; but could be confusing it with something else.) Making it possible (and easy!) to request, and publish, data from one device, which then forwards the reqest to another, and ultimately forward to Internet when connected. Doesn't it make you feel like good ol' FIDO Net is back?
This applies to many forms of data, from electronic content be it a complete ebook, HTML page, Email or some software to download - or publishing of content such as homepage or blog updates, etc. (I think [http://www.oneworld.net/ OneWorld] has an XML-based publishing along those lines; but could be confusing it with something else.) Making it possible (and easy!) to request, and publish, data from one device, which then forwards the reqest to another, and ultimately forward to Internet when connected. Doesn't it make you feel like good ol' FIDO Net is back?
Line 51: Line 39:
[[User:Vorburger|Vorburger]] 20:06, 9 February 2006 (EST)
[[User:Vorburger|Vorburger]] 20:06, 9 February 2006 (EST)


=== Distributed Filesystem? ===
I believe one of the most useful purposes of the laptop will be to distribute electronic text (i.e. e-books). The distribution of any information without a persistent network connection will be difficult.
I imagine a situation where 1 out of 100 or 1000 kids may have access to a network connection. The peer to peer network could be used to distribute e-books from that single network connection to 1000 kids. I'd like to propose the design of a peer-to-peer network client designed specifically for this purpose.


Will the Wikipedia Offline fit into 512 MB (or even 1 GB) ? Even if it does, how about some software and other textbooks loaded at the same time? Clearly, the storage on one device is very limited... but: What if data could be spread over several laptops, a sort of built-in distributed filesystem like [http://www.coda.cs.cmu.edu/ Coda] or [http://www.danga.com/mogilefs/ MogileFS] - do these make any sense on a device like this, with the goal of enhancing storage capacity through distribution? In a school, every of say 100 children has 1/100th of Wikipedia - instead of clogging each device with a complete copy.
A simple, graphical language-localized client would be designed to present a catalog of e-books. The client would pull down a listing of books available in a certain age-range for a targeted language. The student would pick texts that he or she has interest in. This list of requests would consist of a very small packet of data, perhaps a unique identifier of the device and a unique identifier of the text. When the device sees another device, it would off load it's packet to the peer device and vice-versa. Each device would contain a listing of requests from all of the peers that it came in contact with. The next time a device connects to the Internet, it would pull as many texts as allowable by pre-defined memory limits (say 3-5 meg). As the device comes into contact with other devices, it would deliver the texts to the other devices. Hopefully, over time, the requestor would be delivered some of the texts originally requested. As each text is delivered, a delivery or cancellation notice would be sent back through the peer network.


[[User:Vorburger|Vorburger]] 20:06, 9 February 2006 (EST)
The peer to peer network should gather performance intelligence over time. It should be able to guess which routes have better chances of making a request and returning a delivery.


: As of 2006-November the bz2 compressed dump of the english wikipedia is 1.8Gb. This is the dump with only the latest versions of the articles, templates, and image descriptions. (Ie, no talk pages, no user pages, no previous versions of pages, etc). So it's not completely ridiculous to imagine putting it all onto a laptop, though of course it's hard to work with a compressed database, and even so, it's still 4 times the size of the total storage of the laptop. However, all the other language editions of wikipedia are much smaller. For example the same dump for the japanese wiki (3rd largest edition, after english and german) fits into 350Mb, and others are smaller still. Of course, without the actual images, it might not be that useful, and the images are much bigger (76Gb for the english, 7.5Gb for the Japanese)...
If a proof of concept proves to work well, the peer to peer network would be extended to handle two-way communication for interaction such as email or the submission and grading of assignments.


: There is a version of Wikipedia in Simple English that is smaller than the main one. This is more appropriate for an children's educational project and because it uses fewer words than normal English, it probably will compress better. (The same type of dump as above comes to 7.4Mb, which can very easily fit onto the laptop). In any case, dumping an Internet encyclopedia into a kid's laptop makes as much sense as hiring a bus driver to fly a 747. An OLPC encyclopedia needs to be edited severely to reduce its size, make sure the language is understandable by kids and make sure that it has appropriate content. Biographies of all the kings of England are only relevant to English speaking kids, not Hindi speakers (Hindi wikipedia: 1.8Mb) or speaker of Brazilian Portuguese (Portuguese wikipedia: 120Mb).
--[[User:65.7.133.163|65.7.133.163]] 04:41, 27 January 2006 (EST) Jason Hoekstra - jason@solexinc.com
:: Since I assume each school will also have an uplink gateway, maybe the Offline version of Wikipedia could be put on that, and cached on the individual units as they access it?


::: The idea is a good one but your implementation is flawed. Yes, each school will have some sort of system by which content can be downloaded. It could be as simple as a stack of CDROMs and a USB CDROM drive. Or, the teacher could hook the CDROM drive to her own laptop and ''push'' relevant content to the kids. In order for this to work, the encyclopedia editors have to ''chunk'' the text into thematic clusters. This means that a kid still has to do research in the encyclopedia because he has the whole Brazilian national history chunk in his laptop. But he doesn't suffer by having to carry all the Flora and Fauna of Brazil chunk, the World history chunk, and the Birth of Civilization chunk.


:A practical alternative, one that can be done now, is to use content in DVD (as suggested in the previous section). Some "hotspots" covered by these DVD-augmented laptops can be setup in a community, providing distributed servers for giving out content as well as hosting discussions. As the OLPC machine has USB port, adding DVD drive to it is not difficult. - [[User:Raffy]], April 27, 2006.
To continue on with what Jason and Volburger said, there is a fundamental conflict in OLPC. On one hand, you need to keep costs down and must therefore have small persistent storage. On the other hand, the purpose of the laptop is for learning, and learning requires the storage of information. What a child needs to learn with is really nothing more than an encyclopaedia and a simple way to navigate it, but it is probably not feasible to store an entire encyclopaedia in the available space, especially if you include multimedia, which you definitely should.


=== Grid computing ===
The approach suggested by Jason, which is not a bad one at all, is to retrieve information based on interest. I have some experience in mobile, ad hoc, sensor, and peer to peer networking, and what he's proposing is something similar to rumor routing and directed diffusion. There are a couple of problems with the suggestion, though. The first is that you will likely have a lot more requests for books than space to keep them. In a store and forward network with limited space and unpredicatable mobility, you are unlikely to hang onto enough books that you will be able to satisfy the requests. The second problem is that in all likelihood, some children will have far more Internet access than others, due to geographical location, being able to afford transport, or whatever the case may be. This may form a book distribution tree rooted at a few children with many children as leaves, which will cause significant distribution problems. Basically, a few children will need to store and forward books for a large number of children, and will quickly fill up their space without satisfying many requests. This depends on the particular country and situation of course, but in general, the branching factor and depth of the distribution tree can have serious repercussions on how many of the requests are satisfied.
It would be interesting if software were included to allow meshed machines to create an ad-hoc grid/cluster computer. It would be useful for things like compiling software, rendering and other CPU intensive tasks. (Stuff that I imagine some of the more advanced users, High School age, might want to do). A distributed file system would be a central part of that.


*Distributed computing may require special load balance algorithms to take into consideration the cost of electric power of each device and don't discharge hand-powered nodes, if power line connected machines are near enough.
I personally agree with Jason that a distributed, peer to peer filesystem is necessary. You may only have half a gigabyte of space, but there are a lot of half gigabytes running around, hopefully within a few hops of each other. So step one is that you need a routing protocol to form multihop ad hoc networks. There is a lot of literature on this; look at MobiHoc if you need a starting point. Step two is that you need a discovery protocol to learn what books are available and who has them, the aforementioned catalogue. Note that this catalogue needs to be updated as well, but the updates can be distributed using controlled flooding. And the reason I'm writing this whole thing is that I think you need a decent data (book) dissemination protocol. You are dealing with a sparsely connected network of resource starved nodes, with intermittent connectivity and esoteric mobility patterns. I believe you need some sort of centralised logic, such as a tracker in the Bittorrent protocol. If the computers belonging to the children that have Internet access can cooperate on which books to download and store, and if the computers themselves can try to form a rough topology of who is connected to who (in terms of the books they want), then you are in a much better position to allocate your resources to satisfy the most requests.


*Distributed WEP key cracking is useful for getting an internet connection.
I hope this helps and I wish you the very best of luck in your endeavours.


=== Better-performing Flash Filesystem ===
Emerson Farrugia (emerson AolpcT runelands D0T net) - March 17th, 2006


Currently the laptop is using the JFFS2 filesystem.


Compared with previous versions of JFFS2, the version used by OLPC has these characteristics:
==== E-mail client application ====


* Time to mount a 512MiB file system on the OLPC board with the AMD flash controller (which is 10x slower than we should be going) was 5.9s. We ought to be able to get it into the 1-2s range once we have CAFÉ working properly, although actually we're still seeing about 6s.
Perhaps will the project prefer to use a website like GMail for e-mail purposes. If not, as the author of tinymail, I'm willing to make sure tinymail will be suitable for the device. Tinymail is a project that aims to create a E-mail client development infrastructure for creating E-mail clients for small devices. At this moment it can show large IMAP and POP folders using less than 5 megabytes of memory. Tinymail is licensed as LGPL.
* RAM usage has been significantly decreased. Current ram usage can be calculated by 'grep jffs2 /proc/slabinfo', and we have plans to reduce it further by increasing maximum node size.
* JFFS2 supports hard links (although this is apparently not widely known), although we don't support shared writable mmap (and there are no plans to, since its a bad idea on flash and it's hard). .
* JFFS2 supports XATTRs, which is important if we want to use selinux.


There are two other flash file systems avialble for Linux: YAFFS and [[logfs]].
https://svn.cronos.be/svn/tinymail/trunk


OLPC has decided to use JFFS2 rather than YAFFS for the following reasons:
-- Philip Van Hoof <pvanhoof at gnome dot org>

==== Distributed Filesystem? ====

Will the Wikipedia Offline fit into 512 MB (or even 1 GB) ? Even if it does, how about some software and other textbooks loaded at the same time? Clearly, the storage on one device is very limited... but: What if data could be spread over several laptops, a sort of built-in distributed filesystem like [http://www.coda.cs.cmu.edu/ Coda] or [http://www.danga.com/mogilefs/ MogileFS] - do these make any sense on a device like this, with the goal of enhancing storage capacity through distribution? In a school, every of say 100 children has 1/100th of Wikipedia - instead of clogging each device with a complete copy.

[[User:Vorburger|Vorburger]] 20:06, 9 February 2006 (EST)

* Wikipedia will fit only in DVD. Small Linuxes can be embedded in that DVD, but for now, only [http://morphix.org Morphix] and [[PuppyLinux]] are ready for CD/DVD media, with [[PuppyLinux]] well-suited to the limited resources of the OLPC machine.

** Since I assume each school will also have an uplink gateway, maybe the Offline version of Wikipedia could be put on that, and cached on the individual units as they access it?

*** The english wikipedia is around 1.4GB (text only). The other language wikipedias seem to be smaller. I suppose someone could go through and delete a large amount of useless information, but that would be a pretty big job. Still, if pruning useless content from wikipedia were to become a sort of community project, I don't doubt there would be a large enough amount of volunteers that wouldn't mind spending some time on it.

==== Grid computing ====
It would be interesting if software were included to allow meshed machines to create an ad-hoc grid/cluster computer. It would be useful for things like compiling software, rendering and other CPU intensive tasks. (Stuff that I imagine some of the more advanced users, High School age, might want to do). A distributed file system would be a central part of that.

*A practical alternative, one that can be done now, is to use content in DVD (as suggested in the previous section). Some "hotspots" covered by these DVD-augmented laptops can be setup in a community, providing distributed servers for giving out content as well as hosting discussions. As the OLPC machine has USB port, adding DVD drive to it is not difficult. - [[Raffy]], April 27, 2006.

==== Better-performing Flash Filesystem ====

The proposed JFFS2 filesystem was designed for NOR-type Flash memory, which has very different timing characteristics from the cheaper NAND-type Flash memory used in USB thumb drives and, presumably, the laptop. YAFFS is a GPL'ed open-source journalling filesystem designed specifically for NAND Flash memory that is claimed to use less RAM for its tables and generally outperform JFFS2, and they are working on YAFFS2, which is tweaked to be faster and to work with the new larger, 2KB-page-size NAND devices.

YAFFS has the following technical advantages over JFFS2:
* It uses NAND Flash memory better, making it faster (about 2X), more space-efficient and wearing the memory chips out less quickly
* It is faster at mounting a filesystem: a hand-waving example of startup time for a 128MB device is 3 seconds instead of 25
* It uses far less RAM for its internal tables
* It scales better: JFFS2 is said to fall apart above 256MB because its internal data structures get too big while YAFFS is known to work well up to 2GB (the laptop currently aims at 512MB)
* It stores error-correcting codes for all data, which is essential since NAND Flash is supplied not 100% perfect and degrades over time
* YAFFS provides some features lacking from JFFS2 (hard links, memory mapped file writing)

JFFS2 has the following advantages over YAFFS:
* It has built-in write-time data compression
* It has built-in write-time data compression
* It is included in the standard Linux kernel
* It is included in the standard Linux kernel
* JFFS2 is much better tested than YAFFS.
* JFFS2 works out of the box with the MTD subsystem while YAFFS needs tweaks and patches and is hard to adopt to hardware ECC controllers


YAFFS has a [http://www.aleph1.co.uk/yaffs/ home page] and a there is [http://www.aleph1.co.uk/yaffs/jffs2_and_nand.html a technical article] which goes into depth on the differences between NOR and NAND flash memory and the drawbacks of using JFFS2 with the NAND type.


YAFFS has a [http://www.aleph1.co.uk/yaffs/ home page] and a there is [http://www.aleph1.co.uk/yaffs/jffs2_and_nand.html a technical article], although members of the JFFS2 team claim that the comparison is out-of-date.
It would be worth running comparative performance tests on the two filesystems, because there are big potential performance wins on several fronts. In-filesystem compression isn't everything, slows all file operations down and, when used without error correcting codes onto an unreliable medium, risks major data loss.

[[User:Martinwguy|Martin Guy]] 4 March 2006


Jörn Engel is currently working on a new flash file system called [[logfs]]. It is not yet clear if it will hit the mainline kernel in time for consideration for the first generation laptop, but it is progressing fast.
Jörn Engel is currently working on a new flash file system called [[logfs]]. It is not yet clear if it will hit the mainline kernel in time for consideration for the first generation laptop, but it is progressing fast.
It should combine all the advantages listed for either for the two file systems above with a new clean design. In particular, the mount time and memory footprint is independent from the device size, unlike the existing file systems.


It would be worth running comparative performance tests on the two filesystems, because there are big potential performance wins on several fronts. In-filesystem compression isn't everything, slows all file operations down and, when used without error correcting codes onto an unreliable medium, risks major data loss.
I don't think that YAFFS can be considered an option for OLPC at this point because of missing compression and the quality of the code.


The JFFS2 filesystem on the XO already has major data loss problems. It seems that the error correcting codes are not anywhere near good enough. Performance isn't everything.
[[User:Arnd|arnd]] 12 March 2006


==== 3d software rendering ====
=== 3d software rendering ===
As the system does not include hardware accelerated 3d rendering, a software rendering library may be included to wrap the OpenGL (OGL/ES maybe) API and create rendering code on the fly. This, even on a machine with limited clock speed can provide a rendering performance paragonable to that of some integrated 3d chipsets, especially if the resolution is kept low. This could allow educationnal software to use 3d rendering (physics and mathematics softwares could take advantage of this). There are some existing tools that can be leveraged for this; for example, [http://sourceforge.net/projects/ogl-es Vincent] is an OpenGL/ES implementation that provides software rendering for constrained devices like cell phones; SwShader, precursor of transgamings' [http://www.transgaming.com/swiftshader.php SwiftShader] and many others. Having (limited) OpenGL capability does add some capabilities to the device without requiring additional hardware.
As the system does not include hardware accelerated 3d rendering, a software rendering library may be included to wrap the OpenGL (OGL/ES maybe) API and create rendering code on the fly. This, even on a machine with limited clock speed can provide a rendering performance comparable to that of some integrated 3d chipsets, especially if the resolution is kept low. This could allow educational software to use 3d rendering (physics and mathematics softwares could take advantage of this). There are some existing tools that can be leveraged for this; for example, [http://sourceforge.net/projects/ogl-es Vincent] is an OpenGL/ES implementation that provides software rendering for constrained devices like cell phones; SwShader, precursor of transgamings' [http://www.transgaming.com/swiftshader.php SwiftShader] and many others. Having (limited) OpenGL capability does add some capabilities to the device without requiring additional hardware.


==== Software Installation, Package Manager, Central Repository ====
=== Software Installation, Package Manager, Central Repository ===


How relevant is a polished end-user friendly Package Manager? With limited memory, are you more likely to uninstall and try another application and install back one? In the beginning, how important is it to be able to very easily get patched new versions of the software? Underlying question: Is a central repository of applications desirable? Completely open, anybody can submit their (pre-compiled) package?
How relevant is a polished end-user friendly Package Manager? With limited memory, are you more likely to uninstall and try another application and install back one? In the beginning, how important is it to be able to very easily get patched new versions of the software? Underlying question: Is a central repository of applications desirable? Completely open, anybody can submit their (pre-compiled) package?
Line 138: Line 100:


Should there be an easy way to install and remove applications from the device without corrupting the system image? I am thinking of something like klik (http://klik.atekon.de/). -- [[User:DPalmerJr|DPalmerJr]]
Should there be an easy way to install and remove applications from the device without corrupting the system image? I am thinking of something like klik (http://klik.atekon.de/). -- [[User:DPalmerJr|DPalmerJr]]

-> An initial proposal and a proof-of-concept demo is [http://klik.atekon.de/wiki/index.php/OLPC here]. -- [[User:Probono|Probono]]


I am on a team developing a deeply embedded losely connected ARM-based Linux system (64 MiB RAM, 512 MiB disc). We have discovered the hard way that it's best to support in-field upgrades -- right from day 1. Even with an effective release management + testing/validation team, specs will change, improvements will be made, bugs will slip through. Our devices are connected via slow satellite links and connect to our infrastructure as infrequently as once per month. We cannot feed a lot of data through the link without blowing our power budget. Even if/when we are willing to risk an over-the-air in-field upgrade, we may not have the bandwidth/power budget. We have found conventional package managers (dpkg, rpm) are too coarse-grained when dealing with skinny pipes and power budgets. A package manager supporting deltas would be preferable. We have even considered downloading source patches and re-compiling on the embedded device. Your network will be faster than ours, so YMMV.
I am on a team developing a deeply embedded losely connected ARM-based Linux system (64 MiB RAM, 512 MiB disc). We have discovered the hard way that it's best to support in-field upgrades -- right from day 1. Even with an effective release management + testing/validation team, specs will change, improvements will be made, bugs will slip through. Our devices are connected via slow satellite links and connect to our infrastructure as infrequently as once per month. We cannot feed a lot of data through the link without blowing our power budget. Even if/when we are willing to risk an over-the-air in-field upgrade, we may not have the bandwidth/power budget. We have found conventional package managers (dpkg, rpm) are too coarse-grained when dealing with skinny pipes and power budgets. A package manager supporting deltas would be preferable. We have even considered downloading source patches and re-compiling on the embedded device. Your network will be faster than ours, so YMMV.
Line 147: Line 111:
I consider a well-thought-out, secure, trustable, user-controlable package management system to be critical to system stability, extensibility, maintainability, and ultimately to the success of this project. -- [[User:BCL|BCL]]
I consider a well-thought-out, secure, trustable, user-controlable package management system to be critical to system stability, extensibility, maintainability, and ultimately to the success of this project. -- [[User:BCL|BCL]]


-> A system using bundled, self-contained applications like [http://klik.atekon.de/wiki/index.php/OLPC this] could facilitate mesh-sharing of applications. -- [[User:Probono|Probono]]
==== Laptop as USB-Drive ====

=== Laptop as USB-Drive ===


It would probably be useful if the laptop could be accessed as a USB-Drive, like a digital camera.. In the Software Development context hackers could probably also configure File Sharing via the WiFi... but simple "USB cross cabling" could be interesting to end-users because it's: a) most simple, b) secure, probably OK to give access to entire filesystem, if locally attached, c) doesn't need Wifi; the nearest Internet Cafe in a bigger town will let children/teacher USB-connect their laptop to one of their stations to copy over a newly downloaded application, but not have a Wifi basestation; at least not where I have travelled in India.
It would probably be useful if the laptop could be accessed as a USB-Drive, like a digital camera.. In the Software Development context hackers could probably also configure File Sharing via the WiFi... but simple "USB cross cabling" could be interesting to end-users because it's: a) most simple, b) secure, probably OK to give access to entire filesystem, if locally attached, c) doesn't need Wifi; the nearest Internet Cafe in a bigger town will let children/teacher USB-connect their laptop to one of their stations to copy over a newly downloaded application, but not have a Wifi basestation; at least not where I have travelled in India.


:Why take the laptop to the big town when you can take a thumbdrive instead. Better yet, why not just wait for the content to come to you on a CD-ROM. Send an email by motorcycle-net to order the content you want, and next week, the [[Motoman]] motorcycle brings it on CD during the regular delivery. Works in Vietnam.--[[User:Vorburger|Vorburger]]
[[User:Vorburger|Vorburger]]


Maybe a software can be developed for this. Since the system is going to be "Linux Based", just accesing the filesystem should allow to configure almost everything. A software that gives access to the filesystem (and emulate a camera or an USB thumb), could be included. Or maybe, a special cable provided with the laptop (that uses one special of the 3 USB ports) could allow direct access to filesystem. (or with a switch somewhere in the laptop that even without power makes it work as a USB-Drive, even with the posibility of charging batteries while connected).
Maybe a software can be developed for this. Since the system is going to be "Linux Based", just accesing the filesystem should allow to configure almost everything. A software that gives access to the filesystem (and emulate a camera or an USB thumb), could be included. Or maybe, a special cable provided with the laptop (that uses one special of the 3 USB ports) could allow direct access to filesystem. (or with a switch somewhere in the laptop that even without power makes it work as a USB-Drive, even with the posibility of charging batteries while connected).--[[User:Gandolfi|Gandolfi]]


:We are making changes to the XO hardware that should make a "Laptop as USB drive" a reality. It will take some time before we support this in firmware, but it is a possibility.
[[User:Gandolfi|Gandolfi]]


==== Hard-Reset built-in ====
=== Hard-Reset built-in ===


Curious kids will certainly easily manage to screw up the software side of the device - and they should! A built-in hard-reset that can re-initialize the OS etc. from ROM; sort of like some modern laptops have a hidden partition on the HDD that can re-install without the usual Recovery CD, could be useful.
Curious kids will certainly easily manage to screw up the software side of the device - and they should! A built-in hard-reset that can re-initialize the OS etc. from ROM; sort of like some modern laptops have a hidden partition on the HDD that can re-install without the usual Recovery CD, could be useful.


You always have the problem of personal data, files, and configuration settings. Some solution for that would have to be provided; e.g. easily copy to your friend's device over the wireless network?
You always have the problem of personal data, files, and configuration settings. Some solution for that would have to be provided; e.g. easily copy to your friend's device over the wireless network? --[[User:Vorburger|Vorburger]]


:FYI, there is a [[Support_FAQ#How_can_I_restore_my_Give1Get1_XO_to_its_factory_configuration.3F|method of restoring to factory defaults]]. (build 650). --[[User:ixo|ixo]] 06:31, 6 January 2008 (EST)
[[User:Vorburger|Vorburger]]


This is a very good point. If we use a compressed read-only file (or partition) with most of the filesystem (specially the part under /usr) we can not only stuff a lot more software in there, but also resetting would be a much simpler operation. Basically all it had to do was to untar a "factory default" tar file (or something like that) into the writtable part of the flash storage.
This is a very good point. If we use a compressed read-only file (or partition) with most of the filesystem (specially the part under /usr) we can not only stuff a lot more software in there, but also resetting would be a much simpler operation. Basically all it had to do was to untar a "factory default" tar file (or something like that) into the writtable part of the flash storage.
Line 174: Line 140:
The reset operation could be integrated with the patch/upgrade mechanism whereby the system will only install secure signed OS-level packages until either the system or the user decides it's OK to open the doors for business. -- [[User:BCL|BCL]]
The reset operation could be integrated with the patch/upgrade mechanism whereby the system will only install secure signed OS-level packages until either the system or the user decides it's OK to open the doors for business. -- [[User:BCL|BCL]]


if the system is on a readonly area, why not just leave it there and usethat directly without copying anything? changes to that area could still be made using an overlay filesystem, that just shadows the original. (you could also have the system be a tree of links into the rom, and when you want to change something, then you remove the link and replace it with a real file. that would however not be as easy to use/understand, because it would be different from the way a normal system works) -- eMBee
====Font technology====


Which font technology is to be used?


per eMBee's idea, you could use a method similar to a live cd, using software such as unionfs. One of the problems with using true ROM is that as updates are applied, there would be wasted space where the old files are stored. Also, a change in the base system, such as changing from the current plan of linux to the proposed plan of a special Windows OS would render this ROM useless. This problem could be solved by having a "system files" partition that can only be modified by an update program that checks the md5 hash of the update .diff file aginst a list of md5s in a pgp signed file that is redistributed at the same time as a update is released. Also, the system files area should be of a reasonable size bigger than the current distribution (50-100 MB) to make room for updates. A restore could be accomplished by removing all files from the fully read-write side of the unionfs filesystem that affect system operation, while preserving documents (ex. not removing /home/*) . Having the system files area be semi-writable solves the problem mentioned above of the system not being secure at the time a restore is done, because the updates would alredy be in the system files area, which would not be affected by a restore. Just my ideas on the subject. -- Anonymous
The character encoding will be Unicode. In that case it is important that an advanced font technology such as OpenType or Graphite is available. It is also necessary to have rendering software for screen and printer that can intelligently combine glyphs from fonts. The choices are Uniscribe (Windows), ATSUI (Macintosh), SIL Graphite (Linux or Windows), Pango (Linux and any other system that can run Free Software), or TrollTech Scribe (Linux or compatible). Of these, Graphite is the most powerful, but it is not yet in widespread use. Pango is the most widely deployed rendering engine for Linux.


===Font technology===
''The best, of course :-). Fontconfig does fonts substitution on a linguistic level, beyond what Windows and the Mac does. [http://www.pango.org/ Pango] is probably the most advanced layout library around, though further work for some scripts is needed. [http://scripts.sil.org/cms/scripts/page.php?site_id=nrsi&item_id=GraphiteFAQ#WhatIsGraphite The graphite description] says that Sil is working on integrating it with Pango. - jg''


Which font technology is to be used?
For European languages such as French and Spanish an ordinary font technology such as TrueType is fine. For languages using Latin script yet using accented characters which do not each have a precomposed Unicode character, including many in Africa, an advanced font format is necessary. This is so that glyph substitution can take place to convert a sequence of a base character followed by a combining accent into a "looks right" display. Any rendering engine with any font containing the appropriate glyphs can put an accent mark over a character, but only OpenType can specify exactly where the mark should go for best appearance.


The OLPC uses Linux with GTK which includes Pango as a component. It also uses FreeType which means that the OLPC uses cross-platform TrueType fonts.
''Freetype, used by almost everything these days on open source formats, handles a plethora of font types, from Type 1, to TrueType, to OpenType; note that anyone wanting to introduce yet another font format had best be examining how to do it as a Freetype plugin - jg''


Yes, OpenType fonts will much better render complex scripts. The Pango and SIL Graphite projects are cooperating on the design of their rendering engines and the fonts they will need.
Arabic script systems (Arabic, Farsi, Urdu, etc.) need an advanced font technology and an advanced rendering engine. Chinese does not need an advanced font technology system. For languages of the Indian subcontinent typewriter-like displays can be achieved without an advanced font technology. For full support of conjunct ligatures an advanced font technology is needed, and similarly for other Asian alphabets (Sinhalese, Lao, Khmer, Myanmar, Tibetan, Mongolian, etc.).


Anyway, this isn't a problem that OLPC needs to solve. Experts are working on it. OLPC will leverage their work. For an example of why OLPC is not directly working on the font problem, read [http://www.kuro5hin.org/story/2004/2/5/01839/12103 this article on Tibetan writing].
''We know of some open issues with Thai & pango, but believe that they can be solved and that Pango handles most languages already (e.g. Arabic, the Indic languages. Please help determine where further work may be needed. - jg''


It is important that a thorough analysis of the character rendering technology and font technology needed is carried out.
=====The eutofont font format=====


--[http://www.sil.org/ SIL] is the world's foremost research institution on such matters. They work in more than a thousand languages, and maintain the Ethnologue catalog of more than 6,000 documented human languages.
Some time ago William Overington devised a font format using character codes from the Unicode Private Use Area.


--There are such experts at the heart of the font and rendering engine initiatives described throughout this article. I have observed experts from universities and from SIL, Red Hat, Apple, Microsoft, Sun, IBM, Hewlett-Packard, Evertype, commercial font vendors, vendors of font creation software,...since that is who makes up the Unicode Consortium. The portion of their efforts that goes into Linux will inevitably end up on the OLPC products.
(Note by Ed Cherlin: Every font format allows the use of PUA codes. They are used for writing systems not encoded in Unicode, such as Klingon.)


Here is a transcript of what I wrote before.
(Note by William Overington: Ed Cherlin states "Every font format allows the use of PUA codes." Yes. Yet that is an item different from what I was trying to say. I was trying to say that the eutofont font format actually uses Unicode Private Use Area code points in the font format itself with the effect that a font is expressible as a sequence of Unicode Private Use Area characters. So, if the eutofont format were used to produce a font of just the twenty-six letters of the English alphabet, all of them regular Unicode characters, the font would be a string of Unicode characters, most of them from the Private Use Area. An end user need not be aware that Private Use Area codes have been used in the producing of the font and the end user does not need to use the Private Use Area codes directly.)


--I use Pango rendering and properly implemented TrueType fonts on my Linux system to render conjuncts without difficulty. Some TrueType fonts have the glyphs but not the substitution tables; they render with great ugliness. The Akruti fonts, developed in India for all of the major alphabets of India, were placed under the GPL (GNU Public License) as Free Software some time ago (on Gandhi's birthday). There are distributions of Linux in several languages of India, and more on the way.
As far as I know it has never been implemented. However, I mention it here in case readers might like to have a look at the documents and decide whether it might be of any use for the laptop project.


----
I named it the eutofont font format.


''The best, of course :-). Fontconfig does fonts substitution on a linguistic level, beyond what Windows and the Mac does. [http://www.pango.org/ Pango] is probably the most advanced layout library around, though further work for some scripts is needed. [http://scripts.sil.org/cms/scripts/page.php?site_id=nrsi&item_id=GraphiteFAQ#WhatIsGraphite The graphite description] says that Sil is working on integrating it with Pango. - jg''
http://www.users.globalnet.co.uk/~ngo/eutofont.htm


For European languages such as French and Spanish an ordinary font technology such as TrueType is fine. For languages using Latin script yet using accented characters which do not each have a precomposed Unicode character, including many in Africa, an advanced font format is necessary. This is so that glyph substitution can take place to convert a sequence of a base character followed by a combining accent into a "looks right" display. Any rendering engine with any font containing the appropriate glyphs can put an accent mark over a character, but only OpenType can specify exactly where the mark should go for best appearance.
The eutofont font format has glyph substitution facilities and also has chromatic font capability.


''Freetype, used by almost everything these days on open source formats, handles a plethora of font types, from Type 1, to TrueType, to OpenType; note that anyone wanting to introduce yet another font format had best be examining how to do it as a Freetype plugin - jg''
The system could be extended if font technology needs are required which the eutofont font format presently described does not support.


Arabic script systems (Arabic, Farsi, Urdu, etc.) need an advanced font technology and an advanced rendering engine. Chinese does not need an advanced font technology system. For languages of the Indian subcontinent typewriter-like displays can be achieved without an advanced font technology. For full support of conjunct ligatures an advanced font technology is needed, and similarly for other Asian alphabets (Sinhalese, Lao, Khmer, Myanmar, Tibetan, Mongolian, etc.).
''Please note the use of Fontconfig on open source systems for font naming and substitution - jg''


''We know of some open issues with Thai & pango, but believe that they can be solved and that Pango handles most languages already (e.g. Arabic, the Indic languages. Please help determine where further work may be needed. - jg''
Regarding the use of Private Use Area codes: by using them a compactness of font size would be possible which an XML based font system might not be able to achieve: in due course, if the eutofont font system were successful, maybe codes would be added to regular Unicode, though that would lose some compactness as the codes would not be in plane zero; however, in the short term the Private Use Area codes would be needed.


''Please note the use of Fontconfig on open source systems for font naming and substitution - jg''
William Overington


11 March 2006


====Automated Language Localization of some Preset Sentences====


===Email Client requirements===
I have for some time been interested in whether it would be of practical use (rather than just fun in researching what can and cannot be done) to have a collection of sentences and part sentences defined and translated into many languages, each sentence or part sentence having a code number, with the idea that an author may construct a message using one such code number or a sequence of such code numbers and then the code numbers could be used by a software system in the computer of a recipient of the message in conjunction with a small database of code numbers and the text of the sentences in a chosen language so as to produce a localized message displayed for the recipient.


Email is the only well known internet application that doesn't depend on a working TCP/IP connection to the internet. It's model is the paper postal service where there are only one or two connections per day, when the postie visits the letterbox.
For example, suppose that there were only two sentences from which to choose and that these have been encoded as sentences 21011 and 21012.


It is very likely that these laptops will be in the situation where the link to the outside world will be a fragile connection running at very low speeds. If it's a modem line it's likely that the quality is so poor that echo cancellation will fail; this will limit the speed to 2400bps duplex (higher if half duplex). This is not enough for a shared web connection for thirty kids.
The English database would contain the following.


This is okay for email with some rules:
21011 It is raining.
* The email client must be self contained.
* The MTA must be light and capable of very versatile store and forward without help from DNS.
* The MTA on the client must be capable of ad-hoc forwarding. ie the child can tell it to give their mail to another client, one who's going to school today.
* The client must have good facilities for splitting files into multiple emails (and joining) so a maximum message size of say 16kb would not be a problem.
* The ability to put the mail on a USB key. The bandwidth of a real postie with a pocket full of USB keys could be rather high (see [[Sneakernet]] email network via USB Flash Drive).


A good model for this might be the old [http://en.wikipedia.org/wiki/Fidonet FidoNet] networks, though a cleaner addressing scheme would be nice.
21012 It is snowing.


Having just email is not as limiting as you might imagine you can access most of [http://www.faqs.org/faqs/internet-services/access-via-email/ the internet by email.]
The French database would contain the following.


--
21011 Il pleut.
Robert de Bath -- March 2006


PS: I just did the math, I've got a 1Gbyte flash key so my bandwidth on the daily commute to work is 99kbps!
21012 Il neige.


===Motorcycle E-mail Network===
The database could be translated into as many languages as desired and possible.
This is an excellent idea and should be part of the core OLPC project. Here is how it is currently being done in rural Cambodia. http://www.parish-without-borders.net/cditt/cambodia/dailylife/2004/rural-internet.htm


Remember, the OLPC is NOT A LAPTOP. It is a system comprising laptops, children, teachers, applications, content, USB-devices, etc.
So, if someone whose preferred language is English is authoring a message and wishes to send the message "It is raining." then he or she looks throgh the database using whatever search tools that are available at his or her location and encodes the message as 21011 and then sends it.


Also see [[Motoman]] (email by Wi-Fi enabled motorbike) and links to similar projects at the end of [[Sneakernet]].
So, if someone whose preferred language is French receives the message then the text "Il pleut." can be displayed automatically.


===WLAN MAC Address===
So, if there were more sentences than that and also sentences with a parameter such as for "The temperature here is P1 degrees Celsius." where the value of parameter 1 is sent as a digit string (possibly including a decimal point) to accompany the 21852 code of the parameterized sentence, and that list of sentences were available in many languages, then, for example, weather information could be broadcast on a pan-European basis on an interactive television channel and localized automatically in interactive televisions in, for example, England, France, Italy, Finland and Latvia.
There might be privacy issues related to the WLAN MAC address. The MAC is somehow similar to the unique serial number in the CPU-ID except that it is ''additionally broadcast'' around. "Quick, she/he is leaving, lets start eating the apples." A WLAN mesh might allow for relatively fine grained position tracking.


=== Broadcast [[GPS]]/Galileo Position, send tiles of local map ===
As to how to encode such a system, well there are various possibilities. I started off using a deliberately unusual yet valid sequence of regular Unicode characters to act as a key that would be most unlikely to occur in any other use context, namely a comet, a circumflex accent and an enclosing keycap design. I have also looked at using Unicode Private Use Area characters. It has been suggested to me that XML would be the best approach, though I have reservations as I would like a system where a short sequence could be added into a plain text file without having to restructure the whole document, however I am unsure of that so it is possible that XML would be the way to go.
If an OLPC has access to its [[GPS]] coordinates these should optionally be sent via WLAN. Distribution could be combined with a kind of reliability information (maybe like a kind of superset of stratum in the ntp protocol).


OLPCs without direct access should store the coordinates of a nearby OLPCs with position information. This would allow an OLPC connecting to a WLAN to download/display a local map (f.e. download of an area of 80x80km, display 20x20 km (first guess for a compromise between WLAN range, walking area, map detail, bandwith, OLPC distribution)).
I am wondering whether the technique, whether using the key or the Private Use Area codes, or using XML, or otherwise, could be useful for autolocalizing some part of the education process. For example, a sentence such as "Please tell your teacher that you have now completed the task." and such as "You have chosen the correct answer." and "Well done.".
This can help the answer to "Where am I? Where are you from?" (local map, country, continent, earth, and solar system if need should arise:). Tiles of the detailed map could be available for other OLPC. - Frieder Ferlemann 2006-06-06


===Python & kernel memory usage cooperation===
I did a little with the idea theoretically some time ago.
OLPC will push the Linux environment to run in much tighter memory constraints -- small RAM and no swap or paging space. And it's using Python rather than C for many commonly running apps. Currently, running out of memory is handled very primitively -- in the kernel, by killing the biggest application running; in Python applications, by exiting with an error message. This clearly won't suffice, but I haven't seen any plans to improve it.


(Correction: that is not strictly right. In case of memory pressure kernel will start freeing up memory by removing temporary cache buffers, dropping memory pages of the executable files if possible (will not do that if the programs are already executed in place), or shrink network buffers. OOM killer is really used only as last resort. By the way, the fact that kernel can and does manage buffer shrinking automatically actually discourages the applications from having own caches: the kernel has potential for a better control)
http://www.users.globalnet.co.uk/~ngo/c_c00000.htm


Python will have to learn to "give back memory" to the kernel when it doesn't need it. It could do this on a page-by-page basis with mmap calls, subsequent to garbage collection. Also, Python should be able to signal to the application (and/or libraries) that memory is tight and the upper level code should free unnecessary resources (such as caches). This signal should occur whenever an application is suspended, or goes unused for some period. And whenever the kernel runs low of memory.
I never got it beyond English!


A similar strategy should probably exist for other resources, such as filesystem space, and CPU time. The kernel should have a way to tell applications that demand is high, and that they should scale back their demand if they can.
A later development was to incorporate the LOCODE concept so as to specify names of places that were to be localized, such as the way Firenze is expressed as Florence in English and London is expressed as Londres in French.


The kernel will have to learn how to signal applications to reduce their memory usage. This is most important when there is NO memory left -- when the kernel currently picks and kills a process -- but it should be done before that point, when there's more flexibility. E.g. if an application wants to allocate another page temporarily while emptying its cache or doing a garbage collection, it can't do that when zero memory is left.
http://www.unece.org/cefact/locode/service/main.htm


A new signal (SIGSHRINK?) is one way to communicate this to processes. Having them open something from /dev or /proc and listen on it would be another. Or just let applications (or a specialized process, like init) monitor /proc/meminfo and /proc/stat and take actions accordingly. These capabilities would be useful in the upstream kernel and applications.
William Overington


A daemon like inetd could allow applications to totally terminate when idle and/or when signalled to shrink. An application, that was coded to know how to resume on demand, could pass any file descriptors that need to stay open up to the daemon (e.g. open network connections or ttys), then terminate. The daemon would wait for I/O activity on those connections, and fork a new copy of the process when needed. -- John Gilmore
17 March 2006
:Normally [http://www.linuxfocus.org/English/November2000/article175.shtml xinetd] is part of Fedora. It currently seems to have been removed from the OLPC distro.


Applications could check for free memory before starting and refuse to run if too much memory is in use. The only support needed is a reliable way for a Python app to get meaningful numbers for total memory and memory used. (This works assuming the the app knows in advance how much memory it is going to need, independent of what the user does with it. The kernel will actually do this for you: if there's no swap space, and you allocate all your memory early in the process's life, you'll get ENOMEM and you can die cleanly then. -John Gilmore)
====Email Client requirements====


More effort could be put into keeping applications slimmed down. Perhaps some tools to analyze redundancy, i.e. linking to a non-shared library that some other app also links to non-shared. Multiple versions of the same library. Busybox has done a lot of this kind of refactoring for basic UNIX utilities. [http://valgrind.org/ Valgrind] is also a tool to keep in mind: it does a very good job of bookkeeping of all memory allocations and one of the programs shipped with it, cachegrind, helps to minimize CPU cache impacts.
Email is the only well known internet application that doesn't depend on a working TCP/IP connection to the internet. It's model is the paper postal service where there are only one or two connections per day, when the postie visits the letterbox.

It is very likely that these laptops will be in the situation where the link to the outside world will be a fragile connection running at very low speeds. If it's a modem line it's likely that the quality is so poor that echo cancellation will fail; this will limit the speed to 2400bps duplex (higher if half duplex). This is not enough for a shared web connection for thirty kids.

This is okay for email with some rules:
* The email client must be self contained.
* The MTA must be light and capable of very versatile store and forward without help from DNS.
* The MTA on the client must be capable of ad-hoc forwarding. ie the child can tell it to give their mail to another client, one who's going to school today.
* The client must have good facilities for splitting files into multiple emails (and joining) so a maximum message size of say 16kb would not be a problem.
* The ability to put the mail on a USB key. The bandwidth of a real postie with a pocket full of USB keys could be rather high.

A good model for this might be the old [http://en.wikipedia.org/wiki/Fidonet FidoNet] networks, though a cleaner addressing scheme would be nice.

Having just email is not as limiting as you might imagine you can access most of [http://www.faqs.org/faqs/internet-services/access-via-email/ the internet by email.]

--
Robert de Bath -- March 2006

PS: I just did the math, I've got a 1Gbyte flash key so my bandwidth on the daily commute to work is 99kbps!


==Network Protocol==
==Network Protocol==
Line 299: Line 257:


[[User:Ray Van De Walker|Ray Van De Walker]] 10:34, 26 May 2006 (EDT)
[[User:Ray Van De Walker|Ray Van De Walker]] 10:34, 26 May 2006 (EDT)


:Why not go for an open-source mesh network?

:802.11s is mostly a very complex compatibility standard for mesh which is more suited towards commercial metro coverage of a city. As a wireless engineer who has been part of the IEEE Standards development process, I can tell you that 802.11s compatible equipment is probably not the right approach.

:Have you considered following the [http://www.cuwireless.net/ CUWin] Mesh protocol instead of 802.11s?

:CUWin on the other hand, is a decentralized, community-owned mesh network that fosters democratic cultures and local content. The connections are not made to funnel up to an ISP, but instead across town to other CUWin nodes. Each node is encouraged to host its own mail sever, host it's own local content, etc. The equipment is relatively very inexpensive (see below) and you can probably port most of the code.


:[http://meraki.net/ Meraki] Minis and Outdoor are inexpensive have been built to run CUWin - $50-$100. It would make sense to be compatible with this hardware since it's inexpensive and open source. This is an inexpensive way to build a network across a town. This would very much needed in order to connect a communities of laptops together.

:[[User:Kgrr|Kgrr]] 16:25, 22 March 2007 (EDT)

== Peer-to-Peer Recovery ==

If storage limitations don't permit recovery images to be stored, as mentioned in the 'Hard Reset Built-in' section, then a fall back could be to:
#assume another OLPC is meshed
#system software that could perform over-the-air reprogramming/recovery from another working OLPC.
This process could also be used by [[Laptop Service Programme Ideas]]
: [[Nandblaster]] essentially provides this service.
[[Category:Software ideas]]

Latest revision as of 05:43, 6 September 2011

Software Ideas

Usability

Include in the OS an onscreen transluscent/watermark representation of the keyboard that indicated which key is pressed would help users learn to touch type since they would not have to look down at the keyboard, but could be turned on or off (better yet, varied in opacity from 0 to 100%). Otherwise, how many will even know to try typing while looking at the screen? It would also help in low light [as was suggested under hardware by another], as many won't have power for lighting yet would need to use it only when light for after school chores are completed, leaving only late in the day and many will be in latitudes with SHORT winter days.

System Software

A version of Touch Typing software to teach these kids to touch type, the faster you can work with a keyboard whatever age you are then the faster you can get on with solving the world's problems and letting the world know about your solutions...

And does someone not need to make clearer in your marketing for support of this project that it does not have to be the same person who turns the crank as types at the keyboard, that there is a shortage of electricity in developing nations not hands to turn cranks?

Backup Software

I think an automated online backup solution would be important since laptops may be swapped, stolen or otherwise data lost on a child's system. I think that meshed with the suggestion below about a built in "hard reset" capability would allow a child to perform essentially a complete restore of the OS and then their lost files. My company, FileBanc, provides a solution that could be a good start; it runs equally well in a shell environment, web-based console and as a GUI under X. Scheduling happens with cron, and even if a system is lost, one can use another, log into the web console and restore the files through the web browser without having to install anything. We could probably do something like this for $1 per year per laptop. Thoughts? - Sean Stoner

Each school will have a server. It would be easy to set up a Subversion repository on the server, with a graphical client integrated with the file browser on each laptop. Then a student could periodically check in everything changed since the last checkpoint, with a single mouse operation.--Mokurai 06:24, 5 December 2006 (EST)
The School server won't use subversion to implement the backup, but it will be part of the solution. We propose multi-tier backups. The first tier is a backup of the laptop to the school server. This extends the limited storage resources of the laptop to provide a better user experience. But a school server is both frail and itself has limited storage capabilities. A second tier of backup would take student documents from the school server and archive them using a general Internet service (such as that offered above.) An active area of discussion is how to automatically winnow the files in this second step of archiving.
FYI, I just recently started to suggest to some local Non-Profit Organizations to start looking at Mozy Home and Pro. One organization is using it to backup their server, and seem to be very pleased with it. Looking at the options and features, I'll be recommending that each office workstation backup to the server on a regular basis... then the server is backed up continuous (i.e. once every hour). Fully featured restore options via local and web interface. Looks like a winner to me. Maybe a partnership with Mozy to expand/localize the server to XS servers and XO Laptops would serve both businesses well? Other than that, I vision that Subversion/SVN could be an easy combination after a few adjustments for XO Laptops. --ixo 02:21, 6 January 2008 (EST)

A simple backup script, activated by cron or at startup, to sense a USB or SD disk and then backup all the users files would be very helpful. In situations where there is no school server set up, it would be an easy way to save the users content. At the moment, I believe it's a manual process. - Thanks, Lewie K

A solution designed around rdiff-backup might be a good choice. This uses the rsync protocol for efficient transfer and storage of incremental backups (which are automatic), and the most-current versions can be accessed simply by copying files. And it's already packaged for Fedora. Mattdm 18:31, 5 October 2008 (UTC)

Peer To Peer Distribution, for Electronic Text, Software, Email

Extending the original idea from below... this is more general then just about electronic text, though. In lack of a better term, let me call it "built-in support for non-real-time Internet connectivity", provided as shared service and usable by apps.

For example, I myself often read some web pages that I had downloaded while on the network at home while traveling, disconnected from a network, and of course when clicking on a link you get some stupid technical error message. Why can't the thing remember I want to read the linked page later and "queue" it somewhere? (The wwwoffle proxy can do just that.) This idea is probably more much more relevant in some OLPC scenarios than it is for myself; what if you are connected to the "Internet by Motorbike" say only once every two weeks, as in the Motoman project in Cambodia?

This applies to many forms of data, from electronic content be it a complete ebook, HTML page, Email or some software to download - or publishing of content such as homepage or blog updates, etc. (I think OneWorld has an XML-based publishing along those lines; but could be confusing it with something else.) Making it possible (and easy!) to request, and publish, data from one device, which then forwards the reqest to another, and ultimately forward to Internet when connected. Doesn't it make you feel like good ol' FIDO Net is back?

Vorburger 20:06, 9 February 2006 (EST)

Distributed Filesystem?

Will the Wikipedia Offline fit into 512 MB (or even 1 GB) ? Even if it does, how about some software and other textbooks loaded at the same time? Clearly, the storage on one device is very limited... but: What if data could be spread over several laptops, a sort of built-in distributed filesystem like Coda or MogileFS - do these make any sense on a device like this, with the goal of enhancing storage capacity through distribution? In a school, every of say 100 children has 1/100th of Wikipedia - instead of clogging each device with a complete copy.

Vorburger 20:06, 9 February 2006 (EST)

As of 2006-November the bz2 compressed dump of the english wikipedia is 1.8Gb. This is the dump with only the latest versions of the articles, templates, and image descriptions. (Ie, no talk pages, no user pages, no previous versions of pages, etc). So it's not completely ridiculous to imagine putting it all onto a laptop, though of course it's hard to work with a compressed database, and even so, it's still 4 times the size of the total storage of the laptop. However, all the other language editions of wikipedia are much smaller. For example the same dump for the japanese wiki (3rd largest edition, after english and german) fits into 350Mb, and others are smaller still. Of course, without the actual images, it might not be that useful, and the images are much bigger (76Gb for the english, 7.5Gb for the Japanese)...
There is a version of Wikipedia in Simple English that is smaller than the main one. This is more appropriate for an children's educational project and because it uses fewer words than normal English, it probably will compress better. (The same type of dump as above comes to 7.4Mb, which can very easily fit onto the laptop). In any case, dumping an Internet encyclopedia into a kid's laptop makes as much sense as hiring a bus driver to fly a 747. An OLPC encyclopedia needs to be edited severely to reduce its size, make sure the language is understandable by kids and make sure that it has appropriate content. Biographies of all the kings of England are only relevant to English speaking kids, not Hindi speakers (Hindi wikipedia: 1.8Mb) or speaker of Brazilian Portuguese (Portuguese wikipedia: 120Mb).
Since I assume each school will also have an uplink gateway, maybe the Offline version of Wikipedia could be put on that, and cached on the individual units as they access it?
The idea is a good one but your implementation is flawed. Yes, each school will have some sort of system by which content can be downloaded. It could be as simple as a stack of CDROMs and a USB CDROM drive. Or, the teacher could hook the CDROM drive to her own laptop and push relevant content to the kids. In order for this to work, the encyclopedia editors have to chunk the text into thematic clusters. This means that a kid still has to do research in the encyclopedia because he has the whole Brazilian national history chunk in his laptop. But he doesn't suffer by having to carry all the Flora and Fauna of Brazil chunk, the World history chunk, and the Birth of Civilization chunk.
A practical alternative, one that can be done now, is to use content in DVD (as suggested in the previous section). Some "hotspots" covered by these DVD-augmented laptops can be setup in a community, providing distributed servers for giving out content as well as hosting discussions. As the OLPC machine has USB port, adding DVD drive to it is not difficult. - User:Raffy, April 27, 2006.

Grid computing

It would be interesting if software were included to allow meshed machines to create an ad-hoc grid/cluster computer. It would be useful for things like compiling software, rendering and other CPU intensive tasks. (Stuff that I imagine some of the more advanced users, High School age, might want to do). A distributed file system would be a central part of that.

  • Distributed computing may require special load balance algorithms to take into consideration the cost of electric power of each device and don't discharge hand-powered nodes, if power line connected machines are near enough.
  • Distributed WEP key cracking is useful for getting an internet connection.

Better-performing Flash Filesystem

Currently the laptop is using the JFFS2 filesystem.

Compared with previous versions of JFFS2, the version used by OLPC has these characteristics:

  • Time to mount a 512MiB file system on the OLPC board with the AMD flash controller (which is 10x slower than we should be going) was 5.9s. We ought to be able to get it into the 1-2s range once we have CAFÉ working properly, although actually we're still seeing about 6s.
  • RAM usage has been significantly decreased. Current ram usage can be calculated by 'grep jffs2 /proc/slabinfo', and we have plans to reduce it further by increasing maximum node size.
  • JFFS2 supports hard links (although this is apparently not widely known), although we don't support shared writable mmap (and there are no plans to, since its a bad idea on flash and it's hard). .
  • JFFS2 supports XATTRs, which is important if we want to use selinux.

There are two other flash file systems avialble for Linux: YAFFS and logfs.

OLPC has decided to use JFFS2 rather than YAFFS for the following reasons:

  • It has built-in write-time data compression
  • It is included in the standard Linux kernel
  • JFFS2 is much better tested than YAFFS.
  • JFFS2 works out of the box with the MTD subsystem while YAFFS needs tweaks and patches and is hard to adopt to hardware ECC controllers


YAFFS has a home page and a there is a technical article, although members of the JFFS2 team claim that the comparison is out-of-date.

Jörn Engel is currently working on a new flash file system called logfs. It is not yet clear if it will hit the mainline kernel in time for consideration for the first generation laptop, but it is progressing fast.

It would be worth running comparative performance tests on the two filesystems, because there are big potential performance wins on several fronts. In-filesystem compression isn't everything, slows all file operations down and, when used without error correcting codes onto an unreliable medium, risks major data loss.

The JFFS2 filesystem on the XO already has major data loss problems. It seems that the error correcting codes are not anywhere near good enough. Performance isn't everything.

3d software rendering

As the system does not include hardware accelerated 3d rendering, a software rendering library may be included to wrap the OpenGL (OGL/ES maybe) API and create rendering code on the fly. This, even on a machine with limited clock speed can provide a rendering performance comparable to that of some integrated 3d chipsets, especially if the resolution is kept low. This could allow educational software to use 3d rendering (physics and mathematics softwares could take advantage of this). There are some existing tools that can be leveraged for this; for example, Vincent is an OpenGL/ES implementation that provides software rendering for constrained devices like cell phones; SwShader, precursor of transgamings' SwiftShader and many others. Having (limited) OpenGL capability does add some capabilities to the device without requiring additional hardware.

Software Installation, Package Manager, Central Repository

How relevant is a polished end-user friendly Package Manager? With limited memory, are you more likely to uninstall and try another application and install back one? In the beginning, how important is it to be able to very easily get patched new versions of the software? Underlying question: Is a central repository of applications desirable? Completely open, anybody can submit their (pre-compiled) package?

Vorburger

Should there be an easy way to install and remove applications from the device without corrupting the system image? I am thinking of something like klik (http://klik.atekon.de/). -- DPalmerJr

-> An initial proposal and a proof-of-concept demo is here. -- Probono

I am on a team developing a deeply embedded losely connected ARM-based Linux system (64 MiB RAM, 512 MiB disc). We have discovered the hard way that it's best to support in-field upgrades -- right from day 1. Even with an effective release management + testing/validation team, specs will change, improvements will be made, bugs will slip through. Our devices are connected via slow satellite links and connect to our infrastructure as infrequently as once per month. We cannot feed a lot of data through the link without blowing our power budget. Even if/when we are willing to risk an over-the-air in-field upgrade, we may not have the bandwidth/power budget. We have found conventional package managers (dpkg, rpm) are too coarse-grained when dealing with skinny pipes and power budgets. A package manager supporting deltas would be preferable. We have even considered downloading source patches and re-compiling on the embedded device. Your network will be faster than ours, so YMMV.

System development + testing will benefit from a slick patch/upgrade mechanism too.

I don't think it's unreasonable to expect to upgrade the devices via the mesh cluster - upgrade one device and the rest can upgrade from it. Use public-key-encryption to sign 'blessed' packages.

I consider a well-thought-out, secure, trustable, user-controlable package management system to be critical to system stability, extensibility, maintainability, and ultimately to the success of this project. -- BCL

-> A system using bundled, self-contained applications like this could facilitate mesh-sharing of applications. -- Probono

Laptop as USB-Drive

It would probably be useful if the laptop could be accessed as a USB-Drive, like a digital camera.. In the Software Development context hackers could probably also configure File Sharing via the WiFi... but simple "USB cross cabling" could be interesting to end-users because it's: a) most simple, b) secure, probably OK to give access to entire filesystem, if locally attached, c) doesn't need Wifi; the nearest Internet Cafe in a bigger town will let children/teacher USB-connect their laptop to one of their stations to copy over a newly downloaded application, but not have a Wifi basestation; at least not where I have travelled in India.

Why take the laptop to the big town when you can take a thumbdrive instead. Better yet, why not just wait for the content to come to you on a CD-ROM. Send an email by motorcycle-net to order the content you want, and next week, the Motoman motorcycle brings it on CD during the regular delivery. Works in Vietnam.--Vorburger

Maybe a software can be developed for this. Since the system is going to be "Linux Based", just accesing the filesystem should allow to configure almost everything. A software that gives access to the filesystem (and emulate a camera or an USB thumb), could be included. Or maybe, a special cable provided with the laptop (that uses one special of the 3 USB ports) could allow direct access to filesystem. (or with a switch somewhere in the laptop that even without power makes it work as a USB-Drive, even with the posibility of charging batteries while connected).--Gandolfi

We are making changes to the XO hardware that should make a "Laptop as USB drive" a reality. It will take some time before we support this in firmware, but it is a possibility.

Hard-Reset built-in

Curious kids will certainly easily manage to screw up the software side of the device - and they should! A built-in hard-reset that can re-initialize the OS etc. from ROM; sort of like some modern laptops have a hidden partition on the HDD that can re-install without the usual Recovery CD, could be useful.

You always have the problem of personal data, files, and configuration settings. Some solution for that would have to be provided; e.g. easily copy to your friend's device over the wireless network? --Vorburger

FYI, there is a method of restoring to factory defaults. (build 650). --ixo 06:31, 6 January 2008 (EST)

This is a very good point. If we use a compressed read-only file (or partition) with most of the filesystem (specially the part under /usr) we can not only stuff a lot more software in there, but also resetting would be a much simpler operation. Basically all it had to do was to untar a "factory default" tar file (or something like that) into the writtable part of the flash storage.

We could have a boot option, where the user would type "reset" or something like that, to boot a "rescue" kernel and initrd that just did this operation. -- Paulo Marques


There's a problem in the Microsoft Windows world with newly-installed systems. You have to go on-line to get the latest security patches from Microsoft. But as soon as you go on-line with an unpatched system you're at risk of infection from viruses.

The reset operation could be integrated with the patch/upgrade mechanism whereby the system will only install secure signed OS-level packages until either the system or the user decides it's OK to open the doors for business. -- BCL

if the system is on a readonly area, why not just leave it there and usethat directly without copying anything? changes to that area could still be made using an overlay filesystem, that just shadows the original. (you could also have the system be a tree of links into the rom, and when you want to change something, then you remove the link and replace it with a real file. that would however not be as easy to use/understand, because it would be different from the way a normal system works) -- eMBee


per eMBee's idea, you could use a method similar to a live cd, using software such as unionfs. One of the problems with using true ROM is that as updates are applied, there would be wasted space where the old files are stored. Also, a change in the base system, such as changing from the current plan of linux to the proposed plan of a special Windows OS would render this ROM useless. This problem could be solved by having a "system files" partition that can only be modified by an update program that checks the md5 hash of the update .diff file aginst a list of md5s in a pgp signed file that is redistributed at the same time as a update is released. Also, the system files area should be of a reasonable size bigger than the current distribution (50-100 MB) to make room for updates. A restore could be accomplished by removing all files from the fully read-write side of the unionfs filesystem that affect system operation, while preserving documents (ex. not removing /home/*) . Having the system files area be semi-writable solves the problem mentioned above of the system not being secure at the time a restore is done, because the updates would alredy be in the system files area, which would not be affected by a restore. Just my ideas on the subject. -- Anonymous

Font technology

Which font technology is to be used?

The OLPC uses Linux with GTK which includes Pango as a component. It also uses FreeType which means that the OLPC uses cross-platform TrueType fonts.

Yes, OpenType fonts will much better render complex scripts. The Pango and SIL Graphite projects are cooperating on the design of their rendering engines and the fonts they will need.

Anyway, this isn't a problem that OLPC needs to solve. Experts are working on it. OLPC will leverage their work. For an example of why OLPC is not directly working on the font problem, read this article on Tibetan writing.

It is important that a thorough analysis of the character rendering technology and font technology needed is carried out.

--SIL is the world's foremost research institution on such matters. They work in more than a thousand languages, and maintain the Ethnologue catalog of more than 6,000 documented human languages.

--There are such experts at the heart of the font and rendering engine initiatives described throughout this article. I have observed experts from universities and from SIL, Red Hat, Apple, Microsoft, Sun, IBM, Hewlett-Packard, Evertype, commercial font vendors, vendors of font creation software,...since that is who makes up the Unicode Consortium. The portion of their efforts that goes into Linux will inevitably end up on the OLPC products.

Here is a transcript of what I wrote before.

--I use Pango rendering and properly implemented TrueType fonts on my Linux system to render conjuncts without difficulty. Some TrueType fonts have the glyphs but not the substitution tables; they render with great ugliness. The Akruti fonts, developed in India for all of the major alphabets of India, were placed under the GPL (GNU Public License) as Free Software some time ago (on Gandhi's birthday). There are distributions of Linux in several languages of India, and more on the way.

The best, of course :-). Fontconfig does fonts substitution on a linguistic level, beyond what Windows and the Mac does. Pango is probably the most advanced layout library around, though further work for some scripts is needed. The graphite description says that Sil is working on integrating it with Pango. - jg

For European languages such as French and Spanish an ordinary font technology such as TrueType is fine. For languages using Latin script yet using accented characters which do not each have a precomposed Unicode character, including many in Africa, an advanced font format is necessary. This is so that glyph substitution can take place to convert a sequence of a base character followed by a combining accent into a "looks right" display. Any rendering engine with any font containing the appropriate glyphs can put an accent mark over a character, but only OpenType can specify exactly where the mark should go for best appearance.

Freetype, used by almost everything these days on open source formats, handles a plethora of font types, from Type 1, to TrueType, to OpenType; note that anyone wanting to introduce yet another font format had best be examining how to do it as a Freetype plugin - jg

Arabic script systems (Arabic, Farsi, Urdu, etc.) need an advanced font technology and an advanced rendering engine. Chinese does not need an advanced font technology system. For languages of the Indian subcontinent typewriter-like displays can be achieved without an advanced font technology. For full support of conjunct ligatures an advanced font technology is needed, and similarly for other Asian alphabets (Sinhalese, Lao, Khmer, Myanmar, Tibetan, Mongolian, etc.).

We know of some open issues with Thai & pango, but believe that they can be solved and that Pango handles most languages already (e.g. Arabic, the Indic languages. Please help determine where further work may be needed. - jg

Please note the use of Fontconfig on open source systems for font naming and substitution - jg


Email Client requirements

Email is the only well known internet application that doesn't depend on a working TCP/IP connection to the internet. It's model is the paper postal service where there are only one or two connections per day, when the postie visits the letterbox.

It is very likely that these laptops will be in the situation where the link to the outside world will be a fragile connection running at very low speeds. If it's a modem line it's likely that the quality is so poor that echo cancellation will fail; this will limit the speed to 2400bps duplex (higher if half duplex). This is not enough for a shared web connection for thirty kids.

This is okay for email with some rules:

  • The email client must be self contained.
  • The MTA must be light and capable of very versatile store and forward without help from DNS.
  • The MTA on the client must be capable of ad-hoc forwarding. ie the child can tell it to give their mail to another client, one who's going to school today.
  • The client must have good facilities for splitting files into multiple emails (and joining) so a maximum message size of say 16kb would not be a problem.
  • The ability to put the mail on a USB key. The bandwidth of a real postie with a pocket full of USB keys could be rather high (see Sneakernet email network via USB Flash Drive).

A good model for this might be the old FidoNet networks, though a cleaner addressing scheme would be nice.

Having just email is not as limiting as you might imagine you can access most of the internet by email.

-- Robert de Bath -- March 2006

PS: I just did the math, I've got a 1Gbyte flash key so my bandwidth on the daily commute to work is 99kbps!

Motorcycle E-mail Network

This is an excellent idea and should be part of the core OLPC project. Here is how it is currently being done in rural Cambodia. http://www.parish-without-borders.net/cditt/cambodia/dailylife/2004/rural-internet.htm

Remember, the OLPC is NOT A LAPTOP. It is a system comprising laptops, children, teachers, applications, content, USB-devices, etc.

Also see Motoman (email by Wi-Fi enabled motorbike) and links to similar projects at the end of Sneakernet.

WLAN MAC Address

There might be privacy issues related to the WLAN MAC address. The MAC is somehow similar to the unique serial number in the CPU-ID except that it is additionally broadcast around. "Quick, she/he is leaving, lets start eating the apples." A WLAN mesh might allow for relatively fine grained position tracking.

Broadcast GPS/Galileo Position, send tiles of local map

If an OLPC has access to its GPS coordinates these should optionally be sent via WLAN. Distribution could be combined with a kind of reliability information (maybe like a kind of superset of stratum in the ntp protocol).

OLPCs without direct access should store the coordinates of a nearby OLPCs with position information. This would allow an OLPC connecting to a WLAN to download/display a local map (f.e. download of an area of 80x80km, display 20x20 km (first guess for a compromise between WLAN range, walking area, map detail, bandwith, OLPC distribution)).

This can help the answer to "Where am I? Where are you from?" (local map, country, continent, earth, and solar system if need should arise:). Tiles of the detailed map could be available for other OLPC. - Frieder Ferlemann 2006-06-06

Python & kernel memory usage cooperation

OLPC will push the Linux environment to run in much tighter memory constraints -- small RAM and no swap or paging space. And it's using Python rather than C for many commonly running apps. Currently, running out of memory is handled very primitively -- in the kernel, by killing the biggest application running; in Python applications, by exiting with an error message. This clearly won't suffice, but I haven't seen any plans to improve it.

(Correction: that is not strictly right. In case of memory pressure kernel will start freeing up memory by removing temporary cache buffers, dropping memory pages of the executable files if possible (will not do that if the programs are already executed in place), or shrink network buffers. OOM killer is really used only as last resort. By the way, the fact that kernel can and does manage buffer shrinking automatically actually discourages the applications from having own caches: the kernel has potential for a better control)

Python will have to learn to "give back memory" to the kernel when it doesn't need it. It could do this on a page-by-page basis with mmap calls, subsequent to garbage collection. Also, Python should be able to signal to the application (and/or libraries) that memory is tight and the upper level code should free unnecessary resources (such as caches). This signal should occur whenever an application is suspended, or goes unused for some period. And whenever the kernel runs low of memory.

A similar strategy should probably exist for other resources, such as filesystem space, and CPU time. The kernel should have a way to tell applications that demand is high, and that they should scale back their demand if they can.

The kernel will have to learn how to signal applications to reduce their memory usage. This is most important when there is NO memory left -- when the kernel currently picks and kills a process -- but it should be done before that point, when there's more flexibility. E.g. if an application wants to allocate another page temporarily while emptying its cache or doing a garbage collection, it can't do that when zero memory is left.

A new signal (SIGSHRINK?) is one way to communicate this to processes. Having them open something from /dev or /proc and listen on it would be another. Or just let applications (or a specialized process, like init) monitor /proc/meminfo and /proc/stat and take actions accordingly. These capabilities would be useful in the upstream kernel and applications.

A daemon like inetd could allow applications to totally terminate when idle and/or when signalled to shrink. An application, that was coded to know how to resume on demand, could pass any file descriptors that need to stay open up to the daemon (e.g. open network connections or ttys), then terminate. The daemon would wait for I/O activity on those connections, and fork a new copy of the process when needed. -- John Gilmore

Normally xinetd is part of Fedora. It currently seems to have been removed from the OLPC distro.

Applications could check for free memory before starting and refuse to run if too much memory is in use. The only support needed is a reliable way for a Python app to get meaningful numbers for total memory and memory used. (This works assuming the the app knows in advance how much memory it is going to need, independent of what the user does with it. The kernel will actually do this for you: if there's no swap space, and you allocate all your memory early in the process's life, you'll get ENOMEM and you can die cleanly then. -John Gilmore)

More effort could be put into keeping applications slimmed down. Perhaps some tools to analyze redundancy, i.e. linking to a non-shared library that some other app also links to non-shared. Multiple versions of the same library. Busybox has done a lot of this kind of refactoring for basic UNIX utilities. Valgrind is also a tool to keep in mind: it does a very good job of bookkeeping of all memory allocations and one of the programs shipped with it, cachegrind, helps to minimize CPU cache impacts.

Network Protocol

I think the most important single choice is the mesh protocol, because it is likely to have a longer deployment than any implementation of the hardware, OS, or application software.

I figured that the best mesh protocol would minimize total routing waste, in order to reduce power use. Computation will use less power as technology advances, but transmission power is going to be limited by physics at some point.

I researched mesh protocols at the wikipedia.

The hazy-sighted link state protocol just stood out among the choices. It is mathematically optimized to minimize network waste. This means that it minimizes power and won't be easily improved-upon. It also has a fairly old, well-debugged, publicly deployed open source implementation that runs on diverse hardware, and is about the right size and shape (small).

The least surprising choice is probably OLSR (which periodically floods the network with limited routing data). The simplest protocol is probably AODV (a distance vector protocol that floods the network with routing information), The others seem to be research projects, or proprietary, and I would avoid them, even though some are specifically geared to power saving.

Ray Van De Walker 10:34, 26 May 2006 (EDT)


Why not go for an open-source mesh network?
802.11s is mostly a very complex compatibility standard for mesh which is more suited towards commercial metro coverage of a city. As a wireless engineer who has been part of the IEEE Standards development process, I can tell you that 802.11s compatible equipment is probably not the right approach.
Have you considered following the CUWin Mesh protocol instead of 802.11s?
CUWin on the other hand, is a decentralized, community-owned mesh network that fosters democratic cultures and local content. The connections are not made to funnel up to an ISP, but instead across town to other CUWin nodes. Each node is encouraged to host its own mail sever, host it's own local content, etc. The equipment is relatively very inexpensive (see below) and you can probably port most of the code.


Meraki Minis and Outdoor are inexpensive have been built to run CUWin - $50-$100. It would make sense to be compatible with this hardware since it's inexpensive and open source. This is an inexpensive way to build a network across a town. This would very much needed in order to connect a communities of laptops together.
Kgrr 16:25, 22 March 2007 (EDT)

Peer-to-Peer Recovery

If storage limitations don't permit recovery images to be stored, as mentioned in the 'Hard Reset Built-in' section, then a fall back could be to:

  1. assume another OLPC is meshed
  2. system software that could perform over-the-air reprogramming/recovery from another working OLPC.

This process could also be used by Laptop Service Programme Ideas

Nandblaster essentially provides this service.
Retrieved from "http://wiki.laptop.org/mediawiki/index.php?title=System_software&oldid=259990"