Taste the Rainbow:0.7.8: Difference between revisions

From OLPC
Jump to navigation Jump to search
mNo edit summary
 
(6 intermediate revisions by 2 users not shown)
Line 26: Line 26:
== Rainbow's Spool ==
== Rainbow's Spool ==


Rainbow maintains uid- and gid-reservations, tables relating bundle-ids, uids, gids, and backing storage for data-dirs, instance-dirs, and tmp-dirs in a directory called the 'spool'. The spool is presently used at boot time to regenerate the contents of /etc/passwd and /etc/group and is used at run time to keep track of information about activity launches.
Rainbow maintains uid- and gid-reservations, tables relating bundle-ids, uids, gids, and backing storage for data-dirs, instance-dirs, and tmp-dirs in a directory called the 'spool'. The spool is used at boot time by [http://dev.laptop.org/git?p=projects/olpc-utils;a=blob;f=olpc-configure;h=0d6ef55ed49635672d63ae79831bff2a2eea4143#l203 olpc-configure] and [http://dev.laptop.org/git?p=users/mstone/security;a=blob;f=rainbow/bin/rainbow-replay-spool;hb=83eea6528df6a65d9fed508344019cc1e14b24bd#l11 rainbow-replay-spool:replay_spool()] to regenerate the contents of /etc/passwd and /etc/group and is used at run time to keep track of information about activity launches.


=== Location ===
Rainbow's spool is presently located at '/home/olpc/isolation/1'.


In response to [http://dev.laptop.org/ticket/5033 #5033], Rainbow's spool is presently located at ''/home/olpc/isolation/1''.
* This location was chosen in response to [http://dev.laptop.org/ticket/5033 #5033]. As discussed in that ticket, it is important that the rainbow spool persist across invocations of olpc-update (hence its location inside /home, which olpc-update ignores). Since the location, layout, and semantics of the spool directory are part of Rainbow's public interface to the rest of the system (in particular, to Sugar, the Datastore, and to olpc-update), it was also important that this interface be versioned so that future changes can be more easily made.


* As discussed in that ticket, it is important that the rainbow spool persist across invocations of olpc-update (hence its location inside /home, which olpc-update ignores).
The top-level layout of a version '1' spool consists of the following directories:
* Since the location, layout, and semantics of the spool directory are part of Rainbow's public interface to the rest of the system (in particular, to Sugar, the Datastore, and to olpc-update), it was also important that this interface be versioned so that future changes can be more easily made.


=== Layout ===
* uid_pool
* gid_pool
* uid_to_home_dir
* uid_to_instance_dir
* gid_to_data_dir
* bundle_id_to_gid


The top-level layout of a ''version 1'' spool consists of the following directories:
'pool' directories store resource reservations. Resource reservations are intended to allow Rainbow to service several requests in parallel.
'_to_' directories are key-value maps where the keys are file-names and the values are typically directories or strings. Strings are encoded as symlinks. Some extra information is encoded in the owning-uid and owning-gid of the contents of the _to_ directories; for example, during spool replay, we infer the correspondence between uids and gids by examining the ownership information of the home directories in the 'uid_to_home_dir' table.


* ''uid_pool''
I chose to encode key-value maps into a filesystem layout because these maps support the necessary operations at reasonable efficiency and are readily accessible to any program written using libc.
* ''gid_pool''
* ''uid_to_home_dir''
* ''uid_to_instance_dir''
* ''gid_to_data_dir''
* ''bundle_id_to_gid''

=== Data Encoding ===

''pool'' directories store resource reservations. Resource reservations are intended to allow Rainbow to service several requests in parallel.

''_to_'' directories are key-value maps where the keys are file-names and the values are typically directories or strings. Strings are encoded as symlinks. Some extra information is encoded in the owning-uid and owning-gid of the contents of the ''_to_'' directories; for example, during spool replay, we infer the correspondence between uids and gids by examining the ownership information of the home directories in the ''uid_to_home_dir'' table.

* I chose to encode key-value maps into a filesystem layout because these maps support the necessary operations at reasonable efficiency and are readily accessible to any program written using libc.


== Activity Launching ==
== Activity Launching ==
Line 50: Line 58:
Activity launching begins when the rainbow D-Bus service receives a CreateActivity message:
Activity launching begins when the rainbow D-Bus service receives a CreateActivity message:


*[http://dev.laptop.org/git?p=users/mstone/security;a=blob;f=rainbow/rainbow/service.py;hb=83eea6528df6a65d9fed508344019cc1e14b24bd#l38 service.py:Rainbow.CreateActivity()]
*[http://dev.laptop.org/git/users/mstone/security/tree/rainbow/rainbow/service.py?id=rainbow-0.7.27#n109 service.py:Rainbow.CreateActivity()]


The CreateActivity handler first attempts to reap zombie children, then it clones() the current process and delegates control to
The CreateActivity handler first attempts to reap zombie children, then it clones() the current process and delegates control to
*[http://dev.laptop.org/git?p=users/mstone/security;a=blob;f=rainbow/rainbow/inject.py;hb=83eea6528df6a65d9fed508344019cc1e14b24bd#l243 inject.py:run()]
*[http://dev.laptop.org/git/users/mstone/security/tree/rainbow/rainbow/inject.py?id=rainbow-0.7.27#n334 inject.py:run()]


which interleaves calls to the activity-launching primitives:
which interleaves calls to the activity-launching primitives:


*[http://dev.laptop.org/git?p=users/mstone/security;a=blob;f=rainbow/rainbow/inject.py;hb=83eea6528df6a65d9fed508344019cc1e14b24bd#l73 inject.py:strace()],
*[http://dev.laptop.org/git/users/mstone/security/tree/rainbow/rainbow/inject.py?id=rainbow-0.7.27#n84 inject.py:strace()],
*[http://dev.laptop.org/git?p=users/mstone/security;a=blob;f=rainbow/rainbow/inject.py;hb=83eea6528df6a65d9fed508344019cc1e14b24bd#l85 inject.py:reserve_elt()],
*[http://dev.laptop.org/git/users/mstone/security/tree/rainbow/rainbow/inject.py?id=rainbow-0.7.27#n96 inject.py:reserve_elt()],
*[http://dev.laptop.org/git?p=users/mstone/security;a=blob;f=rainbow/rainbow/inject.py;hb=83eea6528df6a65d9fed508344019cc1e14b24bd#l98 inject.py:reserve_credentials()],
*[http://dev.laptop.org/git/users/mstone/security/tree/rainbow/rainbow/inject.py?id=rainbow-0.7.27#n109 inject.py:reserve_credentials()],
*[http://dev.laptop.org/git?p=users/mstone/security;a=blob;f=rainbow/rainbow/inject.py;hb=83eea6528df6a65d9fed508344019cc1e14b24bd#l119 inject.py:grab_home()],
*[http://dev.laptop.org/git/users/mstone/security/tree/rainbow/rainbow/inject.py?id=rainbow-0.7.27#n137 inject.py:grab_home()],
*[http://dev.laptop.org/git?p=users/mstone/security;a=blob;f=rainbow/rainbow/inject.py;hb=83eea6528df6a65d9fed508344019cc1e14b24bd#l147 inject.py:configure_home()], and
*[http://dev.laptop.org/git/users/mstone/security/tree/rainbow/rainbow/inject.py?id=rainbow-0.7.27#n168 inject.py:configure_home()], and
*[http://dev.laptop.org/git?p=users/mstone/security;a=blob;f=rainbow/rainbow/inject.py;hb=83eea6528df6a65d9fed508344019cc1e14b24bd#l179 inject.py:launch()].
*[http://dev.laptop.org/git/users/mstone/security/tree/rainbow/rainbow/inject.py?id=rainbow-0.7.27#n206 inject.py:launch()].


and the assumption-checking procedures:
and the assumption-checking procedures:


*[http://dev.laptop.org/git?p=users/mstone/security;a=blob;f=rainbow/rainbow/inject.py;hb=83eea6528df6a65d9fed508344019cc1e14b24bd#l207 inject.py:check_bundle_id()],
*[http://dev.laptop.org/git/users/mstone/security/tree/rainbow/rainbow/inject.py?id=rainbow-0.7.27#n298 inject.py:check_bundle_id()],
*[http://dev.laptop.org/git?p=users/mstone/security;a=blob;f=rainbow/rainbow/inject.py;hb=83eea6528df6a65d9fed508344019cc1e14b24bd#l210 inject.py:check_bundle_path()],
*[http://dev.laptop.org/git/users/mstone/security/tree/rainbow/rainbow/inject.py?id=rainbow-0.7.27#n301 inject.py:check_bundle_path()],
*[http://dev.laptop.org/git?p=users/mstone/security;a=blob;f=rainbow/rainbow/inject.py;hb=83eea6528df6a65d9fed508344019cc1e14b24bd#l214 inject.py:check_argv()],
*[http://dev.laptop.org/git/users/mstone/security/tree/rainbow/rainbow/inject.py?id=rainbow-0.7.27#n305 inject.py:check_argv()],
*[http://dev.laptop.org/git?p=users/mstone/security;a=blob;f=rainbow/rainbow/inject.py;hb=83eea6528df6a65d9fed508344019cc1e14b24bd#l217 inject.py:check_cwd()],
*[http://dev.laptop.org/git/users/mstone/security/tree/rainbow/rainbow/inject.py?id=rainbow-0.7.27#n308 inject.py:check_cwd()],
*[http://dev.laptop.org/git?p=users/mstone/security;a=blob;f=rainbow/rainbow/inject.py;hb=83eea6528df6a65d9fed508344019cc1e14b24bd#l222 inject.py:check_spool()],
*[http://dev.laptop.org/git/users/mstone/security/tree/rainbow/rainbow/inject.py?id=rainbow-0.7.27#n313 inject.py:check_spool()],
*[http://dev.laptop.org/git?p=users/mstone/security;a=blob;f=rainbow/rainbow/inject.py;hb=83eea6528df6a65d9fed508344019cc1e14b24bd#l230 inject.py:check_owner()],
*[http://dev.laptop.org/git/users/mstone/security/tree/rainbow/rainbow/inject.py?id=rainbow-0.7.27#n322 inject.py:check_owner()],
*[http://dev.laptop.org/git?p=users/mstone/security;a=blob;f=rainbow/rainbow/inject.py;hb=83eea6528df6a65d9fed508344019cc1e14b24bd#l233 inject.py:check_home_dirs()],
*[http://dev.laptop.org/git/users/mstone/security/tree/rainbow/rainbow/inject.py?id=rainbow-0.7.27#n325 inject.py:check_home_dirs()],
*[http://dev.laptop.org/git?p=users/mstone/security;a=blob;f=rainbow/rainbow/inject.py;hb=83eea6528df6a65d9fed508344019cc1e14b24bd#l238 inject.py:check_home()],
*[http://dev.laptop.org/git/users/mstone/security/tree/rainbow/rainbow/inject.py?id=rainbow-0.7.27#n330 inject.py:check_home()],


== Resource Collection ==
== Resource Collection ==
Line 87: Line 95:
== Developing Rainbow ==
== Developing Rainbow ==


I develop Rainbow in two basic modes, which I call 'snapshot', 'candidate', and 'release' modes:
I develop Rainbow in three basic modes, which I call 'snapshot', 'candidate', and 'release' modes:


* By packaging snapshots of a git clone to try out packaging changes.
* By packaging snapshots of a git clone to try out packaging changes.

Latest revision as of 16:11, 26 August 2009

This page is a guided tour of the source code of the rainbow-0.7.8 release.

Source Code Overview

Please start in my rainbow-0.7.8 tree.

 ./
  |--- README : Standard boilerplate about where work gets done; somewhat dated in this release.
  |--- rainbow.spec.in : spec-file template for building RPMS
  |--- Makefile.package : package-specific variables for use in ../Makefile.fedora
  |--- conf : installation-time configuration files
  |     \--- session-olpc.conf : applies some unusual dbus rules to allow many uids
  |                              to use the same session bus and enables OLPC-specific 
  |                              dbus access checks. When /etc/olpc-security exists, 
  |                              session-olpc.conf is loaded by /usr/bin/sugar 
  | 
  |--- docs : explanations & notes
  |     |--- NOTES : various problems I have encountered and thoughts on how to solve them.
  |     *--- rainbow.txt : a sketch & justification of the current design
  |
  \--- rainbow : source code
        |--- util : functions wrapping frequently used idioms or useful syscalls
        |--- inject.py : logic implementing activity launching
        \--- service.py : dbus service entry-point

Rainbow's Spool

Rainbow maintains uid- and gid-reservations, tables relating bundle-ids, uids, gids, and backing storage for data-dirs, instance-dirs, and tmp-dirs in a directory called the 'spool'. The spool is used at boot time by olpc-configure and rainbow-replay-spool:replay_spool() to regenerate the contents of /etc/passwd and /etc/group and is used at run time to keep track of information about activity launches.

Location

In response to #5033, Rainbow's spool is presently located at /home/olpc/isolation/1.

  • As discussed in that ticket, it is important that the rainbow spool persist across invocations of olpc-update (hence its location inside /home, which olpc-update ignores).
  • Since the location, layout, and semantics of the spool directory are part of Rainbow's public interface to the rest of the system (in particular, to Sugar, the Datastore, and to olpc-update), it was also important that this interface be versioned so that future changes can be more easily made.

Layout

The top-level layout of a version 1 spool consists of the following directories:

  • uid_pool
  • gid_pool
  • uid_to_home_dir
  • uid_to_instance_dir
  • gid_to_data_dir
  • bundle_id_to_gid

Data Encoding

pool directories store resource reservations. Resource reservations are intended to allow Rainbow to service several requests in parallel.

_to_ directories are key-value maps where the keys are file-names and the values are typically directories or strings. Strings are encoded as symlinks. Some extra information is encoded in the owning-uid and owning-gid of the contents of the _to_ directories; for example, during spool replay, we infer the correspondence between uids and gids by examining the ownership information of the home directories in the uid_to_home_dir table.

  • I chose to encode key-value maps into a filesystem layout because these maps support the necessary operations at reasonable efficiency and are readily accessible to any program written using libc.

Activity Launching

Activity launching begins when the rainbow D-Bus service receives a CreateActivity message:

The CreateActivity handler first attempts to reap zombie children, then it clones() the current process and delegates control to

which interleaves calls to the activity-launching primitives:

and the assumption-checking procedures:

Resource Collection

When the rainbow-daemon D-Bus service starts, it performs a garbage collection pass to reclaim resources allocated in its spool. At present, this garbage collection pass simply deallocates any allocated uids and unlinks their instance and home directories.

This design is consistent with rainbow-0.7.8's simplifying decision to never reuse uids but is inconsistent with the architectural goal of supporting persistent activity instances. This design may be revisited in future releases.

Developing Rainbow

I develop Rainbow in three basic modes, which I call 'snapshot', 'candidate', and 'release' modes:

  • By packaging snapshots of a git clone to try out packaging changes.
 make snapshot
  • With locally-built or scratch-built packages, when I'm getting ready to tag a release.
 make release