Understanding Sugar code: Difference between revisions
m (Understanding sugar code moved to Walter smells 4: Walter smells) |
(start cleaning up and clarifying the old 2007 info vs. 2010 output) |
||
(6 intermediate revisions by 2 users not shown) | |||
Line 1: | Line 1: | ||
{{ |
{{dated}} |
||
:''This was written in 2007. It's still a useful guide to exploring the code while running Sugar, but the details are different: Python is at 2.6, Sugar has split into sugar and jarabe, the sugar startup components are different, etc.'' |
|||
==Understanding |
==Understanding Sugar code== |
||
This is an attempt to understand how |
This is an attempt to understand how [[Sugar]] runs, based on a moderate level of unix shell experience. Here, we'll break down: |
||
# what packages and software |
# what packages and software Sugar relies on |
||
# where |
# where Sugar sets up scripts, libraries, modules, and configuration files |
||
# how |
# how Sugar manages user identity |
||
# what |
# what Sugar does at runtime (very high level) |
||
# how |
# how Sugar cleans itself up when exiting |
||
Other Docs |
Other Docs |
||
Line 14: | Line 15: | ||
==What |
==What runs as part of Sugar?== |
||
Sugar runs on |
Sugar runs on a variety of computers, including the OLPC XO models. The OLPC runs a stripped down [[Fedora]] [[Linux]] Operating System. |
||
We're going to be using command-line utilities, so the open the [[Terminal activity]]. |
|||
Type whatever is in bold, then press the [Enter] key. |
|||
⚫ | |||
To do this, let's use the superuser command, "su". |
|||
⚫ | |||
: ''2007 Sugar output'' |
|||
[olpc@localhost]$ su |
|||
---- |
|||
⚫ | |||
⚫ | |||
olpc 1290 1285 3 15:46 ? /usr/bin/python /usr/bin/sugar-shell |
olpc 1290 1285 3 15:46 ? /usr/bin/python /usr/bin/sugar-shell |
||
olpc 1293 1290 0 15:46 ? matchbox-window-manager -kbdconfig /usr/share/sugar/kbdconfig ... |
olpc 1293 1290 0 15:46 ? matchbox-window-manager -kbdconfig /usr/share/sugar/kbdconfig ... |
||
Line 34: | Line 33: | ||
olpc 1339 1285 1 15:46 ? grep sugar |
olpc 1339 1285 1 15:46 ? grep sugar |
||
We can see much of Sugar needs the [[Python]] interpreter as it is written in the Python programming language. We can see some key architectural divisions: |
|||
# A shell |
# A shell |
||
# A presence service (this service is accessed via the right side of the |
# A presence service (this service is accessed via the right side of the Sugar interface) |
||
# A clipboard (The left side of the |
# A clipboard (The left side of the Sugar interface) |
||
# A nm applet (The network manager in the upper right) |
# A nm applet (The network manager in the upper right) '' not present in release 10.1.2?'' |
||
# A console (probably the development console) |
# A console (probably the development console) |
||
==What launches Sugar?== |
|||
Though I'm mentioning aspects of the Sugar User Interface, these services are running as processes in the background. I'd expect the |
Though I'm mentioning aspects of the Sugar User Interface, these services are running as processes in the background. I'd expect the Sugar interface to use these services, but the Sugar interface is probably presented by the shell process. That shell would then pass messages to the other utilities as needed. |
||
Also, there are two non-python type processes running, dbus-launch, and a matchbox-window-manager. Matchbox is the windows manager (It implements X Windows.. [http://projects.o-hand.com/matchbox/ Matchbox home page], also notice it's spawned from the sugar-shell), and the dbus is for Inter-process communications (supporting the presence service, amongst other things) |
Also, there are two non-python type processes running, dbus-launch, and a matchbox-window-manager. Matchbox is the windows manager (It implements X Windows.. [http://projects.o-hand.com/matchbox/ Matchbox home page], also notice it's spawned from the sugar-shell), and the dbus is for Inter-process communications (supporting the presence service, amongst other things) |
||
==What starts the |
===What starts the Sugar shell? === |
||
---- |
|||
==== 2010 Sugar output ==== |
|||
It looks like most of these python scripts are stored in /usr/bin. We can read the python scripts and see what they do. |
It looks like most of these python scripts are stored in /usr/bin. We can read the python scripts and see what they do. |
||
⚫ | |||
If you look for its parent in ps output, this is started by /usr/sbin/olpc-dm |
|||
==== 2007 Sugar output ==== |
|||
⚫ | |||
$ less /usr/bin/sugar-shell |
|||
This thing starts a |
This thing starts a Sugar environment. It also relies on a lot of other stuff, and it looks like the OLPC developers expect a lot of changes here as they make progress on getting Sugar into shape. It references GTK, dbus, nm applet, and lots of stuff. |
||
$ '''less /usr/bin/sugar-nm-applet''' |
|||
Oh, look, this is the network manager utility. OK, and then the activity factory will be triggered whenever a new activity is selected on the bottom.... |
Oh, look, this is the network manager utility. OK, and then the activity factory will be triggered whenever a new activity is selected on the bottom.... |
||
Line 62: | Line 65: | ||
[initialization] |
[initialization] |
||
In the output from ps above, sugar-shell process has a parent of 1285... |
|||
$ '''ps -eaf | grep 1285''' |
|||
olpc 1285 1269 0 15:46 ? xinit /home/olpc/.xinitrc -- -auth /home/olpc/.serverauth.1269 |
olpc 1285 1269 0 15:46 ? xinit /home/olpc/.xinitrc -- -auth /home/olpc/.serverauth.1269 |
||
How about that? In the olpc home directory, there is a hidden .xinitrc file. Let's look! |
How about that? In the olpc home directory, there is a hidden .xinitrc file. Let's look! |
||
$ '''less ~olpc/.xinitrc''' |
|||
Well, this thing launches both the tinderbox and the sugar-shell. It looks like the |
Well, this thing launches both the tinderbox and the sugar-shell. It looks like the Sugar shell is launched within the dbus-launch utility. |
||
So that's where the |
So that's where the Sugar user interface starts... it's: |
||
exec dbus-launch --exit-with-session sugar-shell |
exec dbus-launch --exit-with-session sugar-shell |
||
Line 79: | Line 82: | ||
==What sugar python library does /usr/bin/sugar-shell load?== |
==What sugar python library does /usr/bin/sugar-session/shell load?== |
||
You can run python interactively and find out about packages. |
|||
$ '''python''' |
|||
>>> import sys |
>>> '''import sys''' |
||
>>> import sugar |
>>> '''import sugar''' |
||
>>> dir(sugar) |
>>> '''dir(sugar)''' |
||
['ZOOM_ACTIVITY','ZOOM_FRIENDS','ZOOM_HOME','ZOOM_MESH', <i>etc..</i> ] |
['ZOOM_ACTIVITY','ZOOM_FRIENDS','ZOOM_HOME','ZOOM_MESH', <i>etc..</i> ] |
||
>>> sugar |
>>> sugar |
||
<module 'sugar' from '/usr/lib/ |
<module 'sugar' from '/usr/lib/python''VERSION''/site-packages/sugar/__init__.py'> |
||
>>> '''quit()''' |
|||
That seems pretty definitive. Looking in the /usr/lib/ |
That seems pretty definitive. Looking in the /usr/lib/python''VERSION''/site-packages/sugar directory, there are all sorts of interesting files. |
||
ls /usr/lib/ |
$ '''ls /usr/lib/python*/site-packages/sugar/*''' |
||
There are some interesting directories (each directory represents a sub-package): |
There are some interesting directories (each directory represents a python sub-package): |
||
activity |
activity |
||
bundle |
|||
chat |
|||
clipboard |
|||
datastore |
datastore |
||
graphics |
graphics |
||
p2p |
|||
presence |
presence |
||
... |
|||
But the python scripts are even more interesting (each is a module): |
But the python scripts are even more interesting (each is a module): |
||
Line 113: | Line 116: | ||
In particular, the logger.py & env.py are here. It would be nice to see what the sugar-shell script actually does, and we are ready to try it manually. I'm going to skip stuff that doesn't seem right yet. |
In particular, the logger.py & env.py are here. It would be nice to see what the sugar-shell script actually does, and we are ready to try it manually. I'm going to skip stuff that doesn't seem right yet. |
||
$ '''python''' |
|||
>>> import sys |
>>> '''import sys''' |
||
>>> import os |
>>> '''import os''' |
||
>>> from sugar import logger |
>>> '''from sugar import logger''' |
||
>>> from sugar import profile |
>>> '''from sugar import profile''' |
||
>>> from sugar import env |
>>> '''from sugar import env''' |
||
>>> dir(env) |
>>> '''dir(env) |
||
''in Sugar 2010'' |
|||
[..., 'get_profile_path', 'get_user_activities_path', ...] |
|||
>>> '''env.get_user_activities_path()''' |
|||
'/home/olpc/Activities' |
|||
Now we know that the three directories that house Sugar python scripts are /usr/bin, /usr/lib/python''VERSION''/site-packages/sugar, /usr/lib/python''VERSION''/site-packages/jarabe, and that user-installed [[Activities]] live in an Activities subdirectory of the user's home directory. That's a pretty good day's work. |
|||
''in Sugar 2007'' |
|||
[..., 'sugar_activities_dir', 'sugar_activity_info_dir', 'sugar_data_dir', 'sugar_services_dir', ...] |
[..., 'sugar_activities_dir', 'sugar_activity_info_dir', 'sugar_data_dir', 'sugar_services_dir', ...] |
||
>>> env.sugar_activities_dir |
>>> env.sugar_activities_dir |
||
Line 126: | Line 137: | ||
'/usr/share/sugar' |
'/usr/share/sugar' |
||
Now we know that the three directories that house |
Now we know that the three directories that house Sugar python scripts are /usr/bin, /usr/lib/python2.4/site-packages/sugar, and /usr/share/sugar. That's a pretty good day's work. |
||
==How does |
==How does Sugar initialize?== |
||
Next, let's look slowly at what happens as |
Next, let's look slowly at what happens as Sugar initializes. |
||
/usr/bin/sugar-shell is a python script: #!/usr/bin/python |
/usr/bin/sugar-shell is a python script: #!/usr/bin/python |
||
It imports os & sys, and the pygtk. Then |
It imports os & sys, and the pygtk. Then Sugar makes sure the pygtk libraries are in the path by calling a function that adds the path if it's not there. This is just a bit of robustness. |
||
Next, import a bunch of |
Next, import a bunch of Sugar functions into the top namespace of the python environment... We can see all the names in the top namespace by using the dir() |
||
bash-3.1# python |
bash-3.1# python |
||
Line 146: | Line 157: | ||
['GIntiallyUnowned', 'TracebackUtils', ..., 'env', 'gobject', 'gtk', 'logger', 'os', 'profile', 'pygtk', 'sys'] |
['GIntiallyUnowned', 'TracebackUtils', ..., 'env', 'gobject', 'gtk', 'logger', 'os', 'profile', 'pygtk', 'sys'] |
||
That's the imports. Now |
That's the imports. Now Sugar starts setting up the logging utility: |
||
>>> logger.cleanup() |
>>> logger.cleanup() |
||
Line 152: | Line 163: | ||
That initialized the logger function for tracing the sugar shell. Next, |
That initialized the logger function for tracing the sugar shell. Next, Sugar is going to instantiate its shell. Before the shell can be started, Sugar needs to import more stuff. first, let's set paths to that stuff. Set up our python path to look at /usr/share/sugar/shell first. python will find the shared sugar shell scripts first in our environment. |
||
>>> sys.path.insert(0, os.path.join(env.data_dir(), 'shell') |
>>> sys.path.insert(0, os.path.join(env.data_dir(), 'shell') |
||
Line 182: | Line 193: | ||
==Starting the Shell== |
==Starting the Shell== |
||
Before starting the shell, |
Before starting the shell, Sugar needs to assign a session number to the shell execution so it's identified for DBUS services. This snippet does that, but it looks like it's due for future change (I wouldn't actually do this unless you don't have a sugar shell running yet): |
||
dbsa_file = os.path.join(env.get_path_profile(),"session_bus_address") |
dbsa_file = os.path.join(env.get_path_profile(),"session_bus_address") |
||
Line 196: | Line 207: | ||
>>> shell = Shell(model) |
>>> shell = Shell(model) |
||
OK, the model goes out and registers itself with DBUS, and the presence manager, and tries to get proxies, and will try to attach to [[Sugar Architecture | Avahi]] (a DNS discovery service of sorts). If there is already a |
OK, the model goes out and registers itself with DBUS, and the presence manager, and tries to get proxies, and will try to attach to [[Sugar Architecture | Avahi]] (a DNS discovery service of sorts). If there is already a Sugar model instantiated, you'll get a collision on the bottom of a medium size error stack. |
||
Without a model, you can not start a shell. So this means the next step is to comment out lines in the .xinitrc, and manually launch the sugar shell (note, this is <b>dangerous and could break</b> your OLPC OS image). First, make a clean copy of the .xinitrc just in case |
Without a model, you can not start a shell. So this means the next step is to comment out lines in the .xinitrc, and manually launch the sugar shell (note, this is <b>dangerous and could break</b> your OLPC OS image). First, make a clean copy of the .xinitrc just in case |
Latest revision as of 00:47, 14 August 2010
- This was written in 2007. It's still a useful guide to exploring the code while running Sugar, but the details are different: Python is at 2.6, Sugar has split into sugar and jarabe, the sugar startup components are different, etc.
Understanding Sugar code
This is an attempt to understand how Sugar runs, based on a moderate level of unix shell experience. Here, we'll break down:
- what packages and software Sugar relies on
- where Sugar sets up scripts, libraries, modules, and configuration files
- how Sugar manages user identity
- what Sugar does at runtime (very high level)
- how Sugar cleans itself up when exiting
Other Docs
- This is part 1 of the Sugar internals series. Part 2 is at Sugar Components where you can get an idea of the module/class structure.
- Sugar Architecture
- See also this article in red hat magazine which describes the structure of an Activity.
What runs as part of Sugar?
Sugar runs on a variety of computers, including the OLPC XO models. The OLPC runs a stripped down Fedora Linux Operating System. We're going to be using command-line utilities, so the open the Terminal activity. Type whatever is in bold, then press the [Enter] key.
What runs as part of Sugar? Let's see what processes have the keyword sugar using both the "ps" command to list processes, and the grep command to filter for the processes with the word "sugar" in them.
$ ps -eaf | grep sugar
- 2007 Sugar output
olpc 1290 1285 3 15:46 ? /usr/bin/python /usr/bin/sugar-shell olpc 1293 1290 0 15:46 ? matchbox-window-manager -kbdconfig /usr/share/sugar/kbdconfig ... olpc 1296 1 0 15:46 ? dbus-launch --exit-with-session sugar-shell olpc 1298 1 0 15:46 ? /usr/bin/python /usr/bin/sugar-presence-service olpc 1302 1 0 15:46 ? /usr/bin/python /usr/bin/sugar-clipboard olpc 1304 1 0 15:46 ? /usr/bin/python /usr/bin/sugar-nm-applet olpc 1306 1 4 15:46 ? python /usr/bin/sugar-console olpc 1339 1285 1 15:46 ? grep sugar
We can see much of Sugar needs the Python interpreter as it is written in the Python programming language. We can see some key architectural divisions:
- A shell
- A presence service (this service is accessed via the right side of the Sugar interface)
- A clipboard (The left side of the Sugar interface)
- A nm applet (The network manager in the upper right) not present in release 10.1.2?
- A console (probably the development console)
What launches Sugar?
Though I'm mentioning aspects of the Sugar User Interface, these services are running as processes in the background. I'd expect the Sugar interface to use these services, but the Sugar interface is probably presented by the shell process. That shell would then pass messages to the other utilities as needed.
Also, there are two non-python type processes running, dbus-launch, and a matchbox-window-manager. Matchbox is the windows manager (It implements X Windows.. Matchbox home page, also notice it's spawned from the sugar-shell), and the dbus is for Inter-process communications (supporting the presence service, amongst other things)
What starts the Sugar shell?
2010 Sugar output
It looks like most of these python scripts are stored in /usr/bin. We can read the python scripts and see what they do.
$ less /usr/bin/sugar-session
If you look for its parent in ps output, this is started by /usr/sbin/olpc-dm
2007 Sugar output
$ less /usr/bin/sugar-shell
This thing starts a Sugar environment. It also relies on a lot of other stuff, and it looks like the OLPC developers expect a lot of changes here as they make progress on getting Sugar into shape. It references GTK, dbus, nm applet, and lots of stuff.
$ less /usr/bin/sugar-nm-applet
Oh, look, this is the network manager utility. OK, and then the activity factory will be triggered whenever a new activity is selected on the bottom....
[initialization]
In the output from ps above, sugar-shell process has a parent of 1285...
$ ps -eaf | grep 1285 olpc 1285 1269 0 15:46 ? xinit /home/olpc/.xinitrc -- -auth /home/olpc/.serverauth.1269
How about that? In the olpc home directory, there is a hidden .xinitrc file. Let's look!
$ less ~olpc/.xinitrc
Well, this thing launches both the tinderbox and the sugar-shell. It looks like the Sugar shell is launched within the dbus-launch utility.
So that's where the Sugar user interface starts... it's:
exec dbus-launch --exit-with-session sugar-shell
We can read down from the sugar-shell and read almost all the code from that point down. That's pretty nice.
What sugar python library does /usr/bin/sugar-session/shell load?
You can run python interactively and find out about packages.
$ python >>> import sys >>> import sugar >>> dir(sugar) ['ZOOM_ACTIVITY','ZOOM_FRIENDS','ZOOM_HOME','ZOOM_MESH', etc.. ] >>> sugar <module 'sugar' from '/usr/lib/pythonVERSION/site-packages/sugar/__init__.py'> >>> quit()
That seems pretty definitive. Looking in the /usr/lib/pythonVERSION/site-packages/sugar directory, there are all sorts of interesting files.
$ ls /usr/lib/python*/site-packages/sugar/*
There are some interesting directories (each directory represents a python sub-package):
activity bundle datastore graphics presence ...
But the python scripts are even more interesting (each is a module):
Traceback.py emulator.py env.py logger.py profile.py simulator.py util.py
In particular, the logger.py & env.py are here. It would be nice to see what the sugar-shell script actually does, and we are ready to try it manually. I'm going to skip stuff that doesn't seem right yet.
$ python >>> import sys >>> import os >>> from sugar import logger >>> from sugar import profile >>> from sugar import env >>> dir(env)
in Sugar 2010
[..., 'get_profile_path', 'get_user_activities_path', ...] >>> env.get_user_activities_path() '/home/olpc/Activities'
Now we know that the three directories that house Sugar python scripts are /usr/bin, /usr/lib/pythonVERSION/site-packages/sugar, /usr/lib/pythonVERSION/site-packages/jarabe, and that user-installed Activities live in an Activities subdirectory of the user's home directory. That's a pretty good day's work.
in Sugar 2007
[..., 'sugar_activities_dir', 'sugar_activity_info_dir', 'sugar_data_dir', 'sugar_services_dir', ...] >>> env.sugar_activities_dir '/usr/share/sugar/activities' >>> env.sugar_data_dir '/usr/share/sugar'
Now we know that the three directories that house Sugar python scripts are /usr/bin, /usr/lib/python2.4/site-packages/sugar, and /usr/share/sugar. That's a pretty good day's work.
How does Sugar initialize?
Next, let's look slowly at what happens as Sugar initializes.
/usr/bin/sugar-shell is a python script: #!/usr/bin/python
It imports os & sys, and the pygtk. Then Sugar makes sure the pygtk libraries are in the path by calling a function that adds the path if it's not there. This is just a bit of robustness.
Next, import a bunch of Sugar functions into the top namespace of the python environment... We can see all the names in the top namespace by using the dir()
bash-3.1# python >>> import sys, os, pygtk >>> pygtk.require('2.0') >>> import gtk, gobject >>> from sugar import logger, profile, env, TracebackUtils >>> dir() ['GIntiallyUnowned', 'TracebackUtils', ..., 'env', 'gobject', 'gtk', 'logger', 'os', 'profile', 'pygtk', 'sys']
That's the imports. Now Sugar starts setting up the logging utility:
>>> logger.cleanup() >>> logger.start('shell')
That initialized the logger function for tracing the sugar shell. Next, Sugar is going to instantiate its shell. Before the shell can be started, Sugar needs to import more stuff. first, let's set paths to that stuff. Set up our python path to look at /usr/share/sugar/shell first. python will find the shared sugar shell scripts first in our environment.
>>> sys.path.insert(0, os.path.join(env.data_dir(), 'shell') >>> env.data_dir() '/usr/share/sugar' >>> sys.path ['/usr/share/sugar/shell', ... the regular python path ]
What is each service or utility?
Let's discuss this offline in Sugar Components. The following components are pulled in right now:
>>> from view.FirstTimeDialog import FirstTimeDialog >>> from view.Shell import Shell >>> from model.ShellModel import ShellModel
What's the user's name (and color)?
Sugar is going to pull the user name from the profile. If there is no name, FirstTimeDialog is called to get the user's nick name. Then, the profile utility will store the name and color.
name = profile.get_nick_name() if not name or not len(name): dialog = FirstTimeDialog() dialog.run profile.update()
A quick browse through profile.py shows that the nickname is stored in '/home/olpc/.sugar/default/config'. The config file also contains a default color for the user. I bet the color could be changed by adjusting this string.
Also, there are subdirectories here for activities logs, gecko stuff, network management, and a cache. That directory is a nice find.
How is your user color picked? FirstTimeDialog calls iconcolor.py. Iconcolor.py in turn randomly picks a color from the list in color.py and that's your permanent color. This only happens if you don't have a config file, so if you delete that file, you can change your nickname and color. :)
Starting the Shell
Before starting the shell, Sugar needs to assign a session number to the shell execution so it's identified for DBUS services. This snippet does that, but it looks like it's due for future change (I wouldn't actually do this unless you don't have a sugar shell running yet):
dbsa_file = os.path.join(env.get_path_profile(),"session_bus_address") f = open(dsba_file,"w") f.write(os.environ["DBUS_SESSION_BUS_ADDRESS"]) f.close()
That let's all the processes know who to link to... (I think)
Now let's really start the shell (again, this will break things):
>>> model = ModelShell() >>> shell = Shell(model)
OK, the model goes out and registers itself with DBUS, and the presence manager, and tries to get proxies, and will try to attach to Avahi (a DNS discovery service of sorts). If there is already a Sugar model instantiated, you'll get a collision on the bottom of a medium size error stack.
Without a model, you can not start a shell. So this means the next step is to comment out lines in the .xinitrc, and manually launch the sugar shell (note, this is dangerous and could break your OLPC OS image). First, make a clean copy of the .xinitrc just in case
cd ~ cp .xinitrc .xinitrc~
Next, comment out this line:
#exec dbus....
su # /sbin/shutdown now
I'll run back through all the productive steps:
Starting the network manager
Once the sugar shell is started, there are a couple of afterthoughts that get worked out. First, the Network manager needs to be brought online.
gtk's gobject is used to start this up:
args = ['sugar-nm-applet'] flags = gobject.SPAWN_SEARCH_PATH result = gobject.spawn_async(args,flags=flags, standard_output=False)
Considering that I always have to switch to root a run /sbin/dhclient to get network connectivity, I wonder if this chunk of code works.
Starting up the Traceback Helper
I'll have to figure out what this does and report back to you all.
tbh = TracebackUtils.TracebackHelper()
The gtk.main loop?
This looks like the main sugar shell loop in action. [1]
try: gtk.main() except KeyboardInterrupt: print 'Ctrl+c pressed, exiting...'
Cleaning up on a regular shutdown
del tbh os.remove(dsba_file)