Image builder: Difference between revisions

From OLPC
Jump to navigation Jump to search
(Brief documentation for image-builder tool.)
No edit summary
 
(17 intermediate revisions by 9 users not shown)
Line 1: Line 1:
<b><font color=red><big>For information on current-day OLPC software build systems, see [[Build system]].</big></font></b>
Source code is at: http://dev.laptop.org/git/users/cscott/image-builder


{{Deprecated}}
The image builder tool is very simple: it consists of a single python file named 'build.py', which you can download from http://dev.laptop.org/git?p=users/cscott/image-builder;a=blob_plain;f=build.py;hb=HEAD


== Introduction ==
Given an unpacked customization key in /media/custom (say), you invoke the image builder with:


Image builder is a script which allows you to take an official OS image released by OLPC (produced with [[Pilgrim]]), and add in your own customizations. It is primarily intended for adding activities (.xo) and content bundles (.xol), but advanced users can script other customizations too.
$ build.py -v -o per703-6 /media/custom

Note that many XOs (especially those in early deployments) have a security system enabled, which means that the only OS images that can be installed and booted are those that have been cryptographically signed by OLPC or by the deployment in question. For this reason, you may not be able to boot your resultant image on an XO until you sign it with the relevant keys, or alternatively you could disable the security system on the target XOs. See [[Firmware security]] for more information.

== Basic usage ==

The image builder tool is very simple: it consists of a single python file named 'build.py', which you can download from http://dev.laptop.org/git/users/dsd/image-builder/plain/build.py

First, prepare a collection of activity and content bundles in the style of a [[customization stick]]. Make sure you include an OS image on the customization stick, as explained in the optional step on the [[customization stick]] page (this is not optional for the image builder). This means that you should include a file named something like "os767.img" and a "fs.zip" file in the root of the customization stick.

Assuming the customization stick (including OS image) is mounted or located at (say) /media/custom, you invoke the image builder with:

$ build.py -v -o peru703-6.img /media/custom


This will create peru703-6.img and other files needed to install the image with copy-nand. For more information:
This will create peru703-6.img and other files needed to install the image with copy-nand. For more information:
Line 11: Line 23:
<pre>
<pre>
$ build.py --help
$ build.py --help
Usage: build.py [options] [path-to-customization-key]
Usage: build.py [options] [path-to-customization-stick]


Options:
Options:
Line 17: Line 29:
-o FILE, --output=FILE
-o FILE, --output=FILE
Name of custom image file to generate.
Name of custom image file to generate.
--custom-script=CUSTOM_SCRIPT
Run user-specified customization script after bundle
installation.
--fsck-gnu Allow bundles with missing or bad licenses to be
included in the image
-l Use local osXXX.tar.bz2 and osXXX.tar.bz2.md5 instead
of downloading.
-v Display verbose progress information.
-v Display verbose progress information.
-t, --make-tarball Make fs tarball in addition to image.
-q, --quiet Don't output anything if successful.
-q, --quiet Don't output anything if successful.
</pre>
</pre>
Line 23: Line 43:
You will need the following programs installed on your machine for build.py's use: tar, unzip, mkfs.jffs2, sumtool, crcimg. The mkfs.jffs2 and sumtool binaries are in the 'mtd-utils' package on Debian and Fedora; the crcimg tool is [[Customizing_NAND_images#Creating_CRC_file|described here]].
You will need the following programs installed on your machine for build.py's use: tar, unzip, mkfs.jffs2, sumtool, crcimg. The mkfs.jffs2 and sumtool binaries are in the 'mtd-utils' package on Debian and Fedora; the crcimg tool is [[Customizing_NAND_images#Creating_CRC_file|described here]].


PLEASE BE SURE YOU ARE USING AN UP-TO-DATE mkfs.jffs2, since the resulting images are much more robust if they can be generated with hardlinks built-in. The image builder script will complain if you don't have an appropriate mkfs.jffs2; never ignore this warning for production images.
Contact cscott at laptop dot org for more information.

== Avoiding the big download ==

The image builder downloads a large file (at least 150mb) from the internet every time it is run. If you are going to be running this script more than once, you can save a lot of time by manually performing the download before you run the first time.

The files that you need are named osNNN.tar.bz2 and osNNN.tar.bz2.md5. For example, in the case of OLPC OS 8.2.0 build 767, you can find these files here:
* http://download.laptop.org/xo-1/os/official/767/jffs2/os767.tar.bz2
* http://download.laptop.org/xo-1/os/official/767/jffs2/os767.tar.bz2.md5

Place these in the working directory from where that you run the image builder, and use the "-l" argument to instruct the script to use the local files, e.g.
$ build.py -l -v -o per703-6.img /media/custom

== Advanced customizations ==

The procedure described above allows you to make customizations, but you are limited to installing activities, books, and not much more. Sometimes, you may want to make other customizations too. Image builder has functionality which lets you craft a script to make additional modifications.

Note that making modifications outside of the well-supported realm of installing .xo/.xol files is usually a bad idea. It is hacky, and the changes you make are likely to become a maintenance nightmare in future. The correct way to make such changes is to introduce clean & correct mechanisms upstream, in Sugar/Fedora/other affected components.

If you do want to use this functionality, you pass a command to run to the --custom-script parameter to the image builder. This command will be run with one command line parameter: the path to the image that is currently being built. For example:

$ build.py -v -o per703-6.img --custom-script="./extra.sh" /media/custom

And extra.sh in the current directory might start with:

#!/bin/bash
imgdir=$1
# now make modifications to the files under ${imgdir}

Note that you are seeing the "raw" filesystem view here. While the OLPC home directory can be found at ${imgdir}/home/olpc, other parts of the filesystem such as /usr are found under ${imgdir}/versions/pristine/<build-no>/

When using the custom-script option, the image builder regenerates the .contents file inside the image based on any fs modifications that you have made. This is important because otherwise your resultant customized image will not be compatible with olpc-update (it requires that the .contents file in the image corresponds to the actual filesystem data). As the contents file is regenerated by a chroot call into the modified filesystem, image builder must be run on an x86 system when using a custom script.

When customizing images using scripts, you are advised to make additional modifications to the system to clearly show that the image has been customized, and is not identical to the official OS release that came from OLPC. The usual way to do this is to modify the "Build number" text in the Sugar Control Panel ("About my XO" section) so that it clearly indicates some level of customization. This text is read directly from the /boot/olpc_build text file, so in your customization stick you could modify this accordingly. For example, to indicate that this is the 3rd iteration of the customizations on top of OLPC OS 8.2.1 build 802 for the OLPC Nicaragua deployment, you could add the following to your customization script:

echo "802 Nicaragua v3" > ${imgdir}/versions/pristine/802/boot/olpc_build

== Source code ==


http://dev.laptop.org/git/users/dsd/image-builder

Original author: C. Scott Ananian. Enhancements by Daniel Drake.

[[Category:Build system]]

Latest revision as of 17:00, 8 February 2011

For information on current-day OLPC software build systems, see Build system.

Stop hand.png WARNING:
The content of this section is considered
DEPRECATED and OBSOLETE
It is preserved for historical or documenting reasons.

Introduction

Image builder is a script which allows you to take an official OS image released by OLPC (produced with Pilgrim), and add in your own customizations. It is primarily intended for adding activities (.xo) and content bundles (.xol), but advanced users can script other customizations too.

Note that many XOs (especially those in early deployments) have a security system enabled, which means that the only OS images that can be installed and booted are those that have been cryptographically signed by OLPC or by the deployment in question. For this reason, you may not be able to boot your resultant image on an XO until you sign it with the relevant keys, or alternatively you could disable the security system on the target XOs. See Firmware security for more information.

Basic usage

The image builder tool is very simple: it consists of a single python file named 'build.py', which you can download from http://dev.laptop.org/git/users/dsd/image-builder/plain/build.py

First, prepare a collection of activity and content bundles in the style of a customization stick. Make sure you include an OS image on the customization stick, as explained in the optional step on the customization stick page (this is not optional for the image builder). This means that you should include a file named something like "os767.img" and a "fs.zip" file in the root of the customization stick.

Assuming the customization stick (including OS image) is mounted or located at (say) /media/custom, you invoke the image builder with:

$ build.py -v -o peru703-6.img /media/custom

This will create peru703-6.img and other files needed to install the image with copy-nand. For more information:

$ build.py --help
Usage: build.py [options] [path-to-customization-stick]

Options:
  -h, --help            show this help message and exit
  -o FILE, --output=FILE
                        Name of custom image file to generate.
  --custom-script=CUSTOM_SCRIPT
                        Run user-specified customization script after bundle
                        installation.
  --fsck-gnu            Allow bundles with missing or bad licenses to be
                        included in the image
  -l                    Use local osXXX.tar.bz2 and osXXX.tar.bz2.md5 instead
                        of downloading.
  -v                    Display verbose progress information.
  -t, --make-tarball    Make fs tarball in addition to image.
  -q, --quiet           Don't output anything if successful.

You will need the following programs installed on your machine for build.py's use: tar, unzip, mkfs.jffs2, sumtool, crcimg. The mkfs.jffs2 and sumtool binaries are in the 'mtd-utils' package on Debian and Fedora; the crcimg tool is described here.

PLEASE BE SURE YOU ARE USING AN UP-TO-DATE mkfs.jffs2, since the resulting images are much more robust if they can be generated with hardlinks built-in. The image builder script will complain if you don't have an appropriate mkfs.jffs2; never ignore this warning for production images.

Avoiding the big download

The image builder downloads a large file (at least 150mb) from the internet every time it is run. If you are going to be running this script more than once, you can save a lot of time by manually performing the download before you run the first time.

The files that you need are named osNNN.tar.bz2 and osNNN.tar.bz2.md5. For example, in the case of OLPC OS 8.2.0 build 767, you can find these files here:

Place these in the working directory from where that you run the image builder, and use the "-l" argument to instruct the script to use the local files, e.g.

$ build.py -l -v -o per703-6.img /media/custom

Advanced customizations

The procedure described above allows you to make customizations, but you are limited to installing activities, books, and not much more. Sometimes, you may want to make other customizations too. Image builder has functionality which lets you craft a script to make additional modifications.

Note that making modifications outside of the well-supported realm of installing .xo/.xol files is usually a bad idea. It is hacky, and the changes you make are likely to become a maintenance nightmare in future. The correct way to make such changes is to introduce clean & correct mechanisms upstream, in Sugar/Fedora/other affected components.

If you do want to use this functionality, you pass a command to run to the --custom-script parameter to the image builder. This command will be run with one command line parameter: the path to the image that is currently being built. For example:

$ build.py -v -o per703-6.img --custom-script="./extra.sh" /media/custom

And extra.sh in the current directory might start with:

#!/bin/bash
imgdir=$1
# now make modifications to the files under ${imgdir}

Note that you are seeing the "raw" filesystem view here. While the OLPC home directory can be found at ${imgdir}/home/olpc, other parts of the filesystem such as /usr are found under ${imgdir}/versions/pristine/<build-no>/

When using the custom-script option, the image builder regenerates the .contents file inside the image based on any fs modifications that you have made. This is important because otherwise your resultant customized image will not be compatible with olpc-update (it requires that the .contents file in the image corresponds to the actual filesystem data). As the contents file is regenerated by a chroot call into the modified filesystem, image builder must be run on an x86 system when using a custom script.

When customizing images using scripts, you are advised to make additional modifications to the system to clearly show that the image has been customized, and is not identical to the official OS release that came from OLPC. The usual way to do this is to modify the "Build number" text in the Sugar Control Panel ("About my XO" section) so that it clearly indicates some level of customization. This text is read directly from the /boot/olpc_build text file, so in your customization stick you could modify this accordingly. For example, to indicate that this is the 3rd iteration of the customizations on top of OLPC OS 8.2.1 build 802 for the OLPC Nicaragua deployment, you could add the following to your customization script:

echo "802 Nicaragua v3" > ${imgdir}/versions/pristine/802/boot/olpc_build

Source code

http://dev.laptop.org/git/users/dsd/image-builder

Original author: C. Scott Ananian. Enhancements by Daniel Drake.