OS Builder/Run on XO hardware: Difference between revisions

From OLPC
Jump to navigation Jump to search
No edit summary
No edit summary
 
(53 intermediate revisions by 4 users not shown)
Line 1: Line 1:
It is usually possible, if a bit slow, to run OS builder on XO hardware. On platforms where fast machines are available (such as x86), using a server-class "builder" machine is recommended.
It is usually possible, if a bit slow, to run OS builder on XO hardware. On platforms where fast machines are available (such as x86), using a server-class "builder" machine is recommended.


When resources are limited, or if server-class machines are not available for the platform (such as ARM), this recipe is recommended.
When resources are limited, or if server-class machines are not easily available for the platform (such as ARM), this recipe is recommended.


The procedure has been tested with XO-1.75 and XO-4 hardware and builds of the 11.3.x series and 13.2.x series.
The procedure has been tested with XO-1.5, XO-1.75 and XO-4 hardware and 13.2.0-13. (OLPC uses this procedure on XO-4 with 13.2.7, so there's a good chance you'll face no extra difficulties using 13.2.7.)


= Ingredients=
= Ingredients =


Each of these must be dedicated to the task for as long as you need to make builds:
* XO-1.75 or XO-4

* Good, fast SD card for swap,
* an XO-1.5, XO-1.75 or XO-4<ref>use an XO-1.5 to make XO-1 or XO-1.5 builds, and use an XO-1.75 or XO-4 to make XO-1.75 or XO-4 builds. An XO-1 with 13.2.0 cannot be used because there is not enough internal storage space to install the packages described here.</ref>,
* External USB HDD -- fast, large capacity, to store cached RPMs and build files,
* optional<ref>we have tested without swap on XO-1.5, XO-1.75 and XO-4, each of which have at least 1GB of RAM.</ref>; an SD card for [[Swap|swap]]; minimum size 512 MB, good quality, fast access, e.g. class 10,
* a USB hard disk drive for storing RPMs and build files; minimum size 16 GB, fast, large capacity,
* an unfiltered internet connection<ref>the builder must be able to download files from repositories, and filtering may prevent file downloads, in particular one user reported RPMs starting with the name ''tracker'' were blocked.</ref>,
* unsecure laptops to test your builds with,
* a way to distribute your builds to other people.


= Recipe =
= Recipe =


Each step must be done in order:
* Install a recent OS (we used 11.3.1-17 or 13.2.0-8)
* Boot to Sugar or Gnome, log in, connect to network
* Disable automatic power management from the Sugar control panel
* Insert external SD card to be used for swap
* Prepare the SD for swap -- in a terminal, as root
# see what block device id is the external SD
mount
# unmount any existing partition
umount /media/mysdcard
# use fdisk to delete manufacturer's partition
# create a new partition, of at least 2GB, type 82
fdisk /dev/mmcblkN
# prepare the new partition to be swap
# it will report a UUID identifier
partprobe
mkswap /dev/mmcblkNp1
# add this line to fstab to use swap automatically on every boot
UUID=<UUID reported by mkswap> swap swap defaults 0 0
# enable all swap partitions in fstab
swapon -a -v


* mark the system as a builder, so that it is not taken away by accident,
* Prepare the HDD partitioning it with fdisk with a large partition of type "83", then use mkfs to create an ext3 or ext4 partition.
* install [[Release_notes/13.2.0|13.2.0-13]],<!-- or [[Release_notes/13.2.2|13.2.2-15]] on XO-4, -->
* Ensure the system date is correct, if not -- use date -s "current date" to set
* boot to Sugar or Gnome, log in, and connect to your wireless network,
* Follow the [[OS_Builder/Development|OS Builder development installation instructions]] -- remember to disable firstboot, and note that you will want to have your OS Builder directory on the external USB disk
* optional; insert a USB ethernet adapter and connect to your wired network<ref>using a USB ethernet adapter instead of a wireless network can make builds go a bit faster, because network packets are not lost. On an XO-4 with the 8787 combined wireless and bluetooth module, use a USB ethernet adapter or upgrade the kernel, see <trac>12694</trac>. Note sure if you have an 8787? See [[Firmware/Identifying_Wireless_LAN_Device|how to check]].</ref>,
* On F14, you will need to bind-mount /var/tmp to be on your external USB HDD; some large tmp files:
* start a Terminal and become [[root]]:
mkdir /media/externaldisk/vartmp
su
cp -pr /var/tmp/* /media/externaldisk/vartmp/
* disable automatic power management<ref>automatic power management should be disabled because it interferes with the builder; the USB hard drive may not be remounted after a suspend and resume.</ref>:
mount -o bind /media/externaldisk/vartmp /var/tmp
touch /etc/powerd/flags/inhibit-suspend
* For 11.3.x, we want to work on branch v4.0, so
* configure the system to keep documentation after package installation<ref>the change to macros.imgcreate is needed so that the example files in olpc-os-builder are kept, which are classified as documentation for the olpc-os-builder package, and are deleted by default on OLPC OS.</ref>:
git checkout -b v4.0 origin/v4.0
sed -i '/excludedocs/d' /etc/rpm/macros.imgcreate
* For 13.2.x, install the olpc-os-builder package:
* install some packages<ref>we need to install many packages to use the system as a builder, because by design OLPC OS does not have these packages already installed.</ref>:
sed -i '/excludedocs/d' /etc/rpm/macros.imgcreate
yum install olpc-os-builder
yum install -y ntpdate gparted olpc-os-builder binutils git gcc make libtomcrypt-devel zlib-devel
* ensure the system date and time are correct:
* During test builds, generating only one image is faster -- edit examples/olpc-os-11.3.1-xo1.75.ini to disable 8GB image generation and comment out the "usb_update" module.
ntpdate pool.ntp.org
* optional; insert an SD card, and use gparted (Applications -> System Tools -> GParted Partition Editor) to delete the manufacturer's partition and add a ''linux-swap'' partition, then add it to the filesystem table:
eval $(blkid -o export /dev/mmcblk1p1) # get the UUID
echo UUID=$UUID swap swap defaults 0 0 >> /etc/fstab # add to table
swapon -a -v # enable swap
* prepare the USB hard drive using gparted (Applications -> System Tools -> GParted Partition Editor), partitioning it with a single ''ext4'' partition, then add it to the filesystem table:
eval $(blkid -o export /dev/sda1) # get the UUID
umount $DEVNAME # unmount from /media
mkdir /oob # create a mount point
echo UUID=$UUID /oob $TYPE auto,rw 0 0 >> /etc/fstab # add to table
mount -a # mount the filesystem
* download v7.0<ref>the v7.0 branch of olpc-os-builder is what was used to create 13.2.1.</ref> of olpc-os-builder to the USB hard drive:
cd /oob # change directory
git clone -b v7.0 git://dev.laptop.org/projects/olpc-os-builder # download
cd olpc-os-builder # change directory
* build the zhashfs binary:
(cd bin && make)
* optional, for initial testing, not final release; customise your build to make it build faster, by editing examples/olpc-os-13.2.1-xo4.ini
** in the ''[base]'' module, set ''versioned_fs=0'' and ''make_tree_tarball=0'', (this disables the use of ''olpc-update''),
** remove the ''[usb_update]'' module, (this disables the use of ''olpc-update'' with USB drives),
** reduce the ''[global] langs='' to the languages you need,
** '''Note that these configuration changes disable some functionality which you might need later.''' So you should only make these changes permanently if you understand what these options do, and you know that you do not need the relevant functionality.


Your setup is ready, you can now do:
Your setup is ready, you can now make the first build:
sudo ./osbuilder.py examples/olpc-os-11.3.1-xo1.75.ini
./osbuilder.py examples/olpc-os-13.2.1-xo4.ini

The first build will take much longer than subsequent builds, because it will download RPMs and activities to disk. Subsequent builds will still download some package lists, but not as much as the first build.

After the build, the directory /oob/olpc-os-builder/build/output should have the files that can be used to install the build on another system.

= Experiences =

On XO-1.5, XO-1.75 and XO-4 with a typical USB hard drive, the build takes between half an hour and an hour on subsequent runs. See [[/Experiences|experiences]] for further detail.

= Decommissioning =

When the builder is no longer required, shut it down, remove the SD card and repartition it, remove the USB drive and repartition it, and then reinstall OLPC OS on the system.
= Notes =
<references/>

= Fedora 20 Notes =
* use ''modprobe loop'' before build,
* on XO-4, <trac>12837</trac> (yum crash, infrequent 25%) presents a difficulty.

Latest revision as of 10:42, 17 September 2016

It is usually possible, if a bit slow, to run OS builder on XO hardware. On platforms where fast machines are available (such as x86), using a server-class "builder" machine is recommended.

When resources are limited, or if server-class machines are not easily available for the platform (such as ARM), this recipe is recommended.

The procedure has been tested with XO-1.5, XO-1.75 and XO-4 hardware and 13.2.0-13. (OLPC uses this procedure on XO-4 with 13.2.7, so there's a good chance you'll face no extra difficulties using 13.2.7.)

Ingredients

Each of these must be dedicated to the task for as long as you need to make builds:

  • an XO-1.5, XO-1.75 or XO-4<ref>use an XO-1.5 to make XO-1 or XO-1.5 builds, and use an XO-1.75 or XO-4 to make XO-1.75 or XO-4 builds. An XO-1 with 13.2.0 cannot be used because there is not enough internal storage space to install the packages described here.</ref>,
  • optional<ref>we have tested without swap on XO-1.5, XO-1.75 and XO-4, each of which have at least 1GB of RAM.</ref>; an SD card for swap; minimum size 512 MB, good quality, fast access, e.g. class 10,
  • a USB hard disk drive for storing RPMs and build files; minimum size 16 GB, fast, large capacity,
  • an unfiltered internet connection<ref>the builder must be able to download files from repositories, and filtering may prevent file downloads, in particular one user reported RPMs starting with the name tracker were blocked.</ref>,
  • unsecure laptops to test your builds with,
  • a way to distribute your builds to other people.

Recipe

Each step must be done in order:

  • mark the system as a builder, so that it is not taken away by accident,
  • install 13.2.0-13,
  • boot to Sugar or Gnome, log in, and connect to your wireless network,
  • optional; insert a USB ethernet adapter and connect to your wired network<ref>using a USB ethernet adapter instead of a wireless network can make builds go a bit faster, because network packets are not lost. On an XO-4 with the 8787 combined wireless and bluetooth module, use a USB ethernet adapter or upgrade the kernel, see <trac>12694</trac>. Note sure if you have an 8787? See how to check.</ref>,
  • start a Terminal and become root:
 su
  • disable automatic power management<ref>automatic power management should be disabled because it interferes with the builder; the USB hard drive may not be remounted after a suspend and resume.</ref>:
 touch /etc/powerd/flags/inhibit-suspend
  • configure the system to keep documentation after package installation<ref>the change to macros.imgcreate is needed so that the example files in olpc-os-builder are kept, which are classified as documentation for the olpc-os-builder package, and are deleted by default on OLPC OS.</ref>:
 sed -i '/excludedocs/d' /etc/rpm/macros.imgcreate
  • install some packages<ref>we need to install many packages to use the system as a builder, because by design OLPC OS does not have these packages already installed.</ref>:
 yum install -y ntpdate gparted olpc-os-builder binutils git gcc make libtomcrypt-devel zlib-devel
  • ensure the system date and time are correct:
 ntpdate pool.ntp.org
  • optional; insert an SD card, and use gparted (Applications -> System Tools -> GParted Partition Editor) to delete the manufacturer's partition and add a linux-swap partition, then add it to the filesystem table:
 eval $(blkid -o export /dev/mmcblk1p1)               # get the UUID
 echo UUID=$UUID swap swap defaults 0 0 >> /etc/fstab # add to table
 swapon -a -v                                         # enable swap
  • prepare the USB hard drive using gparted (Applications -> System Tools -> GParted Partition Editor), partitioning it with a single ext4 partition, then add it to the filesystem table:
 eval $(blkid -o export /dev/sda1)                    # get the UUID
 umount $DEVNAME                                      # unmount from /media
 mkdir /oob                                           # create a mount point
 echo UUID=$UUID /oob $TYPE auto,rw 0 0 >> /etc/fstab # add to table
 mount -a                                             # mount the filesystem
  • download v7.0<ref>the v7.0 branch of olpc-os-builder is what was used to create 13.2.1.</ref> of olpc-os-builder to the USB hard drive:
 cd /oob                                                         # change directory
 git clone -b v7.0 git://dev.laptop.org/projects/olpc-os-builder # download
 cd olpc-os-builder                                              # change directory
  • build the zhashfs binary:
 (cd bin && make)
  • optional, for initial testing, not final release; customise your build to make it build faster, by editing examples/olpc-os-13.2.1-xo4.ini
    • in the [base] module, set versioned_fs=0 and make_tree_tarball=0, (this disables the use of olpc-update),
    • remove the [usb_update] module, (this disables the use of olpc-update with USB drives),
    • reduce the [global] langs= to the languages you need,
    • Note that these configuration changes disable some functionality which you might need later. So you should only make these changes permanently if you understand what these options do, and you know that you do not need the relevant functionality.

Your setup is ready, you can now make the first build:

 ./osbuilder.py examples/olpc-os-13.2.1-xo4.ini

The first build will take much longer than subsequent builds, because it will download RPMs and activities to disk. Subsequent builds will still download some package lists, but not as much as the first build.

After the build, the directory /oob/olpc-os-builder/build/output should have the files that can be used to install the build on another system.

Experiences

On XO-1.5, XO-1.75 and XO-4 with a typical USB hard drive, the build takes between half an hour and an hour on subsequent runs. See experiences for further detail.

Decommissioning

When the builder is no longer required, shut it down, remove the SD card and repartition it, remove the USB drive and repartition it, and then reinstall OLPC OS on the system.

Notes

<references/>

Fedora 20 Notes

  • use modprobe loop before build,
  • on XO-4, <trac>12837</trac> (yum crash, infrequent 25%) presents a difficulty.