Kernel: Difference between revisions

From OLPC
Jump to navigation Jump to search
(→‎Installing OLPC kernel RPMs: add link to 'testing' build area)
No edit summary
 
(65 intermediate revisions by 13 users not shown)
Line 1: Line 1:
{{OLPC}}
[[Category:Developers]]
[[Category:Developers]]
[[Category:Software]]
[[Category:Software]]
{{Merge | Rebuilding OLPC kernel}}


== The OLPC kernel ==
We use a modified version of the Linux kernel on the OLPC laptop.


OLPC OS [[Releases]] use our OLPC-modified version of the [[Linux]] kernel.
'''Note:''' You will not be able to boot a new kernel on your XO until you have a [[Activation_and_Developer_Keys | Developer Key]].


== The OLPC kernel git repository ==
The OLPC kernel is maintained in git:
* git://dev.laptop.org/olpc-kernel
* http://dev.laptop.org/git/olpc-kernel


Current maintainers: James Cameron.
We use git to track changes that we've made, as well as to easily merge with Linus's (and others!) kernel tree. The git repository can be downloaded with the following command:


We work on specific git branches, which vary by hardware and operating system release.
<pre>
git clone git://dev.laptop.org/olpc-2.6
</pre>


{| border="1" cellspacing="0" cellpadding="2" width="100%" class="wikitable"
Alternatively, you can use gitweb to view and search changes by going to:
! operating system release
! git branches
|-
| (none) ARM XO-1.75
| [http://dev.laptop.org/git/olpc-kernel/log/?h=olpc-5.0 olpc-5.0]
|-
| (none) i386 XO-1 XO-1.5
| olpc-4.6
|-
| [[14.1.0]] i386 XO-1 XO-1.5
| olpc-3.10
|-
| [[Android]] 5.x ARM XO-4
| arm-3.5-android
|-
| [[Android]] 4.x ARM XO-4
| arm-3.5-android-4.x
|-
| [[14.1.0]] [[13.2.1]] [[13.2.0]] [[13.1.0]] ARM XO-4
| arm-3.5
|-
| [[13.2.1]] [[13.2.0]] [[13.1.0]] [[12.1.0]] i386 XO-1 XO-1.5
| x86-3.3
|-
| [[14.1.0]] [[13.2.1]] [[13.2.0]] [[13.1.0]] [[12.1.0]] [[11.3.1]] ARM XO-1.75
| [http://dev.laptop.org/git/olpc-3.0/log/?h=arm-3.0-wip arm-3.0-wip]
|-
| [[11.2.0]] i386
| [http://dev.laptop.org/git/olpc-kernel/log/?h=olpc-2.6.35 olpc-2.6.35]
|-
| [[Release_notes/10.1.3|10.1.3]] i386
| [http://dev.laptop.org/git/olpc-kernel/log/?h=olpc-2.6.31 olpc-2.6.31]
|}


Or looking at git history may identify which branches we are actively working on, and looking at the version of each branch compared to the kernel shipped in each release will help you identify which branch corresponds to which release(s).
http://dev.laptop.org/git?p=olpc-2.6;a=summary


OLPC-specific configurations can be found at arch/x86/configs/ and arch/arm/configs/, these are what we use for automated builds and official releases. This configuration is also installed in /boot in our software releases.
Note that at the bottom of that page, there is a list of branches. We keep around different branches for a number of reasons; within a cloned git repository, you can switch to branch 'FOO' by running:


Questions and contributions should be sent to the devel [[Mailing lists|mailing list]].
<pre>
git checkout FOO
</pre>


=== Installing kernel RPMs ===
The branches to note are:


<b>The following section only applies to pre-13.2 OS releases. On 13.2.0 and newer, installing a new kernel RPM on an XO will automatically activate it on the boot partition as well.</b>
* master - this is the default branch, and is the equivalent of cvs's HEAD. Code that we want in our future releases goes into this branch, and needs to be vetted before being committed.
* stable - this is the branch that we automatically pull kernels from when we build a release. It is usually a snapshot of the 'master' branch, although we sometimes cherry-pick fixes into it.
* testing - this is the branch for joyride kernels. It is usually more recently updated than stable but not as recently updated as master.


Due to OLPC's versioned filesystem layout, installing a new kernel RPM is not enough for it to become active on reboot. This is explained in detail in /boot/README. To summarise, after installing a kernel you must copy it to a special place:
The other branches can be ignored - if you don't already know what they are, they're not going to be of any use to you.


If running a software release that uses partitions, first check to ensure that /bootpart is mounted (it should have several directories in it). If it isn't, mount it:
== OLPC kernel configuration ==
mount /dev/mmcblk0p1 /bootpart


Then:
The Linux kernel has a config file that describes the various options that are built into the kernel. For OLPC, we store that in the root of the git repository, in the 'arch/i386/configs/olpc_defconfig' file. Via gitweb, it can be viewed (for the 'master' branch) here:
rsync --delete-before -av /boot/ /bootpart/boot/


If running a software release that does not use partitions:
http://dev.laptop.org/git?p=olpc-2.6;a=blob;f=arch/i386/configs/olpc_defconfig;h=f1675e86312eee9c1cc9e59ebefca91ff8afe979;hb=HEAD
cp -a /boot/* /versions/boot/current/boot/


=== Building a kernel RPM ===
For official OLPC kernel RPMS (and on OLPC OS images), the config ends up being installed as /boot/config-$VERS. This allows you to see how the currently running kernel is configured.


* If using OLPC OS on your build host, install general development packages needed for kernel compiling:
== Installing OLPC kernel RPMs ==
yum install -y git rpmdevtools gcc make ncurses-devel m4 redhat-rpm-config
* Install build prerequisites:
yum install -y unifdef lzop dracut-modules-olpc
:* On Fedora 14 and older, [[Dracut-modules-olpc#Special considerations for Fedora 14 and older|be careful with dracut-modules-olpc]].
* Install kernel prerequisites:
yum install -y libertas-usb8388-olpc-firmware \
libertas-sd8686-firmware \
libertas-sd8787-firmware
* Clone the git repository; you will need at least 1.5 GBytes free:
git clone git://dev.laptop.org/olpc-kernel
* Checkout the appropriate branch:
git checkout x86-3.3
* Clean the source tree:
make clean distclean
* Make your source changes,
* Make any defconfig changes, for the laptop model,
** XO-1: arch/x86/configs/xo_1_defconfig
** XO-1.5: arch/x86/configs/xo_1.5_defconfig
** XO-1.75: arch/arm/configs/xo_175_defconfig
** XO-4: arch/arm/configs/xo_4_defconfig
* Commit all your changes,
* Run {{code|olpc/buildrpm}}, for the laptop model;
** XO-1: {{code|olpc/buildrpm 1.0}}
** XO-1.5: {{code|olpc/buildrpm 1.5}}
** XO-1.75: {{code|olpc/buildrpm 1.75}}
** XO-4: {{code|olpc/buildrpm 4}}


The output RPM files will be in olpc/RPMS/ which you can install on an XO, with the above versioned filesystem layout issues in mind.
We auto-build kernel RPMs every 30 mins, which means that new, shiny commit that just occurred will show up in the following places (depending upon branch):


=== Cross compiling ===
* master - http://dev.laptop.org/~dilinger/master/
* stable - http://dev.laptop.org/~dilinger/stable/
* testing - http://dev.laptop.org/~dilinger/testing/


Compiling ARM kernels on ARM is sloooow. It's very easy to cross-compile from a faster box.
The SRPMs (Source RPMs) are available in the same place.


Another option (a little more tricky) would be to use [http://crosstool-ng.org/ crosstool-ng] to build and install a cross-compiler.
The RPMs have the dates they were built, and the number of the build. For example, 'kernel-2.6.21-20070322.4.olpc.5fe63334a44da42.i586.rpm' was built on March 22, 2007. It was the 4th build on that day. The '5fe63334a44da42' is part of the git commit ID, if you're trying to match up what specific commits made it into the kernel.


==== Fedora ====
The 'kernel-' RPMs contains all you need to boot into that new kernel; simply install it, reboot, and you're set. The 'kernel-devel-' RPMs contain headers necessary for building out-of-tree kernel modules. The 'kernel-debuginfo-' RPMs contain debugging info necessary for debugging the kernel with, say, systemtap.


On Fedora 16 or later, you can install an ARM cross compiler with:
To install your rpm:
yum install gcc-arm-linux-gnu binutils-arm-linux-gnu
rpm -ivh kernel-....rpm

cp -a /boot/* /versions/boot/current/boot/
To build the kernel, set these environment variables:
(If you omit the -a in the above recipe, the vmlinuz symlink will not be updated correctly)
export ARCH=arm
export CROSS_COMPILE=armv7-linux-gnu-

==== Ubuntu ====

On Ubuntu 13.04 or later, you can install an ARM cross compiler with:
apt-get install gcc-arm-linux-gnueabi

To build the kernel, set these environment variables:
export ARCH=arm
export CROSS_COMPILE=arm-linux-gnueabi-


Now you can compile kernels in the normal way. Here is a recipe:

mkdir -p /tmp/xo/lib/modules
mkdir -p /tmp/xo/bootpart/boot
cp arch/arm/configs/xo*fig .config
make oldconfig zImage
make modules
INSTALL_MOD_STRIP=1 INSTALL_MOD_PATH=/tmp/xo make modules_install
cp arch/arm/boot/zImage /tmp/xo/bootpart/boot/
(cd /tmp/xo/bootpart/boot/ && ln -sf zImage vmlinuz)

After this, /tmp/xo can be tar'd up and extracted on the XO, on USB media, or on an NFS mount point.

=== Automatic RPM generation ===

During development some branches are automatically built every 30 minutes (but only when there are new changes), the results are published at http://dev.laptop.org/~kernels/

The directories you see there are also linked to [[RPM Dropbox|RPM dropboxes]], therefore are automatically published for inclusion in development builds. In other words, to get your kernel changes included in the current software release development efforts, you simply have to push your changes to the correct branch.

The utility behind this magic is olpc-kernel-builder:
* http://dev.laptop.org/git/projects/olpc-kernel-builder/
* git://dev.laptop.org/projects/olpc-kernel-builder
* Maintainer: Chris Ball

When used under OLPC OS 13.2.0 on an XO-4, change the rpm_target to armv7hl in olpc/buildrpm, otherwise an armv7l rpm is generated, resulting in refusal to install with ''is intended for a different architecture''. This is only a packaging architecture and does not affect how the kernel is compiled.

Install mock to use olpc-kernel-builder
yum install -y mock
:Does not work under OLPC OS 13.2.0, mock fails to mount /proc in the chroot. --[[User:Quozl|Quozl]] 05:32, 21 February 2014 (UTC)

== Running development kernels ==

Starting with 13.1.0 development builds, the /boot (or /bootpart/boot) area of the OS images has changed layout slightly. The kernel and initramfs are now <em>only</em> shipped in zip files, in order to reduce duplication for signed builds, and to remove duplication of firmware code in the olpc.fth boot script.

Dropping your own vmlinuz in place (or booting one from USB as described below, pointing at the internal disk for the initrd.img initramfs) will no longer work as standard, since this will cause the system to start looking for an unzipped initramfs too. The story is similar for initramfs development - if you drop a "raw" initramfs in place, you will need to ensure that an unzipped kernel is also installed. Just run, once, as root:

# olpc-dev-kernel

This will unzip the initramfs from the zip file and save it as <tt>initrd.img</tt>, and similarly it will unzip the kernel from its zip file and save it as <tt>vmlinuz</tt>. In both cases, if an existing initrd.img/vmlinuz file is present, it will not be overwritten.

Now drop your files in place as you have done so before, and the system will pick up the "unzipped" partner initramfs/kernel file.

One more detail to be aware of: if you put "unzipped" initramfs/kernel files in place using this method, you will no longer receive automatic firmware updates from bootfw.zip. This should only affect cases where you drop your own development kernel/initramfs in place, <em>then</em> upgrade the olpc-firmware RPM package, <em>and</em> expect that firmware upgrade to be automatically installed.

For more details, read the original proposal: [http://lists.laptop.org/pipermail/devel/2012-March/034439.html Removing duplication in /boot - affects kernel development].

== Running vanilla kernels ==

In late 2010, OLPC started a focused effort to upstream its kernel patches so that unmodified "vanilla" kernels run with complete functionality on the XO-1 and XO-1.5.

As of Linux-3.5 these efforts are mostly concluded - OLPC only carries a small number of patches, which will be upstreamed as time permits.

You may have to hunt around a bit to satisfy the configuration options so that the relevant drivers get built. Start with our defconfig in the appropriate branch. If in doubt, feel free to ask on the devel [[mailing lists|mailing list]] for someone's recent .config file.

== Boot your development kernel from USB ==

When doing kernel development, it is time consuming to commit each small change, make an RPM, install RPM on the XO, reboot, and test. Instead, you can follow this procedure:


On your development system, prepare a USB disk as follows:
See the instructions at [[Kernel Building]] if you need to update modules in the initramfs. If you don't do this, you will likely get modprobe warnings at boot (before the initramfs starts); they are harmless.


<b>XO-1.75 and XO-4:</b>
== Building an OLPC kernel ==
* /boot/olpc.fth (download from [http://dev.laptop.org/~dsd/20111013/olpc.fth here])
* /boot/zImage (your kernel that you want to test)


<b>XO-1.5 and XO-1</b>
So you've decided that you want to build an OLPC kernel? Are you certain? The kernel is filled with arcane magic and sharp pointy things that may draw blood. If you're really serious, see the [[Kernel Building]] page.
* /boot/olpc.fth (download for [http://dev.laptop.org/~dsd/20110221/olpc.fth.xo1 XO-1] or [http://dev.laptop.org/~dsd/20110221/olpc.fth.xo15 XO-1.5], be sure to rename to olpc.fth)
* /boot/irfs.img (download [http://dev.laptop.org/~dsd/20110221/irfs.img here] - this is a hacked OLPC initramfs which doesn't require /ofw support)
* /boot/bzImage (your kernel that you want to test)


Now plug this disk into your XO, turn it on, and the system will load the kernel from USB while booting the rest of the system as normal. If your new kernel crashes badly, just unplug the USB disk and boot the system again - it will revert to booting the "good" kernel from local disk.
== Building an OLPC kernel RPM ==


''Note: for XO-1 this only works with JFFS2. (small tweaks to olpc.fth or the hacked initramfs would be needed for ubifs).''
It takes a bit longer to build, but it's easier to distribute if you build a proper kernel RPM. The instructions for doing so are on the [[Rebuilding OLPC kernel]] page.


[[Category:Subsystems]]
[[Category:Subsystems]]

Latest revision as of 08:04, 20 April 2019


The OLPC kernel

OLPC OS Releases use our OLPC-modified version of the Linux kernel.

The OLPC kernel is maintained in git:

Current maintainers: James Cameron.

We work on specific git branches, which vary by hardware and operating system release.

operating system release git branches
(none) ARM XO-1.75 olpc-5.0
(none) i386 XO-1 XO-1.5 olpc-4.6
14.1.0 i386 XO-1 XO-1.5 olpc-3.10
Android 5.x ARM XO-4 arm-3.5-android
Android 4.x ARM XO-4 arm-3.5-android-4.x
14.1.0 13.2.1 13.2.0 13.1.0 ARM XO-4 arm-3.5
13.2.1 13.2.0 13.1.0 12.1.0 i386 XO-1 XO-1.5 x86-3.3
14.1.0 13.2.1 13.2.0 13.1.0 12.1.0 11.3.1 ARM XO-1.75 arm-3.0-wip
11.2.0 i386 olpc-2.6.35
10.1.3 i386 olpc-2.6.31

Or looking at git history may identify which branches we are actively working on, and looking at the version of each branch compared to the kernel shipped in each release will help you identify which branch corresponds to which release(s).

OLPC-specific configurations can be found at arch/x86/configs/ and arch/arm/configs/, these are what we use for automated builds and official releases. This configuration is also installed in /boot in our software releases.

Questions and contributions should be sent to the devel mailing list.

Installing kernel RPMs

The following section only applies to pre-13.2 OS releases. On 13.2.0 and newer, installing a new kernel RPM on an XO will automatically activate it on the boot partition as well.

Due to OLPC's versioned filesystem layout, installing a new kernel RPM is not enough for it to become active on reboot. This is explained in detail in /boot/README. To summarise, after installing a kernel you must copy it to a special place:

If running a software release that uses partitions, first check to ensure that /bootpart is mounted (it should have several directories in it). If it isn't, mount it:

mount /dev/mmcblk0p1 /bootpart

Then:

rsync --delete-before -av /boot/ /bootpart/boot/

If running a software release that does not use partitions:

cp -a /boot/* /versions/boot/current/boot/

Building a kernel RPM

  • If using OLPC OS on your build host, install general development packages needed for kernel compiling:
yum install -y git rpmdevtools gcc make ncurses-devel m4 redhat-rpm-config
  • Install build prerequisites:
yum install -y unifdef lzop dracut-modules-olpc
  • Install kernel prerequisites:
yum install -y libertas-usb8388-olpc-firmware \
               libertas-sd8686-firmware \
               libertas-sd8787-firmware
  • Clone the git repository; you will need at least 1.5 GBytes free:
git clone git://dev.laptop.org/olpc-kernel
  • Checkout the appropriate branch:
git checkout x86-3.3
  • Clean the source tree:
make clean distclean
  • Make your source changes,
  • Make any defconfig changes, for the laptop model,
    • XO-1: arch/x86/configs/xo_1_defconfig
    • XO-1.5: arch/x86/configs/xo_1.5_defconfig
    • XO-1.75: arch/arm/configs/xo_175_defconfig
    • XO-4: arch/arm/configs/xo_4_defconfig
  • Commit all your changes,
  • Run olpc/buildrpm, for the laptop model;
    • XO-1: olpc/buildrpm 1.0
    • XO-1.5: olpc/buildrpm 1.5
    • XO-1.75: olpc/buildrpm 1.75
    • XO-4: olpc/buildrpm 4

The output RPM files will be in olpc/RPMS/ which you can install on an XO, with the above versioned filesystem layout issues in mind.

Cross compiling

Compiling ARM kernels on ARM is sloooow. It's very easy to cross-compile from a faster box.

Another option (a little more tricky) would be to use crosstool-ng to build and install a cross-compiler.

Fedora

On Fedora 16 or later, you can install an ARM cross compiler with:

yum install gcc-arm-linux-gnu binutils-arm-linux-gnu

To build the kernel, set these environment variables:

export ARCH=arm
export CROSS_COMPILE=armv7-linux-gnu-

Ubuntu

On Ubuntu 13.04 or later, you can install an ARM cross compiler with:

apt-get install gcc-arm-linux-gnueabi

To build the kernel, set these environment variables:

export ARCH=arm
export CROSS_COMPILE=arm-linux-gnueabi-


Now you can compile kernels in the normal way. Here is a recipe:

mkdir -p /tmp/xo/lib/modules
mkdir -p /tmp/xo/bootpart/boot
cp arch/arm/configs/xo*fig .config
make oldconfig zImage
make modules
INSTALL_MOD_STRIP=1 INSTALL_MOD_PATH=/tmp/xo make modules_install
cp arch/arm/boot/zImage /tmp/xo/bootpart/boot/
(cd /tmp/xo/bootpart/boot/ && ln -sf zImage vmlinuz)

After this, /tmp/xo can be tar'd up and extracted on the XO, on USB media, or on an NFS mount point.

Automatic RPM generation

During development some branches are automatically built every 30 minutes (but only when there are new changes), the results are published at http://dev.laptop.org/~kernels/

The directories you see there are also linked to RPM dropboxes, therefore are automatically published for inclusion in development builds. In other words, to get your kernel changes included in the current software release development efforts, you simply have to push your changes to the correct branch.

The utility behind this magic is olpc-kernel-builder:

When used under OLPC OS 13.2.0 on an XO-4, change the rpm_target to armv7hl in olpc/buildrpm, otherwise an armv7l rpm is generated, resulting in refusal to install with is intended for a different architecture. This is only a packaging architecture and does not affect how the kernel is compiled.

Install mock to use olpc-kernel-builder

yum install -y mock
Does not work under OLPC OS 13.2.0, mock fails to mount /proc in the chroot. --Quozl 05:32, 21 February 2014 (UTC)

Running development kernels

Starting with 13.1.0 development builds, the /boot (or /bootpart/boot) area of the OS images has changed layout slightly. The kernel and initramfs are now only shipped in zip files, in order to reduce duplication for signed builds, and to remove duplication of firmware code in the olpc.fth boot script.

Dropping your own vmlinuz in place (or booting one from USB as described below, pointing at the internal disk for the initrd.img initramfs) will no longer work as standard, since this will cause the system to start looking for an unzipped initramfs too. The story is similar for initramfs development - if you drop a "raw" initramfs in place, you will need to ensure that an unzipped kernel is also installed. Just run, once, as root:

# olpc-dev-kernel

This will unzip the initramfs from the zip file and save it as initrd.img, and similarly it will unzip the kernel from its zip file and save it as vmlinuz. In both cases, if an existing initrd.img/vmlinuz file is present, it will not be overwritten.

Now drop your files in place as you have done so before, and the system will pick up the "unzipped" partner initramfs/kernel file.

One more detail to be aware of: if you put "unzipped" initramfs/kernel files in place using this method, you will no longer receive automatic firmware updates from bootfw.zip. This should only affect cases where you drop your own development kernel/initramfs in place, then upgrade the olpc-firmware RPM package, and expect that firmware upgrade to be automatically installed.

For more details, read the original proposal: Removing duplication in /boot - affects kernel development.

Running vanilla kernels

In late 2010, OLPC started a focused effort to upstream its kernel patches so that unmodified "vanilla" kernels run with complete functionality on the XO-1 and XO-1.5.

As of Linux-3.5 these efforts are mostly concluded - OLPC only carries a small number of patches, which will be upstreamed as time permits.

You may have to hunt around a bit to satisfy the configuration options so that the relevant drivers get built. Start with our defconfig in the appropriate branch. If in doubt, feel free to ask on the devel mailing list for someone's recent .config file.

Boot your development kernel from USB

When doing kernel development, it is time consuming to commit each small change, make an RPM, install RPM on the XO, reboot, and test. Instead, you can follow this procedure:

On your development system, prepare a USB disk as follows:

XO-1.75 and XO-4:

  • /boot/olpc.fth (download from here)
  • /boot/zImage (your kernel that you want to test)

XO-1.5 and XO-1

  • /boot/olpc.fth (download for XO-1 or XO-1.5, be sure to rename to olpc.fth)
  • /boot/irfs.img (download here - this is a hacked OLPC initramfs which doesn't require /ofw support)
  • /boot/bzImage (your kernel that you want to test)

Now plug this disk into your XO, turn it on, and the system will load the kernel from USB while booting the rest of the system as normal. If your new kernel crashes badly, just unplug the USB disk and boot the system again - it will revert to booting the "good" kernel from local disk.

Note: for XO-1 this only works with JFFS2. (small tweaks to olpc.fth or the hacked initramfs would be needed for ubifs).