RPM Dropbox
Contents
Introduction
OLPC's dropbox system allows contributors to submit RPM packages for inclusion in XO software releases that are in development stages. The contributor places the RPM files in a specific place under his home directory on OLPC's development server, and the dropbox system then aggregates them alongside packages from other users, turns them into a yum repository, and publishes them at http://xs-dev.laptop.org/~dsd/repos/
During initial development of OLPC OS releases, the build system uses a selection of these repositories a source of packages. The exact set of dropboxes used for each development stream is noted on the release page for the in-development release in question. However, just placing a new package in the dropbox is not enough for inclusion of an entirely new package; the build system lists the packages to be included in the builds, this must be updated so that it gets pulled in.
These repositories are aggregated and published every 30 minutes.
Each repository includes a "filelist.txt" file which shows the sources (i.e. which user) of each package in the dropbox.
If a package gets placed in the dropbox with the same name as a package in Fedora, the version in the dropbox takes precedence -- even if the Fedora repositories include a newer version of the same package. This provides OLPC with a simple method to make modifications to Fedora packages.
Packages in the dropbox are not vetted before being automatically included in development builds. However, once the software release reaches stabilization stages, OLPC switches to using a "frozen repository" system where each and every change is vetted and pushed by the release manager. Dropboxes are no longer used by the build system at that point, but tend to stick around as a way of pushing your new customized packages to the release manager for review and inclusion in the builds. See the Release Process documentation for the details.
Please don't use this system!
This system should only be used in exceptional cases!
OLPC does not have a large software development team and anything put in these repositories becomes entirely our problem.
If you want to add a new package to the system, you should get it packaged directly in Fedora, and get it included in OLPC builds via the regular Fedora package channels.
If you need to fix a bug in a package that comes from Fedora, you may be tempted to whip up a patch, add it to the RPM, and put it in the RPM dropbox. However, the correct approach is to get the bug fixed properly upstream and in Fedora.
For the build system, packages in the dropbox take precedence over Fedora repositories (even if Fedora has a newer version). That means that the following situation is possible:
- You decide take foobar-1.0-1 from Fedora, fork it as foobar-1.0-1.olpc to add a patch that we need to solve a bug
- foobar-1.1 comes out, fixing a really important security or data loss bug
- Fedora pushes fedora-1.1 immediately due to the severity of the bug
- OLPC does not receive this new version due to the presence of foobar-1.0-1.olpc in the dropbox
OLPC does not only work on a short-term basis. Every time we change Fedora version, we must rebuild and rebase everything in the dropbox. This is often a time consuming process. The net effect is that if you put a package in the dropbox, you take away development time from the following development cycle, as you must spend time bringing forward all the non-upstream changes.
As a rule of thumb, when you put something in the dropbox, think about who is going to maintain it in the next 6 months, 1 year, 5 years (hint: this becomes YOUR responsibility). We often only use the dropbox on a short-term basis with a solid route of getting the real fix into Fedora, because forking packages is too much of a drain on our resources.
Usage
Accounts on dev.laptop.org are available to proven contributors, see the Project hosting page.
|
The currency of this article or section may be limited by out-of-date information. There may be relevant discussion on its talk page |
(old content from Build system, to be updated soon)
Dropbox Mechanism
The dropbox mechanism is a way for people who don't have the resources to work through koji to supply us with activities and RPMs.
It is implemented in two pieces, one running on the dropbox-host and one running on the build-host. The dropbox-host simply provides an rsync daemon that will transfer packages to the builder on request.
The build-host uses the program 'collect-pkgs.py' to aggregate packages and change-entries into an RPM repository for composition by pilgrim.
Instructions for Use
Before getting started, you will need an account on dev.laptop.org (See Project hosting).
To add a completely new piece of software (either a new activity or a new package) to a build, three steps are necessary.
1. Include a source code package (.xo or .src.rpm) and a binary package (.xo or .rpm) in your dropbox (which is located at ~/public_rpms/<build>/ on dev.laptop.org). You will need to create the folder, here is an example for joyride:
mkdir -p ~/public_rpms/joyride
Note: do not remove the .xpm's or .xo's from the public_rpms location, not even after they already showed up in an image: public_rpms is not a temporary place where the build systems grab them; it will grab them from that location every run.
2. Update the change log to describe the changes made by the packages you are introducing. We accumulate changelog entries for ~/public_rpms/<build>/ChangeLog and ~/public_rpms/<build>/*.changes, so you can use a separate .changes file for each major group of packages you maintain.
3. Where possible, it's best to add your package as a new dependency of an existing package, so that the reason for the inclusion of this package is clear. If this is not possible, file a bug in trac with component 'distro' and owner 'cscott' requesting that the new package or activity be added to the build.
To verify that a package (here Web-72.xo) has been picked up you can do the following:
First you can look in the repos/joyride if there was a Web-*.xo present. Another interesting file in the same directory is the ChangeLog which is the aggregate ChangeLog that describes all of the packages pulled in from every dropbox. It contains warnings emitted when some problem prevents your RPMs from being picked up. Finally, you can read the buildlog to check if Web-72.xo was installed.
- Bert made a script that aggregates changelogs as joyride-pkgs.html, which is useful for comparing build versions.
ChangeLog Format
The format of this ChangeLog is
<pkg-lines> (containing ".xo", ".rpm")
<detail-lines> (not containing the string ".xo", ".rpm", "--")
-- <signature> (-- is the line marker)
<pkg>
<pkg>.rpm
<detail>
-- <signature>
An example:
Etoys-61.xo etoys-2.2.1710-1.noarch.rpm etoys-2.2.1710-1.src.rpm * changed activity launch to match new Sugar * words in tiles separated by spaces not camelCase * adds "distance to" and "bearing to" tiles * better .pot sort order -- Bert Freudenberg <bert@freudenbergs.de> Tue, 16 Oct 2007 squeak-vm-3.9-12olpc1.i386.rpm squeak-vm-3.9-12olpc1.src.rpm * big merge with trunk * unicode key events * fixed dbus plugin -- Bert Freudenberg <bert@freudenbergs.de> Mon, 15 Oct 2007
Common Mistakes
You must use different time stamps for each ChangeLog entry on the -- author line.
If there are multiple authors of a single build-patch, you should list each person's contribution like so:
pkg.rpm * change -- author 1 pkg.rpm * change -- author 2
rather than like
pkg.rpm * change -- author 1 * change -- author 2
until we improve the ChangeLog parser.
