RPM Dropbox

From OLPC
Jump to: navigation, search

Introduction

OLPC's dropbox system allows contributors to submit RPM packages for inclusion in XO software releases that are in development stages. The contributor places the RPM files in a specific place under his home directory on OLPC's development server, and the dropbox system then aggregates them alongside packages from other users, turns them into a yum repository, and publishes them at http://rpmdropbox.laptop.org

During initial development of OLPC OS releases, the build system uses a selection of these repositories a source of packages. The exact set of dropboxes used for each development stream is noted on the release page for the in-development release in question. However, just placing a new package in the dropbox is not enough for inclusion of an entirely new package; the build system lists the packages to be included in the builds, this must be updated so that it gets pulled in.

These repositories are aggregated and published every 15 minutes.

Each repository includes a "filelist.txt" file which shows the sources (i.e. which user) of each package in the dropbox.

If a package gets placed in the dropbox with the same name as a package in Fedora, the version in the dropbox takes precedence -- even if the Fedora repositories include a newer version of the same package. This provides OLPC with a simple method to make modifications to Fedora packages.

Packages in the dropbox are not vetted before being automatically included in development builds. However, once the software release reaches stabilization stages, OLPC switches to using a frozen repositories system where each and every change is vetted and pushed by the release manager. Dropboxes are no longer used by the build system at that point, but tend to stick around as a way of pushing your new customized packages to the release manager for review and inclusion in the builds. See the Release Process documentation for the details.

User:DanielDrake is the current maintainer of this system. User:Quozl occasionally triggers or changes it.

Please don't use this system!

This system should only be used in exceptional cases!

OLPC does not have a large software development team and anything put in these repositories becomes entirely our problem.

If you want to add a new package to the system, you should get it packaged directly in Fedora, and get it included in OLPC builds via the regular Fedora package channels.

If you need to fix a bug in a package that comes from Fedora, you may be tempted to whip up a patch, add it to the RPM, and put it in the RPM dropbox. However, the correct approach is to get the bug fixed properly upstream and in Fedora.

For the build system, packages in the dropbox take precedence over Fedora repositories (even if Fedora has a newer version). That means that the following situation is possible:

  1. You decide take foobar-1.0-1 from Fedora, fork it as foobar-1.0-1.olpc to add a patch that we need to solve a bug
  2. foobar-1.1 comes out, fixing a really important security or data loss bug
  3. Fedora pushes foobar-1.1 immediately due to the severity of the bug
  4. OLPC does not receive this new version due to the presence of foobar-1.0-1.olpc in the dropbox, and nobody on OLPC's small team notices the severe bug

OLPC does not only work on a short-term basis. Every time we change Fedora version, we must rebuild and rebase everything in the dropbox. This is often a time consuming process. The net effect is that if you put a package in the dropbox, you take away development time from the following development cycle, as you must spend time bringing forward all the non-upstream changes.

As a rule of thumb, when you put something in the dropbox, think about who is going to maintain it in the next 6 months, 1 year, 5 years (hint: this becomes YOUR responsibility). We often only use the dropbox on a short-term basis with a solid route of getting the real fix into Fedora, because forking packages is too much of a drain on our resources.

Conventions

Please submit source RPM (SRPM) too

When adding a package to the dropbox, please submit the corresponding .src.rpm file to the same dropbox. This allows people to identify the changes you have made (if any), do future rebuilds, etc.

Version tagging

If forking a package from Fedora, basing your fork on top of an existing package version, you should add .olpcX to the release tag, after the disttag. The X represents an OLPC-specific release number. You should never modify the release number specified by the original Fedora package, that way it is clear upon which package your fork is based.

For example, if Fedora started with v1.4-4.fc14 as follows: 

Version: 1.4
Release: 4%{?dist}

Then when you fork you would put: 

Version: 1.4
Release: 4%{?dist}.olpc1

A few days later, after applying yet another change on top of that version, you would bump olpc1 to olpc2: 

Version: 1.4
Release: 4%{?dist}.olpc2

Which dropbox to use?

Dropboxes are named according to the base Fedora release being targetted, e.g. f17 for Fedora 17. For each base release, several dropboxes are provided: e.g. f17, f17-xo1.5, f17-xo1.75.

The laptop-specific dropboxes such as f17-xo1.75 are to be used for anything that is specific to the laptop model in question at the hardware level. Two packages that fall into this category are bootfw and kernel, and this list is unlikely to grow in future.

Everything else should go in the main f17 dropbox in this example. This includes our trivially-forked Sugar package, etc (yes, it is OK to mix arm and x86 packages in the same repo, yum does the right thing).

Usage

Accounts on dev.laptop.org are available to proven contributors, see the Project hosting page.

Dropbox usage is simply maintaining a collection of packages under a public_rpms directory in your home directory. Inside public_rpms, create a directory with the name of the dropbox you want to contribute to, and put RPM files inside.

Adding a package

If this is the first time you are contributing to the dropbox in question, you must create the parent directory. For example, for the 'f14' dropbox: 

ssh user@dev.laptop.org
mkdir -p public_rpms/f14

Now it is just a case of putting files in that directory. You could do this from your local machine with the scp command, e.g. 

scp foobar-1.1-1.fc14.i586.rpm foobar-1.1-1.fc14.src.rpm dev.laptop.org:public_rpms/f14

Removing a package

Removing a package from the dropbox is as simple as removing it from your own public_rpms folder. Simply login to dev.laptop.org and use the rm command to remove the files. Note that dropbox files are not permanently archived, so removing files usually equates to deleting them forever; take a backup if you think you might need this file again.

History

The canonical URL for the output of the dropbox system used to be http://xs-dev.laptop.org/~dsd/repos. That system died so we moved the system elsewhere and renamed it to http://rpmdropbox.laptop.org. The old URL is kept working however.

How it works

This section is only for curious people or people performing maintenance/development on the system. You don't need to understand this if you just want to use the dropbox system.

The code can be found at:

The do-repocollect script is run by crontab in account dsd on crank.

The rpmcollector-local.sh variant is the script used at this time. It looks at everyone's home directories for RPM dropboxes, combines them all, creates yum repositories, and outputs them in a directory served by apache. After synchronizing, the switch from old to new repositories is done atomically.

Retrieved from "http://wiki.laptop.org/mediawiki/index.php?title=RPM_Dropbox&oldid=296526"