Firmware release procedures: Difference between revisions

From OLPC
Jump to navigation Jump to search
(udpated to reflect current OFW build/release practice)
Line 3: Line 3:
== Release procedure ==
== Release procedure ==


=== EC Firmware ===
Here is a draft of a BIOS release procedure.

===Stage one: EC:===


* Quanta e-mails EC release and changelog to the OLPC BIOS contact, signed and encrypted with PGP
* Quanta e-mails EC release and changelog to the OLPC BIOS contact, signed and encrypted with PGP
** see notes below
** see notes below
* Quanta and OLPC test this version of EC
* Quanta and OLPC test this version of EC
* When the new EC code has passed preliminary engineering tests, the OLPC firmware team (currently Mitch and Richard) puts it in http://dev.laptop.org/pub/ec/ .
* '''NOTE:''' Due to personnel changes at Quanta, the EC development is currently (as of late Q4 '07)being done by Richard Smith OLPC, for all practical purposes. We may want to revise these EC release procedures to reflect that reality.

=== Wireless Firmware ===

* Marvell sends new wireless firmware to OLPC.
* The connectivity team evaluates it
* If it passes muster, the connectivity team puts it in http://dev.laptop.org/pub/firmware/libertas/
* The connectivity team announces/recommends it by sending email to the firmware and OS teams

=== Source Tree Stabilization ===

* The OFW source repository is at svn://openbios.org/openfirmware
* The firmware team develops and tests OFW changes with private (non-released) builds
* Anyone can make a private test version by checking out the svn source tree and running "make" in cpu/x86/pc/olpc/build/
* To select specific versions of EC and wireless firmware for inclusion in a build, edit cpu/x86/pc/olpc/versions.fth .
* Private versions are tested by the firmware team and selected others, typically by issuing "brown bag" releases distributed from the engineer's public_html directory on dev.laptop.org
* "Brown bag" private versions have names that are distinct from the normal release name schema. Mitch prefers to add a suffix letter - e.g. q2d07a.rom is a brown-bag version that is after the q2d07 release. Richard prefers a name like q2047.rom .
* "Real" release builds are only done after the firmware engineer is fairly certain that the most recent private version is solid and ready to be promoted to an official release.
* Prior to a release build, the firmware release engineer (e.g. Mitch or Richard) ensures that all approved changes have been checked in to the repository and that cpu/x86/pc/olpc/versions.fth refers to the correct versions of EC and wireless firmware. He then builds a private version from the clean tree and tests it well.
* As the final source maintenance step for a new build, the firmware release engineer edits cpu/x86/pc/olpc/versions.fth to set the firmware major and minor version strings to the new release number, then checks in the new version of that file.

===Building a Release===

* Official firmware releases are built on the machine "fc5" (a Xen virtual machine on the host machine "grinch") in the directory "/home/firmware"
* The firmware release engineer ssh's in to grinch, with agent forwarding enabled, then ssh's to fc5.
* Here is an example for how to create a new release "q2d08", when the last release was "q2d07". The following process takes less than a minute to run.

grinch$ ssh fc5
fc5$ cd /home/firmware
fc5$ mkdir q2d08
fc5$ cp q2d07/*me q2d08
fc5$ cd q2d08
fc5$ ./pullme
<Lots of output showing files being checked out>
fc5$ ./buildme
<The output file is in roms/q2d08.rom>

* The firmware release engineer does a quick "smoke test" of the roms/<version>.rom file prior to making it public. Typically that just involves flashing the .rom file into a machine and making sure that it boots. The heavy testing has already been done in the "Source Tree Stabilization" phase.
* If something goes wrong, either a problem with the build process (e.g. a missing file that was not checked in properly) or a test failure on the .rom image, delete all copies of the .rom file, delete the new build directory, fix the source tree, and start over (this is not painful because ./pullme + ./buildme goes quickly).

===Issuing the Release===

* After a clean build and a successful "smoke test", the release engineer makes the release public with:

fc5$ ./releaseme

* That copies the files from the build tree's "roms/" subdirectory to http://dev.laptop.org/pub/firmware/<release_name>/ and changes the symlink http://dev.laptop.org/pub/firmware/LATEST to point to it.
* The release engineer edits the wiki page http://wiki.laptop.org/go/Firmware to create a new index line for the new release
* The release engineer creates a wiki page http://wiki.laptop.org/go/OLPC_Firmware_<release_name>, typically by copying the text of the previous release page and editing it as necessary.
* The release engineer sends an announcement of the new release to devel@laptop.org

===Validating the Release===


* OLPC testers install the new release on some number of systems at 1CC and perhaps other test sites
===Stage two: Buildrom:===
* Assuming that the testing goes well, the ECO team sends mail to Quanta approving the use of the new release in production


===Withdrawing a Release===
⚫
* Pull EC release from http://dev.laptop.org/pub/ec/, check hashes
* Update buildrom changelog and tag for release
* Update SPI flash version string in buildrom binary
* Create buildrom SRPM
* Build two flavors of binary RPM for the two RAM variants


If a dangerous bug is found in a release, it can be withdrawn. A dangerous bug is one that potentially results in bricked systems. To withdraw a release:
===Stage three: Testing:===


⚫
* Manually remove the release directory from http://dev.laptop.org/pub/firmware
* announce build to BIOS team and Ray, release candidate testing begins
* Edit http://wiki.laptop.org/go/Firmware and http://wiki.laptop.org/go/OLPC_Firmware_<release_name> to note that the release has been withdrawn and why.
* test on a 256M board
* Send mail to devel@laptop.org announcing the withdrawal and the reason
* install the binary RPMs on Tinderbox machines
* >12 hours of burn-in warm reboot testing on Tinderbox
* cold boot tests
** we need a cold boot solution; X10 doesn't seem to like the power at OLPC
* After automated tests, send "Who has tested?" mail asking for problem reports
* Release after twelve hours if no problem reports


===Stage four: Release:===
===Cherry-Picking Changes into an Incremental Release===


Releases are normally built from the top of the svn tree, but there are cases where it is necessary to build from an older base version with a few "cherry-picked" changes from later checkins. To see an example of how to do this, look at fc5:/home/firmware/q2d07/pull07 . That script is a point replacement for the generic "pullme". It checks out a specific older version of the source tree, then merges in specific later versions of selected files.
* Sign the firmware
* Release builds kept in a separate directory
* Update the version number from release candidate to final
* Repackage signed copy for the build, and submit to joyride and active stable branches.


==Using PGP for EC code==
==Using PGP for EC code==

Revision as of 00:03, 21 December 2007

  english | 한국어 HowTo [ID# 87097]  +/-  


Release procedure

EC Firmware

  • Quanta e-mails EC release and changelog to the OLPC BIOS contact, signed and encrypted with PGP
    • see notes below
  • Quanta and OLPC test this version of EC
  • When the new EC code has passed preliminary engineering tests, the OLPC firmware team (currently Mitch and Richard) puts it in http://dev.laptop.org/pub/ec/ .
  • NOTE: Due to personnel changes at Quanta, the EC development is currently (as of late Q4 '07)being done by Richard Smith OLPC, for all practical purposes. We may want to revise these EC release procedures to reflect that reality.

Wireless Firmware

  • Marvell sends new wireless firmware to OLPC.
  • The connectivity team evaluates it
  • If it passes muster, the connectivity team puts it in http://dev.laptop.org/pub/firmware/libertas/
  • The connectivity team announces/recommends it by sending email to the firmware and OS teams

Source Tree Stabilization

  • The OFW source repository is at svn://openbios.org/openfirmware
  • The firmware team develops and tests OFW changes with private (non-released) builds
  • Anyone can make a private test version by checking out the svn source tree and running "make" in cpu/x86/pc/olpc/build/
  • To select specific versions of EC and wireless firmware for inclusion in a build, edit cpu/x86/pc/olpc/versions.fth .
  • Private versions are tested by the firmware team and selected others, typically by issuing "brown bag" releases distributed from the engineer's public_html directory on dev.laptop.org
  • "Brown bag" private versions have names that are distinct from the normal release name schema. Mitch prefers to add a suffix letter - e.g. q2d07a.rom is a brown-bag version that is after the q2d07 release. Richard prefers a name like q2047.rom .
  • "Real" release builds are only done after the firmware engineer is fairly certain that the most recent private version is solid and ready to be promoted to an official release.
  • Prior to a release build, the firmware release engineer (e.g. Mitch or Richard) ensures that all approved changes have been checked in to the repository and that cpu/x86/pc/olpc/versions.fth refers to the correct versions of EC and wireless firmware. He then builds a private version from the clean tree and tests it well.
  • As the final source maintenance step for a new build, the firmware release engineer edits cpu/x86/pc/olpc/versions.fth to set the firmware major and minor version strings to the new release number, then checks in the new version of that file.

Building a Release

  • Official firmware releases are built on the machine "fc5" (a Xen virtual machine on the host machine "grinch") in the directory "/home/firmware"
  • The firmware release engineer ssh's in to grinch, with agent forwarding enabled, then ssh's to fc5.
  • Here is an example for how to create a new release "q2d08", when the last release was "q2d07". The following process takes less than a minute to run.
grinch$ ssh fc5
fc5$ cd /home/firmware
fc5$ mkdir q2d08
fc5$ cp q2d07/*me q2d08
fc5$ cd q2d08
fc5$ ./pullme
<Lots of output showing files being checked out>
fc5$ ./buildme
<The output file is in roms/q2d08.rom>
  • The firmware release engineer does a quick "smoke test" of the roms/<version>.rom file prior to making it public. Typically that just involves flashing the .rom file into a machine and making sure that it boots. The heavy testing has already been done in the "Source Tree Stabilization" phase.
  • If something goes wrong, either a problem with the build process (e.g. a missing file that was not checked in properly) or a test failure on the .rom image, delete all copies of the .rom file, delete the new build directory, fix the source tree, and start over (this is not painful because ./pullme + ./buildme goes quickly).

Issuing the Release

  • After a clean build and a successful "smoke test", the release engineer makes the release public with:
 fc5$ ./releaseme

Validating the Release

  • OLPC testers install the new release on some number of systems at 1CC and perhaps other test sites
  • Assuming that the testing goes well, the ECO team sends mail to Quanta approving the use of the new release in production

Withdrawing a Release

If a dangerous bug is found in a release, it can be withdrawn. A dangerous bug is one that potentially results in bricked systems. To withdraw a release:

Cherry-Picking Changes into an Incremental Release

Releases are normally built from the top of the svn tree, but there are cases where it is necessary to build from an older base version with a few "cherry-picked" changes from later checkins. To see an example of how to do this, look at fc5:/home/firmware/q2d07/pull07 . That script is a point replacement for the generic "pullme". It checks out a specific older version of the source tree, then merges in specific later versions of selected files.

Using PGP for EC code

For first release only: This is already done

Download and install GPG4Win from http://www.gpg4win.org/download.html

Create a PGP key and get dwmw2 to sign it to verify that it is Quanta's.

For subsequent releases

Right-click on the EC binary and select the option to create a "detached signature", in plain text. It should create a separate file like 'ECv21.bin.asc', which looks something like this: 

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (MingW32)

iD8DBQBFTxvGmfQ2bFM/BesRAoKzAJ0RNczipB pul5sEUR wCYIQvt /wCguqrV
5GRPVDpdH155fwsDwnu7B4M=
=URby
-----END PGP SIGNATURE-----

Send the EC binary as you normally would, and _also_ attach the separate signature file which is used to verify the binary. There is no need to send another copy of the key itself (0x533F05EB.asc), because we already have that.

Send the EC binary and signature (2 attachments) to software-team@laptop.org

This page needs clean-up and editing for coherence, clarity, and completeness.

Please improve this article if you can.

Retrieved from "http://wiki.laptop.org/mediawiki/index.php?title=Firmware_release_procedures&oldid=87097"