User:Quozl/fs-update-skip-unused-blocks: Difference between revisions

From OLPC
Jump to navigation Jump to search
Line 5: Line 5:
Things to do:
Things to do:
*testing, using [http://dev.laptop.org/~quozl/tinycore/mc.zd mc.zd] a [[Tiny Core Linux]] build, 14Mb download, 64Mb install, takes about five seconds for fs-update, [done],
*testing, using [http://dev.laptop.org/~quozl/tinycore/mc.zd mc.zd] a [[Tiny Core Linux]] build, 14Mb download, 64Mb install, takes about five seconds for fs-update, [done],
*olpc-os-builder, merge zhashfs changes that skip patterned blocks, see [http://lists.laptop.org/pipermail/devel/2011-April/031855.html patch v2],
*olpc-os-builder, merge zhashfs changes that skip patterned blocks, see [http://lists.laptop.org/pipermail/devel/2011-April/031855.html patch v2], maintainer desires rewrite,
*olpc-os-builder, merge sd card image scripts that create patterned blocks, see [http://lists.laptop.org/pipermail/devel/2011-April/031853.html patch],
*olpc-os-builder, merge sd card image scripts that create patterned blocks, see [http://lists.laptop.org/pipermail/devel/2011-April/031853.html patch], no response,
*openfirmware, gain agreement on meaning of zblocks: #eblocks parameter [done], and then correct regression in q3a64 regarding use of the parameter, see [http://lists.laptop.org/pipermail/devel/2011-April/031851.html patch], included in [http://dev.laptop.org/~quozl/q3a64q.rom q3a64q], release q3a65,
*openfirmware, gain agreement on meaning of zblocks: #eblocks parameter [done], correct regression in q3a64 regarding use of the parameter [done], see [http://lists.laptop.org/pipermail/devel/2011-April/031851.html patch], included in [http://dev.laptop.org/~quozl/q3a64q.rom q3a64q], committed in -r2192 [done], to be released in q3a65, [todo]
*openfirmware, related problem, -r2190 causes data timeouts on certain cards, investigating,
*openfirmware, related problem, trac>10825</trac>, -r2190 causes data timeouts on certain cards, investigated, solved in -r2192, [done],
*find or create a trac ticket,
*find or create a trac ticket,
*test 10.1.3 os860, 11.2.0 os16,
*for fs-update erase issues <trac>10825</trac>, compare BLKDISCARD path in e2fsprogs, drivers/mmc/core/core.c:mmc_erase(), called by drivers/mmc/card/block.c:mmc_blk_issue_discard_rq(),
*test olpc-os-builder and f11 fresh local build,
*test olpc-os-builder and f14 fresh local build,
*test nandblaster,
*test nandblaster,
*public testing, with olpc-os-builder and f11,
*public testing, with olpc-os-builder and f11,

Revision as of 07:26, 27 April 2011

By skipping unused blocks during fs-update, the installation time is decreased.

New Builds

Things to do:

  • testing, using mc.zd a Tiny Core Linux build, 14Mb download, 64Mb install, takes about five seconds for fs-update, [done],
  • olpc-os-builder, merge zhashfs changes that skip patterned blocks, see patch v2, maintainer desires rewrite,
  • olpc-os-builder, merge sd card image scripts that create patterned blocks, see patch, no response,
  • openfirmware, gain agreement on meaning of zblocks: #eblocks parameter [done], correct regression in q3a64 regarding use of the parameter [done], see patch, included in q3a64q, committed in -r2192 [done], to be released in q3a65, [todo]
  • openfirmware, related problem, trac>10825</trac>, -r2190 causes data timeouts on certain cards, investigated, solved in -r2192, [done],
  • find or create a trac ticket,
  • test 10.1.3 os860, 11.2.0 os16,
  • test olpc-os-builder and f11 fresh local build,
  • test olpc-os-builder and f14 fresh local build,
  • test nandblaster,
  • public testing, with olpc-os-builder and f11,
  • public testing, with olpc-os-builder and f14,

Existing Builds

How to remove the unused blocks from a .zd file.

This procedure takes a .zd file such as os860 from 10.1.3, and creates a new .zd file that has the unused blocks removed, which results in decreased installation time. In my tests, installation time reduces from 14 minutes to 7 minutes on a 4GB microSD.

Works on Q3A62. Generates a minor warning on Q3A63 and Q3A64, but there is no missing data. Q3A65 may have the fix.

Pristine Install

Select the .zd file you wish to process. In this example, I use os860.zd.

Use fs-update to install the file on a laptop, but do not boot from it:

fs-update u:\os860.zd

Imaging

Boot Tiny Core Linux from USB on the laptop, using a USB drive with at least 4GB free space.

Copy the microSD card to the USB drive as an image:

mount /dev/sda1 /mnt
dd if=/dev/mmcblk0 of=/mnt/fs.img bs=131072 count=29489
umount /mnt

(the block count is the decimal conversion of the hexadecimal 7331 found in the first line of the os860.zd file).

Pattern Fill Unused Blocks

On a high-speed system, mount the USB drive.

Create a pseudo-random pattern fill, which is the same size as the filesystem block size:

img=fs.img
dd if=/dev/urandom of=$img.fill bs=4096 count=1 2>/dev/null

Obtain the partition offsets in bytes:

parted $img "unit B print"

In my test, the output was:

Number  Start      End          Size         Type     File system  Flags
 1      4194304B   71303167B    67108864B    primary  ext2         boot
 2      71303168B  3865051135B  3793747968B  primary  ext3

Setup loop block devices for the partitions within the image:

losetup -o 4194304 /dev/loop0 $img
losetup -o 71303168B /dev/loop1 $img

Fill free blocks in the filesystems with the pseudo-random pattern:

fill-free-blocks -v /dev/loop0 $img.fill
fill-free-blocks -v /dev/loop1 $img.fill

Disassociate the loop block devices:

losetup -d /dev/loop0
losetup -d /dev/loop1

(requires fill-free-blocks.c, see http://dev.laptop.org/~quozl/fs-update-skip-unused-blocks).

Generate .zd File

Regenerate the .zd file, skipping all blocks with the pattern:

zhashfs 0x20000 sha256 $img /tmp/fs.zsp /tmp/fs.zd

(requires a patched version of zhashfs, see http://dev.laptop.org/~quozl/fs-update-skip-unused-blocks).

The .zd file is almost the same size, but will install faster. The block display by OpenFirmware will skip forward over unused blocks.