View source for User:Quozl/fs-update-skip-unused-blocks

By skipping unused blocks during fs-update, the installation time is decreased.

== New Builds ==

Things to do:
*testing, using [http://dev.laptop.org/~quozl/tinycore/mc.zd mc.zd] a [[Tiny Core Linux]] build, 14Mb download, 64Mb install, takes about five seconds for fs-update, [done],
*olpc-os-builder, merge zhashfs changes that skip patterned blocks, see [http://lists.laptop.org/pipermail/devel/2011-April/031855.html patch v2],
*olpc-os-builder, merge sd card image scripts that create patterned blocks, see [http://lists.laptop.org/pipermail/devel/2011-April/031853.html patch],
*openfirmware, gain agreement on meaning of zblocks: #eblocks parameter [done], and then correct regression in q3a64 regarding use of the parameter, see [http://lists.laptop.org/pipermail/devel/2011-April/031851.html patch], included in [http://dev.laptop.org/~quozl/q3a64q.rom q3a64q], release q3a65,
*openfirmware, related problem, -r2190 causes data timeouts on certain cards, investigating,
*find or create a trac ticket,
*for fs-update erase issues <trac>10825</trac>, compare BLKDISCARD path in e2fsprogs, drivers/mmc/core/core.c:mmc_erase(), called by drivers/mmc/card/block.c:mmc_blk_issue_discard_rq(),
*test nandblaster,
*public testing, with olpc-os-builder and f11,
*public testing, with olpc-os-builder and f14,

== Existing Builds ==

How to remove the unused blocks from a .zd file.

This procedure takes a .zd file such as os860 from 10.1.3, and creates a new .zd file that has the unused blocks removed, which results in decreased installation time.  In my tests, installation time reduces from 14 minutes to 7 minutes on a 4GB microSD.

Works on Q3A62.  Generates a minor warning on Q3A63 and Q3A64, but there is no missing data.  Q3A65 may have the fix.

=== Pristine Install ===

Select the .zd file you wish to process.  In this example, I use os860.zd.

Use fs-update to install the file on a laptop, but do not boot from it:

 fs-update u:\os860.zd

=== Imaging ===

Boot Tiny Core Linux from USB on the laptop, using a USB drive with at least 4GB free space.

Copy the microSD card to the USB drive as an image:

 mount /dev/sda1 /mnt
 dd if=/dev/mmcblk0 of=/mnt/fs.img bs=131072 count=29489
 umount /mnt

(the block count is the decimal conversion of the hexadecimal 7331 found in the first line of the os860.zd file).

=== Pattern Fill Unused Blocks ===

On a high-speed system, mount the USB drive.

Create a pseudo-random pattern fill, which is the same size as the filesystem block size:

 img=fs.img
 dd if=/dev/urandom of=$img.fill bs=4096 count=1 2>/dev/null

Obtain the partition offsets in bytes:

 parted $img "unit B print"

In my test, the output was:

 Number  Start      End          Size         Type     File system  Flags
  1      4194304B   71303167B    67108864B    primary  ext2         boot
  2      71303168B  3865051135B  3793747968B  primary  ext3

Setup loop block devices for the partitions within the image:

 losetup -o 4194304 /dev/loop0 $img
 losetup -o 71303168B /dev/loop1 $img

Fill free blocks in the filesystems with the pseudo-random pattern:

 fill-free-blocks -v /dev/loop0 $img.fill
 fill-free-blocks -v /dev/loop1 $img.fill

Disassociate the loop block devices:

 losetup -d /dev/loop0
 losetup -d /dev/loop1

(requires fill-free-blocks.c, see http://dev.laptop.org/~quozl/fs-update-skip-unused-blocks).

=== Generate .zd File ===

Regenerate the .zd file, skipping all blocks with the pattern:
 zhashfs 0x20000 sha256 $img /tmp/fs.zsp /tmp/fs.zd

(requires a patched version of zhashfs, see http://dev.laptop.org/~quozl/fs-update-skip-unused-blocks).

The .zd file is almost the same size, but will install faster.  The block display by OpenFirmware will skip forward over unused blocks.