User:Quozl/fs-update-skip-unused-blocks

By skipping unused blocks during fs-update, the installation time is decreased.

New Builds

Things to do:

• testing, using mc.zd a Tiny Core Linux build, 14Mb download, 64Mb install, takes about five seconds for fs-update, [done],
• olpc-os-builder, merge zhashfs changes that skip patterned blocks, see patch v2, maintainer desires rewrite,
• olpc-os-builder, merge sd card image scripts that create patterned blocks, see patch, no response,
• openfirmware, gain agreement on meaning of zblocks: #eblocks parameter [done], correct regression in q3a64 regarding use of the parameter [done], see patch, included in q3a64q, committed in -r2192 [done], to be released in q3a65, [todo]
• openfirmware, related problem, trac>10825</trac>, -r2190 causes data timeouts on certain cards, investigated, solved in -r2192, [done],
• find or create a trac ticket,
• test 10.1.3 os860, 11.2.0 os16,
• test olpc-os-builder and f11 fresh local build,
• test olpc-os-builder and f14 fresh local build,
• test nandblaster,
• public testing, with olpc-os-builder and f11,
• public testing, with olpc-os-builder and f14,

Existing Builds

How to remove the unused blocks from a .zd file.

This procedure takes a .zd file such as os860 from 10.1.3, and creates a new .zd file that has the unused blocks removed, which results in decreased installation time. In my tests, installation time reduces from 14 minutes to 7 minutes on a 4GB microSD.

Works on Q3A62. Generates a minor warning on Q3A63 and Q3A64, but there is no missing data. Q3A65 may have the fix.

Pristine Install

Select the .zd file you wish to process. In this example, I use os860.zd.

Use fs-update to install the file on a laptop, but do not boot from it:

fs-update u:\os860.zd

Imaging

Boot Tiny Core Linux from USB on the laptop, using a USB drive with at least 4GB free space.

Copy the microSD card to the USB drive as an image:

mount /dev/sda1 /mnt
dd if=/dev/mmcblk0 of=/mnt/fs.img bs=131072 count=29489
umount /mnt

(the block count is the decimal conversion of the hexadecimal 7331 found in the first line of the os860.zd file).

Pattern Fill Unused Blocks

On a high-speed system, mount the USB drive.

Create a pseudo-random pattern fill, which is the same size as the filesystem block size:

img=fs.img
dd if=/dev/urandom of=$img.fill bs=4096 count=1 2>/dev/null

Obtain the partition offsets in bytes:

parted $img "unit B print"

In my test, the output was:

Number  Start      End          Size         Type     File system  Flags
 1      4194304B   71303167B    67108864B    primary  ext2         boot
 2      71303168B  3865051135B  3793747968B  primary  ext3

Setup loop block devices for the partitions within the image:

losetup -o 4194304 /dev/loop0 $img
losetup -o 71303168B /dev/loop1 $img

Fill free blocks in the filesystems with the pseudo-random pattern:

fill-free-blocks -v /dev/loop0 $img.fill
fill-free-blocks -v /dev/loop1 $img.fill

Disassociate the loop block devices:

losetup -d /dev/loop0
losetup -d /dev/loop1

(requires fill-free-blocks.c, see http://dev.laptop.org/~quozl/fs-update-skip-unused-blocks).

Generate .zd File

Regenerate the .zd file, skipping all blocks with the pattern:

zhashfs 0x20000 sha256 $img /tmp/fs.zsp /tmp/fs.zd

(requires a patched version of zhashfs, see http://dev.laptop.org/~quozl/fs-update-skip-unused-blocks).

The .zd file is almost the same size, but will install faster. The block display by OpenFirmware will skip forward over unused blocks.