Image file formats

A collection of file formats.

DjVu

The main site for information on DjVu compression format for ebooks is here http://www.djvuzone.org/
Recently a good overview article was published on News Forge.

In a nutshell, DjVu was invented to solve this problem:

Conventional web formats such as JPEG, GIF, and PNG produce prohibitively large image files at decent resolution. As a result, Web site content developers have been largely unable to leverage existing printed materials.

DjVu is intended to be used with scanned images of book pages, either black & white or full color. It then compresses those scanned pages to produce very highly compressed files.

Given that the target countries for the OLPC have poorly developed computing infrastructures, scanning of existing printed documents into DjVu format may be the fastest way of making a wide variety of educational material and Ebooks available to the kids.

DjVu is supported by the Evince reader which is being used by the OLPC project.

If you want to contribute to the DjVu project in any way, here is the site: http://djvulibre.djvuzone.org/

Why is DjVu important?

In regions where computers are scarce and there is little support for native scripts, DjVu allows existing paper books to be scanned and distributed as ebooks. Even handwritten books can be distributed this way. Tie this together with the OLPC chat application's support for SVG input and the GECKO support for displaying SVG graphics and it is conceivable to distribute a computer with no font support and NO TEXT AT ALL in its user interface. Icons would substitute for text in the UI and handwriting would be a primary mode of input. Note that the OLPC has a wider than normal touchpad that can be used as a handwriting input device.

Of course, this is a bootstrap scenario. Once the OLPC is deployed in this way, native language speakers will begin to work on fonts, and a keyboard layout to enable text use on the OLPC. This could take months or years to sort out, but in the meantime, the kids have an educational tool to use.

Producing DjVu Documents

A simple guide to create DjVu files (from JPEG and PDF) and to bind multiple DjVu files into one, is available at howtoforge.

Workflow Planning

First, you need to think of this in terms of setting up a workflow. There are several steps, some of which require technical expertise and some which do not. In addition, the expertise required to set up and maintain the workflow is different from that required to make encoding decisions and check the quality of scans.

Scanning

Some scanners can handle bound books but they cost a lot more money. However, if you can spare a copy, then you can take it apart and scan the pages on a flatbed scanner. Save the files in an uncompressed TIFF format because they will be processed further. Pages should be scanned in color because the DjVu compression software produces a better result that way.

If your original scans are not perfect, you may need to use software such as Book Restorer or Unpaper to clean them up. This is especially important when you are scanning old, rare books that have been damaged in some way, for instance stains on the pages. In addition, when a book is rare you cannot cut out the pages to do perfectly flat scans. This means that the scans will be curved but software can repair these curves.

Check this wiki for additional scanning advice.

Encoding the Pages

Next, you need to process the individual page scans with various tools to encode the pages. Different encoding tools may be used for different pages depending on the presence of illustrations, photos, colored text, etc. Pages can be segmented into a black and white layer and a color layer so that different encoders can be used on each. In addition, if you have an OCR program for the script that the book is written in, you can run the black and white segment through it. DjVu readers are capable of using the OCR to do text searches and then highlighting the words in the actual scanned text image.

Bundling and Postprocessing

After this you have various pieces which you need to bundle together into a multipage book file. Then, you may wish to further process the book to add text annotations, precompute thumbnail images of pages, etc. Perhaps the book is written in an archaic form of the language and you wish to annotate it with a glossary similar to what we do with Shakespeare's plays.

Testing:

Don't forget to test your book thoroughly using Evince to make sure that there are no problems with using it on the OLPC.

Tools

If you would rather have the scanning done by a company with expertise in the field, that is possible. Once the first pilot country is deployed, there will likely be other companies who can offer this service. But the tools needed are all open source so you can also set up your own production line for scanning books.

Articles and Papers

• this article from Professional Surveyor magazine explains how the National Land Survey of Sweden went about converting their historical archive to DjVu format.

• this article on howtoforge details the entire workflow for producing DjVu books on Linux. It also includes some scripts useful in making the process run more smoothly.

GIF

GIF is old, proprietary but widely supported format. Today it's mostly surpassed by PNG - in all regards except one: support for simple animations. If you need animated icon or something like that - use GIF, by all means (all patents related to GIF are expired by now), but for anything else... there are PNG and JPEG.

There is a page on choosing image formats that will help you to understand the differences and how to know which format will be best for the intended use.

More info can be found on Wikipedia.

JPEG

JPEG 2000 is the most recent work of the JPEG standards committee. It is intended to improve upon several areas of the original JPEG standard such as lossy encoding, blocky appearance, and inefficient compression.

Although this standard is intended to be implemented on a no-charge basis, the JPEG committee does allow for inclusion of patented or otherwise non-free technology as per ITU and ISO policy on reasonable and non-descriminatory terms. It is problematic or impossible for open source/free software to implement such standards.