Image file formats: Difference between revisions
No edit summary |
(+DJVU) |
||
| Line 1: | Line 1: | ||
A collection of '''file formats'''. |
A collection of '''file formats'''. |
||
==What is DJVU?== |
|||
The main site for information on [http://en.wikipedia.org/wiki/Djvu DJVU] compression format for ebooks is here http://www.djvuzone.org/ |
|||
<br>Recently a [http://software.newsforge.com/article.pl?sid=06/03/08/2314247 good overview article] was published on News Forge. |
|||
In a nutshell, DJVU was invented to solve this problem: |
|||
:Conventional web formats such as JPEG, GIF, and PNG produce prohibitively large image files at decent resolution. As a result, Web site content developers have been largely unable to leverage existing printed materials. |
|||
DJVU is intended to be used with scanned images of book pages, either black & white or full color. It then compresses those scanned pages to produce [http://www.djvuzone.org/wid/index.html very highly compressed files]. |
|||
Given that the target countries for the OLPC have poorly developed computing infrastructures, [[scanning]] of existing printed documents into DJVU format may be the fastest way of making a wide variety of educational material and [[Ebooks]] available to the kids. |
|||
DJVU is supported by the [[Evince]] reader which is being used by the OLPC project. |
|||
If you want to contribute to the DJVU project in any way, here is the site: |
|||
http://djvulibre.djvuzone.org/ |
|||
===Why is DJVU important?=== |
|||
In regions where computers are scarce and there is little support for native scripts, DJVU allows existing paper books to be scanned and distributed as ebooks. Even handwritten books can be distributed this way. Tie this together with the OLPC chat application's support for [[SVG]] input and the [[GECKO]] support for displaying [[SVG]] graphics and it is conceivable to distribute a computer with no font support and '''NO TEXT AT ALL''' in its user interface. Icons would substitute for text in the UI and handwriting would be a primary mode of input. Note that the OLPC has a wider than normal touchpad that can be used as a handwriting input device. |
|||
Of course, this is a bootstrap scenario. Once the OLPC is deployed in this way, native language speakers will begin to work on fonts, and a keyboard layout to enable text use on the OLPC. This could take months or years to sort out, but in the meantime, the kids have an educational tool to use. |
|||
===How Do We Produce DJVU Documents?=== |
|||
[http://www.howtoforge.com/creating_djvu_documents_on_linux A simple guide] to create DJVU files (from JPEG and PDF) and to bind multiple DJVU files into one, is available at howtoforge. |
|||
====Workflow Planning==== |
|||
First, you need to think of this in terms of setting up a workflow. There are several steps, some of which require technical expertise and some which do not. In addition, the expertise required to set up and maintain the workflow is different from that required to make encoding decisions and check the quality of scans. |
|||
====Scanning==== |
|||
Some scanners can handle bound books but they cost a lot more money. However, if you can spare a copy, then you can take it apart and scan the pages on a flatbed scanner. Save the files in an uncompressed [[TIFF]] format because they will be processed further. Pages should be scanned in color because the DJVU compression software produces a better result that way. |
|||
If your original scans are not perfect, you may need to use software such as [http://www.i2s-bookscanner.com/en/products_software.asp Book Restorer] or [http://unpaper.berlios.de/ Unpaper] to clean them up. This is especially important when you are scanning old, rare books that have been damaged in some way, for instance stains on the pages. In addition, when a book is rare you cannot cut out the pages to do perfectly flat scans. This means that the scans will be curved but software can repair these curves. |
|||
Check this wiki for additional [[scanning]] advice. |
|||
====Encoding the Pages==== |
|||
Next, you need to process the individual page scans with various tools to [http://djvulibre.djvuzone.org/doc/index.html encode the pages]. Different encoding tools may be used for different pages depending on the presence of illustrations, photos, colored text, etc. Pages can be segmented into a black and white layer and a color layer so that different encoders can be used on each. In addition, if you have an OCR program for the script that the book is written in, you can run the black and white segment through it. DJVU readers are capable of using the OCR to do text searches and then highlighting the words in the actual scanned text image. |
|||
====Bundling and Postprocessing==== |
|||
After this you have various pieces which you need to bundle together into a multipage book file. Then, you may wish to further process the book to add text annotations, precompute thumbnail images of pages, etc. Perhaps the book is written in an archaic form of the language and you wish to annotate it with a glossary similar to what we do with Shakespeare's plays. |
|||
'''Testing''': |
|||
Don't forget to test your book thoroughly using [[Evince]] to make sure that there are no problems with using it on the OLPC. |
|||
====Tools==== |
|||
If you would rather have the scanning done by a [http://www.headway.co.uk/products/capture/lizardtech/lizardtech.htm company with expertise in the field], that is possible. Once the first pilot country is deployed, there will likely be other companies who can offer this service. But the tools needed are all [http://djvulibre.djvuzone.org/ open source] so you can also set up your own production line for scanning books. |
|||
===Articles and Papers=== |
|||
* [http://www.profsurv.com/archive.php?issue=48&article=671 this article from Professional Surveyor magazine] explains how the National Land Survey of Sweden went about converting their historical archive to DJVU format. |
|||
* [http://www.howtoforge.com/creating_djvu_documents_on_linux this article on howtoforge] details the entire workflow for producing DJVU books on Linux. It also includes some scripts useful in making the process run more smoothly. |
|||
== GIF == |
== GIF == |
||
| Line 7: | Line 68: | ||
More info can be found on [http://en.wikipedia.org/wiki/GIF Wikipedia]. |
More info can be found on [http://en.wikipedia.org/wiki/GIF Wikipedia]. |
||
== JPEG == |
== JPEG == |
||
[http://www.jpeg.org/jpeg2000/ JPEG 2000] is the most recent work of the JPEG standards committee. It is intended to improve upon several areas of the original JPEG standard such as lossy encoding, blocky appearance, and inefficient compression. |
[http://www.jpeg.org/jpeg2000/ JPEG 2000] is the most recent work of the JPEG standards committee. It is intended to improve upon several areas of the original JPEG standard such as lossy encoding, blocky appearance, and inefficient compression. |
||
Although this standard is intended to be implemented on a no-charge basis, the JPEG committee does allow for [http://www.jpeg.org/faq.phtml?action=show_answer&question_id=q3f042a68b1081 inclusion of patented or otherwise non-free technology] as per ITU and ISO policy on [http://en.wikipedia.org/wiki/Reasonable_and_non-discriminatory reasonable and non-descriminatory] terms. It is problematic or impossible for open source/free software to implement such standards. |
Although this standard is intended to be implemented on a no-charge basis, the JPEG committee does allow for [http://www.jpeg.org/faq.phtml?action=show_answer&question_id=q3f042a68b1081 inclusion of patented or otherwise non-free technology] as per ITU and ISO policy on [http://en.wikipedia.org/wiki/Reasonable_and_non-discriminatory reasonable and non-descriminatory] terms. It is problematic or impossible for open source/free software to implement such standards. |
||
| ⚫ | |||
[[Category:File formats]] |
[[Category:File formats]] |
||
| ⚫ | |||
Revision as of 07:13, 14 February 2007
A collection of file formats.
What is DJVU?
The main site for information on DJVU compression format for ebooks is here http://www.djvuzone.org/
Recently a good overview article was published on News Forge.
In a nutshell, DJVU was invented to solve this problem:
- Conventional web formats such as JPEG, GIF, and PNG produce prohibitively large image files at decent resolution. As a result, Web site content developers have been largely unable to leverage existing printed materials.
DJVU is intended to be used with scanned images of book pages, either black & white or full color. It then compresses those scanned pages to produce very highly compressed files.
Given that the target countries for the OLPC have poorly developed computing infrastructures, scanning of existing printed documents into DJVU format may be the fastest way of making a wide variety of educational material and Ebooks available to the kids.
DJVU is supported by the Evince reader which is being used by the OLPC project.
If you want to contribute to the DJVU project in any way, here is the site: http://djvulibre.djvuzone.org/
Why is DJVU important?
In regions where computers are scarce and there is little support for native scripts, DJVU allows existing paper books to be scanned and distributed as ebooks. Even handwritten books can be distributed this way. Tie this together with the OLPC chat application's support for SVG input and the GECKO support for displaying SVG graphics and it is conceivable to distribute a computer with no font support and NO TEXT AT ALL in its user interface. Icons would substitute for text in the UI and handwriting would be a primary mode of input. Note that the OLPC has a wider than normal touchpad that can be used as a handwriting input device.
Of course, this is a bootstrap scenario. Once the OLPC is deployed in this way, native language speakers will begin to work on fonts, and a keyboard layout to enable text use on the OLPC. This could take months or years to sort out, but in the meantime, the kids have an educational tool to use.
How Do We Produce DJVU Documents?
A simple guide to create DJVU files (from JPEG and PDF) and to bind multiple DJVU files into one, is available at howtoforge.
Workflow Planning
First, you need to think of this in terms of setting up a workflow. There are several steps, some of which require technical expertise and some which do not. In addition, the expertise required to set up and maintain the workflow is different from that required to make encoding decisions and check the quality of scans.
Scanning
Some scanners can handle bound books but they cost a lot more money. However, if you can spare a copy, then you can take it apart and scan the pages on a flatbed scanner. Save the files in an uncompressed TIFF format because they will be processed further. Pages should be scanned in color because the DJVU compression software produces a better result that way.
If your original scans are not perfect, you may need to use software such as Book Restorer or Unpaper to clean them up. This is especially important when you are scanning old, rare books that have been damaged in some way, for instance stains on the pages. In addition, when a book is rare you cannot cut out the pages to do perfectly flat scans. This means that the scans will be curved but software can repair these curves.
Check this wiki for additional scanning advice.
Encoding the Pages
Next, you need to process the individual page scans with various tools to encode the pages. Different encoding tools may be used for different pages depending on the presence of illustrations, photos, colored text, etc. Pages can be segmented into a black and white layer and a color layer so that different encoders can be used on each. In addition, if you have an OCR program for the script that the book is written in, you can run the black and white segment through it. DJVU readers are capable of using the OCR to do text searches and then highlighting the words in the actual scanned text image.
Bundling and Postprocessing
After this you have various pieces which you need to bundle together into a multipage book file. Then, you may wish to further process the book to add text annotations, precompute thumbnail images of pages, etc. Perhaps the book is written in an archaic form of the language and you wish to annotate it with a glossary similar to what we do with Shakespeare's plays.
Testing:
Don't forget to test your book thoroughly using Evince to make sure that there are no problems with using it on the OLPC.
Tools
If you would rather have the scanning done by a company with expertise in the field, that is possible. Once the first pilot country is deployed, there will likely be other companies who can offer this service. But the tools needed are all open source so you can also set up your own production line for scanning books.
Articles and Papers
- this article from Professional Surveyor magazine explains how the National Land Survey of Sweden went about converting their historical archive to DJVU format.
- this article on howtoforge details the entire workflow for producing DJVU books on Linux. It also includes some scripts useful in making the process run more smoothly.
GIF
GIF is old, proprietary but widely supported format. Today it's mostly surpassed by PNG - in all regards except one: support for simple animations. If you need animated icon or something like that - use GIF, by all means (all patents related to GIF are expired by now), but for anything else... there are PNG and JPEG.
There is a page on choosing image formats that will help you to understand the differences and how to know which format will be best for the intended use.
More info can be found on Wikipedia.
JPEG
JPEG 2000 is the most recent work of the JPEG standards committee. It is intended to improve upon several areas of the original JPEG standard such as lossy encoding, blocky appearance, and inefficient compression.
Although this standard is intended to be implemented on a no-charge basis, the JPEG committee does allow for inclusion of patented or otherwise non-free technology as per ITU and ISO policy on reasonable and non-descriminatory terms. It is problematic or impossible for open source/free software to implement such standards.
