JPEG is international image stadard. It's lossy standard good for photographs. You can reduce image size many times without apparent quality loss!
Since OLPC has limited storage it looks like a good to use JPEG whereever poossible. But in reality it's pretty dumb idea. When you are saving JPEG you are choosing between good quality, medium-sized files and poor quality, small files. You can not open and resave JPEG without quality loss (there are no loseless JPEG unless we are talking about JPEG 2000) - except in a few special cases. So it's not a good idea to use this format for intermediate images! Use PNG or JPEG 2000 instead. Also it's not a good idea to put artwork in JPEG files (you can always overlay artwork with usage of SVG).
There is a page on choosing image formats that will help you to understand the differences and how to know which format will be best for the intended use.
More info can be found on Wikipedia.