[Tilecache] JPG encoding problem and solution

[Marco Nijdam]

Dear list,

This may be a very specific problem and solution, but I thought I'd
share it with you just in case anybody runs into the same problem.

In our maps we have red (#ff0000) shields behind the motorway labels.
For some reason, when I set tilecache to use metatiling, and have it
save the split tiles in JPG, the resulting tiles have a much darker
colored red shields, with some JPG "artifacts" visible.

The attached patch (actually addition) to Tilecache solves this
problem by using ImageMagick to split the metatile into tiles, using
a different JPEG sampling factor that prevents the problem.


Detailed analysis:

In our analysis it turned out that saving the tiles as PNG, and converting
them with ImageMagick or NetPBM to JPG could solve the problem, but only
if we set the JPEG sampling factor to 1x1 (I think default is 2x1).
But the Python PIL does not have any option to change the sampling factor
when saving in JPEG.

For the tiles containing also arial photography, using PNG as intermediate
format resulted in much overhead.

1. Saving a photo in PNG turns out to be very inefficient. Encoding can
take several seconds. And consider that both the WMS and the tilecache
need to save it.

2. We would need a separate batch proces to post-proces the tiles to
convert them to JPEG.

Instead, the attached classes do not require an extra post-process step.
Compared to the first solution (saving as PNG, postprocess to JPEG),
it made the tile-generation process much stabler, and improved speed
(3 to 4 times faster with arial photography).

First step is to request the meta-tile not in PNG but in TIFF. While
a TIFF file is much larger (more network transfer time), it is much
quicker to save and load. It turned out to be a big win for us.

The Layers/WMSChecked class accepts a "metamime_type" config parameter
in the config file, to use a different mime-type for requesting the
meta-tile, than used to serve the tiles.

The Layers/WMSChecked class sends the complete meta-tile to a pipe to an
ImageMagick command. The ImageMagick command splits the metatile in tiles,
and saves them as JPEG with the different sampling factor.
The command is configurable in the config file as well, since you may
need to shave off a some border, or use different options.

ImageMagick can't save the tiles with the required directory
structure and filename as used by Tilecache.
The Layers/WMSChecked class therefore moves (renames) the files on disk
to the correct directory with the correct filename, using the
Caches/RenameDisk class. It tries to prevent loading and resaving the
tile files in Python, to achieve best performance.


The classes also try to solve a completely different problem.
Occasionally the arial photography (cascaded) WMS server fails,
resulting in tiles with only a background color.

The Layers/WMSChecked class contains a retry mechanism to retry
getting a metatile up to three times if the image is invalid, or
contains some (background) color indicating the image was not correctly
generated (since with arial photographs, we normally do not expect to
see the background color).

Kind regards,
--
-- Marco Nijdam
--

-------------- next part --------------
A non-text attachment was scrubbed...
Name: tilecache-colorcheck-pipeconvert.patch
Type: application/octet-stream
Size: 7963 bytes
Desc: not available
Url : http://lists.osgeo.org/pipermail/tilecache/attachments/20100806/828f68f4/tilecache-colorcheck-pipeconvert.obj