[Tilecache] Reduce the size of the PNGs (1/2)
Guillaume Lathoud
glathoud at yahoo.fr
Wed Jul 16 04:16:51 EDT 2008
Hello,
Here is the patch in attachment.
About PNGs: all additional options are fully optional, so if you don't use them you don't have a dependency.
About antialiasing: I entirely agree with you. Unfortunately, although we activated the bicubic antialiasing in Geoserver, it still provided us the exact same result (nearest neighbour), whose quality did not meet our standards for aeral view images.
Best regards,
Guillaume Lathoud
http://www.alpstein-tourismus.de/
--- En date de : Mar 15.7.08, Christopher Schmidt <crschmidt at metacarta.com> a écrit :
> De: Christopher Schmidt <crschmidt at metacarta.com>
> Objet: Re: [Tilecache] Reduce the size of the PNGs (1/2)
> À: "Guillaume Lathoud" <glathoud at yahoo.fr>
> Cc: tilecache at openlayers.org
> Date: Mardi 15 Juillet 2008, 15h42
> On Tue, Jul 15, 2008 at 01:22:48PM +0000, Guillaume Lathoud
> wrote:
> > Our main modifications:
> >
> > * Permit to significantly reduce the size of the PNG
> files using:
> > http://www.imagemagick.org
> > http://pmt.sourceforge.net/pngcrush/
>
> I worry about integrating more dependancies, but I assume
> you've taken
> this into account and made this all optional.
>
> > * Permit to do antialiasing on top of a WMSLayer.
>
> I'm not sure I understand this? Why would you do it?
> I'd assume that the
> WMS server -- which has the original data -- would always
> be in a better
> position to give you the best possible image for a given
> bounding box.
>
> > We appreciate a lot the work done by MetaCarta Labs on
> TileCache, and
> > we hope that our modifications can be useful to
> anyone.
>
> Please:
> * Checkout TileCache from SVN
> http://svn.tilecache.org/trunk/tilecache/
> * Apply your changes to that checkout
> * If you have any added/new files in the checkout, type
> 'svn add' on
> the filename
> * Type 'svn diff', and send the result as a single
> patch as an
> attachment to the list.
> * Fill out http://tilecache.org/ccla.txt and send the
> result via email
> to labs at metacarta.com.
>
> Once you do these, I'll take a look at it; looking
> through the existing
> code for diffs is too time consuming for me, but
> diffs/patches are much
> easier to read and more likely to be applied.
>
> Thanks in advance!
>
> Regards,
> --
> Christopher Schmidt
> MetaCarta
_____________________________________________________________________________
Envoyez avec Yahoo! Mail. Une boite mail plus intelligente http://mail.yahoo.fr
-------------- next part --------------
Index: tilecache.cfg
===================================================================
--- tilecache.cfg (Revision 353)
+++ tilecache.cfg (Arbeitskopie)
@@ -33,7 +33,7 @@
[cache]
type=Disk
-base=/tmp/tilecache
+base=C:\guillaume.lathoud\tmp-tilecache-2.04-alpstein
# [layername] -- all other sections are named layers
#
@@ -91,3 +91,56 @@
url=http://labs.metacarta.com/wms/vmap0
extension=png
+[alpsteinpc43_wege_sbg]
+type=WMSLayer
+layers=alpsteinpc43:wege_sbg
+url=http://localhost:9090/geoserver/wms?
+extension=png
+#
+# "spherical_mercator=true" will override bbox, srs, maxresolution, SRS, Units
+#bbox=-20037508.342789248,6224137.523059905,975514.8322669766,20037508.342789248
+#srs=EPSG:900913
+#maxresolution=156543.03390625
+#
+levels=20
+#tile_y_axis_inverted=True replaced with tms_type=google
+tms_type=google
+spherical_mercator=true
+antialias=true
+palette="f:/guillaume.lathoud/Programme/ImageMagick-6.4.1-Q8/convert.exe"
+pngcrush="f:/guillaume.lathoud/Programme/pngcrush-1.6.4-win32/pngcrush.exe"
+
+[tropical_beach]
+type=ImageLayer
+file=e:/guillaume.lathoud/DATA/co.pics/tropical-beach.jpg
+filebounds=0,-1016,728,0
+scaling=antialias
+maxresolution=16
+levels=10
+tms_type=google
+fillcolor=yellow
+extension=jpg
+jpegquality=5
+
+[alpsteinpc43_wege_sbg_jpg]
+type=WMSLayer
+layers=alpsteinpc43:wege_sbg
+url=http://localhost:9090/geoserver/wms?
+extension=jpg
+levels=20
+tms_type=google
+spherical_mercator=true
+jpegquality=50
+
+[tropical_beach_png]
+type=ImageLayer
+file=e:/guillaume.lathoud/DATA/co.pics/tropical-beach.jpg
+filebounds=0,-1016,728,0
+scaling=antialias
+maxresolution=16
+levels=10
+tms_type=google
+fillcolor=yellow
+extension=png
+palette="f:/guillaume.lathoud/Programme/ImageMagick-6.4.1-Q8/convert.exe"
+pngcrush="f:/guillaume.lathoud/Programme/pngcrush-1.6.4-win32/pngcrush.exe"
Index: TileCache/Layers/Image.py
===================================================================
--- TileCache/Layers/Image.py (Revision 353)
+++ TileCache/Layers/Image.py (Arbeitskopie)
@@ -14,7 +14,9 @@
] + MetaLayer.config_properties
def __init__ (self, name, file = None, filebounds = "-180,-90,180,90",
- transparency = False, scaling = "nearest", **kwargs):
+ transparency = False, scaling = "nearest",
+ fillcolor=None, # modified_imagelayerfillcolor
+ **kwargs):
import PIL.Image as PILImage
MetaLayer.__init__(self, name, **kwargs)
@@ -30,6 +32,7 @@
if isinstance(transparency, str):
transparency = transparency.lower() in ("true", "yes", "1")
self.transparency = transparency
+ self.fillcolor = fillcolor # modified_imagelayerfillcolor
def renderTile(self, tile):
import PIL.Image as PILImage
@@ -50,7 +53,13 @@
scaling = PILImage.NEAREST
crop_size = (max_x-min_x, max_y-min_y)
- if min(min_x, min_y, max_x, max_y) < 0:
+
+ # modified_imagelayerfillbugfix begin
+ fill_needed = ( min(min_x, min_y, max_x, max_y ) < 0 or
+ max( min_x, max_x ) > abs( self.filebounds[ 2 ] - self.filebounds[ 0 ] ) or
+ max( min_y, max_y ) > abs( self.filebounds[ 3 ] - self.filebounds[ 1 ] ) )
+ if fill_needed:
+ # modified_imagelayerfillbugfix end
if self.transparency and self.image.mode in ("L", "RGB"):
self.image.putalpha(PILImage.new("L", self.image_size, 255));
sub = self.image.transform(crop_size, PILImage.EXTENT, (min_x, min_y, max_x, max_y))
@@ -60,6 +69,44 @@
scaling = PILImage.BICUBIC
sub = sub.resize(size, scaling)
+ # modified_imagelayerfillcolor begin
+
+ if fill_needed and self.fillcolor:
+
+ sub2 = PILImage.new( sub.mode, sub.size, self.fillcolor )
+
+ # Convert self.filebounds into the pixel coordinate system of the tile
+
+ a_res = self.resolutions[ tile.z ]
+
+ fill_x0, fill_y0 = ( -min_x / a_res, -min_y / a_res, )
+ fill_x1 = ( self.filebounds[ 2 ] - bounds[ 0 ] ) / self.image_res[ 0 ] / a_res
+ fill_y1 = ( bounds[ 3 ] - self.filebounds[ 1 ] ) / self.image_res[ 1 ] / a_res
+
+ # Determine the relevant part of sub
+
+ rel_x0, rel_x1 = [ max( 0, min( size[ 0 ], x ) ) for x in ( fill_x0, fill_x1 ) ]
+ rel_y0, rel_y1 = [ max( 0, min( size[ 1 ], y ) ) for y in ( fill_y0, fill_y1 ) ]
+
+ import math
+
+ rel_x0 = math.floor( min( rel_x0, rel_x1 ) )
+ rel_x1 = math.ceil( max( rel_x0, rel_x1 ) )
+ rel_y0 = math.floor( min( rel_y0, rel_y1 ) )
+ rel_y1 = math.ceil( max( rel_y0, rel_y1 ) )
+
+ # Copy the relevant part of sub into sub2
+
+ rel_bbox = ( rel_x0, rel_y0, rel_x1, rel_y1 )
+
+ sub2.paste( sub.crop( rel_bbox ), rel_bbox )
+
+ # Update
+
+ sub = sub2
+
+ # modified_imagelayerfillcolor end
+
buffer = StringIO.StringIO()
if self.image.info.has_key('transparency'):
sub.save(buffer, self.extension, transparency=self.image.info['transparency'])
Index: TileCache/Layers/WMS.py
===================================================================
--- TileCache/Layers/WMS.py (Revision 353)
+++ TileCache/Layers/WMS.py (Arbeitskopie)
@@ -25,6 +25,8 @@
"srs": self.srs,
"format": self.format(),
"layers": self.layers,
+ 'antialias': self.antialias, # modified_antialias
+ 'extension': self.extension, # modified_antialias
}, self.user, self.password)
tile.data, response = wms.fetch()
return tile.data
Index: TileCache/Layer.py
===================================================================
--- TileCache/Layer.py (Revision 353)
+++ TileCache/Layer.py (Arbeitskopie)
@@ -99,7 +99,12 @@
"cache", "debug", "description",
"watermarkimage", "watermarkopacity",
"extent_type", "tms_type", "units", "mime_type",
- "spherical_mercator", "metadata")
+ "spherical_mercator", "metadata", # modified_antialias
+ "antialias", # modified_antialias
+ "palette", # modified_palette
+ "pngcrush", # modified_pngcrush
+ "jpegquality", # modified_jpegquality
+ )
config_properties = [
{'name':'spherical_mercator', 'description':'Layer is in spherical mercator. (Overrides bbox, maxresolution, SRS, Units)', 'type': 'boolean'},
@@ -117,9 +122,14 @@
extension = "png", mime_type = None, cache = None, debug = True,
watermarkimage = None, watermarkopacity = 0.2,
spherical_mercator = False,
- extent_type = "strict", units = "degrees", tms_type = "", **kwargs ):
+ extent_type = "strict", units = "degrees", tms_type = "", # modified_antialias
+ antialias = False, # modified_antialias
+ palette = False, # modified_palette
+ pngcrush = False, # modified_pngcrush
+ jpegquality = False, # modified_jpegquality
+ **kwargs ):
"""Take in parameters, usually from a config file, and create a Layer.
-
+
>>> l = Layer("Name", bbox="-12,17,22,36", debug="no")
>>> l.bbox
[-12.0, 17.0, 22.0, 36.0]
@@ -194,7 +204,39 @@
self.watermarkimage = watermarkimage
self.watermarkopacity = float(watermarkopacity)
+
+ # modified_antialias begin
+ if isinstance( antialias, str ):
+ antialias = antialias.lower() in ("true","yes","on","1")
+ self.antialias = antialias
+ # modified_antialias end
+
+ # modified_palette begin
+ self.palette = False
+ if isinstance( palette, str ):
+ if len( palette ) > 1:
+ if ( palette[ 0 ] == palette[ -1 ] ) and ( palette[ 0 ] in ( '"', "'", ) ):
+ palette = palette[ 1:-1 ]
+ self.palette = palette
+ # modified_palette end
+ # modified_pngcrush begin
+ self.pngcrush = False
+ if isinstance( pngcrush, str ):
+ if len( pngcrush ) > 1:
+ if ( pngcrush[ 0 ] == pngcrush[ -1 ] ) and ( pngcrush[ 0 ] in ( '"', "'", ) ):
+ pngcrush = pngcrush[ 1:-1 ]
+ self.pngcrush = pngcrush
+ # modified_pngcrush end
+
+ # modified_jpegquality begin
+ self.jpegquality = False
+ if isinstance( jpegquality, str ):
+ jpegquality = int( jpegquality )
+ if isinstance( jpegquality, int ):
+ self.jpegquality = jpegquality
+ # modified_jpegquality end
+
self.metadata = {}
prefix_len = len("metadata_")
@@ -348,8 +390,9 @@
pass
def render (self, tile):
+
return self.renderTile(tile)
-
+
class MetaLayer (Layer):
__slots__ = ('metaTile', 'metaSize', 'metaBuffer')
Index: TileCache/Client.py
===================================================================
--- TileCache/Client.py (Revision 353)
+++ TileCache/Client.py (Arbeitskopie)
@@ -4,14 +4,19 @@
import sys, urllib, urllib2, time, os, math
import httplib
+import copy # modified_antialias
# setting this to True will exchange more useful error messages
# for privacy, hiding URLs and error messages.
HIDE_ALL = False
class WMS (object):
- fields = ("bbox", "srs", "width", "height", "format", "layers", "styles")
- defaultParams = {'version': '1.1.1', 'request': 'GetMap', 'service': 'WMS'}
+ fields = ("bbox", "srs", "width", "height", "format", "layers", "styles", # modified_antialias
+ 'antialias', 'extension', # modified_antialias
+ )
+ defaultParams = {'version': '1.1.1', 'request': 'GetMap', 'service': 'WMS', # modified_antialias
+ 'antialias': False, 'extension':'png', # modified_antialias
+ }
__slots__ = ("base", "params", "client", "data", "response")
def __init__ (self, base, params, user=None, password=None):
@@ -42,9 +47,29 @@
self.params[key] = ""
def url (self):
- return self.base + urllib.urlencode(self.params)
+
+ # modified_antialias begin
+
+ params_copy = copy.deepcopy( self.params )
+
+ if self.params[ 'antialias' ]:
+ params_copy[ 'width' ] *= 4
+ params_copy[ 'height' ] *= 4
+
+ for a_str in ( 'antialias',
+ 'palette', # modified_palette
+ 'pngcrush', # modified_pngcrush
+ 'jpegquality', # modified_jpegquality
+ ):
+ while a_str in params_copy:
+ del params_copy[ a_str ]
+
+ return self.base + urllib.urlencode( params_copy )
+
+ # modified_antialias end
def fetch (self):
+
urlrequest = urllib2.Request(self.url())
# urlrequest.add_header("User-Agent",
# "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)" )
@@ -64,6 +89,34 @@
raise Exception("Did not get image data back. \nURL: %s\nContent-Type Header: %s\nResponse: \n%s" % (self.url(), ctype, data))
except httplib.BadStatusLine:
response = None # try again
+
+ # modified_antialias begin
+
+ if self.params[ 'antialias' ]:
+
+ import StringIO, Image
+
+ try:
+ image = Image.open( StringIO.StringIO( data ) )
+ newimage = image.resize( ( self.params[ 'width' ], self.params[ 'height' ] ), Image.ANTIALIAS )
+ antialias_out = StringIO.StringIO()
+
+ if newimage.info.has_key('transparency'):
+ newimage.save(antialias_out, self.params[ 'extension' ],
+ transparency=image.info['transparency'])
+ else:
+ newimage.save(antialias_out, self.params[ 'extension' ])
+
+ # Replace the (e.g. 1024x1024) image data with the (e.g. 256x256) image data
+ data = antialias_out.getvalue()
+
+ except StandardError, e:
+ import tempfile, os
+ open( os.path.join( 'c:/tmp', 'antialias_error.txt' ), 'wt' ).write( str( e ) )
+ raise
+
+ # modified_antialias end
+
return data, response
def setBBox (self, box):
Index: TileCache/Service.py
===================================================================
--- TileCache/Service.py (Revision 353)
+++ TileCache/Service.py (Arbeitskopie)
@@ -136,8 +136,165 @@
if not force: image = self.cache.get(tile)
if not image:
data = layer.render(tile)
- if (data): image = self.cache.set(tile, data)
- else: raise Exception("Zero length data returned from layer.")
+
+ if (data):
+
+ # modified_palette begin
+
+ if layer.palette:
+
+ import PIL.Image as PILImage
+ import StringIO
+
+ pilimage = PILImage.open( StringIO.StringIO( data ) )
+
+ if pilimage.format.lower() in ( 'png', ):
+
+ pilcolors = pilimage.getcolors()
+
+ # Sometimes (PIL 1.1.6) getcolors() returns None
+ if pilcolors == None:
+ pilcolors = []
+ pilpixels = pilimage.load()
+ if pilpixels != None:
+ for a in range( pilimage.size[ 0 ] ):
+ for b in range( pilimage.size[ 1 ] ):
+ pilc = pilpixels[ a, b ]
+ if pilc not in pilcolors:
+ pilcolors.append( pilc )
+ if len( pilcolors ) > 1:
+ break
+ if len( pilcolors ) > 1:
+ break
+
+ if len( pilcolors ) < 2:
+
+ # Work around a bug in Image Magick 6.4.1-Q8:
+ # "convert.exe <src.png> -colors 256 png8:<dest.png>"
+ # ...when all pixels in src.png have the same color.
+
+ pilimage = PILImage.open( StringIO.StringIO( data ) )
+ newpilimage = pilimage.convert( 'P' )
+
+ palette_out = StringIO.StringIO()
+ newpilimage.save( palette_out, layer.extension )
+
+ data = palette_out.getvalue()
+
+ else: # Call Image Magick's convert on non-empty images
+
+ import tempfile, os, datetime, subprocess
+
+ tmp_convert_exefullpath = layer.palette
+ if tmp_convert_exefullpath[ 0 ] == tmp_convert_exefullpath[ -1 ]:
+ if tmp_convert_exefullpath[ 0 ] in ( '"', "'", ):
+ tmp_convert_exefullpath = tmp_convert_exefullpath[ 1:-1 ]
+
+ tmp_orig_imagefile = tempfile.NamedTemporaryFile( suffix = '.' + layer.extension )
+ tmp_orig_imagefilename = tmp_orig_imagefile.name
+ tmp_orig_imagefile.close()
+
+ open( tmp_orig_imagefilename, 'wb' ).write( data )
+
+ tmp_dest_imagefile = tempfile.NamedTemporaryFile( suffix = '.' + layer.extension )
+ tmp_dest_imagefilename = tmp_dest_imagefile.name
+ tmp_dest_imagefile.close()
+
+ tmp_format_str = 'png8:' if ( layer.extension.lower() == 'png' ) else ''
+
+ tmp_pipe = subprocess.Popen( tmp_convert_exefullpath +
+ " %s -colors 256 %s%s" % ( tmp_orig_imagefilename,
+ tmp_format_str,
+ tmp_dest_imagefilename, ),
+ shell=True,
+ stdin=subprocess.PIPE,
+ stdout=subprocess.PIPE,
+ stderr=subprocess.PIPE )
+
+ tmp_stdout = tmp_pipe.stdout.read()
+ tmp_stderr = tmp_pipe.stderr.read()
+ tmp_exit_code = tmp_pipe.wait()
+
+ assert tmp_exit_code == 0
+
+ data = open( tmp_dest_imagefilename, 'rb' ).read()
+
+ os.remove( tmp_orig_imagefilename )
+ os.remove( tmp_dest_imagefilename )
+
+ # modified_palette end
+
+ # modified_pngcrush begin
+
+ if layer.pngcrush:
+
+ import StringIO
+ import PIL.Image as PILImage
+
+ pilimage = PILImage.open( StringIO.StringIO( data ) )
+
+ if pilimage.format.lower() in ( 'png', ):
+
+ import tempfile, os, datetime, subprocess
+
+ tmp_pngcrush_exefullpath = layer.pngcrush
+ if tmp_pngcrush_exefullpath[ 0 ] == tmp_pngcrush_exefullpath[ -1 ]:
+ if tmp_pngcrush_exefullpath[ 0 ] in ( '"', "'", ):
+ tmp_pngcrush_exefullpath = tmp_pngcrush_exefullpath[ 1:-1 ]
+
+ tmp_orig_imagefile = tempfile.NamedTemporaryFile( suffix = '.' + layer.extension )
+ tmp_orig_imagefilename = tmp_orig_imagefile.name
+ tmp_orig_imagefile.close()
+
+ open( tmp_orig_imagefilename, 'wb' ).write( data )
+
+ tmp_dest_imagefile = tempfile.NamedTemporaryFile( suffix = '.' + layer.extension )
+ tmp_dest_imagefilename = tmp_dest_imagefile.name
+ tmp_dest_imagefile.close()
+
+ tmp_pipe = subprocess.Popen( tmp_pngcrush_exefullpath +
+ " %s %s" % ( tmp_orig_imagefilename,
+ tmp_dest_imagefilename, ),
+ shell=True,
+ stdin=subprocess.PIPE,
+ stdout=subprocess.PIPE,
+ stderr=subprocess.PIPE )
+
+ tmp_stdout = tmp_pipe.stdout.read()
+ tmp_stderr = tmp_pipe.stderr.read()
+ tmp_exit_code = tmp_pipe.wait()
+
+ assert tmp_exit_code == 0
+
+ data = open( tmp_dest_imagefilename, 'rb' ).read()
+
+ os.remove( tmp_orig_imagefilename )
+ os.remove( tmp_dest_imagefilename )
+
+ # modified_pngcrush end
+
+ # modified_jpegquality begin
+
+ if layer.jpegquality:
+
+ import StringIO
+ import PIL.Image as PILImage
+
+ pilimage = PILImage.open( StringIO.StringIO( data ) )
+
+ if pilimage.format.lower() in ( 'jpg', 'jpeg', ):
+ jpegquality_out = StringIO.StringIO()
+ jpegquality_opt = { 'quality': layer.jpegquality, }
+ pilimage.save( jpegquality_out, layer.extension,
+ **jpegquality_opt )
+ data = jpegquality_out.getvalue()
+
+ # modified_jpegquality end
+
+ image = self.cache.set(tile, data)
+ else:
+ raise Exception("Zero length data returned from layer.")
+
if layer.debug:
sys.stderr.write(
"Cache miss: %s, Tile: x: %s, y: %s, z: %s, time: %s\n" % (
Index: tilecache.cgi
===================================================================
--- tilecache.cgi (Revision 353)
+++ tilecache.cgi (Arbeitskopie)
@@ -1,5 +1,8 @@
+#!F:/Python25/python.exe -u
#!/usr/bin/env python
+import cgitb; cgitb.enable() # great for debugging through a web browser xxx debug
+
from TileCache import Service, cgiHandler, cfgfiles
if __name__ == '__main__':
More information about the Tilecache
mailing list