[Tilecache] http 304 Not Modifies responses ?

[Attila Csipa]

On Wednesday 02 January 2008 15:01:45 Christopher Schmidt wrote:
> On Wed, Jan 02, 2008 at 02:00:59PM +0100, Attila Csipa wrote:
> > I've been looking at ways of improving the speed of the web mapping
> > projects I'm in and that's how I ran accross TileCache. While it works
> > pretty smoothly after a quick glance at the source it occured to me that
> > it does not support any client side caching (with the help of f.e. 304
> > Not Modified responses).
>
> In general, I simply use mod_expires on the Apache side to do this for
> me: I set the expiration time to 1 month, and it stays in cache for as
> long as neccesary.

That's very different from I'm doing. I have datasets that change rarely, but 
when they do, it's critical that the data show by the clients is up-to-date. 
In my experience, if you set an expires header, the browser won't bother 
checking back but rather say the image in the browser cache is still A-OK if 
you are within the given period. 304 headers should be somewhere inbetween, 
slower than expires based cache as they need to check what the server has, 
but faster than regular requests as no actual data needs to be transferred on 
a cache hit.

> > I made a patch that does the job (inserts Last-Modified and ETag
> > headers based on the date and content of the generated tile and
> > responds with a 304 if the header comes back for a reload of a tile it
> > already has),
>
> I'm interested as to how you're doing this :) I've sat down to implement
> it a couple times, and never been happy with it -- since it wasn't
> designed in from the beginning, tile metainformation isn't really
> returned from the cache, and I couldn't figure out how to make it so
> without breaking things.

Well, very simple, I let tilecache do whatever it needs to do to generate the 
tile, and when it's already present in the disk cache, I use the timestamp of 
the image file in the cache for Last-Modified, and an MD5 of the image file 
as an ETag (a bit expensive, but see motivation below). Code-wise it's very 
similar how you set Content-Type. When I refresh (=delete/expire) the image 
tiles in the disk cache the clients pick up the new tiles automatically. 

Example:

Status=OK - 200
Status=200 OK
Content-Type=image/jpeg
Last-Modified=Sun, 30 Dec 2007 20:40:11 GMT
Etag="c05e0072b585a71167cee39eb48a526d"
Content-Length=25265
Date=Wed, 02 Jan 2008 14:50:52 GMT
Server=lighttpd/1.4.18

And on the following request for the same tile the browser should give:

Host=localhost
User-Agent=Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.8.1.11) 
Gecko/20071204 Ubuntu/7.10 (gutsy) Firefox/2.0.0.11
Accept=text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
Accept-Language=en-us,en;q=0.5
Accept-Encoding=gzip,deflate
Accept-Charset=ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive=300
Connection=keep-alive
If-Modified-Since=Sun, 30 Dec 2007 20:40:11 GMT
If-None-Match="c05e0072b585a71167cee39eb48a526d"
Cache-Control=max-age=0

And the reply to this is:

Status=Not Modified - 304
Status=304 Tile Not Modified
Content-Length=0
Date=Wed, 02 Jan 2008 14:50:55 GMT
Server=lighttpd/1.4.18

... and no data (25K in the example) need be transferred on the second try IF 
the image the browser has is up-to-date

As a future possibility I was thinking of using the etag as a base for 
transparently supporting/caching/replacing common (pure black/white) tiles 
without actually loading them (there was talk about this earlier, although 
I'm not sure if here or on the mapserver-dev list).

> I definitely think that it is worth putting it back into trunk so long
> as it isn't completely cockneyed -- I'd love to get a look at it
> regardless.

OK, I'll finish it up (as I only did it for the fastcgi handler, and would 
only be right if both cgi and fastcgi versions knew about this feature and 
acted the same way) and post it here.

> If-Newer-Than doesn't sound like the right header from memory: I believe
> "If-Modified-Since" is the one that should get sent in that case. Since

Right, I was writing from (my apparently not flawless :) ) memory, 
If-Modified-Since (and its sibling, If-None-Match) were the ones I wanted to 
refer to.