[Tilecache] http 304 Not Modifies responses ?

[Christopher Schmidt]

On Wed, Jan 02, 2008 at 04:04:29PM +0100, Attila Csipa wrote:
> On Wednesday 02 January 2008 15:01:45 Christopher Schmidt wrote:
> > On Wed, Jan 02, 2008 at 02:00:59PM +0100, Attila Csipa wrote:
> > > I've been looking at ways of improving the speed of the web mapping
> > > projects I'm in and that's how I ran accross TileCache. While it works
> > > pretty smoothly after a quick glance at the source it occured to me that
> > > it does not support any client side caching (with the help of f.e. 304
> > > Not Modified responses).
> >
> > In general, I simply use mod_expires on the Apache side to do this for
> > me: I set the expiration time to 1 month, and it stays in cache for as
> > long as neccesary.
> 
> That's very different from I'm doing. I have datasets that change rarely, but 
> when they do, it's critical that the data show by the clients is up-to-date. 
> In my experience, if you set an expires header, the browser won't bother 
> checking back but rather say the image in the browser cache is still A-OK if 
> you are within the given period. 304 headers should be somewhere inbetween, 
> slower than expires based cache as they need to check what the server has, 
> but faster than regular requests as no actual data needs to be transferred on 
> a cache hit.

Yep. I understand the motiations for both, but my data (seldom|never)
changes, so I just use expires headers.

> > > I made a patch that does the job (inserts Last-Modified and ETag
> > > headers based on the date and content of the generated tile and
> > > responds with a 304 if the header comes back for a reload of a tile it
> > > already has),
> >
> > I'm interested as to how you're doing this :) I've sat down to implement
> > it a couple times, and never been happy with it -- since it wasn't
> > designed in from the beginning, tile metainformation isn't really
> > returned from the cache, and I couldn't figure out how to make it so
> > without breaking things.
> 
> Well, very simple, I let tilecache do whatever it needs to do to generate the 
> tile, and when it's already present in the disk cache, I use the timestamp of 
> the image file in the cache for Last-Modified, 

Right, this is what I'm trying to figure out. To me, it looks like
TileCache.Service does:

if not force: image = self.cache.get(tile)

since self.cache.get(tile) returns data, and not something like a file
pointer, I don't understand where the datestamp is coming from.

I have a feeling that the answer is either: "I've changed the return
from the diskcache 'get' method to include metadata" -- which breaks the
Cache class API, such as it exists -- or "I've changed this single
function call to also return something different which I use to look at
the file directly" -- which is brekaing an abstraction barrier.

The former is not an unsolvable problem, but since users acn write their
own cache classes, it's something we need to be careful with.

The latter is a problem because disk caching is *not* the only way tiles
can be stored -- there are existing users of the memcached class, for
example, and they don't have a file on disk to get the information from. 

> and an MD5 of the image file 
> as an ETag (a bit expensive, but see motivation below). Code-wise it's very 
> similar how you set Content-Type. When I refresh (=delete/expire) the image 
> tiles in the disk cache the clients pick up the new tiles automatically. 

Right, that's essentially what I expected, but I as curious about the
way you were getting the datestamp out.

> As a future possibility I was thinking of using the etag as a base for 
> transparently supporting/caching/replacing common (pure black/white) tiles 
> without actually loading them (there was talk about this earlier, although 
> I'm not sure if here or on the mapserver-dev list).

I think that we should talk about this is you're interested before you
implement it: I have some vague implementation notions that I'd like to
bounce about before we do too much coding on it.

> 
> > I definitely think that it is worth putting it back into trunk so long
> > as it isn't completely cockneyed -- I'd love to get a look at it
> > regardless.
> 
> OK, I'll finish it up (as I only did it for the fastcgi handler, and would 
> only be right if both cgi and fastcgi versions knew about this feature and 
> acted the same way) and post it here.

mod_python should get touched too -- if you can't test CGI/mod_python
though, I'll be glad to take over and update those if you offer the rest
of the patch.

Regards,
-- 
Christopher Schmidt
MetaCarta