[gdal-dev] Cache when dealing with several processes and COG

[Guy Doulberg]

Hi guys,

I am working on a tileserver use case on top of cogs.

I  want to find a cache mechanism to my architecture.

The tile-server architecture is several python processes(gunicorn) running
on several VMs.

I understand how GDAL caches the curl blocks or the raster using
intra-process caching, but I can't use this cache in the other
processes/vms.

I was thinking maybe to use some kind of http proxy server that will cache
the bytes content retrieved from the http server holding the cogs (Azure
blob storage)

There is some data that can be reused(therefore cached) across all tile
requests for example:
1. The file size (HEAD)
2. The first header block
3. The other header blocks
4. maybe in some cases the image blocks themselves (in case you take the
same blocks all the time but change something in the presentation layer)

did any of you tried this architecture or used a different way to cache
across servers?  maybe there is a way to share GDAL_CACHE across process
that I missed?

Thanks,
Guy
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20180703/cd8b1647/attachment.html>