[MapProxy] FW: Which cache is better

Sat Jun 10 10:35:04 PDT 2017

Some filesystems don't like thousands of small files. Even backing these up or syncing them might be very painful.

-- 
Von einem iPhone gesendet und wird daher Fehler enthalten…

Am 10.06.2017 um 10:14 schrieb Trond Michelsen <trondmm-mapproxy at crusaders.no>:

> What is the advantage of an sqlite cache over a simple file cache?
> 
> On Fri, Jun 09, 2017 at 09:18:33AM -0400, Travis Kirstine wrote:
>> David,
>> 
>> According to the doc you could try using the ArcGIS tiles as a source which
>> may increase your caching performance.
>> 
>> https://mapproxy.org/docs/nightly/sources.html#tiles
>> see arcgiscache_path
>> 
>> 
>> We are using riak in a 5 node cluster N=3 with a leveldb backend, we found
>> that the bitcask memory footprint was unrealistic.  We went with the
>> default 64 partitions which maybe should have been increased to 128. We've
>> found that debugging issues and simply accessing the data a little more
>> difficult than expected and required scripting to perform basic queries /
>> tasks.  That being said the cluster is performing very well on both read /
>> writes
>> 
>> We did also briefly test couchdb and found that it worked well, I would
>> have preferred it over riak but outside factors forced our decision.
>> 
>> If I was you I would still try a sqlite based cache first to see if it
>> meets your need.
>> 
>> 
>> 
>> 
>> On 8 June 2017 at 17:53, Andrew, David (DPFEM) <
>> David.Andrew at dpfem.tas.gov.au> wrote:
>> 
>>> Hi Travis,
>>> 
>>> I need to supply a selection of base maps in a 24/7 configuration from the
>>> same datacentre as the applications servers.
>>> 
>>> The application reading htem is an olderversion of geoserver, which cannot
>>> directly connect to arcgis.
>>> 
>>> The source is
>>> 
>>> http://services.thelist.tas.gov.au/arcgis/rest/services/
>>> Basemaps/Orthophoto/ImageServer
>>> 
>>> 
>>> 
>>> yes to ARCGIS as a WMS source for MapProxy.
>>> 
>>> 
>>> 
>>> *David Andrew*
>>> 
>>> IT Services – Enterprise Systems
>>> 
>>> Business and Executive Services
>>> 
>>> 
>>> 
>>> Department of Police, Fire and Emergency Management
>>> 
>>> Cnr Argyle and Melville Streets, Hobart TAS 7000
>>> 
>>> David.Andrew at police.tas.gov.au| www.dpfem.tas.gov.au
>>> 
>>> 
>>> 
>>> *From:* Travis Kirstine [mailto:traviskirstine at gmail.com]
>>> *Sent:* 08 June 2017 22:26
>>> *To:* Andrew, David (DPFEM) <David.Andrew at dpfem.tas.gov.au>
>>> *Cc:* mapproxy at lists.osgeo.org
>>> *Subject:* Re: [MapProxy] FW: Which cache is better
>>> 
>>> 
>>> 
>>> David,
>>> 
>>> 
>>> 
>>> If your primary concern is interoperability and portability I would look
>>> at mbtiles or geopackage.  Both cache types store the tiles in a single
>>> file which could be used in a few platforms and easily backed up.  The only
>>> issue with these caches (sqlite) it a tendency to lock up during heavy
>>> concurrent writes, mapproxy has implemented bulk writing to help address
>>> this issue for mbtiles (not sure about geopackage).  If your users are
>>> using a fully seeded cache this won't be a big issue.
>>> 
>>> 
>>> 
>>> When you indicate that it takes 2 weeks to download the cache from ArcGIS
>>> do you me that you are using ArcGIS as a WMS source for MapProxy?
>>> 
>>> 
>>> 
>>> Regards
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> On 8 June 2017 at 00:29, Andrew, David (DPFEM) <
>>> David.Andrew at dpfem.tas.gov.au> wrote:
>>> 
>>> Hi all,
>>> 
>>> Totally a newbie regarding all things GIS
>>> 
>>> I’m trying to establish which cache type is better for a photo cache.
>>> 
>>> 
>>> 
>>> I’m looking to cache the orthographic photo maps for the state of Tasmania
>>> Australia, to a zoom level of 18
>>> 
>>> This will then be used by the new emergency services application to have
>>> an offline pre seeded copy.
>>> 
>>> Around 30 clients, up to 50 in a peak load like New Years eve.
>>> 
>>> 
>>> 
>>> I cannot seem to find anything that defines which cache style would be
>>> better or worse.
>>> 
>>> CouchDB seems interesting and replicatable
>>> 
>>> Riak seem like very fast fetch
>>> 
>>> Geopackage looks like we can copy the flat file once we get it.
>>> 
>>> 
>>> 
>>> If anyone has any numbers around which is better that would be great.
>>> 
>>> At the moment to download the cache takes a week or 2 for levels 17-18
>>> from an arcgis server(10.31 newer than the cache option allow)
>>> 
>>> 
>>> 
>>> Thanks in advance
>>> 
>>> 
>>> 
>>> *David Andrew*
>>> 
>>> IT Services – Enterprise Systems
>>> 
>>> Business and Executive Services
>>> 
>>> 
>>> 
>>> Department of Police, Fire and Emergency Management
>>> 
>>> Cnr Argyle and Melville Streets, Hobart TAS 7000
>>> 
>>> David.Andrew at police.tas.gov.au| www.dpfem.tas.gov.au
>>> 
>>> 
>>> 
>>> 
>>> ------------------------------
>>> 
>>> 
>>> CONFIDENTIALITY NOTICE AND DISCLAIMER
>>> The information in this transmission may be confidential and/or protected
>>> by legal professional privilege, and is intended only for the person or
>>> persons to whom it is addressed. If you are not such a person, you are
>>> warned that any disclosure, copying or dissemination of the information is
>>> unauthorised. If you have received the transmission in error, please
>>> immediately contact this office by telephone, fax or email, to inform us of
>>> the error and to enable arrangements to be made for the destruction of the
>>> transmission, or its return at our cost. No liability is accepted for any
>>> unauthorised use of the information contained in this transmission.
>>> 
>>> 
>>> _______________________________________________
>>> MapProxy mailing list
>>> MapProxy at lists.osgeo.org
>>> https://lists.osgeo.org/mailman/listinfo/mapproxy
>>> 
>>> 
>>> 
>>> ------------------------------
>>> 
>>> CONFIDENTIALITY NOTICE AND DISCLAIMER
>>> The information in this transmission may be confidential and/or protected
>>> by legal professional privilege, and is intended only for the person or
>>> persons to whom it is addressed. If you are not such a person, you are
>>> warned that any disclosure, copying or dissemination of the information is
>>> unauthorised. If you have received the transmission in error, please
>>> immediately contact this office by telephone, fax or email, to inform us of
>>> the error and to enable arrangements to be made for the destruction of the
>>> transmission, or its return at our cost. No liability is accepted for any
>>> unauthorised use of the information contained in this transmission.
> 
>> _______________________________________________
>> MapProxy mailing list
>> MapProxy at lists.osgeo.org
>> https://lists.osgeo.org/mailman/listinfo/mapproxy
> _______________________________________________
> MapProxy mailing list
> MapProxy at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/mapproxy