[MapProxy] FW: Which cache is better
Tobias Wendorff
tobias.wendorff at tu-dortmund.de
Sat Jun 10 10:35:04 PDT 2017
Some filesystems don't like thousands of small files. Even backing these up or syncing them might be very painful.
--
Von einem iPhone gesendet und wird daher Fehler enthalten…
Am 10.06.2017 um 10:14 schrieb Trond Michelsen <trondmm-mapproxy at crusaders.no>:
> What is the advantage of an sqlite cache over a simple file cache?
>
> On Fri, Jun 09, 2017 at 09:18:33AM -0400, Travis Kirstine wrote:
>> David,
>>
>> According to the doc you could try using the ArcGIS tiles as a source which
>> may increase your caching performance.
>>
>> https://mapproxy.org/docs/nightly/sources.html#tiles
>> see arcgiscache_path
>>
>>
>> We are using riak in a 5 node cluster N=3 with a leveldb backend, we found
>> that the bitcask memory footprint was unrealistic. We went with the
>> default 64 partitions which maybe should have been increased to 128. We've
>> found that debugging issues and simply accessing the data a little more
>> difficult than expected and required scripting to perform basic queries /
>> tasks. That being said the cluster is performing very well on both read /
>> writes
>>
>> We did also briefly test couchdb and found that it worked well, I would
>> have preferred it over riak but outside factors forced our decision.
>>
>> If I was you I would still try a sqlite based cache first to see if it
>> meets your need.
>>
>>
>>
>>
>> On 8 June 2017 at 17:53, Andrew, David (DPFEM) <
>> David.Andrew at dpfem.tas.gov.au> wrote:
>>
>>> Hi Travis,
>>>
>>> I need to supply a selection of base maps in a 24/7 configuration from the
>>> same datacentre as the applications servers.
>>>
>>> The application reading htem is an olderversion of geoserver, which cannot
>>> directly connect to arcgis.
>>>
>>> The source is
>>>
>>> http://services.thelist.tas.gov.au/arcgis/rest/services/
>>> Basemaps/Orthophoto/ImageServer
>>>
>>>
>>>
>>> yes to ARCGIS as a WMS source for MapProxy.
>>>
>>>
>>>
>>> *David Andrew*
>>>
>>> IT Services – Enterprise Systems
>>>
>>> Business and Executive Services
>>>
>>>
>>>
>>> Department of Police, Fire and Emergency Management
>>>
>>> Cnr Argyle and Melville Streets, Hobart TAS 7000
>>>
>>> David.Andrew at police.tas.gov.au| www.dpfem.tas.gov.au
>>>
>>>
>>>
>>> *From:* Travis Kirstine [mailto:traviskirstine at gmail.com]
>>> *Sent:* 08 June 2017 22:26
>>> *To:* Andrew, David (DPFEM) <David.Andrew at dpfem.tas.gov.au>
>>> *Cc:* mapproxy at lists.osgeo.org
>>> *Subject:* Re: [MapProxy] FW: Which cache is better
>>>
>>>
>>>
>>> David,
>>>
>>>
>>>
>>> If your primary concern is interoperability and portability I would look
>>> at mbtiles or geopackage. Both cache types store the tiles in a single
>>> file which could be used in a few platforms and easily backed up. The only
>>> issue with these caches (sqlite) it a tendency to lock up during heavy
>>> concurrent writes, mapproxy has implemented bulk writing to help address
>>> this issue for mbtiles (not sure about geopackage). If your users are
>>> using a fully seeded cache this won't be a big issue.
>>>
>>>
>>>
>>> When you indicate that it takes 2 weeks to download the cache from ArcGIS
>>> do you me that you are using ArcGIS as a WMS source for MapProxy?
>>>
>>>
>>>
>>> Regards
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> On 8 June 2017 at 00:29, Andrew, David (DPFEM) <
>>> David.Andrew at dpfem.tas.gov.au> wrote:
>>>
>>> Hi all,
>>>
>>> Totally a newbie regarding all things GIS
>>>
>>> I’m trying to establish which cache type is better for a photo cache.
>>>
>>>
>>>
>>> I’m looking to cache the orthographic photo maps for the state of Tasmania
>>> Australia, to a zoom level of 18
>>>
>>> This will then be used by the new emergency services application to have
>>> an offline pre seeded copy.
>>>
>>> Around 30 clients, up to 50 in a peak load like New Years eve.
>>>
>>>
>>>
>>> I cannot seem to find anything that defines which cache style would be
>>> better or worse.
>>>
>>> CouchDB seems interesting and replicatable
>>>
>>> Riak seem like very fast fetch
>>>
>>> Geopackage looks like we can copy the flat file once we get it.
>>>
>>>
>>>
>>> If anyone has any numbers around which is better that would be great.
>>>
>>> At the moment to download the cache takes a week or 2 for levels 17-18
>>> from an arcgis server(10.31 newer than the cache option allow)
>>>
>>>
>>>
>>> Thanks in advance
>>>
>>>
>>>
>>> *David Andrew*
>>>
>>> IT Services – Enterprise Systems
>>>
>>> Business and Executive Services
>>>
>>>
>>>
>>> Department of Police, Fire and Emergency Management
>>>
>>> Cnr Argyle and Melville Streets, Hobart TAS 7000
>>>
>>> David.Andrew at police.tas.gov.au| www.dpfem.tas.gov.au
>>>
>>>
>>>
>>>
>>> ------------------------------
>>>
>>>
>>> CONFIDENTIALITY NOTICE AND DISCLAIMER
>>> The information in this transmission may be confidential and/or protected
>>> by legal professional privilege, and is intended only for the person or
>>> persons to whom it is addressed. If you are not such a person, you are
>>> warned that any disclosure, copying or dissemination of the information is
>>> unauthorised. If you have received the transmission in error, please
>>> immediately contact this office by telephone, fax or email, to inform us of
>>> the error and to enable arrangements to be made for the destruction of the
>>> transmission, or its return at our cost. No liability is accepted for any
>>> unauthorised use of the information contained in this transmission.
>>>
>>>
>>> _______________________________________________
>>> MapProxy mailing list
>>> MapProxy at lists.osgeo.org
>>> https://lists.osgeo.org/mailman/listinfo/mapproxy
>>>
>>>
>>>
>>> ------------------------------
>>>
>>> CONFIDENTIALITY NOTICE AND DISCLAIMER
>>> The information in this transmission may be confidential and/or protected
>>> by legal professional privilege, and is intended only for the person or
>>> persons to whom it is addressed. If you are not such a person, you are
>>> warned that any disclosure, copying or dissemination of the information is
>>> unauthorised. If you have received the transmission in error, please
>>> immediately contact this office by telephone, fax or email, to inform us of
>>> the error and to enable arrangements to be made for the destruction of the
>>> transmission, or its return at our cost. No liability is accepted for any
>>> unauthorised use of the information contained in this transmission.
>
>> _______________________________________________
>> MapProxy mailing list
>> MapProxy at lists.osgeo.org
>> https://lists.osgeo.org/mailman/listinfo/mapproxy
> _______________________________________________
> MapProxy mailing list
> MapProxy at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/mapproxy
More information about the MapProxy
mailing list