[mapserver-users] Mapcache tileset only shows sporadically in KML / Google Earth

Sun Jun 9 06:22:51 PDT 2013

On 6/9/2013 6:23 AM, Rahkonen Jukka wrote:
> Hi,
>
> This is so important thing to know that it should be mentioned in
> http://www.mapserver.org/trunk/mapcache/seed.html I believe I am not
> the only GeoWebCache users who experiment with MapCache and think
> that MapCache can also do seeding/reseeding/truncating for an active
> cache because that is the standard with GWC.

I tend to agree with you on this point. You might want to write a ticket 
for this as a new feature with the use case(s) that you provided below. 
That said, Thomas has been very busy and getting any priority on this 
might need some funding.

> The need to use seeding tool with inative cache makes me to wonder
> how to implement partial updating of big cache like 1 TB in size in
> production environment. The service can't be stopped for truncating
> and reseeding. Making a copy of the active cache, truncating and
> reseeding the copy and finally switching MapCache to use the
> refreshed cache should work, but copying 1 TB of data is slow and
> keeping an extra terabyte on a hosted disk is costly.
>
> I guess that by using a GeoTIFF cache it would be possible to use the
> seeding tool for creating a new cache from the updated areas.  The
> tiff files in the new cache are using the same naming convention and
> therefore they could be copied on top of the outdated tiles in the
> active cache and this could happen on-the-fly. Would this work?

The way I have updated my live cache in the past is:

1. update the source with new data
2. run a script to delete tiles in the cache for the updated area of 
interest or 1/N of it if I want to do a multi-day N day update to spread 
the load out

This is relatively safe, but runs the risk of an occasional tile getting 
deleted while it is getting downloaded, but on the next refresh from the 
client a new tile will get loaded.

I have also written a script (can't find it at the moment) that computes 
the top-left tile in each metatile for a give area of interest and 
request it from apache. This in effect creates a seeder that uses apache 
to do the work and therefore works cooperatively with the live cache.

-Steve

> -Jukka Rahkonen-
>
>
>
> ________________________________________ Stephen Woodbridge wrote:
>
>> In general, The seeder can not be run on an active cache because it
>> does
> not coordinate tile generation with apache so if they both try to
> create the same tiles things can get confusing and there is a chance
> that an incomplete tile might get served with both writing the
> resulting tile might be damaged.
>
> -Steve W.
>
> On 6/8/2013 3:41 PM, John D. Evans wrote:
>> Hi,
>>
>> Thomas Bonfort wrote on 06/06/2013:
>>> As for the root issue, are your tilesets fully preseeded or are
>>> they populated on demand? KML superoverlays are organized in a
>>> tree, i.e. to access a tile at zoom level N you first need to
>>> access all parent tiles from level 0 to N-1; on an unseeded
>>> tileset with a slow wms backend this might be a very long wait.
>>> Also please check the apache error log for any mapcache related
>>> messages.
>>
>> Thanks for the input! Sorry for the delayed reply. My tiles had
>> been populated on demand but I finally found, installed, and ran
>> "mapcache_seed". It's now been running for 10+ hours now (*) but I
>> still get the same effect in Google Earth. (A very low-resolution
>> view displays when I'm zoomed out very far, but disappears as soon
>> as I try to zoom in.) Does the seeding have to be completely
>> finished before Google Earth can see beyond the top-level KML
>> tiles?
>>
>> Thanks again.
>>
>> - John E.
>>
>> (*) I'm running this on a fairly puny virtual machine (2 cores); I
>> guess I should try seeding with 8 or 10 threads on an 8-core
>> machine. I will also try limiting my zoom levels (I suspect I'm
>> seeding 0-17 by default) and my spatial extent (I'm probably doing
>> the whole globe by default -- lots of blank tiles :-).
>>
>>>
>>>
>>> On 6 June 2013 05:19, John D. Evans <john.evans at gst.com
>>> <mailto:john.evans at gst.com>> wrote:
>>>
>>> Hello,
>>>
>>> Not sure my earlier query made it to the list.
>>>
>>> I'm using MapCache to serve for some satellite image products.
>>> It sort of works: the tms and wmts browsers at /mapcache/demo
>>> show full coverage of my data area, and let users zoom in to see
>>> full resolution image mosaics. But the KML superoverlay (my
>>> reason for using MapCache) only displays in Google Earth when I'm
>>> zoomed very far out (so that the entire globe is not much bigger
>>> than a golf ball on my screen). If I zoom in, the image usually
>>> disappears; though sometimes when zoomed in I do see [only] the
>>> right (east) half of my tileset, at a very low resolution.
>>>
>>> Do you see anything wrong with my mapcache.xml (below)? Is there
>>> anything else I should be checking?
>>>
>>> (I'm using a simple disk cache in /tmp, with mod_mapcache from
>>> the Ubuntu libmapcache 1.0.0-3 package; Mapserver 6.2.1 is on the
>>> same machine, running in 64-bit Ubuntu 12.04. Mapserver works
>>> fine, and reports no errors [I think]. My client is Google Earth
>>> 7.)
>>>
>>> Thanks for taking a look,
>>>
>>> - John D. Evans GST, Inc / Greenbelt, MD
>>>
>>> Below is my mapcache.xml file (actually an excerpt of it: the
>>> real one has more tilesets and sources).
>>>
>>> <mapcache> <metadata> <title>my mapcache service</title>
>>> <abstract>woot! this is a service abstract!</abstract>
>>> </metadata>
>>>
>>> <cache name="disk" type="disk"> <base>/tmp</base>
>>> <symlink_blank/> </cache>
>>>
>>> <format name="PNG_BEST" type ="PNG">
>>> <compression>best</compression> </format> <format name="PNG_FAST"
>>> type ="PNG"> <compression>fast</compression> </format>
>>>
>>> <source name="CSPP_CloudPhase" type="wms"> <http>
>>> <url>http://localhost/cgi-bin/mapserv?</url> </http> <getmap>
>>> <params> <FORMAT>image/png</FORMAT>
>>> <LAYERS>iicmocp_daily</LAYERS> <MAP>/raid/maps/viirs.map</MAP>
>>> </params> </getmap> </source>
>>>
>>> <tileset name="CSPP_Cloud_Phase_EDR"> <metadata> <title>VIIRS
>>> Cloud Phase from NASA GSFC DRL</title> <abstract>Detailed
>>> description goes here</abstract> </metadata>
>>> <source>CSPP_CloudPhase</source> <cache>disk</cache>
>>> <format>PNG</format> <grid>WGS84</grid> <metatile>2 2</metatile>
>>> <expires>3600 <tel:3600></expires> <auto_expire>3600
>>> <tel:3600></auto_expire> <!-- <metabuffer>10</metabuffer> -->
>>> </tileset>
>>>
>>> <default_format>PNG_FAST</default_format>
>>>
>>> <service type="wms" enabled="true">
>>> <full_wms>assemble</full_wms>
>>> <resample_mode>bilinear</resample_mode>
>>> <format>PNG_FAST</format> <!-- <maxsize>4096</maxsize> -->
>>> </service>
>>>
>>> <service type="wmts" enabled="true"/> <service type="tms"
>>> enabled="true"/> <service type="kml" enabled="true"/> <service
>>> type="gmaps" enabled="true"/> <service type="ve"
>>> enabled="true"/> <service type="demo" enabled="true"/>
>>>
>>> <lock_dir>/tmp</lock_dir> <errors>report</errors>
>>> <threaded_fetching>true</threaded_fetching>
>>>
>>> </mapcache> _______________________________________________
>>> mapserver-users mailing list mapserver-users at lists.osgeo.org
>>> <mailto:mapserver-users at lists.osgeo.org>
>>> http://lists.osgeo.org/mailman/listinfo/mapserver-users
>>>
>>>
>>> ------------------------------------------------------------------------
>>>
>>>
NOTE: This message was trained as non-spam. If this is wrong, please
>>> correct the training as soon as possible. Spam
>>> <https://filter.gst.com/canit/b.php?i=01JIUgvAE&m=e50f357136f6&c=s>
>>>
>>>
Not spam
>>> <https://filter.gst.com/canit/b.php?i=01JIUgvAE&m=e50f357136f6&c=n>
>>>
>>>
Forget previous vote
>>> <https://filter.gst.com/canit/b.php?i=01JIUgvAE&m=e50f357136f6&c=f>
>>
>>
>>
>>
>>>
_______________________________________________
>> mapserver-users mailing list mapserver-users at lists.osgeo.org
>> http://lists.osgeo.org/mailman/listinfo/mapserver-users
>>
>
> _______________________________________________ mapserver-users
> mailing list mapserver-users at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/mapserver-users
> _______________________________________________ mapserver-users
> mailing list mapserver-users at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/mapserver-users
>