[mapguide-internals] Avoiding unnecessary Tile Cache invalidation (Trac Ticket 1332)

Wed Apr 28 01:58:29 EDT 2010

Pushing tile sets into a separate resource would ensure that only
changes affecting the tile set would trigger a flush of the cache,
addressing the original issue. That's neither here nor there though.

I do like the idea of making tile cache deletion a manually triggered
event, with UI in Maestro (with four or five confirmation prompts) to
initiate :)  There's already an API call for this on a per-set basis,
isn't there?

Jason

On 2010-04-27, Tom Fukushima <tom.fukushima at autodesk.com> wrote:
> There are two problems being discussed now: the problem in the ticket (ie.,
> tile cache being deleted when we don't want it to be) and shared tile sets.
> This is only about the problem in the ticket.
>
> I had originally thought that option 2 would be a good short term solution
> as well, but in the end I don't like it and I think we should remove the
> autodelete altogether. Then as time permits put an interface into the server
> admin or Studio&Maestro that allows us to flush the cache manually.  The
> interface could inform us when a tile set becomes stale due to a resource
> change and then we could either undo the resource change or clear the cache
> or ignore the warning.
>
> For the short term, I think you can keep the server from deleting your tile
> sets by making the tile directory (and subdirectories and files) read only.
> I just tried this (made my C:\Program
> Files\MapGuide\Server\Repositories\TileCache\Samples_Sheboygan_MapsTiled_Sheboygan
> recursively read only) and it seems to work (and you can still create new
> tiles), but we should look at the code to verify.  Otherwise, (still for the
> short term) if the read only option doesn't pan out, I think that what we
> users should be doing is diligently making copies of the tile sets (a tool
> like XXCOPY can do this easily) and then use the copies to restore things if
> the tiles get deleted.
>
> Even though I suggested a solution in the ticket; I've changed my mind and I
> don't like trying to be smarter about automatic regeneration of the tiles,
> because in the end we can never be completely correct since we can't
> accurately monitor changes in the data (ie., we don't know when someone has
> changed the feature geometry or attributes in the Oracle database).
>
>
> Tom
>
>
> -----Original Message-----
> From: mapguide-internals-bounces at lists.osgeo.org
> [mailto:mapguide-internals-bounces at lists.osgeo.org] On Behalf Of Zac Spitzer
> Sent: Tuesday, April 27, 2010 9:49 PM
> To: MapGuide Internals Mail List
> Subject: Re: [mapguide-internals] Avoiding unnecessary Tile Cache
> invalidation (Trac Ticket 1332)
>
> Uv coded up a solution for option 2, but we are yet to test it and he's
> travelling at the moment
>
> if studio/maestro could be aware of this setting, and could check the bounds
> for a temp copy of the new map in a session repo before modifying the real
> map this could be made a little more robust?
>
> z
>
> On 28 April 2010 13:14, Trevor Wekel <trevor_wekel at otxsystems.com> wrote:
>> Option 2 might be appropriate for MGOS 2.2.
>>
>> Longer term, I think ApplicationDefinition and WebLayout could be extended
>> to reference a "TileSetDefinition" resource.  Any data that takes days or
>> weeks to generate probably merits its own resource type.  Having a
>> separate resource type would also facilitate tile set replication from one
>> server to another.
>>
>> TileSetDefinition
>> {
>> ResourceId mapDefinition;
>> CoordinateSystem tileSetCoordinateSystem;
>> Extent tileSetExtent;
>> double[] scales;
>> Int tileWidth;
>> Int tileHeight;
>> Int metaTilingFactor;
>> Int32 backgroundColor;
>> // other tile related stuff
>> }
>>
>> The TileSetDefinition would be referenced by ApplicationDefinition and
>> WebLayout and the current tiling mechanism in MapDefinition would be
>> deprecated.  TileSetDefinition could also be extended to reference third
>> party tile sets like Google Maps, or even a different MapGuide server.
>>
>> The pre-rendered tiles would be associated with the TileSetDefinition so
>> MapDefinition changes would not immediately affect rendered tiles.
>>
>> Regards,
>> Trevor
>>
>> -----Original Message-----
>> From: mapguide-internals-bounces at lists.osgeo.org
>> [mailto:mapguide-internals-bounces at lists.osgeo.org] On Behalf Of Walt
>> Welton-Lair
>> Sent: April 27, 2010 8:41 PM
>> To: MapGuide Internals Mail List
>> Subject: RE: [mapguide-internals] Avoiding unnecessary Tile Cache
>> invalidation (Trac Ticket 1332)
>>
>> Lots of people have complained about this over the years.  Some options:
>>
>> - rip out the functionality
>>  - benefit: no more unexpected loss of tiles, less server overhead,
>> extremely easy to implement
>>  - disadvantage: map authors have to remember to delete invalid tile sets
>>
>> - add a serverconfig flag (a setting under tile service) to allow
>> disabling the behavior
>>  - benefit: no more unexpected loss of tiles, less server overhead, still
>> easy to implement
>>
>> - design and implement a way to tag/lock individual tile sets, to prevent
>> them from being deleted
>>  - disadvantage: requires the most effort
>>
>>
>> I think the second option should be considered.  The default behavior
>> would be to enable the functionality, and it could be turned off for
>> production servers.
>>
>> Walt
>>
>> ________________________________________
>> From: mapguide-internals-bounces at lists.osgeo.org
>> [mapguide-internals-bounces at lists.osgeo.org] On Behalf Of Zac Spitzer
>> [zac.spitzer at gmail.com]
>> Sent: Tuesday, April 27, 2010 10:11 PM
>> To: MapGuide Internals Mail List
>> Subject: Re: [mapguide-internals] Avoiding unnecessary Tile Cache
>> invalidation  (Trac Ticket 1332)
>>
>> a base layer group could be treated as a layer?
>>
>> this is really a frustrating issue for my clients as well
>>
>> z
>>
>> On 28 April 2010 11:04, Jason Birch <jason at jasonbirch.com> wrote:
>>> WOULD increase (no way of referencing external tile sets currently)
>>>
>>> On 2010-04-27, Jason Birch <jason at jasonbirch.com> wrote:
>>>> My suggestion has always been to treat tile sets as separate resources
>>>> and allow them to be included in multiple map definitions. This
>>>> increases usefulness and stability all in one go.
>>>>
>>>> On 2010-04-27, Jackie Ng <jumpinjackie at gmail.com> wrote:
>>>>>
>>>>> Hi All,
>>>>>
>>>>> At Zac's suggestion, I'm spawning this thread off of discussions on
>>>>> Trac
>>>>> #1332 (http://trac.osgeo.org/mapguide/ticket/1332) to stimulate some
>>>>> more
>>>>> discussion on this list.
>>>>>
>>>>> Sometime last week, I heard a tale of total frustration about how a
>>>>> Tile
>>>>> Cache of 1.5 million tiles that was built over 1.5-2 days was instantly
>>>>> trashed the moment a dynamic untiled layer was added to the map
>>>>> definition
>>>>> and saved back into the repository.
>>>>>
>>>>> That got me thinking about whether something as innocuous as changing
>>>>> the
>>>>> description or background color of a map definition w/ tiled layers
>>>>> would
>>>>> invalidate the tile cache so I tested this out and just as I feared, it
>>>>> indeed wiped out the tile cache.
>>>>>
>>>>> My initial thoughts were that tiling's only concerned with the base
>>>>> layer
>>>>> settings in the Map Definition. So theoretically speaking, doing
>>>>> modifications that don't affect the base layers should not affect the
>>>>> integrity of the original tiles.
>>>>>
>>>>> Now according to Tom, there is a whole stack of dependencies that would
>>>>> have
>>>>> to be checked such as Feature Source changes, Symbol changes, Layer
>>>>> Definition changes in order to determine whether a Map Definition's
>>>>> Tile
>>>>> Cache needs to be rebuilt. I wonder what overhead would be incurred by
>>>>> performing these checks (if it hasn't been done already).
>>>>>
>>>>> There's also renaming the tile cache directory before saving (also
>>>>> suggested
>>>>> by Tom) and then renaming back. I wonder what other things can be done
>>>>> so
>>>>> that the Tile Cache is cleared *only* if it has to be.
>>>>>
>>>>> - Jackie
>>>>> --
>>>>> View this message in context:
>>>>> http://osgeo-org.1803224.n2.nabble.com/Avoiding-unnecessary-Tile-Cache-invalidation-Trac-Ticket-1332-tp4971985p4971985.html
>>>>> Sent from the MapGuide Internals mailing list archive at Nabble.com.
>>>>> _______________________________________________
>>>>> mapguide-internals mailing list
>>>>> mapguide-internals at lists.osgeo.org
>>>>> http://lists.osgeo.org/mailman/listinfo/mapguide-internals
>>>>>
>>>>
>>> _______________________________________________
>>> mapguide-internals mailing list
>>> mapguide-internals at lists.osgeo.org
>>> http://lists.osgeo.org/mailman/listinfo/mapguide-internals
>>>
>>
>>
>>
>> --
>> Zac Spitzer
>> Solution Architect / Director
>> Ennoble Consultancy Australia
>> http://www.ennoble.com.au
>> http://zacster.blogspot.com
>> +61 405 847 168
>> _______________________________________________
>> mapguide-internals mailing list
>> mapguide-internals at lists.osgeo.org
>> http://lists.osgeo.org/mailman/listinfo/mapguide-internals
>> _______________________________________________
>> mapguide-internals mailing list
>> mapguide-internals at lists.osgeo.org
>> http://lists.osgeo.org/mailman/listinfo/mapguide-internals
>>
>>
>> _______________________________________________
>> mapguide-internals mailing list
>> mapguide-internals at lists.osgeo.org
>> http://lists.osgeo.org/mailman/listinfo/mapguide-internals
>>
>>
>
>
>
> --
> Zac Spitzer
> Solution Architect / Director
> Ennoble Consultancy Australia
> http://www.ennoble.com.au
> http://zacster.blogspot.com
> +61 405 847 168
> _______________________________________________
> mapguide-internals mailing list
> mapguide-internals at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/mapguide-internals
> _______________________________________________
> mapguide-internals mailing list
> mapguide-internals at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/mapguide-internals
>