[mapguide-internals] RFC 90 Meta Tiling Support

Mon Mar 29 21:09:03 EDT 2010

So far its using a config variable  x = UseMetaTiles
this creates a logical modulo x grid over the map. (Currently we are 
working with values of 3 and 4 (9x  16x metatiles))
Tileservice & Rendering has been augmented to deal with this.
Taking request to tilex,tiley... remap to metax, metay... keep metatile 
bitmap and return to tileservice instead of creating image,
then slice up the metatile and save x*x tiles in tilecache. Thats working.

The current issue is synchronisation (the current file locking algorithm 
is not thread safe when requesting the same tile)
In a high load scenarios we keep on exposing race condition in the 
insufficient locking code.
So this needs some more engineering and I am currently investigating 
options.
Extending the mutex scope would be simple but eliminating possible 
concurrency.
Generally lockfiles seem not such a good way to deal with this 
particular metatiling scenario.

----------------
Using the sandbox is more like a favour to the community than anything 
else, so other people can have a look.
Its named after the rfc so a simple and intuitive process has been followed!
If this is not good enough I am also happy to do things in my own git 
repository and present results after (which was my original plan)

Comments on the locking issues are very much appreciated. The original 
code has never been fully thread safe, just the problematic usecase was
difficult to exploit (requesting the same tile simultaneously in the 
unprotected code stretch (ServerTileService.cpp:GetTile before the mutex)

Tom Fukushima wrote:
> Hi Zac,
>
>   
>>> "there's a hell of lot of stuff done by AutoDesk Developers which doesn't really follow this pattern at all either"
>>>       
> Can you send me specific examples of this here?  I'm trying to make sure everything done is transparent so I would like to know where you still see problems. BTW, I only do this for MGOS, I don't handle FDO, CsMap or Fusion.
>
> For the RFC, I think this is a good idea, but the information that you have in this email seems a lot more detailed than what's in the actual RFC. I'm assuming here that for now, you are trying to get some feedback to fill in the RFC details.  
>
> I'm assuming that the meta tiles form a fixed grid over the map.  I.e., if I ask for tile x,y, there is a invariant(?) mapping to meta tile a,b.  Is this correct?
>
> "which then works in the back ground" - what does this mean?
>
> Thanks
> Tom
>
> -----Original Message-----
> From: mapguide-internals-bounces at lists.osgeo.org [mailto:mapguide-internals-bounces at lists.osgeo.org] On Behalf Of Zac Spitzer
> Sent: Friday, March 26, 2010 7:20 PM
> To: MapGuide Internals Mail List
> Subject: Re: [mapguide-internals] RFC 90 Meta Tiling Support
>
> Sorry everyone, we should of posted to internals first.
>
> that said there's a hell of lot of stuff done by AutoDesk Developers which
> doesn't really follow this pattern at all either... just saying :) but
> regardless
> that's no excuse, we should of posted to the list first.
>
> sandboxes are play grounds and the meta tiling work Uv is doing is very
> much a work in progress at the moment. Once he has nutted out the
> details he will post more info to internals
>
> so, lets leave that behind and rewind the clock....
>
> -----------------------------------------------------
>
> I have posted a Draft of RFC 90 on trac
>
> http://trac.osgeo.org/mapguide/wiki/MapGuideRfc90
>
> This is a proposal to add meta tiling support to the mapguide engine.
>
> The current approach is define a meta tile size factor in the serverconfig.ini
> which then works in the back ground, rendering a large tile and then
> slicing it into the normal tile sizes.
>
> Given the static nature and global nature of anything done via serverconfig.ini,
> this is all or nothing. I think we have resource headers which could be
> used a lot more than we do currently, that way we can make map specific
> configuration overrides.
>
> For example, image format, selection color and meta tile factor would
> all be good candidates for setting via resource headers. That would
> avoid having to rev the xml def's to add such functionality.
>
> Being able to tune maps configurationwithout changing a global setting
> and restarting the server would be a good thing... RFC worthy?
>
> Back to meta tiling...
>
> Another approach would be to add support for a RENDERMETATILE mapagent,
> which would avoid the locking and polling in the current model.
>
> Uv's work so far at the base implementation is the rfc90 sandbox
>
> Regards
>
> Zac
>
>
>
>
>
>
> On 27 March 2010 06:34, Jason Birch <jason at jasonbirch.com> wrote:
>   
>> The sandbox doesn't necessarily need to be tied to an RFC.
>>
>> Although the proper procedure should be followed in the future, I'm happy to
>> retroactively approve the creation of the sandbox once the description email
>> is sent.  Our SVN churns enough without having to delete / recreate that
>> branch :)
>>
>> It would be good to get the RFC into discussion/approval before too much
>> work is done so we can ensure that the approach is acceptable.  Wouldn't
>> want to have to re-do substantial parts or (as a project) to lose this
>> contribution.
>>
>> Jason
>>
>>
>> On 26 March 2010 09:27, Trevor Wekel  wrote:
>>
>>     
>>> Hello UV,
>>>
>>> I have been having offline discussions with Zac regarding Meta Tiling
>>> Support.  I do implicitly support this RFC because of the potential
>>> performance improvements.
>>>
>>> Did I miss something?  I see that a draft RFC has been posted and a new
>>> RFC90 sandbox has been created.  I do not recall see a posting to
>>> mapguide-internals stating that you are
>>> a) Working on meta tiling support
>>> b) Going to create an RFC
>>> c) Requesting a Sandbox
>>>
>>> The Sandbox rules are clearly posted on
>>> http://trac.osgeo.org/mapguide/wiki/Sandboxes.  Sandboxes must be
>>> requested.
>>>
>>> It is great that you are contributing to the MapGuide project.  We
>>> definitely need more developers in the community.  Please keep in mind that
>>> all developers have to play by the rules.
>>>
>>> If a sandbox for RFC 90 was officially requested on mapguide-internals,
>>> please disregard this email.
>>>
>>>
>>> Sincerely,
>>> Trevor
>>>
>>>
>>> _______________________________________________
>>> mapguide-internals mailing list
>>> mapguide-internals at lists.osgeo.org
>>> http://lists.osgeo.org/mailman/listinfo/mapguide-internals
>>>
>>>
>>>       
>> _______________________________________________
>> mapguide-internals mailing list
>> mapguide-internals at lists.osgeo.org
>> http://lists.osgeo.org/mailman/listinfo/mapguide-internals
>>
>>     
>
>
>
>   