[mapguide-internals] RFC60 finalization

Mon Oct 26 10:20:16 EDT 2009

It's implementing the simplest effective way of caching -- generally you have lots of features sharing the same style within a request and it will only parse the style once for those features. Having a global cache is more complicated because it also has to monitor changes to symbol resources -- and it would have been even more annoying to implement back when we theoretically allowed for the resource service to run on a separate machine than the mapping service.

Do you have any empirical results of a real world map showing that the current implementation results in a high relative overhead of symbol parsing compared to, say, FDO requests, or PNG saving or the symbol stylization itself?

Traian

> -----Original Message-----
> From: mapguide-internals-bounces at lists.osgeo.org [mailto:mapguide-
> internals-bounces at lists.osgeo.org] On Behalf Of UV
> Sent: Monday, October 26, 2009 8:21 AM
> To: MapGuide Internals Mail List
> Subject: Re: [mapguide-internals] RFC60 finalization
> 
> USAGE of SEMgSymbolManager????
> 
> Trying to understand the current implementation and usage of the
> SEMgSymbolManager some questions occur.
> 
> Why is there an instance for each thread?
> 
> In my current understanding there is a map object which could be cached
> previously.
> Upon reception of the gettile request this get transformed into a
> runtime map.
> 
> So in a very typical GetTile client use case we have a number of
> parallel neighboring tiles requested from the same map and scalerange.
> 
> What is the reason to not keep a shared instance of the map object
> between the threads instead of repeating the whole kaboodle...
> Symbol manager instances..... map parsing etc. this seems like a read
> only data structure which can be created once and then be made
> accessible to all threads via a singleton....
> 
> what am I overlooking here? I have a hard time seeing the benefit of
> caching stylization objects when the cache is created per request/
> There could be a lot of potential regarding computational expense  if
> we
> can leverage the server code to share more of those RO objects between
> the threads.
> 
> Instead of creating a cache instance inside the rendering method of the
> rendering service we could have a global cache structure managing
> maps and related scaleranges with stylization info including all
> referenced resources.....
> 
> any ideas?
> 
> 
> UV wrote:
> > MOTIVATION
> > The important question is how far the color search in the map should
> go.
> > To solve the problem of our client we were only looking at areas and
> > line styles.
> > Those are the relevant use cases as they are easily spotted on
> > bordering tiles!
> > (and it is not so computational expensive)
> >
> > To include all the symbolization data including the referenced
> symbols
> > seems like an computational overkill with no justification.
> > I am not sure if this should be enabled by default!
> >
> > Dont forget that we are only tuning a color quantization algorithm
> for
> > the paletted PNG8 /GIF  tiles with this information!!
> > Its only a gradual improvement of color correctness with limited
> impact.
> >
> >
> > Tom Fukushima wrote:
> >> I'm trying to understand this...so correct me if I'm wrong.  But it
> seems that the problem is not with attribute-based stylization.  It's
> that if someone uses referenced symbols (that is, the LayerDefinition
> specifies a resource using a resource ID such as
> Library://mysymbols/a.SymbolDefinition instead of embedded XML for a
> SymbolDefinition in the LayerDefinition) then a large part of the code
> will need to be rewritten to support it.
> >>
> > Refactoring is not really necessary. The access points are very
> > limited. The string collection is capable of dealing with variables
> > and references.
> > Only adding functionality for the bits we did not need and did not
> > implement!
> > This is why I don't think this is my responsibility to do this
> without
> > funding, no good test cases (maps), and no client.
> >> If the above is the case, I understand that we need to make
> tradeoffs all the time, and this may be where we need to make one.  I
> think though that it would be advantageous for the current developer to
> do the requested refactoring now while it is fresh.  Later, if defects
> are found or refinements (e.g., attribute-based stylization) are done,
> they could be addressed at that time---and this would be manageable
> because wouldn't require refactoring large parts of the code.
> >>
> >> Could we get some idea of the work that would be required to get
> support for the referenced symbol definitions?
> >>
> >> Note that I'm not saying it has to be done (but of course I would
> prefer it to be done), I'm trying to get more information on this
> sticking point (I believe it is the only one right?) to try to make a
> judgment.
> >>
> >> Tom
> >>
> >>
> >> -----Original Message-----
> >> From: mapguide-internals-bounces at lists.osgeo.org [mailto:mapguide-
> internals-bounces at lists.osgeo.org] On Behalf Of Jason Birch
> >> Sent: Friday, October 23, 2009 9:24 AM
> >> To: MapGuide Internals Mail List
> >> Subject: Re: [mapguide-internals] RFC60 finalization
> >>
> >> It seems to me that we have frequently (too often perhaps?) favoured
> >> expediency over correctness when faced with resourcing or timeline
> >> issues.
> >>
> >> In this case, I'm in favour of placing the burden of change on the
> >> whoever deems that it is critical to support pallette reservation
> for
> >> attribute-based stylization.
> >>
> >> Jason
> >>
> >> On 2009-10-23, Walt Welton-Lair <walt.welton-lair at autodesk.com>
> wrote:
> >>
> >>>> Secondly, it think it is wrong to request the implementation of
> the new
> >>>> stylization for such transparent feature addition.
> >>>> RFC60 works fine for our case, why should we implement something
> that
> >>>> does not harm anything but what we do not need?
> >>>>
> >>> The way your code is designed - where it does the work of grabbing
> the
> >>> colors inside the MdfModel project - WILL NOT WORK for enhanced
> stylization
> >>> in which there are referenced symbol definitions.  It's not
> possible to load
> >>> referenced symbols from the resource service from inside the
> MdfModel code.
> >>> This is not a minor defect.  It requires that a lot of your code be
> >>> refactored.  I already explained the problem last March.  Here's
> the
> >>> excerpt:
> >>>
> >>> "With the new enhanced stylization (RFC 14) the layer definition
> can
> >>> reference symbol definitions (in addition to inlining them).  The
> symbols,
> >>> of course, define colors which need to be accounted for.  Accessing
> >>> referenced symbol definitions requires the resource service - we
> need to get
> >>> the symbol definition resource from the service.  The MdfModel
> project
> >>> (where VectorLayerDefinition is stored) does not have access (nor
> want
> >>> access) to the resource service.  So if you want to properly
> support the new
> >>> enhanced stylization with your RFC (you should) then we'll probably
> have to
> >>> move this ComputeUsedColors method somewhere else.  There's a
> method -
> >>> MgMappingUtil::ComputeStylizationOffset - which does something
> analogous to
> >>> ComputeUsedColors, so possibly we can add ComputeUsedColors to
> >>> MgMappingUtil."
> >>>
> >>> So even if you don't add code to actually parse colors for enhanced
> style
> >>> layers, you need to keep the bigger picture in mind and design your
> new code
> >>> so it will be easy to add this support.
> >>>
> >>> All along that's the real reason why I wanted you to take a look at
> this.
> >>> So that you would come to realize that your current design doesn't
> fully
> >>> work as is.
> >>>
> >>>
> >>>
> >>>> Open source is more like: Who needs it should code it!
> >>>>
> >>> True, but it's not an excuse for not doing the right thing.  People
> can't
> >>> just add anything they want without taking account the bigger
> picture.
> >>> That's my take on this.
> >>>
> >>>
> >>> Walt
> >>>
> >>> -----Original Message-----
> >>> From: mapguide-internals-bounces at lists.osgeo.org
> >>> [mailto:mapguide-internals-bounces at lists.osgeo.org] On Behalf Of UV
> >>> Sent: Friday, October 23, 2009 9:03 AM
> >>> To: MapGuide Internals Mail List
> >>> Subject: [mapguide-internals] RFC60 finalization
> >>>
> >>> Hi all,
> >>>
> >>> I looked into the RFC60 code again and added some more features to
> it
> >>> e.g. finding label colors.
> >>>
> >>> However, a redirection of referenced featureIds which would cause a
> FDO
> >>> lookup seems prohibitive expensive in this context.
> >>> I would not like to add this into the code simply as it could
> increase
> >>> delay tile computation a lot.
> >>> So the interesting test here are performance related. Simple unit-
> tests
> >>> are not sufficient for this.
> >>> Creating complex test data to test performance issues is tedious
> and
> >>> very badly motivated!!! (specification vs. implementation from same
> hand)
> >>>
> >>> Secondly, it think it is wrong to request the implementation of the
> new
> >>> stylization for such transparent feature addition.
> >>> RFC60 works fine for our case, why should we implement something
> that
> >>> does not harm anything but what we do not need?
> >>> This is not thought along open source paradigms.  Open source is
> more
> >>> like: Who needs it should code it!
> >>> So we should leave this open and the next person that has real test
> data
> >>> which doesn't work can post a defect!
> >>> Very easy process!
> >>>
> >>>
> >>> RFC60 is not trivial at all but it solves a real problem in real
> maps.
> >>> To block the inclusion of RFC60 into the code base on those matters
> is
> >>> simply wrong in open source terms as it keeps others from using a
> new
> >>> working feature.
> >>> _______________________________________________
> >>> mapguide-internals mailing list
> >>> mapguide-internals at lists.osgeo.org
> >>> http://lists.osgeo.org/mailman/listinfo/mapguide-internals
> >>> _______________________________________________
> >>> mapguide-internals mailing list
> >>> mapguide-internals at lists.osgeo.org
> >>> http://lists.osgeo.org/mailman/listinfo/mapguide-internals
> >>>
> >>>
> >> _______________________________________________
> >> mapguide-internals mailing list
> >> mapguide-internals at lists.osgeo.org
> >> http://lists.osgeo.org/mailman/listinfo/mapguide-internals
> >> _______________________________________________
> >> mapguide-internals mailing list
> >> mapguide-internals at lists.osgeo.org
> >> http://lists.osgeo.org/mailman/listinfo/mapguide-internals
> >>
> >>
> >
> 
> _______________________________________________
> mapguide-internals mailing list
> mapguide-internals at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/mapguide-internals