MS RFC 22a: Feature cache for long running processes and query processing (update)

[Steve Lime]

I agree on the ordering issue. I don't use it often but when I do it's for a
good reason. The other benefit of maintaining order is consistent labeling. I
mean, labels always drop in/out as one pan or zooms but for the same map 
they are consistent over time. It would be confusing if the same map generated
seconds apart had totally different labels. Even with the prioritization addition
to 5.0 order is important within a priority level.

Steve

>>> On 6/28/2007 at 8:47 AM, in message <4683BBFF.3010604 at swoodbridge.com>, Stephen
Woodbridge <woodbri at SWOODBRIDGE.COM> wrote:
> Daniel Morissette wrote:
>> Hi Tamas,
>> 
>> Tamas Szekeres wrote:
>>>
>>>> and even if it has a NextItem method
>>>> to walk through all objects, the order of objects is not maintained by a
>>>> hashtable, so if a user has data sorted (by sortshp) then the sort oder
>>>> will be lost and rendering order will become pseudo-random if done via a
>>>> cache layer (unless I'm missing something?).
>>>>
>>>
>>> That's true. I'm not aware of the order of the renderings in this case.
>>> In my practice I haven't found such a problem it was required.
>>> However we could use an additional list to treat this issue if it is
>>> significant.
>>>
>> 
>> This ordering of shapes at render time is a feature of MapServer, hence 
>> the command-line program sortshp. I don't use it myself but some users 
>> must rely on it otherwise it would not exist. I think it's a sad 
>> side-effect to not try to maintain the ordering but I'll let those who 
>> need this feature fight for it.
> 
> Shape ordering is a problem!
> 
> This is a good point as I do use ordered data. I primarily use ordered 
> shapefiles to control the rendering and labeling of data. It would be a 
> little disconcerting to render an image and then when it is rendered out 
> of the cache all the labels would change because the object ordering has 
> changed. I think this would be very bad. At Where2getit we serve 500-750 
> Million map draws a year and I would hate to have to deal with questions 
> and issues from those users and clients. I suppose I do not need to use 
> the cache in which case I would get the old behavior which might be OK.
> 
> A few additional thoughts on this RFC:
> 
> 1) Would it make sense or be possible to construct a cache provider that 
>   also maintained a spatial index for its cache items?
> 
> 2) On a zoom in, I assume that the cache will be used exclusively since 
> the extents will be in side the previous extents, or does the new scale 
> invalidate the cache and require it to be rebuilt. Would it make sense 
> for the cache to be "aware" of the zoom scales it can support?
> 
> 3) Performance is a BIGGY for me. It takes a lot of servers to process 
> the map draws mentioned above. Mapserver is basically I/O bound in our 
> case of using shapefiles 30-50 GB per map dataset. Mapserver does NOT 
> use much memory. I would trade memory for performance any day. I have 
> found that I can get a 4-5x performance benefit by creating a 12 GB ram 
> disk and moving frequently hit data into it. That still leave 4 GB for 
> OS and mapserver processes on 16 GB system and I would like to see 
> mapserver use more memory if we can make it go faster.
> 
> -Steve
> -Steve