[UMN_MAPSERVER-USERS] MS RFC 22: Data adapter support

[Stephen Woodbridge]

Sorry Steve,

I noticed that the reply did not make it to the dev list after it was 
sent. Here is Tamas' reply for the dev list.

Tamas,

Thank you for the overview of your plans. This puts things into better 
context for me. BTW, I don't have any issues per say, I'm just asking 
questions that I think need to be thought.

Best regards,
  -Steve W

Tamas Szekeres wrote:
> 2006/10/24, Stephen Woodbridge <woodbri at swoodbridge.com>:
>> Tamas,
>>
>> I think some of the generic issues involved with a cache are caste into
>> the context of this RFC follow. I think answers to questions like this
>> will help the rfc provide a high level overview of how this will work
>> and deal with specific cache related issues.
>>
>> 1. How large can it get? Can I configure a max size? How?
>> 2. What happens to the behavior when it fills up?
>> 3. How are items expired from the cache?
>> 4. What happens when the cache gets fragmented?
>> 5. What happens when there is not room in the cache to add to it? Either
>> it is fragmented or full?
>> 6. There has been some implementation of layer/feature? security, I
>> thought, how will the cache deal with that?
>> 7. Can/do items have a Expiration time set on them? after which they are
>> invalid and can be purged or assumed to be purged?
>> 8. Is a cache - mapfile specific or mapserver instance specific or user
>> session specific? How will this be handled from a config and
>> installation point of view? In a live production site with multiple
>> users, multiple mapfiles, and multiple mapserver instances how does all
>> this work? How to the requests get to their respective cache instance.
>>
>> Answers to many of these questions probably represent config options for
>> the cache, where would these be configured on the installation server?
>>
>> I did not check the rfc, you may have addressed some of these already,
>> this is a dump of the problems I have had to consider working with other
>> caching applications.
>>
> 
> Steve,
> 
> This RFC would establish the infrastructure to write data adapters but not
> addressed to deal with a particular adapter implementation. I think it's 
> not
> so easy to digest these things in itself and would not want to introduce 
> new
> issues into this proposal.
> 
> However I am planning to create an additional RFC specifically to the 
> proposed
> feature cache implementation, but I would like to see how this one is 
> acceptable
> by the others, because that RFC will highly rely on this RFC.
> 
> But I can summarize my proposal here for further discussions. I would not
> keen to impement a fairly general and complex solution this implementation
> seems to be somewhat mapserver specific and will provide solution for my 
> issues
> have been coming up in the past.
> 
> 1. Primairly the cache will retain features from a previous provider
> query and will not
> aggregate features from multiple queries.
> 
> 2. The user can specify whether the features should be fetched using a 
> constant
> extent or using the actual extent of the map multiplied by a number
> (>=1). When selecting the
> constant extent option the reloading of the cache should be triggered
> by the user
> (from the mapscript API). The reload call can be issued even from a
> background thread.
> 
> 3. The user can specify how the fetching of the features should be
> done. The user can
> select that the loading of the features will not be done until the
> actual drawing area falls
> inside the cache area.
> 
> 4. The user can configure the cache from the mapfile of from the
> mapscript interface
> Every LAYER section can have arbitrary number of DATAADAPTER sections
> The metadata item of the dataadapter will contain the adapter specific
> configuration options
> For example
> LAYER
>  CONNECTIONTYPE OGR
>  ..
>  DATAADAPTER
>      NAME "MyAdapter1"
>      TYPE FEATURECACHE
>      METADATA
>          "option1"    "value1"
>          "option2"    "value2"
>      END
>   END
> END
> 
> 5. The cache will preserve the shapeObj references from LayerNextShape
> calls. The adapter
> will copy the requested shape to the client. For a faster access the
> adapter will also provide to get
> the shape objects by reference
> 
> 6. A quadtree index will be built up to provide faster access to the 
> shapes of a
> particuar rendering area
> 
> 
> Best Regards,
> 
> Tamas