MS RFC 22: Data adapter support

[Tamas Szekeres]

2006/10/24, Stephen Woodbridge <woodbri at swoodbridge.com>:
> Tamas,
>
> I think some of the generic issues involved with a cache are caste into
> the context of this RFC follow. I think answers to questions like this
> will help the rfc provide a high level overview of how this will work
> and deal with specific cache related issues.
>
> 1. How large can it get? Can I configure a max size? How?
> 2. What happens to the behavior when it fills up?
> 3. How are items expired from the cache?
> 4. What happens when the cache gets fragmented?
> 5. What happens when there is not room in the cache to add to it? Either
> it is fragmented or full?
> 6. There has been some implementation of layer/feature? security, I
> thought, how will the cache deal with that?
> 7. Can/do items have a Expiration time set on them? after which they are
> invalid and can be purged or assumed to be purged?
> 8. Is a cache - mapfile specific or mapserver instance specific or user
> session specific? How will this be handled from a config and
> installation point of view? In a live production site with multiple
> users, multiple mapfiles, and multiple mapserver instances how does all
> this work? How to the requests get to their respective cache instance.
>
> Answers to many of these questions probably represent config options for
> the cache, where would these be configured on the installation server?
>
> I did not check the rfc, you may have addressed some of these already,
> this is a dump of the problems I have had to consider working with other
> caching applications.
>

Steve,

This RFC would establish the infrastructure to write data adapters but not
addressed to deal with a particular adapter implementation. I think it's not
so easy to digest these things in itself and would not want to introduce new
issues into this proposal.

However I am planning to create an additional RFC specifically to the proposed
feature cache implementation, but I would like to see how this one is acceptable
by the others, because that RFC will highly rely on this RFC.

But I can summarize my proposal here for further discussions. I would not
keen to impement a fairly general and complex solution this implementation
seems to be somewhat mapserver specific and will provide solution for my issues
have been coming up in the past.

1. Primairly the cache will retain features from a previous provider
query and will not
aggregate features from multiple queries.

2. The user can specify whether the features should be fetched using a constant
extent or using the actual extent of the map multiplied by a number
(>=1). When selecting the
constant extent option the reloading of the cache should be triggered
by the user
(from the mapscript API). The reload call can be issued even from a
background thread.

3. The user can specify how the fetching of the features should be
done. The user can
select that the loading of the features will not be done until the
actual drawing area falls
inside the cache area.

4. The user can configure the cache from the mapfile of from the
mapscript interface
Every LAYER section can have arbitrary number of DATAADAPTER sections
The metadata item of the dataadapter will contain the adapter specific
configuration options
For example
LAYER
  CONNECTIONTYPE OGR
  ..
  DATAADAPTER
      NAME "MyAdapter1"
      TYPE FEATURECACHE
      METADATA
          "option1"	"value1"
          "option2"	"value2"
      END
   END
END

5. The cache will preserve the shapeObj references from LayerNextShape
calls. The adapter
will copy the requested shape to the client. For a faster access the
adapter will also provide to get
the shape objects by reference

6. A quadtree index will be built up to provide faster access to the shapes of a
particuar rendering area


Best Regards,

Tamas