[fdo-internals] RFC 23 for Review

Thu Jul 10 13:56:01 EDT 2008

I just want to clarify that MapGuide does cache schemas now; it didn't in the original 1.0 and 1.1 releases but does in 1.2 and higher.

Cheers
Tom

-----Original Message-----
From: fdo-internals-bounces at lists.osgeo.org [mailto:fdo-internals-bounces at lists.osgeo.org] On Behalf Of Haris Kurtagic
Sent: Thursday, July 10, 2008 11:57 AM
To: FDO Internals Mail List
Subject: RE: [fdo-internals] RFC 23 for Review

Right now King.Oracle is using 1 SQL statement for getting all spatial
tables and properties.
And additional SQL statement for every table to get primary keys.
To optimize this further King.Oracle have OracleSchema parameter in
connection so tables of interest could be set into particular Oracle
schema and provider will work with only tables in that schema.

It could be benefit that we have two stage Describe Schema. So in first
stage it would be enough to run 1 SQL statement (Problem is not number
of tables but number of executed statements). And second stage to fully
describe particular class.
But again it will be slow if application would call it all the time.
I think bigger problem is because how often application  use it.

- Providers are doing caching of schema because MapGuide is calling it
all the time.
  I think that is wrong. Provider will return false values if there are
changes in database.
  Provider should not be smarter than application. If application wants
to execute and get correct new schema provider should do so.
  Now it is caching just because practical reason, otherwise MapGuide
becomes too slow.

I think It would be very nice that we can with FDO add new property for
our GIS feature and see that property in database as well as in FDO
class without need to restart server. Now, if we add new property (new
column to table) we need to restart server to get it.

Haris

-----Original Message-----
From: fdo-internals-bounces at lists.osgeo.org
[mailto:fdo-internals-bounces at lists.osgeo.org] On Behalf Of Robert
Fortin
Sent: Thursday, July 10, 2008 3:40 PM
To: FDO Internals Mail List
Subject: RE: [fdo-internals] RFC 23 for Review

Although must provider do caching, it is not always the case and doesn't
have to be.  It's a trade off that is directly related to performance.
I don't think it is a big issue to requrest the application to
reinitialize itself after a schema modification.

The fact that there is currently only one command that require to get
the entire schema is not efficient for all application because it
requires a full reverse engineering of the entire database instead of
requesting the information relevant at a given time.  For example,
Mapguide doesn't have to get the entire schema as it works mostly with
one at a time.

If the provider decide to do caching of the entire schema when only the
class name are requested (as proposed by the RFC) or if caching is done
on the first call, there won't be any benefit.  But it doesn't have to
be this way.  In case it does just retrieve the name of the classes,
there will be benefit for the application, saving the reverse
engineering of the entire schema which - with large database - can be an
overhead that doesn't have to be.

I agree with Harris and Bruno, it has to be consistant across the board.
No distinction or special case between RBDMS and other provider although
the benefit might be more significant for RBDMS provider.  I understand
your strategy might be to limit the scope of the effort.  See a proposal
below to get rid of this special case.

Comments on the proposed RFC below:

FdoIEnumerateFeatureClasses: for consistancy with existing command doing
similar things in FDO

- The class should be named FdoIGetFeatureClasses (like
FdoIGetSpatialContexts, FdoIGetLongTransactions)
- Execute could return a reader to enable iteration instead of a string
collection.
- This return reader could by itself offer to return the class
definition for a given class instead of going back to the DescribeSchema
command.  This would also avoid the problem that you would be
introducing by reducing the scope to only the RDBMS function because

1) The new command could be limited to a subset of provider and not
expose by the ones where it less relevant.
2) No need to modify DescribeSchema command.
3) There won't be special case handling for DescribeSchema comamnd to
limit the scope to RDBMS provider.

Robert

-----Original Message-----
From: fdo-internals-bounces at lists.osgeo.org
[mailto:fdo-internals-bounces at lists.osgeo.org] On Behalf Of Haris
Kurtagic
Sent: Thursday, July 10, 2008 8:31 AM
To: FDO Internals Mail List
Subject: RE: [fdo-internals] RFC 23 for Review

Hi Bruno,
I saw your post now and I see I wrote same thing as you did.

Yes, King.Oracle provider is caching schema. Although I needed to add
deep Copy of Schema from cache before returning it to caller because of
bug in MapGuide. It makes it little slower after that.
Problem was in case of multi-user updating of data trough MapGuide.
MapGuide is deleting and adding class from schema in one of functions.
And if another user would do edit in that timeslot, exception in
MapGuide..

It is like MapGuide would assume that for every DescribeSchema he would
get new instance of schema. That could be correct in a way but then
MapGuide as application should be written to reuse schema instead of
calling it in almost every function.

I personally think for every request for Describe Schema providers
should do exactly that (not caching) and applications should count on
that.

Haris

-----Original Message-----
From: fdo-internals-bounces at lists.osgeo.org
[mailto:fdo-internals-bounces at lists.osgeo.org] On Behalf Of Bruno Scott
Sent: Thursday, July 10, 2008 8:52 AM
To: fdo-internals at lists.osgeo.org
Subject: Re: [fdo-internals] RFC 23 for Review

Hi Ronnie,
I know a little about the PostGIS provider, and the PostGIS provider
caches
the schema description internally on the first connection. So the first
call
is slow but all subsequent access to the DescribeSchema command is as
fast
as possible because it does not make any access to the rdbms.
The drawback of this approach is when we make a modification in the
rdbms
schema, we have to restart the mapguide server to make it available.

I know also that when we make a modification in an oracle schema, we
have
also to restart the mapguide server. So i bet other provider use this
same
technique.

So i guess for all of these providers that implement schema description
cache, we wont have much performance improvement.

Bruno

Ronnie Louie wrote:
>
> Hi,
>
> RFC 23 (http://trac.osgeo.org/fdo/wiki/FDORfc23) has been posted.
This
> RFC proposes APIs for retrieving feature class information without
having
> to perform a full DescribeSchema.  The main benefit will be improved
> performance in applications connecting to RDBMS datasources.  Please
> review the proposal and reply to this list with your comments by July
15,
> 2008.
>
> Thanks
> Ronnie
>
> _______________________________________________
> fdo-internals mailing list
> fdo-internals at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/fdo-internals
>
>

--
View this message in context:
http://www.nabble.com/RFC-23-for-Review-tp18366282p18376986.html
Sent from the FDO Internals mailing list archive at Nabble.com.

_______________________________________________
fdo-internals mailing list
fdo-internals at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/fdo-internals
_______________________________________________
fdo-internals mailing list
fdo-internals at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/fdo-internals
_______________________________________________
fdo-internals mailing list
fdo-internals at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/fdo-internals
_______________________________________________
fdo-internals mailing list
fdo-internals at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/fdo-internals