[fdo-internals] FDO PostGIS provider developments

Dan Stoica dan.stoica at autodesk.com
Thu Dec 10 11:30:15 EST 2009 

Hi Jackie,

> but dog slow in SQL Server 2008

SQL Server 2008 is not fast. Even native select is about 10,000 simple polygons/sec. This is confirmed by MS on their site. When comes to inserts it is expected to be way slower because of building the spatial index. 

I'm not saying that Generic RDBMS insert cannot be optimized but the expectations have to be reasonable. Anybody is free to contribute :-)
 

Dan.

-----Original Message-----
From: fdo-internals-bounces at lists.osgeo.org [mailto:fdo-internals-bounces at lists.osgeo.org] On Behalf Of Traian Stanev
Sent: Thursday, December 10, 2009 10:18 AM
To: fdo-internals at lists.osgeo.org
Subject: RE: [fdo-internals] FDO PostGIS provider developments



The situation with the PostGIS provider is similar to the recent situation with the ArcSDE provider -- essentially someone comes in and says they can write a better provider. It's the job of the programmer who writes the code to decide what APIs to use (or not to use) -- in the end all everyone else should care about is that the provider works well and that it is updated in timely manner to reflect any new mandatory FDO API changes.

Traian


-----Original Message-----
From: fdo-internals-bounces at lists.osgeo.org [mailto:fdo-internals-bounces at lists.osgeo.org] On Behalf Of Jackie Ng
Sent: Wednesday, December 09, 2009 1:09 AM
To: fdo-internals at lists.osgeo.org
Subject: Re: [fdo-internals] FDO PostGIS provider developments


My turn to chime in...

I'm in favour of a move to a Generic RDBMS based PostGIS provider, even if
it means an elevated barrier for contribution from non-ADSK developers.
While there has been documented difficulties with an earlier attempt at such
a provider, that was over 2 years ago. Lots of things can happen/change in 2
years, technologically speaking.

In terms of performance, I find Generic RDBMS based providers to be a mixed
bag. For example, I find bulk inserting of data is reasonably fast in MySQL
but dog slow in SQL Server 2008, and they are both derived from the same
shared code! I personally wouldn't take any arguments for/against the move
based on performance with much seriousness.

As to what to do with the current provider if the move were to go ahead, I'm
in favour of keeping it around for a while as a "legacy" provider. I don't
think anyone has mentioned anything about having to maintain both.

- Jackie


Orest Halustchak wrote:
> 
> Hi,
> 
> Autodesk has had interest from Map3D customers who want to use the FDO
> PostGIS provider to access and edit data using Map3D and other products.
> 
> These customers have not been able to use this provider reliably with
> Map3D. We had a close look at the provider code to determine what work
> would be needed to complete the implementation of schema and edit support
> and fix other issues. Unfortunately, we found that the provider as it
> stands today requires a lot of work to complete the implementation of
> required FDO interfaces and to add good unit test coverage. Things that
> still need a fair bit of work include:
> 
> *         Creating new schema and new datastore
> 
> *         Spatial filter handling
> 
> *         Huge memory leaks on insert.
> 
> *         Not all schema commands are implemented.
> 
> *         Enable and fix transaction support.
> 
> *         Constraint and default values support.
> 
> *         Lots of TODOs spread all over the code
> 
> *         Virtually no unit tests exist.
> 
> We looked at the level of effort needed to complete that work. It was
> quite high. So, we looked at an alternative. There exists an earlier open
> source community code base for a PostGIS provider that was started a
> couple of years ago but not finished. That code base used the generic
> rdbms framework that is shared with the SQL Server Spatial, MySQL, and
> ODBC providers. Most of the schema processing is handled with that shared
> code. We spent some time working with the current provider and the other
> code base to determine the most efficient way to get to a completed
> provider that would be robust, perform well, and be maintainable.
> 
> In the end, we determined that taking the earlier code base, adding
> support for the recent fdo interface changes, and completing other parts
> that weren't finished would take much less time. Also, based on
> performance comparisons, we would get something that was much faster on
> inserts and selects, e.g. the select performance is about six times faster
> and schema describe is about three times faster. We couldn't compare
> insert times very well because the current provider kept crashing after a
> certain point and we couldn't insert a large number of features.
> 
> So, what we would like to do is complete our work to get a working PostGIS
> provider and then replace the current open source code with our new copy.
> Note that we plan to use native PostGIS schema without adding additional
> metadata tables just as the current provider does. It will be able to read
> any schemas created by the current version of the provider and itself will
> generate generic PostGIS schemas.
> 
> We'll need to submit an RFC for this, but we wanted to get this
> information out to you ahead of time.
> 
> At the same time, we are planning to change the connection parameters to
> separate out the database name from the service name. This will make it
> easier for users. They can identify the service (e.g. localhost:5432), and
> then see the available datastores from which they can choose in a UI.
> Then, PostGIS schema simply will map to FDO schema. The main drawback to
> this is that any users with existing MapGuide feature sources and layer
> definitions will have to update them.
> 
> In the end, we will end up with a good functioning provider that performs
> well.
> 
> 
> Regards,
> Orest.
> 
> 
> _______________________________________________
> fdo-internals mailing list
> fdo-internals at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/fdo-internals
> 
> 

-- 
View this message in context: http://n2.nabble.com/FDO-PostGIS-provider-developments-tp4085231p4137693.html
Sent from the FDO Internals mailing list archive at Nabble.com.
_______________________________________________
fdo-internals mailing list
fdo-internals at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/fdo-internals
_______________________________________________
fdo-internals mailing list
fdo-internals at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/fdo-internals

More information about the fdo-internals mailing list