[fdo-internals] FDO PostGIS provider developments

Mon Nov 30 10:55:14 EST 2009

Hi Haris,

One of the problems that the PostGIS provider user-community are experiencing is a lack of development support from the community on some fairly fundamental problems including connectivity, schema management, stability, and unit testing. In all honestly, the current code base as it stands simply does not meet the standards expected of a robust real-world deployment into a customer site. In Orest's email, he outlined a number of issues that have been identified as road-blocks, and are not on anyone's radar to address in the 3.5 timeframe. While the GenericRDBMS code base is slightly more complex, it has been proven to work, and work extremely well. Stability is not an issue. Performance and schema management are sound. Memory management is under control, etc.

If you strongly feel that the current code base is the better option, how would address the fundamental problems seen in the current code base, so that the provider could be successfully deployed to customer sites in the 3.5 release timeframe?

Things that still need a fair bit of work include:

*         Creating new schema and new data store

*         Spatial filter handling

*         Huge memory leaks on insert.

*         Not all schema commands are implemented.

*         Enable and fix transaction support.

*         Constraint and default values support.

*         Lots of TODOs spread all over the code

*         Virtually no unit tests exist.

Greg

From: fdo-internals-bounces at lists.osgeo.org [mailto:fdo-internals-bounces at lists.osgeo.org] On Behalf Of Haris Kurtagic
Sent: Monday, November 30, 2009 10:26 AM
To: FDO Internals Mail List
Subject: Re: [fdo-internals] FDO PostGIS provider developments

I wouldn't like to go to much into discussion about generic rdbms stuff. I think that priority should be having good PostGIS provider which we can get with current resources.

Just in short, when I started looking how to make provider for Oracle, Generic RDBMS was first place to start looking. For me Generic RDBMS was to complex , hard to work with and extend . Performance was also bad.
That was few years ago, I would assume that it has improved since but still I am afraid that Autodesk would be only maintaner of that code. That would be main reason why I would like to have current PostGIS available, which has not only Autodesk developers which could work on it.

Haris

On Mon, Nov 30, 2009 at 2:36 PM, Zac Spitzer <zac.spitzer at gmail.com<mailto:zac.spitzer at gmail.com>> wrote:
2009/12/1 Haris Kurtagic <haris at sl-king.com<mailto:haris at sl-king.com>>:
> Is it necessary to replace current version, can't that be another version of
> provider ?
> I was never able to get comfortable with generic rdbms stuff. In my opinion
> there are minuses too.
like?

> I would like to have option of current code base to be continued if there
> would be option at least for some time.
> Haris
>
> On Mon, Nov 30, 2009 at 2:14 AM, Orest Halustchak
> <orest.halustchak at autodesk.com<mailto:orest.halustchak at autodesk.com>> wrote:
>>
>> Hi,
>>
>>
>>
>> Autodesk has had interest from Map3D customers who want to use the FDO
>> PostGIS provider to access and edit data using Map3D and other products.
>>
>>
>>
>> These customers have not been able to use this provider reliably with
>> Map3D. We had a close look at the provider code to determine what work would
>> be needed to complete the implementation of schema and edit support and fix
>> other issues. Unfortunately, we found that the provider as it stands today
>> requires a lot of work to complete the implementation of required FDO
>> interfaces and to add good unit test coverage. Things that still need a fair
>> bit of work include:
>>
>> *         Creating new schema and new datastore
>>
>> *         Spatial filter handling
>>
>> *         Huge memory leaks on insert.
>>
>> *         Not all schema commands are implemented.
>>
>> *         Enable and fix transaction support.
>>
>> *         Constraint and default values support.
>>
>> *         Lots of TODOs spread all over the code
>>
>> *         Virtually no unit tests exist.
>>
>>
>>
>> We looked at the level of effort needed to complete that work. It was
>> quite high. So, we looked at an alternative. There exists an earlier open
>> source community code base for a PostGIS provider that was started a couple
>> of years ago but not finished. That code base used the generic rdbms
>> framework that is shared with the SQL Server Spatial, MySQL, and ODBC
>> providers. Most of the schema processing is handled with that shared code.
>> We spent some time working with the current provider and the other code base
>> to determine the most efficient way to get to a completed provider that
>> would be robust, perform well, and be maintainable.
>>
>>
>>
>> In the end, we determined that taking the earlier code base, adding
>> support for the recent fdo interface changes, and completing other parts
>> that weren't finished would take much less time. Also, based on performance
>> comparisons, we would get something that was much faster on inserts and
>> selects, e.g. the select performance is about six times faster and schema
>> describe is about three times faster. We couldn't compare insert times very
>> well because the current provider kept crashing after a certain point and we
>> couldn't insert a large number of features.
>>
>>
>>
>> So, what we would like to do is complete our work to get a working PostGIS
>> provider and then replace the current open source code with our new copy.
>> Note that we plan to use native PostGIS schema without adding additional
>> metadata tables just as the current provider does. It will be able to read
>> any schemas created by the current version of the provider and itself will
>> generate generic PostGIS schemas.
>>
>>
>>
>> We'll need to submit an RFC for this, but we wanted to get this
>> information out to you ahead of time.
>>
>>
>>
>> At the same time, we are planning to change the connection parameters to
>> separate out the database name from the service name. This will make it
>> easier for users. They can identify the service (e.g. localhost:5432), and
>> then see the available datastores from which they can choose in a UI. Then,
>> PostGIS schema simply will map to FDO schema. The main drawback to this is
>> that any users with existing MapGuide feature sources and layer definitions
>> will have to update them.
>>
>>
>>
>> In the end, we will end up with a good functioning provider that performs
>> well.
>>
>>
>>
>>
>>
>> Regards,
>>
>> Orest.
>>
>>
>>
>> _______________________________________________
>> fdo-internals mailing list
>> fdo-internals at lists.osgeo.org<mailto:fdo-internals at lists.osgeo.org>
>> http://lists.osgeo.org/mailman/listinfo/fdo-internals
>>
>
>
> _______________________________________________
> fdo-internals mailing list
> fdo-internals at lists.osgeo.org<mailto:fdo-internals at lists.osgeo.org>
> http://lists.osgeo.org/mailman/listinfo/fdo-internals
>
>

--
Zac Spitzer
Solution Architect / Director
Ennoble Consultancy Australia
http://www.ennoble.com.au
http://zacster.blogspot.com
+61 405 847 168
_______________________________________________
fdo-internals mailing list
fdo-internals at lists.osgeo.org<mailto:fdo-internals at lists.osgeo.org>
http://lists.osgeo.org/mailman/listinfo/fdo-internals

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.osgeo.org/pipermail/fdo-internals/attachments/20091130/a5af42c6/attachment.html