[fdo-internals] RE: [fdo-users] Re: SDE and slow multi-geometries

[Greg Boone]

Hi Oyvind,

I have committed your changes to the FDO Trunk. At this point there is no plan to back port them to FDO 3.6. That may change depending on your needs.

Refer to:

http://trac.osgeo.org/fdo/changeset/6245

Regards,
Greg

From: fdo-users-bounces at lists.osgeo.org [mailto:fdo-users-bounces at lists.osgeo.org] On Behalf Of Oyvind Idland
Sent: Wednesday, June 22, 2011 4:09 AM
To: FDO Users Mail List
Subject: Re: [fdo-users] Re: SDE and slow multi-geometries

Hello,

I just submitted a patch, see Ticket #735. Sorry for taking to long time !

The patch was created from 3.5.0, so it should be merged into newer versions.

As mentioned in a different thread, some of the unit-tests crashes (not related to this patch) and problems related to this should probably be fixed by somebody.

The new code is affected by the tests that works, and has been running in our production environment for a while without problems.


-- Oyvind
On Tue, Feb 22, 2011 at 4:36 PM, Greg Boone <greg.boone at autodesk.com<mailto:greg.boone at autodesk.com>> wrote:
>> I assume there are tests that validates geometries

I would not assume this. You may want to take a look and see if those tests exist.

Greg

From: fdo-users-bounces at lists.osgeo.org<mailto:fdo-users-bounces at lists.osgeo.org> [mailto:fdo-users-bounces at lists.osgeo.org<mailto:fdo-users-bounces at lists.osgeo.org>] On Behalf Of Oyvind Idland
Sent: Saturday, February 19, 2011 9:02 AM

To: FDO Users Mail List
Subject: Re: [fdo-users] Re: SDE and slow multi-geometries

Hello Dan and Greg,

we are currently using 3.5, and we are in the middle of a release. I could patch 3.5 locally and use
our own binaries.

I'll try to run the test suite during beginning of the week, and check if the tests triggers new code. I assume
there are tests that validates geometries. I can add necessary test code if I find that something is left out.

After testing I can attach the patch.

PS the memory leak I mentioned was fixed in a different way than I described.


-- Oyvind


On Fri, Feb 18, 2011 at 5:23 PM, Dan Stoica <dan.stoica at autodesk.com<mailto:dan.stoica at autodesk.com>> wrote:
Hi Oyvind,

In addition, before submitting the patch please run the unit tests suite.
Does your change needs a specific unit test thus making sure it is exercised?

Dan.

From: fdo-users-bounces at lists.osgeo.org<mailto:fdo-users-bounces at lists.osgeo.org> [mailto:fdo-users-bounces at lists.osgeo.org<mailto:fdo-users-bounces at lists.osgeo.org>] On Behalf Of Greg Boone
Sent: Friday, February 18, 2011 10:19 AM

To: FDO Users Mail List
Subject: RE: [fdo-users] Re: SDE and slow multi-geometries

Can you create an svn patch and attach it to the ticket related to this defect? Once that is done I can have a few people look at the changes and have it submitted.

At this point these changes can go into the trunk, and possibly the 3.6 branch. Is there a requirement to have the changes ported back to 3.5?

Greg

From: fdo-users-bounces at lists.osgeo.org<mailto:fdo-users-bounces at lists.osgeo.org> [mailto:fdo-users-bounces at lists.osgeo.org<mailto:fdo-users-bounces at lists.osgeo.org>] On Behalf Of Oyvind Idland
Sent: Friday, February 18, 2011 5:05 AM
To: FDO Users Mail List
Subject: [fdo-users] Re: SDE and slow multi-geometries

The code is now fixed. As I got started, I also did some related code cleanup:

- the macro function EXPAND() are now gone. Instead, I made a simple templated
array class with an array subscript operator. This looks much cleaner now and can be
debugged.


- the macro function ADD_POINT is also gone, replaced with an inline static function.


- memory leak: ArcSDEConnection, line 885:

CHAR *qualified_table_name = new CHAR[SE_QUALIFIED_TABLE_NAME+1];

This one never gets deleted. I can't see any reason to use new, so I replaced it
with an array instead:

CHAR qualified_table_name [SE_QUALIFIED_TABLE_NAME+1];



However, there are still things I'd like to do with the code: The geometry buffering is
attached to the ArcSDEConnection class, and functions like convert_sde_shape_to_fgf()
are poking around in these arrays (XY, Z, parts etc). What I would like to do, is to
wrap all these arrays together with the indexing code into a new SDE geometry
handling class. This won't add anything new, just make it more clean.


Where can I submit/send the changes ?


-- Oyvind

On Wed, Feb 16, 2011 at 4:53 PM, Oyvind Idland <oyvind.idland at gmail.com<mailto:oyvind.idland at gmail.com>> wrote:
Hello,

I have been looking into an issue that was brought up here earlier. Turns out that the SDE API
is slower than cold molassis when using SE_shape_get_num_points() on multipart geometries.

I did some experimentation using the index tables returned from SE_shape_get_all_points()
directly. The total processing speed on my dataset went from roughly 20 secs down to less than
1 sec. There are several huge polygons with 200+ holes in them, and multilines.

Why the native SDE API is so slow is a mystery, those indices should not require heavy
calculation.

The code is experimental so far, but when I finish writing the code, I can submit a patch
if desired.


-- Øyvind Idland


_______________________________________________
fdo-users mailing list
fdo-users at lists.osgeo.org<mailto:fdo-users at lists.osgeo.org>
http://lists.osgeo.org/mailman/listinfo/fdo-users


_______________________________________________
fdo-users mailing list
fdo-users at lists.osgeo.org<mailto:fdo-users at lists.osgeo.org>
http://lists.osgeo.org/mailman/listinfo/fdo-users

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.osgeo.org/pipermail/fdo-internals/attachments/20110912/4de1d371/attachment.html