[fdo-users] Improving performance for NEN1878 provider, howto?

Thu Oct 4 09:56:51 EDT 2007

"What else can I do to trace performance or to make GetData faster?"

Do you have a profiling tool to run against a simple FDO test? If not,
use some timers and narrow down the bottleneck. E.g.

-          How long takes SpatialExtents(), Count()

-          How long takes IsNull()

-          How long takes GetGeometry()

Dan.

From: fdo-users-bounces at lists.osgeo.org
[mailto:fdo-users-bounces at lists.osgeo.org] On Behalf Of Badreddine
Karoui
Sent: Thursday, October 04, 2007 9:20 AM
To: FDO Users Mail List
Subject: RE: [fdo-users] Improving performance for NEN1878 provider,
howto?

Try replace the SHP provider dll in the providers xml file with you
provider's dll and see if that makes a difference. Keep the provider
name as is. Now, Map will think your provider is the SHP provider and
you will need to provide the connection parameters through the SHP
connect dialog or just hard code them in your connection object.

Badreddine

From: fdo-users-bounces at lists.osgeo.org
[mailto:fdo-users-bounces at lists.osgeo.org] On Behalf Of Piet Bibber
Sent: Thursday, October 04, 2007 6:17 AM
To: FDO Users Mail List
Subject: Re: [fdo-users] Improving performance for NEN1878 provider,
howto?

Dan,

SelectAggregates does instanciate the OptimizedAggregateReader  (at the
bottom of the Execute command).

Where I believe it is very slow is in getting data (through the IsNull
and GetData methods); 

The dialog with "Polulating Feature Cache" is on screen during 3 to 5
seconds; the same dataset in sdf takes sub seconds to load. The "
Polulating Feature Cache" dialog doesn't even appear.

My features are all in memory, no disk access is needed, yet is multiple
times slower.

i must be doing something wrong.

When I return true to IsNull (and avoid the GetData call), its still
slow....

What else can I do to trace performance or to make GetData faster?

Thanks for the help so far

Piet

On 10/1/07, Dan Stoica <dan.stoica at autodesk.com> wrote: 

Indeed, OptimizedAggregateReader is the place. Make sure your
SelectAggregates command is instantiating such a reader.

Dan.

From: fdo-users-bounces at lists.osgeo.org
[mailto:fdo-users-bounces at lists.osgeo.org] On Behalf Of Piet Bibber 
Sent: Monday, October 01, 2007 10:16 AM
To: FDO Users Mail List
Subject: Re: [fdo-users] Improving performance for NEN1878 provider,
howto?

Robert, Badreddine, Gavin

I do provide the extends and counts in Nen1878OptimizedAggregateReader.
Is that what you refer to?

Piet

On 10/1/07, Robert Fortin <robert.fortin at autodesk.com> wrote: 

Badreddine,

Isn't it the combination of count and spatialExtents that need to be
optimized?
I believe Map looks for the equivalent of

Select SpatialExtents(geometry), count(*)  from <class> 

RF

-----Original Message-----
From: fdo-users-bounces at lists.osgeo.org 
[mailto:fdo-users-bounces at lists.osgeo.org] On Behalf Of Badreddine
Karoui
Sent: Monday, October 01, 2007 9:42 AM 
To: fdo-users at lists.osgeo.org
Subject: RE: [fdo-users] Improving performance for NEN1878 provider, 
howto?

Hi,

You may want to implement the FdoISelectAggregates command plus the
SpatialExtents and Count functions. If these two functions are not 
implemented, then Map will scan all the geometries to find the minimum 
bounding rectangle of all the data.

Badreddine

-----Original Message-----
From: fdo-users-bounces at lists.osgeo.org
[mailto:fdo-users-bounces at lists.osgeo.org] On Behalf Of Piet Bibber 
Sent: Monday, October 01, 2007 7:46 AM
To: fdo-users at lists.osgeo.org
Subject: [fdo-users] Improving performance for NEN1878 provider, howto? 

Hello,

My NEN1878 provider is almost done, but performance is not so fantastic
when
reading the geometry; reading the schema is alright. 
That is stange, as I cache in the geometry when I read the schema 
(NEN1878
schema discovery requires a full file scan; no seperate schema header).
(The
schema is read within 3 seconds for a 2.3MB file, including the geometry

and
attributes (all in memory (using STL)). 
The SDF and SHP all seem so fast :-)

Map 3D seem to spend most of his time caching in the geometry (see
source
code below)

I based my provider on the SHP provider, the reader seem to use 
FDO_READER 
template, but I can't find where that points to. Maybe there are better
optimized readers.

Thanks.
Piet

Excerpt from the source files:

   FdoByteArray * CreateCachedGeometry(FdoByteArray* fgfByteArray_I) 
   {
       FdoByteArray*   fgfByteArray = fgfByteArray_I; // it may be
reallocated
       int             count = mFeature->GetNumPoints();
       FdoInt32        numPointOrds = 2;
       FdoInt32        numBytesForOrdinates = numPointOrds * 
sizeof(double)
* count;
       FdoInt32        numParts = 0;
       FdoGeometryType fdoGeomType  = GetFdoGeometryType();
       FdoInt32        dimensionality = FdoDimensionality_XY;
       FdoInt32        numPositions   = mFeature->GetNumPoints(); 

       fgfByteArray->Clear();

       fgfByteArray = FdoByteArray::Append(fgfByteArray,
sizeof(fdoGeomType), (FdoByte *) &fdoGeomType);

       switch (fdoGeomType)
       {
       case FdoGeometryType_None: 
           break;

           case FdoGeometryType_Point:
           fgfByteArray = FdoByteArray::Append(fgfByteArray,
sizeof(dimensionality), (FdoByte*)&dimensionality);
           fgfByteArray = FdoByteArray::Append(fgfByteArray, 
numBytesForOrdinates,   (FdoByte*)mFeature->GetPoints());
                   break;

           case FdoGeometryType_LineString:
           fgfByteArray = FdoByteArray::Append(fgfByteArray,
sizeof(dimensionality), (FdoByte *) &dimensionality); 
           fgfByteArray = FdoByteArray::Append(fgfByteArray,
sizeof(count),
(FdoByte *)&count);
           fgfByteArray = FdoByteArray::Append(fgfByteArray,
numBytesForOrdinates, (FdoByte *)mFeature->GetPoints()); 
                   break;

           default:
           throw
FdoException::Create(FdoException::NLSGetMessage(FDO_NLSID(FDO_10_UNSUPP
ORTEDGEOMETRYTYPE)));
       }

           return (fgfByteArray); 
   }

// Returns pointer to vector of DoublePoints
inline DoublePoint* Nen1878Feature::GetPoints()
{
   return (&mCoordinates[0]);
}

--
View this message in context:
http://www.nabble.com/Improving-performance-for-NEN1878-provider%2C-howt
o--tf4547817s18162.html#a12977669
Sent from the fdo-users mailing list archive at Nabble.com
<http://nabble.com/> .

_______________________________________________ 
fdo-users mailing list
fdo-users at lists.osgeo.org 
http://lists.osgeo.org/mailman/listinfo/fdo-users

_______________________________________________ 
fdo-users mailing list
fdo-users at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/fdo-users 

_______________________________________________
fdo-users mailing list
fdo-users at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/fdo-users

_______________________________________________
fdo-users mailing list
fdo-users at lists.osgeo.org 
http://lists.osgeo.org/mailman/listinfo/fdo-users

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.osgeo.org/pipermail/fdo-users/attachments/20071004/fc6ae00f/attachment-0001.html