[fdo-users] Improving performance for NEN1878 provider, howto?
Piet Bibber
piet.bibber at gmail.com
Thu Oct 4 14:25:19 EDT 2007
Thanks Badreddine,
That doesn't really work :-(.
No schema is fetched.
I will try with the UnitTest's to get some more performance statistics
// Piet
On 10/4/07, Badreddine Karoui <badreddine.karoui at autodesk.com> wrote:
>
> Try replace the SHP provider dll in the providers xml file with you
> provider's dll and see if that makes a difference. Keep the provider name as
> is. Now, Map will think your provider is the SHP provider and you will need
> to provide the connection parameters through the SHP connect dialog or just
> hard code them in your connection object.
>
>
>
> Badreddine
>
>
>
> *From:* fdo-users-bounces at lists.osgeo.org [mailto:
> fdo-users-bounces at lists.osgeo.org] *On Behalf Of *Piet Bibber
> *Sent:* Thursday, October 04, 2007 6:17 AM
> *To:* FDO Users Mail List
> *Subject:* Re: [fdo-users] Improving performance for NEN1878 provider,
> howto?
>
>
>
> Dan,
>
>
>
> SelectAggregates does instanciate the OptimizedAggregateReader (at the
> bottom of the Execute command).
>
>
>
> Where I believe it is very slow is in getting data (through the IsNull and
> GetData methods);
>
> The dialog with "Polulating Feature Cache" is on screen during 3 to 5
> seconds; the same dataset in sdf takes sub seconds to load. The " Polulating
> Feature Cache" dialog doesn't even appear.
>
> My features are all in memory, no disk access is needed, yet is multiple
> times slower.
>
>
>
> i must be doing something wrong.
>
>
>
> When I return true to IsNull (and avoid the GetData call), its still
> slow....
>
>
>
> What else can I do to trace performance or to make GetData faster?
>
>
>
> Thanks for the help so far
>
> Piet
>
>
>
>
>
>
> On 10/1/07, *Dan Stoica* <dan.stoica at autodesk.com> wrote:
>
> Indeed, OptimizedAggregateReader is the place. Make sure your
> SelectAggregates command is instantiating such a reader.
>
>
>
> Dan.
>
>
>
> *From:* fdo-users-bounces at lists.osgeo.org [mailto:
> fdo-users-bounces at lists.osgeo.org] *On Behalf Of *Piet Bibber
> *Sent:* Monday, October 01, 2007 10:16 AM
> *To:* FDO Users Mail List
> *Subject:* Re: [fdo-users] Improving performance for NEN1878 provider,
> howto?
>
>
>
> Robert, Badreddine, Gavin
>
>
>
> I do provide the extends and counts in Nen1878OptimizedAggregateReader. Is
> that what you refer to?
>
> Piet
>
>
>
>
>
> On 10/1/07, *Robert Fortin* <robert.fortin at autodesk.com> wrote:
>
> Badreddine,
>
> Isn't it the combination of count and spatialExtents that need to be
> optimized?
> I believe Map looks for the equivalent of
>
> Select SpatialExtents(geometry), count(*) from <class>
>
> RF
>
>
> -----Original Message-----
> From: fdo-users-bounces at lists.osgeo.org
> [mailto:fdo-users-bounces at lists.osgeo.org] On Behalf Of Badreddine
> Karoui
> Sent: Monday, October 01, 2007 9:42 AM
> To: fdo-users at lists.osgeo.org
> Subject: RE: [fdo-users] Improving performance for NEN1878 provider,
> howto?
>
> Hi,
>
> You may want to implement the FdoISelectAggregates command plus the
> SpatialExtents and Count functions. If these two functions are not
> implemented, then Map will scan all the geometries to find the minimum
> bounding rectangle of all the data.
>
> Badreddine
>
> -----Original Message-----
> From: fdo-users-bounces at lists.osgeo.org
> [mailto:fdo-users-bounces at lists.osgeo.org] On Behalf Of Piet Bibber
> Sent: Monday, October 01, 2007 7:46 AM
> To: fdo-users at lists.osgeo.org
> Subject: [fdo-users] Improving performance for NEN1878 provider, howto?
>
>
> Hello,
>
> My NEN1878 provider is almost done, but performance is not so fantastic
> when
> reading the geometry; reading the schema is alright.
> That is stange, as I cache in the geometry when I read the schema
> (NEN1878
> schema discovery requires a full file scan; no seperate schema header).
> (The
> schema is read within 3 seconds for a 2.3MB file, including the geometry
> and
> attributes (all in memory (using STL)).
> The SDF and SHP all seem so fast :-)
>
> Map 3D seem to spend most of his time caching in the geometry (see
> source
> code below)
>
> I based my provider on the SHP provider, the reader seem to use
> FDO_READER
> template, but I can't find where that points to. Maybe there are better
> optimized readers.
>
> Thanks.
> Piet
>
> Excerpt from the source files:
>
> FdoByteArray * CreateCachedGeometry(FdoByteArray* fgfByteArray_I)
> {
> FdoByteArray* fgfByteArray = fgfByteArray_I; // it may be
> reallocated
> int count = mFeature->GetNumPoints();
> FdoInt32 numPointOrds = 2;
> FdoInt32 numBytesForOrdinates = numPointOrds *
> sizeof(double)
> * count;
> FdoInt32 numParts = 0;
> FdoGeometryType fdoGeomType = GetFdoGeometryType();
> FdoInt32 dimensionality = FdoDimensionality_XY;
> FdoInt32 numPositions = mFeature->GetNumPoints();
>
> fgfByteArray->Clear();
>
> fgfByteArray = FdoByteArray::Append(fgfByteArray,
> sizeof(fdoGeomType), (FdoByte *) &fdoGeomType);
>
> switch (fdoGeomType)
> {
> case FdoGeometryType_None:
> break;
>
> case FdoGeometryType_Point:
> fgfByteArray = FdoByteArray::Append(fgfByteArray,
> sizeof(dimensionality), (FdoByte*)&dimensionality);
> fgfByteArray = FdoByteArray::Append(fgfByteArray,
> numBytesForOrdinates, (FdoByte*)mFeature->GetPoints());
> break;
>
> case FdoGeometryType_LineString:
> fgfByteArray = FdoByteArray::Append(fgfByteArray,
> sizeof(dimensionality), (FdoByte *) &dimensionality);
> fgfByteArray = FdoByteArray::Append(fgfByteArray,
> sizeof(count),
> (FdoByte *)&count);
> fgfByteArray = FdoByteArray::Append(fgfByteArray,
> numBytesForOrdinates, (FdoByte *)mFeature->GetPoints());
> break;
>
> default:
> throw
> FdoException::Create(FdoException::NLSGetMessage(FDO_NLSID(FDO_10_UNSUPP
> ORTEDGEOMETRYTYPE)));
> }
>
> return (fgfByteArray);
> }
>
>
> // Returns pointer to vector of DoublePoints
> inline DoublePoint* Nen1878Feature::GetPoints()
> {
> return (&mCoordinates[0]);
> }
>
> --
> View this message in context:
> http://www.nabble.com/Improving-performance-for-NEN1878-provider%2C-howt
> o--tf4547817s18162.html#a12977669
> Sent from the fdo-users mailing list archive at Nabble.com<http://nabble.com/>
> .
>
> _______________________________________________
> fdo-users mailing list
> fdo-users at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/fdo-users
>
> _______________________________________________
> fdo-users mailing list
> fdo-users at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/fdo-users
>
> _______________________________________________
> fdo-users mailing list
> fdo-users at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/fdo-users
>
>
>
>
> _______________________________________________
> fdo-users mailing list
> fdo-users at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/fdo-users
>
>
>
> _______________________________________________
> fdo-users mailing list
> fdo-users at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/fdo-users
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.osgeo.org/pipermail/fdo-users/attachments/20071004/dce6bc87/attachment-0001.html
More information about the fdo-users
mailing list