[fdo-internals] SHP performance problem may be due to .idx

[Dan Stoica]

It is not a bug.

Greg is right, the idx file timestamp is changed but the file is not recreated. I didn't measure but I doubt that just updating the header can introduce such penalty.

The only thing I can think of is Ticket #776: SHP provider: spatial index out of date is not recreated, submitted in 2011.

	In ShpFileSet::GetSpatialIndex() it reads:

            		// Check the timestamps for the SI and the SHP. The IDX file is always modified after SHP
           		// but it can get stale when the SHP has been edited by 3th party applications.

Could you please check, before doing Validate, that the IDX file timestamp is as recent as the SHP file timestamp? I don't have MG to confirm your findings, I just ran a SHP provider unit test in debug.

I didn't understand the " writing a total of 900 idx files" part. There should be only 30 idx files. 

Regards.


-----Original Message-----
From: fdo-internals-bounces at lists.osgeo.org [mailto:fdo-internals-bounces at lists.osgeo.org] On Behalf Of everling
Sent: Saturday, October 06, 2012 4:50 AM
To: fdo-internals at lists.osgeo.org
Subject: Re: [fdo-internals] SHP performance problem may be due to .idx

Is that a confirmation that it is a bug? I hope that this is an issue that may be fixed.

When I do a Validate on a SHP FeatureSource with 30 SHP files, MGOS 2.4 RC2 will recreate all 30 idx files for each SHP file, writing a total of 900 idx files when it did not have to write anything at all. Even marking the idx files as read-only did not mitigate this undesirable behaviour.

I apologise if I may be whiny, but it is quite painful to realise that MGOS is rewriting the idx files needlessly and having to wait long minutes for MGOS to stop chasing its tail to get a validation report, or to do anything else with SHP files where MGOS is determined to rewrite the idx files.

On an additional note, I have checked a MapGuide Enterprise 2011 installation to discover that it does not rewrite the idx files needlessly.
Is this a possible hint of regression?

Thank you.



--
View this message in context: http://osgeo-org.1560.n6.nabble.com/SHP-performance-problem-may-be-due-to-idx-tp5003321p5006817.html
Sent from the FDO Internals mailing list archive at Nabble.com.
_______________________________________________
fdo-internals mailing list
fdo-internals at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/fdo-internals