[mapguide-users] perfomance test

Sun Oct 29 19:10:10 EST 2006

Oh I think it's a perfectly reasonable test to do, and I do very similar things to measure performance of providers. However, one has to recognize that in a well written provider, the majority of time would be going into the underlying database, which the provider writer can do very little about. In the end, if performance is not good enough, one can always convert the data to SDF for publishing as a map with MapGuide. :-)

Traian

	-----Original Message----- 
	From: Haris Kurtagic [mailto:haris at sl-king.com] 
	Sent: Sun 10/29/2006 6:30 PM 
	To: users at mapguide.osgeo.org 
	Cc: 
	Subject: RE: [mapguide-users] perfomance test

	I've tried to loop 100  times and got little difference not much.

	I suppose this was not good test at the end, If I could I would delete first email.
	And again, this was test for testing OSGEO.SHP provider OSGEO.SDF provider and King.Oracle provider and not underlying databases.

	P.S.
	I  now know why EULA is not allowing everyone to publish test results. :)

	Haris

  _____  

	From: Traian Stanev [mailto:traian.stanev at autodesk.com] 
	Sent: Sunday, October 29, 2006 11:48 PM
	To: users at mapguide.osgeo.org; users at mapguide.osgeo.org
	Subject: RE: [mapguide-users] perfomance test

	If you are looping 16 times and then dividing the total time by 16, then it's probably fine. Your timing includes both executing the select command and then reading through the resulting feature reader?

	Traian

		-----Original Message----- 
		From: Haris Kurtagic [mailto:haris at sl-king.com] 
		Sent: Sun 10/29/2006 5:40 PM 
		To: users at mapguide.osgeo.org 
		Cc: 
		Subject: RE: [mapguide-users] perfomance test

		Thank you for all your suggestions. This is my first test so :)

		Yes, I also think that other factors have more influence.
		I' ve used clock() on windows and I ran it in loop of 16.

		I suppose I should take some much larger data set.

		Thanks,
		Haris

  _____  

		From: Traian Stanev [mailto:traian.stanev at autodesk.com] 
		Sent: Sunday, October 29, 2006 11:32 PM
		To: users at mapguide.osgeo.org; users at mapguide.osgeo.org
		Subject: RE: [mapguide-users] perfomance test

		OK. All I was saying is that EnvelopeIntersects does not incur a secondary geometry filter processing for each feature in the SDF provider, which Intersects does. I would have expected a significant difference between Intersects and EnvelopeIntersects for the SDF provider. Also, EnvelopeIntersects is what MapGuide uses to render every map.

		About the FGF thing -- yes SDF has the luxury of not having to convert geometry formats, so in the limit, it would be faster than a provider that converts, but (in my opinion) there are many other factors which have a much bigger impact on performance, than a simple geometry conversion loop.

		Also, another question... How do you do the timing? If you use the Windows clock, it has a resolution of 16 milliseconds, which makes the accuracy of the 0.031 sec measurement for SDF very doubtful to me, unless you repeated the operation 100 times and then took the total time and divided by 100, for example.

		Traian

			-----Original Message----- 
			From: Haris Kurtagic [mailto:haris at sl-king.com] 
			Sent: Sun 10/29/2006 5:18 PM 
			To: users at mapguide.osgeo.org 
			Cc: 
			Subject: RE: [mapguide-users] perfomance test

			Yes, I've  tried envelope but I wanted to try more complicated query (to use primary and secondary filter in Oracle) and also couldn't see huge difference on this small set.

			If I understood you correctly you mean sdf provider is not loosing time in converting to FGF format ?
			I agree on that but that is format use in MG, so that is one + for sdf :)

			Haris

  _____  

			From: Traian Stanev [mailto:traian.stanev at autodesk.com] 
			Sent: Sunday, October 29, 2006 11:13 PM
			To: users at mapguide.osgeo.org; users at mapguide.osgeo.org
			Subject: RE: [mapguide-users] perfomance test

			Did you try an EnvelopeIntersects filter? That one is by far the most common one, rather than Intersects.
			It is not surprising that SDF is fastest by far... The SDF provider does not add any intelligence to the data and stores everything in FGF format so you are basically getting back a pointer into database memory when you ask for the geometry.

			Traian

				-----Original Message----- 
				From: Haris Kurtagic [mailto:haris at sl-king.com] 
				Sent: Sun 10/29/2006 4:57 PM 
				To: users at mapguide.osgeo.org 
				Cc: 
				Subject: [mapguide-users] perfomance test

				Hi,

				After some discussion's here on this list I got interested in comparing different data source and fdo providers.

				So, I have used an test application written as pure FDO client.
				I have run test again sdf, shp , 10gr2 and xe, using osgeo sdf, shp and king.oracle provider.
				Data used in test is Sheboygan_Parcels.sdf from MG unittest data.

				Before I wrote the results I would like to make strong point that this could be altogether false results, I don't know anything about writing true fair tests.
				As my lawyer told me I should put in here : whatever is written is not promise, could be wrong and ..... :)

				17565 features read in case 1,2, 6681 fetaures in 3,4
				results are in seconds

				Test case 1: No geometry filter, Fetch all attributes 

				SDF        0.937 

				SHP        8.531 
				10GR2    18.172
				XE            18.500

				Test case 2: No geometry filter, Fetch Geometry and Identity

				SDF        0.031 

				SHP        0.109 
				10GR2    1.938
				XE           1.735

				Test Case 3:  Geometry Filter, Fetch all attributes

				SDF        0.390
				SHP        3.391
				10GR2     6.906
				XE            6.218

				Test Case 4:  Geometry Filter, Fetch Geometry and Identity

				SDF        0.031
				SHP        0.140
				10GR2     0.891
				XE           0.797

				Geometry Filter was: SHPGEOM INTERSECTS GeomFromText('POLYGON ((-87.72 44, -86 44, -86 42, -87.72 42, -87.72 44))')
				and for SDF and King.Oracle it returns same number of features 6683 which is good
				shape provider returned : 6681 ?

				Original SDF file was copied to Oracle using that same tool, I call it Fdo2Sdo.
				Oracle 10GR2 is running on my same desktop in virtual machine and XE on localhost.

				My Remarks:
				SDF is really fast and highly optimized for this kind of queries. Test Case 4 would be the most used case in real word app. and differences there are smallest.
				I think if you add overhead of MG and other overheads than influence would be even smaller.
				I think these tests are very unfavorable to Oracle.
				This test is run with very small data set (just 17565 polygons), I believe much larger data sets would be more preferable to Oracle.
				I also couldn't run test with ordering or grouping (sdf,shp don't support it). I run this test immediately after installing Oracle, not using any spatial index optimization
				and also provider is not highly optimized as could be.
				Here I am comparing file system to database ( it is little unusual ), It would make more sense to test against other db.

				I will make this tool freely available very soon, so you will be able to run test by your self's, and also it would be nice to test other data sources and providers.

				I hope that someone will find this tests useful.
				I will welcome every comment on this and if I made this tests wrongly please do tell me.

				Haris

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.osgeo.org/pipermail/mapguide_users/attachments/20061029/61f961ca/attachment.html