[mapguide-internals] RE: RFC110 Profiling API enhancement

Sun Apr 24 23:22:21 EDT 2011

Not sure if it's useful for MG server to store the profile results.
How about is just sending these data to client and visualize the stats at client side (like admin page).

Arthur

-----Original Message-----
From: mapguide-internals-bounces at lists.osgeo.org [mailto:mapguide-internals-bounces at lists.osgeo.org] On Behalf Of Zac Spitzer
Sent: Monday, April 25, 2011 10:55 AM
To: MapGuide Internals Mail List
Subject: Re: [mapguide-internals] RE: RFC110 Profiling API enhancement

I had this related idea floating around for a while, could we pipe this data in a db (sqlite?) and then use mapguide to then visualise these stats?

On Mon, Apr 25, 2011 at 12:49 PM, Arthur Liu <arthur.liu at autodesk.com> wrote:
> Hi Traian and Zac,
>
> Thanks for your feedback, these information are very helpful to me.
>
> As Zac said, besides developers of MG, end users could also get benefits from this profiling API. We have an idea about providing a dashboard to demonstrate the profile result, so that end users also could improve the map rendering performance by reorganizing the resources in map.
>
> I also has some concern about injecting too many code into current MG code. A higher proportion of profiling code coverage means more detailed information we could get.  It's very important that we should balance the two factor.
>
> Traian also raised a very good question about the CPU timing tool. I noticed that the timing tool used for debugging in MG was GetTickCount(), which works only on Windows platform and has accurate and overflow issues.  I would like to use QueryPerformanceFrequency() and QueryPerformanceCounter() API provided by Windows and gettimeofday() for Linux to implement an accurate timer.
>
> Taking the advantage of existing logging mechanism is interesting,  which also answers my second question about the MdfModel and MdfParser.  However, the current log information is not detailed enough as profiling result, not sure if an enriched log is useful for other end users who don't care the details. When it comes to multiple users, the content in log getting chaos, and the implementation of the analytics script could be complex.  It's a great suggestion, I will consider to this solution more deep.

>
> Regards,
> Arthur
>
> -----Original Message-----
> From: mapguide-internals-bounces at lists.osgeo.org 
> [mailto:mapguide-internals-bounces at lists.osgeo.org] On Behalf Of Zac 
> Spitzer
> Sent: Saturday, April 23, 2011 10:26 AM
> To: MapGuide Internals Mail List
> Subject: Re: [mapguide-internals] RE: RFC110 Profiling API enhancement
>
> absolutely we want to avoid any production overhead.
>
> it would be nice to use this information to profile builds, to see how 
> new features/fixes affect overall performance, similiar to what the 
> Firefox folks do
>
> I was thinking the detailed logging already caputures a lot of this information, in a verbose and unstructed manner, perhaps an analytics script for the log files could produce similiar info?
>
> On Sat, Apr 23, 2011 at 12:20 PM, Traian Stanev <traian.stanev at autodesk.com> wrote:
>>
>> In such case, my preference would be to write the code in such a way that this profiling can be either compiled out of the code or at least to keep it outside of the stylization and renderer core, in order to not force everyone to pay for the reporting overhead.
>>
>> Ideally, it would be something that wraps calls to the renderer/feature/resource service/etc. from the rendering service.
>>
>> Also, I would be interested to know what CPU timing tools will be used for this.
>>
>> Traian
>>
>>
>> -----Original Message-----
>> From: mapguide-internals-bounces at lists.osgeo.org
>> [mailto:mapguide-internals-bounces at lists.osgeo.org] On Behalf Of Zac 
>> Spitzer
>> Sent: Friday, April 22, 2011 9:56 PM
>> To: MapGuide Internals Mail List
>> Subject: Re: [mapguide-internals] RE: RFC110 Profiling API 
>> enhancement
>>
>> I read this as being something that could be wrapped up so end users could leverage this info for tuning maps. If the performance overhead is roughtly consistant, the information would still be very useful.
>>
>> Maestro already has some basic profiling in it, in MG Classic Author, being able to see how much data was being returned for a layer was a great way to optimise performance.
>>
>> It's a tangent off this RFC, but I think some sort of support for 
>> generalizing datasets would really help performance. Sort of like an 
>> automated layer atop a featuresource which allows the generalized 
>> data to be cached in sqlite? (fdo class naming would be a problem)
>>
>>
>> z
>>
>>
>>
>>
>>
>>
>> On Sat, Apr 23, 2011 at 11:34 AM, Traian Stanev <traian.stanev at autodesk.com> wrote:
>>>
>>> Hi Arthur,
>>>
>>> Who would be the target user of such an API? It seems to me it would be a MapGuide core developer interested in optimizing MapGuide performance. In such case, what would the API provide above and beyond the Visual Studio built-in sampling profiler?
>>>
>>> Wouldn't the time spent in injecting all those checkpoints in the code (which might themselves add a further performance overhead) be better spent optimizing the actual code using existing profiling tools, like the built-in Visual Studio profiler or VTune.
>>>
>>> Having spent a lot of time optimizing MapGuide rendering, I find that the above-mentioned tools do an excellent job already.
>>>
>>> Traian
>>>
>>>
>>> -----Original Message-----
>>> From: mapguide-internals-bounces at lists.osgeo.org
>>> [mailto:mapguide-internals-bounces at lists.osgeo.org] On Behalf Of 
>>> Arthur Liu
>>> Sent: Friday, April 22, 2011 2:37 AM
>>> To: MapGuide Internals Mail List
>>> Subject: [mapguide-internals] RFC110 Profiling API enhancement
>>
>>> Hi all,
>>>
>>> There's an RFC110(http://trac.osgeo.org/mapguide/wiki/MapGuideRfc110)  about providing some profiling APIs to help identify MapGuide performance bottlenecks.
>>>
>>> Before making this RFC be reviewed formally, I would like to know you experts' ideas about following questions:
>>>
>>>
>>> 1.      Which of the following implementation is better?
>>>
>>> a.      Solution 1:
>>>
>>> Copy the implementation of profiling related APIs to MgServerProfilingService and add profiling code into the copied version.
>>>
>>> For example, to profile the map rendering process, we need to copy the implementation of RenderMap in MgServerRenderingService(and all other MG server APIs being invoked in RenderMap) to MgServerProfilingService. The advantage and disadvantage of this implementation is obvious: less influence on the original implementation but heavy maintenance to do in the future.
>>>
>>>
>>>
>>> b.      Solution 2:
>>>
>>> Inject the profiling code into the original functions being profiled directly, which means MgServerServices should be extended to accept a ProfileResult and the implementation of some APIs will updated to support profiling.
>>>
>>> For example, to support profiling RenderMap, all the MgServerServices in the RenderMap workflow will be affected - MgServerRenderingService, MgServerResourceService, MgServerFeatureService, MgServerDescribeSchema, MgServerSelectFeatures, etc.  Some new internal interface like Get/SetProfileResult should be added to these MgServerServices.
>>>
>>>
>>> For both solutions, how many code should be copied or updated will be decided by how detailed the information would be gathered.
>>>
>>>
>>>
>>> 2.      Is it a good idea that we take the advantage of MdfModel and MdfParser to store the ProfileResult object model and do the serialization?
>>>
>>>
>>> Thanks,
>>> Arthur
>>> _______________________________________________
>>> mapguide-internals mailing list
>>> mapguide-internals at lists.osgeo.org
>>> http://lists.osgeo.org/mailman/listinfo/mapguide-internals
>>> _______________________________________________
>>> mapguide-internals mailing list
>>> mapguide-internals at lists.osgeo.org
>>> http://lists.osgeo.org/mailman/listinfo/mapguide-internals
>>>
>>
>>
>>
>> --
>> Zac Spitzer
>> Solution Architect / Director
>> Ennoble Consultancy Australia
>> http://www.ennoble.com.au
>> http://zacster.blogspot.com
>> +61 405 847 168
>> _______________________________________________
>> mapguide-internals mailing list
>> mapguide-internals at lists.osgeo.org
>> http://lists.osgeo.org/mailman/listinfo/mapguide-internals
>>
>> _______________________________________________
>> mapguide-internals mailing list
>> mapguide-internals at lists.osgeo.org
>> http://lists.osgeo.org/mailman/listinfo/mapguide-internals
>>
>>
>
>
>
> --
> Zac Spitzer
> Solution Architect / Director
> Ennoble Consultancy Australia
> http://www.ennoble.com.au
> http://zacster.blogspot.com
> +61 405 847 168
> _______________________________________________
> mapguide-internals mailing list
> mapguide-internals at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/mapguide-internals
>
> _______________________________________________
> mapguide-internals mailing list
> mapguide-internals at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/mapguide-internals
>
>

--
Zac Spitzer
Solution Architect / Director
Ennoble Consultancy Australia
http://www.ennoble.com.au
http://zacster.blogspot.com
+61 405 847 168
_______________________________________________
mapguide-internals mailing list
mapguide-internals at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapguide-internals