[mapguide-internals] Release Build not working???? Debug build is fine!!! Architectural Solution: RFC 63

[Traian Stanev]

Yes, that's all I wanted to point out -- the fact that it doesn't work for your map does not mean that it's a general problem, although it may be.

Before making a general solution out of making giant raster tiles and splitting them up, you also have to keep in mind that there are some algorithms in the renderer which scale quadratically -- for example labeling. So if a map has lots of labels, it will get significantly slower to render as you increase tile size. Yes, I know that it is *generally* faster to render one giant raster, because of the very high request overhead -- I used to test RenderMap with 5kx5k pixel requests. However, it is a lot slower if you have lots of labels. 

I will repeat my suggestion of reducing the number of worker threads to see if it alleviates the memory usage problem... It seems like the cheapest workaround in your case.

Traian 

________________________________________
From: mapguide-internals-bounces at lists.osgeo.org [mapguide-internals-bounces at lists.osgeo.org] On Behalf Of UV [uvwild at gmail.com]
Sent: Tuesday, April 21, 2009 10:56 PM
To: MapGuide Internals Mail List
Subject: Re: [mapguide-internals] Release Build not working???? Debug build     is fine!!! Architectural Solution: RFC 63

the windows analogy is just a similar scenario but for windows disk IO only.

In the mapguide case this is not just about disk I/O as also about
general resource usage (e.g. memory)
You are absolutely correct to point out that the caching strategies
implemented in various areas of the server
alleviate the disk I/O issues a lot. But obviously caching cost memory.

This assessment of the server behaviour is fully empirical based on
studying the processing of a very large map for quite a long time on
different machines now.
This is a performance and stability improvement that needs parameters
for tuning for the installation at hand. Not the same settings are
approprate for each and every server deployment.

However, observing the current failures and thinking ahead into the
future this seems to be a qay to go.
We cannot always solve problems by upgrading the memory of the server.
Furthermore this is an open source project so its doomed to be deployed
to servers with limited resources.
And the maps will not get smaller in the future either.

Traian Stanev wrote:
> I don't see how the analogy with Windows copying is right. If you have 10 threads reading the same piece of data from disk, it's most likely that the data is in the I/O cache, so no disk reads are really done after the first one. The other 9 threads would get the data in memory for free.
>
> Have you tried reducing the number of simultaneous threads in your server thread pool in order to alleviate your low memory condition?
>
>
> Traian
>
> ________________________________________
> From: mapguide-internals-bounces at lists.osgeo.org [mapguide-internals-bounces at lists.osgeo.org] On Behalf Of UV [uvwild at gmail.com]
> Sent: Tuesday, April 21, 2009 10:17 PM
> To: MapGuide Internals Mail List
> Subject: Re: [mapguide-internals] Release Build not working???? Debug build     is fine!!! Architectural Solution: RFC 63
>
> I finally came to the conclusion that this has been caused by resource
> limits. (the  infamous Out Of Memory exception)
>
> A memory limitation causes the mapping / rendering service to abort
> rendering the tile before all features are determined.
> Its very interesting that the serialization enforced by STDOUT of the
> debug build solves this problem already!
>
> Firstly this is a bug because the faulty Tile gets returned without an
> error message. (like #462)
>
> However, this gives us as reason to suggest a redesign of the Tiling
> service.
> Basically, the server does not do anything efficient if we start to
> request mutliple tiles of the same map and scale at the same time.
> This is like windows copying. If you start n parallel copy streams
> between the same disks you end up thrashing your disk.
>
> In our mapguide case this means that the server should maintain a
> request queue and serialize request to the same map and scale
> and maybe even compute 4 - 16 tiles at once and slice them up on the
> bitmap level.
> This does not seem very hard to do and will improve performance and
> stability of the mapguide server dramatically.
>
> See also RFC 63
>
> UV wrote:
>
>> Hi,
>>
>> now for the second time I tried to run my 8GB map sample in a release
>> build.
>> In the debug build the server renders all tiles just fine.
>>
>> When I did the same test with the release build the server created
>> only tiles in a single color - no more features....
>> (the sea is blue.... the desert is beige) the good news is that its
>> deterministic..... the same wrong tiles each time.
>>
>> I ran the test suite.... OK 430 tests
>> The server INFO Trace.log does not show anything special.
>>
>> The server is executed from the exact same location in the default
>> installation directory.... I verified the DLL versions....
>> No error messages... the access.log shows the error free access to the
>> tiles..... but the computed tiles are crap.
>>
>> Anyone seen this before?
>> Any idea what to look for? As its a release build and the debug
>> version is working sorting this out is a challenging task.
>> If there were any error messages this would be useful but there aint any.
>>
>> PS:
>> The server repeatedly creates an additional directory in the tilecache
>> called C-0 (as opposed to C0)
>> containing also very boring featureless tiles  named 0_-1.png to
>> 9_-1.png. This looks like a bug.
>> _______________________________________________
>> mapguide-internals mailing list
>> mapguide-internals at lists.osgeo.org
>> http://lists.osgeo.org/mailman/listinfo/mapguide-internals
>>
>>
>
> _______________________________________________
> mapguide-internals mailing list
> mapguide-internals at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/mapguide-internals
> _______________________________________________
> mapguide-internals mailing list
> mapguide-internals at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/mapguide-internals
>
>

_______________________________________________
mapguide-internals mailing list
mapguide-internals at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapguide-internals