[mapguide-internals] Release Build not working???? Debug build is fine!!! Architectural Solution: RFC 63

[UV]

the windows analogy is just a similar scenario but for windows disk IO only.

In the mapguide case this is not just about disk I/O as also about 
general resource usage (e.g. memory)
You are absolutely correct to point out that the caching strategies 
implemented in various areas of the server 
alleviate the disk I/O issues a lot. But obviously caching cost memory.

This assessment of the server behaviour is fully empirical based on 
studying the processing of a very large map for quite a long time on 
different machines now.
This is a performance and stability improvement that needs parameters 
for tuning for the installation at hand. Not the same settings are 
approprate for each and every server deployment.

However, observing the current failures and thinking ahead into the 
future this seems to be a qay to go.
We cannot always solve problems by upgrading the memory of the server.
Furthermore this is an open source project so its doomed to be deployed 
to servers with limited resources.
And the maps will not get smaller in the future either.

Traian Stanev wrote:
> I don't see how the analogy with Windows copying is right. If you have 10 threads reading the same piece of data from disk, it's most likely that the data is in the I/O cache, so no disk reads are really done after the first one. The other 9 threads would get the data in memory for free.
>
> Have you tried reducing the number of simultaneous threads in your server thread pool in order to alleviate your low memory condition?
>
>
> Traian
>
> ________________________________________
> From: mapguide-internals-bounces at lists.osgeo.org [mapguide-internals-bounces at lists.osgeo.org] On Behalf Of UV [uvwild at gmail.com]
> Sent: Tuesday, April 21, 2009 10:17 PM
> To: MapGuide Internals Mail List
> Subject: Re: [mapguide-internals] Release Build not working???? Debug build     is fine!!! Architectural Solution: RFC 63
>
> I finally came to the conclusion that this has been caused by resource
> limits. (the  infamous Out Of Memory exception)
>
> A memory limitation causes the mapping / rendering service to abort
> rendering the tile before all features are determined.
> Its very interesting that the serialization enforced by STDOUT of the
> debug build solves this problem already!
>
> Firstly this is a bug because the faulty Tile gets returned without an
> error message. (like #462)
>
> However, this gives us as reason to suggest a redesign of the Tiling
> service.
> Basically, the server does not do anything efficient if we start to
> request mutliple tiles of the same map and scale at the same time.
> This is like windows copying. If you start n parallel copy streams
> between the same disks you end up thrashing your disk.
>
> In our mapguide case this means that the server should maintain a
> request queue and serialize request to the same map and scale
> and maybe even compute 4 - 16 tiles at once and slice them up on the
> bitmap level.
> This does not seem very hard to do and will improve performance and
> stability of the mapguide server dramatically.
>
> See also RFC 63
>
> UV wrote:
>   
>> Hi,
>>
>> now for the second time I tried to run my 8GB map sample in a release
>> build.
>> In the debug build the server renders all tiles just fine.
>>
>> When I did the same test with the release build the server created
>> only tiles in a single color - no more features....
>> (the sea is blue.... the desert is beige) the good news is that its
>> deterministic..... the same wrong tiles each time.
>>
>> I ran the test suite.... OK 430 tests
>> The server INFO Trace.log does not show anything special.
>>
>> The server is executed from the exact same location in the default
>> installation directory.... I verified the DLL versions....
>> No error messages... the access.log shows the error free access to the
>> tiles..... but the computed tiles are crap.
>>
>> Anyone seen this before?
>> Any idea what to look for? As its a release build and the debug
>> version is working sorting this out is a challenging task.
>> If there were any error messages this would be useful but there aint any.
>>
>> PS:
>> The server repeatedly creates an additional directory in the tilecache
>> called C-0 (as opposed to C0)
>> containing also very boring featureless tiles  named 0_-1.png to
>> 9_-1.png. This looks like a bug.
>> _______________________________________________
>> mapguide-internals mailing list
>> mapguide-internals at lists.osgeo.org
>> http://lists.osgeo.org/mailman/listinfo/mapguide-internals
>>
>>     
>
> _______________________________________________
> mapguide-internals mailing list
> mapguide-internals at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/mapguide-internals
> _______________________________________________
> mapguide-internals mailing list
> mapguide-internals at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/mapguide-internals
>
>