[mapguide-internals] Release Build not working???? Debug build is fine!!! Architectural Solution: RFC 63

[Traian Stanev]

Is it thrashing when saving all the tile images or when rendering maps? In the former case there really isn't much you can do -- even if you make giant tiles with MapGuide, you still have to split them up into small tiles and save them to disk. I guess the way to test that would be to render all the tiles without saving them to disk and see if it is still thrashing.

Traian

________________________________________
From: mapguide-internals-bounces at lists.osgeo.org [mapguide-internals-bounces at lists.osgeo.org] On Behalf Of Zac Spitzer [zac.spitzer at gmail.com]
Sent: Tuesday, April 21, 2009 10:44 PM
To: MapGuide Internals Mail List
Subject: Re: [mapguide-internals] Release Build not working???? Debug build is  fine!!! Architectural Solution: RFC 63

In my experience mapguide always thrashes the disk when tiling using my seeder
which is always requesting the same scale and area

depending on the way the data is read, unless you have a very small
dataset, the I/O cache
is going to be probably too small...

watching the IO reads / writes under task manager I often see
100-200mbs (ish approx )
a second on a raid 10 array..

in 32-bit land there isn't going to be a lot of memory for the I/O cache

z

On Wed, Apr 22, 2009 at 12:30 PM, Traian Stanev
<traian.stanev at autodesk.com> wrote:
>
> I don't see how the analogy with Windows copying is right. If you have 10 threads reading the same piece of data from disk, it's most likely that the data is in the I/O cache, so no disk reads are really done after the first one. The other 9 threads would get the data in memory for free.
>
> Have you tried reducing the number of simultaneous threads in your server thread pool in order to alleviate your low memory condition?
>
>
> Traian
>
> ________________________________________
> From: mapguide-internals-bounces at lists.osgeo.org [mapguide-internals-bounces at lists.osgeo.org] On Behalf Of UV [uvwild at gmail.com]
> Sent: Tuesday, April 21, 2009 10:17 PM
> To: MapGuide Internals Mail List
> Subject: Re: [mapguide-internals] Release Build not working???? Debug build     is fine!!! Architectural Solution: RFC 63
>
> I finally came to the conclusion that this has been caused by resource
> limits. (the  infamous Out Of Memory exception)
>
> A memory limitation causes the mapping / rendering service to abort
> rendering the tile before all features are determined.
> Its very interesting that the serialization enforced by STDOUT of the
> debug build solves this problem already!
>
> Firstly this is a bug because the faulty Tile gets returned without an
> error message. (like #462)
>
> However, this gives us as reason to suggest a redesign of the Tiling
> service.
> Basically, the server does not do anything efficient if we start to
> request mutliple tiles of the same map and scale at the same time.
> This is like windows copying. If you start n parallel copy streams
> between the same disks you end up thrashing your disk.
>
> In our mapguide case this means that the server should maintain a
> request queue and serialize request to the same map and scale
> and maybe even compute 4 - 16 tiles at once and slice them up on the
> bitmap level.
> This does not seem very hard to do and will improve performance and
> stability of the mapguide server dramatically.
>
> See also RFC 63
>
> UV wrote:
>> Hi,
>>
>> now for the second time I tried to run my 8GB map sample in a release
>> build.
>> In the debug build the server renders all tiles just fine.
>>
>> When I did the same test with the release build the server created
>> only tiles in a single color - no more features....
>> (the sea is blue.... the desert is beige) the good news is that its
>> deterministic..... the same wrong tiles each time.
>>
>> I ran the test suite.... OK 430 tests
>> The server INFO Trace.log does not show anything special.
>>
>> The server is executed from the exact same location in the default
>> installation directory.... I verified the DLL versions....
>> No error messages... the access.log shows the error free access to the
>> tiles..... but the computed tiles are crap.
>>
>> Anyone seen this before?
>> Any idea what to look for? As its a release build and the debug
>> version is working sorting this out is a challenging task.
>> If there were any error messages this would be useful but there aint any.
>>
>> PS:
>> The server repeatedly creates an additional directory in the tilecache
>> called C-0 (as opposed to C0)
>> containing also very boring featureless tiles  named 0_-1.png to
>> 9_-1.png. This looks like a bug.
>> _______________________________________________
>> mapguide-internals mailing list
>> mapguide-internals at lists.osgeo.org
>> http://lists.osgeo.org/mailman/listinfo/mapguide-internals
>>
>
> _______________________________________________
> mapguide-internals mailing list
> mapguide-internals at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/mapguide-internals
> _______________________________________________
> mapguide-internals mailing list
> mapguide-internals at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/mapguide-internals
>



--
Zac Spitzer -
http://zacster.blogspot.com
+61 405 847 168
_______________________________________________
mapguide-internals mailing list
mapguide-internals at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapguide-internals