[mapguide-internals] Release Build not working???? Debug build is fine!!! ALL IS FINE

Thu Apr 23 11:57:22 EDT 2009

What recent submission fixed this?  What revision were you using previously?

The main recent submissions were:

Bruce fixed ticket 462 (Server Crash and Unresponsive using OSGeo.Gdal FDO provider)
- does your tiled map use GDAL?

I fixed ticket 969 (Zero-width lines for AGG renderer are too thick)
- default behavior is legacy, so this should not have changed any behavior for your map (unless you enabled one or more of the new settings)

-----Original Message-----
From: mapguide-internals-bounces at lists.osgeo.org [mailto:mapguide-internals-bounces at lists.osgeo.org] On Behalf Of UV
Sent: Thursday, April 23, 2009 11:21 AM
To: MapGuide Internals Mail List
Subject: Re: [mapguide-internals] Release Build not working???? Debug build is fine!!! ALL IS FINE

I am happy to see that the recent checkins have massively increased the 
stability of the server.
My R3838 build including the RFC60 patch is working flawlessly now.
The peak usage curve does not cause faulty tiles anymore. Well Done!

As those checkins also broke my patch (too many conflicts) I have to 
upload a new one for RFC60 asap.
The code is now stable.

UV wrote:
> clearly our EAP Map does not fit into memory.... so there is disk 
> access for the rendering and for the tiles.
>
> Thanks for the advice about for the number of worker threads...   
> which server classes are dealing with this?
> (I wanted to ask Bruce to augment his very instructive architectural 
> description with real classnames at some point.
> Without a reference to the actual code base its not easy to find 
> things there and it remains a more abstract description than necessary.)
>
> I am now looking at a number of configuration parameters. I gonna try 
> to adjust the threadpoolsize.
> Probably the data connection cache can be tuned also.
> This is for sure a useful way to make a deployment work for a specific 
> map on a specific server.
>
> However, this RFC is about providing additional parameters to tune the 
> server based on a better understanding of its inner workings.
> It would increase the efficiency of the resource use in many scenarios 
> in a more flexible way than the simple throtteling down.
> This is not an urgent request anyway... just brainstorming for now.
>
> Firstly we have to find out why out of memory exceptions dont fail the 
> tile generation.....
> Avoiding the memory exhaustion seems not the right solution for this 
> behaviour.
>
> Thanks for the comments.
> UV
>
> Traian Stanev wrote:
>> Is it thrashing when saving all the tile images or when rendering 
>> maps? In the former case there really isn't much you can do -- even 
>> if you make giant tiles with MapGuide, you still have to split them 
>> up into small tiles and save them to disk. I guess the way to test 
>> that would be to render all the tiles without saving them to disk and 
>> see if it is still thrashing.
>>
>> Traian
>>
>> ________________________________________
>> From: mapguide-internals-bounces at lists.osgeo.org 
>> [mapguide-internals-bounces at lists.osgeo.org] On Behalf Of Zac Spitzer 
>> [zac.spitzer at gmail.com]
>> Sent: Tuesday, April 21, 2009 10:44 PM
>> To: MapGuide Internals Mail List
>> Subject: Re: [mapguide-internals] Release Build not working???? Debug 
>> build is  fine!!! Architectural Solution: RFC 63
>>
>> In my experience mapguide always thrashes the disk when tiling using 
>> my seeder
>> which is always requesting the same scale and area
>>
>> depending on the way the data is read, unless you have a very small
>> dataset, the I/O cache
>> is going to be probably too small...
>>
>> watching the IO reads / writes under task manager I often see
>> 100-200mbs (ish approx )
>> a second on a raid 10 array..
>>
>> in 32-bit land there isn't going to be a lot of memory for the I/O cache
>>
>> z
>>
>> On Wed, Apr 22, 2009 at 12:30 PM, Traian Stanev
>> <traian.stanev at autodesk.com> wrote:
>>  
>>> I don't see how the analogy with Windows copying is right. If you 
>>> have 10 threads reading the same piece of data from disk, it's most 
>>> likely that the data is in the I/O cache, so no disk reads are 
>>> really done after the first one. The other 9 threads would get the 
>>> data in memory for free.
>>>
>>> Have you tried reducing the number of simultaneous threads in your 
>>> server thread pool in order to alleviate your low memory condition?
>>>
>>>
>>> Traian
>>>
>>> ________________________________________
>>> From: mapguide-internals-bounces at lists.osgeo.org 
>>> [mapguide-internals-bounces at lists.osgeo.org] On Behalf Of UV 
>>> [uvwild at gmail.com]
>>> Sent: Tuesday, April 21, 2009 10:17 PM
>>> To: MapGuide Internals Mail List
>>> Subject: Re: [mapguide-internals] Release Build not working???? 
>>> Debug build     is fine!!! Architectural Solution: RFC 63
>>>
>>> I finally came to the conclusion that this has been caused by resource
>>> limits. (the  infamous Out Of Memory exception)
>>>
>>> A memory limitation causes the mapping / rendering service to abort
>>> rendering the tile before all features are determined.
>>> Its very interesting that the serialization enforced by STDOUT of the
>>> debug build solves this problem already!
>>>
>>> Firstly this is a bug because the faulty Tile gets returned without an
>>> error message. (like #462)
>>>
>>> However, this gives us as reason to suggest a redesign of the Tiling
>>> service.
>>> Basically, the server does not do anything efficient if we start to
>>> request mutliple tiles of the same map and scale at the same time.
>>> This is like windows copying. If you start n parallel copy streams
>>> between the same disks you end up thrashing your disk.
>>>
>>> In our mapguide case this means that the server should maintain a
>>> request queue and serialize request to the same map and scale
>>> and maybe even compute 4 - 16 tiles at once and slice them up on the
>>> bitmap level.
>>> This does not seem very hard to do and will improve performance and
>>> stability of the mapguide server dramatically.
>>>
>>> See also RFC 63
>>>
>>> UV wrote:
>>>    
>>>> Hi,
>>>>
>>>> now for the second time I tried to run my 8GB map sample in a release
>>>> build.
>>>> In the debug build the server renders all tiles just fine.
>>>>
>>>> When I did the same test with the release build the server created
>>>> only tiles in a single color - no more features....
>>>> (the sea is blue.... the desert is beige) the good news is that its
>>>> deterministic..... the same wrong tiles each time.
>>>>
>>>> I ran the test suite.... OK 430 tests
>>>> The server INFO Trace.log does not show anything special.
>>>>
>>>> The server is executed from the exact same location in the default
>>>> installation directory.... I verified the DLL versions....
>>>> No error messages... the access.log shows the error free access to the
>>>> tiles..... but the computed tiles are crap.
>>>>
>>>> Anyone seen this before?
>>>> Any idea what to look for? As its a release build and the debug
>>>> version is working sorting this out is a challenging task.
>>>> If there were any error messages this would be useful but there 
>>>> aint any.
>>>>
>>>> PS:
>>>> The server repeatedly creates an additional directory in the tilecache
>>>> called C-0 (as opposed to C0)
>>>> containing also very boring featureless tiles  named 0_-1.png to
>>>> 9_-1.png. This looks like a bug.
>>>> _______________________________________________
>>>> mapguide-internals mailing list
>>>> mapguide-internals at lists.osgeo.org
>>>> http://lists.osgeo.org/mailman/listinfo/mapguide-internals
>>>>
>>>>       
>>> _______________________________________________
>>> mapguide-internals mailing list
>>> mapguide-internals at lists.osgeo.org
>>> http://lists.osgeo.org/mailman/listinfo/mapguide-internals
>>> _______________________________________________
>>> mapguide-internals mailing list
>>> mapguide-internals at lists.osgeo.org
>>> http://lists.osgeo.org/mailman/listinfo/mapguide-internals
>>>
>>>     
>>
>>
>>
>> -- 
>> Zac Spitzer -
>> http://zacster.blogspot.com
>> +61 405 847 168
>> _______________________________________________
>> mapguide-internals mailing list
>> mapguide-internals at lists.osgeo.org
>> http://lists.osgeo.org/mailman/listinfo/mapguide-internals
>> _______________________________________________
>> mapguide-internals mailing list
>> mapguide-internals at lists.osgeo.org
>> http://lists.osgeo.org/mailman/listinfo/mapguide-internals
>>
>>   
>
> _______________________________________________
> mapguide-internals mailing list
> mapguide-internals at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/mapguide-internals
>

_______________________________________________
mapguide-internals mailing list
mapguide-internals at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapguide-internals