[fdo-internals] RE: [mapguide-internals] bad behaviour in low memory situation verified on yesterday build from build server #480,

[Zac Spitzer]

Trevor,

UV is working with the same map base I have been wokring on. It's
mostly SDF with some roads
still in SHP files. The various label bugs test cases I had filed a
while back contain the most complex
layer definitions

z

On Mon, Apr 20, 2009 at 7:07 PM, Trevor Wekel <trevor.wekel at autodesk.com> wrote:
> I suspect we are looking at a memory leak somewhere in the MapGuide code base.  I am currently investigating memory leaks in MapGuide and may know more in a couple of days.
>
> UV, were you using a strictly SDF based map?  How many tiles were generated before you started running into memory issues? How many layers were on the map?  Were you using any of the advanced stylization?  I'm just trying to scope the problem a bit.
>
> Thanks,
> Trevor
>
> ________________________________________
> From: mapguide-internals-bounces at lists.osgeo.org [mapguide-internals-bounces at lists.osgeo.org] On Behalf Of UV [uvwild at gmail.com]
> Sent: Sunday, April 19, 2009 10:50 PM
> To: MapGuide Internals Mail List; FDO Internals Mail List
> Subject: [mapguide-internals] bad behaviour in low memory situation verified on yesterday build from build server  #480,
>
> To verify the low memory behaviour of the server I was rendering the
> tiles of our big map
> on a 1GB windows server machine using a most recent build from the build
> server.
>
> I can observer a relative consistent behaviour now....
> First  the FDO reader falls over trying to load a large file. (SDF
> database access)
> After this happened a few times, the MappingService fails to allocate
> memory itself.
> In the meanwhile SUCCESS gets returned for wrong tiles.
>
> I think the OoM exception inside FDO does not get propagated to MapGuide
> correctly nor handled in a useful manner.
> I remember seeing some exception free code under the sdf reader....
> Similarly the OoM exception does not get handled in a useful manner in
> MapGuide.
>
> Generally, the behaviour of the server when memory runs out is now
> almost worse than before because the tileserver returns SUCCESS for
> wrongly rendered tiles instead of INVALID STREAM HEADER.
> I am happy to help but I need help.
> I supplied the retry routine for the ByteSink which was a step forward
> in stability,
> But the underlying problem of this multi-threaded server when it comes
> to memory shortage IS NOT SOLVED.
>
> [Approach1]
> A wait retry loop around each memory allocation to catch out of memory
> exception .... this should serialize memory requests sufficiently to
> make things stable. We could create a clever macro for all memory
> allocation which would do exactly this.
> Also the allocator base class for std library containers can be used for
> that. I think an important place would be the FDO readers which open the
> few hundred MB files. There is no point in doing that in MapGuide only.
> This could be done by propagating more meaningful errors from FDO to
> mapguide so this can be dealt with correctly (recognize OoM and attempt
> retry) Black boxing FDO and having only a general exception there seems
> the wrong way to deal with such problems.
>
> This is a lot of work but I assume that doing that for the few memory
> hungry parts will sort most problems. (see also ByteSink)
>
> [Approach2]
> The minimal useful workaround would be to have the server return an
> error message as a HTTP response or on the tile without creating the
> tile in the cache! This way the client layer can attempt to retry later.
>
> # Log Type: Error Log
> # Log Parameters: CLIENT,CLIENTIP,USER,ERROR,STACKTRACE
> <2009-04-20T13:30:01>     2088
>  Success: Server started.
> <2009-04-20T14:04:41>     5212        192.168.99.99    Anonymous
>  Error: Failed to stylize layer: aug 06-ROADS - HWY
>        Cannot establish connection.
>  StackTrace:
>  - MgMappingUtil.StylizeLayers line 763 file
> c:\ci\mapguide\build\mgdev\server\src\services\mapping\MappingUtil.cpp
> <2009-04-20T14:07:28>     5540        192.168.99.99    Anonymous
>  Error: Failed to stylize layer: aug 06-ROADS - FWY
>        An exception occurred in FDO component.
>        An exception occurred in FDO component.
>        An error occurred during SDF database access.
>  StackTrace:
>  - MgMappingUtil.StylizeLayers line 763 file
> c:\ci\mapguide\build\mgdev\server\src\services\mapping\MappingUtil.cpp
> <2009-04-20T14:07:29>     1768        192.168.99.99    Anonymous
>  Error: Failed to stylize layer: aug 06-ROADS - MAJ
>        An exception occurred in FDO component.
>        An exception occurred in FDO component.
>        An error occurred during SDF database access.
>  StackTrace:
>  - MgMappingUtil.StylizeLayers line 763 file
> c:\ci\mapguide\build\mgdev\server\src\services\mapping\MappingUtil.cpp
> <2009-04-20T14:23:54>     3380        192.168.99.99    Anonymous
>  Error: Failed to stylize layer: Feb 08-ABORIG_WA
>        Out of memory.
>        bad allocation
>  StackTrace:
>  - MgMappingUtil.StylizeLayers line 763 file
> c:\ci\mapguide\build\mgdev\server\src\services\mapping\MappingUtil.cpp
> .....
> _______________________________________________
> mapguide-internals mailing list
> mapguide-internals at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/mapguide-internals_______________________________________________
> fdo-internals mailing list
> fdo-internals at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/fdo-internals
>



-- 
Zac Spitzer -
http://zacster.blogspot.com
+61 405 847 168