[Mapserver-dev] Unfreed Buffer in Lexer

[Frank Warmerdam]

Folks,

As part of my effort at testing new additions I make to MapServer, I like
to check for memory leaks.  I have cleaned up most of my own leaks, but
I always get reports about a buffer in the lexer code.  It appears to be
some sort of a working buffer that never gets freed, but is re-used (or
replaced) by subsequent lexing operations.  So this isn't a serious leak.
It can't accumulate over time or anything like that.

But it does make these sorts of "noise leaks" do make it harder to identify
real leaks, especially if there are a lot of them.  So I would like to clean
it up.

The leaked buffer(s) look like this:

==3238== 12 bytes in 1 blocks are still reachable in loss record 1 of 3
==3238==    at 0x400268D5: malloc (vg_replace_malloc.c:160)
==3238==    by 0x808F6DE: msyy_scan_bytes (maplexer.c:3957)
==3238==    by 0x808F6C0: msyy_scan_string (maplexer.c:3936)
==3238==    by 0x808E1F9: msyylex (maplexer.l:34)
==3238==    by 0x808E1A1: msyyparse (/usr/share/bison/bison.simple:573)
==3238==    by 0x8099641: msGetClass (mapraster.c:140)
==3238==    by 0x80AA55C: msDrawRasterLayerGDAL (mapdrawgdal.c:564)
==3238==    by 0x809BC68: msDrawRasterLayerLow (mapraster.c:1625)
==3238==    by 0x8069FC5: msDrawLayer (mapdraw.c:581)
==3238==    by 0x80696DA: msDrawMap (mapdraw.c:295)
==3238==
==3238==
==3238== 40 bytes in 1 blocks are still reachable in loss record 2 of 3
==3238==    at 0x400268D5: malloc (vg_replace_malloc.c:160)
==3238==    by 0x808F63E: msyy_scan_buffer (maplexer.c:3903)
==3238==    by 0x808F70F: msyy_scan_bytes (maplexer.c:3966)
==3238==    by 0x808F6C0: msyy_scan_string (maplexer.c:3936)
==3238==    by 0x808E1F9: msyylex (maplexer.l:34)
==3238==    by 0x808E1A1: msyyparse (/usr/share/bison/bison.simple:573)
==3238==    by 0x8099641: msGetClass (mapraster.c:140)
==3238==    by 0x80AA55C: msDrawRasterLayerGDAL (mapdrawgdal.c:564)
==3238==    by 0x809BC68: msDrawRasterLayerLow (mapraster.c:1625)
==3238==    by 0x8069FC5: msDrawLayer (mapdraw.c:581)

The code calling the parser looks roughly like this:


       msyystate = 4; msyystring = tmpstr1;
       if(msyyparse() != 0) return(-1); // error parsing the expression

       free(tmpstr1);

Other than freeing the buffer I pass in to be parsed, I don't try to do
anything else to force flushing of working buffers.  Looking through the
maplexer.c code (generated by flex I believe) the allocated memory seems
to be for a YY_BUFFER_STATE.  But I don't know how to ensure the lexer
cleans everything up when it is done.

Are there any lex/flex guru's out there who can suggest ways of a tight
cleanup?

On this point, I would note there is an msCleanup() function which isn't
really needed in normal use, but can be used to force cleanup of various
other one-time memory allocations such as GDAL/OGR driver registrations.
If you are trying to do memory leak testing, I encourage you to call it.
The shp2img utility (which I use for much of my testing) already calls it.
I don't think mapserv does, but I'm not sure.  It is exposed in SWIG MapScript
now in CVS.

Best regards,
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam, warmerdam at pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | Geospatial Programmer for Rent