msSHPReadShape and potential memory leak (and sorthshp.c)

[Richard Roger]

Dear List

I have been writing a program to re-write a shapefile in a spatially 
sorted order.  My test file is about 1.3 GB in size and has over 4 million 
polygons.  My program is based on "sortshp.c" and "shptree.c" .  I am 
using Mapserver-4.6.0 distribution on a Sun system using Solaris 8 and 
using gcc 3.4.2.

The heart of the program is a simple loop, something like

for( i=0; i<num_records; i++ ) {
  msSHPReadShape( inSHP, entity[i], shape );
  msSHPWriteShape( outSHP, shape );
}

where shape is of type shapeObj *.

As written, this loop causes the program to use about  1GB (not MB!) of 
memory over and above its other demands.  If a call to msFreeShape is 
added, so the loop is changed to 

for( i=0; i<num_records; i++ ) {
  msSHPReadShape( inSHP, entity[i], shape );
  msSHPWriteShape( outSHP, shape );
  msFreeShape( shape );
}

then the memory usage is only about 50MB (twenty times less). 

This memory "leak" in the first loop is caused by my initially poor 
understanding and use of msSHPReadShape, but I think it's a mistake others 
might easily make.  I think this "feature" could be avoided by the 
following change :

*** replace the call to "msInitShape" in "msSHPReadShape" by a call to a 
"msFreeShape".

I am not sure if "msFreeShape" would need to be modified to deal with an 
entirely null shape.

I have not submitted this as a bug as I am not sure if I am using the 
mapshape functions correctly.

If this change is not made to msSHPReadShape, then I'd advocate adding an 
"msFreeShape" to "sorthshp.c" right after the two calls
"msSHPReadShape/msSHPWriteShape".

Regards

Richard E. Roger

Dr. R. E. Roger                             NSW Department of 
                                         Primary Industries
Spatial Information Officer Systems                    161 Kite St
Resource Information Unit                     Locked Bag 21
                                           ORANGE NSW 2800
ph: (02) 6391 3697                               fax: (02) 6391 3740

This message is intended for the addressee named and may contain 
confidential information. If you are not the intended recipient or 
received it in error, please delete the message and notify sender. Views 
expressed are those of the individual sender and are not necessarily the 
views of their organisation.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.osgeo.org/pipermail/mapserver-dev/attachments/20050711/cf6002d7/attachment.html