[Mapserver-dev] Optimizing msAddLine()

Steve Lime steve.lime at dnr.state.mn.us
Mon Jan 6 17:29:47 EST 2003

Thanks David, lemme (and others?) look through the code and test it a
bit. Certainly looks
promising though. Wonder how much affect it has in web use...


>>> David Turover <dturover at student.santarosa.edu> 01/06/03 04:03PM
I wrote a small test program that links against Mapserver code and
makes a lot of msDrawMap() calls on the same data set. In testing it,
I've noticed one place where Mapserver can be sped up.

Under cygwin/gcc2.95.3-5/w2k, the program starts out fast and visibly
slows down after it begins using a lot of memory due to fragmentation
(and possibly memory leaks, but I didn't see any). gprof results are:

  %   cumulative   self              self     total
 time   seconds   seconds    calls  Ts/call  Ts/call  name
 80.43     59.01    59.01                             msAddLine
  4.10     62.02     3.01                             get_metrics
  3.05     64.26     2.24                            
  2.24     65.90     1.64                             SwapWord
  1.79     67.21     1.31                             msSHPReadShape

msAddLine() has two for() loops iterating through a pair of arrays
and copying the data to new arrays -- this can use up a lot
of CPU time. Each of these arrays is freed and a new array of
size + 1 is malloced -- a recipe for fragmentation, since each
new array is too large to fit into the space vacated by the
old array.

I rewrote msAddLine() to use a buffer whose length grows in
fixed increments so that one of the two loops and mallocs only
needs to be run once out of, say, 64 msAddLine() calls on the
same shape. New function code:

int msAddLine(shapeObj *p, lineObj *new_line){
        lineObj *extended_line;
        int v;

        if(p->numlines % LINE_SEGS_PER_BUFFER == 0){
                /* Create an extended line array */
                if((extended_line = (lineObj *)calloc(
                        p->numlines + LINE_SEGS_PER_BUFFER,
                         sizeof(lineObj))) == NULL) {
                        msSetError(MS_MEMERR, NULL, "msAddLine()");

                /* Copy the old data to the new array */
                for (v= 0; v < p->numlines; v++)
                        extended_line[v]= p->line[v];

                /* Dispose of the old line */
                p->line = extended_line;
        /* Otherwise, p->line is large enough to hold the new line
        ** and doesn't need to be messed with. Sweet, eh? */

        /* Update the polygon information */
        extended_line = &p->line[p->numlines];

        /* Copy the points to the new line */
        extended_line->numpoints = new_line->numpoints;
        if((extended_line->point = (pointObj
                 * sizeof(pointObj))) == NULL) {
                msSetError(MS_MEMERR, NULL, "msAddLine()");
        for (v= 0; v < new_line->numpoints; v++)
                extended_line->point[v]= new_line->point[v];


This makes it a tad more complicated and difficult to debug if
anything goes wrong, but it seems to work for my needs and
cuts running time down from 2 minutes (plus 6 minutes in msFreeMap()
for cygwin to free everything) to 1 minute (plus 1 minute in
and memory usage down to 252 megabytes from 292 megs. New gprof

 16.22      0.84     0.84                             msSHPReadShape
 15.44      1.64     0.80                             msAddLine
 11.39      2.23     0.59                             get_metrics
 11.39      2.82     0.59                            
 10.62      3.37     0.55                            
  7.72      3.77     0.40                             msSHPOpen

Other than msAddLine, the biggest obvious slowdowns are in cygwin's
memory manager choking on the high memory use (probably more 
fragmentation), and unnecessary repetition of file i/o, neither of
effects normal Mapserver use.

The file i/o slowdown is due to msDrawMap() reloading the
shapefiles from the disk every time it is called. While this
is fine for Mapserver which only needs to call msDrawMap() once,
my program is calling msDrawMap 1,364 times and so a lot of time
is wasted on i/o and in msSHPReadShape() and related functions.
I've been working on cacheing the shapefile after it is first read,
but I haven't figured this one out yet.

Please note, as I haven't made it clear, that I haven't tested
the above function in an actual Mapserver program.

Test program code:
/* This is an attempt to write a program using MapServer */

bash-2.05a$ gcc test.c mapfile.c maputil.c maperror.c mapstring.c
mapprimitive.c mapshape.c maphash.c maptree.c mapbits.c mapsearch.c
mapxbase.c maplayer.c mappostgis.c maporaclespatial.c mapsde.c
mapsymbol.c maplabel.c maplexer.c mapparser.c mapraster.c mapwms.c
maplegend.c mapscale.c mapproject.c mapwmslayer.c
mapgd.c mapdraw.c mapoutput.c -lgd -lproj -lpng -DUSE_PROJ

#include "map.h"

int main(int argc, char ** argv){
        mapObj * map = msLoadMap("sonoma.map", NULL);
        int x, y;
        int depth=0, depthMax = 5;
        double minx, miny, maxx, maxy, mwidth, mheight, xunit, yunit;
        int zoomX[] = {2,4,8,16,32,0};
        int zoomY[] = {2,4,8,16,32,0};
        char fname[32];

        /* gdImagePtr img; /* 3.6.3 version */
        imageObj* img;

        if(map == NULL){
                printf("Can't load map.\n");
                return 0;

        img = msDrawMap(map);
        if(img == NULL){
                printf("Can't draw map image!\n");
                return 0;
        /*gdImageDestroy(img); */

        minx = map->extent.minx;
        maxx = map->extent.maxx;
        miny = map->extent.miny;
        maxy = map->extent.maxy;

        mwidth = maxx - minx;
        mheight = maxy - miny;

        printf("Extent is %d %d %d %d\n", minx, miny, maxx, maxy);

        while(depth < depthMax){
                xunit = mwidth / zoomX[depth];
                yunit = mheight / zoomY[depth];
                for(x = 0; x < zoomX[depth]; x++){
                        for(y = 0; y < zoomX[depth]; y++){
                                map->extent.minx = minx + xunit 
                                map->extent.miny = miny + yunit 
                                map->extent.maxx = map->extent.minx + 
                                map->extent.maxy = map->extent.miny + 
                                if((img = msDrawMap(map)) != NULL){
                                        printf("drew map at %d %d
                                                depth, x, y);
                                        /*gdImageDestroy(img); /* 3.6
                                                depth, x, y);
                                        msSaveImage(map, img, fname);
                                } else {
                                        printf("Can't draw map at %d %d

                                                depth, x, y);

        return 0;

Mapserver-dev mailing list
Mapserver-dev at lists.gis.umn.edu 

More information about the mapserver-dev mailing list