[Mapserver-dev] Optimizing msAddLine()
Steve Lime
steve.lime at dnr.state.mn.us
Mon Jan 6 17:29:47 EST 2003
Thanks David, lemme (and others?) look through the code and test it a
bit. Certainly looks
promising though. Wonder how much affect it has in web use...
Steve
>>> David Turover <dturover at student.santarosa.edu> 01/06/03 04:03PM
>>>
I wrote a small test program that links against Mapserver code and
makes a lot of msDrawMap() calls on the same data set. In testing it,
I've noticed one place where Mapserver can be sped up.
Under cygwin/gcc2.95.3-5/w2k, the program starts out fast and visibly
slows down after it begins using a lot of memory due to fragmentation
(and possibly memory leaks, but I didn't see any). gprof results are:
% cumulative self self total
time seconds seconds calls Ts/call Ts/call name
80.43 59.01 59.01 msAddLine
4.10 62.02 3.01 get_metrics
3.05 64.26 2.24
msClipPolygonRect
2.24 65.90 1.64 SwapWord
1.79 67.21 1.31 msSHPReadShape
msAddLine() has two for() loops iterating through a pair of arrays
and copying the data to new arrays -- this can use up a lot
of CPU time. Each of these arrays is freed and a new array of
size + 1 is malloced -- a recipe for fragmentation, since each
new array is too large to fit into the space vacated by the
old array.
I rewrote msAddLine() to use a buffer whose length grows in
fixed increments so that one of the two loops and mallocs only
needs to be run once out of, say, 64 msAddLine() calls on the
same shape. New function code:
#define LINE_SEGS_PER_BUFFER 64
int msAddLine(shapeObj *p, lineObj *new_line){
lineObj *extended_line;
int v;
if(p->numlines % LINE_SEGS_PER_BUFFER == 0){
/* Create an extended line array */
if((extended_line = (lineObj *)calloc(
p->numlines + LINE_SEGS_PER_BUFFER,
sizeof(lineObj))) == NULL) {
msSetError(MS_MEMERR, NULL, "msAddLine()");
return(-1);
}
/* Copy the old data to the new array */
for (v= 0; v < p->numlines; v++)
extended_line[v]= p->line[v];
/* Dispose of the old line */
msFree(p->line);
p->line = extended_line;
}
/* Otherwise, p->line is large enough to hold the new line
** and doesn't need to be messed with. Sweet, eh? */
/* Update the polygon information */
extended_line = &p->line[p->numlines];
/* Copy the points to the new line */
extended_line->numpoints = new_line->numpoints;
if((extended_line->point = (pointObj
*)malloc(new_line->numpoints
* sizeof(pointObj))) == NULL) {
msSetError(MS_MEMERR, NULL, "msAddLine()");
return(-1);
}
for (v= 0; v < new_line->numpoints; v++)
extended_line->point[v]= new_line->point[v];
p->numlines++;
return(0);
}
This makes it a tad more complicated and difficult to debug if
anything goes wrong, but it seems to work for my needs and
cuts running time down from 2 minutes (plus 6 minutes in msFreeMap()
for cygwin to free everything) to 1 minute (plus 1 minute in
msFreeMap())
and memory usage down to 252 megabytes from 292 megs. New gprof
results:
16.22 0.84 0.84 msSHPReadShape
15.44 1.64 0.80 msAddLine
11.39 2.23 0.59 get_metrics
11.39 2.82 0.59
msClipPolygonRect
10.62 3.37 0.55
msImageFilledPolygon
7.72 3.77 0.40 msSHPOpen
Other than msAddLine, the biggest obvious slowdowns are in cygwin's
memory manager choking on the high memory use (probably more
fragmentation), and unnecessary repetition of file i/o, neither of
which
effects normal Mapserver use.
The file i/o slowdown is due to msDrawMap() reloading the
shapefiles from the disk every time it is called. While this
is fine for Mapserver which only needs to call msDrawMap() once,
my program is calling msDrawMap 1,364 times and so a lot of time
is wasted on i/o and in msSHPReadShape() and related functions.
I've been working on cacheing the shapefile after it is first read,
but I haven't figured this one out yet.
Please note, as I haven't made it clear, that I haven't tested
the above function in an actual Mapserver program.
Test program code:
-----
/* This is an attempt to write a program using MapServer */
/*
bash-2.05a$ gcc test.c mapfile.c maputil.c maperror.c mapstring.c
mapprimitive.c mapshape.c maphash.c maptree.c mapbits.c mapsearch.c
mapxbase.c maplayer.c mappostgis.c maporaclespatial.c mapsde.c
mapogr.cpp
mapsymbol.c maplabel.c maplexer.c mapparser.c mapraster.c mapwms.c
maplegend.c mapscale.c mapproject.c mapwmslayer.c
mapgd.c mapdraw.c mapoutput.c -lgd -lproj -lpng -DUSE_PROJ
-DUSE_GD_PNG
*/
#include "map.h"
int main(int argc, char ** argv){
mapObj * map = msLoadMap("sonoma.map", NULL);
int x, y;
int depth=0, depthMax = 5;
double minx, miny, maxx, maxy, mwidth, mheight, xunit, yunit;
int zoomX[] = {2,4,8,16,32,0};
int zoomY[] = {2,4,8,16,32,0};
char fname[32];
/* gdImagePtr img; /* 3.6.3 version */
imageObj* img;
if(map == NULL){
printf("Can't load map.\n");
msWriteError(stdout);
return 0;
}
img = msDrawMap(map);
if(img == NULL){
printf("Can't draw map image!\n");
msWriteError(stdout);
return 0;
}
/*gdImageDestroy(img); */
msFreeImage(img);
minx = map->extent.minx;
maxx = map->extent.maxx;
miny = map->extent.miny;
maxy = map->extent.maxy;
mwidth = maxx - minx;
mheight = maxy - miny;
printf("Extent is %d %d %d %d\n", minx, miny, maxx, maxy);
while(depth < depthMax){
xunit = mwidth / zoomX[depth];
yunit = mheight / zoomY[depth];
for(x = 0; x < zoomX[depth]; x++){
for(y = 0; y < zoomX[depth]; y++){
map->extent.minx = minx + xunit
*(double)x;
map->extent.miny = miny + yunit
*(double)y;
map->extent.maxx = map->extent.minx +
xunit;
map->extent.maxy = map->extent.miny +
yunit;
if((img = msDrawMap(map)) != NULL){
printf("drew map at %d %d
%d\n",
depth, x, y);
/*gdImageDestroy(img); /* 3.6
*/
sprintf(fname,
"tmp/%d_%d_%d.png",
depth, x, y);
msSaveImage(map, img, fname);
msFreeImage(img);
} else {
printf("Can't draw map at %d %d
%d\n",
depth, x, y);
msWriteError(stdout);
}
}
}
depth++;
}
msFreeMap(map);
return 0;
}
_______________________________________________
Mapserver-dev mailing list
Mapserver-dev at lists.gis.umn.edu
http://lists.gis.umn.edu/mailman/listinfo/mapserver-dev
More information about the mapserver-dev
mailing list