[mapserver-dev] Mapserver getenv crashes (+ valgrind logs)

[Henrik K]

Hello,

We use a self-compiled stack of latest versions of mapserver (7.6.1), proj,
gdal, geos etc.  The production server is pretty busy and we see lots of
crashing related to getenv().

I tried running valgrind, had a hard time producing any crashes since it
seems to relate on query speed.  But with 25 concurrent connection flooding
I managed to get few dumps too..

(gdb) where
#0  0x0000000008e262bd in getenv () from /lib64/libc.so.6
#1  0x000000000753e7c1 in CPLGetConfigOption () at cpl_conv.cpp:1717
#2  0x0000000006db696c in GTiffDataset::Finalize (this=0x1c617970) at geotiff.cpp:7744
#3  0x0000000006db6f9a in GTiffDataset::~GTiffDataset (this=0x1c617970, __in_chrg=<optimized out>) at geotiff.cpp:7714
#4  0x0000000006db70c9 in GTiffDataset::~GTiffDataset (this=0x1c617970, __in_chrg=<optimized out>) at geotiff.cpp:7711
#5  0x0000000004f7fab1 in msGDALCleanup () at /home/wms/src/mapserver-7.6.1/mapgdal.c:90
#6  0x0000000004f134b7 in msCleanup () at /home/wms/src/mapserver-7.6.1/maputil.c:2105
#7  0x0000000000401554 in msCleanupOnSignal (nInData=<optimized out>) at /home/wms/src/mapserver-7.6.1/mapserv.c:64
#8  <signal handler called>
#9  0x0000000008eec61d in accept () from /lib64/libc.so.6
#10 0x00000000052f19ea in OS_Accept () from /lib64/libfcgi.so.0
#11 0x00000000052ef9ac in FCGX_Accept_r () from /lib64/libfcgi.so.0
#12 0x00000000052efae5 in FCGX_Accept () from /lib64/libfcgi.so.0
#13 0x00000000052efc6a in FCGI_Accept () from /lib64/libfcgi.so.0
#14 0x0000000000401234 in main (argc=<optimized out>, argv=<optimized out>) at /home/wms/src/mapserver-7.6.1/mapserv.c:137

I have a large package of valgrind logs if someone more experienced could
have a look?  You can download it here:

https://drive.google.com/file/d/1wgl7CgfBhyVyKMkqlOyNoAS6IZc1CEND/view?usp=sharing

Some random examples:

==20510== Process terminating with default action of signal 11 (SIGSEGV): dumping core
==20510==  Access not within mapped region at address 0x24ABBAA0
==20510==    at 0x8E262BD: getenv (in /usr/lib64/libc-2.17.so)
==20510==    by 0x753E7C0: CPLGetConfigOption (cpl_conv.cpp:1717)
==20510==    by 0x6DB696B: GTiffDataset::Finalize() [clone .part.500] (geotiff.cpp:7744)
==20510==    by 0x6DB6F99: GTiffDataset::~GTiffDataset() (geotiff.cpp:7714)
==20510==    by 0x6DB70C8: GTiffDataset::~GTiffDataset() (geotiff.cpp:7720)
==20510==    by 0x4F7FAB0: msGDALCleanup (mapgdal.c:90)
==20510==    by 0x4F134B6: msCleanup (maputil.c:2105)
==20510==    by 0x401553: msCleanupOnSignal (mapserv.c:64)
==20510==    by 0x8E233FF: ??? (in /usr/lib64/libc-2.17.so)
==20510==    by 0x8EEC61C: ??? (in /usr/lib64/libc-2.17.so)
==20510==    by 0x52F19E9: OS_Accept (in /usr/lib64/libfcgi.so.0.0.0)
==20510==    by 0x52EF9AB: FCGX_Accept_r (in /usr/lib64/libfcgi.so.0.0.0)

==20510== 6,624 bytes in 6 blocks are definitely lost in loss record 11,010 of 11,185
==20510==    at 0x4C29F73: malloc (vg_replace_malloc.c:309)
==20510==    by 0x4F23C89: msRASTERLayerGetExtent (maprasterquery.c:1339)
==20510==    by 0x4EE4D16: msLayerGetExtent (maplayer.c:558)
==20510==    by 0x4E9E43A: msOWSGetLayerExtent (mapows.c:2370)
==20510==    by 0x4E81A50: msDumpLayer (mapwms.c:2349)
==20510==    by 0x4E85093: msWMSGetCapabilities (mapwms.c:3613)
==20510==    by 0x4E8A345: msWMSDispatch (mapwms.c:5175)
==20510==    by 0x4E9B168: msOWSDispatch (mapows.c:289)
==20510==    by 0x4EA408C: msCGIDispatchRequest (mapservutil.c:1709)
==20510==    by 0x4EA408C: msCGIDispatchRequest (mapservutil.c:1691)
==20510==    by 0x401385: main (mapserv.c:283)

==18996== Invalid read of size 8
==18996==    at 0x8E26298: getenv (in /usr/lib64/libc-2.17.so)
==18996==    by 0x753E7C0: CPLGetConfigOption (cpl_conv.cpp:1717)
==18996==    by 0x6DB696B: GTiffDataset::Finalize() [clone .part.500] (geotiff.cpp:7744)
==18996==    by 0x6DB6F99: GTiffDataset::~GTiffDataset() (geotiff.cpp:7714)
==18996==    by 0x6DB70C8: GTiffDataset::~GTiffDataset() (geotiff.cpp:7720)
==18996==    by 0x4F7FAB0: msGDALCleanup (mapgdal.c:90)
==18996==    by 0x4F134B6: msCleanup (maputil.c:2105)
==18996==    by 0x401553: msCleanupOnSignal (mapserv.c:64)
==18996==    by 0x8E233FF: ??? (in /usr/lib64/libc-2.17.so)
==18996==    by 0x8EEC61C: ??? (in /usr/lib64/libc-2.17.so)
==18996==    by 0x52F19E9: OS_Accept (in /usr/lib64/libfcgi.so.0.0.0)
==18996==    by 0x52EF9AB: FCGX_Accept_r (in /usr/lib64/libfcgi.so.0.0.0)
==18996==  Address 0x1718ebd0 is 0 bytes inside a block of size 480 free'd
==18996==    at 0x4C2B06D: free (vg_replace_malloc.c:540)
==18996==    by 0x52ED2CA: ??? (in /usr/lib64/libfcgi.so.0.0.0)
==18996==    by 0x52EF6A3: FCGX_Free (in /usr/lib64/libfcgi.so.0.0.0)
==18996==    by 0x52EF92E: FCGX_Accept_r (in /usr/lib64/libfcgi.so.0.0.0)
==18996==    by 0x52EFAE4: FCGX_Accept (in /usr/lib64/libfcgi.so.0.0.0)
==18996==    by 0x52EFC69: FCGI_Accept (in /usr/lib64/libfcgi.so.0.0.0)
==18996==    by 0x401233: main (mapserv.c:247)
==18996==  Block was alloc'd at
==18996==    at 0x4C2C291: realloc (vg_replace_malloc.c:836)
==18996==    by 0x52ED383: ??? (in /usr/lib64/libfcgi.so.0.0.0)
==18996==    by 0x52ED9A2: ??? (in /usr/lib64/libfcgi.so.0.0.0)
==18996==    by 0x52EFA36: FCGX_Accept_r (in /usr/lib64/libfcgi.so.0.0.0)
==18996==    by 0x52EFAE4: FCGX_Accept (in /usr/lib64/libfcgi.so.0.0.0)
==18996==    by 0x52EFC69: FCGI_Accept (in /usr/lib64/libfcgi.so.0.0.0)
==18996==    by 0x401233: main (mapserv.c:247)

==18996== 360 bytes in 30 blocks are definitely lost in loss record 68 of 75
==18996==    at 0x4C29F73: malloc (vg_replace_malloc.c:309)
==18996==    by 0x4EB4F6E: msStrdup (mapstring.c:2116)
==18996==    by 0x4EAE95C: msHTTPAuthProxySetup (maphttp.c:398)
==18996==    by 0x4E9688B: msPrepareWMSLayerRequest (mapwmslayer.c:1252)
==18996==    by 0x4F0CE11: msDrawMap (mapdraw.c:309)
==18996==    by 0x4E86601: msWMSGetMap (mapwms.c:3977)
==18996==    by 0x4E8A580: msWMSDispatch (mapwms.c:5350)
==18996==    by 0x4E9B168: msOWSDispatch (mapows.c:289)
==18996==    by 0x4EA408C: msCGIDispatchRequest (mapservutil.c:1709)
==18996==    by 0x4EA408C: msCGIDispatchRequest (mapservutil.c:1691)
==18996==    by 0x401385: main (mapserv.c:283)

Cheers,
Henrik