[mapserver-users] Expression segmentation fault

Fri Sep 6 05:20:52 PDT 2013

Ok, so I did a little bit of geometry testing. Don't know why that 
should cause an expression crash, but well ...

here's the query I ran on planet_osm_line and planet_osm_polygon in PostGIS:

select

osm_id, boundary, admin_level, tags,
st_isvalid(way) as isvalid,
st_isvalidreason(way) as reason,
st_isclosed(way) as isclosed,
st_isempty(way) as isempty,
st_geometrytype(way) as geometrytype,
st_length(way) as length,
st_perimeter(way) as perimeter,
st_numgeometries(way) as numgeometries,
st_numinteriorrings(way) as numinteriorrings,
st_astext(way) as wkt

from planet_osm_polygon

where way && st_transform(st_setsrid('BOX3D(0 0, 150000 200000)'::box3d, 
2169), 3857)
and tags ?& ARRAY['boundary', 'admin_level']

order by geometrytype, admin_level, name

There's one record that differs from the rest in the planet_osm_polygon 
table. I've pasted the result below. It has a geometrytype of 
GEOMETRYCOLLECTION and is empty, so that may cause a crash ...

229382895;"administrative";"8";""area"=>"yes", "name"=>"RW 10", 
"boundary"=>"administrative", "admin_level"=>"8", "flood_prone"=>"no", 
"is_in:hamlet"=>"MENTENG DALAM", "is_in:district"=>"Jakarta Selatan", 
"is_in:province"=>"DKI Jakarta", "is_in:subdistrict"=>"TEBET"";t;"Valid 
Geometry";;t;"ST_GeometryCollection";0;0;0;;"GEOMETRYCOLLECTION EMPTY"

The attributes confirm, that this record could be a remain of 
http://www.openstreetmap.org/browse/way/229382895 , but I really wonder 
why it gets caught in the bbox which should center around Luxembourg 
with parts of the surrounding countries. But when the geometry is a 
black hole you never know :-)

Am 2013-09-05 16:30, schrieb Stephen Woodbridge:
> Hi,
>
> This is a great idea. Regardless we should try to identify the case that
> is causing the crash and trap that. Mapserver should never crash as a
> general rule so finding the specific case is important so Thomas can
> reproduce it.
>
> If you have (or can load) the data in a database table, then it should
> be fairly easy to do a manual binary search to find the offending object
> by adding  " where gid between <lower> and <upper>" to your query and
> adjusting the value of lower and upper to narrow the search to the
> problem object.
>
> Thanks,
>    -Steve W
>
> On 9/5/2013 10:11 AM, Rahkonen Jukka wrote:
>> Hi,
>>
>> Osm2pgsql can create invalid polygons with self-intersections and/or
>> three-vertex A-B-A rings. It might be good to run MakeValid or simply
>> find possibly invalid features with IsValid and delete them and then
>> see if Mapserver gets happy.  Actually, run IsValid and save the
>> faulty geometries somewhere before correcting or deleting them. It
>> would be nice to catch them if they happen to be the reason for the
>> crash.
>>
>> -Jukka Rahkonen-
>>
>> Frank Broniewski wrote:
>>>
>>> Ok, this is weird. It is, as I know now, data source related. I was
>>> poking in the layer configuration to find out why the core dump
>>> happens. Reinstalling didn't solve the issue, btw.
>>>
>>> So first I converted the data to shapefile with ogr2ogr (as always
>>> very handy!) and changed the layer accordingly. And no core dump
>>> happened! Then I tried several versions of the data and filter
>>> statements without success. Since this is a osm2pgsql database I
>>> swapped finally the table from planet_osm_polygon to
>>> planet_osm_line (same table schema apart from the geometry) - and
>>> voilà - no core dump.
>>>
>>> So I can relate the problem to the planet_osm_polygon table - but
>>> I'm not sure what difference actually is responsible for the crash
>>> ...
>>>
>>> The most apparent difference is of course the geometry type:
>>> planet_osm_line is linestring, planet_osm_polygon is geometry ...
>>>
>>> I will investigate the attributes further tomorrow, but now I need
>>> to earn some money. It's always the deadlines ...
>>>
>>>
>>> Frank
>>>
>>>
>>> Am 2013-09-05 09:47, schrieb thomas bonfort:
>>>> Frank, This is such a simple use-case that I doubt it's a bug in
>>>> the code per-se. Can you make sure that you are using a clean
>>>> build (make clean && make && make install) as I suspect this
>>>> could be happening due to a compilation problem (as a rule of
>>>> thumb, with mapserver <6.4, always run make clean after
>>>> re-running configure)
>>>>
>>>> -- thomas
>>>>
>>>>
>>>> On Wed, Sep 4, 2013 at 2:39 PM, Frank Broniewski
>>>> <brfr at metrico.lu> wrote:
>>>>> Am 2013-09-04 14:33, schrieb thomas bonfort:
>>>>>
>>>>>> Frank, please supply a backtrace of the crash (`bt` in gdb
>>>>>> once it has halted at the segfault)
>>>>>>
>>>>>> -- thomas
>>>>>>
>>>>>> On Wed, Sep 4, 2013 at 2:25 PM, Frank Broniewski
>>>>>> <brfr at metrico.lu>
>>> wrote:
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> I just updated my mapserver installation to the latest
>>>>>>> version (6.2.1). I'm running FreeBSD 9.1 and I'm getting a
>>>>>>> segmentation fault related to expression usage in my
>>>>>>> mapfile. I've already compiled mapserver with debug symbols
>>>>>>> and I loaded the core file into gdb:
>>>>>>>
>>>>>>>> gdb /usr/local/www/apache22/cgi-bin/mapserv mapserv.core
>>>>>>>
>>>>>>> <snip> Reading symbols from /lib/libc.so.7...done. Loaded
>>>>>>> symbols for /lib/libc.so.7 Reading symbols from
>>>>>>> /usr/local/lib/libintl.so.9...done. Loaded symbols for
>>>>>>> /usr/local/lib/libintl.so.9 Reading symbols from
>>>>>>> /usr/lib/libsupc++.so.1...done. Loaded symbols for
>>>>>>> /usr/lib/libsupc++.so.1 Reading symbols from
>>>>>>> /libexec/ld-elf.so.1...done. Loaded symbols for
>>>>>>> /libexec/ld-elf.so.1 #0  0x00000008008e2520 in yylex
>>>>>>> (lvalp=0x7fffffffd250, p=0x7fffffffd3b0) at
>>>>>>> mapparser.y:649 649     mapparser.y: No such file or
>>>>>>> directory. in mapparser.y [New Thread 809007400 (LWP
>>>>>>> 101134/mapserv)]
>>>>>>>
>>>>>>> So apparently there seems to be a file missing!? Looking at
>>>>>>> the source code (git), line 649 is a blank line, so I'm not
>>>>>>> sure what's missing exactly. I've got yacc, bison and flex
>>>>>>> installed. For testing purposes I'm using the mapserver cgi
>>>>>>> on the command line:
>>>>>>>
>>>>>>> /usr/local/www/apache22/cgi-bin/mapserv -nh
>>>>>>>
>>>>>>>
>>> "QUERY_STRING=map=/data/web/mapserver/cnra/cnra.map&mode=map&laye
>>> r=boundaries"
>>>>>>>
>>>>>>> The boundaries layer:
>>>>>>>
>>>>>>> Layer
>>>>>>>
>>>>>>> #    Classitem "level" Connection "host=10.0.0.2 dbname=osm
>>>>>>> user=user
>>> password=guessme"
>>>>>>> Connectiontype Postgis Data "geom FROM (SELECT osm_id, way
>>>>>>> AS geom, name, admin_level
>>> AS
>>>>>>> level, tags FROM planet_osm_polygon) AS foo USING UNIQUE
>>>>>>> osm_id USING
>>> SRID=3857"
>>>>>>> Filter "tags ?& ARRAY['boundary', 'admin_level']" Name
>>>>>>> "boundaries" Processing "CLOSE_CONNECTION=DEFER" Status on
>>>>>>> Type line Units meters
>>>>>>>
>>>>>>> Metadata "ows_title" "Boundary Map" "ows_abstract"
>>>>>>> "Boundary map - data from OpenStreetMap, ODbl licensed"
>>>>>>> End
>>>>>>>
>>>>>>> Projection "init=epsg:3857" End
>>>>>>>
>>>>>>> Class
>>>>>>>
>>>>>>> Expression ("[level]" = "6") #        Expression "6" Name
>>>>>>> "communes"
>>>>>>>
>>>>>>> Style Color 10 10 10 Opacity 50 Width 2 End
>>>>>>>
>>>>>>> End End
>>>>>>>
>>>>>>>
>>>>>>> The classitem / simple expression (Expression "6") works
>>>>>>> with the mapserver cgi, but python mapscript throws an
>>>>>>> error: _mapscript.MapServerError: msEvalExpression():
>>>>>>> General error message. Invalid item index.
>>>>>>>
>>>>>>> I've read the mapserver expressions documentation [1] and
>>>>>>> the note that says something about the working environment
>>>>>>> might be linked to more than
>>> one
>>>>>>> expression library. But my operating system skills are not
>>>>>>> high enough to turn this paragraph into something useful
>>>>>>> for me.
>>>>>>>
>>>>>>> So any help is greatly appreciated.
>>>>>>>
>>>>>>> Frank
>>>>>>>
>>>>>>>
>>>>>>> Frank BRONIEWSKI
>>>>>>>
>>>>>>> METRICO s.à r.l. géomètres technologies d'information
>>>>>>> géographique rue des Romains 36 L-5433 NIEDERDONVEN
>>>>>>>
>>>>>>> tél.: +352 26 74 94 - 28 fax.: +352 26 74 94 99
>>>>>>> http://www.metrico.lu
>>>>>>> _______________________________________________
>>>>>>> mapserver-users mailing list
>>>>>>> mapserver-users at lists.osgeo.org
>>>>>>> http://lists.osgeo.org/mailman/listinfo/mapserver-users
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>> Hi Thomas,
>>>>>
>>>>> thanks for the fast response! Here's the bt:
>>>>>
>>>>> (gdb) bt
>>>>>
>>>>> #0  0x00000008008e2520 in yylex (lvalp=0x7fffffffd250,
>>>>> p=0x7fffffffd3b0) at mapparser.y:649 #1  0x00000008008e0146 in
>>>>> yyparse (p=0x7fffffffd3b0) at mapparser.c:1500 #2
>>>>> 0x00000008009239b2 in msEvalExpression (layer=0x809009000,
>>>>> shape=0x7fffffffd4e0, expression=0x8090a0280, itemindex=-1) at
>>>>> maputil.c:478 #3  0x0000000800923f03 in msShapeGetClass
>>>>> (layer=0x809009000, map=0x8090bd800, shape=0x7fffffffd4e0,
>>>>> classgroup=0x0, numclasses=1) at maputil.c:581 #4
>>>>> 0x0000000800980ab2 in msDrawVectorLayer (map=0x8090bd800,
>>>>> layer=0x809009000, image=0x8090a91c0) at mapdraw.c:989 #5
>>>>> 0x00000008009800ed in msDrawLayer (map=0x8090bd800,
>>> layer=0x809009000,
>>>>> image=0x8090a91c0) at mapdraw.c:808 #6  0x000000080097ed18 in
>>>>> msDrawMap (map=0x8090bd800, querymap=0)
>>> at
>>>>> mapdraw.c:437 #7  0x0000000800a16e6e in
>>>>> msCGIDispatchImageRequest
>>> (mapserv=0x8090b2300) at
>>>>> mapservutil.c:1448 #8  0x0000000800a179c2 in
>>>>> msCGIDispatchRequest (mapserv=0x8090b2300)
>>> at
>>>>> mapservutil.c:1690 #9  0x00000000004012c1 in main (argc=3,
>>>>> argv=0x7fffffffd9f0) at mapserv.c:259 (gdb)
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> -- Frank BRONIEWSKI
>>>>>
>>>>> METRICO s.à r.l. géomètres technologies d'information
>>>>> géographique rue des Romains 36 L-5433 NIEDERDONVEN
>>>>>
>>>>> tél.: +352 26 74 94 - 28 fax.: +352 26 74 94 99
>>>>> http://www.metrico.lu
>>>>
>>>>
>>>
>>>
>>> -- Frank BRONIEWSKI
>>>
>>> METRICO s.à r.l. géomètres technologies d'information géographique
>>> rue des Romains 36 L-5433 NIEDERDONVEN
>>>
>>> tél.: +352 26 74 94 - 28 fax.: +352 26 74 94 99
>>> http://www.metrico.lu
>>> _______________________________________________ mapserver-users
>>> mailing list mapserver-users at lists.osgeo.org
>>> http://lists.osgeo.org/mailman/listinfo/mapserver-users
>> _______________________________________________ mapserver-users
>> mailing list mapserver-users at lists.osgeo.org
>> http://lists.osgeo.org/mailman/listinfo/mapserver-users
>>
>
> _______________________________________________
> mapserver-users mailing list
> mapserver-users at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/mapserver-users

-- 
Frank BRONIEWSKI

METRICO s.à r.l.
géomètres
technologies d'information géographique
rue des Romains 36
L-5433 NIEDERDONVEN

tél.: +352 26 74 94 - 28
fax.: +352 26 74 94 99
http://www.metrico.lu