[GRASS-dev] Re: ctypes problem with Vect_open_old

[Glynn Clements]

Martin Landa wrote:

> seems to be problem with LFS. The script works when GRASS compiled
> without LFS. Can anyone confirm it?

struct Map_info contains a field of type "struct gvfile", and another
of type "struct Plus_head" which itself contains a field of type
"struct gvfile". "struct gvfile" contains fields of type off_t.

Within grass/lib/vector.py, I have:

	__off_t = c_long # /usr/include/bits/types.h: 141
	
	__off64_t = __quad_t # /usr/include/bits/types.h: 142
	
	off_t = __off64_t # /usr/include/sys/types.h: 90
	
	struct_gvfile._fields_ = [
	    ('file', POINTER(FILE)),
	    ('start', String),
	    ('current', String),
	    ('end', String),
	    ('size', off_t),
	    ('alloc', off_t),
	    ('loaded', c_int),
	]

Here, vector.py is being built with _FILE_OFFSET_BITS=64:

	make[5]: Entering directory `/usr/local/src/grass/7.0/lib/python/ctypes'

	...

	GISRC=/usr/local/src/grass/7.0/dist.i686-pc-linux-gnu/demolocation/.grassrc70
	GISBASE=/usr/local/src/grass/7.0/dist.i686-pc-linux-gnu
	PATH="/usr/local/src/grass/7.0/dist.i686-pc-linux-gnu/bin:$PATH"
	PYTHONPATH="/usr/local/src/grass/7.0/dist.i686-pc-linux-gnu/etc/python:$PYTHONPATH"
	LD_LIBRARY_PATH="/usr/local/src/grass/7.0/dist.i686-pc-linux-gnu/bin:/usr/local/src/grass/7.0/dist.i686-pc-linux-gnu/lib:/usr/local/src/grass/7.0/dist.i686-pc-linux-gnu/lib:/usr/local/src/grass/svn/dist.i686-pc-linux-gnu/lib"
	LC_ALL=C ./ctypesgen.py --cpp "gcc -E -D_FILE_OFFSET_BITS=64
	-I/usr/local/src/grass/7.0/dist.i686-pc-linux-gnu/include
	-I/usr/local/src/grass/7.0/dist.i686-pc-linux-gnu/include"
	-lgrass_vector.7.0.svn -lgrass_dig2.7.0.svn -lgrass_dgl.7.0.svn
	-lgrass_rtree.7.0.svn -lgrass_linkm.7.0.svn -lgrass_dbmiclient.7.0.svn
	-lgrass_dbmibase.7.0.svn -lgrass_btree2.7.0.svn -lgrass_gproj.7.0.svn
	-I/usr/include -I/usr/include
	/usr/local/src/grass/7.0/dist.i686-pc-linux-gnu/include/grass/vector.h
	/usr/local/src/grass/7.0/dist.i686-pc-linux-gnu/include/grass/vect/dig_structs.h
	/usr/local/src/grass/7.0/dist.i686-pc-linux-gnu/include/grass/vect/dig_defines.h
	-o vector.py

	...

	Status: gcc -E -D_FILE_OFFSET_BITS=64
	-I/usr/local/src/grass/7.0/dist.i686-pc-linux-gnu/include
	-I/usr/local/src/grass/7.0/dist.i686-pc-linux-gnu/include -U __GNUC__
	-dD -I/usr/include -I/usr/include "-Dinline=" "-D__inline__="
	"-D__extension__=" "-D_Bool=uint8_t" "-D__const=const" "-D__asm__(x)="
	"-D__asm(x)=" "-DCTYPESGEN=1" /home/glynn/.tmp/tmpQU1gtk.h

If the ctypes wrappers weren't built with the same compilation
switches as the libraries which they're trying to wrap, they'll
contain incorrect definitions for off_t and, by extension, anything
which uses it.

If this is happening when the ctypes wrappers and vector libraries are
built at the same time, then we have a problem with our build system. 
If it's caused by mixing ctypes wrappers and vector libraries from
different builds (e.g. PYTHONPATH pointing to a different version),
then you need to prevent that.

Note that similar issues apply for C code. If you try to compile a
module which uses the vector library with a different setting of
_FILE_OFFSET_BITS to that used when compiling the library, many of the
structures will have the wrong size and layout. The layout doesn't
matter (unless you access structure fields from your code), but the
size does (i.e. if your code doesn't use LFS but the vector library
does, a variable of type "struct Map_info" will be too small, and the
vector library is likely to overrun it and corrupt whatever follows it
in memory).

-- 
Glynn Clements <glynn at gclements.plus.com>