[GRASS-dev] Re: About the vector changes

[Glynn Clements]

Markus Metz wrote:

> Yes, if you want to work with vectors not in the current mapset using 
> grass7, you have to rebuild topology for all vectors in the 
> corresponding mapset first.

So it's intentional that GRASS7 is no longer compatible with previous
vector maps?

That isn't necessarily a problem, but I must have missed the
announcement. But the error message needs to be clear; and accurate:
the problem isn't a lack of LFS support, and the map *wan't* created
with the LFS enabled (the map hasn't changed in 5 years).

> > FWIW, the complete structure is:

[snip]

> > Other than off_t_size = 42, ptr->spidx_port is all zeros.
> 
> Another problem is ptr->port.off_t_size = 0, it should be 4, as 
> ptr->sidx_port.off_t_size.
> topo was read successfully, reading sidx failed, cidx was not yet read. 
> The libraries should not even attempt to read sidx if that file doesn't 
> exist. The same test is done for all vector files. I don't understand 
> why that happens, I get the correct error messages.

PERMANENT/vector/fields contains:

	-rw-r--r-- 1 root root  4117 May 25  2004 cidx
	-rw-r--r-- 1 root root 27905 May 25  2004 coor
	-rw-r--r-- 1 root root    55 May 25  2004 dbln
	-rw-r--r-- 1 root root   278 May 25  2004 head
	-rw-r--r-- 1 root root   186 May 25  2004 hist
	-rw-r--r-- 1 root root 24538 May 25  2004 sidx
	-rw-r--r-- 1 root root 25161 May 25  2004 topo

MD5 hashes:

	6d564fa2eec15fc5080694d74e106f26  cidx
	a09c6ee5263e9404c3890ad2f3d75aed  coor
	f47b9255570ced6368981740ffc9ebb2  dbln
	7ed61317b7d9a658acb5bd305c9fbb7c  head
	0679efebdb1616566d03243eb5b8c867  hist
	35c963fd5ececd8364d1c018353442c0  sidx
	4e5b18920309348c8df961f869d01793  topo

The files are unmodified from:

	-rw-r--r-- 1 glynn root 19965571 Oct 16  2004 spearfish_grass57data.tar.gz

	c7a356fcc9ab649f73973315cba205cb  spearfish_grass57data.tar.gz

The 5.7 Spearfish maps worked until quite recently.

> >> No, because this is set system-wide in Grass.make
> >> https://trac.osgeo.org/grass/browser/grass/trunk/include/Make/Grass.make#L82
> >
> > That's likely to cause problems for anything which uses $(VECT_CFLAGS)
> > but which isn't itself 64-bit aware (i.e. uses fseek/ftell, calculates
> > offsets as int/long).
> >
> >> because not only the library but also all modules including vector.h 
> >> must have 64bit enabled.
> >
> > That means all such modules must be made 64-bit safe. Without
> > _FILE_OFFSET_BITS=64, open(), fopen() etc will simply refuse to open
> > files larger than 2GiB. If you set _FILE_OFFSET_BITS=64 in code which
> > wasn't designed for it, ftell() will fail once you get past the 2GiB
> > mark, storing the result of lseek() in a long will silently truncate
> > it, etc.
> >
> > This change has the potential to corrupt files, so it needs to be
> > completed ASAP.
> 
> OK. So I have to check and correct all occurrences of ftell and fseek in 
> all vector modules and make sure the result of lseek is stored in off_t 
> not long, and that offset passed to lseek is of type off_t not long. 
> What else needs to be done?

Offsets need to be calculated as off_t, not int or long. Casting them
when lseek/G_fseek are called is too late. IOW:

	int row, cols, cols, bytes_per_cell;
	off_t offset = row * (col * rows + col) * bytes_per_cell;

needs to be e.g.:

	off_t offset = ((off_t) row * cols + col) * bytes_per_cell;

so that off_t is used throughout the calculation.

Also, this needs to be checked for anything which uses VECT_CFLAGS,
which isn't limited to v.* modules (a few g.* and r.* modules use
vector functions).

> >> Instead of adding the appropriate lines to each 
> >> single Makefile I decided to set it system-wide, that also avoids 
> >> problems with addon modules. It worked so far on Linux 32bit and Linux 
> >> 64bit. What platform are you using?
> >>     
> >
> > Linux/x86 (32-bit).
> >   
> With or without LFS?

With.

-- 
Glynn Clements <glynn at gclements.plus.com>