[GRASS-user] Large vector files

[Glynn Clements]

Brad Douglas wrote:

> > > > I would _highly_recommend trying r.in.xyz if you have not already done
> > > > so. Especially with LIDAR and other forms of remotely-sensed data.
> > > > I've had good success with it. Note there is also a parameter in
> > > > r.in.xyz to control how much of the input map to keep in memory,
> > > > allowing you to run the data import in multiple passes.  
> > > 
> > > note the r.in.xyz memory parameter is to help with massive RASTER
> > > regions, nothing directly to do with the size of the input file.
> > > The input file is not kept in memory! The output raster grid is.
> > > 
> > > I am always looking for feedback on how r.in.xyz goes with massive input
> > > data. (>2gb? >4gb?)
> > 
> > r.in.xyz doesn't use LFS, so it will be limited to 2Gb on 32-bit
> > systems (any system where "long" is 32 bits). As it uses ANSI stdio
> > functions (including ftell/fseek), extending it to support large files
> > would be non-trivial.
> 
> Attached is a quick patch to enable LFS.  It's "poorly" implemented with
> fseeko/ftello, so I'm not sure if I should commit it.

It shouldn't be committed.

fseeko/ftello aren't available on all platforms. That's why I said
that fixing it is non-trivial; you have to first check that
fseeko/ftello are available, and only use them if they are.

-- 
Glynn Clements <glynn at gclements.plus.com>