[GRASS-user] Large vector files

[Glynn Clements]

Jonathan Greenberg wrote:

> Unfortunately, I was hoping to work in a vector environment with the data --
> I'm sure I could think up raster analogs to the analyses I'm trying to do
> right now, but as Michael pointed out earlier, this is a problem that does
> need to be solved -- Lidar, in particular, is getting more popular, and
> software support remains primitive -- while my problem is not a Lidar one,
> it still has the same underlying issue -- I need to be able to create and
> manipulate massive vector files.
> 
> I am hearing a lot of suggestions about using things like PostGIS and
> PostGRESQL here and elsewhere, but I am a total novice to this -- is there a
> "dummy's guide" to working with these DB instead of shapefiles?  I'd like to
> be able to use all of the various GRASS vector commands on a "large vector"
> (what would the format be called?) that already exist -- I'm noticing
> there's a lot more setup involved in getting a DB running, and the process
> has yet to be streamlined.  Is it possible to simply substitute some
> postgres driven vector DB for a GRASS vector in the GRASS algorithms, or do
> the v.[whatever] algorithms need to be reworked to support this?

You can use any supported database system to store vectors. Use
db.connect to select which database to use.

After DBF, SQLite is the next easiest to use, as it doesn't involve a
separate server process. Unlike the DBF driver, SQLite will normally
have been built with large file support.

Client-server database systems normally require a non-trivial amount
of setup before you can use them. E.g. installing the software,
setting the daemon to start on boot, creating a database, configuring
access to that database, etc.

However, none of this will help if you are running into memory limits
(because the module tries to keep the entire vector map in memory)
rather than file size limits.

I'm not particularly familiar with the vector side of GRASS, but it
appears that whether or not the map includes topology information has
a significant effect upon the maximum size. Maps which lack topology
information are less likely to cause problems than those which have
it.

-- 
Glynn Clements <glynn at gclements.plus.com>