[GRASSLIST:923] Re: Handling millions of points in GRASS...

[Sandro Klippel]

I worked with approximately 4 millions points of batimetric data of
southern Brazil continental margin. For performance reasons I preferred
working with GMT tools and awk scripts. But the awk scripts was used
only for filter out bad data. 'blockmean' and 'surface' grid data in few
seconds. Then I imported to grass/nviz with 'r.in.bin' for 3d
visualization.

You can use the 'gmtselect' for spatial selections.

Sandro.

On Thu, 2003-08-07 at 06:00, Bernhard Reiter wrote:
> On Wed, Aug 06, 2003 at 09:23:56AM -0700, David Finlayson wrote:
> 
> > I know that RDBMS support is coming soon and that (apparently) PostGIS 
> > is already functional on some platforms (Linux, Cygwin?).  But what is 
> > the word from the trenches?  I have been very happy with MySQL and would 
> > rather use this if functionality is comparable to PostGres, but it would 
> > be better to move now rather than wait if necessary...
> > 
> > A major task that needs to be handled is some way to select points both 
> > by spatial location (polygons not boxes) and by attributes such as age, 
> > survey, and platform.  Currently, I have a table of soundings (x,y,z,ID) 
> > and a table of headers  (ID,date,platform,etc.) and I load it into 
> > ArcGIS (ESRI, inc.) through ODBC where I can grid up the data into 
> > rasters for further analysis. Hence, I am currently dependent on Windows.
> 
> How to hold the data depends on your usage requirements.
> If you do many queries that could benefit from sofisticated indexes
> you should think about using PostGIS.
> Also if you need data security.
> 
> Also check out this post comparing the upcoming geographical features
> of MySQL 4.1 and postgis:
> http://intevation.de/pipermail/freegis-list/2003-July/001449.html
> Basically MySQL 
> does not offer transactional security and nice index technology.
> 
> Radim and Sandro already pointed out that there are several
> ways to get points out of the database into GRASS.
> 
> The more complicated approaches (like moving to PostGIS 
> and optimising the indexing) only make sense if you really
> have the usage pattern for it. I agree with Radim that cleverly used
> text files might give good results.
-- 
Sandro Klippel <sandro.klippel at talha-mar.com.br>

talha-mar projetos ambientais
Conselheiro d'Avila 190 
Porto Alegre,RS 91040-450
+55 (51) 3344-7451

w w w . t a l h a - m a r . c o m . b r