[postgis-users] PostGIS and LIDAR Massive Point Sets

[Norman Vine]

collin writes:
> 
> We were wondering why a sample 200MB text file of 8 million points 
> ballooned into over a gigabyte when loaded into postgres.   I for one 
> would vote for making two data structures: one for point data and one 
> general for all other types.  Point data is somewhat unique in its 
> simplicity and commonality.  There is also the potential to represent 
> grids using stored point data. 
> 
> With this current generation of remote sensing devices, data resolution 
> is so high that an overflight of one county (Napa, ca, us) of raw LIDAR 
> data is 160 gigabytes.  I don't look forward to causing that to go up 
> over a terrabyte after loading.  This is only one county, one project.  
> After a hundred of these kind of projects....  
> 
> Databases are really the best way to store this data, but if there isn't 
> an efficient way to do it, I think we may stick to flat binary files or 
> loading the data into a simple non-spatial table with a fixed 
> projection.   So, for point data, a space/performance efficient custom 
> structure would have clear benefit, at least from my point of view :)

One could argue that a hybird approach is the best way to store and 
use this type of data.  

i.e.  Use something like PostGIS to store the metadata and the filename 
of the raw binary data.  Then use something like gdal, gmt or mbsystem*
to process the data into the desired form with scripts generated on the
fly based on queries of the metadata.

* I am most used to dealing with sonar data which has many similarities
to LIDAR so just substitute the equivalant LIDAR tools.  The point is that
I think you will be better off using tools that are designed to work with the
raw data natively then trying to use PostGIS proceedures once you know
which 'pieces' of the data you need.  

Cheers

Norman