[postgis-users] PostGIS and LIDAR Massive Point Sets

[collin]

Glad to hear you've had good experiences with large datasets. 

Another question: what is your recommended method for inserting point 
data?  Even with transactions turned off, the INSERT command is 
painfully slow.  Is there any way to use COPY FROM with point data?  The 
shapefile loader doesn't work, becuase our dataset would crash 
Arcview/ArcMap.  Any help appreciated.

If it looks promising, I can try recommending that we help out 
financially with implementing WKB.  No guarantees.

Collin
GIS Informatics Researcher
UC Berkeley


---------- David Blasby wrote ----------

>Date: Thu, 12 Feb 2004 14:35:59 -0800
>From: David Blasby <dblasby at refractions.net>
>Reply-To: PostGIS Users Discussion <postgis-users at postgis.refractions.net>
>To: PostGIS Users Discussion <postgis-users at postgis.refractions.net>
>Subject: Re: [postgis-users] PostGIS and LIDAR Massive Point Sets
>
>collin at socrates.Berkeley.EDU wrote:
>
>  
>
>>I am participating in a project to improve LIDAR processing.  We have a
>>20 gigabyte dataset of filtered x,y,z points in ascii files.
>>
>>I have questions:
>>1. Can PostGIS handle that kind of data at usable speeds? We are using a
>>new Sun Blade server running solaris.
>>2. What data types are the point arrays? text? float? Text could cause an
>>unusable overhead on a dataset this massive.
>>3. How large a dataset has anyone tried so far? And what have been the
>>problems/limitations?
>>
>>Any help, commentary, or opinions would be appreciated.
>>    
>>
>
>
>I've been using multi-gigabyte datasets in PostGIS for quite a while and
>havent had any problems.   The speed depends on what you're doing -
>searching 20 million rows for a small-ish bounding box is pretty much
>instant.
>
>The geometry data is stored in its own datastructure.  The details are
>in postgis.h and there's documentation on the website.  There is quite a
>bit of overhead for storing single points.
>
>The structure is holding unused data (offsetX,offsetY,scale, bvol.zmin,
>bvol.zmax, and type should be char), so some of the overhead could be
>minimized.
>
>If this overhead (after you take out the unused data) is an issue, we
>could upgrade the WKB type so it was indexable, and auto-convertable to
>and from the PostGIS types (so you have all the functionality of
>PostGIS).  I discussed this on the list a little while ago.
>Unfortunately, this would be several days of development.  I've been
>wanting someone to either do this or hire us to do it since it would be
>very nice feature.
>
>dave
>
>
>
>_______________________________________________
>postgis-users mailing list
>postgis-users at postgis.refractions.net
>http://postgis.refractions.net/mailman/listinfo/postgis-users
>
>
>
>
>  
>