[postgis-users] Massive Lidar Dataset Datatype Suggestions?
Gerry Creager N5JXS
gerry.creager at tamu.edu
Sat Nov 13 14:31:20 PST 2004
Collin,
I've nothing to add that will help you, but I'd be interested in hearing
the results. I expect to be working with similar datasets *very* soon
in a coastal and ocean observing project, associated with DEM generation
for inundation projections.
I suspect there are some issues with memory in your install. I'm
implementing a 3GHz Xeon 4-way SMP with 16 GB RAM for our PostGIS
implementation... a pair of 733 MHz processors with 1 GB RAM seems a
little light. Also, have you considered a RAID 1 system for mirroring,
to improve data integrity?
Please keep me in the loop if at all possible. We *are* interested.
Regards,
Gerry
collin wrote:
> Greetings all,
>
> I asked some questions about massive point datasets in April, before the
> implementation of LWGEOM. Then I fell off the face of the earth for a
> while. Know I'm back working on this issue again and would love some
> input.
>
> I have a LIDAR dataset of 473 million points with 2 point geometries
> (first & last returns), timestamp and 2 laser intensity returns (integer).
>
> I am trying to figure the best setup for storing, extracting and
> processing this dataset. btw, it is a smallish dataset. We will be
> processing 2 billion+ point projects in the near future.
>
> Currently I compiled & installed postgresql 8.0 beta 4, with postgis 0.9
> release, geos 2.0.1, and proj 4. This is on Fedora Core 2, smp 733mhz,
> 1GB ram, 160gb hdd Intellistation.
>
> I am using HWGEOM, with WKT right now and managed to create a table with
> oid & the_geom for one point per return.
> The upload took 48 hours and is roughly 85GB.
> The GiST indexing took ~80 hours and is ~35GB.
>
> This is obviously non-optimal considering we now have WKB and LWGEOM to
> play with. I couldn't get LWGEOM to install properly from the cvs
> extract, which is why I reverted to the 0.9 version.
>
> So, any suggestions on how to get the full 9-column dataset uploaded
> with a more efficient data structure? (note: current machine is just a
> test machine. Production will have a LOT more drivespace).
>
> Also, I intend to perform a fair amount of point processing inside the
> database using either plpgsql or java api. Is this a bad idea?
>
> Thanks for any input.
>
> ________________
> Collin Bode
> GIS Informatics Researcher
> Power Lab, Integrative Biology
> University of California, Berkeley
> _______________________________________________
> postgis-users mailing list
> postgis-users at postgis.refractions.net
> http://postgis.refractions.net/mailman/listinfo/postgis-users
--
Gerry Creager -- gerry.creager at tamu.edu
Texas Mesonet -- AATLT, Texas A&M University
Cell: 979.229.5301 Office: 979.458.4020 FAX: 979.847.8578
Page: 979.228.0173
Office: 903A Eller Bldg, TAMU, College Station, TX 77843
More information about the postgis-users
mailing list