[GRASS-user] v.in.ascii - put1 of 4 columns to 3D point file attribute

[M S]

On Mon, Apr 14, 2008 at 1:33 AM, Hamish <hamish_b at yahoo.com> wrote:

> Mark wrote:
> > The points are about 5 million per tile, and there are 23 tiles (about
> > 115 million total).  I'm not sure if this is considered a large amount
> > of mass input points.
>
> Yes, it's a lot. The current vector engine allocates a small amount of
> memory per feature for topology. With the DBF as database and default
> v.in.ascii settings I've only ever managed to import about 3 million
> points before running out of memory. I am interested to hear that you
> could load 5 million into a SQLite DB, maybe that is more efficient.


The SQlite setup was so easy (thanks to the book), and then able to start
working with the tables was a breeze (literally as easy as db.connect
driver=sqlite database="$GISDBASE/$LOCATION_NAME/$MAPSET/sqlite.db.)  I'm
not a SQL expert, but learned enough to do some sweet tasks.  SQlite was
rather impressive in working with 5.2 million records.  Through v.in.ascii,
it did load all the data (4 columns) into the sqlite database without
problem (about 7 minutes on a 64bit 2.0GHz 2GB ram system - no table and no
topology)


>
> With the v.in.ascii -t and -b flags you should be able to load 25million+
> points into a single vector map, but there is a limited number of modules
> that will be able to use a vector map without topology. (importantly
> v.surf.rst still works)
>

Ayone know if the v.outlier, v.lidar.edgedetect, v. lidar.grow and
v.lidar.correction need topology?



>
> If you are willing to abandon your "code" data column, you can strip off
> that column and import the rest as a 3D vector (-z z=), in which case no
> table is created (or just ignore it with -t -z). If you are just
> interested in the 3D coordinate, then a DB table is unneeded overhead and
> is best skipped.
>

This approach turned out to be the best.  You (Hamish) were right that the
code was an intensity or strength of return signal.  I made a nice
"intensity" grayscale map, that was great.  Since I dont need the intensity,
or know how to use it to help separate the DEM from DSM features, the 3D
point worked perfect. Thanks!


>
>
> Question: what do you want to do with the data? Simply create a raster
> DEM or do more fancy cleaning with v.lidar.*? If just creating a raster
> DEM you might skip v.in.ascii altogether and use the r.in.xyz module.
>
>
I will definitely look more closely at the r.in.xyz module.  I had used it
for some binning and statistical analysis of the point distribution of the
lidar.  it is pretty tight data.

I want/need to do fancy cleaning with the v.lidar.* tools.  However, not
having pulse return counts to work with like the example in the book, I am
not sure how well it will work, because only having an intensity value, I'm
not sure how that relates to first and last returns.  What I would like to
do with the data is create a bare earth DEM, and a DSM of the elevated
surface features.  Then perhaps add elevated surface features like solid
objects (e.g. buildings, etc) that would block terrain for shallow overland
flow or r.lake simulations for flooding.

Since the application is for watershed analysis (feature building, and
ultimately running hydrology models in GRASS), so my goal is to create the
most realistic hydrologic surface model to properly route the water.  In
theory, it seems to me that water should not route through solid features
like buildings (extrude as walls?), but should be able to (and often does)
route through shrubby or forested areas.  In these areas, where there is
canopy, it seems that those canopy elevations should be replaced with bare
earth elevations, but allow water to flow through these areas.  This is the
first time I have worked with raw LiDAR data to create features or do
watershed analysis, so this is a fun learning experience.  Typically, I had
worked with contour data in the past.

Mark
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.osgeo.org/pipermail/grass-user/attachments/20080414/717de0f5/attachment.html