[GRASS-user] Re: new r.in.xyz Benchmarks

[Hamish]

Patton, Eric wrote:
> Good news with r.in.xyz. This program is really performing. I ran the
> following tests:
> 
> Input file: 2006_Garry_Trough_1.txt
> Size: 2.8GB
> Time to Import: 8min, 24sec.
> # of Points: 100,055,397
> 
> Input File: 2006_Garry_Trough_2.txt
> Size: 3.2GB
> Time to Import: 9min, 22sec.
> # of Points: 115,947,033
> 
> Input File: 2006_Garry_Trough_3.txt
> Size: 3.6GB
> Time to Import: 10min, 33sec
> # of Points: 126,162,261
> 
> This is really fantastic.

excellent. I'd still like to figure out a way to get LFS filesize
(probably kb is best).  system("ls -s") isn't portable..


> Before, I had to chop each input file into 1GB sections using awk,
> import, then patch back together. I couldn't even script that, as I
> was getting the double free/corruption errors, which have altogther
> vanished.

the double free bug has been fixed in CVS HEAD and the 6.2 branch.

It's a rare thing depending on points falling just a tiny fraction east
of the eastern bound of your region. So chaging the region slightly could
make it go away.

hmmm...
if the scan mode returns region values as "%11f" (1.0e-11), and
10*GRASS_EPSILON is 1.0e-14, there's the (tiny) space between those two
where it can overshoot? or resampling/resolution mismatch? rounding
error somewhere in the code or libgis?

oh well, it doesn't really matter. I think the code is fine now, those
points are "bonus" case anyway so it isn't a bug to lose one or two.

If you run in 6.3 and use 'g.gisenv set="DEBUG=3"' you now see a message
in the odd case that was triggering that bug (message is like: "skipping
extraneous data point"). It'll have something to do with the data file's
bounds and the region's bounds. If you do see that message, I'd like to
know the specifics of those.

> I'll be able to test it even better in about a month's time, as I'm
> getting massive LIDAR dataset that was just flown - about 0.25m
> resolution, at an area of about 5km X 15km - all in xyz format! ;)

Nice,

I've been waiting on commiting the LFS patches. I'll post another email
about them to the dev list.


Hamish