[Liblas-devel] I/O performance? -- your help requested!

[Smith, Michael ERDC-CRREL-NH]

lidar at lidarora1 tmp]$ uname -a
Linux lidarora1 2.6.32-100.0.19.el5 #1 SMP Fri Sep 17 17:51:41 EDT 2010
x86_64 x86_64 x86_64 GNU/Linux

2 CPU, 4cores/cpu + hyperthreading = 16 virtual cores

processor    : 15
vendor_id    : GenuineIntel
cpu family    : 6
model        : 26
model name    : Intel(R) Xeon(R) CPU           X5570  @ 2.93GHz
stepping    : 5
cpu MHz        : 1600.000
cache size    : 8192 KB



[lidar at lidarora1 tmp]$ time laszippertest -n 1000000
(skipping range coder test)
laszipper1 wrote 7000000 bytes in 0.03 seconds
laszipper2 wrote 3898142 bytes in 0.13 seconds
SUCCESS: lasunzipper1 read 7000000 bytes in 0.02 seconds
SUCCESS: lasunzipper2 read 3898142 bytes in 0.18 seconds

real    0m0.379s
user    0m0.349s
sys    0m0.029s
[lidar at lidarora1 tmp]$ time laszippertest -n 1000000
(skipping range coder test)
laszipper1 wrote 7000000 bytes in 0.03 seconds
laszipper2 wrote 3898142 bytes in 0.13 seconds
SUCCESS: lasunzipper1 read 7000000 bytes in 0.02 seconds
SUCCESS: lasunzipper2 read 3898142 bytes in 0.17 seconds

real    0m0.366s
user    0m0.336s
sys    0m0.029s
[lidar at lidarora1 tmp]$ time laszippertest -n 1000000
(skipping range coder test)
laszipper1 wrote 7000000 bytes in 0.03 seconds
laszipper2 wrote 3898142 bytes in 0.13 seconds
SUCCESS: lasunzipper1 read 7000000 bytes in 0.02 seconds
SUCCESS: lasunzipper2 read 3898142 bytes in 0.16 seconds

real    0m0.366s
user    0m0.338s
sys    0m0.026s
[lidar at lidarora1 tmp]$ time laszippertest -n 1000000 -s
(skipping range coder test)
laszipper1 wrote 7000000 bytes in 0.03 seconds
laszipper2 wrote 3898142 bytes in 0.13 seconds
SUCCESS: lasunzipper1 read 7000000 bytes in 0.02 seconds
SUCCESS: lasunzipper2 read 3898142 bytes in 0.16 seconds

real    0m0.364s
user    0m0.338s
sys    0m0.024s
[lidar at lidarora1 tmp]$ time laszippertest -n 1000000 -s
(skipping range coder test)
laszipper1 wrote 7000000 bytes in 0.03 seconds
laszipper2 wrote 3898142 bytes in 0.13 seconds
SUCCESS: lasunzipper1 read 7000000 bytes in 0.02 seconds
SUCCESS: lasunzipper2 read 3898142 bytes in 0.17 seconds

real    0m0.369s
user    0m0.339s
sys    0m0.028s
[lidar at lidarora1 tmp]$ time laszippertest -n 1000000 -s
(skipping range coder test)
laszipper1 wrote 7000000 bytes in 0.03 seconds
laszipper2 wrote 3898142 bytes in 0.13 seconds
SUCCESS: lasunzipper1 read 7000000 bytes in 0.02 seconds
SUCCESS: lasunzipper2 read 3898142 bytes in 0.17 seconds

real    0m0.364s
user    0m0.335s
sys    0m0.027s



-- 
Michael Smith
Remote Sensing/GIS Center
US Army Corps of Engineers


On 2/4/11 11:39 AM, "Michael P. Gerlek" <mpg at flaxen.com> wrote:

> Do you have 5-10 minutes to spare today?
> 
> Your libLAS team (well, me anyway) is wondering about I/O performance of the
> liblas kit -- specifically, when doing binary reading and writing, is there
> any fundamental performance difference between using C-style FILE* I/O and
> C++-style stream I/O?  And if streams are better, would boost's stream be
> better still?  If you google around a bit, you'll find lots of contradictory
> (and sometimes overly passionate) statements about this topic.  At the end
> of the day, though, the consensus seems to be that:
> 
>   (1) you need to be "smart" if you're using C++ I/O -- it is easy to shoot
> yourself in the foot
>   (2) modern C++ streams are implemented on top of the native OS APIs
>   (3) under Visual Studio, FILE* operations and streams are both implemented
> using the win32 APIs, but streams have an additional lock (that is claimed
> by some to be not needed)
> 
> and, most importantly,
> 
>   (4) performance varies greatly with different I/O patterns, e.g. large
> sequential block reads vs small random reads
> 
> Very fortunately, we happen to already have a rough, 1st-order I/O
> performance test built into the laszip tree.  If you have that tree built
> (http://hg.liblas.org/zip), in Release mode, could you please send me the
> results of running the "laszippertest" test app, as follows?
> 
>     time ./laszippertest -n 1000000
>     time ./laszippertest -n 1000000
>     time ./laszippertest -n 1000000
>     time ./laszippertest -n 1000000 -s
>     time ./laszippertest -n 1000000 -s
>     time ./laszippertest -n 1000000 -s
> 
> The first three runs will encode and decode 1 million random points using
> FILEs, and the second three will do it with streams.  This is not a perfect
> test, but it represents something approximating the real I/O footprint or
> traces that liblas uses.
> 
> Oh, and be sure to include the kind of platform (processor speed, compiler,
> OS) you're running it on.
> 
> Thanks much!
> 
> -mpg
> 
> 
> _______________________________________________
> Liblas-devel mailing list
> Liblas-devel at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/liblas-devel
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.osgeo.org/pipermail/liblas-devel/attachments/20110205/25cfc9f8/attachment.html