<HTML>
<HEAD>
<TITLE>Re: [Liblas-devel] I/O performance? -- your help requested!</TITLE>
</HEAD>
<BODY>
<FONT FACE="Calibri, Verdana, Helvetica, Arial"><SPAN STYLE='font-size:11pt'>lidar@lidarora1 tmp]$ uname -a<BR>
Linux lidarora1 2.6.32-100.0.19.el5 #1 SMP Fri Sep 17 17:51:41 EDT 2010 x86_64 x86_64 x86_64 GNU/Linux<BR>
<BR>
2 CPU, 4cores/cpu + hyperthreading = 16 virtual cores <BR>
<BR>
processor : 15<BR>
vendor_id : GenuineIntel<BR>
cpu family : 6<BR>
model : 26<BR>
model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz<BR>
stepping : 5<BR>
cpu MHz : 1600.000<BR>
cache size : 8192 KB<BR>
<BR>
<BR>
<BR>
[lidar@lidarora1 tmp]$ time laszippertest -n 1000000<BR>
(skipping range coder test)<BR>
laszipper1 wrote 7000000 bytes in 0.03 seconds<BR>
laszipper2 wrote 3898142 bytes in 0.13 seconds<BR>
SUCCESS: lasunzipper1 read 7000000 bytes in 0.02 seconds<BR>
SUCCESS: lasunzipper2 read 3898142 bytes in 0.18 seconds<BR>
<BR>
real 0m0.379s<BR>
user 0m0.349s<BR>
sys 0m0.029s<BR>
[lidar@lidarora1 tmp]$ time laszippertest -n 1000000<BR>
(skipping range coder test)<BR>
laszipper1 wrote 7000000 bytes in 0.03 seconds<BR>
laszipper2 wrote 3898142 bytes in 0.13 seconds<BR>
SUCCESS: lasunzipper1 read 7000000 bytes in 0.02 seconds<BR>
SUCCESS: lasunzipper2 read 3898142 bytes in 0.17 seconds<BR>
<BR>
real 0m0.366s<BR>
user 0m0.336s<BR>
sys 0m0.029s<BR>
[lidar@lidarora1 tmp]$ time laszippertest -n 1000000<BR>
(skipping range coder test)<BR>
laszipper1 wrote 7000000 bytes in 0.03 seconds<BR>
laszipper2 wrote 3898142 bytes in 0.13 seconds<BR>
SUCCESS: lasunzipper1 read 7000000 bytes in 0.02 seconds<BR>
SUCCESS: lasunzipper2 read 3898142 bytes in 0.16 seconds<BR>
<BR>
real 0m0.366s<BR>
user 0m0.338s<BR>
sys 0m0.026s<BR>
[lidar@lidarora1 tmp]$ time laszippertest -n 1000000 -s<BR>
(skipping range coder test)<BR>
laszipper1 wrote 7000000 bytes in 0.03 seconds<BR>
laszipper2 wrote 3898142 bytes in 0.13 seconds<BR>
SUCCESS: lasunzipper1 read 7000000 bytes in 0.02 seconds<BR>
SUCCESS: lasunzipper2 read 3898142 bytes in 0.16 seconds<BR>
<BR>
real 0m0.364s<BR>
user 0m0.338s<BR>
sys 0m0.024s<BR>
[lidar@lidarora1 tmp]$ time laszippertest -n 1000000 -s<BR>
(skipping range coder test)<BR>
laszipper1 wrote 7000000 bytes in 0.03 seconds<BR>
laszipper2 wrote 3898142 bytes in 0.13 seconds<BR>
SUCCESS: lasunzipper1 read 7000000 bytes in 0.02 seconds<BR>
SUCCESS: lasunzipper2 read 3898142 bytes in 0.17 seconds<BR>
<BR>
real 0m0.369s<BR>
user 0m0.339s<BR>
sys 0m0.028s<BR>
[lidar@lidarora1 tmp]$ time laszippertest -n 1000000 -s<BR>
(skipping range coder test)<BR>
laszipper1 wrote 7000000 bytes in 0.03 seconds<BR>
laszipper2 wrote 3898142 bytes in 0.13 seconds<BR>
SUCCESS: lasunzipper1 read 7000000 bytes in 0.02 seconds<BR>
SUCCESS: lasunzipper2 read 3898142 bytes in 0.17 seconds<BR>
<BR>
real 0m0.364s<BR>
user 0m0.335s<BR>
sys 0m0.027s<BR>
<BR>
<BR>
<BR>
-- <BR>
Michael Smith<BR>
Remote Sensing/GIS Center<BR>
US Army Corps of Engineers<BR>
<BR>
<BR>
On 2/4/11 11:39 AM, "Michael P. Gerlek" <<a href="mpg@flaxen.com">mpg@flaxen.com</a>> wrote:<BR>
<BR>
</SPAN></FONT><BLOCKQUOTE><FONT FACE="Calibri, Verdana, Helvetica, Arial"><SPAN STYLE='font-size:11pt'>Do you have 5-10 minutes to spare today?<BR>
<BR>
Your libLAS team (well, me anyway) is wondering about I/O performance of the<BR>
liblas kit -- specifically, when doing binary reading and writing, is there<BR>
any fundamental performance difference between using C-style FILE* I/O and<BR>
C++-style stream I/O? And if streams are better, would boost's stream be<BR>
better still? If you google around a bit, you'll find lots of contradictory<BR>
(and sometimes overly passionate) statements about this topic. At the end<BR>
of the day, though, the consensus seems to be that:<BR>
<BR>
(1) you need to be "smart" if you're using C++ I/O -- it is easy to shoot<BR>
yourself in the foot<BR>
(2) modern C++ streams are implemented on top of the native OS APIs<BR>
(3) under Visual Studio, FILE* operations and streams are both implemented<BR>
using the win32 APIs, but streams have an additional lock (that is claimed<BR>
by some to be not needed)<BR>
<BR>
and, most importantly,<BR>
<BR>
(4) performance varies greatly with different I/O patterns, e.g. large<BR>
sequential block reads vs small random reads<BR>
<BR>
Very fortunately, we happen to already have a rough, 1st-order I/O<BR>
performance test built into the laszip tree. If you have that tree built<BR>
(<a href="http://hg.liblas.org/zip">http://hg.liblas.org/zip</a>), in Release mode, could you please send me the<BR>
results of running the "laszippertest" test app, as follows?<BR>
<BR>
time ./laszippertest -n 1000000<BR>
time ./laszippertest -n 1000000<BR>
time ./laszippertest -n 1000000<BR>
time ./laszippertest -n 1000000 -s<BR>
time ./laszippertest -n 1000000 -s<BR>
time ./laszippertest -n 1000000 -s<BR>
<BR>
The first three runs will encode and decode 1 million random points using<BR>
FILEs, and the second three will do it with streams. This is not a perfect<BR>
test, but it represents something approximating the real I/O footprint or<BR>
traces that liblas uses.<BR>
<BR>
Oh, and be sure to include the kind of platform (processor speed, compiler,<BR>
OS) you're running it on.<BR>
<BR>
Thanks much!<BR>
<BR>
-mpg<BR>
<BR>
<BR>
_______________________________________________<BR>
Liblas-devel mailing list<BR>
<a href="Liblas-devel@lists.osgeo.org">Liblas-devel@lists.osgeo.org</a><BR>
<a href="http://lists.osgeo.org/mailman/listinfo/liblas-devel">http://lists.osgeo.org/mailman/listinfo/liblas-devel</a><BR>
<BR>
</SPAN></FONT></BLOCKQUOTE>
</BODY>
</HTML>