<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=us-ascii"><meta name=Generator content="Microsoft Word 14 (filtered medium)"><title>Re: [Liblas-devel] I/O performance? -- your help requested!</title><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=EN-US link=blue vlink=purple><div class=WordSection1><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Your box is too fast. :-(<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Could you maybe do the runs again, but with “–n 100000000” (100x larger)?<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>-mpg<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><div style='border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt'><div><div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in'><p class=MsoNormal><b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span></b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'> Smith, Michael ERDC-CRREL-NH [mailto:michael.smith@usace.army.mil] <br><b>Sent:</b> Saturday, February 05, 2011 4:48 AM<br><b>To:</b> mpg@flaxen.com; liblas-devel@lists.osgeo.org<br><b>Subject:</b> Re: [Liblas-devel] I/O performance? -- your help requested!<o:p></o:p></span></p></div></div><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal style='margin-bottom:12.0pt'><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>lidar@lidarora1 tmp]$ uname -a<br>Linux lidarora1 2.6.32-100.0.19.el5 #1 SMP Fri Sep 17 17:51:41 EDT 2010 x86_64 x86_64 x86_64 GNU/Linux<br><br>2 CPU, 4cores/cpu + hyperthreading = 16 virtual cores <br><br>processor : 15<br>vendor_id : GenuineIntel<br>cpu family : 6<br>model : 26<br>model name : Intel(R) Xeon(R) CPU X5570 @ 2.93GHz<br>stepping : 5<br>cpu MHz : 1600.000<br>cache size : 8192 KB<br><br><br><br>[lidar@lidarora1 tmp]$ time laszippertest -n 1000000<br>(skipping range coder test)<br>laszipper1 wrote 7000000 bytes in 0.03 seconds<br>laszipper2 wrote 3898142 bytes in 0.13 seconds<br>SUCCESS: lasunzipper1 read 7000000 bytes in 0.02 seconds<br>SUCCESS: lasunzipper2 read 3898142 bytes in 0.18 seconds<br><br>real 0m0.379s<br>user 0m0.349s<br>sys 0m0.029s<br>[lidar@lidarora1 tmp]$ time laszippertest -n 1000000<br>(skipping range coder test)<br>laszipper1 wrote 7000000 bytes in 0.03 seconds<br>laszipper2 wrote 3898142 bytes in 0.13 seconds<br>SUCCESS: lasunzipper1 read 7000000 bytes in 0.02 seconds<br>SUCCESS: lasunzipper2 read 3898142 bytes in 0.17 seconds<br><br>real 0m0.366s<br>user 0m0.336s<br>sys 0m0.029s<br>[lidar@lidarora1 tmp]$ time laszippertest -n 1000000<br>(skipping range coder test)<br>laszipper1 wrote 7000000 bytes in 0.03 seconds<br>laszipper2 wrote 3898142 bytes in 0.13 seconds<br>SUCCESS: lasunzipper1 read 7000000 bytes in 0.02 seconds<br>SUCCESS: lasunzipper2 read 3898142 bytes in 0.16 seconds<br><br>real 0m0.366s<br>user 0m0.338s<br>sys 0m0.026s<br>[lidar@lidarora1 tmp]$ time laszippertest -n 1000000 -s<br>(skipping range coder test)<br>laszipper1 wrote 7000000 bytes in 0.03 seconds<br>laszipper2 wrote 3898142 bytes in 0.13 seconds<br>SUCCESS: lasunzipper1 read 7000000 bytes in 0.02 seconds<br>SUCCESS: lasunzipper2 read 3898142 bytes in 0.16 seconds<br><br>real 0m0.364s<br>user 0m0.338s<br>sys 0m0.024s<br>[lidar@lidarora1 tmp]$ time laszippertest -n 1000000 -s<br>(skipping range coder test)<br>laszipper1 wrote 7000000 bytes in 0.03 seconds<br>laszipper2 wrote 3898142 bytes in 0.13 seconds<br>SUCCESS: lasunzipper1 read 7000000 bytes in 0.02 seconds<br>SUCCESS: lasunzipper2 read 3898142 bytes in 0.17 seconds<br><br>real 0m0.369s<br>user 0m0.339s<br>sys 0m0.028s<br>[lidar@lidarora1 tmp]$ time laszippertest -n 1000000 -s<br>(skipping range coder test)<br>laszipper1 wrote 7000000 bytes in 0.03 seconds<br>laszipper2 wrote 3898142 bytes in 0.13 seconds<br>SUCCESS: lasunzipper1 read 7000000 bytes in 0.02 seconds<br>SUCCESS: lasunzipper2 read 3898142 bytes in 0.17 seconds<br><br>real 0m0.364s<br>user 0m0.335s<br>sys 0m0.027s<br><br><br><br>-- <br>Michael Smith<br>Remote Sensing/GIS Center<br>US Army Corps of Engineers<br><br><br>On 2/4/11 11:39 AM, "Michael P. Gerlek" <<a href="mpg@flaxen.com">mpg@flaxen.com</a>> wrote:</span><o:p></o:p></p><p class=MsoNormal style='margin-bottom:12.0pt'><span style='font-size:11.0pt;font-family:"Calibri","sans-serif"'>Do you have 5-10 minutes to spare today?<br><br>Your libLAS team (well, me anyway) is wondering about I/O performance of the<br>liblas kit -- specifically, when doing binary reading and writing, is there<br>any fundamental performance difference between using C-style FILE* I/O and<br>C++-style stream I/O? And if streams are better, would boost's stream be<br>better still? If you google around a bit, you'll find lots of contradictory<br>(and sometimes overly passionate) statements about this topic. At the end<br>of the day, though, the consensus seems to be that:<br><br> (1) you need to be "smart" if you're using C++ I/O -- it is easy to shoot<br>yourself in the foot<br> (2) modern C++ streams are implemented on top of the native OS APIs<br> (3) under Visual Studio, FILE* operations and streams are both implemented<br>using the win32 APIs, but streams have an additional lock (that is claimed<br>by some to be not needed)<br><br>and, most importantly,<br><br> (4) performance varies greatly with different I/O patterns, e.g. large<br>sequential block reads vs small random reads<br><br>Very fortunately, we happen to already have a rough, 1st-order I/O<br>performance test built into the laszip tree. If you have that tree built<br>(<a href="http://hg.liblas.org/zip">http://hg.liblas.org/zip</a>), in Release mode, could you please send me the<br>results of running the "laszippertest" test app, as follows?<br><br> time ./laszippertest -n 1000000<br> time ./laszippertest -n 1000000<br> time ./laszippertest -n 1000000<br> time ./laszippertest -n 1000000 -s<br> time ./laszippertest -n 1000000 -s<br> time ./laszippertest -n 1000000 -s<br><br>The first three runs will encode and decode 1 million random points using<br>FILEs, and the second three will do it with streams. This is not a perfect<br>test, but it represents something approximating the real I/O footprint or<br>traces that liblas uses.<br><br>Oh, and be sure to include the kind of platform (processor speed, compiler,<br>OS) you're running it on.<br><br>Thanks much!<br><br>-mpg<br><br><br>_______________________________________________<br>Liblas-devel mailing list<br><a href="Liblas-devel@lists.osgeo.org">Liblas-devel@lists.osgeo.org</a><br><a href="http://lists.osgeo.org/mailman/listinfo/liblas-devel">http://lists.osgeo.org/mailman/listinfo/liblas-devel</a></span><o:p></o:p></p></div></div></body></html>