<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
OK I feel better now. I see Michael's and mine are about the same.<br>
<br>
I did 100x and something went out of range I think:<br>
<br>
$ time laszippertest -n 100000000<br>
Settings:<br>
num_points=100000000, use_iostream=false, run_forever=false,
use_random=false, user_seed=1297100531<br>
laszipper wrote -1589934592 bytes in 58.12 seconds<br>
<br>
Note the negative, also had to control-c.<br>
<br>
6 10x below and 2 million points. Hope this helps...<br>
<br>
[dburken@tazz build]$ time laszippertest -n 10000000<br>
Settings:<br>
num_points=10000000, use_iostream=false, run_forever=false,
use_random=false, user_seed=1297100699<br>
laszipper wrote 700000000 bytes in 5.85 seconds<br>
SUCCESS: lasunzipper read 700000000 bytes in 3.85 seconds<br>
laszipper wrote 381625423 bytes in 17.62 seconds<br>
SUCCESS: lasunzipper read 381625422 bytes in 28.49 seconds<br>
Finished 1 runs<br>
<br>
<br>
real 0m59.820s<br>
user 0m51.617s<br>
sys 0m5.002s<br>
[dburken@tazz build]$ <br>
[dburken@tazz build]$ time laszippertest -n 10000000<br>
Settings:<br>
num_points=10000000, use_iostream=false, run_forever=false,
use_random=false, user_seed=1297100974<br>
laszipper wrote 700000000 bytes in 4.91 seconds<br>
SUCCESS: lasunzipper read 700000000 bytes in 2.63 seconds<br>
laszipper wrote 381625423 bytes in 18.51 seconds<br>
SUCCESS: lasunzipper read 381625422 bytes in 30.89 seconds<br>
Finished 1 runs<br>
<br>
<br>
real 0m58.074s<br>
user 0m53.394s<br>
sys 0m3.840s<br>
[dburken@tazz build]$ time laszippertest -n 10000000<br>
Settings:<br>
num_points=10000000, use_iostream=false, run_forever=false,
use_random=false, user_seed=1297101035<br>
laszipper wrote 700000000 bytes in 4.79 seconds<br>
SUCCESS: lasunzipper read 700000000 bytes in 2.67 seconds<br>
laszipper wrote 381625423 bytes in 20.48 seconds<br>
SUCCESS: lasunzipper read 381625422 bytes in 23.44 seconds<br>
Finished 1 runs<br>
<br>
<br>
real 0m52.201s<br>
user 0m47.626s<br>
sys 0m4.031s<br>
[dburken@tazz build]$ time laszippertest -n 10000000 -s<br>
Settings:<br>
num_points=10000000, use_iostream=true, run_forever=false,
use_random=false, user_seed=1297101129<br>
laszipper wrote 700000000 bytes in 6.81 seconds<br>
SUCCESS: lasunzipper read 700000000 bytes in 4.76 seconds<br>
laszipper wrote 381625423 bytes in 21.66 seconds<br>
SUCCESS: lasunzipper read 381625422 bytes in 28.8 seconds<br>
Finished 1 runs<br>
<br>
<br>
real 1m2.768s<br>
user 0m55.831s<br>
sys 0m6.590s<br>
[dburken@tazz build]$ time laszippertest -n 10000000 -s<br>
Settings:<br>
num_points=10000000, use_iostream=true, run_forever=false,
use_random=false, user_seed=1297101209<br>
laszipper wrote 700000000 bytes in 5.09 seconds<br>
SUCCESS: lasunzipper read 700000000 bytes in 3.53 seconds<br>
laszipper wrote 381625423 bytes in 17.39 seconds<br>
SUCCESS: lasunzipper read 381625422 bytes in 27.96 seconds<br>
Finished 1 runs<br>
<br>
<br>
real 0m54.278s<br>
user 0m49.023s<br>
sys 0m5.235s<br>
[dburken@tazz build]$ time laszippertest -n 10000000 -s<br>
Settings:<br>
num_points=10000000, use_iostream=true, run_forever=false,
use_random=false, user_seed=1297101272<br>
laszipper wrote 700000000 bytes in 5.2 seconds<br>
SUCCESS: lasunzipper read 700000000 bytes in 3.58 seconds<br>
laszipper wrote 381625423 bytes in 17.02 seconds<br>
SUCCESS: lasunzipper read 381625422 bytes in 24.59 seconds<br>
Finished 1 runs<br>
<br>
<br>
real 0m50.899s<br>
user 0m45.558s<br>
sys 0m5.113s<br>
[dburken@tazz build]$ time laszippertest -n 1000000<br>
Settings:<br>
num_points=1000000, use_iostream=false, run_forever=false,
use_random=false, user_seed=1297101353<br>
laszipper wrote 70000000 bytes in 0.47 seconds<br>
SUCCESS: lasunzipper read 70000000 bytes in 0.27 seconds<br>
laszipper wrote 38271321 bytes in 1.55 seconds<br>
SUCCESS: lasunzipper read 38271321 bytes in 2.45 seconds<br>
Finished 1 runs<br>
<br>
<br>
real 0m5.060s<br>
user 0m4.435s<br>
sys 0m0.594s<br>
[dburken@tazz build]$ time laszippertest -n 1000000 -s<br>
Settings:<br>
num_points=1000000, use_iostream=true, run_forever=false,
use_random=false, user_seed=1297101361<br>
laszipper wrote 70000000 bytes in 0.5 seconds<br>
SUCCESS: lasunzipper read 70000000 bytes in 0.36 seconds<br>
laszipper wrote 38271321 bytes in 1.69 seconds<br>
SUCCESS: lasunzipper read 38271321 bytes in 2.48 seconds<br>
Finished 1 runs<br>
<br>
<br>
<br>
<br>
On 02/07/2011 12:02 PM, Michael P. Gerlek wrote:
<blockquote cite="mid:000601cbc6e8$d8015770$88040650$@flaxen.com"
type="cite">
<meta http-equiv="Content-Type" content="text/html;
charset=ISO-8859-1">
<meta name="Generator" content="Microsoft Word 14 (filtered
medium)">
<title>Re: [Liblas-devel] I/O performance? -- your help requested!</title>
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size: 11pt; font-family:
"Calibri","sans-serif"; color: rgb(31,
73, 125);">Your box is too fast. :-(<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size: 11pt; font-family:
"Calibri","sans-serif"; color: rgb(31,
73, 125);"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size: 11pt; font-family:
"Calibri","sans-serif"; color: rgb(31,
73, 125);">Could you maybe do the runs again, but with “–n
100000000” (100x larger)?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size: 11pt; font-family:
"Calibri","sans-serif"; color: rgb(31,
73, 125);"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size: 11pt; font-family:
"Calibri","sans-serif"; color: rgb(31,
73, 125);">-mpg<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size: 11pt; font-family:
"Calibri","sans-serif"; color: rgb(31,
73, 125);"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size: 11pt; font-family:
"Calibri","sans-serif"; color: rgb(31,
73, 125);"><o:p> </o:p></span></p>
<div style="border-width: medium medium medium 1.5pt;
border-style: none none none solid; border-color:
-moz-use-text-color -moz-use-text-color -moz-use-text-color
blue; padding: 0in 0in 0in 4pt;">
<div>
<div style="border-right: medium none; border-width: 1pt
medium medium; border-style: solid none none;
border-color: rgb(181, 196, 223) -moz-use-text-color
-moz-use-text-color; padding: 3pt 0in 0in;">
<p class="MsoNormal"><b><span style="font-size: 10pt;
font-family:
"Tahoma","sans-serif";">From:</span></b><span
style="font-size: 10pt; font-family:
"Tahoma","sans-serif";"> Smith,
Michael ERDC-CRREL-NH
[<a class="moz-txt-link-freetext" href="mailto:michael.smith@usace.army.mil">mailto:michael.smith@usace.army.mil</a>] <br>
<b>Sent:</b> Saturday, February 05, 2011 4:48 AM<br>
<b>To:</b> <a class="moz-txt-link-abbreviated" href="mailto:mpg@flaxen.com">mpg@flaxen.com</a>;
<a class="moz-txt-link-abbreviated" href="mailto:liblas-devel@lists.osgeo.org">liblas-devel@lists.osgeo.org</a><br>
<b>Subject:</b> Re: [Liblas-devel] I/O performance? --
your help requested!<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal" style="margin-bottom: 12pt;"><span
style="font-size: 11pt; font-family:
"Calibri","sans-serif";">lidar@lidarora1
tmp]$ uname -a<br>
Linux lidarora1 2.6.32-100.0.19.el5 #1 SMP Fri Sep 17
17:51:41 EDT 2010 x86_64 x86_64 x86_64 GNU/Linux<br>
<br>
2 CPU, 4cores/cpu + hyperthreading = 16 virtual cores <br>
<br>
processor : 15<br>
vendor_id : GenuineIntel<br>
cpu family : 6<br>
model : 26<br>
model name : Intel(R) Xeon(R) CPU X5570 @
2.93GHz<br>
stepping : 5<br>
cpu MHz : 1600.000<br>
cache size : 8192 KB<br>
<br>
<br>
<br>
[lidar@lidarora1 tmp]$ time laszippertest -n 1000000<br>
(skipping range coder test)<br>
laszipper1 wrote 7000000 bytes in 0.03 seconds<br>
laszipper2 wrote 3898142 bytes in 0.13 seconds<br>
SUCCESS: lasunzipper1 read 7000000 bytes in 0.02 seconds<br>
SUCCESS: lasunzipper2 read 3898142 bytes in 0.18 seconds<br>
<br>
real 0m0.379s<br>
user 0m0.349s<br>
sys 0m0.029s<br>
[lidar@lidarora1 tmp]$ time laszippertest -n 1000000<br>
(skipping range coder test)<br>
laszipper1 wrote 7000000 bytes in 0.03 seconds<br>
laszipper2 wrote 3898142 bytes in 0.13 seconds<br>
SUCCESS: lasunzipper1 read 7000000 bytes in 0.02 seconds<br>
SUCCESS: lasunzipper2 read 3898142 bytes in 0.17 seconds<br>
<br>
real 0m0.366s<br>
user 0m0.336s<br>
sys 0m0.029s<br>
[lidar@lidarora1 tmp]$ time laszippertest -n 1000000<br>
(skipping range coder test)<br>
laszipper1 wrote 7000000 bytes in 0.03 seconds<br>
laszipper2 wrote 3898142 bytes in 0.13 seconds<br>
SUCCESS: lasunzipper1 read 7000000 bytes in 0.02 seconds<br>
SUCCESS: lasunzipper2 read 3898142 bytes in 0.16 seconds<br>
<br>
real 0m0.366s<br>
user 0m0.338s<br>
sys 0m0.026s<br>
[lidar@lidarora1 tmp]$ time laszippertest -n 1000000 -s<br>
(skipping range coder test)<br>
laszipper1 wrote 7000000 bytes in 0.03 seconds<br>
laszipper2 wrote 3898142 bytes in 0.13 seconds<br>
SUCCESS: lasunzipper1 read 7000000 bytes in 0.02 seconds<br>
SUCCESS: lasunzipper2 read 3898142 bytes in 0.16 seconds<br>
<br>
real 0m0.364s<br>
user 0m0.338s<br>
sys 0m0.024s<br>
[lidar@lidarora1 tmp]$ time laszippertest -n 1000000 -s<br>
(skipping range coder test)<br>
laszipper1 wrote 7000000 bytes in 0.03 seconds<br>
laszipper2 wrote 3898142 bytes in 0.13 seconds<br>
SUCCESS: lasunzipper1 read 7000000 bytes in 0.02 seconds<br>
SUCCESS: lasunzipper2 read 3898142 bytes in 0.17 seconds<br>
<br>
real 0m0.369s<br>
user 0m0.339s<br>
sys 0m0.028s<br>
[lidar@lidarora1 tmp]$ time laszippertest -n 1000000 -s<br>
(skipping range coder test)<br>
laszipper1 wrote 7000000 bytes in 0.03 seconds<br>
laszipper2 wrote 3898142 bytes in 0.13 seconds<br>
SUCCESS: lasunzipper1 read 7000000 bytes in 0.02 seconds<br>
SUCCESS: lasunzipper2 read 3898142 bytes in 0.17 seconds<br>
<br>
real 0m0.364s<br>
user 0m0.335s<br>
sys 0m0.027s<br>
<br>
<br>
<br>
-- <br>
Michael Smith<br>
Remote Sensing/GIS Center<br>
US Army Corps of Engineers<br>
<br>
<br>
On 2/4/11 11:39 AM, "Michael P. Gerlek" <<a
moz-do-not-send="true" href="mpg@flaxen.com">mpg@flaxen.com</a>>
wrote:</span><o:p></o:p></p>
<p class="MsoNormal" style="margin-bottom: 12pt;"><span
style="font-size: 11pt; font-family:
"Calibri","sans-serif";">Do you have
5-10 minutes to spare today?<br>
<br>
Your libLAS team (well, me anyway) is wondering about I/O
performance of the<br>
liblas kit -- specifically, when doing binary reading and
writing, is there<br>
any fundamental performance difference between using
C-style FILE* I/O and<br>
C++-style stream I/O? And if streams are better, would
boost's stream be<br>
better still? If you google around a bit, you'll find
lots of contradictory<br>
(and sometimes overly passionate) statements about this
topic. At the end<br>
of the day, though, the consensus seems to be that:<br>
<br>
(1) you need to be "smart" if you're using C++ I/O -- it
is easy to shoot<br>
yourself in the foot<br>
(2) modern C++ streams are implemented on top of the
native OS APIs<br>
(3) under Visual Studio, FILE* operations and streams
are both implemented<br>
using the win32 APIs, but streams have an additional lock
(that is claimed<br>
by some to be not needed)<br>
<br>
and, most importantly,<br>
<br>
(4) performance varies greatly with different I/O
patterns, e.g. large<br>
sequential block reads vs small random reads<br>
<br>
Very fortunately, we happen to already have a rough,
1st-order I/O<br>
performance test built into the laszip tree. If you have
that tree built<br>
(<a moz-do-not-send="true" href="http://hg.liblas.org/zip">http://hg.liblas.org/zip</a>),
in Release mode, could you please send me the<br>
results of running the "laszippertest" test app, as
follows?<br>
<br>
time ./laszippertest -n 1000000<br>
time ./laszippertest -n 1000000<br>
time ./laszippertest -n 1000000<br>
time ./laszippertest -n 1000000 -s<br>
time ./laszippertest -n 1000000 -s<br>
time ./laszippertest -n 1000000 -s<br>
<br>
The first three runs will encode and decode 1 million
random points using<br>
FILEs, and the second three will do it with streams. This
is not a perfect<br>
test, but it represents something approximating the real
I/O footprint or<br>
traces that liblas uses.<br>
<br>
Oh, and be sure to include the kind of platform (processor
speed, compiler,<br>
OS) you're running it on.<br>
<br>
Thanks much!<br>
<br>
-mpg<br>
<br>
<br>
_______________________________________________<br>
Liblas-devel mailing list<br>
<a moz-do-not-send="true"
href="Liblas-devel@lists.osgeo.org">Liblas-devel@lists.osgeo.org</a><br>
<a moz-do-not-send="true"
href="http://lists.osgeo.org/mailman/listinfo/liblas-devel">http://lists.osgeo.org/mailman/listinfo/liblas-devel</a></span><o:p></o:p></p>
</div>
</div>
<pre wrap="">
<fieldset class="mimeAttachmentHeader"></fieldset>
_______________________________________________
Liblas-devel mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Liblas-devel@lists.osgeo.org">Liblas-devel@lists.osgeo.org</a>
<a class="moz-txt-link-freetext" href="http://lists.osgeo.org/mailman/listinfo/liblas-devel">http://lists.osgeo.org/mailman/listinfo/liblas-devel</a>
</pre>
</blockquote>
</body>
</html>