<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
</head>
<body style="color: rgb(0, 0, 0); font-size: 14px; font-family: Calibri, sans-serif; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">
<div>Roger,</div>
<span id="OLK_SRC_BODY_SECTION">
<div>
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px; ">
<br>
</div>
<div><font class="Apple-style-span" face="Calibri,sans-serif">That seems very slow to me. With the oracle driver, I was able to extract an area of 0.352 sq miles with 1047792 points and it took 11.9 seconds. </font><span class="Apple-style-span" style="font-family: Calibri, sans-serif; ">The
pg driver is much newer than the oracle driver however and hasn't had as much testing but that is a lot slower than I would expect. One note, w</span><span class="Apple-style-span" style="font-family: Calibri, sans-serif; ">e do store our data with 100k points
per block/row. For oracle we've found that more points per row is more efficient.</span></div>
<div><font class="Apple-style-span" face="Calibri,sans-serif"><br>
</font></div>
<div><font class="Apple-style-span" face="Calibri,sans-serif">One thing I would check is the access plan of your spatial query. Is it using a spatial index to access the correct rows or is it using a full table scan. You could change your where option to set
to a set of primary keys and see if its a lot faster.</font></div>
<div><font class="Apple-style-span" face="Calibri,sans-serif"><br>
</font></div>
<div><font class="Apple-style-span" face="Calibri,sans-serif">I'd also check that your PDAL is compiled into release mode. That can also make a huge difference.</font></div>
<div><font class="Apple-style-span" face="Calibri,sans-serif"><br>
</font></div>
<div><font class="Apple-style-span" face="Calibri,sans-serif">Mike</font></div>
<div><font class="Apple-style-span" face="Calibri,sans-serif"><br>
</font></div>
<div><font class="Apple-style-span" face="Calibri,sans-serif">
<div>
<div>
<div>-- </div>
<div>Michael Smith</div>
</div>
<div>US Army Corps</div>
<div>Remote Sensing GIS/Center</div>
</div>
</font></div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px; ">
<br>
</div>
<span id="OLK_SRC_BODY_SECTION" style="color: rgb(0, 0, 0); font-size: 14px; font-family: Calibri, sans-serif; ">
<div style="font-family:Calibri; font-size:11pt; text-align:left; color:black; BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 0in; PADDING-RIGHT: 0in; BORDER-TOP: #b5c4df 1pt solid; BORDER-RIGHT: medium none; PADDING-TOP: 3pt">
<span style="font-weight:bold">From: </span>Roger Bedell <<a href="mailto:rbedell@coordinatesolutions.com">rbedell@coordinatesolutions.com</a>><br>
<span style="font-weight:bold">Date: </span>Saturday, August 31, 2013 9:55 AM<br>
<span style="font-weight:bold">To: </span>"<a href="mailto:pgpointcloud@lists.osgeo.org">pgpointcloud@lists.osgeo.org</a>" <<a href="mailto:pgpointcloud@lists.osgeo.org">pgpointcloud@lists.osgeo.org</a>><br>
<span style="font-weight:bold">Cc: </span>David Lowther <<a href="mailto:dlowther@coordinatesolutions.com">dlowther@coordinatesolutions.com</a>><br>
<span style="font-weight:bold">Subject: </span>[EXTERNAL] [pgpointcloud] Retrieval seems slow<br>
<span style="font-weight:bold">Resent-From: </span>Michael Smith <<a href="mailto:michael.smith@usace.army.mil">michael.smith@usace.army.mil</a>><br>
</div>
<div><br>
</div>
<div>
<div>
<div dir="ltr">
<div>Hello again,</div>
<div><br>
</div>
<div>Ok, finally got a reasonable number of points to import into our database. Took a while because of a hardware failure, but now it is done.</div>
<div><br>
</div>
<div>I imported about 2500 LAS files, each one about 8mb into a single table. The resulting table has 55 million records (counted not estimated). Each record has 400 points. Database size is about 375gb.</div>
<div><br>
</div>
<div>I am attempting to do the following from pcpipeline:</div>
<div><br>
</div>
<div>
<div><?xml version="1.0" encoding="utf-8"?></div>
<div><Pipeline version="1.0"></div>
<div> <Writer type="drivers.las.writer"></div>
<div> <Option name="filename">test_small.las</Option></div>
<div> <Option name="spatialreference">EPSG:4326</Option></div>
<div> <Reader type="drivers.pgpointcloud.reader"></div>
<div><span style="white-space:pre-wrap"></span><Option name="connection">host='ogisrv102' port='8080' dbname='NRCS_LIDAR' user='postgres'password='5Tb3CwBu353w' </Option></div>
<div> <Option name="table">all_returns</Option></div>
<div> <Option name="column">pa</Option></div>
<div> <Option name="srid">4326</Option></div>
<div> <Option name="where">PC_Intersects(pa, ST_MakeEnvelope(-97.27021, 34.46225, -97.26987, 34.46251, 4326))</Option></div>
<div> </Reader></div>
<div> </Writer></div>
<div></Pipeline></div>
</div>
<div><br>
</div>
<div>Start time 8:24, end time 8:30 for the program to get here:</div>
<div><br>
</div>
<div>
<div>F:\PDAL64>pdal64\pcpipeline -i export_pg_to_las_test.xml</div>
<div>Requested to read 700 points</div>
<div>Requested to write 700 points</div>
<div>Buffer capacity is 700</div>
<div>0</div>
</div>
<div><br>
</div>
<div>Finally completes at 08:35:</div>
<div><br>
</div>
<div>
<div>F:\PDAL64>pdal64\pcpipeline -i export_pg_to_las_test.xml</div>
<div>Requested to read 700 points</div>
<div>Requested to write 700 points</div>
<div>Buffer capacity is 700</div>
<div>0.100</div>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div>The new database server has reasonable horsepower. 2 Xeon e5-2690 CPUs, 32gb RAM. Postgresql is 9.2.4, 64 bit on a Windows 2008 Server OS. The 64 bit pgpointcloud build for Windows was provided by Regina. I built the 64 bit PDAL.</div>
<div><br>
</div>
<div>11 minutes seems excessive for extracting 700 points. Am I missing something, or is this to be expected?</div>
<div><br>
</div>
<div>Thanks for any insight,</div>
Roger<br clear="all">
<div><br>
</div>
-- <br>
Roger Bedell<br>
Coordinate Solutions Inc.<br>
</div>
</div>
</div>
</span></div>
</div>
</span>
</body>
</html>