[pdal] looping multiple bounding coordinates in PDAL Pipeline
Nicolas Cadieux
nicolas.cadieux at archeotec.ca
Tue Dec 10 10:43:54 PST 2019
Hi,
This will print the bounds for each object in a shapefile.
Nicolas
On 2019-12-10 9:49 a.m., Jason McVay wrote:
> Hi Adam, and Nicolas,
>
> I think I understand conceptually how using shapely/geopandas to loop
> in coordinates to a pdal pipeline should work. But if you can pass
> along an example that would be greatly appreciated!
>
> Thanks again,
>
> Jason McVay
>
> MS Geography, Virginia Tech
> BA Environmental Studies, University of Montana
> www.linkedin.com/in/jasonmcvay86/
> <http://www.linkedin.com/in/jasonmcvay86/>
> https://twitter.com/jasonmcvay
>
>
> /"May your trails be crooked, winding, lonesome, dangerous, leading to
> the most amazing view"/
> - Ed Abbey
>
>
> On Mon, Dec 9, 2019 at 6:09 PM Nicolas Cadieux
> <nicolas.cadieux at archeotec.ca <mailto:nicolas.cadieux at archeotec.ca>>
> wrote:
>
> Hi,
> I will send him a similar python loop tomorrow for inspiration.
> Did not have time to look at this today.
> Nicolas
>
>> Le 9 déc. 2019 à 17:36, adam steer <adam.d.steer at gmail.com
>> <mailto:adam.d.steer at gmail.com>> a écrit :
>>
>>
>> Hi Jason
>>
>> Weighing in late here, it’s possible to cobble together
>> fiona/shapely/pdal to loop through a bunch of polygons (or
>> process them in parallel) and do what you need. It’s a task
>> that’s on my list of things to do when I get time :)
>>
>> That way you can assemble a processing pipeline which goes
>> straight from some geometries to data, without waiting for the
>> new PR..
>>
>> Cheers,
>>
>> Adam
>>
>>
>>
>> On Tue, 10 Dec 2019 at 07:42, Jason McVay <jasonmcvay09 at gmail.com
>> <mailto:jasonmcvay09 at gmail.com>> wrote:
>>
>> Thanks Howard! I think this is the way to go. I would be
>> interested in exploring the pull request version as well, but
>> I may have to wait until after the holiday break to get to that.
>> Jason McVay
>>
>> MS Geography, Virginia Tech
>> BA Environmental Studies, University of Montana
>> www.linkedin.com/in/jasonmcvay86/
>> <http://www.linkedin.com/in/jasonmcvay86/>
>> https://twitter.com/jasonmcvay
>>
>> /"May your trails be crooked, winding, lonesome, dangerous,
>> leading to the most amazing view"/
>> - Ed Abbey
>>
>>
>> On Mon, Dec 9, 2019 at 8:36 AM Howard Butler <howard at hobu.co
>> <mailto:howard at hobu.co>> wrote:
>>
>>
>>
>>> On Dec 8, 2019, at 7:09 PM, Jason McVay
>>> <jasonmcvay09 at gmail.com <mailto:jasonmcvay09 at gmail.com>>
>>> wrote:
>>>
>>> I'm looking for some advice on the best way/how to loop
>>> in thousands of bounding coordinates into a pdal pipeline.
>>>
>>> I have a csv (and a geojson) of several thousand min/max
>>> x/y and a unique ID. The AOI's are not very big, so the
>>> pipeline runs quickly, but there are a lot of AOIs to
>>> capture! I'm querying an entwine dataset, the extent of
>>> which is national, so I'm limiting the data with a
>>> bounding box of each AOI.
>>>
>>> My pipeline currently runs HAG and Ferry Z filter, then
>>> uses the gdal.writer to make a GeoTiff at 1m resolution.
>>> It works perfectly when I manually enter in a set of
>>> test coordinates. How can I scale this to loop and
>>> update the bounds automatically?
>>>
>>> I'm running this locally on a MacBook Pro.
>>>
>>> Thank you, any advice is appreciated!
>>
>> Jason,
>>
>> PDAL doesn't multithread or operate in a parallel fashion
>> for you. You must use external tools to do this yourself.
>> I have had good success using GNU parallel or xargs on
>> bash along with the Python multiprocessing library to
>> achieve that.
>>
>> You scenario would seem to fit that model quite well.
>> Here's a GNU parallel example. In short, use your
>> favorite scripting language (or sed/awk/cat) to write a
>> script that contains all of the job entries you need to
>> run (bounds entries are all the same in my example, but
>> you should get the point:
>>
>>> pdal pipeline pipe.json
>>> --readers.ept.filename="ept://http://path/to/location"
>>> --readers.ept.bounds="([-10063436.56, -10060190.36],
>>> [5038996.16, 5043062.79])"
>>> --writers.gdal.filename="hag_mean_henry_co.tif"
>>> pdal pipeline pipe.json
>>> --readers.ept.filename="ept://http://path/to/location"
>>> --readers.ept.bounds="([-10063436.56, -10060190.36],
>>> [5038996.16, 5043062.79])"
>>> --writers.gdal.filename="hag_mean_howard_co.tif"
>>> pdal pipeline pipe.json
>>> --readers.ept.filename="ept://http://path/to/location"
>>> --readers.ept.bounds="([-10063436.56, -10060190.36],
>>> [5038996.16, 5043062.79])"
>>> --writers.gdal.filename="hag_mean_james_co.tif"
>>> pdal pipeline pipe.json
>>> --readers.ept.filename="ept://http://path/to/location"
>>> --readers.ept.bounds="([-10063436.56, -10060190.36],
>>> [5038996.16, 5043062.79])"
>>> --writers.gdal.filename="hag_mean_mike_co.tif"
>>
>>
>> Then run that script:
>>
>>> parallel -j 16 < jobs.txt
>>
>> Filtering EPT resources with boundaries is a common
>> desire. I recently added a pull request to master (not
>> yet released) that allows you to specify filtering (for
>> faster query) and cropping (eliminating an extra stage
>> specification) for EPT resources. See
>> https://github.com/PDAL/PDAL/pull/2771#issue-323371431
>> <https://smex-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fgithub.com%2fPDAL%2fPDAL%2fpull%2f2771%23issue%2d323371431&umid=1ef705ee-0b38-4b1b-9373-2ccf4d0eb417&auth=ab4b424674be62c9f8f9e1c1a31e433d534186a3-f8290d8713ab4ed5b3666d9bbdf34b2c63f8b5c8> The
>> goal with the approach in the pull request is to not have
>> to change format of the bounding geometries to text
>> simply to feed them into a pipeline. We may add similar
>> capability to other drivers if it is indeed useful in
>> other contexts.
>>
>> With the PR, you could express your query boundaries as
>> an OGR query and then iterate through your EPT resources.
>> The current PR implementation doesn't "split" by the
>> polygons, however. We might need to add the same
>> capability to filters.crop to achieve that. Feedback is
>> appreciated so we can learn how people wish to use this.
>>
>> Howard
>>
>> _______________________________________________
>> pdal mailing list
>> pdal at lists.osgeo.org <mailto:pdal at lists.osgeo.org>
>> https://lists.osgeo.org/mailman/listinfo/pdal
>>
>>
>>
>> --
>> Dr. Adam Steer
>> http://spatialised.net
>> <https://smex-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=http%3a%2f%2fspatialised.net&umid=73b42ec6-c163-4304-9ef2-a1d38eac8f79&auth=72ce7397d0db234fdd09ad1e9584ffcc03ba0336-066b12425d4ebeb1bf4522439ac18d619101f119>
>> https://www.researchgate.net/profile/Adam_Steer
>> http://au.linkedin.com/in/adamsteer
>> http://orcid.org/0000-0003-0046-7236
>> +61 427 091 712 :: @adamdsteer
>>
>> Suits are bad for business:
>> http://www.spatialised.net/business-penguins/
>> _______________________________________________
>> pdal mailing list
>> pdal at lists.osgeo.org <mailto:pdal at lists.osgeo.org>
>> https://lists.osgeo.org/mailman/listinfo/pdal
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/pdal/attachments/20191210/91e80010/attachment-0001.html>
-------------- next part --------------
import fiona
import shapely
from shapely.geometry import shape
shpfile = fiona.open(r"C:\temp\Sample_data\INPUT_NETWORK_FILE_SHP_191105.shp")
for lines in shpfile:
geoms = shape(lines['geometry'])
print "\n"
#print geoms #print the object geometry
boundairies = geoms.bounds #print the object bounding box
print boundairies
More information about the pdal
mailing list