[gdal-dev] Speeding up gdalwarp process

[Grigory Pomadchin]

Hey Simon,

We had a successful enough experience with working with GDAL & Spark.

Initially there was a try to use GDAL JNI bindings for these purposes, but
it turned out that GDAL, and especially WARP (using its VRT driver), is not
completely thread safe.

You can have a look into an alternative version of JNI bindings (very
limited and covering warp only)
https://github.com/geotrellis/gdal-warp-bindings

However, even though bindings are thread safe, it was achieved by dataset
locks. Usage of GDAL in a highly multithreaded environment can still be
challenging in terms of a desired performance.

To resolve your need in Spark you can in general try GeoTrellis (and there
is a GDAL support to perform all warp steps).

On Fri, Jan 24, 2020 at 7:04 PM Marius Jigmond <mariusjigmond at hotmail.com>
wrote:

> We've been using Geotrellis (https://geotrellis.io/) for Spark jobs.
>
> -marius
> ------------------------------
> *From:* gdal-dev <gdal-dev-bounces at lists.osgeo.org> on behalf of Simon <
> twinbirds at protonmail.com>
> *Sent:* Thursday, January 23, 2020 6:30 PM
> *To:* gdal-dev at lists.osgeo.org <gdal-dev at lists.osgeo.org>
> *Subject:* [gdal-dev] Speeding up gdalwarp process
>
>
> Hi gdal-devs,
> I have a question, if there is some way to use gdalwarp in a clustering
> system (e.g., Sparks) or with GPU to speed up the process? My task involves
> re-projection and re-sampling of hundreds of high-resolution images. Any
> ideas to make use of Sparks or GPU is welcomed. Thank you.
> Simon
>
> _______________________________________________
> gdal-dev mailing list
> gdal-dev at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/gdal-dev



-- 
Grigory Pomadchin
LinkedIn: http://www.linkedin.com/in/grpomadchin
<https://vk.com/away.php?to=http%3A%2F%2Fwww.linkedin.com%2Fin%2Fgrpomadchin&cc_key=>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20200124/55ab2d06/attachment.html>