[gdal-dev] Use GDAL on AWS Lambda

[Harris, Tim]

I haven't used Serverless, but I'm running GDAL in AWS Lambda using binary wheels:

https://github.com/youngpm/gdalmanylinux

This repo is based on Sean Gillies's wheel build for Fiona, Rasterio, and Shapely (https://github.com/sgillies/frs-wheel-builds). I believe it has instructions, but the general idea is that it uses an old linux Docker image to build GDAL wheels for a few different Python versions. Once those are built, I use the pip --target option to install my Lambda code, any required Python modules, and the GDAL wheel into a local directory. Then I just zip up that directory, upload to S3, and run my Lambda function. The zip ends up being about 30 MB.

For example, to install the GDAL wheel into the local directory, I would do something like this:
pip install --target ./lambda_zip_dir ./wheels/GDAL-2.3.0-cp36-cp36m-manylinux1_x86_64.whl

Same idea for any other dependencies and for my own Lambda code.

-----Original Message-----
From: gdal-dev [mailto:gdal-dev-bounces at lists.osgeo.org] On Behalf Of kch
Sent: Thursday, July 12, 2018 1:31 PM
To: gdal-dev at lists.osgeo.org
Subject: [gdal-dev] Use GDAL on AWS Lambda

Hello, 

I am looking for any help from anyone who has experience with using GDAL on
AWS Lambda. I am trying to deploy a python function which uses GDAL onto AWS
Lambda. At the moment I am using Serverless to package and deploy my
function. I am at a roadblock because I can't figure out how to get all the
dependencies into my deployment package. My Serverless function packages the
dependencies listed in a requirements.txt using the
serverless-python-requirements plugin and dockerizePip: true option. At
deployment, I get the error: 

Serverless: Installing requirements of requirements.txt in .serverless…
Serverless: Docker Image: lambci/lambda:build-python3.6
Error --------------------------------------------------
The directory ‘/.cache/pip/http’ or its parent directory is not owned by the
current user and the cache has been disabled. Please check the permissions
and owner of that directory. If executing pip with sudo, you may want sudo’s
-H flag.
The directory ‘/.cache/pip’ or its parent directory is not owned by the
current user and caching wheels has been disabled. check the permissions and
owner of that directory. If executing pip with sudo, you may want sudo’s -H
flag.
Command “python setup.py egg_info” failed with error code 1 in
/tmp/pip-install-yo9exd91/pygdal/
For debugging logs, run again after setting the “SLS_DEBUG=*” environment
variable.

I have already updated setuptools to the latest version and even included
the latest version of setuptools in requirements.txt. I think the problem is
that at deployment, Serverless does not pick up all the non-python
dependencies of GDAL. I don't know how to work around this. 

I am running on Ubuntu 16.04 and Python3.6. Thank you in advance.




--
Sent from: http://osgeo-org.1560.x6.nabble.com/GDAL-Dev-f3742093.html
_______________________________________________
gdal-dev mailing list
gdal-dev at lists.osgeo.org
https://lists.osgeo.org/mailman/listinfo/gdal-dev