[gdal-dev] How to access remote data that requires authentication?
Even Rouault
even.rouault at spatialys.com
Tue Oct 5 12:10:55 PDT 2021
There are a few historically exported VSIInstaller symbols, but
VSIInstallCurlFileHandler is never exported. None of those symbols need
to be explicitly called as GDAL will automatically register the
builtin-handlers. Perhaps your gdal.dll lacks curl support ? I assumed
you pass the needed config options / env variables for that service.
Perhaps you could try with something simpler that doesn't require
authentication.
Le 05/10/2021 à 21:05, Joaquim Manuel Freire Luís a écrit :
>
> Even, sorry to continue this.
>
> Now is from Julia
>
> julia>
> gdalinfo("/vsicurl/https://lpdaac.earthdata.nasa.gov/lp-prod-protected/HLSS30.015/HLS.S30.T10TEK.2020273T190109.v1.5.B8A.tif")
>
> ERROR 4:
> `/vsicurl/https://lpdaac.earthdata.nasa.gov/lp-prod-protected/HLSS30.015/HLS.S30.T10TEK.2020273T190109.v1.5.B8A.tif'
> not recognized as a supported file format.
>
> I saw a reference to VSIInstallCurlFileHandler() so I thought “right I
> need this guy too” but when I try to install it
>
> julia> GMT.Gdal.VSIInstallCurlFileHandler()
>
> ERROR: could not load symbol "VSIInstallCurlFileHandler":
>
> The specified procedure could not be found.
>
> And indeed my gdal.dll has a couple of VSIInstaller symbols but not
> this one (nor zip, gzip and others).
>
> Is the “not supported file format” related to this? Shouldn’t that
> symbol be exported to the dll?
>
> *From:*gdal-dev <gdal-dev-bounces at lists.osgeo.org> *On Behalf Of
> *Joaquim Manuel Freire Luís
> *Sent:* Tuesday, October 5, 2021 5:59 PM
> *To:* Even Rouault <even.rouault at spatialys.com>; gdal-dev at lists.osgeo.org
> *Subject:* Re: [gdal-dev] How to access remote data that requires
> authentication?
>
> >The mention to _netrc is weird...
>
> On Windows is _netrc, not .netrc
>
> And I finally made it (twisted). The problem was that I detest where
> MS decides is my home dir (and all the hidden dirs stuff that it puts
> there) and always have had a home at “HOME=c:\j” and it’s there that
> libcurl (I presume) seeks for the netrc file.
>
> Thanks for all the tips. But I still wonder how GMT manages to read
> the file without anything of this.
>
> For the record if any other Windows user needs it
>
> gdalinfo
> /vsicurl/https://lpdaac.earthdata.nasa.gov/lp-prod-protected/HLSS30.015/HLS.S30.T10TEK.2020273T190109.v1.5.B8A.tif
> --config GDAL_DISABLE_READDIR_ON_OPEN EMPTY_DIR --config
> CPL_VSIL_CURL_USE_HEAD FALSE --config GDAL_HTTP_COOKIEFILE
> c:/TEMP/cookies.txt --config GDAL_HTTP_COOKIEJAR c:/TEMP/cookies.txt
>
> Driver: GTiff/GeoTIFF
>
> Files:
> /vsicurl/https://lpdaac.earthdata.nasa.gov/lp-prod-protected/HLSS30.015/HLS.S30.T10TEK.2020273T190109.v1.5.B8A.tif
>
> Size is 3660, 3660
>
> Coordinate System is:
>
> PROJCRS["UTM Zone 10, Northern Hemisphere",
>
> Regarding authentication issues, this service is quite annoying and
> requires enabling cookies. See
> https://lists.osgeo.org/pipermail/gdal-dev/2021-October/054728.html
> <https://lists.osgeo.org/pipermail/gdal-dev/2021-October/054728.html>
>
> You need to add things like /--config GDAL_HTTP_COOKIEFILE
> /tmp/cookies.txt --config GDAL_HTTP_COOKIEJAR /tmp/cookies.txt/
>
> Le 05/10/2021 à 18:20, Joaquim Manuel Freire Luís a écrit :
>
> OK, tried more things from that thread.
>
> gdalinfo /vsicurl/https://data.lpdaac.earthdatacloud.nasa.gov/lp-prod-protected/HLSL30.020/HLS.L30.T10TEK.2021192T184511.v2.0/HLS.L30.T10TEK.2021192T184511.v2.0.B04.tif <https://data.lpdaac.earthdatacloud.nasa.gov/lp-prod-protected/HLSL30.020/HLS.L30.T10TEK.2021192T184511.v2.0/HLS.L30.T10TEK.2021192T184511.v2.0.B04.tif> --config GDAL_DISABLE_READDIR_ON_OPEN EMPTY_DIR --config CPL_CURL_VERBOSE ON --config CPL_VSIL_CURL_USE_HEAD FALSE
>
> * Couldn't find host data.lpdaac.earthdatacloud.nasa.gov in the _netrc file; using defaults ...
>
> OK, right it's not there.
>
> But latter down it says
>
> * Couldn't find host urs.earthdata.nasa.gov in the _netrc file; using defaults
>
> Now this is not right.
>
> < HTTP/1.1 401 Unauthorized
>
> ...
>
> < WWW-Authenticate: Basic realm="Please enter your Earthdata Login credentials. If you do not have a Earthdata Login, create one athttps://urs.earthdata.nasa.gov//users/new <https://urs.earthdata.nasa.gov/users/new>"
>
> ...
>
> ERROR 11: HTTP response code: 401
>
> So I'm back to the Authentication problem. I do have an _netrc file (and .netrc btw) in my home dir as well as current dir but it does seem to find it. Is there something else that I must to in order to that file be found/used?
>
> -----Original Message-----
>
> From: thomas bonfort<thomas.bonfort at gmail.com> <mailto:thomas.bonfort at gmail.com>
>
> Sent: Tuesday, October 5, 2021 4:31 PM
>
> To: Joaquim Manuel Freire Luís<jluis at ualg.pt> <mailto:jluis at ualg.pt>
>
> Subject: Re: [gdal-dev] How to access remote data that requires authentication?
>
> for the 206 there seems to be a similar issue posted here a few days ago, search for "Problem accessing NASA Cloud Optimized GeoTIFF data"
>
> in the archives
>
> On Tue, Oct 5, 2021 at 5:26 PM Joaquim Manuel Freire Luís<jluis at ualg.pt> <mailto:jluis at ualg.pt> wrote:
>
> you should also change your password, now you have posted it on a
>
> public mailing list :/
>
> Shit, thanks for spotting it.
>
> But it doesn't work with /vsicur/ neither (had tried it before). Now
>
> the error is 206
>
> gdalinfo
>
> /vsicurl/https://user:pass@lpdaac.earthdata.nasa.gov/lp-prod-protected <https://user:pass@lpdaac.earthdata.nasa.gov/lp-prod-protected>
>
> /HLSS30.015/HLS.S30.T10TEK.2020273T190109.v1.5.B8A.tif
>
> ERROR 11: HTTP response code: 206
>
> On Tue, Oct 5, 2021 at 5:12 PM Joaquim Manuel Freire Luís<jluis at ualg.pt> <mailto:jluis at ualg.pt> wrote:
>
> Hi,
>
> I’ve read a lot of the docs, tried many -co options but can’t get through this mystery.
>
> I can access the data through GMT, which uses GDAL to do this job,
>
> but can’t do it with GDAL directly
>
> gdalinfo
>
> https://user:pass@lpdaac.earthdata.nasa.gov/lp-prod-protected/HLSS30 <https://user:pass@lpdaac.earthdata.nasa.gov/lp-prod-protected/HLSS30>
>
> .0 15/HLS.S30.T10TEK.2020273T190109.v1.5.B8A.tif
>
> ERROR 1: HTTP error code : 401
>
> gdalinfo failed - unable to open 'https://jluis:abaixo0Earthdata@lpdaac.earthdata.nasa.gov/lp-prod-protected/HLSS30.015/HLS.S30.T10TEK.2020273T190109.v1.5.B8A.tif <https://jluis:abaixo0Earthdata@lpdaac.earthdata.nasa.gov/lp-prod-protected/HLSS30.015/HLS.S30.T10TEK.2020273T190109.v1.5.B8A.tif>'.
>
> So passing longin:password via url does not work either. But it does
>
> if indirectly used
>
> grdinfo
>
> https://user:pass@lpdaac.earthdata.nasa.gov/lp-prod-protected/HLSS30 <https://user:pass@lpdaac.earthdata.nasa.gov/lp-prod-protected/HLSS30>
>
> .0 15/HLS.S30.T10TEK.2020273T190109.v1.5.B8A.tif
>
> HLS.S30.T10TEK.2020273T190109.v1.5.B8A.tif: Title: Grid imported via
>
> GDAL
>
> HLS.S30.T10TEK.2020273T190109.v1.5.B8A.tif: Command:
>
> HLS.S30.T10TEK.2020273T190109.v1.5.B8A.tif: Remark:
>
> HLS.S30.T10TEK.2020273T190109.v1.5.B8A.tif: Pixel node registration
>
> used [Cartesian grid]
>
> HLS.S30.T10TEK.2020273T190109.v1.5.B8A.tif: Grid file format: gd =
>
> Import/export through GDAL
>
> HLS.S30.T10TEK.2020273T190109.v1.5.B8A.tif: x_min: 499980 x_max:
>
> 609780 x_inc: 30 name: x n_columns: 3660
>
> HLS.S30.T10TEK.2020273T190109.v1.5.B8A.tif: y_min: 4390200 y_max:
>
> 4500000 y_inc: 30 name: y n_rows: 3660
>
> HLS.S30.T10TEK.2020273T190109.v1.5.B8A.tif: v_min: -0.0149 v_max:
>
> 0.8833 name: z
>
> HLS.S30.T10TEK.2020273T190109.v1.5.B8A.tif: scale_factor: 0.0001
>
> add_offset: 0 packed z-range: [-149,8833]
>
> +proj=utm +zone=10 +ellps=WGS84 +units=m +no_defs
>
> Joaquim
>
> _______________________________________________
>
> gdal-dev mailing list
>
> gdal-dev at lists.osgeo.org <mailto:gdal-dev at lists.osgeo.org>
>
> https://lists.osgeo.org/mailman/listinfo/gdal-dev <https://lists.osgeo.org/mailman/listinfo/gdal-dev>
>
> _______________________________________________
>
> gdal-dev mailing list
>
> gdal-dev at lists.osgeo.org <mailto:gdal-dev at lists.osgeo.org>
>
> https://lists.osgeo.org/mailman/listinfo/gdal-dev <https://lists.osgeo.org/mailman/listinfo/gdal-dev>
>
> --
> http://www.spatialys.com <http://www.spatialys.com>
> My software is free, but my time generally not.
--
http://www.spatialys.com
My software is free, but my time generally not.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20211005/f9557574/attachment-0001.html>
More information about the gdal-dev
mailing list