[gdal-dev] How to access remote data that requires authentication?

Tue Oct 5 09:59:22 PDT 2021

>The mention to _netrc is weird...

On Windows is _netrc, not .netrc

And I finally made it (twisted). The problem was that I detest where MS decides is my home dir (and all the hidden dirs stuff that it puts there) and always have had a home at “HOME=c:\j” and it’s there that libcurl (I presume) seeks for the netrc file.

Thanks for all the tips. But I still wonder how GMT manages to read the file without anything of this.

For the record if any other Windows user needs it

gdalinfo /vsicurl/https://lpdaac.earthdata.nasa.gov/lp-prod-protected/HLSS30.015/HLS.S30.T10TEK.2020273T190109.v1.5.B8A.tif --config GDAL_DISABLE_READDIR_ON_OPEN EMPTY_DIR --config CPL_VSIL_CURL_USE_HEAD FALSE --config GDAL_HTTP_COOKIEFILE c:/TEMP/cookies.txt --config GDAL_HTTP_COOKIEJAR c:/TEMP/cookies.txt

Driver: GTiff/GeoTIFF

Files: /vsicurl/https://lpdaac.earthdata.nasa.gov/lp-prod-protected/HLSS30.015/HLS.S30.T10TEK.2020273T190109.v1.5.B8A.tif

Size is 3660, 3660

Coordinate System is:

PROJCRS["UTM Zone 10, Northern Hemisphere",

Regarding authentication issues, this service is quite annoying and requires enabling cookies. See https://lists.osgeo.org/pipermail/gdal-dev/2021-October/054728.html

You need to add things like --config GDAL_HTTP_COOKIEFILE /tmp/cookies.txt --config GDAL_HTTP_COOKIEJAR /tmp/cookies.txt
Le 05/10/2021 à 18:20, Joaquim Manuel Freire Luís a écrit :

OK, tried more things from that thread.

gdalinfo /vsicurl/https://data.lpdaac.earthdatacloud.nasa.gov/lp-prod-protected/HLSL30.020/HLS.L30.T10TEK.2021192T184511.v2.0/HLS.L30.T10TEK.2021192T184511.v2.0.B04.tif --config GDAL_DISABLE_READDIR_ON_OPEN EMPTY_DIR --config CPL_CURL_VERBOSE ON --config CPL_VSIL_CURL_USE_HEAD FALSE

* Couldn't find host data.lpdaac.earthdatacloud.nasa.gov in the _netrc file; using defaults ...

OK, right it's not there.

But latter down it says

* Couldn't find host urs.earthdata.nasa.gov in the _netrc file; using defaults

Now this is not right.

< HTTP/1.1 401 Unauthorized

...

< WWW-Authenticate: Basic realm="Please enter your Earthdata Login credentials. If you do not have a Earthdata Login, create one at https://urs.earthdata.nasa.gov//users/new<https://urs.earthdata.nasa.gov/users/new>"

...

ERROR 11: HTTP response code: 401

So I'm back to the Authentication problem. I do have an _netrc file (and .netrc btw) in my home dir as well as current dir but it does seem to find it. Is there something else that I must to in order to that file be found/used?

-----Original Message-----

From: thomas bonfort <thomas.bonfort at gmail.com><mailto:thomas.bonfort at gmail.com>

Sent: Tuesday, October 5, 2021 4:31 PM

To: Joaquim Manuel Freire Luís <jluis at ualg.pt><mailto:jluis at ualg.pt>

Subject: Re: [gdal-dev] How to access remote data that requires authentication?

for the 206 there seems to be a similar issue posted here a few days ago, search for "Problem accessing NASA Cloud Optimized GeoTIFF data"

in the archives

On Tue, Oct 5, 2021 at 5:26 PM Joaquim Manuel Freire Luís <jluis at ualg.pt><mailto:jluis at ualg.pt> wrote:

you should also change your password, now you have posted it on a

public mailing list :/

Shit, thanks for spotting it.

But it doesn't work with /vsicur/ neither (had tried it  before). Now

the error is  206

gdalinfo

/vsicurl/https://user:pass@lpdaac.earthdata.nasa.gov/lp-prod-protected

/HLSS30.015/HLS.S30.T10TEK.2020273T190109.v1.5.B8A.tif

ERROR 11: HTTP response code: 206

On Tue, Oct 5, 2021 at 5:12 PM Joaquim Manuel Freire Luís <jluis at ualg.pt><mailto:jluis at ualg.pt> wrote:

Hi,

I’ve read a lot of the docs, tried many -co options but can’t get through this mystery.

I can access the data through GMT, which uses GDAL to do this job,

but can’t do it with GDAL directly

gdalinfo

https://user:pass@lpdaac.earthdata.nasa.gov/lp-prod-protected/HLSS30

.0 15/HLS.S30.T10TEK.2020273T190109.v1.5.B8A.tif

ERROR 1: HTTP error code : 401

gdalinfo failed - unable to open 'https://jluis:abaixo0Earthdata@lpdaac.earthdata.nasa.gov/lp-prod-protected/HLSS30.015/HLS.S30.T10TEK.2020273T190109.v1.5.B8A.tif'.

So passing longin:password via url does not work either. But it does

if indirectly used

grdinfo

https://user:pass@lpdaac.earthdata.nasa.gov/lp-prod-protected/HLSS30

.0 15/HLS.S30.T10TEK.2020273T190109.v1.5.B8A.tif

HLS.S30.T10TEK.2020273T190109.v1.5.B8A.tif: Title: Grid imported via

GDAL

HLS.S30.T10TEK.2020273T190109.v1.5.B8A.tif: Command:

HLS.S30.T10TEK.2020273T190109.v1.5.B8A.tif: Remark:

HLS.S30.T10TEK.2020273T190109.v1.5.B8A.tif: Pixel node registration

used [Cartesian grid]

HLS.S30.T10TEK.2020273T190109.v1.5.B8A.tif: Grid file format: gd =

Import/export through GDAL

HLS.S30.T10TEK.2020273T190109.v1.5.B8A.tif: x_min: 499980 x_max:

609780 x_inc: 30 name: x n_columns: 3660

HLS.S30.T10TEK.2020273T190109.v1.5.B8A.tif: y_min: 4390200 y_max:

4500000 y_inc: 30 name: y n_rows: 3660

HLS.S30.T10TEK.2020273T190109.v1.5.B8A.tif: v_min: -0.0149 v_max:

0.8833 name: z

HLS.S30.T10TEK.2020273T190109.v1.5.B8A.tif: scale_factor: 0.0001

add_offset: 0 packed z-range: [-149,8833]

+proj=utm +zone=10 +ellps=WGS84 +units=m +no_defs

Joaquim

_______________________________________________

gdal-dev mailing list

gdal-dev at lists.osgeo.org<mailto:gdal-dev at lists.osgeo.org>

https://lists.osgeo.org/mailman/listinfo/gdal-dev

_______________________________________________

gdal-dev mailing list

gdal-dev at lists.osgeo.org<mailto:gdal-dev at lists.osgeo.org>

https://lists.osgeo.org/mailman/listinfo/gdal-dev

--

http://www.spatialys.com

My software is free, but my time generally not.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20211005/ef413a53/attachment-0001.html>