[gdal-dev] GDAL WMTS get tiles no documentation

Fri May 31 06:57:41 PDT 2024

As you proposed I tested *gdal_translate *utility but for the following
sample command I get an empty image (tested with jpg and png)

gdal_translate "WMTS:
http://maps.wien.gv.at/wmts/1.0.0/WMTSCapabilities.xml,layer=fmzk3dviewer"
-srcwin 1820157 6144777 512 512 tile.jpg

[image: image.png]

pt., 31 maj 2024 o 15:31 Michał Kowalczuk <michkowalczuk at gmail.com>
napisał(a):

> Thank you Rukka,
> Since *-srcwin* and *-projwin* are options only for *gdal_translate*
> utility, your suggestion leads me to use *GDALTranslate()* function with
> these parameters.
>
> This means that I cannot use directly created WMTS dataset to retrieve
> data (using GDALDatasetRasterIO), but create a new (in-memory or
> file-based) for each map extent change.
>
> And this is the standard way of doing it?
> Do I understand it correctly?
>
> Michal
>
> pt., 31 maj 2024 o 15:13 Rahkonen Jukka <
> jukka.rahkonen at maanmittauslaitos.fi> napisał(a):
>
>> Hi,
>>
>>
>>
>> Before starting to program you can familiar yourself with the topic by
>> using gdal_translate gdal_translate — GDAL documentation
>> <https://gdal.org/programs/gdal_translate.html>.
>>
>> Play with the -srcwin and -projwin / -projwin_srs options and gather some
>> trust that is is possible to select and download any part of the large WMTS
>> layer. Add “-- debug on” and you will see that GDAL downloads tiles
>> separately just like QGIS. Actually, with WMTS it is the only way to
>> download data. Maybe you have read the standard already
>> https://portal.ogc.org/files/?artifact_id=35326. It is not necessary
>> because GDAL knows what to do, but a review every now and then does not
>> harm. Especially the TileMatrix and TileMatrixSet things are not
>> self-evident, but they provide the answer to the question about how a WMTS
>> client knows which row, col, and zoom to download.
>>
>>
>>
>> Do not touch the DataWindow. It defines the extent of the whole WMTS
>> layer and it is used for example if gdal_translate is run without spatial
>> selection (-srcwin or -projwin). Touch in your future code things which are
>> equivalent to -srcwin or -projwin of gdal_translate.
>>
>>
>>
>> -Jukka Rahkonen-
>>
>>
>>
>> *Lähettäjä:* Michał Kowalczuk <michkowalczuk at gmail.com>
>> *Lähetetty:* perjantai 31. toukokuuta 2024 15.45
>> *Vastaanottaja:* Even Rouault <even.rouault at spatialys.com>
>> *Kopio:* Rahkonen Jukka <jukka.rahkonen at maanmittauslaitos.fi>;
>> gdal-dev at lists.osgeo.org
>> *Aihe:* Re: [gdal-dev] GDAL WMTS get tiles no documentation
>>
>>
>>
>> Hi, thank you for your replies!
>> As I'm still a newbie especially with WMTS. I need to take a step back to
>> the basics. Let's forget about multithreading for now.
>>
>>    - I'm in the stage that I can open a dataset using WMTS service path
>>    from data provider
>>    - I can list subdatasets and open the selected layer as a new dataset
>>    using path from *SUBDATASET_id_NAME*.
>>    - This dataset reports large height and width values, which is
>>    obvious.
>>    - The first question is how to read any fragment from this dataset
>>    using *GDALDatasetRasterIO*.
>>    Is there a similar way to *SpatialFIlter *from vector layers?
>>    I see in the docs there is a *<DataWindow>* tag for a local service
>>    descriptions which defines extents of the data. Does it mean that for each
>>    map extent change I need to create a new local XML file and open a new
>>    dataset to read demanded bbox?
>>    Is this the only way for doing it?
>>    - From other site I noticed that QGIS downloads tiles separately? But
>>    how it knows which row/col and zoom should be used in GET request? Does
>>    it use GDAL for WMTS reading at all?
>>
>> I will be grateful for any tip and trucks for a newbie ;-)
>>
>>
>>
>> Regards,
>>
>> Michal
>>
>>
>>
>> śr., 29 maj 2024 o 12:41 Even Rouault <even.rouault at spatialys.com>
>> napisał(a):
>>
>> Hi,
>>
>> The WMS cache (which is used underneath by the WMTS driver) writes each
>> tiles in a separate file. So if you use multi-threaded, as long as you read
>> in parallel different GDALDataset* object on the WMTS filename, and read
>> disjoint areas, that should be fine. There might be a slight risk of things
>> going bad if you happened to read the same tile from different GDALDataset*
>> object at the same time, as it could potentially read a partially written
>> cached tile, or concurrently write the tile in the cache and corrupt it.
>> The logic is in frmts/wms/gdalwmscache.cpp
>>
>> Even
>>
>> Le 29/05/2024 à 10:14, Michał Kowalczuk via gdal-dev a écrit :
>>
>> Because this is not clear in documentation, especially in WMTS driver
>> docs, my question is strictly about this issue to programmers who have
>> already explored this topic.
>>
>>
>>
>> Regards
>>
>> Michal
>>
>>
>>
>> W dniu śr., 29.05.2024 o 10:04 Rahkonen Jukka <
>> jukka.rahkonen at maanmittauslaitos.fi> napisał(a):
>>
>> Hi,
>>
>>
>>
>> No, I am not sure because I do not program myself. Reading this document
>> https://gdal.org/user/multithreading.html makes me feel that it is
>> possible to do some things in parallel, but the programmer must know how to
>> do it right.
>>
>>
>>
>> -Jukka Rahkonen-
>>
>>
>>
>> *Lähettäjä:* Javier Jimenez Shaw <j1 at jimenezshaw.com>
>> *Lähetetty:* keskiviikko 29. toukokuuta 2024 10.41
>> *Vastaanottaja:* Rahkonen Jukka <jukka.rahkonen at maanmittauslaitos.fi>
>> *Kopio:* Michał Kowalczuk <michkowalczuk at gmail.com>;
>> gdal-dev at lists.osgeo.org
>> *Aihe:* Re: [gdal-dev] GDAL WMTS get tiles no documentation
>>
>>
>>
>>
>>
>>
>>
>> On Wed, 29 May 2024 at 08:59, Rahkonen Jukka via gdal-dev <
>> gdal-dev at lists.osgeo.org> wrote:
>>
>> Hi,
>>
>>
>>
>> When you have a RasterBand  from the WMTS data source, it is abstracted
>> and you can read the raster data just like from any other data source and
>> raster band
>> https://gdal.org/tutorials/raster_api_tut.html#reading-raster-data
>> “There are a few ways to read raster data, but the most common is via the
>> GDALRasterBand::RasterIO() method. This method will automatically take care
>> of data type conversion, up/down sampling and windowing.” GDAL knows which
>> tiles to read.
>>
>>
>>
>> I do not know if the WMTS driver can do parallel tile downloads. If not,
>> it is possible to run many RasterIO() at the same time, each reading data
>> from a different window like in this rasterio document
>> https://rasterio.readthedocs.io/en/latest/topics/concurrency.html.
>>
>>
>>
>> Jukka, are you sure you can run several RasterIO in parallel (on the same
>> dataset)? in GDAL GeoTIFF you cannot: the cache may be corrupted. In that
>> case I open several datasets over the same file. I do not know about WMTS.
>>
>>
>>
>>
>>
>> -Jukka Rahkonen-
>>
>>
>>
>> *Lähettäjä:* gdal-dev <gdal-dev-bounces at lists.osgeo.org> *Puolesta *Michal
>> Kowalczuk via gdal-dev
>> *Lähetetty:* keskiviikko 29. toukokuuta 2024 9.08
>> *Vastaanottaja:* gdal-dev at lists.osgeo.org
>> *Aihe:* [gdal-dev] GDAL WMTS get tiles no documentation
>>
>>
>>
>> Hi GDAL fellows
>>
>> This is my first post on this mailing list, so I'm asking for
>> understanding.
>>
>>
>>
>> As all we know the purpose of using WMTS over WMS, I'd like to implement
>> parallel downloading tiles from service using C API.
>>
>> In my opinion GDAL documentation (
>> https://gdal.org/drivers/raster/wmts.html) says nothing on this topic.
>>
>> I can get capabilities from WMTS, I can open the selected subdataset but
>> how to get tiles for given extent? I could not find any information how to
>> do it, even in the GDAL tests on github.
>>
>>
>>
>> I am kindly asking for tips. It also can be in python. How using pure
>> GDAL API fetch tiles to dynamically complete the displayed map. This is my
>> goal.
>>
>>
>>
>> Thank you!
>>
>> Michal
>>
>> _______________________________________________
>> gdal-dev mailing list
>> gdal-dev at lists.osgeo.org
>> https://lists.osgeo.org/mailman/listinfo/gdal-dev
>>
>>
>>
>> _______________________________________________
>>
>> gdal-dev mailing list
>>
>> gdal-dev at lists.osgeo.org
>>
>> https://lists.osgeo.org/mailman/listinfo/gdal-dev
>>
>> --
>>
>> http://www.spatialys.com
>>
>> My software is free, but my time generally not.
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20240531/b8fd5722/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 25778 bytes
Desc: not available
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20240531/b8fd5722/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 28257 bytes
Desc: not available
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20240531/b8fd5722/attachment-0003.png>