[gdal-dev] GDAL WMTS get tiles no documentation

Fri May 31 11:40:54 PDT 2024

Thank you again.
Now I know which direction should I go.This is my goal to have a control
over this process using the low-level RasterIO.
I browse GDAL github repository a lot and it's very helpful, especially the
tests.
This time I was confused with WMTS and I needed a support at the begining.

Have a great weekend!
Michal

pt., 31 maj 2024 o 16:14 Rahkonen Jukka <jukka.rahkonen at maanmittauslaitos.fi>
napisał(a):

> Hi,
>
>
>
> You do not need to use GDALTranslate even if may be convenient. In the
> RasterIO() function GDALRasterBand C++ API — GDAL documentation
> <https://gdal.org/api/gdalrasterband_cpp.html> the region is selected by
> using XOff, YOff, XSize and YSize which map directly with -srcwin. But I
> was thinking that very soon you’ll ask about how to convert a georeferenced
> coordinates into pixel space, and gdal_translate supports that. So maybe
> you want to read the source code
> https://github.com/OSGeo/gdal/blob/master/apps/gdal_translate_lib.cpp.
> And meanwhile you can read and convert data from WMTS with gdal_translate,
> enjoy in seeing that it works, and be confident that you will be able to do
> the same and more with your own code. If programmers think that way, I do
> not know.
>
>
>
> -Jukka-
>
>
>
> *Lähettäjä:* Michał Kowalczuk <michkowalczuk at gmail.com>
> *Lähetetty:* perjantai 31. toukokuuta 2024 16.32
> *Vastaanottaja:* Rahkonen Jukka <jukka.rahkonen at maanmittauslaitos.fi>
> *Kopio:* Even Rouault <even.rouault at spatialys.com>;
> gdal-dev at lists.osgeo.org
> *Aihe:* Re: [gdal-dev] GDAL WMTS get tiles no documentation
>
>
>
> Thank you Rukka,
>
> Since *-srcwin* and *-projwin* are options only for *gdal_translate*
> utility, your suggestion leads me to use *GDALTranslate()* function with
> these parameters.
>
>
>
> This means that I cannot use directly created WMTS dataset to retrieve
> data (using GDALDatasetRasterIO), but create a new (in-memory or
> file-based) for each map extent change.
>
>
>
> And this is the standard way of doing it?
>
> Do I understand it correctly?
>
>
>
> Michal
>
>
>
> pt., 31 maj 2024 o 15:13 Rahkonen Jukka <
> jukka.rahkonen at maanmittauslaitos.fi> napisał(a):
>
> Hi,
>
>
>
> Before starting to program you can familiar yourself with the topic by
> using gdal_translate gdal_translate — GDAL documentation
> <https://gdal.org/programs/gdal_translate.html>.
>
> Play with the -srcwin and -projwin / -projwin_srs options and gather some
> trust that is is possible to select and download any part of the large WMTS
> layer. Add “-- debug on” and you will see that GDAL downloads tiles
> separately just like QGIS. Actually, with WMTS it is the only way to
> download data. Maybe you have read the standard already
> https://portal.ogc.org/files/?artifact_id=35326. It is not necessary
> because GDAL knows what to do, but a review every now and then does not
> harm. Especially the TileMatrix and TileMatrixSet things are not
> self-evident, but they provide the answer to the question about how a WMTS
> client knows which row, col, and zoom to download.
>
>
>
> Do not touch the DataWindow. It defines the extent of the whole WMTS layer
> and it is used for example if gdal_translate is run without spatial
> selection (-srcwin or -projwin). Touch in your future code things which are
> equivalent to -srcwin or -projwin of gdal_translate.
>
>
>
> -Jukka Rahkonen-
>
>
>
> *Lähettäjä:* Michał Kowalczuk <michkowalczuk at gmail.com>
> *Lähetetty:* perjantai 31. toukokuuta 2024 15.45
> *Vastaanottaja:* Even Rouault <even.rouault at spatialys.com>
> *Kopio:* Rahkonen Jukka <jukka.rahkonen at maanmittauslaitos.fi>;
> gdal-dev at lists.osgeo.org
> *Aihe:* Re: [gdal-dev] GDAL WMTS get tiles no documentation
>
>
>
> Hi, thank you for your replies!
> As I'm still a newbie especially with WMTS. I need to take a step back to
> the basics. Let's forget about multithreading for now.
>
>    - I'm in the stage that I can open a dataset using WMTS service path
>    from data provider
>    - I can list subdatasets and open the selected layer as a new dataset
>    using path from *SUBDATASET_id_NAME*.
>    - This dataset reports large height and width values, which is obvious.
>    - The first question is how to read any fragment from this dataset
>    using *GDALDatasetRasterIO*.
>    Is there a similar way to *SpatialFIlter *from vector layers?
>    I see in the docs there is a *<DataWindow>* tag for a local service
>    descriptions which defines extents of the data. Does it mean that for each
>    map extent change I need to create a new local XML file and open a new
>    dataset to read demanded bbox?
>    Is this the only way for doing it?
>    - From other site I noticed that QGIS downloads tiles separately? But
>    how it knows which row/col and zoom should be used in GET request? Does
>    it use GDAL for WMTS reading at all?
>
> I will be grateful for any tip and trucks for a newbie ;-)
>
>
>
> Regards,
>
> Michal
>
>
>
> śr., 29 maj 2024 o 12:41 Even Rouault <even.rouault at spatialys.com>
> napisał(a):
>
> Hi,
>
> The WMS cache (which is used underneath by the WMTS driver) writes each
> tiles in a separate file. So if you use multi-threaded, as long as you read
> in parallel different GDALDataset* object on the WMTS filename, and read
> disjoint areas, that should be fine. There might be a slight risk of things
> going bad if you happened to read the same tile from different GDALDataset*
> object at the same time, as it could potentially read a partially written
> cached tile, or concurrently write the tile in the cache and corrupt it.
> The logic is in frmts/wms/gdalwmscache.cpp
>
> Even
>
> Le 29/05/2024 à 10:14, Michał Kowalczuk via gdal-dev a écrit :
>
> Because this is not clear in documentation, especially in WMTS driver
> docs, my question is strictly about this issue to programmers who have
> already explored this topic.
>
>
>
> Regards
>
> Michal
>
>
>
> W dniu śr., 29.05.2024 o 10:04 Rahkonen Jukka <
> jukka.rahkonen at maanmittauslaitos.fi> napisał(a):
>
> Hi,
>
>
>
> No, I am not sure because I do not program myself. Reading this document
> https://gdal.org/user/multithreading.html makes me feel that it is
> possible to do some things in parallel, but the programmer must know how to
> do it right.
>
>
>
> -Jukka Rahkonen-
>
>
>
> *Lähettäjä:* Javier Jimenez Shaw <j1 at jimenezshaw.com>
> *Lähetetty:* keskiviikko 29. toukokuuta 2024 10.41
> *Vastaanottaja:* Rahkonen Jukka <jukka.rahkonen at maanmittauslaitos.fi>
> *Kopio:* Michał Kowalczuk <michkowalczuk at gmail.com>;
> gdal-dev at lists.osgeo.org
> *Aihe:* Re: [gdal-dev] GDAL WMTS get tiles no documentation
>
>
>
>
>
>
>
> On Wed, 29 May 2024 at 08:59, Rahkonen Jukka via gdal-dev <
> gdal-dev at lists.osgeo.org> wrote:
>
> Hi,
>
>
>
> When you have a RasterBand  from the WMTS data source, it is abstracted
> and you can read the raster data just like from any other data source and
> raster band
> https://gdal.org/tutorials/raster_api_tut.html#reading-raster-data “There
> are a few ways to read raster data, but the most common is via the
> GDALRasterBand::RasterIO() method. This method will automatically take care
> of data type conversion, up/down sampling and windowing.” GDAL knows which
> tiles to read.
>
>
>
> I do not know if the WMTS driver can do parallel tile downloads. If not,
> it is possible to run many RasterIO() at the same time, each reading data
> from a different window like in this rasterio document
> https://rasterio.readthedocs.io/en/latest/topics/concurrency.html.
>
>
>
> Jukka, are you sure you can run several RasterIO in parallel (on the same
> dataset)? in GDAL GeoTIFF you cannot: the cache may be corrupted. In that
> case I open several datasets over the same file. I do not know about WMTS.
>
>
>
>
>
> -Jukka Rahkonen-
>
>
>
> *Lähettäjä:* gdal-dev <gdal-dev-bounces at lists.osgeo.org> *Puolesta *Michal
> Kowalczuk via gdal-dev
> *Lähetetty:* keskiviikko 29. toukokuuta 2024 9.08
> *Vastaanottaja:* gdal-dev at lists.osgeo.org
> *Aihe:* [gdal-dev] GDAL WMTS get tiles no documentation
>
>
>
> Hi GDAL fellows
>
> This is my first post on this mailing list, so I'm asking for
> understanding.
>
>
>
> As all we know the purpose of using WMTS over WMS, I'd like to implement
> parallel downloading tiles from service using C API.
>
> In my opinion GDAL documentation (
> https://gdal.org/drivers/raster/wmts.html) says nothing on this topic.
>
> I can get capabilities from WMTS, I can open the selected subdataset but
> how to get tiles for given extent? I could not find any information how to
> do it, even in the GDAL tests on github.
>
>
>
> I am kindly asking for tips. It also can be in python. How using pure GDAL
> API fetch tiles to dynamically complete the displayed map. This is my goal.
>
>
>
> Thank you!
>
> Michal
>
> _______________________________________________
> gdal-dev mailing list
> gdal-dev at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/gdal-dev
>
>
>
> _______________________________________________
>
> gdal-dev mailing list
>
> gdal-dev at lists.osgeo.org
>
> https://lists.osgeo.org/mailman/listinfo/gdal-dev
>
> --
>
> http://www.spatialys.com
>
> My software is free, but my time generally not.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20240531/29b8976e/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 25778 bytes
Desc: not available
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20240531/29b8976e/attachment-0001.png>