<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:"Segoe UI Emoji";
panose-1:2 11 5 2 4 2 4 2 2 3;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
p.xmsonormal, li.xmsonormal, div.xmsonormal
{mso-style-name:x_msonormal;
margin:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.Shkpostityyli21
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:70.85pt 2.0cm 70.85pt 2.0cm;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="FI" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">Hi,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">Mapserver is not clever at all when it finds many overlapping images. It renders them all one by one. If the list of images is sorted correctly then the end result is what was desired,
but with all too much work. See <a href="https://github.com/MapServer/MapServer/issues/5899">
https://github.com/MapServer/MapServer/issues/5899</a>.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">We use two tricks:<br>
1) We have computed the time range of validity for each image and we save the start data and end date into two attributes in the tileindex table in a database. Start date is the acquisition date of the image, and end date is the date when the last pixel of
the image gets covered by some newer image. If the requested time is not within the range of validity then the image is skipped. Skipping is based on the SQL query that is used as DATA for the tileindex layer.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">2) Because time is not supported as a dimension we have written a small utility that is manipulating the incoming http GetCoverage requests before they are sent to Mapserver. The utility
converts the time subset, for example SUBSET=time("2012-12-31T02:00:00Z") into something like "&start_date<2012-12-31T02:00:00Z&end_date>2012-12-31T02:00:00Z. The start_date and end_date variables are defined and validated in the mapfile.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US">As a result Mapserver usually does not get overlapping images from the tileindex so it does not need to spend time for multiple rendering either. We have typically about 10 aerial photos
from different times at each place and without the above mentioned tweaks the service was all too slow.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">-Jukka Rahkonen-<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b>Lähettäjä:</b> MapServer-users <mapserver-users-bounces@lists.osgeo.org>
<b>Puolesta </b>Marcin Niemyjski via MapServer-users<br>
<b>Lähetetty:</b> torstai 27. heinäkuuta 2023 12.26<br>
<b>Vastaanottaja:</b> MapServer Users <mapserver-users@lists.osgeo.org><br>
<b>Aihe:</b> Re: [MapServer-users] Speed of WMS vs Speed of WCS; How to speed up WCS service?<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black">Hello Jukka,<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black"><o:p> </o:p></span></p>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black">Yup I guess that's the case, I assumed that WCS has the same capabilities as WMS when it comes to handling time dimension and it would use the default time from mapfile while requesting the data.<br>
This is my mapserv log file from one getCoverage request (very small, that's why It took only 30 sec to process it). It would be great if you check it
</span><span style="font-size:12.0pt;font-family:"Segoe UI Emoji",sans-serif;color:black">🙂</span><span style="font-size:12.0pt;color:black">:<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black"><a href="https://s3.waw3-1.cloudferro.com/swift/v1/Julien/log2.txt">https://s3.waw3-1.cloudferro.com/swift/v1/Julien/log2.txt</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black">as you can see mapserver lists all of tile item objects (or at least I guess it does haha) which intersect with requested BBOX. So, this is the problem, that's why it takes so long to generate
file. I've got to pass somehow time parameter to request, or at least to query(?) to reduce number of returning shapes.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black">You mentioned your own supporting time WCS service, is it open source? If not, can you give me advice on how to start writing my own? Just some basics ^^<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black">Thanks for your complex answer it helped a lot,
<o:p></o:p></span></p>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black">Marcin<o:p></o:p></span></p>
</div>
<p class="MsoNormal"><span style="font-size:12.0pt;color:black"><o:p> </o:p></span></p>
</div>
<div id="Signature">
<div>
<table class="MsoNormalTable" border="0" cellspacing="3" cellpadding="0" width="1440" style="width:1080.0pt">
<tbody>
<tr style="height:33.75pt">
<td width="1440" style="width:1080.0pt;padding:.75pt .75pt .75pt .75pt;height:33.75pt">
</td>
</tr>
<tr style="height:18.4pt">
<td width="1440" style="width:1080.0pt;padding:.75pt .75pt .75pt .75pt;height:18.4pt;box-sizing: border-box">
</td>
</tr>
<tr style="height:18.4pt">
<td width="1440" style="width:1080.0pt;padding:.75pt .75pt .75pt .75pt;height:18.4pt;box-sizing: border-box">
</td>
</tr>
<tr style="height:18.4pt">
<td width="1440" style="width:1080.0pt;padding:.75pt .75pt .75pt .75pt;height:18.4pt;box-sizing: border-box">
</td>
</tr>
</tbody>
</table>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
</div>
<div class="MsoNormal" align="center" style="text-align:center">
<hr size="2" width="98%" align="center">
</div>
<div id="divRplyFwdMsg">
<p class="MsoNormal"><b><span style="color:black">From:</span></b><span style="color:black"> Rahkonen Jukka <<a href="mailto:jukka.rahkonen@maanmittauslaitos.fi">jukka.rahkonen@maanmittauslaitos.fi</a>><br>
<b>Sent:</b> Thursday, July 27, 2023 12:05 AM<br>
<b>To:</b> Marcin Niemyjski <<a href="mailto:mniemyjski@cloudferro.com">mniemyjski@cloudferro.com</a>>; MapServer Users <<a href="mailto:mapserver-users@lists.osgeo.org">mapserver-users@lists.osgeo.org</a>><br>
<b>Subject:</b> Re: [MapServer-users] Speed of WMS vs Speed of WCS; How to speed up WCS service?</span>
<o:p></o:p></p>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="xmsonormal">Hi,<o:p></o:p></p>
<p class="xmsonormal"> <o:p></o:p></p>
<p class="xmsonormal"><span lang="EN-US">Does the tileindex query ” DATA 'geometry from (select * from plua order by "maxCC" desc) “ return several images from the requested BBOX?</span><o:p></o:p></p>
<p class="xmsonormal"><span lang="EN-US"> </span><o:p></o:p></p>
<p class="xmsonormal"><span lang="EN-US">I know that MapServer WCS can be fast and we have also a demo at
<a href="https://beta-karttakuva.maanmittauslaitos.fi/demo/ortokuvat_ja_korkeusmallit/#15.05/67.01553/27.16873">
https://beta-karttakuva.maanmittauslaitos.fi/demo/ortokuvat_ja_korkeusmallit/#15.05/67.01553/27.16873</a></span><o:p></o:p></p>
<p class="xmsonormal"><span lang="EN-US">but unfortunately the user interface is only in Finnish. The source images (maybe 10 TB or so) are GeoTIFFs and they are not in the cloud. The time for downloading a small GetCoverage response as GeoTIFF seems to take
around 400 milliseconds. We had to write a small service that makes our WCS service to support time as a dimension because MapServer does not support it with WCS
<a href="https://www.mail-archive.com/mapserver-users@lists.osgeo.org/msg26265.html">
https://www.mail-archive.com/mapserver-users@lists.osgeo.org/msg26265.html</a>.</span><o:p></o:p></p>
<p class="xmsonormal"><span lang="EN-US"> </span><o:p></o:p></p>
<p class="xmsonormal"><span lang="EN-US">-Jukka Rahkonen-</span><o:p></o:p></p>
<p class="xmsonormal"><span lang="EN-US"> </span><o:p></o:p></p>
<p class="xmsonormal"><span lang="EN-US"> </span><o:p></o:p></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="xmsonormal"><b>Lähettäjä:</b> MapServer-users <<a href="mailto:mapserver-users-bounces@lists.osgeo.org">mapserver-users-bounces@lists.osgeo.org</a>>
<b>Puolesta </b>Marcin Niemyjski via MapServer-users<br>
<b>Lähetetty:</b> keskiviikko 26. heinäkuuta 2023 15.06<br>
<b>Vastaanottaja:</b> MapServer Users <<a href="mailto:mapserver-users@lists.osgeo.org">mapserver-users@lists.osgeo.org</a>><br>
<b>Aihe:</b> Re: [MapServer-users] Speed of WMS vs Speed of WCS; How to speed up WCS service?<o:p></o:p></p>
</div>
</div>
<p class="xmsonormal"> <o:p></o:p></p>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black">Hi Seth,</span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black"> </span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black">as you suggested, I've added the Extent parameter to all of my layers, and it resulted in decreasing time to 5m33.534s. But still, it's quite a lot of time for generating 1.5 MiB file.</span><o:p></o:p></p>
</div>
<div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black"> </span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black">Any other suggestions?<br>
Maybe there is something wrong with projection so beyond just serving data mapserver have to reproject it multiple times? (more than one time by using tilesrs option)?</span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black"> </span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black">Thanks,</span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black">Marcin</span><o:p></o:p></p>
</div>
<div id="x_Signature">
<div>
<table class="MsoNormalTable" border="0" cellspacing="3" cellpadding="0" width="95%" style="width:95.0%">
<tbody>
<tr style="height:22.5pt">
<td style="padding:.75pt .75pt .75pt .75pt;height:22.5pt"></td>
</tr>
<tr>
<td style="padding:.75pt .75pt .75pt .75pt"></td>
</tr>
<tr>
<td style="padding:.75pt .75pt .75pt .75pt"></td>
</tr>
<tr>
<td style="padding:.75pt .75pt .75pt .75pt"></td>
</tr>
</tbody>
</table>
<p class="xmsonormal"> <o:p></o:p></p>
</div>
</div>
</div>
<div class="MsoNormal" align="center" style="text-align:center">
<hr size="1" width="98%" align="center">
</div>
<div id="x_divRplyFwdMsg">
<p class="xmsonormal"><b><span style="color:black">From:</span></b><span style="color:black"> Seth G <<a href="mailto:sethg@geographika.co.uk">sethg@geographika.co.uk</a>><br>
<b>Sent:</b> Wednesday, July 26, 2023 12:58 PM<br>
<b>To:</b> Marcin Niemyjski <<a href="mailto:mniemyjski@cloudferro.com">mniemyjski@cloudferro.com</a>>; MapServer Users <<a href="mailto:mapserver-users@lists.osgeo.org">mapserver-users@lists.osgeo.org</a>><br>
<b>Subject:</b> Re: [MapServer-users] Speed of WMS vs Speed of WCS; How to speed up WCS service?</span>
<o:p></o:p></p>
<div>
<p class="xmsonormal"> <o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="xmsonormal">Hi Marcin,<o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"> <o:p></o:p></p>
</div>
<div>
<p class="xmsonormal">As a first test I'd recommend adding in EXTENTs to all your layers (and ensuring they are correct):<o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"> <o:p></o:p></p>
</div>
<div>
<p class="xmsonormal">EXTENT 1334552.857598 5342648.587634 4715044.821842 7547620.486088<o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"> <o:p></o:p></p>
</div>
<div>
<p class="xmsonormal">If these are missing MapServer will often try and calculate these from the datasource automatically. Depending on the request type this calculation could be made multiple times.<o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"> <o:p></o:p></p>
</div>
<div>
<p class="xmsonormal">Seth<o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"> <o:p></o:p></p>
</div>
<div id="x_x_sig62266145">
<div>
<p class="xmsonormal">--<o:p></o:p></p>
</div>
<div>
<p class="xmsonormal">web:<a href="https://geographika.net/">https://geographika.net</a> & <a href="https://mapserverstudio.net/">https://mapserverstudio.net</a><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal">twitter: @geographika<o:p></o:p></p>
</div>
</div>
<div>
<p class="xmsonormal"> <o:p></o:p></p>
</div>
<div>
<p class="xmsonormal">On Wed, Jul 26, 2023, at 11:53 AM, Marcin Niemyjski via MapServer-users wrote:<o:p></o:p></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt" id="x_x_qt">
<div>
<p class="xmsonormal">Hello,<o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"> <o:p></o:p></p>
</div>
<div>
<p class="xmsonormal">I have a problem, Namely, I have created a WCS and here is its mapfile:<o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><a href="https://pastecode.io/s/6v3qfssb">https://pastecode.io/s/6v3qfssb</a><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"> <o:p></o:p></p>
</div>
<div>
<p class="xmsonormal">For a better understanding, here is a view of the tileindex through which the data is served:<o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><a href="https://pasteboard.co/ydgvDiqxz4lf.png">https://pasteboard.co/ydgvDiqxz4lf.png</a><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"> <o:p></o:p></p>
</div>
<div>
<p class="xmsonormal">I am connecting to files located in a private bucket, so I am using vsis3.<o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"> <o:p></o:p></p>
</div>
<div>
<p class="xmsonormal">The WMS loads quite fast, but the issue arises with the time it takes to generate a requested WCS tile:<o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"> <a href="https://pasteboard.co/8G8uF41jqgHc.png">https://pasteboard.co/8G8uF41jqgHc.png</a><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"> <o:p></o:p></p>
</div>
<div>
<p class="xmsonormal">It takes almost 5 minutes to generate the tile, and the received data is 1.5 MiB (which is it's actual size).<o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"> <o:p></o:p></p>
</div>
<div>
<p class="xmsonormal">Additionally, I am sending you URLs to sample data:<o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><a href="https://s3.waw3-1.cloudferro.com/swift/v1/Julien/Data/T34UFD_20230708T093549_TCI_60m.jp2">https://s3.waw3-1.cloudferro.com/swift/v1/Julien/Data/T34UFD_20230708T093549_TCI_60m.jp2</a><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><a href="https://s3.waw3-1.cloudferro.com/swift/v1/Julien/Data/T37TCN_20220605T083559_TCI_60m.jp2">https://s3.waw3-1.cloudferro.com/swift/v1/Julien/Data/T37TCN_20220605T083559_TCI_60m.jp2</a><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"> <o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black">The case is - how to speed it up? Am I making any goofy mistakes?</span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal">I would be grateful for any recommendations or if you notice any errors, I might be making :)<o:p></o:p></p>
</div>
<div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black"> </span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black">Best,</span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-size:12.0pt;color:black">Marcin</span><o:p></o:p></p>
</div>
</div>
<div>
<p class="xmsonormal">_______________________________________________<o:p></o:p></p>
</div>
<div>
<p class="xmsonormal">MapServer-users mailing list<o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><a href="mailto:MapServer-users@lists.osgeo.org">MapServer-users@lists.osgeo.org</a><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><a href="https://lists.osgeo.org/mailman/listinfo/mapserver-users">https://lists.osgeo.org/mailman/listinfo/mapserver-users</a><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"> <o:p></o:p></p>
</div>
</blockquote>
<div>
<p class="xmsonormal"> <o:p></o:p></p>
</div>
</div>
</div>
</div>
</div>
</body>
</html>