<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
span.Shkpostityyli17
{mso-style-type:personal;
font-family:Consolas;
color:windowtext;
font-weight:normal;
font-style:normal;
text-decoration:none none;}
span.Shkpostityyli18
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="FI" link="#0563C1" vlink="#954F72">
<div class="WordSection1">
<p class="MsoNormal"><span lang="EN-US" style="color:#1F497D">Hi,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="color:#1F497D">There seems to be 37800 aerial photos in my tileindex, 10000 x 10000 or 12000 x 12000 pixels each, and system works like a charm. I wouldn’t call 69.2 GB as a massive dataset.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="color:#1F497D">First thing to do is to stop using ASCII GRID which must be one of the most unsuitable formats for a web map service. Server must always open the whole image into memory because format does not support
direct access to region of interest. Your mode=map requests puts Mapserver to read all your 466 ASCII GRID files into memory.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="color:#1F497D">Convert your DSM files into tiled GeoTIFFs, create overviews, and set a reasonable maxscaledenom for the layer so that no more than 40 or so files are opened from the tileindex layer. Make a layer
group and use some other layer at small scales. All this is FAQ and you should find lot of reading from the internet. The “Geoserver on steroids” presentations are good start. The challenge is the same for all server brands.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="color:#1F497D">-Jukka Rahkonen-<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="color:#1F497D"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><span style="mso-fareast-language:FI">Wright, Simon M wrote:<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:10.0pt;font-family:Consolas">Dear All<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:10.0pt;font-family:Consolas"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:10.0pt;font-family:Consolas">I’m trying to use the MapServer Tile Indexes to create a map service, and ultimately a WMS, of some DSM data but am getting an Internal Server Error.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:10.0pt;font-family:Consolas"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:10.0pt;font-family:Consolas">The DSM data are organised as 466 5 × 5 km rasters with a resolution of 1 × 1 m with a total size of 69.2 GB. The data format is Esri ASCII GRID.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:10.0pt;font-family:Consolas"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:10.0pt;font-family:Consolas">To test the use of Tile Indexes in MapServer I used the gdaltindex to create an index shapefile for just twenty of the 5 × 5 km rasters and created a basic map file:<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:10.0pt;font-family:Consolas"><o:p> </o:p></span></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-GB" style="font-size:10.0pt;font-family:Consolas">MAP<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-GB" style="font-size:10.0pt;font-family:Consolas"> IMAGETYPE PNG<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-GB" style="font-size:10.0pt;font-family:Consolas"> EXTENT 240000 100000 250000 150000<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-GB" style="font-size:10.0pt;font-family:Consolas"> SIZE 400 2000<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-GB" style="font-size:10.0pt;font-family:Consolas"> PROJECTION<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-GB" style="font-size:10.0pt;font-family:Consolas"> "init=epsg:27700"<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-GB" style="font-size:10.0pt;font-family:Consolas"> END<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-GB" style="font-size:10.0pt;font-family:Consolas"> LAYER<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-GB" style="font-size:10.0pt;font-family:Consolas"> NAME "tiled-rasters"<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-GB" style="font-size:10.0pt;font-family:Consolas"> STATUS ON<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-GB" style="font-size:10.0pt;font-family:Consolas"> TILEINDEX "/datastore/b81071f2-85b3-4e31-8506-cabe899f989a/b81071f2-85b3-4e31-8506-cabe899f989a_copy/index_file"<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-GB" style="font-size:10.0pt;font-family:Consolas"> TILEITEM "location"<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-GB" style="font-size:10.0pt;font-family:Consolas"> TYPE RASTER<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-GB" style="font-size:10.0pt;font-family:Consolas"> PROJECTION<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-GB" style="font-size:10.0pt;font-family:Consolas"> "init=epsg:27700"<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-GB" style="font-size:10.0pt;font-family:Consolas"> END<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-GB" style="font-size:10.0pt;font-family:Consolas"> END<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:36.0pt"><span lang="EN-GB" style="font-size:10.0pt;font-family:Consolas">END<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:10.0pt;font-family:Consolas"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:10.0pt;font-family:Consolas"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:10.0pt;font-family:Consolas">When I request an image of the map with the URL
<a href="http://localhost:8080/cgi-bin/mapserv?map=/vagrant/maps/tiled-rasters.map&layer=tiled-rasters&mode=map">
http://localhost:8080/cgi-bin/mapserv?map=/vagrant/maps/tiled-rasters.map&layer=tiled-rasters&mode=map</a> I get an “Internal Server Error”.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:10.0pt;font-family:Consolas"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:10.0pt;font-family:Consolas">Therefore I created a second index shapefile using the gdaltindex tool that only contained one of the 5 × 5 km rasters and updated the map file to point to the second index
shapefile. When I request an image of the map I can see an image of the DSM data as expected.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:10.0pt;font-family:Consolas"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:10.0pt;font-family:Consolas">Therefore, am I asking too much of the tile indexing approach in MapServer to work against a massive dataset – 69.2 GB in 466 raster datasets – or have MapServer users been
able to get tile indexing too work in MapServer against large data volumes and can provide some guidance.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:10.0pt;font-family:Consolas"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:10.0pt;font-family:Consolas">Many thanks in advance of any help that MapServer users can offer.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:10.0pt;font-family:Consolas"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:10.0pt;font-family:Consolas"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:10.0pt;font-family:Consolas"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:10.0pt;font-family:Consolas">Best wishes, Simon.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:10.0pt;font-family:Consolas"><o:p> </o:p></span></p>
<div class="MsoNormal" align="center" style="text-align:center"><span lang="EN-GB" style="font-size:12.0pt;font-family:"Times New Roman",serif;mso-fareast-language:FI">
<hr size="2" width="100%" align="center">
</span></div>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:10.0pt;font-family:"Times New Roman",serif;mso-fareast-language:FI">This message (and any attachments) is for the recipient only. NERC is subject to the Freedom of Information Act 2000 and the contents
of this email and any reply you make may be disclosed by NERC unless it is exempt from release under the Act. Any material supplied to NERC may be stored in an electronic records management system</span><span lang="EN-GB" style="font-size:12.0pt;font-family:"Times New Roman",serif;mso-fareast-language:FI">.
<o:p></o:p></span></p>
<div class="MsoNormal" align="center" style="text-align:center"><span lang="EN-GB" style="font-size:12.0pt;font-family:"Times New Roman",serif;mso-fareast-language:FI">
<hr size="2" width="100%" align="center">
</span></div>
</div>
</body>
</html>