<br><br><div class="gmail_quote">On Fri, Aug 13, 2010 at 1:59 PM, Ivan Lucena <span dir="ltr"><<a href="mailto:ivan.lucena@pmldnet.com">ivan.lucena@pmldnet.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
Sebastian,<br>
<br>
You are using a very small block size, maybe because you are also using ArcGIS products and they only support Oracle GeoRaster with 256x256 blocking. Is that right?<br></blockquote><div><br>nop... I guess that 256x256 if the default gdal_translate block size ?<br>
</div><blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
<br>
Because of that, to access your GeoRaster object, the GDAL driver is requesting a very large array of BLOB (1000000 approximately). That can make your sever such-up memory from the machine where it is running and it is probably using virtual memory (disk) what is really slow. </blockquote>
<div><br>do you mean where MapServer is running ? <br><br>top - 14:41:31 up 7 days, 6:20, 10 users, load average: 2.81, 1.92, 1.20<br>Tasks: 340 total, 2 running, 338 sleeping, 0 stopped, 0 zombie<br>Cpu(s): 11.8%us, 10.4%sy, 7.7%ni, 62.1%id, 0.0%wa, 0.0%hi, 8.0%si, 0.0%st<br>
Mem: 12331240k total, 5992136k used, 6339104k free, 189416k buffers<br>Swap: 9084716k total, 703708k used, 8381008k free, 1481992k cached<br><br> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND<br>
24850 www-data 20 0 533m 330m 18m R 14 2.7 0:00.94 mapserv<br>24852 www-data 20 0 532m 329m 18m S 19 2.7 0:01.22 mapserv<br>24849 www-data 20 0 529m 326m 18m S 16 2.7 0:01.00 mapserv<br>24851 www-data 20 0 517m 314m 18m S 13 2.6 0:00.96 mapserv<br>
24853 www-data 20 0 499m 296m 18m S 8 2.5 0:00.75 mapserv<br>24854 www-data 20 0 470m 267m 18m S 5 2.2 0:00.53 mapserv<br><br> </div><blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
It also uses some memory on the client side just to hold the array too but if you are running everything (Mapserver,Oracle) on the same machine that will adds up.<br></blockquote><div><br>Oracle is running on another Machine... and is very likely that it is the bottle neck... <br>
<br></div><blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
<br>
We are aware of that problem but it would be very helpful if you could file a bug report on GDAL's trac [<a href="http://trac.osgeo.org/gdal/" target="_blank">http://trac.osgeo.org/gdal/</a>] Thanks.<br>
<br></blockquote><div><br>I guess that the problem is that MapServer triys to read the whole map instead of reading only tile ???? is that correct ?<br> </div><blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
Just as curiosity. Did you loaded that image with gdal_translate? How long it took and long does it takes to generate pyramids? Ha, you forgot to say what version of Oracle you are using.<br>
<br></blockquote><div><br>I loaded it with gdal in around 5 minutes... the pyramid (all levels) has been created overnight.... <br> </div><blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
Best regards,<br>
<br>
Ivan<br>
<div class="im"><br>
<br>
<br>
> -------Original Message-------<br>
> From: Sebastian E. Ovide <<a href="mailto:sebastian.ovide@gmail.com">sebastian.ovide@gmail.com</a>><br>
> To: Lucena, Ivan <<a href="mailto:ivan.lucena@pmldnet.com">ivan.lucena@pmldnet.com</a>><br>
> Cc: <a href="mailto:mapserver-users@lists.osgeo.org">mapserver-users@lists.osgeo.org</a><br>
> Subject: Re: [mapserver-users] logging GDAL queries<br>
> Sent: Aug 13 '10 06:44<br>
><br>
> sure...<br>
><br>
><br>
> sebas@SeansPC:~/mapserver/mapserver-5.6.5$ gdalinfo --version<br>
> GDAL 1.7.2, released 2010/04/23<br>
><br>
> sebas@SeansPC:~/mapserver/mapserver-5.6.5$ gdalinfo<br>
</div><div class="im">> georaster:geoserver,geoserver,MFPRODUK_11G,fluvd04q200pj,georaster,id=1<br>
</div><div><div></div><div class="h5">> -mdd oracle<br>
> Driver: GeoRaster/Oracle Spatial GeoRaster<br>
> Files: none associated<br>
> Size is 190325, 271423<br>
> Coordinate System is:<br>
> PROJCS["Popular Visualisation CRS / Mercator (deprecated)",<br>
> GEOGCS["Popular Visualisation CRS",<br>
> DATUM["Popular_Visualisation_Datum",<br>
> SPHEROID["Popular Visualisation Sphere",6378137,0,<br>
> AUTHORITY["EPSG","7059"]],<br>
> TOWGS84[0,0,0,0,0,0,0],<br>
> AUTHORITY["EPSG","6055"]],<br>
> PRIMEM["Greenwich",0,<br>
> AUTHORITY["EPSG","8901"]],<br>
> UNIT["degree",0.01745329251994328,<br>
> AUTHORITY["EPSG","9122"]],<br>
> AUTHORITY["EPSG","4055"]],<br>
> UNIT["metre",1,<br>
> AUTHORITY["EPSG","9001"]],<br>
> PROJECTION["Mercator_1SP"],<br>
> PARAMETER["central_meridian",0],<br>
> PARAMETER["scale_factor",1],<br>
> PARAMETER["false_easting",0],<br>
> PARAMETER["false_northing",0],<br>
> EXTENSION["PROJ4","+proj=merc +a=6378137 +b=6378137 +lat_ts=0.0<br>
> +lon_0=0.0 +x_0=0.0 +y_0=0 +k=1.0 +units=m +nadgrids=@null +wktext <br>
> +no_defs"],<br>
> AUTHORITY["EPSG","3785"],<br>
> AXIS["X",EAST],<br>
> AXIS["Y",NORTH]]<br>
> Origin = (-703633.464883987908252,7805920.616936270147562)<br>
> Pixel Size = (4.999999999999999,-5.000000000000000)<br>
> Metadata (oracle):<br>
> TABLE_NAME=fluvd04q200pj<br>
> COLUMN_NAME=georaster<br>
> RDT_TABLE_NAME=RDT_FLUVD04Q200PJ<br>
> RASTER_ID=62<br>
</div></div>> METADATA=<georasterMetadata xmlns="[LINK:<br>
> <a href="http://xmlns.oracle.com/spatial/georaster" target="_blank">http://xmlns.oracle.com/spatial/georaster</a>]<br>
<div><div></div><div class="h5">> <a href="http://xmlns.oracle.com/spatial/georaster" target="_blank">http://xmlns.oracle.com/spatial/georaster</a>"><br>
> <objectInfo><br>
> <rasterType>20001</rasterType><br>
> <isBlank>false</isBlank><br>
> <defaultRed>1</defaultRed><br>
> <defaultGreen>1</defaultGreen><br>
> <defaultBlue>1</defaultBlue><br>
> </objectInfo><br>
> <rasterInfo><br>
> <cellRepresentation>UNDEFINED</cellRepresentation><br>
> <cellDepth>8BIT_U</cellDepth><br>
> <NODATA>255</NODATA><br>
> <totalDimensions>2</totalDimensions><br>
> <dimensionSize type="ROW"><br>
> <size>271423</size><br>
> </dimensionSize><br>
> <dimensionSize type="COLUMN"><br>
> <size>190325</size><br>
> </dimensionSize><br>
> <ULTCoordinate><br>
> <row>0</row><br>
> <column>0</column><br>
> </ULTCoordinate><br>
> <blocking><br>
> <type>REGULAR</type><br>
> <totalRowBlocks>1061</totalRowBlocks><br>
> <totalColumnBlocks>744</totalColumnBlocks><br>
> <rowBlockSize>256</rowBlockSize><br>
> <columnBlockSize>256</columnBlockSize><br>
> </blocking><br>
> <interleaving>BIP</interleaving><br>
> <pyramid><br>
> <type>DECREASE</type><br>
> <resampling>NN</resampling><br>
> <maxLevel>11</maxLevel><br>
> </pyramid><br>
> <compression><br>
> <type>DEFLATE</type><br>
> </compression><br>
> </rasterInfo><br>
> <spatialReferenceInfo><br>
> <isReferenced>true</isReferenced><br>
> <isRectified>true</isRectified><br>
> <SRID>3785</SRID><br>
> <spatialResolution dimensionType="X"><br>
> <resolution>5</resolution><br>
> </spatialResolution><br>
> <spatialResolution dimensionType="Y"><br>
> <resolution>5</resolution><br>
> </spatialResolution><br>
> <modelCoordinateLocation>CENTER</modelCoordinateLocation><br>
> <modelType>FunctionalFitting</modelType><br>
> <polynomialModel rowOff="0" columnOff="0" xOff="0" yOff="0" zOff="0"<br>
> rowScale="1" columnScale="1" xScale="1" yScale="1" zScale="1"><br>
> <pPolynomial pType="1" nVars="2" order="1" nCoefficients="3"><br>
> <polynomialCoefficients>1561183.623387254 0<br>
> -0.2</polynomialCoefficients><br>
> </pPolynomial><br>
> <qPolynomial pType="1" nVars="0" order="0" nCoefficients="1"><br>
> <polynomialCoefficients>1</polynomialCoefficients><br>
> </qPolynomial><br>
> <rPolynomial pType="1" nVars="2" order="1" nCoefficients="3"><br>
> <polynomialCoefficients>140726.1929767976 0.2<br>
> 0</polynomialCoefficients><br>
> </rPolynomial><br>
> <sPolynomial pType="1" nVars="0" order="0" nCoefficients="1"><br>
> <polynomialCoefficients>1</polynomialCoefficients><br>
> </sPolynomial><br>
> </polynomialModel><br>
> </spatialReferenceInfo><br>
> <layerInfo><br>
> <layerDimension>BAND</layerDimension><br>
> <objectLayer><br>
> <layerNumber>0</layerNumber><br>
> <layerDimensionOrdinate>0</layerDimensionOrdinate><br>
> <layerID/><br>
> <colorMap><br>
> <colors><br>
> <cell value="1" red="255" green="0" blue="0"<br>
> alpha="100"/><br>
> <cell value="2" red="0" green="255" blue="0"<br>
> alpha="100"/><br>
> <cell value="3" red="0" green="0" blue="255"<br>
> alpha="100"/><br>
> <cell value="4" red="150" green="50" blue="100"<br>
> alpha="100"/><br>
> <cell value="255" red="0" green="0" blue="0" alpha="0"/><br>
> </colors><br>
> </colorMap><br>
> </objectLayer><br>
> </layerInfo><br>
> </georasterMetadata><br>
><br>
> Image Structure Metadata:<br>
> INTERLEAVE=PIXEL<br>
> COMPRESSION=DEFLATE<br>
> Corner Coordinates:<br>
> Upper Left ( -703633.465, 7805920.617) ( 6d19'15.05"W, 57d<br>
> 2'52.80"N)<br>
> Lower Left ( -703633.465, 6448805.617) ( 6d19'15.05"W,<br>
> 49d49'30.04"N)<br>
> Upper Right ( 247991.535, 7805920.617) ( 2d13'39.89"E, 57d<br>
> 2'52.80"N)<br>
> Lower Right ( 247991.535, 6448805.617) ( 2d13'39.89"E,<br>
> 49d49'30.04"N)<br>
> Center ( -227820.965, 7127363.117) ( 2d 2'47.58"W,<br>
> 53d35'23.19"N)<br>
> Band 1 Block=256x256 Type=Byte, ColorInterp=Gray<br>
> NoData Value=255<br>
> Overviews: 95162x135711, 47581x67855, 23790x33927, 11895x16963,<br>
> 5947x8481, 2973x4240, 1486x2120, 743x1060, 371x530, 185x265, 92x132<br>
><br>
><br>
</div></div><div class="im">> On Fri, Aug 13, 2010 at 12:32 PM, Lucena, Ivan <[LINK:<br>
> mailto:<a href="mailto:ivan.lucena@pmldnet.com">ivan.lucena@pmldnet.com</a>] <a href="mailto:ivan.lucena@pmldnet.com">ivan.lucena@pmldnet.com</a>> wrote:<br>
> Interesting.<br>
> Can you post your gdalinfo report with "-mdd oracle" option?<br>
> What are the versions your are running (GDAL,Oracle)?<br>
> Older version of the driver used to run a new query every time you change<br>
> levels.<br>
> Sebastian E. Ovide wrote:<br>
><br>
> yep I did... and the weir thing is that generating the images from "far<br>
> away" zoom under 12... it performs very well... not sure why it doesn't<br>
> in the levels 13-17.... for that reason I was trying to find the SQL<br>
> queries....<br>
><br>
><br>
</div>> On Fri, Aug 13, 2010 at 12:01 PM, Lucena, Ivan <[LINK:<br>
> mailto:<a href="mailto:ivan.lucena@pmldnet.com">ivan.lucena@pmldnet.com</a>] <a href="mailto:ivan.lucena@pmldnet.com">ivan.lucena@pmldnet.com</a> <mailto:[LINK:<br>
<div class="im">> mailto:<a href="mailto:ivan.lucena@pmldnet.com">ivan.lucena@pmldnet.com</a>] <a href="mailto:ivan.lucena@pmldnet.com">ivan.lucena@pmldnet.com</a>>> wrote:<br>
><br>
> Sebastian,<br>
><br>
><br>
</div><div class="im">> Sebastian E. Ovide wrote:<br>
><br>
> Hi All<br>
><br>
> I am using MapServer + GDAL to render GeoRasters and I am<br>
> having<br>
> very poor performance reading GeoRasters at a high zoom<br>
> level...<br>
> over 13... it would take even 20 seconds per tile...<br>
><br>
><br>
</div>> Did you generate pyramids?<br>
<div class="im">><br>
><br>
> Is it possible to log the GDAL queries that MapServer is<br>
> doing?<br>
> (and the SQL queries that GDAL is doing ?)<br>
><br>
> thanks<br>
><br>
> -- Sebastian E. Ovide<br>
><br>
><br>
> <br>
> ------------------------------------------------------------------------<br>
><br>
><br>
> _______________________________________________<br>
> mapserver-users mailing list<br>
</div>> [LINK: mailto:<a href="mailto:mapserver-users@lists.osgeo.org">mapserver-users@lists.osgeo.org</a>]<br>
<div class="im">> <a href="mailto:mapserver-users@lists.osgeo.org">mapserver-users@lists.osgeo.org</a><br>
</div>> <mailto:[LINK: mailto:<a href="mailto:mapserver-users@lists.osgeo.org">mapserver-users@lists.osgeo.org</a>]<br>
<div class="im">> <a href="mailto:mapserver-users@lists.osgeo.org">mapserver-users@lists.osgeo.org</a>><br>
><br>
</div>> [LINK:<br>
> <a href="http://lists.osgeo.org/mailman/listinfo/mapserver-users" target="_blank">http://lists.osgeo.org/mailman/listinfo/mapserver-users</a>]<br>
<div class="im">> <a href="http://lists.osgeo.org/mailman/listinfo/mapserver-users" target="_blank">http://lists.osgeo.org/mailman/listinfo/mapserver-users</a><br>
><br>
><br>
> --<br>
> Sebastian E. Ovide<br>
><br>
><br>
</div>> --<br>
> Sebastian E. Ovide<br>
</blockquote></div><br><br clear="all"><br>-- <br>Sebastian E. Ovide<br><br><br><br><br>