[UMN_MAPSERVER-USERS] Question re: PostGIS and MapScript getExtent() function

Fernando Simon fsimon at UNIVALI.BR
Tue Jul 12 17:38:10 EDT 2005

Hi folks,
    I added the support for getExtent function in 4.4 version and can 
run using two ways: using MBR and CONVEXHULL database function. The 
bugs: 1136 and 1112 is related about this issue.
    The column DIMINFO can store the extent for the table, but it's 
don't guarantee that it is the real extent, in many cases it's store: 
+180, -180, -90 and +90. So, when an user apply a filter or any 
restriction this values can be very wrong. Of course that I can use this 
for a fast extent, but it's not guarantee that it's real, many users can 
report in the future that the extents it is not real....
    The MBR function can be very slow when the table store many data, I 
needed to use it for 9i version and geodetic values. Convexhull function 
is more fast, but cannot be apply for 9i version. I used these functions 
because an user can change a layer filter and call the getextent 
function without need to change the data parameter for the layer. The 
performance is very good for the most requests, and increase with a good 
spatial index.
    The idea about the fast extent is very nice, more ideas? Only for 
WMS, like Paul suggest? What is the impact for this change in the others 
Mapserv modules? Hints?
    Thanks again.

Fernando Simon
Mapserver and Oracle Spatial developer
G10 - Laboratorio de Computacao Aplicada - Brazil
http://g10.cttmar.univali.br - UNIVALI/CTTMAR

Steve Hall wrote:

> In my experience with Oracle it too can be pretty slow getting an MBR 
> on a table that big  ;-)
> I don't know much about other data sources, but Oracle's spatial 
> metadata registry (USER_SDO_GEOM_METADATA) does maintain layer extent 
> data in the DIMINFO column so could be retrieved without needing to 
> perform a spatial aggregate, and hence would be a suitable candidate 
> for the *Fast function you descibe here.
> Like you say PostGIS also has a couple of choices in this area.
> I guess by definition the *Fast functions would just ignore, (or 
> fail?) if a filter was specified...?   Or perhaps if a filter is 
> specified, the *Fast functions could call the non-Fast functions to 
> get the answer in the (blind?) hope that the filter makes them 
> sufficiently performant.
> Thanks,
> Steve
> ----- Original Message ----- From: "Paul Ramsey" 
> <pramsey at refractions.net>
> To: "Steve Hall" <steve.hall at mercatorgeosystems.co.uk>
> Sent: Tuesday, July 12, 2005 7:37 PM
> Subject: Re: [UMN_MAPSERVER-USERS] Question re: PostGIS and MapScript 
> getExtent() function
>> Steve,
>> Try it on a 20M record table in Oracle. Both implementations,  
>> Oracle's and your uncommented PostGIS one, do the same functional  
>> thing, which is a full table scan.  It is possible that Oracle is  
>> slightly smarter about reading the index for the AGGR_MBR case, 
>> which  might make it work better on large tables: one for the testing 
>> range.
>> I think your tuppence might be right on the money:
>> - Enable ms*GetLayerExtent() to provide the "right" answer in all  
>> cases, even if it is expensive.
>> - Add a ms*GetLayerExtentFast() that for the easy data sources just  
>> calls GetLayerExtent, and for the hard data sources does whatever  
>> combination of fast hacks and cop-outs are required.
>> WMS GetCapabilities should call GetLayerExtentFast and mapscript  
>> should call GetLayerExtent.
>> Unrelated question for the gurus: does GetLayerExtent respect 
>> filters  for all data sources?
>> Paul
>> On 12-Jul-05, at 11:30 AM, Steve Hall wrote:
>>> Thanks Paul,
>>> This all sounds very reasonable to me....  I'll keep this to myself  
>>> for now then.
>>> I'm curious about the fact that this is implemented for Oracle  
>>> which is performing a very similar operation (i.e a spatial  
>>> aggregate, SDO_AGGR_MBR).  Is Oracle more efficient than PostGIS in  
>>> this area or is there a difference in approach that i'm missing...?
>>> For my two-pennorth a function like this is very useful, as it  
>>> allows me to change the filter for a layer, and alter the map  
>>> extent to match the extent of the "new" layer, without needing to  
>>> do any "dirty" SQL on the back-end database manually to find the  
>>> new extent.  Perhaps if it can't / shouldn't be done via these  
>>> msXXXLayerGetExtent() functions there is another way, or could it  
>>> be listed as an enhancement?
>>> Thanks again, i'm new to the contributing side of Open Source so  
>>> forgive my early mistakes!
>>> Steve
>>> ----- Original Message ----- From: "Paul Ramsey" 
>>> <pramsey at refractions.net>
>>> To: "Steve Hall" <steve.hall at mercatorgeosystems.co.uk>
>>> Sent: Tuesday, July 12, 2005 6:09 PM
>>> Subject: Re: [UMN_MAPSERVER-USERS] Question re: PostGIS and  
>>> MapScript getExtent() function
>>>> Steve,
>>>> If you grep through the source, you'll see the calls, which are   
>>>> mainly in the WMS code (pulling extents for capabilities).  The   
>>>> problem is that the return time for the call can be very large  
>>>> for  very large tables.  Which is not really acceptable for a  
>>>> capabilities  return. Maybe it is acceptable for your  
>>>> application.  This is the  real problem.  We want something with  
>>>> very low return times (as a  shapefile extents call is) but we  
>>>> cannot get it with the extent()  function, because that requires a  
>>>> sequence scan.
>>>> So basically, we cannot simply stick in your implementation, or  
>>>> we  are building a big obscure "gotcha" into mapserver. ("Why does  
>>>> it  take forever to return a capabilities file for my PostGIS  
>>>> backed WMS? Everything else is fast.")
>>>> Perhaps the folks on -DEV can return some zen wisdom on the  
>>>> correct approach to this.
>>>> Paul
>>>> On 12-Jul-05, at 9:52 AM, Steve Hall wrote:
>>>>> Hi Paul,
>>>>> Exactly right - it definitely needs reviewing by those that know   
>>>>> more about MapServer internals than I do.  (which is most of you  
>>>>> I expect).
>>>>> How is this function used my MapServer itself then?  The context  
>>>>> in which I am using it is via an explicit $layerObj->getExtent()  
>>>>> call from PHP/MapScript, so in this respect the current  
>>>>> implementation  is not much use, but I suspect there is more to  
>>>>> this function than  just an implementation for this call.   The  
>>>>> main issue I had was  that this function worked as I expected for  
>>>>> Oracle, yet not for  PostGIS, meaning I'd need some ugly code in  
>>>>> my app to deal with the inconsistency...
>>>>> Anyway, i've attached my version of the function for you to take  
>>>>> a look at. My implemenation is basically the version that was   
>>>>> commented out, with the compilcation errors & warning resolved,   
>>>>> added support for the FILTER expression and support for either  
>>>>> BOX  or BOX3D return clauses from PostGIS extent().
>>>>> Many thanks,
>>>>> Steve
>>>>> ----- Original Message ----- From: "Paul Ramsey" 
>>>>> <pramsey at refractions.net>
>>>>> To: "Steve Hall" <steve.hall at MERCATORGEOSYSTEMS.CO.UK>
>>>>> Sent: Tuesday, July 12, 2005 3:49 PM
>>>>> Subject: Re: [UMN_MAPSERVER-USERS] Question re: PostGIS and   
>>>>> MapScript getExtent() function
>>>>>> msPOSTGISLayerGetExtent() is potentially extremely inefficient, 
>>>>>> depending on how it is implemented, so I would have to see your 
>>>>>> patch to judge if it does more harm than good. It may be doing   
>>>>>> good things for you and bad things for others.  One thing it   
>>>>>> *could* do without harm is read the wms_extent meta-parameter  
>>>>>> and  use that if it is provided by the user. There are also a  
>>>>>> few  version specific fast extent functions available in  
>>>>>> PostGIS, but  they are not guaranteed to work in all cases, at  
>>>>>> all times,  correctly.
>>>>>> On 12-Jul-05, at 2:58 AM, Steve Hall wrote:
>>>>>>> Hi All,
>>>>>>> I'm using MapServer 4.6.0 with Oracle Spatial and PostGIS data 
>>>>>>> sources via PHP/MapScript.  Part of my application code looks   
>>>>>>> like this pseudo-code :
>>>>>>>     $map->setLayerFilter("ThematicData", "REQUEST_ID=121");
>>>>>>>     $map->zoomToLayer("ThematicData");
>>>>>>> Here i'm setting a specific FILTER expression for a layer, and   
>>>>>>> then zooming to the extent of the (now filtered) layer.  I use   
>>>>>>> the layerObj->getExtent() and then mapObj->setExtent()  
>>>>>>> functions  to achieve this behind the scenes.  This works great  
>>>>>>> for Oracle,  the query is fast and makes for nice, tidy code in  
>>>>>>> my  application without the need for Oracle specific SQL  
>>>>>>> statements  to derive the new extent.
>>>>>>> However when I try this on PostGIS I just get a massive   
>>>>>>> (useless) extent returned.  Digging into the code I can see  
>>>>>>> that  the msPOSTGISLayerGetExtent() function in mappostgis.c  
>>>>>>> is  effectively doing nothing although a lot of code is  
>>>>>>> commented out.
>>>>>>> Working on my local code I have re-enabled this code, fixed the 
>>>>>>> compilation errors and changed it so that when a FILTER is   
>>>>>>> applied to the layer this is used as part of the SQL call to   
>>>>>>> PG.  It seems to work great (from PHP/MapScript) and  
>>>>>>> performance  is fine and is really useful for me meaning I  
>>>>>>> don't need to  calculate the extents for a map myself -  
>>>>>>> MapServer can supply  the information in a generic and  abstract 
>>>>>>> fashion.
>>>>>>> Ideally I would like to get this code fed back into MapServer,   
>>>>>>> but of course only if it's acceptable to you all.  So,   should  
>>>>>>> I instead post this to mapserver-dev or can I pass on  the new  
>>>>>>> code to someone from here?
>>>>>>> There may be good, historical, reasons why this wasn't ever 
>>>>>>> implemented though so please let me know!
>>>>>>> Many thanks,
>>>>>>> Steve Hall
>>>>>>> <msPOSTGISLayerGetExtent.c>

More information about the mapserver-dev mailing list