[mapserver-dev] WFS + GEOS + native Oracle Spatial + <ogc:Intersects> = not working?

[Vitali Diatchkov]

Hello.

I have several issues about MapServer as a WFS service.

I am using MapServer as a WFS service from UDIG.
Recently I have found that  raw <ogc:Intersects>  filter doesn't work on 
latest MapServer that I compile myself against layer with native Oracle 
Spatial connection. The reason was absence of GEOS support, so I 
complied MapServer with GEOS. And things became even more unexpected.

First of all after switching debug tracing in MAP file I have found that 
the log file contains:
-------------------------------------------------------------------------------
[Thu Mar 11 22:03:29 2010].859000 msOracleSpatialLayerResultGetShape(): 
OracleSpatial error. msOracleSpatialLayerResultGetShape record out of range
[Thu Mar 11 22:03:29 2010].859000 msOracleSpatialLayerResultGetShape(): 
OracleSpatial error. msOracleSpatialLayerResultGetShape record out of range
-------------------------------------------------------------------------------

The next step was debugging from Visual Studio and what a surprise.
Well it seems native Oracle  Spatial connector can directly deal only 
with WINDOW-based queries (or BBOX). All spatial filters like 
<ogc:Intersects> are processed in GEOS. But this is implemented in 
context of legacy architecture and I wouldn't be the first to mention 
that is is not efficient at all.
What I have found that if Intersects filter provided (as an example of 
non BBOX filter) then during first iteration all records are extracted 
from the table in  msOracleSpatialLayerWhichShapes function. At the same 
time an extent of the whole layer is used to get those records. If there 
are 1000000 records, means all are loaded. Intersection is done then 
using GEOS spatial predicates with a geometry provided in filter for all 
loaded records. Resuts are somehow cached.

Then the next step starts in FLTLayerApplyPlainFilterToLayer calling 
FLTAddToLayerResultCache. And there a bug is in: a) architecture b) 
"maporaclespatial.c" logic.
Let's assume there are 1300 records in Oracle table. The buffer in 
msOracleSpatialStatement structure is 1024 by default.  When last 
element in buffer is processed then the next bunch of items are fetched 
from a result set. In our case first time buffer is fully loaded and has 
1024 items. In next fetching the rest 276 items are loaded into the 
buffer starting from 1st cell. Fetching I am talking about takes place 
in msOracleSpatialLayerNextShape function.
What information is cached?
-------------------------------------------------------------
addResult(lp->resultcache, shape.classindex, shape.index, shape.tileindex);
-------------------------------------------------------------
in msQueryByOperator function.

In my test case shape.index is a primary key of a feature from a  
database, like "214789" and shape.tileindex is a kind of index of the 
cell in a buffer "msOracleSpatialStatemen.obj".  Let's assume 5 features 
are actually in a result of intersction operation. In current logic 
their shape.tileindex value might be 45, 567 or 1238 e.g. that is out of 
mentioned buffer limits. Actually is is assigned in
-------------------------------------------------------------------
shape->tileindex = sthand->row_num; /* Index into cursor */
-------------------------------------------------------------------
where row_num just grows permanently.

FLTAddToLayerResultCache leads to a call of
----------------------------------------------------------------------
int msOracleSpatialLayerResultGetShape( layerObj *layer, shapeObj 
*shape, int record, long pkey )
----------------------------------------------------------------------
where arguments are sort of:
record=-1 (always)
pkey=214789

where "record" is equal to -1 (always, comes from  "status = 
msLayerResultsGetShape(psLayer, &shape, -1, anValues[i]);")

this -1 actually kills all the logic in msOracleSpatialLayerResultGetShape.

In a result we have nothing to return as a WFS response and dummy XML is 
output:
--------------------------------------------------------------------
<wfs:FeatureCollection xsi:schemaLocation="http://www.opengis.net/wfs 
http://schemas.opengis.net/wfs/1.0.0/WFS-basic.xsd" >
<gml:boundedBy>
<gml:Box srsName="EPSG:2393">
<gml:coordinates>-1.000000,-1.000000 -1.000000,-1.000000</gml:coordinates>
</gml:Box>
</gml:boundedBy>
</wfs:FeatureCollection>
------------------------------------------------------------------------------

After some studying of code in maporaclespatial.c I can not understand 
how this scenario might work at all. Is it so that WFS with GEOS and 
MapServer's native Oracle Spatial is not a solution for this use case 
and only simple BBOX queries works?
But even if to imagine that Intersects filter works and there is no 
problem with buffer and ietms indexing overlapping, loading of all 
records during first step for processing by GEOS is not appropriate also.

The codebase of MapServer is large and I could miss something 
important.. Any comments on that?

Regards,
Vitali Diatchkov.

P.S. Latest branch-5-6 codebase is used.