SWIG Mapscript

[Sean Gillies]

On Sep 30, 2004, at 9:57 AM, Ryan, Adam wrote:
>
>> -----Original Message-----
>> From: Sean Gillies [mailto:sgillies at frii.com]
>> Sent: Thursday, September 30, 2004 7:45 AM
>> To: Ryan, Adam
>> Cc: MAPSERVER-USERS at LISTS.UMN.EDU
>> Subject: Re: [UMN_MAPSERVER-USERS] SWIG Mapscript
>>
>>
>> On Sep 29, 2004, at 2:37 PM, Ryan, Adam wrote:
>>
>>>
>>>> -----Original Message-----
>>>> From: Sean Gillies [mailto:sgillies at FRII.COM]
>>>> Sent: Wednesday, September 29, 2004 12:19 PM
>>>> To: MAPSERVER-USERS at LISTS.UMN.EDU
>>>> Subject: Re: [UMN_MAPSERVER-USERS] SWIG Mapscript
>>>>
>>>>
>>>>> On Sep 29, 2004, at 12:54 PM, Ryan, Adam wrote:
>>>>>
>>>>>> Mapservers,
>>>>>>
>>>>>> Two methods I use in php mapscript don't seem to be in the SWIG
>>>>>> version.
>>>>>>
>>>>>> Layer->queryByIndex - this is non-critical since I can use
>>>>>> queryByAttribute
>>>>>> with an FID
>>>>>>
>>>>>> Map->loadQuery - this is critical.  There is a saveQuery
>>>> method, but
>>>>>> no
>>>>>> loadQuery?
>>>>>>
>>>>>> Thanks for any insight.
>>>>>>
>>>>>> Adam
>>>>>>
>>>>>
>>>>
>>>> Sorry, meant to say that I've *never* used the PHP mapscript and
>>>> don't understand the use for saving and loading queries.
>>>>
>>>> Sean
>>>>
>>>
>>> Hey, thanks for the reply.
>>>
>>> Am I the only one using saveQuery?  I think saveQuery and loadQuery
>>> are terrific methods.  They generate/read small files of
>> shape index
>>> values - similar, to a bitmap in Avenue I think.  By
>> loading a saved
>>> query, I can
>>> return a layer to a previous query 'state', for lack of a
>> better word
>>> (resultCacheObj state?).  This is extremely useful.
>>>
>>> Building shapefiles does not allow for dynamic queries
>> using AND, OR,
>>> or XOR
>>> with previous queries, and it's too slow.  The layer index
>> problem is
>>> easily
>>> handled by changing the layer index value in the saved query file
>>> itself
>>> prior to loading.  As long as only one layer is queried at
>> a time, this
>>> works well.
>>>
>>> I'm trying to re-write an application using Python and without this
>>> functionality it's a no go.  I can't even emulate a
>> saveQuery because
>>> there is no queryByIndex method, and even if there was, stepping
>>> through a list of
>>> indexes and calling queryByIndex is MUCH slower than the loadQuery
>>> method.
>>>
>>> I'm at a loss.
>>>
>>> Adam
>>>
>>
>> Adam,
>>
>> Thanks for the explanation.  I am not against implementing the PHP
>> savequery
>> and loadquery, although I do question why these methods write
>> and read
>> from
>> the filesystem when your logical combination needs to be done
>> in memory.
>>
>> Meanwhile, for 4.2, I have an idea.  Instead of manipulating
>> the layer's resultcache in place, create Python sets of query
>> results and do your
>> logical
>> operations on them.  For example::
>>
>>      import sets
>>
>>      ...
>>      layer.query1()  # assume success
>>      qset1 = sets.Set()
>>      for i in range(layer.getNumResults()):
>>          result = layer.getResult(i)
>>          result_tuple = (result.shapeindex, result.tileindex,
>> result.classindex)
>>          qset1.add(result_tuple)
>>
>>      layer.query2()  # assume success
>>      qset2 = sets.Set()
>>      for i in range(layer.getNumResults()):
>>          result = layer.getResult(i)
>>          result_tuple = (result.shapeindex, result.tileindex,
>> result.classindex)
>>          qset2.add(result_tuple)
>>
>>      # Union
>>      qunion = qset1.union(qset2)
>>
>>      # Intersection
>>      qintersection = qset2.intersection(qset1)
>>
>>      # Get shapes from the intersection
>>      for member in qintersection:
>>          layer.getShape(member[0], member[1])
>>          ...
>>
>>
>> Sets are a first class type in the upcoming Python 2.4, well
>> worth learning.
>>
>> cheers,
>> Sean
>>
>> --
>> Sean Gillies
>> sgillies at frii dot com
>> http://users.frii.com/sgillies
>>
>
>
> Sean,
>
> Thanks for the reply.  Your solution is what I've essentially been
> doing for
> a while now in PHP.  But this sort of thing is MUCH more useful when
> you are
> able to do a saveQuery.  For example:
>
> 1.)  User searches by Attributes: [NAME] = 'Smith'.  Query1.qy is
> saved and
> the file name is passed to the user along with a map showing all
> highlighted
> features.
>
> 2.)  User clicks the "AND" selection method and draws a box on the
> map.  The
> box extent and the query file name are passed back.  I do a
> queryByRect on
> the layer using the box to get result set1.  I load in the old query
> (set2).
> I check the selection method and perform the relate (AND,OR,XOR) and
> produce
> set3 (JUST AS YOU HAVE DESCRIBED ABOVE).  I save this as query2.qy,
> load it
> into the mapb object, drawQuery and spit back results.  Now only those
> features with Name = Smith AND inside the box are displayed.
>
> 3.)  User searches by Attributes: [YEARBUILT] < 1970 with selection
> method
> again set to AND which will further narrow the selection...
>
> ...and this can go on and on indefinitely.  This is the only way to do
> multiple and successive spatial and attribute queries in order to
> refine a
> selection set.
>
> THIS IS GIS BABY!
>

:)

Adam, I didn't understand that you were using drawQuery ... that changes
the equation a bit.


> And here's why my interface wouldn't work with your solution above:
>
> 1.)  Iterating through large selection sets using mapscript to build an
> array, or set, or whatever is too slow.  I know this to be true in PHP
> after
> doing a number of tests, and I can't say for sure in Python, but I
> suspect
> so.  It is MUCH faster to do map.saveQuery to a file, and then write a
> little routine to read the binary file into an array.  This seemed
> counter-intuitive to me, but I ran a bunch of tests and that's what I
> found.
> Much faster!
>

Yes, getting query results one by one with mapscript is the slowest
possible method.  In my opinion, what's *really* needed is a mapscript
method
that returns arrays, or Python lists, of results.  Then you wouldn't
have to
write/read from the filesystem either.  Faster yet.

> 2.)  At the end of your solution you have an array of indexes but
> nothing to
> do with them but get the shape.  What are you going to do with the
> shapes,
> draw each one?  That's way to slow.  What you need is a way to define
> the
> resultCache with your array, but queryByIndex is not implemented.
> And, as I
> said before, even if it was implemented, iterating though a large set
> and
> using queryByIndex on each index is WAY TO SLOW!  It is MUCH faster to
> dump
> that new set to a file and do a map.loadQuery.
>

Agreed, drawing individual shapes is slow.

> And if you build shapefiles (or entirely new files in whatever format
> you
> prefer), as Lowell has suggested, you loose the initial indexing of the
> original file and are no longer able to do successive spatial and
> attribute
> queries.  You could use a FID attribute, but then you'd be back to
> iterating
> through a bunch of queryByAttributes...ugggh!
>
> Please let me know what you think about this.
>
> I don't know C, I'm on Windows, and I don't even compile my own
> mapscript so
> I'm really just using the software and not contributing to it's
> development.
> In light of that, if there is any way I can help with this, I am more
> than
> willing to put in the time.  Let me know.
>
> Best regards,
>
> Adam
>

You use hobu's mapscript, right?  So you are asking for an enhancement
to mapserver 4.3/4.4?  I think a fully functional saveQuery/loadQuery
is a very reasonable request.  Please confirm, and I'll add an issue to
Bugzilla.

cheers,
Sean

--
Sean Gillies
sgillies at frii dot com
http://users.frii.com/sgillies