[mapserver-dev] mappostgis.c

[Stephen Woodbridge]

Daniel,

Thank you for the excellent explanation that make prefect sense. So 
project is what needs to be fixed :)

Well it might be fixed if PROJ had a very small abstraction API to 
virtualize the access to the EPSG data files and parsing the data. Then 
existing apps could use the default file access driver, but anyone so 
inclined could overload the methods with their own drivers to postgres, 
sqlite, memcache, whatever.

So postGIS just implements its own version of PROJ library or what, 
given that it has its own spatial_ref_sys table or it just feeds the 
proj4text column into the existing PROJ functions.

Does postGIS have the same performance issues with projections?
If not, why not? can the postGIS pattern be mirrored in mapserver for 
example?
Since Paul is looking into this, I'm sure he has thought of these things 
and their is probably some good reason why this will not work efficiently.

-Steve W



Daniel Morissette wrote:
> At the moment, the handling of the epsg definitions uses a mechanism 
> that's built into PROJ and that is shared by all apps that use PROJ on a 
> given system. That doesn't mean things can't or shouldn't change, but if 
> we come up with our own mechanism then it will be MapServer-specific and 
> won't be shared by other apps that use PROJ... unless we embed the new 
> mechanism directly in PROJ, but I doubt that adding a SQLite dependency 
> in PROJ would fly very well.
> 
> All that being said, fixing that epsg performance issue has been on my 
> list for a while (for too long)... I just need to find the time to do it.
> 
> Daniel
> 
> Stephen Woodbridge wrote:
>> Paul, Steve,
>>
>> I think I idly asked this in the past and it was ignored. Why would we 
>> not consider using something like SQLite to:
>>   1) cache the parse epsg codes into, or
>>   2) just compile the epsg codes into a SQLite DB that could be 
>> accessed instead of the textfile.
>>
>> Here are some random thoughts on the subject:
>>
>> o the data could be preparsed into fields or a C struct that is copied 
>> into a blob and retrieved as needed
>> o SQLite licensing is compatible with mapserver
>> o yes - people might not want to have to compile the epsg file into a DB
>> o yes - it increases the technology footprint slightly
>> o but it might be useful for many other mapserver caching needs
>> o its thread safe, stable and very fast
>> o it could be configured to support both models: DB or text file by a 
>> PROCESSING directive in the MAP PROJECT block to point to the DB. 
>> Although it is probably cleaner to just support one. A simple utility 
>> could be provided like "epsg2db /path/to/proj.db" to compile the text 
>> file into a db.
>>
>> I'm just thinking out loud here. Does this make sense to anyone else?
>> 25% of the processing time is a huge win! that means a server would 
>> have 25% more capacity or you could serve the same number of requests 
>> with 75% of the hardware you currently need.
>>
>> -Steve W
>>
>> Paul Ramsey wrote:
>>> On Tue, Nov 4, 2008 at 10:54 AM, Steve Lime
>>> <Steve.Lime at dnr.state.mn.us> wrote:
>>>> A good threat, probably long overdue. Do you have any feel for the
>>>> performance end of things? I recall the Geoserver tests showing
>>>> GeoServer performing substantially better than MapServer w/regards 
>>>> to PostGIS.
>>>
>>> OK, this isn't testing against Geoserver, just against 5.2, and the 
>>> result is, 5.4trunk is maybe 5% slower. Which is not unsurprising, 
>>> since now the geometry is transiting a text encoding on the way. I
>>> had my fingers crossed that the simpler database handling would speed
>>>  things up, but no go.
>>>
>>> In terms of straight-up profiling information:
>>>
>>> 25% of cycles are still spent in pj_init, and this is *after* I moved
>>>  the two relevant EPSG codes to the top of the file. Geoserver has 
>>> objects cached. 20% of cycles are spent in image compression (gif or
>>> png). The Geoserver ImageIO might have optimized image compressors
>>> (my work with IPP impresses on me how much gain is available in this
>>> area).
>>>
>>> The penalty for parsing the proj4 definitions every time, even for 
>>> layers that aren't being drawn, is certainly high, and of course
>>> grows the more layers you have.  However, these overheads also exist
>>> in the shape file chain, so the difference between PostGIS and Shape
>>> relative to Geoserver remains unexplained.
>>>
>>> P. _______________________________________________ mapserver-dev
>>> mailing list mapserver-dev at lists.osgeo.org 
>>> http://lists.osgeo.org/mailman/listinfo/mapserver-dev
>>
>> _______________________________________________
>> mapserver-dev mailing list
>> mapserver-dev at lists.osgeo.org
>> http://lists.osgeo.org/mailman/listinfo/mapserver-dev
> 
>