[mapserver-dev] EPSG File Overhead

[Stephen Woodbridge]

Paul Ramsey wrote:
> It's not a mapserver problem, it's a proj problem. If we turn it into
> a mapserver problem, we complicate things.
> 
> Which is not to say that a "postmap" style lookup in proj would not
> actually be a reasonable way to go. Except then the index needs to be
> kept in sync with the text file, which people might alter and then not
> update the index file. (And if you've administered postfix, you know
> this happens, oh it happens often.)
> 
> Perhaps you are right, mapserver could read proj/epsg and write down
> /tmp/epsg.idx and do date checks and re-write it whenever they get out
> of sync. That approach is at least pretty much guaranteed to always
> work.
> 
> However, my understanding is that Frank wants to attack this at a proj
> level. We shall see in Toronto.

We already have the sync problem with .shp and .qix files and I think 
most people that use mapserver a lot get it, but have the occasional 
brain fart and forget.

Not that I run into this problem all that often, fixing it in proj would 
be the right place to do it, so all applications benefit from the change.

> Oh, one more thing: because this is an overhead that happens for every
> PROJECTION in a map file, regardless of whether the LAYER is being
> rendered, and because PROJECTIONs have a tendency to be homogeneous,
> something as simple as an in-Mapserver cache of projectionObj that
> just re-used the objects between layers would probably cut the
> overhead by a factor of 5 to 10 for most users. That might actually be
> enough to knock the overhead down to an acceptable level. Not zero,
> but not so obviously terrible anymore. And it wouldn't involve huge
> changes to Mapserver or huge changes to proj.

Doing a mapserver cache thing would be a good mapserver only solution.

-Steve W

> P
> 
> On Wed, Feb 4, 2009 at 8:08 PM, Stephen Woodbridge
> <woodbri at swoodbridge.com> wrote:
>> Paul,
>>
>> I do not understand why we don't just compile the project files into a
>> binary form that does not require parsing and is indexed/hashed for fast
>> lookups. It doesn't mean that we can't also use the text versions of the
>> files. A simple MS utility to compile/dump the binary version that would
>> solve the problem and keep it pretty simple and light weight.
>>
>> -Steve W
>>
>> Paul Ramsey wrote:
>>> It wouldn't be that hard to add something to the construction of the
>>> projectionObj that hashes all the configuration arguments into one key
>>> for identity comparison... frankly, the overhead in ESPG searching is
>>> so much higher (you pay it for every layer, every parse run, even if
>>> you don't use the layer) than the overhead for reprojection, you're
>>> still better off with the explicit strings.
>>>
>>> P.
>>>
>>> On Wed, Feb 4, 2009 at 1:47 PM, Paul Ramsey <pramsey at opengeo.org> wrote:
>>>> Yes, it's a pretty blunt test:
>>>>
>>>> http://trac.osgeo.org/mapserver/browser/trunk/mapserver/mapproject.c#L67
>>>>
>>>> On Wed, Feb 4, 2009 at 1:08 PM, Paul Spencer <pspencer at dmsolutions.ca>
>>>> wrote:
>>>>> I would like to note that changing mapfile projection blocks from init=
>>>>> to
>>>>> the equivalent proj strings does cause problems with WMS services as
>>>>> every
>>>>> feature is reprojected (even though the two projections are identical).
>>>>>
>>>>> Paul
>>>>>
>>>>> On 4-Feb-09, at 3:56 PM, Paul Ramsey wrote:
>>>>>
>>>>>> I'm firing up Andrea's suite and getting some profiling results...
>>>>>>
>>>>>> Using the tiger_pg test harness, and his default map file, with the
>>>>>> EPSG codes at the top of the EPSG file, I get throughput of 35.5 req/s
>>>>>> and profiling shows fscanf taking 26% of time. Editing the map file to
>>>>>> replace init=epsg: entries with full proj4 definitions, and running
>>>>>> again, I get throughput of 44.7 req/s and profiling shows that proj4
>>>>>> overhead has dropped to invisible (probably because there are no
>>>>>> actual reprojections in this test). At that point, the biggest CPU
>>>>>> overhead is compressing the output GIF, so we're going very well
>>>>>> indeed.
>>>>>> _______________________________________________
>>>>>> mapserver-dev mailing list
>>>>>> mapserver-dev at lists.osgeo.org
>>>>>> http://lists.osgeo.org/mailman/listinfo/mapserver-dev
>>>>> __________________________________________
>>>>>
>>>>>  Paul Spencer
>>>>>  Chief Technology Officer
>>>>>  DM Solutions Group Inc
>>>>>  http://research.dmsolutions.ca/
>>>>>
>>>>>
>>> _______________________________________________
>>> mapserver-dev mailing list
>>> mapserver-dev at lists.osgeo.org
>>> http://lists.osgeo.org/mailman/listinfo/mapserver-dev
>> _______________________________________________
>> mapserver-dev mailing list
>> mapserver-dev at lists.osgeo.org
>> http://lists.osgeo.org/mailman/listinfo/mapserver-dev
>>