[mapserver-dev] EPSG File Overhead

[Paul Ramsey]

It's not a mapserver problem, it's a proj problem. If we turn it into
a mapserver problem, we complicate things.

Which is not to say that a "postmap" style lookup in proj would not
actually be a reasonable way to go. Except then the index needs to be
kept in sync with the text file, which people might alter and then not
update the index file. (And if you've administered postfix, you know
this happens, oh it happens often.)

Perhaps you are right, mapserver could read proj/epsg and write down
/tmp/epsg.idx and do date checks and re-write it whenever they get out
of sync. That approach is at least pretty much guaranteed to always
work.

However, my understanding is that Frank wants to attack this at a proj
level. We shall see in Toronto.

Oh, one more thing: because this is an overhead that happens for every
PROJECTION in a map file, regardless of whether the LAYER is being
rendered, and because PROJECTIONs have a tendency to be homogeneous,
something as simple as an in-Mapserver cache of projectionObj that
just re-used the objects between layers would probably cut the
overhead by a factor of 5 to 10 for most users. That might actually be
enough to knock the overhead down to an acceptable level. Not zero,
but not so obviously terrible anymore. And it wouldn't involve huge
changes to Mapserver or huge changes to proj.

P

On Wed, Feb 4, 2009 at 8:08 PM, Stephen Woodbridge
<woodbri at swoodbridge.com> wrote:
> Paul,
>
> I do not understand why we don't just compile the project files into a
> binary form that does not require parsing and is indexed/hashed for fast
> lookups. It doesn't mean that we can't also use the text versions of the
> files. A simple MS utility to compile/dump the binary version that would
> solve the problem and keep it pretty simple and light weight.
>
> -Steve W
>
> Paul Ramsey wrote:
>>
>> It wouldn't be that hard to add something to the construction of the
>> projectionObj that hashes all the configuration arguments into one key
>> for identity comparison... frankly, the overhead in ESPG searching is
>> so much higher (you pay it for every layer, every parse run, even if
>> you don't use the layer) than the overhead for reprojection, you're
>> still better off with the explicit strings.
>>
>> P.
>>
>> On Wed, Feb 4, 2009 at 1:47 PM, Paul Ramsey <pramsey at opengeo.org> wrote:
>>>
>>> Yes, it's a pretty blunt test:
>>>
>>> http://trac.osgeo.org/mapserver/browser/trunk/mapserver/mapproject.c#L67
>>>
>>> On Wed, Feb 4, 2009 at 1:08 PM, Paul Spencer <pspencer at dmsolutions.ca>
>>> wrote:
>>>>
>>>> I would like to note that changing mapfile projection blocks from init=
>>>> to
>>>> the equivalent proj strings does cause problems with WMS services as
>>>> every
>>>> feature is reprojected (even though the two projections are identical).
>>>>
>>>> Paul
>>>>
>>>> On 4-Feb-09, at 3:56 PM, Paul Ramsey wrote:
>>>>
>>>>> I'm firing up Andrea's suite and getting some profiling results...
>>>>>
>>>>> Using the tiger_pg test harness, and his default map file, with the
>>>>> EPSG codes at the top of the EPSG file, I get throughput of 35.5 req/s
>>>>> and profiling shows fscanf taking 26% of time. Editing the map file to
>>>>> replace init=epsg: entries with full proj4 definitions, and running
>>>>> again, I get throughput of 44.7 req/s and profiling shows that proj4
>>>>> overhead has dropped to invisible (probably because there are no
>>>>> actual reprojections in this test). At that point, the biggest CPU
>>>>> overhead is compressing the output GIF, so we're going very well
>>>>> indeed.
>>>>> _______________________________________________
>>>>> mapserver-dev mailing list
>>>>> mapserver-dev at lists.osgeo.org
>>>>> http://lists.osgeo.org/mailman/listinfo/mapserver-dev
>>>>
>>>> __________________________________________
>>>>
>>>>  Paul Spencer
>>>>  Chief Technology Officer
>>>>  DM Solutions Group Inc
>>>>  http://research.dmsolutions.ca/
>>>>
>>>>
>> _______________________________________________
>> mapserver-dev mailing list
>> mapserver-dev at lists.osgeo.org
>> http://lists.osgeo.org/mailman/listinfo/mapserver-dev
>
> _______________________________________________
> mapserver-dev mailing list
> mapserver-dev at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/mapserver-dev
>