[postgis-devel] Transform fails in new transform caching code

[Jeremy Palmer]

> -----Original Message-----
> From: postgis-devel-bounces at postgis.refractions.net [mailto:postgis-devel-
> bounces at postgis.refractions.net] On Behalf Of Mark Cave-Ayland
> Sent: Wednesday, 23 November 2005 10:23 p.m.
> To: 'PostGIS Development Discussion'
> Subject: RE: [postgis-devel] Transform fails in new transform caching code
> 
> 
> > -----Original Message-----
> > From: postgis-devel-bounces at postgis.refractions.net [mailto:postgis-
> devel-
> > bounces at postgis.refractions.net] On Behalf Of palmerj at xtra.co.nz
> > Sent: 22 November 2005 23:36
> > To: postgis-devel at postgis.refractions.net
> > Subject: [postgis-devel] Transform fails in new transform caching code
> >
> > Hi,
> >
> > I'm getting the following error when using the new transform function:
> >
> > ERROR:  AddToPROJ4SRSCache: couldn't parse proj4 string: '+proj=tmerc
> > +lat_0=-43.59055555555556 +lon_0=172.7269444444445 +k=1.000000
> +x_0=400000
> > +y_0=800000 +ellps=GRS80 +towgs84=0,0,0,0,0,0,0 +units=m +no_defs ': no
> > system list, errno: 12
> >
> > Note I do not get this error immediately after running my transform
> query.
> > I have also tested the srid (2124) on multiple points and have not had
> > this problem before, so I'm sure the proj4test string is correct. It's
> > also interesting that err no = 12, which is a non list proj4 error...
> >
> > Just for a bit of background, the query I'm running is completing the
> > transformation from the Cartesian product of 60 crdsys to another 30
> > coordinate system. While this does not seem like a normal thing to do,
> it
> > is required for some projection statistics I'm trying to produce.
> >
> > Maybe there is a problem with the cache becoming full?
> >
> > Cheers
> > Jeremy
> 
> 
> Hi Jeremy,
> 
> Thanks for helping test the new projection code. Since we appear to be in
> totally opposing timezones, I've come up with several checks below that
> would really help me determine what is happening:
> 
> 
> 1) Ensure that you have the latest CVS checkout since there have been a
> couple of bugs fixed since I replied to your original post on the list -
> it
> may be that this will solve the problem on its own.
> 
> 2) Can you give the exact SQL and the proj4text for both SRIDs that you
> are
> using that causes the new code to fail.
> 
> 3) To check what the cache is doing, can you uncomment the
> PROJ4_CACHE_DEBUG
> definition around line 71 of lwgeom_transform.c, do "make install" again,
> and then re-connect using psql. You should then find that you get lots of
> debugging NOTICEs which will tell us exactly what is happening with the
> cache.
> 
> 4) You mentioned in your post that the query doesn't fail straight away.
> Can
> you try declaring a cursor something like:
> 
> BEGIN;
> DECLARE a CURSOR FOR <select statement>;
> FETCH n FROM a;
> 
> 
> By changing the value of n and repeating the FETCH, it is possible to
> fetch
> the next n transformed rows from your dataset. Using this you should be
> able
> to get an estimate of how far through your dataset you are managing to get
> before the error occurs.
> 
> If you could repeat this in several psql sessions, this allow us to see
> how
> many geometries can be processed before the transform fails, and whether
> this value changes between psql sessions. The largest set I've tested on
> is
> about 7 million records without any problems.
> 
> 
> Many thanks,
> 
> Mark.
> 
> ------------------------
> WebBased Ltd
> 17 Research Way
> Plymouth
> PL6 8BT
> 
> T: +44 (0)1752 797131
> F: +44 (0)1752 791023
> 
> http://www.webbased.co.uk
> http://www.infomapper.com
> http://www.swtc.co.uk
> 
> This email and any attachments are confidential to the intended recipient
> and may also be privileged. If you are not the intended recipient please
> delete it from your system and notify the sender. You should not copy it
> or
> use it for any purpose nor disclose or distribute its contents to any
> other
> person.
> 
> 
> _______________________________________________
> postgis-devel mailing list
> postgis-devel at postgis.refractions.net
> http://postgis.refractions.net/mailman/listinfo/postgis-devel

Hi Mark,

Thanks for the reply. I followed your suggestions and have upgraded to the
latest cvs. The problem seems to have gone away for the moment. It still
seems a little strange. I have defined PROJ4_CACHE_DEBUG as recommended and
will see if anything like this happens again during my testing this week.

As a side question, I noticed when looking through the lwgeom_transform.c
source that the proj cache size is hard set to 8. Is there a reason for
this? Why not just have a cache as large as require for the transaction? I
suppose for my query I can just sort by srid to minimise the number of db
lookups, but this will not always be appropriate.

Thanks for your help

Jeremy