[gdal-dev] Python-OGR not looping over more than 500 PGSQL features

[Matthieu Rigal]

Chaitanya,

I looked into the log files and could see the following lines :
2009-10-09 15:23:56 CEST lua_db lua_admin STATEMENT:  FETCH 500 in 
OGRPGResultLayerReader
2009-10-09 15:23:56 CEST lua_db lua_admin STATEMENT:  CLOSE 
OGRPGResultLayerReader

So it is OGR which is requesting only 500 features.
I may understand the problems to not have a limit but shouldn't there be some 
kind of a loop to be able to go automatically through all the features ?
If not, this should be mentioned on the PG driver page, isn't it ?

And furthermore, I never noticed this behaviour when previously working with 
older GDAL versions. Was I blind or did something changed ?

Best regards,
Matthieu


On Friday 09 October 2009 15:21:05 Chaitanya kumar CH wrote:
> Matthieu,
>
> GetFeatureCount in postgres driver is implemented with the COUNT()
> function. (
> http://trac.osgeo.org/gdal/browser/trunk/gdal/ogr/ogrsf_frmts/pg/ogrpgtable
>layer.cpp#L1895 ) So it was never returning more than 500 features. IMHO,
> the number 500 looks like an intentional limitation, probably set on the
> server connection.
>
> On Fri, Oct 9, 2009 at 4:11 PM, Matthieu Rigal <rigal at rapideye.de> wrote:
> > Hi people,
> >
> > I recently discovered a strange behaviour which was not happening before,
> > I think...
> > I'm using Python 2.5 with GDAL/OGR 1.6 and PGsql 8.3 on Linux.
> >
> > I am reading a table containing 858 features with several columns and
> > geometry.
> > I first do a SELECT on the table and the GetFeatureCount tells me 858
> > lines are being readen.
> > Then I am looping on the OGR object Feature by feature, to make some
> > calculation and I send an UPDATE for this particular line through another
> > OGR object.
> > I have a counter after every UPDATE.
> >
> > Whereas it should go through all the 858 features, it stops after having
> > gone through 500 features, my counter at the end shows 500 and looking to
> > the database, it clearly stops after 500.
> > I've done a test on a second database and it also stopped at 500...
> >
> > Is there some recently introduced limitations on OGR side ? I have
> > searched all the website without finding something about it... May it
> > come from some Python memory limitation ( I doubt) ? Or from some PGSQL
> > server limitation (I doubt also because 858 features are returned at the
> > beginning ?
> >
> > Thanks for your help,
> > Best Regards,
> > Matthieu Rigal
> >

RapidEye AG
Molkenmarkt 30
14776 Brandenburg an der Havel
Germany
 
Follow us on Twitter! www.twitter.com/rapideye_ag
 
Head Office/Sitz der Gesellschaft: Brandenburg an der Havel
Management Board/Vorstand: Wolfgang G. Biedermann
Chairman of Supervisory Board/Vorsitzender des Aufsichtsrates: 
Juergen Breitkopf 
Commercial Register/Handelsregister Potsdam HRB 17 796
Tax Number/Steuernummer: 048/100/00053
VAT-Ident-Number/Ust.-ID: DE 199331235
DIN EN ISO 9001 certified
 
*************************************************************************
Diese E-Mail enthaelt vertrauliche und/oder rechtlich geschuetzte
Informationen. Wenn Sie nicht der richtige Adressat sind oder diese
E-Mail irrtuemlich erhalten haben, informieren Sie bitte sofort den
Absender und vernichten Sie diese E-Mail. Das unerlaubte Kopieren sowie
die unbefugte Weitergabe dieser E-Mail ist nicht gestattet.
 
The information in this e-mail is intended for the named recipients
only. It may contain privileged and confidential information. If you
have received this communication in error, any use, copying or
dissemination of its contents is strictly prohibited. Please erase all
copies of the message along with any included attachments and notify
RapidEye AG or the sender immediately by telephone at the number
indicated on this page.