[Gdal-dev] ogr2ogr segfaults with large SQL statement

[Frank Warmerdam]

Bruce Raup wrote:
>> I'm not too sure what version 1.6 is.  Perhaps 1.2.6 of GDAL?
> 
> I got that number from the libgdal file:  "libgdal.so.1.6.0", which I
> built and installed on 2005-02-28.

Bruce,

Ah.  That is the "libtool" version number.  I'm not really sure how to
relate it back to meaningful version numbers.

>> My understanding of the current logic is that it should list all
>> tables and views that have geometry columns in the list of named
>> layers.  The current query for layers looks like:
>>
>> SELECT c.relname FROM pg_class c, geometry_columns g
>>     WHERE (c.relkind in ('r','v') AND c.relname !~ '^pg'
>>     AND c.relname::TEXT = g.f_table_name::TEXT)
>>
>> Hmm, I see the documentation claims views are not supported.  Just
>> how does one go about creating a named view in Postgres?
> 
> Not supported in what?  PostgreSQL?  Views are definitely supported in
> PostgreSQL.  In psql, "\h create" gives (among other things):

I meant that the OGR PG driver docs say that the OGR PG driver does not
support views.

> Command:     CREATE VIEW
> Description: define a new view
> Syntax:
> CREATE [ OR REPLACE ] VIEW name [ ( column_name [, ...] ) ] AS query

Believe it ir not, even the above seems a bit abstract to me.  What
is "query", a full SELECT statement?  Why do I need to list column
names?  Just if I want to rename them?

>> I was going to suggest you use the special datasource syntax
>> for listing explicit tables but I see that isn't supported for
>> postgres.  Perhaps I need to add it.
> 
> I don't follow.  This is a special syntax for ogr2ogr?

For the RDBMS based drivers there is an issue with what tables and
views ought to be treated as layers for OGR.  In a big RDBMS there
can be a substantial performance cost to querying the details of
all tables in the system and turning them into OGR layers.  Not too
speak of burying the user in a long list of layers.

So I generally try strategies like "only show spatial layers", or
"only show layers in the users schema".  The problem is this makes
it difficult to address other tables through OGR.   One solution
to this I have used in serveral other RDBMS based drivers is to
allow a list of tables (or named views) to be treated as layers
to be listed in the "data source name".  That is what I was talking
about.

>>> - If the WHERE clause in the SQL statement exceeds around 5000
>>> characters, it still segfaults.
>> I have added a test for large attribute filters, and for large
>> SQL statements to the test suite, and identified a serious bug
>> which I have corrected.  Large attribute filters should now work
>> with OGR CVS.
> 
> Great!  By "large", what do you mean?  Is there still a hard limit? 
> Or does it use a dynamically allocated buffer?

I tested with a 10 or 15K query.  But I changed the code to use
dynamically sized strings.  I thought I had got all the fixed
sized buffers before but the problem didn't become apparent to me
till I prepared and ran an actual stress test.

Best regards,
-- 
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam, warmerdam at pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | President OSGF, http://osgeo.org