[gdal-dev] Coding for appending to a database in OGR

[Peter J Halls]

Frank and team,

    having reviewed the code again and the comments made by others, for which 
thanks, it seems to me that there is a possible solution that might resolve all 
the concerns.  I've asked my DBA for performance expectation comments, but have 
not yet heard back from him.

    Instead of trying to track the current FID and where the caller has not 
specified a value  'SELECT max(OGR_FID) + 1 FROM <tablename>' could be used 
instead.  This will always return the maximum value plus one and consigns the 
work to Oracle.  This will be 'safe' for multiple updaters and saves the 
overhead / risk of enquiring the value and updating it in the code.

    However, one side effect of this is that the value is not readily known to 
OGR.  As this is only when writing a new feature to the database, does this 
matter?  Is there any reliance on the FID within a newly created / written 
feature having the destination FID value on return - I suspect that, in the 
majority of instances, the next call is to destroy the feature handle?  As this 
would be a change in behaviour, at least it will be if I am correct that only 
the BoundCreateFeature method is actually used, then this may be significant.

How do others regard this?  Is it a big issue or a non-issue?

Best wishes,

Peter

Frank Warmerdam wrote:
> Peter J Halls wrote:
>> Even and team,
>>
>>    following the hint from Even's post below, I have been 
>> experimenting with OGR_F_SetFID to get the FID for appends to 
>> increment beyond the existing value, rather than always to start at 
>> 1.  I'm testing against the OCI driver and am not (yet) using 1.8.
>>
>>    I've found that there appears to be no way to 'reset' the starting 
>> FID value used by the driver for output.  I can certainly set the FID 
>> to an arbitrary value for a specific record, but the internal counter 
>> does not appear to be reset-able.  This means that I must manage the 
>> FID values externally to OGR when appending to an existing dataset in 
>> order to avoid new records being added with OGR_FID values restarting 
>> from 1.
>>
>>    Is this what is expected / intended?
> 
> Peter,
> 
> This is a flaw in the OCI driver.  FIDs are supposed to be unique, and
> if the CreateFeature() method is assigning one, it should ensure it is
> unique.
> 
> I have reviewed the trunk code, and this seems to be a problem with it too.
> 
> My feeling is that the first time iNextShapeId is used in the lifetime of
> an OGROCITableLayer we ought to do a query on the corresponding column
> to find it's max, and use that +1 as iNextShapeId.  This would generally
> avoid conflicts, and would not be as expensive as querying for each
> create feature.
> 
> The downside to this approach is that it assumes no one else is adding
> features to the table during the lifetime of the OGROCITableLayer object.
> This is usually safe, but not always.  This is one example of how the
> OGR RDBMS drivers are often not *really* prepared to live cleanly in
> a multi-user / multi-application environment.  It could be "done right"
> but the cost for bulk loads might be quite painful.  Fast bulk loads
> was the original reason for existence of the OCI driver.
> 
> If you would like to prepare code for that it would be great.  If not,
> please create a ticket for the issue and hopefully I'll have a chance to
> do it.
> 
> Best regards,

-- 
--------------------------------------------------------------------------------
Peter J Halls, GIS Advisor, University of York
Telephone: 01904 323806     Fax: 01904 323740
Snail mail: IT Services, University of York, Heslington, York YO10 5DD
This message has the status of a private and personal communication
--------------------------------------------------------------------------------