[fdo-dev] FdoIInsert Batch Insert

[Pierre Dalcourt]

Hi Haris,

 

SHP Provider implements batch insert, but I don't think there is much
optimization there currently.  The current implementation basically
allows batch insert but I don't think  it doesn't provide any
significant advantage performance-wise over regular insert.  It could
very likely be optimized but not sure how much gain you could get out of
it.

 

Pierre

  _____  

From: Haris Kurtagic [mailto:haris at sl-king.com] 
Sent: October 28, 2006 8:06 AM
To: dev at fdo.osgeo.org
Subject: [fdo-dev] FdoIInsert Batch Insert

 

Hi,

 

I am trying to figure out how Batch Insert should work. From reading
documentation and looking in implementation in providers, it is very
unclear how it should be used.

Looking at shp,sdf and my sql, I see only shp provider is implementing
it.

 

Documentation say batch parameters is collection of parameter values for
each row.

 

I see from shp file that it is matching batch parameter value to
property values collection ( GetPropertValues ) using parameter name.

I see from shp implementation that property values are supposed to be of
type FdoParameterValue ( I think that needs to be added to
documentation).

 

SHP provider is implementing it in such way that it is matching this
parameters to properties for every row, and then converting to
properties value for each row. Altogether is very inefficient.

it also means that for every row we need to send parameter name + value.

 

I think that batch insert should be implemented in such way that batch
parameters values are send as collection of values and parameter names
are sent just as one row of parameter names.

In batch inserts, number of parameters or their order won't changed.
This would be minimum optimization could go further.

 

Another thing I saw in shp implementation is that it is accepting just
FdoParameter as type for property value (throw exception for other),

which would mean that it won't accept constant values in batch insert (
I am ok with that, but also should be documented ). 

If this is true for general way of FDO batch insert implementation than
batch inserts can be even more optimized.

 

Try to summarize:

 

I think we need to define what are rules for batch inserts and according
to that optimize batch inserts.

Batch inserts could become very important if FDO gets wider acceptance.

 

 

Haris

 

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.osgeo.org/pipermail/fdo_dev/attachments/20061101/457cad51/attachment.html