[gdal-dev] Missing features after copying layers from Shapefiles to SQLite

[Chaitanya kumar CH]

Nik,

One option I frequently use when uploading shapefiles to PostGIS is to set
the -nlt option in ogr2ogr. Setting it as
multipolygon/multilinestring/multipoint for a shapefile defined as
polygon/linestring/point usually overcomes the problem.

Check if Spatialite allows adding single geometries when the datasource is
defined as a multigeometry. If not, you can force the geometry to
multigeometry before adding.



On Tue, Mar 4, 2014 at 5:39 AM, Nik Sands <nixanz at nixanz.com> wrote:

> So now, instead of the 'OGR_DS_CopyLayer', I'm using the code below to run
> SQL queries to split each layer from the source data set into multiple
> layers in the destination (Spatialite) dataset based on the geometry types.
>  (The addNewLayerNames: method will not add empty layers).
>
> It appears to be working OK for now (at least the maps LOOK right on
> screen).
>
> Is this a reasonable approach to overcoming the problem?
>
> Do I need to include more than just the 6 geometry types in the
> dictionary/hash below?
>
> Cheers,
> Nik.
>
>
> CODE:
> -----
>         NSDictionary *geomTypeExts = @{
>                 @"POINT" : @"pt",
>                 @"MULTIPOINT" : @"mpt",
>                 @"LINESTRING" : @"ls",
>                 @"MULTILINESTRING" : @"mls",
>                 @"POLYGON" : @"pg",
>                 @"MULTIPOLYGON" : @"mpg",
>         };
>
>         for ( NSString *geomType in [geomTypeExts allKeys] )
>         {
>                 NSString *sql = [NSString stringWithFormat:@"SELECT *
> FROM %@ WHERE OGR_GEOMETRY = '%@'", sourceLayerName, geomType];
>                 NSString *destLayerName = [NSString stringWithFormat:@"%@_%@",
> sourceLayerName, [geomTypeExts valueForKey:geomType]];
>
>                 [self addNewLayerNamed:destLayerName withSQL:sql fromDS:ds
> toDS:destDS];
>         }
>
>
> On 4 Mar 2014, at 9:54 am, Nik Sands <nixanz at nixanz.com> wrote:
>
> > Thanks for this tip.  I think you're on the right track with mixing
> single line strings with multi line strings and single polygons with multi
> polygons.
> >
> > (I don't think the number of features is causing any problems as it
> happens for some very small groups of features also.)
> >
> > It appears as though the first time it hits a feature type that is
> different to all of the others so far, it just bails out silently.
> >
> > So this implies that it's OK to mix feature types (multi/single) in
> Shapefiles (at least for reading) but not in Spatialite (at least for
> writing).  Is this correct?
> >
> > If this is so, then what is the correct way to convert/copy a layer from
> Shapefiles to Spatialite?
> >
> > (Note that is seems to work OK for plain SQLite but not for Spatialite).
> >
> >
> > On 3 Mar 2014, at 4:37 pm, Jukka Rahkonen <jukka.rahkonen at mmmtike.fi>
> wrote:
> >
> >> Nik Sands <nixanz <at> nixanz.com> writes:
> >>
> >>>
> >>> Hi all,
> >>>
> >>> I'm planning to switch from using a standard file format of Shapefiles
> to
> >> Spatialite for internal storage
> >>> of spatial data.
> >>>
> >>> When importing from other formats, the app attempts to convert these to
> >> SQLite (Spatialite) using
> >>> 'OGR_DS_CopyLayer' to copy each layer from the source data set to a new
> >> layer in the new SQLite dataset.
> >>>
> >>> However, when converting Shapefiles to SQLite, it only copies across
> about
> >> a third of the features for many
> >>> of the layers and therefore the new data set is of no use.
> >>>
> >>> The relevant code and output is below.
> >>>
> >>> I'm at a loss to figure out what is going wrong here and I'd be
> grateful
> >> is somebody could assist in getting
> >>> this to work as expected (or am I doing the wrong thing completely?).
> >>>
> >>> NB:  This is using Spatialite on iOS (simulator).
> >>>
> >>> When I use ogr2ogr on Mac OS X with SQLite (without Spatialite) it
> works
> >> fine and doesn't miss any features.
> >>>
> >>> Cheers,
> >>> Nik.
> >>>
> >>> CODE:
> >>>
> >>>     OGRLayerH sourceLayer = OGR_DS_GetLayer(ds, i);
> >>>     int sourceFeatureCount = OGR_L_GetFeatureCount(sourceLayer, YES);
> >>>
> >>>     OGR_DS_CopyLayer(destDS, sourceLayer, OGR_L_GetName(sourceLayer),
> NULL);
> >>>
> >>>     OGRLayerH destLayer = OGR_DS_GetLayerByName(destDS,
> >> OGR_L_GetName(sourceLayer));
> >>>     printf("sourceFeatureCount:  %d\n", sourceFeatureCount);
> >>>     printf("  destFeatureCount:  %d\n",
> OGR_L_GetFeatureCount(destLayer, YES));
> >>>
> >>> OUTPUT:
> >>>
> >>>     sourceFeatureCount:  3882
> >>>       destFeatureCount:  1200
> >>>
> >>
> >> Hi,
> >>
> >> My guess is that CopyLayer is making 200 feature sized transactions and
> only
> >> 6 transactions are accepted by Spatialite which makes 1200 features. The
> >> rest transactions are not accepted for some reason. With Spatialite my
> guess
> >> is that it complains about constraints and some of the features do not
> share
> >> the same geometry type and SRID. Perhaps trying to write multipolygons
> into
> >> a layer that accepts only polygons or vice versa.
> >>
> >> -Jukka Rahkonen-
> >>
> >>
> >>
> >> _______________________________________________
> >> gdal-dev mailing list
> >> gdal-dev at lists.osgeo.org
> >> http://lists.osgeo.org/mailman/listinfo/gdal-dev
> >
> > _______________________________________________
> > gdal-dev mailing list
> > gdal-dev at lists.osgeo.org
> > http://lists.osgeo.org/mailman/listinfo/gdal-dev
>
> _______________________________________________
> gdal-dev mailing list
> gdal-dev at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/gdal-dev
>



-- 
Best regards,
Chaitanya kumar CH.

+91-9494447584
17.2416N 80.1426E
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20140304/fd2fcc15/attachment-0001.html>