[GRASS-user] Problem importing/viewing a ~160MB shapefile

[Nikos Alexandris]

On Mon, 2008-10-13 at 17:07 -0700, Hamish wrote:
> Nikos Alexandris wrote:
> > Dear All, apologies for the cross-posting but I have trouble to work
> > with a BIG shapefil (~160MB). I already used that shapefile in the past
> > without problems. But now I can't use it.
> 
> ? what changed?  did you install a new version of GDAL/OGR?
> I use 400mb shapefiles without problem..
> 
>  
> > In GRASS: I (almost) managed to import in GRASS this BIG shapefile but
> > the process *stucks* somewhere at "breaking intersections" point (...it
> > was running for 10 hours). I broke the process and the file was in
> > GRASS' database. I run v.build but it still is problematic. I can't get
> > anything on-display.
> 
> It just takes a long time when lines are very long and unbroken. It
> eventually gets there.
> 
> See Radim's vector TODO:
>   http://trac.osgeo.org/grass/browser/grass/trunk/doc/vector/TODO#L242
> 
> and search the archives for the "Florida Coastline" problem.
> 
> 
> the v.in.ogr spatial= option might help.
> 
> 
> > In QGIS: when I try to load directly the shapefile (not the GRASS
> > vector) my system freezes!! I haven't seen my linuxbox frozen since I
> > don't remember when. What could cause such behaviour?
> > Is there anyway to get the shapefile fixed out of GRASS?
> 
> once loaded into GRASS you can run the v.split command to split the long
> lines, then it is much faster (at least for other grass commands).
> 
> 
> Hamish

Hamish and all,
back to this "old" post.

********
Attemp 1
********
# using gdal 1.5.3, grass6_devel
# import shapefile
v.in.ogr dsn=... out=...
# buffer overflow error (or segfault --- I can't recall).


********
Attemp 2
********
# using gdal-1.6.0beta2, grass6_devel
# importing the usual way
v.in.ogr dsn=coastlines.shp out=coastlines_longtime -ew --o

Projection of input dataset and current location appear to match
Layer: coastlines
WARNING: Column name changed: 'AREA' -> 'area'
WARNING: Column name changed: 'PERIMETER' -> 'perimeter'
[...]
Importing map 181148 features...
-----------------------------------------------------
Building topology for vector map <global_coastlines_longtime>...
Registering primitives...
181386 primitives registered
9764427 vertices registered
Building areas...
 100%
181386 areas built
181386 isles built
Attaching islands...
 100%
Attaching centroids...
 100%
Number of nodes: 181386
Number of primitives: 181386
Number of points: 0
Number of lines: 0
Number of boundaries: 181386
Number of centroids: 0
Number of areas: 181386
Number of isles: 181386
Number of areas without centroid: 181386
-----------------------------------------------------
WARNING: Cleaning polygons, result is not guaranteed!
 100%
Building topology for vector map <global_coastlines_longtime>...
Number of nodes: 181386
Number of primitives: 181386
Number of points: 0
Number of lines: 0
Number of boundaries: 181386
Number of centroids: 0
Number of areas: -
Number of isles: -
-----------------------------------------------------
Break polygons:
^[^[-----------------------------------------------------
Remove duplicates:
-----------------------------------------------------
Break boundaries:

# file get's imported and, as it can be seen from the output "breaking
polygons" and "removing duplicates" is successful
# the "^[^[" characters was me pressing something to check whether the
system is still running or not
# breaking boundaries is still there even after more than 30 hours!


********
Attemp 3
********
# v.in.ogr -c ## takes some time but is successful

# v.split (vertices=3) ## I have really no idea which number is
appropriated depending on what. ### it works

# v.clean tool=break ## never ends or it might need more than 2-3 days!?


I can open the file with QGIS preview (unstable) but not with QGIS
0.11.0. The latter freezes the whole system, as I have reported in my
first post about this.

Regards, Nikos