[GRASS-dev] [GRASS GIS] #2185: Painfully Slow 'v.in.ogr' Vector Import

[GRASS GIS]

#2185: Painfully Slow 'v.in.ogr' Vector Import
--------------------------------------+-------------------------------------
 Reporter:  justinzane                |       Owner:  grass-dev@…              
     Type:  defect                    |      Status:  new                      
 Priority:  normal                    |   Milestone:  7.0.0                    
Component:  Vector                    |     Version:  svn-trunk                
 Keywords:  import, OGR, performance  |    Platform:  Linux                    
      Cpu:  Unspecified               |  
--------------------------------------+-------------------------------------
 I'm trying to import shapefiles (in this case, OpenStreetMap data) into
 GRASS (grass70-svn) using `v.in.ogr`; and it is '''painfully, brutally
 slow'''. By slow, I mean between 13 and 28 ''primitives per second''.

 At first I thought this was because I was using the default sqlite
 database on an NFS mount. But, trying sqlite on a local disk and trying
 MySQL to separate server both failed to improve performance. I'm getting a
 consistent read/write of ~125Kbps/~550Kbps no matter which shapefile or
 which database I use.

 I cannot see any bottlenecks anywhere, as the two `python /path/to/grass-
 gui` processes are averaging 10% of one core combined, the network
 utilization is below 10% consistently, and the DB server is almost
 quiescent. To give an example of how painful this is:

     v.in.ogr dsn=/home/justin/downloads/osm_CA/places.shp
     WARNING: All available OGR layers will be imported into vector map
 <places>
     Check if OGR layer <places> contains polygons...
     Importing 7020 features (OGR layer <places>)...
     -----------------------------------------------------
     Building topology for vector map <places at OSM>...
     Registering primitives...
     7020 primitives registered
     7020 vertices registered
     Building areas...
     0 areas built
     0 isles built
     Attaching islands...
     Attaching centroids...
     Number of nodes: 0
     Number of primitives: 7020
     Number of points: 7020
     Number of lines: 0
     Number of boundaries: 0
     Number of centroids: 0
     Number of areas: 0
     Number of isles: 0
     (Fri Jan 31 19:01:08 2014) Command finished (4 min 38 sec)

 Note that this may belong under Database.

-- 
Ticket URL: <http://trac.osgeo.org/grass/ticket/2185>
GRASS GIS <http://grass.osgeo.org>