[GRASS-dev] [GRASS GIS] #2185: Painfully Slow 'v.in.ogr' Vector Import

[GRASS GIS]

#2185: Painfully Slow 'v.in.ogr' Vector Import
--------------------------------------+-------------------------------------
 Reporter:  justinzane                |       Owner:  grass-dev@…              
     Type:  defect                    |      Status:  new                      
 Priority:  normal                    |   Milestone:  7.0.0                    
Component:  Vector                    |     Version:  svn-trunk                
 Keywords:  import, OGR, performance  |    Platform:  Linux                    
      Cpu:  Unspecified               |  
--------------------------------------+-------------------------------------

Comment(by mmetz):

 Replying to [ticket:2185 justinzane]:
 > I'm trying to import shapefiles (in this case, OpenStreetMap data) into
 GRASS (grass70-svn) using `v.in.ogr`; and it is '''painfully, brutally
 slow'''. By slow, I mean between 13 and 28 ''primitives per second''.

 >
 > I cannot see any bottlenecks anywhere, as the two `python /path/to
 /grass-gui` processes are averaging 10% of one core combined, the network
 utilization is below 10% consistently, and the DB server is almost
 quiescent. To give an example of how painful this is:
 >
 {{{
      v.in.ogr dsn=/home/justin/downloads/osm_CA/places.shp
      WARNING: All available OGR layers will be imported into vector map
 <places>
      Check if OGR layer <places> contains polygons...
      Importing 7020 features (OGR layer <places>)...
      -----------------------------------------------------
      Building topology for vector map <places at OSM>...
      Registering primitives...
      7020 primitives registered
      7020 vertices registered
      Building areas...
      0 areas built
      0 isles built
      Attaching islands...
      Attaching centroids...
      Number of nodes: 0
      Number of primitives: 7020
      Number of points: 7020
      Number of lines: 0
      Number of boundaries: 0
      Number of centroids: 0
      Number of areas: 0
      Number of isles: 0
      (Fri Jan 31 19:01:08 2014) Command finished (4 min 38 sec)
 }}}
 >
 On a standard laptop with default settings, the import of points is about
 140 x faster. 7000 points from a shapefile should be imported in less than
 2 seconds.

 You would need to monitor the `v.in.ogr` process and the corresponding db
 driver process (`sqlite` for the default sqlite database in GRASS 7)
 instead of any Python processes and see if `v.in.ogr` uses close to 100%
 CPU. If not, `sqlite` should be very busy, but that should not happen
 because all changes to the database are committed at once after importing
 the primitives.

-- 
Ticket URL: <http://trac.osgeo.org/grass/ticket/2185#comment:2>
GRASS GIS <http://grass.osgeo.org>