[GRASS-dev] [GRASS GIS] #2185: Painfully Slow 'v.in.ogr' Vector Import
GRASS GIS
trac at osgeo.org
Mon Mar 20 19:43:14 PDT 2017
#2185: Painfully Slow 'v.in.ogr' Vector Import
-------------------------+------------------------------------------------
Reporter: justinzane | Owner: grass-dev@…
Type: defect | Status: new
Priority: normal | Milestone: 7.0.6
Component: Vector | Version: svn-trunk
Resolution: | Keywords: import, OGR, performance, v.in.ogr
CPU: x86-64 | Platform: Linux
-------------------------+------------------------------------------------
Comment (by neteler):
Strange, here it is not fast at all:
{{{
GRASS 7.2.1svn (latlong_wgs84):~ > time -p v.in.ogr
in=world_AOI_latlong.geojson out=world_AOI_latlong
Check if OGR layer <OGRGeoJSON> contains polygons...
100%
WARNING: Width for column properties set to 255 (was not specified by
OGR),
some strings may be truncated!
Importing 1 features (OGR layer <OGRGeoJSON>)...
100%
-----------------------------------------------------
Registering primitives...
One primitive registered
7 vertices registered
Number of nodes: 1
Number of primitives: 1
Number of points: 0
Number of lines: 0
Number of boundaries: 1
Number of centroids: 0
Number of areas: -
Number of isles: -
-----------------------------------------------------
Cleaning polygons
-----------------------------------------------------
Breaking polygons...
Breaking polygons (pass 1: select break points)...
100%
Breaking polygons (pass 2: break at selected points)...
100%
-----------------------------------------------------
Removing duplicates...
100%
-----------------------------------------------------
Breaking boundaries...
98% ^C
real 373.04
user 371.13
sys 0.97
GRASS 7.2.1svn (latlong_wgs84):~ > g.version -g
version=7.2.1svn
date=2017
revision=r70769
build_date=2017-03-20
build_platform=x86_64-pc-linux-gnu
build_off_t_size=8
GRASS 7.2.1svn (latlong_wgs84):~ > cs2cs
Rel. 4.9.2, 08 September 2015
usage: cs2cs [ -eEfIlrstvwW [args] ] [ +opts[=arg] ]
[+to [+opts[=arg] [ files ]
GRASS 7.2.1svn (latlong_wgs84):~ > gdal-config --version
2.1.2
db.connect -p
driver: sqlite
database: /home/mundialis/grassdata/latlong_wgs84/user1/sqlite/sqlite.db
...
}}}
"svn status" does not show a single local modification.
The disk is a SSD drive. The system is a Fedora 25 installation:
{{{
rpm -qa | sort | grep 'geos\|gdal\|proj'
gdal-2.1.2-5.fc25.x86_64
gdal-devel-2.1.2-5.fc25.x86_64
gdal-grass-2.1.2-1.fc25.x86_64
gdal-libs-2.1.2-5.fc25.x86_64
gdal-python-2.1.2-5.fc25.x86_64
geos-3.5.0-3.fc25.x86_64
geos-devel-3.5.0-3.fc25.x86_64
proj-4.9.2-2.fc24.x86_64
proj-devel-4.9.2-2.fc24.x86_64
proj-epsg-4.9.2-2.fc24.x86_64
proj-nad-4.9.2-2.fc24.x86_64
pyproj-1.9.5.1-3.fc25.x86_64
}}}
strace shows a lot of lseek() if that matters:
{{{
...
lseek(6, 814250, SEEK_SET) = 814250
lseek(6, 814250, SEEK_SET) = 814250
lseek(6, 814250, SEEK_SET) = 814250
lseek(6, 814250, SEEK_SET) = 814250
lseek(6, -138, SEEK_CUR) = 814112
write(6, "\f", 1) = 1
fstat(6, {st_mode=S_IFREG|0664, st_size=814250, ...}) = 0
lseek(6, 811008, SEEK_SET) = 811008
read(6,
"l\fK\300&U@\304p\327\354y1U\300L\205\334i\322\27U\300\23_\363\0\254\350T@\r"...,
3242) = 3242
write(6,
"\r\6\0\0\0\246n\300\212l]e\300\305\240g\372\204\344e\300d\216\25\215\1ve@\357B\276"...,
101) = 101
fstat(6, {st_mode=S_IFREG|0664, st_size=814351, ...}) = 0
lseek(6, 811008, SEEK_SET) = 811008
read(6,
"l\fK\300&U@\304p\327\354y1U\300L\205\334i\322\27U\300\23_\363\0\254\350T@\r"...,
3343) = 3343
write(6,
"\r\2\0\0\0\246n\300\212l]e\300\246n\300\212l]e\300\23_\363\0\254\350T@\24_\363"...,
37) = 37
lseek(6, 811008, SEEK_SET) = 811008
read(6,
"l\fK\300&U@\304p\327\354y1U\300L\205\334i\322\27U\300\23_\363\0\254\350T@\r"...,
3205) = 3205
read(6,
"\r\2\0\0\0\246n\300\212l]e\300\246n\300\212l]e\300\24_\363\0\254\350T@\23_\363"...,
4096) = 175
lseek(6, 814388, SEEK_SET) = 814388
lseek(6, 814388, SEEK_SET) = 814388
lseek(6, 814388, SEEK_SET) = 814388
lseek(6, 814388, SEEK_SET) = 814388
lseek(6, 814388, SEEK_SET) = 814388
lseek(6, 814388, SEEK_SET) = 814388
lseek(6, 814388, SEEK_SET) = 814388
lseek(6, 814388, SEEK_SET) = 814388
lseek(6, -138, SEEK_CUR) = 814250
write(6, "\f", 1) = 1
fstat(6, {st_mode=S_IFREG|0664, st_size=814388, ...}) = 0
lseek(6, 811008, SEEK_SET) = 811008
read(6,
"l\fK\300&U@\304p\327\354y1U\300L\205\334i\322\27U\300\23_\363\0\254\350T@\r"...,
3380) = 3380
write(6,
"\r\6\0\0\0\246n\300\212l]e\300\305\240g\372\204\344e\300d\216\25\215\1ve@\357B\276"...,
101) = 101
fstat(6, {st_mode=S_IFREG|0664, st_size=814489, ...}) = 0
lseek(6, 811008, SEEK_SET) = 811008
read(6,
"l\fK\300&U@\304p\327\354y1U\300L\205\334i\322\27U\300\23_\363\0\254\350T@\r"...,
3481) = 3481
write(6,
"\r\2\0\0\0\246n\300\212l]e\300\246n\300\212l]e\300\24_\363\0\254\350T@\23_\363"...,
37) = 37
lseek(6, 811008, SEEK_SET) = 811008
read(6,
"l\fK\300&U@\304p\327\354y1U\300L\205\334i\322\27U\300\23_\363\0\254\350T@\r"...,
3343) = 3343
read(6,
"\r\2\0\0\0\246n\300\212l]e\300\246n\300\212l]e\300\23_\363\0\254\350T@\24_\363"...,
4096) = 175
lseek(6, 814526, SEEK_SET) = 814526
lseek(6, 814526, SEEK_SET) = 814526
lseek(6, 814526, SEEK_SET) = 814526
lseek(6, 814526, SEEK_SET) = 814526
lseek(6, -138, SEEK_CUR) = 814388
write(6, "\f", 1) = 1
fstat(6, {st_mode=S_IFREG|0664, st_size=814526, ...}) = 0
lseek(6, 811008, SEEK_SET) = 811008
read(6,
"l\fK\300&U@\304p\327\354y1U\300L\205\334i\322\27U\300\23_\363\0\254\350T@\r"...,
3518) = 3518
write(6,
"\r\6\0\0\0\246n\300\212l]e\300\305\240g\372\204\344e\300d\216\25\215\1ve@\357B\276"...,
101) = 101
fstat(6, {st_mode=S_IFREG|0664, st_size=814627, ...}) = 0
lseek(6, 811008, SEEK_SET) = 811008
read(6,
"l\fK\300&U@\304p\327\354y1U\300L\205\334i\322\27U\300\23_\363\0\254\350T@\r"...,
3619) = 3619
write(6,
"\r\2\0\0\0\246n\300\212l]e\300\246n\300\212l]e\300\23_\363\0\254\350T@\24_\363"...,
37) = 37
lseek(6, 811008, SEEK_SET) = 811008
read(6,
"l\fK\300&U@\304p\327\354y1U\300L\205\334i\322\27U\300\23_\363\0\254\350T@\r"...,
3481) = 3481
read(6,
"\r\2\0\0\0\246n\300\212l]e\300\246n\300\212l]e\300\24_\363\0\254\350T@\23_\363"...,
4096) = 175
lseek(6, 814664, SEEK_SET) = 814664
lseek(6, 814664, SEEK_SET) = 814664
lseek(6, 814664, SEEK_SET) = 814664
lseek(6, 814664, SEEK_SET) = 814664
lseek(6, 814664, SEEK_SET) = 814664
lseek(6, 814664, SEEK_SET) = 814664
lseek(6, 814664, SEEK_SET) = 814664
lseek(6, 814664, SEEK_SET) = 814664
lseek(6, -138, SEEK_CUR) = 814526
write(6, "\f", 1) = 1
fstat(6, {st_mode=S_IFREG|0664, st_size=814664, ...}) = 0
lseek(6, 811008, SEEK_SET) = 811008
read(6,
"l\fK\300&U@\304p\327\354y1U\300L\205\334i\322\27U\300\23_\363\0\254\350T@\r"...,
3656) = 3656
write(6,
"\r\6\0\0\0\246n\300\212l]e\300\305\240g\372\204\344e\300d\216\25\215\1ve@\357B\276"...,
101) = 101
fstat(6, {st_mode=S_IFREG|0664, st_size=814765, ...}) = 0
lseek(6, 811008, SEEK_SET) = 811008
read(6,
"l\fK\300&U@\304p\327\354y1U\300L\205\334i\322\27U\300\23_\363\0\254\350T@\r"...,
3757) = 3757
write(6,
"\r\2\0\0\0\246n\300\212l]e\300\246n\300\212l]e\300\24_\363\0\254\350T@\23_\363"...,
37) = 37
lseek(6, 811008, SEEK_SET) = 811008
read(6,
"l\fK\300&U@\304p\327\354y1U\300L\205\334i\322\27U\300\23_\363\0\254\350T@\r"...,
3619) = 3619
read(6,
"\r\2\0\0\0\246n\300\212l]e\300\246n\300\212l]e\300\23_\363\0\254\350T@\24_\363"...,
4096) = 175
lseek(6, 814802, SEEK_SET) = 814802
lseek(6, 814802, SEEK_SET) = 814802
lseek(6, 814802, SEEK_SET) = 814802
lseek(6, 814802, SEEK_SET) = 814802
lseek(6, -138, SEEK_CUR) = 814664
write(6, "\f", 1) = 1
fstat(6, {st_mode=S_IFREG|0664, st_size=814802, ...}) = 0
lseek(6, 811008, SEEK_SET) = 811008
read(6,
"l\fK\300&U@\304p\327\354y1U\300L\205\334i\322\27U\300\23_\363\0\254\350T@\r"...,
3794) = 3794
write(6,
"\r\6\0\0\0\246n\300\212l]e\300\305\240g\372\204\344e\300d\216\25\215\1ve@\357B\276"...,
101) = 101
fstat(6, {st_mode=S_IFREG|0664, st_size=814903, ...}) = 0
lseek(6, 811008, SEEK_SET) = 811008
read(6,
"l\fK\300&U@\304p\327\354y1U\300L\205\334i\322\27U\300\23_\363\0\254\350T@\r"...,
3895) = 3895
write(6,
"\r\2\0\0\0\246n\300\212l]e\300\246n\300\212l]e\300\23_\363\0\254\350T@\24_\363"...,
37) = 37
lseek(6, 811008, SEEK_SET) = 811008
read(6,
"l\fK\300&U@\304p\327\354y1U\300L\205\334i\322\27U\300\23_\363\0\254\350T@\r"...,
3757) = 3757
read(6,
"\r\2\0\0\0\246n\300\212l]e\300\246n\300\212l]e\300\24_\363\0\254\350T@\23_\363"...,
4096) = 175
lseek(6, 814940, SEEK_SET) = 814940
lseek(6, 814940, SEEK_SET) = 814940
lseek(6, 814940, SEEK_SET) = 814940
lseek(6, 814940, SEEK_SET) = 814940
lseek(6, 814940, SEEK_SET) = 814940
lseek(6, 814940, SEEK_SET) = 814940
lseek(6, 814940, SEEK_SET) = 814940
lseek(6, 814940, SEEK_SET) = 814940
lseek(6, -138, SEEK_CUR) = 814802
write(6, "\f", 1) = 1
fstat(6, {st_mode=S_IFREG|0664, st_size=814940, ...}) = 0
lseek(6, 811008, SEEK_SET) = 811008
read(6,
"l\fK\300&U@\304p\327\354y1U\300L\205\334i\322\27U\300\23_\363\0\254\350T@\r"...,
3932) = 3932
write(6,
"\r\6\0\0\0\246n\300\212l]e\300\305\240g\372\204\344e\300d\216\25\215\1ve@\357B\276"...,
101) = 101
fstat(6, {st_mode=S_IFREG|0664, st_size=815041, ...}) = 0
lseek(6, 811008, SEEK_SET) = 811008
read(6,
"l\fK\300&U@\304p\327\354y1U\300L\205\334i\322\27U\300\23_\363\0\254\350T@\r"...,
4033) = 4033
write(6,
"\r\2\0\0\0\246n\300\212l]e\300\246n\300\212l]e\300\24_\363\0\254\350T@\23_\363"...,
37) = 37
lseek(6, 811008, SEEK_SET) = 811008
read(6,
"l\fK\300&U@\304p\327\354y1U\300L\205\334i\322\27U\300\23_\363\0\254\350T@\r"...,
3895) = 3895
read(6,
"\r\2\0\0\0\246n\300\212l]e\300\246n\300\212l]e\300\23_\363\0\254\350T@\24_\363"...,
4096) = 175
lseek(6, 815078, SEEK_SET) = 815078
lseek(6, 815078, SEEK_SET) = 815078
lseek(6, 815078, SEEK_SET) = 815078
lseek(6, 815078, SEEK_SET) = 815078
lseek(6, -138, SEEK_CUR) = 814940
write(6, "\f", 1) = 1
fstat(6, {st_mode=S_IFREG|0664, st_size=815078, ...}) = 0
lseek(6, 811008, SEEK_SET) = 811008
read(6,
"l\fK\300&U@\304p\327\354y1U\300L\205\334i\322\27U\300\23_\363\0\254\350T@\r"...,
4070) = 4070
write(6,
"\r\6\0\0\0\246n\300\212l]e\300\305\240g\372\204\344e\300d\216\25\215\1",
26) = 26
write(6,
"ve@\357B\276I\231$e@\305\240g\372\204\344e\300\246n\300\212l]e\300\23_\363\0\254"...,
75) = 75
fstat(6, {st_mode=S_IFREG|0664, st_size=815179, ...}) = 0
lseek(6, 815104, SEEK_SET) = 815104
read(6,
"ve@\357B\276I\231$e@\305\240g\372\204\344e\300\246n\300\212l]e\300\23_\363\0\254"...,
75) = 75
write(6,
"\r\2\0\0\0\246n\300\212l]e\300\246n\300\212l]e\300\23_\363\0\254\350T@\24_\363"...,
37) = 37
lseek(6, 811008, SEEK_SET) = 811008
read(6,
"l\fK\300&U@\304p\327\354y1U\300L\205\334i\322\27U\300\23_\363\0\254\350T@\r"...,
4033) = 4033
read(6,
"\r\2\0\0\0\246n\300\212l]e\300\246n\300\212l]e\300\24_\363\0\254\350T@\23_\363"...,
4096) = 175
lseek(6, 815216, SEEK_SET) = 815216
lseek(6, 815216, SEEK_SET) = 815216
lseek(6, 815216, SEEK_SET) = 815216
lseek(6, 815216, SEEK_SET) = 815216
lseek(6, 815216, SEEK_SET) = 815216
lseek(6, 815216, SEEK_SET) = 815216
lseek(6, 815216, SEEK_SET) = 815216
lseek(6, 815216, SEEK_SET) = 815216
lseek(6, -138, SEEK_CUR) = 815078
write(6, "\f", 1) = 1
fstat(6, {st_mode=S_IFREG|0664, st_size=815216, ...}) = 0
lseek(6, 815104, SEEK_SET) = 815104
read(6,
"ve@\357B\276I\231$e@\305\240g\372\204\344e\300\246n\300\212l]e\300\23_\363\0\254"...,
112) = 112
write(6,
"\r\6\0\0\0\246n\300\212l]e\300\305\240g\372\204\344e\300d\216\25\215\1ve@\357B\276"...,
101) = 101
fstat(6, {st_mode=S_IFREG|0664, st_size=815317, ...}) = 0
lseek(6, 815104, SEEK_SET) = 815104
read(6,
"ve@\357B\276I\231$e@\305\240g\372\204\344e\300\246n\300\212l]e\300\23_\363\0\254"...,
213) = 213
write(6,
"\r\2\0\0\0\246n\300\212l]e\300\246n\300\212l]e\300\24_\363\0\254\350T@\23_\363"...,
37) = 37
lseek(6, 815104, SEEK_SET) = 815104
read(6,
"ve@\357B\276I\231$e@\305\240g\372\204\344e\300\246n\300\212l]e\300\23_\363\0\254"...,
75) = 75
read(6,
"\r\2\0\0\0\246n\300\212l]e\300\246n\300\212l]e\300\23_\363\0\254\350T@\24_\363"...,
4096) = 175
...
}}}
I'm a bit clueless... do you have uncommitted improvements? :-)
Can anyone else please test this tiny GeoJSON in a latlong location?
thanks.
--
Ticket URL: <https://trac.osgeo.org/grass/ticket/2185#comment:28>
GRASS GIS <https://grass.osgeo.org>
More information about the grass-dev
mailing list