[GRASS-dev] GRASS GIS nightly builds
Newcomb, Doug
doug_newcomb at fws.gov
Sun Feb 24 17:49:42 PST 2013
Hamish,
On Sun, Feb 24, 2013 at 1:41 AM, Hamish <hamish_b at yahoo.com> wrote:
> Doug wrote:
> > Compiled with openmp and liblas?
>
> fwiw I would not really recommend to compile with OpenMP support
> unless you are willing to help develop that. Most of it is fine I
> think, but the LU parts used by v.surf.rst and v.surf.bspline are
> not very efficient, overhead is like 70%. So you do get a little speedup
> verus single core, but use a lot of power to do it.
> (currently that spawns 10s of thousands of threads when it would be better
> to spawn nearer to the number of local CPU cores)
>
> I've been playing with generating surfaces at home on an 8 core AMD
vishera system 3.5 ghz Ubuntu 12.04 system with G7 using this point layer:
+----------------------------------------------------------------------------+
| Name: nc_2a
|
| Mapset: PERMANENT
|
| Location: nc_stpft
|
| Database: /data2/grass70_data
|
| Title:
|
| Map scale: 1:1
|
| Name of creator: dnewcomb
|
| Organization:
|
| Source date: Sat Feb 23 08:16:54 2013
|
| Timestamp (first layer): none
|
|----------------------------------------------------------------------------|
| Map format: native
|
|----------------------------------------------------------------------------|
| Type of map: vector (level: 1)
|
|
|
| Number of points: 73232338 Number of centroids: 0
|
| Number of lines: 0 Number of boundaries: 0
|
| Number of areas: 0 Number of islands: 0
|
| Number of faces: 0 Number of kernels: 0
|
| Number of volumes: 0 Number of holes: 0
|
|
|
| Map is 3D: Yes
|
| Number of dblinks: 0
|
|
|
| Projection: Lambert Conformal Conic
|
|
|
| N: 694901.76 S: 482704.72
|
| E: 580721.99 W: 492486.01
|
| B: 1083.51 T: 5558.44
|
|
|
| Digitization threshold: 0
|
| Comment:
|
|
|
+----------------------------------------------------------------------------+
(Sat Feb 23 09:49:39 2013) Command finished (34 sec)
----------------------------------------------------------------------------------------------------------------------------------------------------------------------
I get the following 20ft resolution surface using v.usrf.bspline
---------------------------------------------------------------------------------------------------------------------------------------------------------------------
r.info map=nc_2_bspline_cubic at PERMANENT
+----------------------------------------------------------------------------+
| Layer: nc_2_bspline_cubic at PERMANENT Date: Sat Feb 23 19:13:29 2013
|
| Mapset: PERMANENT Login of Creator: dnewcomb
|
| Location: nc_stpft
|
| DataBase: /data2/grass70_data
|
| Title: bicubic interpolation with Tykhonov regularization (
nc_2_bsplin |
| Timestamp: none
|
|----------------------------------------------------------------------------|
|
|
| Type of Map: raster Number of Categories: 0
|
| Data Type: DCELL
|
| Rows: 10568
|
| Columns: 4412
|
| Total Cells: 46626016
|
| Projection: Lambert Conformal Conic
|
| N: 693610 S: 482260 Res: 19.99905375
|
| E: 580722 W: 492486 Res: 19.99909338
|
| Range of data: min = 92.2649807704968 max = 5558.35850509857
|
|
|
| Data Description:
|
| generated by v.surf.bspline
|
|
|
| Comments:
|
| v.surf.bspline -z input="nc_2a at PERMANENT" layer="1" raster_output="n\
|
| c_2_bspline_cubic" sie=40.0 sin=40.0 method="bicubic" lambda_i=0.01 \
|
| solver="cholesky" maxit=200 error=0.000001 memory=3000
|
|
|
+----------------------------------------------------------------------------+
(Sun Feb 24 19:53:17 2013) Command finished (0 sec)
subregion 920 of 920...
v.surf.bspline complete.
(Sun Feb 24 08:39:12 2013) Command finished (805 min 43 sec)
----------------------------------------------------------------------------------------------------------------------------------------------------------------------
As you can see , the surface generation ran in 805 minutes. I'm currently
rerunning the analysis with maxit = 100 to see how long it takes with the
iterations lowered.
Now with v.surf.rst, I generated the following grid:
---------------------------------------------------------------------------------------------------------------------------------------------------------------------
---------------------------------------------------------------------------+
| Layer: nc_2_rst_npmin_120 at PERMANENT Date: Sat Feb 23 01:14:43 2013
|
| Mapset: PERMANENT Login of Creator: dnewcomb
|
| Location: nc_stpft
|
| DataBase: /data2/grass70_data
|
| Title: ( nc_2_rst_npmin_120 )
|
| Timestamp: none
|
|----------------------------------------------------------------------------|
|
|
| Type of Map: raster Number of Categories: 0
|
| Data Type: FCELL
|
| Rows: 10568
|
| Columns: 4412
|
| Total Cells: 46626016
|
| Projection: Lambert Conformal Conic
|
| N: 693610 S: 482260 Res: 19.99905375
|
| E: 580722 W: 492486 Res: 19.99909338
|
| Range of data: min = -1.409014e+28 max = 7.890661e+27
|
|
|
| Data Source:
|
| vector map nc_2 at PERMANENT
|
|
|
|
|
| Data Description:
|
| generated by v.surf.rst
|
|
|
| Comments:
|
| tension=40.000000, smoothing=0.100000
|
| dnorm=215.219236, dmin=9.999527, zmult=1.000000
|
| segmax=40, npmin=120, rmsdevi=29786443112213122633433088.000000
|
| zmin_data=1083.510000, zmax_data=9999.000000
|
| zmin_int=-14090138501368785958609092608.000000,
zmax_int=78906611338755 |
|
|
| v.surf.rst input="nc_2 at PERMANENT" layer="1" elevation="nc_2_rst_npmi\
|
| n_120" tension=40. segmax=40 npmin=120 dmin=9.999527 dmax=49.997634 \
|
| zmult=1.0
|
|
|
+----------------------------------------------------------------------------+
(Sat Feb 23 08:10:21 2013) Command finished (0 sec)
v.surf.rst complete.
(Sat Feb 23 17:29:55 2013) Command finished (458 min 33 sec)
-------------------------------------------------------------------------------------------------------------------------------------------------------------------
As you can see, it took only 458 minutes to generate the same resolution
surface from the same input points. On the 8 core system, it is using 650%
of the cpu to process the data, bouncing up and down 30% or so as it
processes.
v.surf.bspline, on the other hand parses the data up into zones and uses
100% cpu up to the time it writes the individual zone out. When it writes
the zone out, it spikes up to 750% cpu. For this vector set and grid
extent, it generated 920 zones.
V.surf.rst seems to use much more memory than v.surf.bspline
I am of course, speaking as an end user who has not looked at the source
code. ( I'm trying to back myself into learning C :-))
>
> For that matter I would not recommend that new users be
> encouraged to use development versions at all, unless they are
> willing to help develop it, or at least debug it. And if
> development nightly builds are made available, it should be made
> very clear that it could eat all your data, harm your dog, etc.
>
>
Agreed , and I plan on giving the output to other folks to check against
the official state grid and the QA points. If this is inefficient, I would
really like to try it out after it was made more efficient :-) Thanks for
all the work!
Doug
> best,
> Hamish
>
--
Doug Newcomb
USFWS
Raleigh, NC
919-856-4520 ext. 14 doug_newcomb at fws.gov
---------------------------------------------------------------------------------------------------------
The opinions I express are my own and are not representative of the
official policy of the U.S.Fish and Wildlife Service or Dept. of the
Interior. Life is too short for undocumented, proprietary data formats.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/grass-dev/attachments/20130224/6d02acdf/attachment-0001.html>
More information about the grass-dev
mailing list