[GRASS-user] How to open tiff images
Pawan Joshi
josh_0882 at yahoo.com
Thu Mar 1 12:13:21 EST 2007
hello,
Please help me to open tiff images i m using grass6.1 in windows
this is not supporting r.in.tiff when i m opening tif file by r.in.gdal there is only
a grey image.
grassuser-request at grass.itc.it wrote: Send grassuser mailing list submissions to
grassuser at grass.itc.it
To subscribe or unsubscribe via the World Wide Web, visit
http://grass.itc.it/mailman/listinfo/grassuser
or, via email, send a message with subject or body 'help' to
grassuser-request at grass.itc.it
You can reach the person managing the list at
grassuser-owner at grass.itc.it
When replying, please edit your Subject line so it is more specific
than "Re: Contents of grassuser digest..."
Today's Topics:
1. Re: 64 bit or parallel processing (S?ren Gebbert)
2. using LLID numbers with the linear referencing system
(Lindsay Mico)
3. Re: 64 bit or parallel processing (Hamish)
4. RE: 64 bit or parallel processing (Charles Ehlschlaeger)
5. RE: 64 bit or parallel processing (Jonathan Greenberg)
6. Open Source GIS Gathering San Diego (OSG-SD) - June 18th,
2007 (Dana Nibby)
7. Re: 64 bit or parallel processing (Glynn Clements)
8. Re: help with ogr2ogr and .shp file (Hamish)
----------------------------------------------------------------------
Message: 1
Date: Wed, 28 Feb 2007 23:48:49 +0100
From: S?ren Gebbert
Subject: Re: [GRASS-user] 64 bit or parallel processing
To: Glynn Clements
Cc: grassuser at grass.itc.it
Message-ID: <45E606D1.50907 at gmx.de>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Glynn Clements wrote:
> Hamish wrote:
>
>> It may be interesting to try and parallelize the segmentation library,
>> but most of GRASS's raster code needs to be rewritten.
>>
>> two problems:
>> 1) most grass raster modules are serial row based. maybe if the raster
>> format gets updated to a tiled model it would be possible to start
>> parallizing it. (not impossible, NULL and FP support was added in the past)
>> or break up lines into chunks like raster modules that have rows= or
>> percent= options already.
>
> Rows versus tiles doesn't have any impact upon the ability to
> parallelise the code.
It will be easier with a tile based approach to parallelize the code.
The g3d lib implementation allows different tile sizes. So you are able
to define tile sizes (befor open a map) which will fit perfectly into
the memory. Those tiles can be processed in parallel.
I guess a similar approach can be implemented using the segment library
for raster maps.
I hope to get a prototype running using tile based parallelizing
approach in 5 - 6 months. It will be part of the new gpde library and
will use the gpde array implementation for easy data access and because
its thread safe (AFAIKT).
The gpde lib uses OpenMP www.openmp.org for parallel computation in
grass. But only the linear equation solver and the linear equation
system assembler are parallelized. This will hopefully change in the future.
To enable parallel data access will be a hard work in grass and will
only work on cluster file systems. One approach will be to distribute
the rows and tiles to different storage places to enable parallel data
access.
Just my two cent
Soeren
>> 2) the raster file format is split over may directories making it hard
>> to "lock" a map so another job doesn't try and edit the same map. the
>> future plan is to have the raster format stored like the vector format,
>> $MAPSET/raster/$MAPNAME/element, edited as a copy and moved into position
>> in full when the raster is closed.
>>
>> 3) (bonus problem) we need to ensure that the region (WIND) file isn't
>> modified by any module but g.region, and that modules read the WIND file
>> only one when they first start. (in case it changes mid-process)
>
> This isn't an issue for parallelising individual modules (i.e. using
> multiple threads in order to utilise multiple CPU cores). It is an
> issue for being able to run multiple modules concurrently.
>
> The core problem for parallelising GRASS is that the libraries aren't
> remotely thread-safe, and the way that they are written is such that
> making them thread-safe would be a lot of work.
>
> The two biggest issues are:
>
> 1. Use of global/static variables; libgis alone has ~180 of them (not
> including the 181 GRASS_copyright variables).
>
> 2. Use of "scratch" buffers. Current policy deprecates the use of
> alloca(), so eliminating scratch buffers would involve lots of
> malloc/free calls for short-lived buffers, which is inefficient.
>
> Consequently, it isn't feasible to make the bulk of GRASS thread-safe.
> We might be able to do this for limited portions, e.g. the core raster
> I/O code (get/put row operations), but you would still need to ensure
> that everything else was called from the main thread.
>
> It would be more feasible to modify individual modules; e.g. a
> multi-threaded version of r.mapcalc might be feasible. But unless the
> raster I/O code was made thread-safe, you would still be limited by
> the rate at which map data could be read and written by a single
> thread, so it would only be worthwhile for cases where the bulk of the
> overhead was in the actual calculations.
>
------------------------------
Message: 2
Date: Wed, 28 Feb 2007 16:13:53 -0800
From: "Lindsay Mico"
Subject: [GRASS-user] using LLID numbers with the linear referencing
system
To: grassuser at grass.itc.it
Message-ID:
Content-Type: text/plain; format=flowed
Fellow GRASS users,
I would like to build a new point data layer from a large database of stream
surveys which have 2 pieces of spatial info:
1) an LLID number for that stream
2) distance from the start of that stream
Based on what I have seen, I would use the linear referencing system
modules. I am not sure if I need to build a new reference layer as I have a
stream layer with the LLID info already in it. Additionally, I could not
find an example of how v.lrs.segment works in practice. I have attempted to
summarize things as I understand it to help make it easier to answer my
questions. As I understand it the input parameters for v.lrs.segment would
be as follows:
1) input= the stream layer with the associated LLID info
2) output = the name of the output layer
3) llayer = whether it is a point or line (is it 1 for line and 0 for
point?)
4) rsdriver = I don't know what this means
5) rsdatabase = the location of the file with the LLID # and distance
6) rstable = the name of the file which has the LLID number and distance?
will it automatically add in the other data in the database? Or do I do
that later using a spreadsheet program on the .dbf
As I am still learning how to do this, I expect I will have more questions
in the future. Hopefully this will be of use to others as well. Many
thanks in advance. Cheers.
Lindsay Mico
Environmental Consultant
Demeter Design LLC
(503) 368-7195
http://demeterdesign.net
"Thoughtful Planning is the Surest Investment"
------------------------------
Message: 3
Date: Thu, 1 Mar 2007 13:14:14 +1300
From: Hamish
Subject: Re: [GRASS-user] 64 bit or parallel processing
To: Glynn Clements
Cc: grassuser at grass.itc.it
Message-ID: <20070301131414.69298749.hamish_nospam at yahoo.com>
Content-Type: text/plain; charset=US-ASCII
Glynn Clements wrote:
> It would be more feasible to modify individual modules; e.g. a
> multi-threaded version of r.mapcalc might be feasible. But unless the
> raster I/O code was made thread-safe, you would still be limited by
> the rate at which map data could be read and written by a single
> thread, so it would only be worthwhile for cases where the bulk of the
> overhead was in the actual calculations.
Right, so working on individual modules which can typically take 10+
hours to run (*.rst, watersheds, viewsheds (incl. r.sun), indices from
RS data, etc) seems like a much better focus of effort for the greatest
gain:work ratio. And as seen in the MPI add-ons this is exactly where
the previous ad-hoc effort has gone.
IIRC Helena said that the RST modules have changed substantially sice
GRASS 5 and the s.surf.rst.mpi work would have to be largely redone.
Any thoughts on gains that could be made by MPI'ing the segmentation
lib? Do do modules using that usually do so for memory needs not
processing speed?
Hamish
------------------------------
Message: 4
Date: Wed, 28 Feb 2007 18:52:38 -0600
From: "Charles Ehlschlaeger"
Subject: RE: [GRASS-user] 64 bit or parallel processing
To:
Message-ID: <5mf1q2$2ba67f at asav15.insightbb.com>
Content-Type: text/plain; charset="US-ASCII"
Most GRASS raster functions, r.mapcalc and similar, won't speed up by more
than a couple of percent from parallelization. 90% of the time spent by
these commands comes from moving the data off the hard drive and getting it
to the CPU and visa versa (based on parallel processing research I did in
the mid '90s). For "normal" GRASS raster commands, having four hard drives
on RAID 0 will dramatically increase GRASS performance more than anything
one thing you could do to a computer (or GRASS itself). The reason ORNL
parallelized my code was that the computational requirements was over c^2,
where c is the number of grid cells on a map, for some of the analyses they
wanted to do.
As Hamish wrote in recent email, good candidates for parallelization would
be viewshed analysis or other functions requiring many cells to be read to
get a value for each output grid cell. r.watershed won't benefit from
parallelization.
Chuck Ehlschlaeger, Associate Professor & GIS Center Director
Department of Geography, Western Illinois University
215 Tillman Hall, 1 University Circle, Macomb, IL 61455
cre111 at wiu.edu, phone: 309-298-1841, fax: 309-298-3003
-----Original Message-----
From: grassuser-bounces at grass.itc.it [mailto:grassuser-bounces at grass.itc.it]
On Behalf Of Glynn Clements
Sent: Wednesday, February 28, 2007 3:33 PM
To: Hamish
Cc: grassuser at grass.itc.it
Subject: Re: [GRASS-user] 64 bit or parallel processing
Hamish wrote:
> It may be interesting to try and parallelize the segmentation library,
> but most of GRASS's raster code needs to be rewritten.
>
> two problems:
> 1) most grass raster modules are serial row based. maybe if the raster
> format gets updated to a tiled model it would be possible to start
> parallizing it. (not impossible, NULL and FP support was added in the
past)
> or break up lines into chunks like raster modules that have rows= or
> percent= options already.
Rows versus tiles doesn't have any impact upon the ability to
parallelise the code.
> 2) the raster file format is split over may directories making it hard
> to "lock" a map so another job doesn't try and edit the same map. the
> future plan is to have the raster format stored like the vector format,
> $MAPSET/raster/$MAPNAME/element, edited as a copy and moved into position
> in full when the raster is closed.
>
> 3) (bonus problem) we need to ensure that the region (WIND) file isn't
> modified by any module but g.region, and that modules read the WIND file
> only one when they first start. (in case it changes mid-process)
This isn't an issue for parallelising individual modules (i.e. using
multiple threads in order to utilise multiple CPU cores). It is an
issue for being able to run multiple modules concurrently.
The core problem for parallelising GRASS is that the libraries aren't
remotely thread-safe, and the way that they are written is such that
making them thread-safe would be a lot of work.
The two biggest issues are:
1. Use of global/static variables; libgis alone has ~180 of them (not
including the 181 GRASS_copyright variables).
2. Use of "scratch" buffers. Current policy deprecates the use of
alloca(), so eliminating scratch buffers would involve lots of
malloc/free calls for short-lived buffers, which is inefficient.
Consequently, it isn't feasible to make the bulk of GRASS thread-safe.
We might be able to do this for limited portions, e.g. the core raster
I/O code (get/put row operations), but you would still need to ensure
that everything else was called from the main thread.
It would be more feasible to modify individual modules; e.g. a
multi-threaded version of r.mapcalc might be feasible. But unless the
raster I/O code was made thread-safe, you would still be limited by
the rate at which map data could be read and written by a single
thread, so it would only be worthwhile for cases where the bulk of the
overhead was in the actual calculations.
--
Glynn Clements
_______________________________________________
grassuser mailing list
grassuser at grass.itc.it
http://grass.itc.it/mailman/listinfo/grassuser
------------------------------
Message: 5
Date: Wed, 28 Feb 2007 17:06:33 -0800
From: "Jonathan Greenberg"
Subject: RE: [GRASS-user] 64 bit or parallel processing
To:
Message-ID: <000301c75b9d$dab6bab0$b0a6eda9 at terra.cstars.ucdavis.edu>
Content-Type: text/plain; charset="us-ascii"
I'm now working at UCD on a project with > 65 (!) hyperspectral images, and
a LOT of the slow downs stem from I/O intensive processes -- with that said,
there are plenty of "standard" raster processing routines that start
becoming processor intensive (e.g. linear spectral unmixing) that, with a
decent i/o setup (we have a lot of cheap-o 2-drive RAID0s that we back up to
a permanent location, this nearly doubles our I/O speed). Not having any
idea of how grass's raster access backbone functions, it does seem you could
have a generic tiling routine that could be called from all of the raster
commands that would be highly parallizable (I note that RSI ENVI does this
out of the box now, it detects the # of processors and simply starts forking
a single image across all of the processors, I assume one line/tile per
processor). Mapcalc and related processes would be a good start, and keep
in mind *spatial* analyses (e.g. texture windows) as you design it.
My suggestion to the GRASS programmers: if you decide this is useful, I'd
HIGHLY recommend trying to do this in an out-of-the-box approach, e.g.
having an install-time "mp" capability -- if it requires being a) experts in
raster processing, b) experts in GRASS and c) experts in parallel coding,
you'll end up with 3 people doing SMP :)
--j
-----Original Message-----
From: grassuser-bounces at grass.itc.it [mailto:grassuser-bounces at grass.itc.it]
On Behalf Of Hamish
Sent: Wednesday, February 28, 2007 4:14 PM
To: Glynn Clements
Cc: grassuser at grass.itc.it
Subject: Re: [GRASS-user] 64 bit or parallel processing
Glynn Clements wrote:
> It would be more feasible to modify individual modules; e.g. a
> multi-threaded version of r.mapcalc might be feasible. But unless the
> raster I/O code was made thread-safe, you would still be limited by
> the rate at which map data could be read and written by a single
> thread, so it would only be worthwhile for cases where the bulk of the
> overhead was in the actual calculations.
Right, so working on individual modules which can typically take 10+
hours to run (*.rst, watersheds, viewsheds (incl. r.sun), indices from
RS data, etc) seems like a much better focus of effort for the greatest
gain:work ratio. And as seen in the MPI add-ons this is exactly where
the previous ad-hoc effort has gone.
IIRC Helena said that the RST modules have changed substantially sice
GRASS 5 and the s.surf.rst.mpi work would have to be largely redone.
Any thoughts on gains that could be made by MPI'ing the segmentation
lib? Do do modules using that usually do so for memory needs not
processing speed?
Hamish
_______________________________________________
grassuser mailing list
grassuser at grass.itc.it
http://grass.itc.it/mailman/listinfo/grassuser
------------------------------
Message: 6
Date: Wed, 28 Feb 2007 18:23:11 -0800 (PST)
From: Dana Nibby
Subject: [GRASS-user] Open Source GIS Gathering San Diego (OSG-SD) -
June 18th, 2007
To: grass mailing list
Message-ID: <672492.17452.qm at web37112.mail.mud.yahoo.com>
Content-Type: text/plain; charset="iso-8859-1"
Folks, save the datum.
We're tentatively planning a 2nd Annual Open Source GIS Gathering to be held on Monday, June 18th - probably starting early afternoon, ending late in the evening. The same Monday of the Plenary Session(s) of another famous GIS conference held in San Diego.
Don't be lost in space... San Diego State University's Visualization Lab is easy to find via trolley from the San Diego Convention Center area.
We filled the Lab in 2006, and arrangements are being made to accommodate more. Suggested ideas for this year include technical workshops and roundtable discussions.
If you, or anyone you know, would be interested in presenting on an open source GIS topic, or conducting a technical workshop, please contact Dana Nibby here: danaspatial at yahoo.com.
There's now a mailing list for OSG-SD 2007. If you think you may be interested in attending, please sign-up below:
http://lists.telascience.org/mailman/listinfo/osg-sd
Spread the word to the extent possible. Geotag, you're "it."
Spatial Regards,
Dana
---------------------------------
Check out the all-new Yahoo! Mail beta - Fire up a more powerful email and get things done faster.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://grass.itc.it/pipermail/grassuser/attachments/20070228/da62997e/attachment-0001.html
------------------------------
Message: 7
Date: Thu, 1 Mar 2007 02:27:10 +0000
From: Glynn Clements
Subject: Re: [GRASS-user] 64 bit or parallel processing
To: Hamish
Cc: grassuser at grass.itc.it
Message-ID: <17894.14846.975053.90603 at cerise.gclements.plus.com>
Content-Type: text/plain; charset=us-ascii
Hamish wrote:
> Any thoughts on gains that could be made by MPI'ing the segmentation
> lib? Do do modules using that usually do so for memory needs not
> processing speed?
Modules which use the segment library normally do so because they
access the raster data in an order other than top-to-bottom.
If an algorithm reads the data in a linear fashion, there's no point
in using the segment library.
Ultimately, we need to look into replacing the core raster I/O code.
However, this needs to be done incrementally; we can't afford to put
everything on hold while we re-write everything which uses
G_{get,put}_raster_row() etc (i.e. most of GRASS).
--
Glynn Clements
------------------------------
Message: 8
Date: Thu, 1 Mar 2007 17:56:32 +1300
From: Hamish
Subject: Re: [GRASS-user] help with ogr2ogr and .shp file
To: antonio rodriguez
Cc: grass list
Message-ID: <20070301175632.2212ecdb.hamish_nospam at yahoo.com>
Content-Type: text/plain; charset=US-ASCII
antonio rodriguez wrote:
> I need some help and advice to transform a .shp file to a format
> readable by GRASS.
from within your lat lon GRASS location:
v.in.ogr location=
to import the shape file and create a new location for it based on its
.prj projection info. (assuming it has a .prj file. if not you'll have
to create the needed location manually and import using v.in.ogr from
within the new location)
> Extent: (100528.742188, 3991489.250000) - (610083.250000, 4288506.900918)
> Layer SRS WKT: (unknown)
Looks like you'll have to create a new location and set the projection
settings manually...
then (in the lat/lon location) suck it into the lat/lon projection with
v.proj.
Hamish
------------------------------
_______________________________________________
grassuser mailing list
grassuser at grass.itc.it
http://grass.itc.it/mailman/listinfo/grassuser
End of grassuser Digest, Vol 11, Issue 1
****************************************
Pawan Joshi
0120-4314870
G-5, Sector 20
Noida
"Do what you love @};- , Love what you do."
---------------------------------
Bored stiff? Loosen up...
Download and play hundreds of games for free on Yahoo! Games.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.osgeo.org/pipermail/grass-user/attachments/20070301/8a982b50/attachment.html
More information about the grass-user
mailing list