[postgis-users] Re: Re: rasters in PostGIS...

[Chapman, Martin]

Patrick,

Those are good comments and I agree.  I think maybe the optimal solution
would be if you could configure the database to work in either mode.
One mode to store rasters in the DB, one to store rasters outside the
DB, or perhaps even a mixed mode.  There seems to be valid arguments for
each case.  In any case, I think the discussion is an important one
since efficient use and management of rasters are fundamental to any
complete GIS solution these days, and I personally see
PostGIS/PostgreSQL as a major player both currently and in the future of
Spatially enabled databases.

On another note, I also think it would be interesting for the PostGIS
people to entertain the concept of de-coupling some of the processes
related to raster ingest/management/analysis to be separate executables
that could run from different servers in order to off-load some of the
processing from the database server.  An example of this would be a
program that creates raster overviews on another server and then once
finished, inserts the pyramids into the database.  That way, you could
run your long running ingestion processes without affecting the
performance of your database for other users.  Other examples, would be
backups and raster processes like mosaicking.  I'm not sure if that is
possible today, but if not I think that would help improve the
scalability of the database.  Just food for thought.

Martin

-----Original Message-----
From: postgis-users-bounces at postgis.refractions.net
[mailto:postgis-users-bounces at postgis.refractions.net] On Behalf Of
Patrick
Sent: Friday, September 30, 2005 1:46 AM
To: postgis-users at postgis.refractions.net
Subject: [postgis-users] Re: Re: rasters in PostGIS...


Hi Martin,

"Chapman, Martin" <MChapman at sanz.com> wrote in message 
news:ED3A48B9840E594890A2BC172D11946502F9C0B1 at mailman.san.com...
> Please understand that comments were only meant to offer help.

I'd like to first apologize for my shrill tone in that previous post. It
got 
late, I was cranky. Sorry about that.

My main point in the in-or-out discussion is that whether the DB is GB 
without or TB with raster data does not seem to be particularly relevant

because in both cases there are TBs of data that need loading, analysis,
and 
managing. In the enterprise environment where PostGIS is or could be 
prominent (and that specifically excludes data powerhouses like EDC,
GSFC, 
USDA, and the commercial outfits) data will be accessed by multiple
users 
over a network. So from the point of view of hardware capacity, CPU
load, or 
network congestion there isn't too much of a difference.

What really speaks for putting raster data in a DB is the management of 
access and concurrency, and the uniformity of interface. I am not
thinking 
of some static data warehousing here, but an active repository that is 
frequently accessed for data analysis. In that case, where multiple
users 
are accessing and editing data, a layout that optimizes data access (by 
tiling large datasets), prevents unauthorized access, and guards against

corruption due to multiple concurrent edits would be more than just a 
trivial feature set.

Mind you that most raster formats are actually not optimally organized
for 
the typical analysis task. Whereas many spatial operators use a kernel
of 
data around the current pixel (e.g. the slope function) most formats
store 
data in rows. This is trivial on small files, but not so on larger
files. If 
rasters are to be implemented in PostGIS it will definitely have some
form 
of tiling, to optimize data management at the DB level. This opens up
myriad 
opportunities for optimized analysis (where an application does not need
to 
load large amount of data to work on a small section), mosaicking (e.g.
all 
SRTM tiles as a single raster), etc.

Backups and hardware failures? Yes, well, the latter is as much part of
life 
as hunger and death and it affects TB databases as badly as GB
databases. 
And I have been doing my backups incrementally for the last 10 years and

although I have never had to backup TBs of data I very much believe that

PG-Joe will find himself in the same situation.

Even if it is not for the USGS, I think that many users would welcome 
rasters in PostGIS. Read the posts. But doing that right requires
critical 
review, so welcome aboard the PostGIS Raster Effort! {;-D

Cheers,
Patrick 



_______________________________________________
postgis-users mailing list postgis-users at postgis.refractions.net
http://postgis.refractions.net/mailman/listinfo/postgis-users