[OSGeo-Discuss] Image Management in an RDBMS...(was OS Spatial environment 'sizing')

[Arnulf Christl]

Bruce.Bannerman at dpi.vic.gov.au wrote:
> IMO:
> 
> 
> 
> Hi Tyler,
> 
> Thanks for the reply.
> 
> 
>> Am I correct in believing that the two things people desire with 
>> images in an RDB,  is having an abstract 1) storage framework 
>> (tables) and 
>> 2) a common access language (SQL) for managing the 
>> framework.   You could have the most complex storage set up behind 
>> the scenes, but as long as the access interface plays well, the 
>> complexity could be minimised by good UI design.   At least I think 
>> so, but haven't done it before.
> 
> 
> 
> I can't speak for other people's needs, but only give my own opinion.

Hi,
sorry to bother again but I still see the need to clarify that there are two issues here. One is pragmatic experience with raster storage in a proprietary set of software. The other is whether there is a *need* to store rasters in a database at all. We have a customer who loves her SDE/Oracle and would never switch back to file based access because she feels it is falling back into stone age. I tell her that she has been brain washed by certain creative minds who "sell things" (instead of develop software) into thinking that files based systems are stone age. She hates me for it and curses her vendors at the same time now... There are arguments for file based approaches, one of them is that hardware is still developing really fast. Even disk read head caches are part of the spatial data infrastructure. It is just a question on whether you make use of them. Whether they are accessed by an operating system directly or by a database that sits on top of the operating system will 
surely make a difference in performance. 

> My experience with storing spatial data in a database is mainly limited to 
> ESRI's solutions based on ArcSDE/Oracle (~7 years). I have had a cursory 
> look at Oracle Spatial and PostGres/PostGIS and intend looking a lot 
> closer at both. I've also used a number of spatial tools over the years 
> from a number of vendors.
> 
> I've implemented and managed a number of ArcSDE instances over the years. 
> As skeptical as I am about the decision to rename the product as ArcGIS 
> Server basic (or whatever its called now), ESRI have done a great job with 
> the product. Particularly with its integration with ArcGIS Desktop as the 
> primary user interface for adding, maintaining and managing spatial data 
> from a GUI. You don't need to use SQL, but it also has its advantages 
> (when appropriate).

This should probably rather read "with ArcGIS Desktop as the *only* user interface" - which makes you depend on that vendor. I would amend the second part to read "Unfortunately you can't use SQL,..." but thats also just a personal opinion.

> I've found a number of benefits with managing spatial data in a corporate 
> database environment. These comments apply to both vector and image  data. 
> I'm sure that these comments are equally pertinent to most RDBMS 
> maintained spatial data. Some examples are: 

Just to make sure: "corporate database environment" refers to any database software, not just Oracle? SDE is not a corporate database environment or do you see it as a part of it? I can follow and underline the arguments related to holding vector data in a database but still fail to understand the need for rasters (which is probably mainly due to my ignorance).

> - Within a large organisation, it is possible to get rid of most of the 
> islands of data that are hidden in a wide variety of departments. If 
> implemented right, people come to see the database as the authoritive 
> source of their data and respect it as such.

Yes. But you would not want to have people talk to the database (ugh - SQL) but rather to a service. Give people a link to a service instead of a file to store on their own machine. There is no explicit need for a database, just encapsulate whatever you have behind a service. Users don't care what is behind the service, it can be a database or a lump of files on a SAN. 

> - This can remove the situation where you get multiple copies of the same 
> dataset around your organisation, with different people making their own 
> independent edits to the data and expecting someone to reconcile the edits 
> with the authoritative data set at a later time (if you're lucky).

Absolutely. But I fail to see the need to store rasters in a database arise from this argument.

> - It can also remove the situation where someone takes a copy of a 
> critical data set and does not update it for several years, leaving 
> business people making critical decisions on suspect data.
> 
> - You can start managing your data for a given geographic phenomena as a 
> single entity covering a large geographic region, without having to resort 
> to tiles and all the related edge matching problems that we had in the 
> past (e.g. mismatching pixels, lines, polygons that just end at the tile 
> boundary or have an incorrect attibute on the matching sheet etc). 

I am probably too vector oriented to understand the problems involved here but my experience is that there should be no issue if you have your services configured all right. 

> - Some of the biggest advantages though, come from the corporate IT 
> support that you come to rely on, e.g. regular backups, large disk 
> capacity on fast SAN devices, secure access to data by authorised 
> custodians, redundant databases for disaster recovery, point in time 
> restoration of data etc.

But there is no difference whether you back up files or a database. Well, actually there is a difference. Backing up files is a lot easier, more robust and much much better scalable than cramming everything into one or even several databases and trying to replicate that once it needs to be scaled up.

> To date, I have not found a suitable solution for managing imagery that 
> includes multi and hyper spectral data in a database. But I'm looking.
> 
> 
> 
> Ideally the solution will use open data formats for storage and delivery. 
> I'm getting sick of having to redevelop corporate applications with the 
> same functionality because a vendor has decided to change their technology 
> and data formats. This results in a lot of wasted time and money that 
> would be better used implementing effective decision support tools that 
> allow businesses to better understand and exploit their data. 

I can hear you loud and clear. 

Best regards, 
Arnulf

>> Some more recent raster in db discussion here:
>> http://spatialgalaxy.net/2008/02/15/rasters-in-the-database-why-bother/
>>
> 
> Thanks for the heads-up Tyler. I obviously don't know what I'm talking 
> about ;-) (eh Tim?).
> 
> 
> Bruce
> 
> 
> 
> 
> Notice:
> This email and any attachments may contain information that is personal, 
> confidential, legally privileged and/or copyright.No part of it should be reproduced, 
> adapted or communicated without the prior written consent of the copyright owner. 
> 
> It is the responsibility of the recipient to check for and remove viruses.
> If you have received this email in error, please notify the sender by return email, delete 
> it from your system and destroy any copies. You are not authorised to use, communicate or rely on the information 
> contained in this email.
> 
> Please consider the environment before printing this email.
> 
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> Discuss mailing list
> Discuss at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/discuss