[OSGeo-Discuss] Image Management in an RDBMS...(was OS Spatial environment 'sizing')

[Bruce.Bannerman at dpi.vic.gov.au]

IMO:


Thanks for the reply Arnulf.


> 

> I tell 
> her that she has been brain washed by certain creative minds who 
> "sell things" (instead of develop software) into thinking that files
> based systems are stone age. She hates me for it and curses her 
> vendors at the same time now... There are arguments for file based 
> approaches, one of them is that hardware is still developing really 
> fast. Even disk read head caches are part of the spatial data 
> infrastructure. It is just a question on whether you make use of 
> them. 

Each person/organisation has their own problem space for managing spatial 
data. Some are working with large amounts of data over a large regional 
area where they want consistant data delivered. For others, they are only 
interested in working with a small subset of data for a particular 
analysis. There are countless other uses in between.

For many uses, a database makes sense, for others a file based approach 
makes sense.

I'm not trying to force you down one track, make your own choice based on 
your needs.

wrt brainwashing, I like to think that I'm healthily sceptical of vendors 
and OS claims alike. I prefer to have a good look at a solution and make 
up my own mind. My experiences tell me that the use of a database for 
managing spatial data (either vector or raster) is the right approach 
...... for my problem space.


> > 
> > I've implemented and managed a number of ArcSDE instances over the 
years. 
> > As skeptical as I am about the decision to rename the product as 
ArcGIS 
> > Server basic (or whatever its called now), ESRI have done a great job 
with 
> > the product. Particularly with its integration with ArcGIS Desktop as 
the 
> > primary user interface for adding, maintaining and managing spatial 
data 
> > from a GUI. You don't need to use SQL, but it also has its advantages 
> > (when appropriate).
> 
> This should probably rather read "with ArcGIS Desktop as the *only* 
> user interface" - which makes you depend on that vendor. I would 
> amend the second part to read "Unfortunately you can't use SQL,..." 
> but thats also just a personal opinion.

This is partly true as I understand it. You can use ArcSDE command line 
tools, Python Scripts, SQL and there may be scope for WFS-T in the future 
for maintaining data. There are also a number of OSGeo apps that can read 
ArcSDE.

A lot depends on how closely you want to follow ESRI's recommendations and 
directions (and how much you value the integrity of your data). My 
personal preference is to take advantage of the benefits of 
multi-versioned editting of spatial data using ArcSDE and ArcGIS, but to 
deliver the data and develop applications around the use of Open 
Standards. I'm currently looking at the use of OS Spatial apps to serve 
this data.

I'm also monitoring OSGeo apps for potential future desktop use (GRASS 
springs to mind).


> 
> Just to make sure: "corporate database environment" refers to any 
> database software, not just Oracle? SDE is not a corporate database 
> environment or do you see it as a part of it? 


These are generic comments relating to the use of a database.

Yes you are correct. ArcSDE can be thought of as an interface that 
controls access to the data stored in the underlying database.



> 
> > - Within a large organisation, it is possible to get rid of most of 
the 
> > islands of data that are hidden in a wide variety of departments. If 
> > implemented right, people come to see the database as the authoritive 
> > source of their data and respect it as such.
> 
> Yes. But you would not want to have people talk to the database (ugh
> - SQL) but rather to a service. Give people a link to a service 
> instead of a file to store on their own machine. There is no 
> explicit need for a database, just encapsulate whatever you have 
> behind a service. Users don't care what is behind the service, it 
> can be a database or a lump of files on a SAN. 

Yes. You want an interface between yourself and the DB. But there are use 
cases for controlled access via SQL (as long as the integrity of you data 
is assured).



> 
> > - This can remove the situation where you get multiple copies of the 
same 
> > dataset around your organisation, with different people making their 
own 
> > independent edits to the data and expecting someone to reconcile the 
edits 
> > with the authoritative data set at a later time (if you're lucky).
> 
> Absolutely. But I fail to see the need to store rasters in a 
> database arise from this argument.


See my earlier description of state wide mosaics with regular updates.

> 
> > - It can also remove the situation where someone takes a copy of a 
> > critical data set and does not update it for several years, leaving 
> > business people making critical decisions on suspect data.
> > 
> > - You can start managing your data for a given geographic phenomena as 
a 
> > single entity covering a large geographic region, without having to 
resort 
> > to tiles and all the related edge matching problems that we had in the 

> > past (e.g. mismatching pixels, lines, polygons that just end at the 
tile 
> > boundary or have an incorrect attibute on the matching sheet etc). 
> 
> I am probably too vector oriented to understand the problems 
> involved here but my experience is that there should be no issue if 
> you have your services configured all right. 
> 

I don't agree. But then my requirements are for spatial data that covers a 
large geographic area and is used by many people in a corporate 
environment. 

SOA, while it has definite advantages does not provide all of the answers.



> > - Some of the biggest advantages though, come from the corporate IT 
> > support that you come to rely on, e.g. regular backups, large disk 
> > capacity on fast SAN devices, secure access to data by authorised 
> > custodians, redundant databases for disaster recovery, point in time 
> > restoration of data etc.
> 
> But there is no difference whether you back up files or a database. 
> Well, actually there is a difference. Backing up files is a lot 
> easier, more robust and much much better scalable than cramming 
> everything into one or even several databases and trying to 
> replicate that once it needs to be scaled up.

Two examples:

- One day in December last year, I needed to restore a dataset as it was 
at 11am the previous day to save some maintenance people a weeks work in 
recreating their data. We were able to do this.

- Several years ago, we had a major power outage that took out several 
disks in our SAN. The disks just happened to contain all of our spatial 
data. We had services back up and running in several hours by cranking up 
our DR database.


Thanks for the debate. But all said, if a database doesn't meet your use 
case, don't use it.


Bruce




Notice:
This email and any attachments may contain information that is personal, 
confidential, legally privileged and/or copyright.No part of it should be reproduced, 
adapted or communicated without the prior written consent of the copyright owner. 

It is the responsibility of the recipient to check for and remove viruses.
If you have received this email in error, please notify the sender by return email, delete 
it from your system and destroy any copies. You are not authorised to use, communicate or rely on the information 
contained in this email.

Please consider the environment before printing this email.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/discuss/attachments/20080222/9d31a098/attachment-0002.html>