[OSGeodata] Public Geodata Hosting: Large-scale Raster Hosting

[Allan Doyle]

I'd like to add that this kind of hosting can provide huge amounts of  
visibility into the ability of FOSS tools to do a job that up until  
now may well be considered to be in the province of the proprietary  
vendors by most people. We know better, but what better way to show  
what open source is capable of doing.

In every exercise that we do like this, I'd like to encourage a  
parallel (but integrated) activity of writing up a case-study of how  
it is/was done. This turns into a double bonus since we then not only  
have proof of ability but educational materials as well.

	Allan

On Jul 7, 2006, at 14:35, Chris Holmes wrote:

> I think it's definitely a worthwhile project.  Just the kind of  
> data I'd like to see us help out.  I had a great conversation with  
> a guy who's a planner for a pretty large corporation (though with  
> do-gooder roots, as he started off working for a small planning  
> organization which got bought by a bigger org, which got bought yet  
> again).  His job is basically to find lots of spatial data for  
> their projects and bring it all together for them to use.  Since  
> it's the US most of it is freely available, and if there was an  
> easy way for him to put it some repository, share it with others,  
> he'd happily upload/submit the stuff he finds there.  And indeed he  
> said it'd be a great benefit if such a place existed, since he  
> spends his time looking for such things in the first place.
>
> One of the things I feel strongly is that we just need to start  
> making spatial data more available, and if there's value there then  
> companies will start to make it easier to find.  So I think a great  
> thing the geodata committee could do is help out anyone who wants  
> to make geodata available.  From talking to John Graham, he seemed  
> to have quite a few resources, past what's offered to OSGeo  
> already, to host large amounts of data, especially of this sort.   
> So we should talk to him first, see if he's into the idea.  I  
> believe bandwidth isn't really a concern for him as he's at a  
> university?  And I imagine there might be others who could donate  
> resources.  We have a couple servers at TOPP ready to stand up  
> stuff (we're hosting WMS and WFS for TIGER of the whole US, soon  
> should tidy it up and announce).
>
> C
>
> Christopher Schmidt wrote:
>> I'm currently in a conversation with a representative of MassGIS  
>> about
>> hosting of their 2005 orthophotos for the state of Massachusetts.
>> Currently, these files are available from the MassGIS website in  
>> lossy
>> MrSID format.
>> Among the concerns with putting up the newer data in the larger  
>> JPEG2000
>> format is the bandwidth hit that MassGIS will suffer once these  
>> images
>> go up. Any organization hosting gigabytes of interesting data  
>> probably recognizes the problems inherent in the hosting --  
>> namely, that you can
>> quickly become a product of your own success.
>> As a result, MassGIS is currently seeking an organization to help  
>> mirror
>> these files on a fast link for public consumption, at least for the
>> first 6 months when interest in the photos will be at its peak.
>> Among the stated goals of the Public Geodata committee are:
>>  # Run a repository of open geodata
>>  # Promote public access to state-collected geodata
>>  # Promote the use of open geospatial formats
>> I think it's obvious how offering to help MassGIS with hosting of  
>> such a
>> dataset would be an excellent demo and test case of any repository of
>> open geodata. Granted, this information is not the typical form of
>> 'geodata' that is lobbied for in this situation, as vector data  
>> can in
>> many cases be used more effectively. However, large-scale sets of  
>> raster
>> data are clearly not to be ignored.
>> Additionally, MassGIS licensing is very permissive -- only  
>> attribution
>> is required, which makes it a case where it's high quality geodata  
>> that
>> definitely falls under the umbrella of 'open'.
>> By working with MassGIS to host such a dataset, we can practice  
>> what we
>> preach by helping to make access to the state-collected  
>> orthophotos more
>> publicly visible.
>> Additionally, by helping alleviate the bandwidth concerns associated
>> with more 'open' image format that is JPEG2000 over MrSID, the public
>> geodata committee can demonstrate by example their ability to help
>> promote more open formats. (Note that MrSID is still more broadly
>> supported than JPEG2000, so publishing those files is absolutely
>> appropriate for MassGIS to reach the most users -- but publishing  
>> MrSID
>> is not, it seems, the more difficult part of the problem.)
>> Some ideas were bandied about about searching for hosting at local
>> universities, etc., but I think this could be an interesting  
>> technical
>> project for the public geodata committee to take on: a large- 
>> scale, well
>> defined set of data with an enthusiastic governmental organization
>> looking to help users. (Governments looking to help their  
>> constituents!
>> excuse me while I faint!) However, the technical problems may be  
>> hard to
>> solve: the data is 15GB in lossy MrSID, so storing lossless  
>> JPEG2000 may
>> be quite a chore. The bandwidth might be much even for a non-MassGIS
>> entity, etc. I'm looking to get a feel for other people's  
>> feeelings on this topic:
>> how feasable is it? Is it a worthwhile project? And most importantly
>> because I'm extemely impatient, what's the time frame on being  
>> able to
>> set up resources for this type of thing?
>> Note: I do not speak for MassGIS -- I'm just acting as a go-between
>> since I feel that it's easier to hear this from a member of the group
>> than outside to start. :)
>
> -- 
> Chris Holmes
> The Open Planning Project
> http://topp.openplans.org
> <cholmes.vcf>

-- 
Allan Doyle
+1.781.433.2695
adoyle at eogeo.org