[OSGeo-Discuss] Open File Formats andProprietaryAlgorithms

Mon Aug 24 09:20:41 PDT 2009

Landon, 

Just had another thought . . . 

What about setting up a (openSource) tool set specifically for handling
Raster images for pre-processing purposes.  Might even be something that
publishers could re-distribute with their datasets, as in this processor
stack works with our data. 

Just thinking on the run here, no detail (or ramifications though
through [at all  :c) ]) 

bobb 

>>> "Bob Basques" <Bob.Basques at ci.stpaul.mn.us> wrote:

Landon, 

>It would be interesting to come up with a standard structure on a
computer file system that could be used to accessed tiled raster data,
if this hasn’t been done already. One the file system structure was
defined, it would be fairly easy to write open source software that
accessed this structure and provided individual tiles as a service to
desktop GIS applications.   

Hmm, interesting angle, to expand on your idea a bit more, what about a
processing suite (or set of suites) that process data for different
types of uses, visual display, DEM analysis, etc.  Each "processor"
stack would/could have it's own rules associated with data resolution vs
files sizes, etc.   

bobb 

>>> "Landon Blake" <lblake at ksninc.com> wrote:

Bobb wrote: “Here's my reasoning, we're never (ever?) going to hit the
top end on how big files ever get, resolution just keeps going up and
up, so there is always going to be some upper limit that will need to be
breached somehow.  Working out a proper method for segregating the data
up front (dare I say it), as some sort of standard (which can be
adjusted as time passes) will make everything work nicely, then all will
work with available tools when they are available, if tools to handle
larger datasets become available, and the community feels there is a
reason/need that these new larger files need to be handled, then they
get to change the standard.” 

I agree with some of the points you are making in your argument Bobb.
There is certainly a practical limit to how much you data you should put
in a single file. That is why we have lumber cut to 8 foot lengths. You
don’t need a flatbed semi to carry it to your house. :] 

When you refer to a standard for splitting data up front, what do you
mean? 

It would be interesting to come up with a standard structure on a
computer file system that could be used to accessed tiled raster data,
if this hasn’t been done already. One the file system structure was
defined, it would be fairly easy to write open source software that
accessed this structure and provided individual tiles as a service to
desktop GIS applications. 

Landon 

From: 

discuss-bounces at lists.osgeo.org [mailto:discuss-bounces at lists.osgeo.org]

On Behalf Of 

Bob Basques

Sent: 

Monday, August 24, 2009 7:33 AM

To: 

OSGeo Discussions

Subject: 

RE: [OSGeo-Discuss] Open File Formats and ProprietaryAlgorithms 

All, 

Ok, I'm probably going to get someone irritated, but here goes . . 

Why not approach this from the other end of the spectrum and work at
making the original files smaller.  Work with the providers to make the
images smaller in the first place, or at least come up with a maximum
practical size to work with, I mean if this is the only (or biggest
reason) for implementing JP2, then getting folks to make the smaller
deliverables seems like a better long term approach. 

Here's my reasoning, we're never (ever?) going to hit the top end on how
big files ever get, resolution just keeps going up and up, so there is
always going to be some upper limit that will need to be breached
somehow.  Working out a proper method for segregating the data up front
(dare I say it), as some sort of standard (which can be adjusted as time
passes) will make everything work nicely, then all reason/need that these new larger files need to be handled, then they
get to change the standard. 

bobb 

>>> "Fawcett, David" <David.Fawcett at state.mn.us> wrote: 

I realize that there are likely not a large number of people who have
the expertise and experience to write this kind of code. 

Is this a project that should be shopped around for funding?  Google
Summer of Code?  A grant from our ~benevolent overlord Google?  Some
other foundation or org interested in open data formats? 

David.
-----Original Message-----
From: discuss-bounces at lists.osgeo.org
[mailto:discuss-bounces at lists.osgeo.org] On Behalf Of Michael P. Gerlek
Sent: Thursday, August 20, 2009 4:36 PM
To: OSGeo Discussions
Subject: RE: [OSGeo-Discuss] Open File Formats and Proprietary
Algorithms
<snip>

> Do you know why there hasn't been a broader adoption of JP2?

Not through lack of trying on my part :-)

I think the two biggest reasons are:

(1) The algorithms for handling large images in memory really are rocket
science, and no one in the FOSS community has gotten the "itch"
sufficiently bad enough to go and do the work needed inside the existing
open source packages.  Hopefully someday someone will.

_______________________________________________
Discuss mailing list
Discuss at lists.osgeo.org http://lists.osgeo.org/mailman/listinfo/discuss
_______________________________________________
Discuss mailing list
Discuss at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/discuss 

Warning:
Information provided via electronic media is not guaranteed against
defects including translation and transmission errors. If the reader is
not the intended recipient, you are hereby notified that any
dissemination, distribution or copying of this communication is strictly
prohibited. If you have received this information in error, please
notify the sender immediately. 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/discuss/attachments/20090824/1333235b/attachment-0002.html>