[OSGeo-Discuss] Does rasdaman CE solve an open source geospatial problem?

Sun May 22 19:22:13 PDT 2016

Hi Edzer,

Thank you for your timely email.

I agree that the functionality of the community version has been
constrained and have noted your comments below on aa earlier thread on the
Rasdaman lists that proceeded down a similar line with direct responses to
issues avoided through diverting the discussion around the nuances of the
query used.

I have also noted that Rasdaman Gmbh has been gradually reducing the
difference between the two versions by porting some functionality across to
the Community version, just not the performance aspects that matter.

I tend to be quite pragmatic on these issues. The approach that I took was
that we need to focus on getting open project governance with many parties
involved as per [b1].

I considered that once we got a wider representation and a truely open
project we could begin to address these types of issues within an
appropriate framework.

In part, this is why I deliberately chose the wording at [2], in order to
expose and to force the issue.

This approach has clearly not worked and I cannot see the governance issue
resolved to the benefit of a truely open community as thinks currently
stand.

On reflecting through the past archives of the Incubation list relating to
Rasdaman Incubation, I see that these have been issues right from the
beginning.

We should not dictate to a commercial organisation how they run their
business. They will need to make decisions that are appropriate for their
future growth.

IMO, I see an organisation that is trying to find a viable way to make a
living from open source. They have chosen this approach. Many organisations
have this issue. Other organisations take alternate approaches.

It is time to call this for what it is and pull the plug on incubation.
Perhaps market forces will dictate an alternate approach for Rasdaman, but
I doubt it.

After (sadly) watching how this has played out and the intransigence
adopted, I’m personally treating any further investment of my time into
Rasdaman in its current structure to be a very poor investment. Though this
could quickly change if the governance issue is addressed constructively.

I have therefore shelved any plans that I have for use of Rasdaman. I’d say
that Thredds and PyWPS will meet my immediate needs.

I agree with you that this is a lost opportunity. The project has so much
potential.

As has been discussed recently on this list, there is always the option to
fork Rasdaman, but I do not think that this is something to be undertaken
lightly.

Bruce

[b1] http://lists.osgeo.org/pipermail/incubator/2016-May/003050.html

[b2] http://lists.osgeo.org/pipermail/incubator/2016-April/002996.html

On 22/05/2016, 23:23, "Discuss on behalf of Edzer Pebesma"

> <discuss-bounces at lists.osgeo.org on behalf of
> edzer.pebesma at uni-muenster.de> wrote:
>
> >Cameron, my comment is in-line:
> >
> >On 21/05/16 14:06, Cameron Shorter wrote:
> >> Hi Edzer,
> >> Thank you for raising this topic questioning the value of radsaman
> >> community edition. It is pertinent considering recent discussions about
> >> Rasdaman incubation.
> >>
> >> Peter, your comments about programmers wanting to get paid for their
> >> work is valid, but does not provide justification for OSGeo promoting a
> >> proprietary business model. OSGeo is in the business of promoting open
> >> source software, and helping people who create open source software.
> >>
> >> Rasdaman's business model is in a grey zone. It provides a community
> >> edition and a proprietary edition. This is often referred to as an "open
> >> core" business model, or sometimes less favorably called "crippleware".
> >> I think Rasdaman is the only OSGeo (proposed) project which provides an
> >> open core model. All prior projects have been pure open source.
> >>
> >> Although a an open core model deviates from OSGeo's original principles,
> >> one could argue that Rasdaman community edition stands on its own as a
> >> valuable, quality open source geospatial application by itself, worthy
> >> of OSGeo promotion.
> >>
> >> Edzer's comments appear to counter this argument. Edzer, I understand
> >> you suggest Rasdaman community edition is of little value for real world
> >> problems?
> >
> >Yes; the example Peter mentions, http://planetserver.eu/ does solve a
> >problem: providing a download service for large scale imagery, and it is
> >good if rasdaman CE can do this for 20 Tb, with sufficient performance.
> >There are however several other open source technologies that can also
> >do this, potentially much simpler (e.g. thredds data server, maybe even
> >gdal VRT).
> >
> >The problems where array data management systems really come into play
> >is scalable computing: doing something useful with large data sets
> >without having to download all the data first ("bringing the
> >computations to the data, instead of data to the computations") - this
> >is what much of the big data scalability fanfare is about, not about
> >serving or downloading data. And this is where rasdaman CE does not
> >scale - only EE seems to do so.
> >
> >The longer we (the open source community) wait with a good solution to
> >this problem, the more ground we loose to Google Earth Engine, a very
> >nice service that is bad for science. See also the last slide ("Google
> >Earth Engine is Evil") in the excellent presentation by Jordi Inglada,
> >held last week at the Living Planet Symposium:
> >
> >http://jordiinglada.net/stok/LivingPlanet/LandCoverSlides.pdf
> >
> >You might see this as a call to arms, and it is, but a new thread will
> >be needed to follow up on this.
> >
> >> Extending from this, OSGeo endorsement of Rasdaman should be questioned
> >> and potentially withdrawn.
> >>
> >> I'd be interested to hear opinions of others in the field as to whether
> >> Rasdaman community version is of value for real-world production systems
> >> by itself.
> >>
> >> A deeper question for the greater OSGeo community is should OSGeo
> >> endorse Open Core business models?
> >>
> >> Warm regards, Cameron
> >>
> >> On 21/05/2016 6:07 pm, Peter Baumann wrote:
> >>> oh, just looking at the subject again:
> >>>
> >>> several service providers believe indeed rasdaman community does offer
> >>> a significant advantage:
> >>> - see the download figures on www.rasdaman.org
> >>> - concretely, see www.planetserver.eu which is running rasdaman
> >>> community on - I believe - about 20 TB of Planetary Science data.
> >>>
> >>> -Peter
> >>>
> >>>
> >>> On 05/21/2016 09:56 AM, Peter Baumann wrote:
> >>>> Hm, first of all: this is opening a different thread, talking about
> >>>> functionality of rasdaman community. Next, it is based on assumptions
> >>>> - without details (because off topic): conclusions are wrong.
> >>>>
> >>>> But to respond to the core message emphasized in the first paragraph:
> >>>> I respectfully disagree. In particular, such a position does not
> >>>> benefit the open source community very much as I am trying to explain
> >>>> below.
> >>>>
> >>>> TL;DR:
> >>>>
> >>>> You have a strong expertise in Geoinformatics, I know something about
> >>>> Computer Science. This is where we can talk as professors and
> >>>> scientists. Your statement is about economics, industry etc. Having
> >>>> an opinion there (and articulate it) is fair, but in these fields our
> >>>> opinion weighs not more than anyone else's in the street. We should
> >>>> not attempt to attain importance through inapplicable roles.
> >>>>
> >>>> Let us look at a professor. They have a conveniently high salary
> >>>> which is paid by society, that is: tax payers. Nobody can influence
> >>>> what a professor does and how much return s/he generates for society.
> >>>>
> >>>> A single open source developer (or a small group, whatever) do not
> >>>> experience this convenience. They have a dream where they invest,
> >>>> they try to not make money for getting richer than a professor ;-)
> >>>> but merely for their economic survival. Some (in particular
> >>>> scientists) enjoy the money rain coming from publicly funded projects
> >>>> (again: the tax payer subsidizes), but most in the community have to
> >>>> struggle hard. They face reluctant customers, competition by the
> >>>> giants in the market, and many more obstacles.
> >>>>
> >>>> From the cosy place of a lifelong position with a secured salary and
> >>>> decent retirement funds it is easy to say that all software should be
> >>>> free like free beer (quote from below: "can be reproduced by other
> >>>> scientists without prohibitive license costs").
> >>>>
> >>>> If the open source movement cannibalizes itself it will make it all
> >>>> so easy for the big players to maintain their dominance, they will
> >>>> silently applaud. Quoting Jeroen:
> >>>> > NEVER IGNORE COMPANIES AGAIN IN OSGEO OR FOSS4G! THEY ARE NOT A
> >>>> THREAT, THEY ARE A NECESSITY.
> >>>>
> >>>> That said: It is entirely ok to have the opinion you have. Others,
> >>>> though, may disagree. I am one of those.
> >>>>
> >>>> respectfully,
> >>>> Peter
> >>>>
> >>>>
> >>>>
> >>>> On 05/20/2016 09:30 AM, Edzer Pebesma wrote:
> >>>>> As a scientist, I teach my students that for doing science it is a
> >>>>> requirement to work with open source software, because only then
> >>>>> workflows are fully transparent and can be reproduced by other
> >>>>> scientists without prohibitive license costs. Currently, working
> >>>>> with large amounts of earth observation (EO) or climate model data
> >>>>> typically requires to download these data tile by tile, stitch them
> >>>>> together, and go through all of them. Array databases may simplify
> >>>>> this substantially: after ingesting the tiles, they can directly
> >>>>> work on the whole data as a multi-dimensinal array ("data cube").
> >>>>> Computations on these array are typically embarassingly parallel,
> >>>>> and scale up with the number of cores in a cluster.
> >>>>>
> >>>>> Rasdaman is an array data base that comes in two flavours, the open
> >>>>> source community edition (CE) and the commercial enterprise edition
> >>>>> (EE). The differences between the two are clear [1]. When I want to
> >>>>> use rasdaman CE (open source) for scalable image analysis, I get
> >>>>> stuck waiting for one core to finish everything [1]. This is not
> >>>>> going to solve any problems related to computing on large data,
> >>>>> and is not scalable. The bold claim that rasdaman.org opens with
> >>>>> ("This worldwide leading array analytics engine distinguishes itself
> >>>>> by its flexibility, performance, and scalability") is not true for
> >>>>> the CE advertised. This has been mentioned in the past on mailing
> >>>>> lists [2,3], but the typical answer from Peter Baumann diverts into
> >>>>> other arguments. Also the benchmark graph (photo from an AGU poster)
> >>>>> [4] that Peter sent this week [5] must refer to the enterprise
> >>>>> edition, since Spark and Hive both scale, but rasdaman CE does not
> >>>>>[3].
> >>>>>
> >>>>> I assume that on the discussions on this list, ONLY the open
> >>>>> source community edition is considered, compared, and discussed,
> >>>>> as a potential future OSGeo project.
> >>>>>
> >>>>> OSGeo supports the needs of the open source geospatial community [6].
> >>>>>
> >>>>> Given
> >>>>>
> >>>>>  * the bold claims and continuing confusion about whether,
> >>>>>    and which, rasdaman is scalable,
> >>>>>  * the need for OSGeo to give good advice to prospective users
> >>>>>    about technologies that do scale EO data analysis,
> >>>>>  * the current (unfilled!) needs of scientists for good, open source
> >>>>>    software for such analysis, and
> >>>>>  * the potential conflict of interest of its creator [7],
> >>>>>
> >>>>> I wonder wether OSGeo should recommend rasdaman CE to the open
> >>>>> source geospatial community.
> >>>>>
> >>>>>
> >>>>> [1] http://rasdaman.org/wiki/Features
> >>>>> [2]
> >>>>>https://lists.osgeo.org/pipermail/incubator/2014-October/002540.html
> >>>>> [3]
> >>>>>https://groups.google.com/forum/#!topic/rasdaman-users/66XL3tmDDQI
> >>>>> [4]
> >>>>>
> >>>>>
> https://lists.osgeo.org/pipermail/discuss/attachments/20160515/49200cd
> >>>>>4/attachment.jpg
> >>>>> [5] https://lists.osgeo.org/pipermail/discuss/2016-May/016099.html
> >>>>> [6] http://www.osgeo.org/content/faq/foundation_faq.html
> >>>>> [7] https://lists.osgeo.org/pipermail/discuss/2016-May/016045.html
> >>>>>
> >>>>>
> >>>>> _______________________________________________
> >>>>> Discuss mailing list
> >>>>> Discuss at lists.osgeo.org
> >>>>> http://lists.osgeo.org/mailman/listinfo/discuss
> >>>>
> >>>> --
> >>>> Dr. Peter Baumann
> >>>>  - Professor of Computer Science, Jacobs University Bremen
> >>>>    www.faculty.jacobs-university.de/pbaumann
> >>>>    mail: p.baumann at jacobs-university.de
> >>>>    tel: +49-421-200-3178, fax: +49-421-200-493178
> >>>>  - Executive Director, rasdaman GmbH Bremen (HRB 26793)
> >>>>    www.rasdaman.com, mail: baumann at rasdaman.com
> >>>>    tel: 0800-rasdaman, fax: 0800-rasdafax, mobile: +49-173-5837882
> >>>> "Si forte in alienas manus oberraverit hec peregrina epistola
> >>>>incertis ventis dimissa, sed Deo commendata, precamur ut ei reddatur
> >>>>cui soli destinata, nec preripiat quisquam non sibi parata." (mail
> >>>>disclaimer, AD 1083)
> >>>>
> >>>>
> >>>
> >>> --
> >>> Dr. Peter Baumann
> >>>  - Professor of Computer Science, Jacobs University Bremen
> >>>    www.faculty.jacobs-university.de/pbaumann
> >>>    mail: p.baumann at jacobs-university.de
> >>>    tel: +49-421-200-3178, fax: +49-421-200-493178
> >>>  - Executive Director, rasdaman GmbH Bremen (HRB 26793)
> >>>    www.rasdaman.com, mail: baumann at rasdaman.com
> >>>    tel: 0800-rasdaman, fax: 0800-rasdafax, mobile: +49-173-5837882
> >>> "Si forte in alienas manus oberraverit hec peregrina epistola incertis
> >>>ventis dimissa, sed Deo commendata, precamur ut ei reddatur cui soli
> >>>destinata, nec preripiat quisquam non sibi parata." (mail disclaimer,
> >>>AD 1083)
> >>>
> >>>
> >>>
> >>>
> >>> _______________________________________________
> >>> Discuss mailing list
> >>> Discuss at lists.osgeo.org
> >>> http://lists.osgeo.org/mailman/listinfo/discuss
> >>
> >> --
> >> Cameron Shorter,
> >> Software and Data Solutions Manager
> >> LISAsoft
> >> Suite 112, Jones Bay Wharf,
> >> 26 - 32 Pirrama Rd, Pyrmont NSW 2009
> >>
> >> P +61 2 9009 5000,  W www.lisasoft.com,  F +61 2 9009 5099
> >>
> >
> >--
> >Edzer Pebesma
> >Institute for Geoinformatics  (ifgi),  University of Münster
> >Heisenbergstraße 2, 48149 Münster, Germany; +49 251 83 33081
> >Journal of Statistical Software:   http://www.jstatsoft.org/
> >Computers & Geosciences:   http://elsevier.com/locate/cageo/
> >Spatial Statistics Society http://www.spatialstatistics.info
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/discuss/attachments/20160523/99d859b3/attachment-0002.html>