[gdal-dev] Installing test databases/servers online for the GDAL project

[Tamas Szekeres]

Frank, Even, Ivan,

Thank you all for your comments,

It great to hear that most of these are available for free, so we
'only' have to deal with setting them up. Yes, a 'gdaltest' VM image
would be helpful, we could easily set up and run all of those
databases locally on each buildbot machine so we wouldn't have to
concern about the security very much.

However when looking into the scripts I have further technical issues
that should also be addressed. Some of the tests use fixed dbnames on
the localhost (like for postgis, mysql and ingres) therefore we cannot
install the databases in separate machines/VM images. Moreover
currently all the windows slaves are hosted on a single machine and we
should use at least separate database names for each of the slaves in
order to avoid the interferece between them.

The ogr_oci tests use the OCI_DSNAME environment variable which is
much better because I can set the environment for each instance
separately. The only drawback of this is that the buildbot dumps the
environment variables of the host machine to the web, so we cannot
store passwords in these variables safely. I'd prefer if the tests
were load the connection strings from external files instead.

With regards to the OGR SDE tests all of the credentials are contained
by the script with a fixed server IP address, therefore I wouldn't
have the chance to set up a proper environment even if I had an SDE
instance locally.

The lack of an available SDE instance is a big problem of some of the
developers. Would it be expensive to purchase a valid license for
GDAL/OsGeo? Do they have a developer like construction for testing
purposes?


Best regards,

Tamas



2008/10/28 Frank Warmerdam <warmerdam at pobox.com>:
> Tamas Szekeres wrote:
>>
>> Folks,
>>
>> I'm not totally sure whether this topic has already been discussed or
>> not, however I'd like to see your opinion about.
>>
>> I've just worked on adding further drivers to Windows buildbot slaves,
>> as I consider it would be reasonable to have them regularly tested
>> especialy during the beta/release period of the GDAL project. I've
>> noticed that many of the tests rely on the existence of some external
>> servers like for mysql, oci, sde, pgsql, wcs. Currently none of these
>> servers are available online therefore those tests are skipped in the
>> testing process.
>>
>> I'd wonder if we could somehow get those servers online, however I'm
>> not sure who is responsible to maintain the corresponding databases.
>> Assuming that there's no explicit owner of these servers would it be
>> reasonable to establish such test servers for the GDAL project
>> permanently?
>
> Tamas,
>
> I am interested in having as many of the databases enabled in at least
> one of the buildbot slave instances.  Possibly Ari and I can get some of
> them working on buildtest.osgeo.org as he migrates some of our GDAL
> build slaves there.
>
> I will note that SDE is pretty involved and non-free.  Currently I'm hoping
> that Howard will be able to test SDE reasonably frequently.
>
> As Ivan mentioned, it is permitted to install the full Oracle software
> without cost as long as it is only being used for development, not
> production.  So I'd like to get an oracle instance running somewhere
> reasonably centrally accessable for testing and development.  However, the
> operating system on buildtest.osgeo.org seems to be too old/odd to
> accomplish this.  I tried and failed.  There has been some discussion of
> OSGeo purchasing a server beefy enough to run several VMs.  It might be
> practical to have one of those VMs used for testing and include an oracle
> instance.
>
> There is no reason, other than time, that we can't get mysql, postgres,
> and ingres working on buildtest.osgeo.org.   I'd also like to get a
> dependable WCS and WMS instance going somewhere.  I've setup my own
> WCS test instance in the past, but the system ended up dying
> (geodata.telascience.org) and I never got around to setting it up again.
> Once again, just a matter of someone willing to do the work and document
> it.
>
> I'm a bit leery about providing open database services on the internet.
> One concern is that it may make for very slow automated testing for
> those with significant network latency.  My second concern is security.
> I have no idea how dangerous a publically accessable mysql or postgres
> instance is.  But I'm not keen on leaving a door wide open to crackers.
> This might be more practical if we could provide it in a VM that we
> could just restart from a clean image when needed.
>
> Best regards,
> --
> ---------------------------------------+--------------------------------------
> I set the clouds in motion - turn up   | Frank Warmerdam,
> warmerdam at pobox.com
> light and sound - activate the windows | http://pobox.com/~warmerdam
> and watch the world go round - Rush    | Geospatial Programmer for Rent
>
>