[mapserver-dev] the state of msautotest

[Tamas Szekeres]

Mateusz Loskot has already set up a buildbot configuration at osgeo
according to:
http://wiki.osgeo.org/wiki/BuildBot_Configuration

Mapserver buildslaves has been available at
http://buildbot.osgeo.org/mapserver

However this config seems to be offline at the moment.

Best regards,

Tamas




2012/9/12 Alan Boudreault <aboudreault at mapgears.com>

> I totally agree with you guys. msautotests is in a critical state. :P
>
> I'm aware of 3 tools that seem to be popular these days:
>
> - Buildbot: self-hosted solution in python.
>
> - Jenkins: self-hosted. More complete than buildbot. Nice solution for
> continuous integration and regression tests.
>
> - Travis-CI: web service, current trend with opensource softwares. Seems
> to work very well.  Nice Integration with github projects. (each commit are
> tested)
>
> Unfortunately, I don't know much about them at the moment. The easier for
> sure would be to test Travis-CI and see how it works.
>
> Alan
>
>
> On 12-09-12 07:14 AM, thomas bonfort wrote:
>
>> After maybe a couple beers too many with Stephan the other day we came
>> to talk about mapserver's autotests and it's current deficiencies with
>> respect to how we are using them.
>>
>> To summarize, here are the principle issues that arose:
>>
>> - The tests aren't run on each commit, which is understandable as it
>> takes an additional effort to do so
>> - We have a large number of failing tests, which can be attributed to:
>>    - the afore-mentioned reason: is a failing test a result of the last
>> commit or was it already failing beforehand?
>>    - image comparison tests can produce false positives, due to
>> different cpu architectures and underlying dependency libraries
>> (namely freetype)
>>    - image and xml/gml tests can diverge depending on the compile-time
>> configuration options that were chosen.
>> - we've ended up in a chicken vs. egg situation where tests aren't
>> being run because they are failing, and test fail in the long run
>> because they are never being used/updated.
>>
>> To alleviate this, we'd like to set the ball rolling on these ideas:
>> - provide a "reference" platform with a kitchen-sink (i.e. compiled
>> with all options) mapserver instance. This could be an osgeo ad-hoc vm
>> or a dedicated server provided by one of us or our users. (second
>> option might be better given the limited resources available at osgeo)
>> - go through all the current tests, make sure they pass on this
>> reference platform, and update the expected result if not.
>> - automatically run the whole test-suite on each commit, and provide a
>> web-page or email alert when a test has failed. Does anyone have
>> experience with a software solution that could provide that to us?
>>
>> Thoughts? It's certainly not a miracle solution as it will require
>> acting upon when a test starts failing, however we believe that
>> automating the runs and removing the false positives will be a nice
>> step forward (and give more peace-of-mind to the release-manager when
>> packaging a release :) )
>>
>> cheers,
>> thomas
>> ______________________________**_________________
>> mapserver-dev mailing list
>> mapserver-dev at lists.osgeo.org
>> http://lists.osgeo.org/**mailman/listinfo/mapserver-dev<http://lists.osgeo.org/mailman/listinfo/mapserver-dev>
>>
>>
>
> --
> Alan Boudreault
> http://www.mapgears.com/
>
> ______________________________**_________________
> mapserver-dev mailing list
> mapserver-dev at lists.osgeo.org
> http://lists.osgeo.org/**mailman/listinfo/mapserver-dev<http://lists.osgeo.org/mailman/listinfo/mapserver-dev>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/mapserver-dev/attachments/20120912/845cd875/attachment-0001.html>