[pycsw-devel] MS4W installer containing pycsw-dev for Windows users

[Jeff McKenna]

Thanks for your support on this Tom, comments inline below:


On 2019-01-19 3:27 PM, Tom Kralidis wrote:
> On Wed, Jan 16, 2019 at 1:48 PM Jeff McKenna
> <jmckenna at gatewaygeomatics.com> wrote:
>>
>> Hi pycsw community!
>>
>> Short summary
>> -------------
>>
>> Windows users might be interested that MS4W (the popular installer for
>> MapServer https://ms4w.com) is almost ready to be released, with pycsw
>> running (!!!).  It will be named "MS4W 4.0", and there is now an alpha
>> package containing pycsw ready to test at
>> https://ms4w.com/release/experimental/ms4w_4.0.0_alpha19.zip  (not
>> familiar with running MS4W or how to install it?  You can wait for the
>> setup.exe installer which will come shortly and install pycsw, Apache,
>> for you)
>>
> 
> Wow this is great news/work Jeff!  Thanks for this valuable addition
> to ms4w.

Thanks!!!

> 
>> Longer explanation
>> ------------------
>>
>> I've been working very hard on a big MS4W release, containing PHP 7 and
>> Python, Java, CSharp mapscripts, and also its own included Python
>> instance (hundreds of libraries in total, all compiled from source with
>> Visual Studio 2017).  I've also now included pycsw (master), running on
>> Python 3.7.0  (some of you devs are definitely cringing already I bet,
>> seeing that version, but please hear me out)
>>
>> Note that such an installer, for PHP7 and Python, GDAL, MapServer,
>> Apache, mod_wsgi (pycsw is running through mod_wsgi) demands a recent
>> compiler that can handle building all of that together, which is why
>> Visual Studio 2017 was chosen.
>>
>> Of course, to install pycsw I realize the hardcoded requirements (in
>> requirements.txt) for lxml, pyproj, Shapely, but to make all of this
>> work in the new 2017 compiler, I've used newer library versions for
>> those (lxml 4.3.0, pyproj 1.9.6, Shapely 1.6.4.post2).
>>
> 
> Are the newer versions the latest stable versions?  I'm also wondering
> whether we should review our current requirements/dependencies (it's
> been awhile) and whether we should add 3.7 to our Travis CI setup.

Yes those are the latest stable versions for lxml, pyproj, Shapely.  It 
would be great if you could add those (and Python 3.7) to your TravisCI 
setup.

> 
>> The only change I had to make was to use "WKTReadingError" in
>> pycsw/core/repository.py (line#41), because I am using a much newer
>> version of Shapely and GEOS.  For the record, I couldn't find anyone
>> else reporting that change.
>>
> 
> I've filed an issue in https://github.com/geopython/pycsw/issues/584)
> to safeguard
> across Shapely versions.

Great, I have expanded the explanation there (with exact error and 
changes made).

> 
>> I guess my question is that if anyone sees any show stoppers for using
>> these updated library versions.  I know, it is not recommended, but, out
>> of this is an actual Windows installer for the many Windows users (MS4W
>> gets between 4,000 to 6,000 installs a month, I monitor it closely).
>>
> 
> IHMO I don't see any issues.  If we do find issues we are committed to
> fixing them accordingly.

Ok thanks, this is good to know.

> 
>> The Alpha now includes a running catalogue service with just the SQLite
>> cite.db records, but I am sure we can expand that later (note that I
>> discovered that most if not all records as part of the 'gisdata' package
>> are outdated/broken, which you can see for yourself in the pycsw viewer
>> at http://demo.pycsw.org/viewer/ (that viewer leverages the 'gisdata'
>> package for its records)
>>
> 
> The view also has a 'service' endpoint which has a selected set of
> harvested OGC web services.  I've filed an issue in
> https://github.com/geopython/demo.pycsw.org/issues/2
> to update the endpoints on demo.pycsw.org

Ok thanks.  I think having some working sample services is very useful 
to new users.

> 
>> It would be great to have a lighter-weight viewer to demonstrate the
>> power of pycsw for MS4W users (I realize pycsw is headless, but it is
>> nice for new users to see it in action).  For sure I will push MS4W
>> users to test their CSW services with the QGIS MetaSearch plugin (great
>> work thanks!).
>>
> 
> Can we start with the viewer on demo.pycsw.org? We can definitely
> implement updates to enhance its useability.

I honestly originally had more written here about the existing viewer, 
but deleted it.

The issue is how heavy it is (requires GeoExt, OL, leaflet, gxp, ...).

But, 'beggars cannot be choosers' so I deleted my email text on it.

It's about 200MB, and I was not sure to include it inside MS4W, without 
confirmation from pycsw devs that the source of demo.pycsw.org will be 
actively maintained (it hasn't been touched in several years, and 
leverages broken data services).  So you can see my concern as a packager.

I didn't want to appear here as complaining, so I deleted my email text 
before.  But it is just about my concern as a packager (of MS4W): should 
the demo be distributed (even though it is based on old software 
versions, many old software packages, and is quite heavy/lots of 
packages and space required, and points to broken data).  As a packager, 
it is a lot to commit to, before adding it to MS4W.

I feel that more important is having about 10 working catalogue services 
in a maintained endpoint, that is easy for new pycsw users to get (how 
about inside the pycsw source at /pycsw/data/sample.db ?), so then new 
pycsw users can goto something like the QGIS metasearch plugin and 
access their new pycsw service easily.

Thoughts?

> 
>> Well, I hope this news is accepted well, even if it pushes the limits of
>> 'recommended/required' software versions.  I hope this big effort helps
>> the whole pycsw community.
>>
> 
> This is greatly appreciated, thank you Jeff!
> 

Awesome.  I think it is a win-win for all!!  Thank you pycsw team!

-jeff