[pycsw-devel] MS4W installer containing pycsw-dev for Windows users

[Tom Kralidis]

Hi Jeff: comments interleaved:
On Mon, Jan 21, 2019 at 10:43 AM Jeff McKenna
<jmckenna at gatewaygeomatics.com> wrote:
>
> Thanks for your support on this Tom, comments inline below:
>
>
> On 2019-01-19 3:27 PM, Tom Kralidis wrote:
> > On Wed, Jan 16, 2019 at 1:48 PM Jeff McKenna
> > <jmckenna at gatewaygeomatics.com> wrote:
> >>
> >> Hi pycsw community!
> >>
> >> Short summary
> >> -------------
> >>
> >> Windows users might be interested that MS4W (the popular installer for
> >> MapServer https://ms4w.com) is almost ready to be released, with pycsw
> >> running (!!!).  It will be named "MS4W 4.0", and there is now an alpha
> >> package containing pycsw ready to test at
> >> https://ms4w.com/release/experimental/ms4w_4.0.0_alpha19.zip  (not
> >> familiar with running MS4W or how to install it?  You can wait for the
> >> setup.exe installer which will come shortly and install pycsw, Apache,
> >> for you)
> >>
> >
> > Wow this is great news/work Jeff!  Thanks for this valuable addition
> > to ms4w.
>
> Thanks!!!
>
> >
> >> Longer explanation
> >> ------------------
> >>
> >> I've been working very hard on a big MS4W release, containing PHP 7 and
> >> Python, Java, CSharp mapscripts, and also its own included Python
> >> instance (hundreds of libraries in total, all compiled from source with
> >> Visual Studio 2017).  I've also now included pycsw (master), running on
> >> Python 3.7.0  (some of you devs are definitely cringing already I bet,
> >> seeing that version, but please hear me out)
> >>
> >> Note that such an installer, for PHP7 and Python, GDAL, MapServer,
> >> Apache, mod_wsgi (pycsw is running through mod_wsgi) demands a recent
> >> compiler that can handle building all of that together, which is why
> >> Visual Studio 2017 was chosen.
> >>
> >> Of course, to install pycsw I realize the hardcoded requirements (in
> >> requirements.txt) for lxml, pyproj, Shapely, but to make all of this
> >> work in the new 2017 compiler, I've used newer library versions for
> >> those (lxml 4.3.0, pyproj 1.9.6, Shapely 1.6.4.post2).
> >>
> >
> > Are the newer versions the latest stable versions?  I'm also wondering
> > whether we should review our current requirements/dependencies (it's
> > been awhile) and whether we should add 3.7 to our Travis CI setup.
>
> Yes those are the latest stable versions for lxml, pyproj, Shapely.  It
> would be great if you could add those (and Python 3.7) to your TravisCI
> setup.
>
> >
> >> The only change I had to make was to use "WKTReadingError" in
> >> pycsw/core/repository.py (line#41), because I am using a much newer
> >> version of Shapely and GEOS.  For the record, I couldn't find anyone
> >> else reporting that change.
> >>
> >
> > I've filed an issue in https://github.com/geopython/pycsw/issues/584)
> > to safeguard
> > across Shapely versions.
>
> Great, I have expanded the explanation there (with exact error and
> changes made).
>
> >
> >> I guess my question is that if anyone sees any show stoppers for using
> >> these updated library versions.  I know, it is not recommended, but, out
> >> of this is an actual Windows installer for the many Windows users (MS4W
> >> gets between 4,000 to 6,000 installs a month, I monitor it closely).
> >>
> >
> > IHMO I don't see any issues.  If we do find issues we are committed to
> > fixing them accordingly.
>
> Ok thanks, this is good to know.
>
> >
> >> The Alpha now includes a running catalogue service with just the SQLite
> >> cite.db records, but I am sure we can expand that later (note that I
> >> discovered that most if not all records as part of the 'gisdata' package
> >> are outdated/broken, which you can see for yourself in the pycsw viewer
> >> at http://demo.pycsw.org/viewer/ (that viewer leverages the 'gisdata'
> >> package for its records)
> >>
> >
> > The view also has a 'service' endpoint which has a selected set of
> > harvested OGC web services.  I've filed an issue in
> > https://github.com/geopython/demo.pycsw.org/issues/2
> > to update the endpoints on demo.pycsw.org
>
> Ok thanks.  I think having some working sample services is very useful
> to new users.
>
> >
> >> It would be great to have a lighter-weight viewer to demonstrate the
> >> power of pycsw for MS4W users (I realize pycsw is headless, but it is
> >> nice for new users to see it in action).  For sure I will push MS4W
> >> users to test their CSW services with the QGIS MetaSearch plugin (great
> >> work thanks!).
> >>
> >
> > Can we start with the viewer on demo.pycsw.org? We can definitely
> > implement updates to enhance its useability.
>
> I honestly originally had more written here about the existing viewer,
> but deleted it.
>
> The issue is how heavy it is (requires GeoExt, OL, leaflet, gxp, ...).
>

Note we carry 2 viewers at demo.pycsw.org:

1. https://github.com/geopython/demo.pycsw.org/blob/master/www/viewer/index.html
(Leaflet, jQuery, bootstrap)
2. https://github.com/geopython/demo.pycsw.org/blob/master/www/viewer/index2.html
(GeoExt, gxp, etc.)

Viewer 1 is a lightweight approach (we can't do much without
Leaflet/jQuery/bootstrap) and
is infact a single page of HTML/JavaScript. With the dependencies the
whole thing is around
400KB if you decide to store dependencies locally.

> But, 'beggars cannot be choosers' so I deleted my email text on it.
>
> It's about 200MB, and I was not sure to include it inside MS4W, without
> confirmation from pycsw devs that the source of demo.pycsw.org will be
> actively maintained (it hasn't been touched in several years, and
> leverages broken data services).  So you can see my concern as a packager.
>
> I didn't want to appear here as complaining, so I deleted my email text
> before.  But it is just about my concern as a packager (of MS4W): should
> the demo be distributed (even though it is based on old software
> versions, many old software packages, and is quite heavy/lots of
> packages and space required, and points to broken data).  As a packager,
> it is a lot to commit to, before adding it to MS4W.
>
> I feel that more important is having about 10 working catalogue services
> in a maintained endpoint, that is easy for new pycsw users to get (how
> about inside the pycsw source at /pycsw/data/sample.db ?), so then new
> pycsw users can goto something like the QGIS metasearch plugin and
> access their new pycsw service easily.
>

Per https://github.com/geopython/demo.pycsw.org/issues/2 can you suggest
some WMS/WFS/WCS endpoints we could use?

> Thoughts?
>
> >
> >> Well, I hope this news is accepted well, even if it pushes the limits of
> >> 'recommended/required' software versions.  I hope this big effort helps
> >> the whole pycsw community.
> >>
> >
> > This is greatly appreciated, thank you Jeff!
> >
>
> Awesome.  I think it is a win-win for all!!  Thank you pycsw team!
>
> -jeff
>
>
> _______________________________________________
> pycsw-devel mailing list
> pycsw-devel at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/pycsw-devel