[pycsw-devel] PyCSW Harvesting.

Thu Nov 12 15:34:11 PST 2020

Hi Brian: good to hear.  pycsw does not provide HTML output.  You would have
to write a script or XSLT to transform the XML into HTML.

..Tom

On Thu, Nov 12, 2020 at 6:26 PM BRIAN STIFFENN LUNA BOLIVAR
<bslunab at correo.udistrital.edu.co> wrote:
>
> Hi Tom, sorry for the late response, I've been really busy, but I have some really good news:
>
> Last week I applied your recommendation, so I fellow this to configure correctly the harvest:
>
> 1. Change my file post.xml and now look like this:
>
> <?xml version="1.0" encoding="UTF-8"?>
> <Harvest xmlns="http://www.opengis.net/cat/csw/2.0.2"
> xmlns:ogc="http://www.opengis.net/ogc"
> xmlns:gmd="http://www.isotc211.org/2005/gmd"
> xmlns:ows="http://www.opengis.net/ows"
> xmlns:xsd="http://www.w3.org/2001/XMLSchema"
> xmlns:dc="http://purl.org/dc/elements/1.1/"
> xmlns:dct="http://purl.org/dc/terms/"
> xmlns:gml="http://www.opengis.net/gml"
> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> xsi:schemaLocation="http://www.opengis.net/cat/csw/2.0.2
> http://schemas.opengis.net/csw/2.0.2/CSW-publication.xsd"
> service="CSW" version="2.0.2">
>   <Source>http://127.0.0.1/cgi-bin/mapserv.exe?map=C:/ms4w/DATA/geoserviciosgeiper.map</Source>
>   <ResourceType>http://www.opengis.net/wms</ResourceType>
>   <ResourceFormat>application/xml</ResourceFormat>
> </Harvest>
>
> (I change http://localhost/... to  http://127.0.0.1/)
>
> 2. Run the following code: python bin/pycsw-admin.py -c post_xml -u http://127.0.0.1/pycsw/csw.py -x C:/ms4w/DATA/post.xml
>
> - Harvest works, at the end show:
> ...</csw:HarvestResponse>'
> Done
>
> 3. Then execute the following: pycsw-admin.py -c load_records -p /path/to/records -f default.cfg -y
>
> 4. Prove the csw in QGIS with MetaSearch and here appear my layers, so it works PERFECTLY! Thank you so much for your help.
>
> Now I have a question: Does pyCSW generate an html document with the layers and information in? If not, how can I do this automatically just with the GetCapabilities response?
>
> Regards,
>
>
> El mié., 14 oct. 2020 a las 5:54, Tom Kralidis (<tomkralidis at gmail.com>) escribió:
>>
>> Hi Brian: it looks like the "&" characters in the <Source> element are
>> causing XML parsing issues.  They are also
>> not required in the context of a CSW Harvest request.  Can you update
>> your XML request with just the base URL
>> in <Source>?  The request parameters are autogenerated by pycsw accordingly:
>>
>> <?xml version="1.0" encoding="UTF-8"?>
>> <Harvest xmlns="http://www.opengis.net/cat/csw/2.0.2"
>> xmlns:ogc="http://www.opengis.net/ogc"
>> xmlns:gmd="http://www.isotc211.org/2005/gmd"
>> xmlns:ows="http://www.opengis.net/ows"
>> xmlns:xsd="http://www.w3.org/2001/XMLSchema"
>> xmlns:dc="http://purl.org/dc/elements/1.1/"
>> xmlns:dct="http://purl.org/dc/terms/"
>> xmlns:gml="http://www.opengis.net/gml"
>> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>> xsi:schemaLocation="http://www.opengis.net/cat/csw/2.0.2
>> http://schemas.opengis.net/csw/2.0.2/CSW-publication.xsd"
>> service="CSW" version="2.0.2">
>>   <Source>http://localhost/cgi-bin/mapserv.exe?map=C:/ms4w/DATA/geoserviciosgeiper.map</Source>
>>   <ResourceType>http://www.opengis.net/wms</ResourceType>
>>   <ResourceFormat>application/xml</ResourceFormat>
>> </Harvest>
>>
>> Are you able to try again and report back?
>>
>> Thanks
>>
>> ..Tom
>>
>> On Tue, Oct 13, 2020 at 10:04 PM BRIAN STIFFENN LUNA BOLIVAR
>> <bslunab at correo.udistrital.edu.co> wrote:
>> >
>> > Thanks a lot for your response Tom, so I have a question, should I follow this steps to harvest a wms service:
>> >
>> > 1. Modify the following XML request from my WMS service: https://github.com/geopython/pycsw/blob/master/tests/functionaltests/suites/harvesting/post/Harvest-wms-run1.xmlv (I named: post.xml and save it in c:\geiper\post\post.xml)
>> > 2. Set in my default.cfg transactions=true
>> > 3. When it's done, I should execute this command line:
>> >
>> > python bin/pycsw-admin.py -c post_xml -u http://localhost/pycsw/csw.py -x c:\geiper\post\post.xml
>> >
>> > 4. Refresh the harvested records with the following command line:
>> >
>> > python bin/pycsw-admin.py  -c refresh_harvested_records -f default.cfg
>> >
>> > Those were all the steps I must follow to harvest records from WMS?
>> >
>> > If yes, I get the following error:
>> >
>> > b'<?xml version="1.0" encoding="UTF-8" standalone="no"?>\n<!-- pycsw 2.4.2 -->\n<ows:ExceptionReport xmlns:csw="http://www.opengis.net/cat/csw/2.0.2" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:dct="http://purl.org/dc/terms/" xmlns:gmd="http://www.isotc211.org/2005/gmd" xmlns:gml="http://www.opengis.net/gml" xmlns:ows="http://www.opengis.net/ows" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" version="1.2.0" language="en-US" xsi:schemaLocation="http://www.opengis.net/ows http://schemas.opengis.net/ows/1.0.0/owsExceptionReport.xsd"><ows:Exception exceptionCode="NoApplicableCode" locator="service"><ows:ExceptionText>Exception: document not well-formed.\nError: EntityRef: expecting \';\', line 3, column 95 (<string>, line 3).</ows:ExceptionText></ows:Exception></ows:ExceptionReport>'
>> >
>> > And My post.xml is formed like this:
>> >
>> > <?xml version="1.0" encoding="UTF-8"?>
>> > <Harvest xmlns="http://www.opengis.net/cat/csw/2.0.2" xmlns:ogc="http://www.opengis.net/ogc" xmlns:gmd="http://www.isotc211.org/2005/gmd" xmlns:ows="http://www.opengis.net/ows" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:dct="http://purl.org/dc/terms/" xmlns:gml="http://www.opengis.net/gml" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.opengis.net/cat/csw/2.0.2 http://schemas.opengis.net/csw/2.0.2/CSW-publication.xsd" service="CSW" version="2.0.2">
>> >
>> >   <Source>http://localhost/cgi-bin/mapserv.exe?map=C:/ms4w/DATA/geoserviciosgeiper.map&SERVICE=WMS&VERSION=1.3.0</Source>
>> >
>> >   <ResourceType>http://www.opengis.net/wms</ResourceType>
>> >   <ResourceFormat>application/xml</ResourceFormat>
>> > </Harvest>
>> >
>> > Thank you so much for your help.
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> > El mar., 13 oct. 2020 a las 19:28, Tom Kralidis (<tomkralidis at gmail.com>) escribió:
>> >>
>> >> Hi Brian: thanks for the info.  Comments interleaved:
>> >>
>> >> On Tue, Oct 13, 2020 at 4:13 PM BRIAN STIFFENN LUNA BOLIVAR
>> >> <bslunab at correo.udistrital.edu.co> wrote:
>> >> >
>> >> > Hello everyone, good day. I have been trying for over a month to install and run pycsw, but I have not been fortunate enough to do it properly for my college degree work.
>> >> >
>> >>
>> >> What kind of installation problems are you having?
>> >>
>> >> > I currently have a WMS service in place which I need to run Harvest operation and be able to update the GetCapabilities of the CSW however, I have not found the right way to do it.
>> >> >
>> >> > Could someone please provide me with a methodological guide to be able to run the Harvest operation to a WMS?
>> >> >
>> >>
>> >> You can run a CSW-T Harvest XML POST request with the following XML request:
>> >>
>> >> https://github.com/geopython/pycsw/blob/master/tests/functionaltests/suites/harvesting/post/Harvest-wms-run1.xml
>> >>
>> >> Here, you would replace the <Source> element content with your WMS
>> >> base URL.  To invoke XML POST requests, see the following
>> >> FAQ item: https://pycsw.org/faq/#how-can-i-make-csw-post-xml-requests
>> >>
>> >> Hope this helps.
>> >>
>> >> ..Tom
>> >>
>> >> > Q: I'm currently using Mapserver 7.7.0-dev in windows.
>> >> >
>> >> > Regards.
>> >> >