[pycsw-devel] PyCSW Harvesting.

Tom Kralidis tomkralidis at gmail.com
Thu Nov 12 15:34:41 PST 2020 

Hi Brian: this is a QGIS issue and better tabled with the QGIS community.

..Tom

On Thu, Nov 12, 2020 at 6:32 PM BRIAN STIFFENN LUNA BOLIVAR
<bslunab at correo.udistrital.edu.co> wrote:
>
> I have to report a problem but I don't know if this happens because of pyCSW or qgis:
>
> When I use the MetaSearch plugin, appears my layers from WMS, but when I try to load a layer via WMS call, appears this error:
>
> AttributeError: 'NoneType' object has no attribute 'addRasterLayer'
>
> Traceback (most recent call last):
>   File "C:/OSGEO4~1/apps/qgis-ltr/./python/plugins\MetaSearch\dialogs\maindialog.py", line 757, in add_to_ows
>     ows_provider.addRasterLayer.connect(self.iface.addRasterLayer)
> AttributeError: 'NoneType' object has no attribute 'addRasterLayer'
>
>
> Versión de Python: 3.7.0 (v3.7.0:1bf9cc5093, Jun 27 2018, 04:59:51) [MSC v.1914 64 bit (AMD64)]
> Versión de QGIS: 3.10.6-A Coruña A Coruña, ec80021f49
>
> Ruta de Python:
>
> C:/OSGEO4~1/apps/qgis-ltr/./python
> C:/Users/Brian Luna/AppData/Roaming/QGIS/QGIS3\profiles\default/python
> C:/Users/Brian Luna/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins
> C:/OSGEO4~1/apps/qgis-ltr/./python/plugins
> C:\OSGEO4~1\apps\Python37
> C:\OSGEO4~1\apps\Python37\Scripts
> C:\OSGeo4W64\bin\python37.zip
> C:\OSGEO4~1\apps\Python37\DLLs
> C:\OSGEO4~1\apps\Python37\lib
> C:\OSGeo4W64\bin
> C:\OSGEO4~1\apps\Python37\lib\site-packages
> C:\OSGEO4~1\apps\Python37\lib\site-packages\win32
> C:\OSGEO4~1\apps\Python37\lib\site-packages\win32\lib
> C:\OSGEO4~1\apps\Python37\lib\site-packages\Pythonwin
> C:/Users/Brian Luna/AppData/Roaming/QGIS/QGIS3\profiles\default/python
>
> Does anyone know the propertly way to solve it?
>
>
> El jue., 12 nov. 2020 a las 18:25, BRIAN STIFFENN LUNA BOLIVAR (<bslunab at correo.udistrital.edu.co>) escribió:
>>
>> Hi Tom, sorry for the late response, I've been really busy, but I have some really good news:
>>
>> Last week I applied your recommendation, so I fellow this to configure correctly the harvest:
>>
>> 1. Change my file post.xml and now look like this:
>>
>> <?xml version="1.0" encoding="UTF-8"?>
>> <Harvest xmlns="http://www.opengis.net/cat/csw/2.0.2"
>> xmlns:ogc="http://www.opengis.net/ogc"
>> xmlns:gmd="http://www.isotc211.org/2005/gmd"
>> xmlns:ows="http://www.opengis.net/ows"
>> xmlns:xsd="http://www.w3.org/2001/XMLSchema"
>> xmlns:dc="http://purl.org/dc/elements/1.1/"
>> xmlns:dct="http://purl.org/dc/terms/"
>> xmlns:gml="http://www.opengis.net/gml"
>> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>> xsi:schemaLocation="http://www.opengis.net/cat/csw/2.0.2
>> http://schemas.opengis.net/csw/2.0.2/CSW-publication.xsd"
>> service="CSW" version="2.0.2">
>>   <Source>http://127.0.0.1/cgi-bin/mapserv.exe?map=C:/ms4w/DATA/geoserviciosgeiper.map</Source>
>>   <ResourceType>http://www.opengis.net/wms</ResourceType>
>>   <ResourceFormat>application/xml</ResourceFormat>
>> </Harvest>
>>
>> (I change http://localhost/... to  http://127.0.0.1/)
>>
>> 2. Run the following code: python bin/pycsw-admin.py -c post_xml -u http://127.0.0.1/pycsw/csw.py -x C:/ms4w/DATA/post.xml
>>
>> - Harvest works, at the end show:
>> ...</csw:HarvestResponse>'
>> Done
>>
>> 3. Then execute the following: pycsw-admin.py -c load_records -p /path/to/records -f default.cfg -y
>>
>> 4. Prove the csw in QGIS with MetaSearch and here appear my layers, so it works PERFECTLY! Thank you so much for your help.
>>
>> Now I have a question: Does pyCSW generate an html document with the layers and information in? If not, how can I do this automatically just with the GetCapabilities response?
>>
>> Regards,
>>
>>
>> El mié., 14 oct. 2020 a las 5:54, Tom Kralidis (<tomkralidis at gmail.com>) escribió:
>>>
>>> Hi Brian: it looks like the "&" characters in the <Source> element are
>>> causing XML parsing issues.  They are also
>>> not required in the context of a CSW Harvest request.  Can you update
>>> your XML request with just the base URL
>>> in <Source>?  The request parameters are autogenerated by pycsw accordingly:
>>>
>>> <?xml version="1.0" encoding="UTF-8"?>
>>> <Harvest xmlns="http://www.opengis.net/cat/csw/2.0.2"
>>> xmlns:ogc="http://www.opengis.net/ogc"
>>> xmlns:gmd="http://www.isotc211.org/2005/gmd"
>>> xmlns:ows="http://www.opengis.net/ows"
>>> xmlns:xsd="http://www.w3.org/2001/XMLSchema"
>>> xmlns:dc="http://purl.org/dc/elements/1.1/"
>>> xmlns:dct="http://purl.org/dc/terms/"
>>> xmlns:gml="http://www.opengis.net/gml"
>>> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>>> xsi:schemaLocation="http://www.opengis.net/cat/csw/2.0.2
>>> http://schemas.opengis.net/csw/2.0.2/CSW-publication.xsd"
>>> service="CSW" version="2.0.2">
>>>   <Source>http://localhost/cgi-bin/mapserv.exe?map=C:/ms4w/DATA/geoserviciosgeiper.map</Source>
>>>   <ResourceType>http://www.opengis.net/wms</ResourceType>
>>>   <ResourceFormat>application/xml</ResourceFormat>
>>> </Harvest>
>>>
>>> Are you able to try again and report back?
>>>
>>> Thanks
>>>
>>> ..Tom
>>>
>>> On Tue, Oct 13, 2020 at 10:04 PM BRIAN STIFFENN LUNA BOLIVAR
>>> <bslunab at correo.udistrital.edu.co> wrote:
>>> >
>>> > Thanks a lot for your response Tom, so I have a question, should I follow this steps to harvest a wms service:
>>> >
>>> > 1. Modify the following XML request from my WMS service: https://github.com/geopython/pycsw/blob/master/tests/functionaltests/suites/harvesting/post/Harvest-wms-run1.xmlv (I named: post.xml and save it in c:\geiper\post\post.xml)
>>> > 2. Set in my default.cfg transactions=true
>>> > 3. When it's done, I should execute this command line:
>>> >
>>> > python bin/pycsw-admin.py -c post_xml -u http://localhost/pycsw/csw.py -x c:\geiper\post\post.xml
>>> >
>>> > 4. Refresh the harvested records with the following command line:
>>> >
>>> > python bin/pycsw-admin.py  -c refresh_harvested_records -f default.cfg
>>> >
>>> > Those were all the steps I must follow to harvest records from WMS?
>>> >
>>> > If yes, I get the following error:
>>> >
>>> > b'<?xml version="1.0" encoding="UTF-8" standalone="no"?>\n<!-- pycsw 2.4.2 -->\n<ows:ExceptionReport xmlns:csw="http://www.opengis.net/cat/csw/2.0.2" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:dct="http://purl.org/dc/terms/" xmlns:gmd="http://www.isotc211.org/2005/gmd" xmlns:gml="http://www.opengis.net/gml" xmlns:ows="http://www.opengis.net/ows" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" version="1.2.0" language="en-US" xsi:schemaLocation="http://www.opengis.net/ows http://schemas.opengis.net/ows/1.0.0/owsExceptionReport.xsd"><ows:Exception exceptionCode="NoApplicableCode" locator="service"><ows:ExceptionText>Exception: document not well-formed.\nError: EntityRef: expecting \';\', line 3, column 95 (<string>, line 3).</ows:ExceptionText></ows:Exception></ows:ExceptionReport>'
>>> >
>>> > And My post.xml is formed like this:
>>> >
>>> > <?xml version="1.0" encoding="UTF-8"?>
>>> > <Harvest xmlns="http://www.opengis.net/cat/csw/2.0.2" xmlns:ogc="http://www.opengis.net/ogc" xmlns:gmd="http://www.isotc211.org/2005/gmd" xmlns:ows="http://www.opengis.net/ows" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:dct="http://purl.org/dc/terms/" xmlns:gml="http://www.opengis.net/gml" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.opengis.net/cat/csw/2.0.2 http://schemas.opengis.net/csw/2.0.2/CSW-publication.xsd" service="CSW" version="2.0.2">
>>> >
>>> >   <Source>http://localhost/cgi-bin/mapserv.exe?map=C:/ms4w/DATA/geoserviciosgeiper.map&SERVICE=WMS&VERSION=1.3.0</Source>
>>> >
>>> >   <ResourceType>http://www.opengis.net/wms</ResourceType>
>>> >   <ResourceFormat>application/xml</ResourceFormat>
>>> > </Harvest>
>>> >
>>> > Thank you so much for your help.
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>> > El mar., 13 oct. 2020 a las 19:28, Tom Kralidis (<tomkralidis at gmail.com>) escribió:
>>> >>
>>> >> Hi Brian: thanks for the info.  Comments interleaved:
>>> >>
>>> >> On Tue, Oct 13, 2020 at 4:13 PM BRIAN STIFFENN LUNA BOLIVAR
>>> >> <bslunab at correo.udistrital.edu.co> wrote:
>>> >> >
>>> >> > Hello everyone, good day. I have been trying for over a month to install and run pycsw, but I have not been fortunate enough to do it properly for my college degree work.
>>> >> >
>>> >>
>>> >> What kind of installation problems are you having?
>>> >>
>>> >> > I currently have a WMS service in place which I need to run Harvest operation and be able to update the GetCapabilities of the CSW however, I have not found the right way to do it.
>>> >> >
>>> >> > Could someone please provide me with a methodological guide to be able to run the Harvest operation to a WMS?
>>> >> >
>>> >>
>>> >> You can run a CSW-T Harvest XML POST request with the following XML request:
>>> >>
>>> >> https://github.com/geopython/pycsw/blob/master/tests/functionaltests/suites/harvesting/post/Harvest-wms-run1.xml
>>> >>
>>> >> Here, you would replace the <Source> element content with your WMS
>>> >> base URL.  To invoke XML POST requests, see the following
>>> >> FAQ item: https://pycsw.org/faq/#how-can-i-make-csw-post-xml-requests
>>> >>
>>> >> Hope this helps.
>>> >>
>>> >> ..Tom
>>> >>
>>> >> > Q: I'm currently using Mapserver 7.7.0-dev in windows.
>>> >> >
>>> >> > Regards.
>>> >> >

More information about the pycsw-devel mailing list