[OWSLib-users] csw getrecords2 : request rejected

[Julie Pierson]

Thanks again for your quick answer James. I will try contacting them and 
in the meanwhile I'll explore the other ways you mention.

Interestingly, when I use curl with the same request I get a "400 bad 
request error" with the address "Apache Server at 
beta.schemas.opengis.net Port 80" so I'll also try looking into schemas.

Julie.


Le 02/08/2018 à 10:26, Passmore, James H. a écrit :
> Julie,
>
> It just breaks a proxy rule; it just probably means that whoever set up the CSW server didn't tell the proxy administrator to allow some requests through.  So probably you just need to contact the service provider, and tell them what requests you were trying, and get them to change the rule, after all the service GetCaps response does have `<ows:AccessConstraints>Pas de contraintes d'accès</ows:AccessConstraints>`
>
> It could be your user agent isn't allowed, it could be the size of the request, it could be a word in the URL...
>
> If you want an alternative way of gathering all the records, and as the server exposes a sitemap of all records ~
> http://www.data.eaufrance.fr/geosource/srv/fre/portal.sitemap then you could just loop though the records, construct the URL you want if different from the listed URL for example from something like http://www.data.eaufrance.fr:/geosource/?uuid=2e703ba0-08eb-4029-a9b1-a6f7f3667b62 (in the sitemap) to something like http://www.data.eaufrance.fr/geosource/srv/fre/xml.metadata.get?uuid=2e703ba0-08eb-4029-a9b1-a6f7f3667b62 (for the full ISO record).
>
> That's something I've done before, but it didn't use OWSLib.
>
> James
>
>
>
>
>
> -----Original Message-----
> From: Julie Pierson <julie.pierson at cnrs.fr>
> Sent: 02 August 2018 08:43
> To: Passmore, James H. <jpass at bgs.ac.uk>
> Cc: owslib-users at lists.osgeo.org
> Subject: Re: [OWSLib-users] csw getrecords2 : request rejected
>
> Thanks for your answer James. It is indeed a French government website.
> I understand the request by blocked, but would you know why ? Could I bypass this somehow ?
>
> Thanks
>
> Julie.
>
>
> Le 31/07/2018 à 19:43, Passmore, James H. a écrit :
>> It looks very much like the request was blocked by the server you are querying.
>>
>> I've had similar messages when querying French government web sites
>>
>> James
>>
>> -----Original Message-----
>> From: OWSLib-users <owslib-users-bounces at lists.osgeo.org> On Behalf Of
>> Julie Pierson
>> Sent: 31 July 2018 15:41
>> To: owslib-users at lists.osgeo.org
>> Subject: [OWSLib-users] csw getrecords2 : request rejected
>>
>> Hi all,
>>
>> I'm trying to get the records from this csw : http://www.data.eaufrance.fr:/geosource/srv/eng/csw?
>> Getting one record with getrecordbyid works ok, but my request is rejected when I use getrecords2 or getrecords.
>>
>> Here's a snippet of code for Python2.7 :
>> from owslib.csw import CatalogueServiceWeb url = 'http://www.data.eaufrance.fr:/geosource/srv/eng/csw?'
>> csw = CatalogueServiceWeb(url)
>> csw.getrecords2()
>>
>> And here's the error I get :
>> Traceback (most recent call last):
>>     File "<stdin>", line 1, in <module>
>>     File "/usr/local/lib/python2.7/dist-packages/owslib/csw.py", line 376, in getrecords2
>>       self._invoke()
>>     File "/usr/local/lib/python2.7/dist-packages/owslib/csw.py", line 676, in _invoke
>>       self._exml = etree.parse(BytesIO(self.response))
>>     File "src/lxml/lxml.etree.pyx", line 3427, in lxml.etree.parse (src/lxml/lxml.etree.c:85131)
>>     File "src/lxml/parser.pxi", line 1799, in lxml.etree._parseDocument (src/lxml/lxml.etree.c:124241)
>>     File "src/lxml/parser.pxi", line 1819, in lxml.etree._parseMemoryDocument (src/lxml/lxml.etree.c:124533)
>>     File "src/lxml/parser.pxi", line 1707, in lxml.etree._parseDoc (src/lxml/lxml.etree.c:123074)
>>     File "src/lxml/parser.pxi", line 1079, in lxml.etree._BaseParser._parseDoc (src/lxml/lxml.etree.c:117114)
>>     File "src/lxml/parser.pxi", line 573, in lxml.etree._ParserContext._handleParseResultDoc (src/lxml/lxml.etree.c:110510)
>>     File "src/lxml/parser.pxi", line 683, in lxml.etree._handleParseResult (src/lxml/lxml.etree.c:112276)
>>     File "src/lxml/parser.pxi", line 613, in
>> lxml.etree._raiseParseError (src/lxml/lxml.etree.c:111124)
>> lxml.etree.XMLSyntaxError: Opening and ending tag mismatch: br line 1
>> and body, line 1, column 242
>>
>> If I then try :
>> csw.response
>>
>> I get :
>> "<html><head><title>Request Rejected</title></head><body>The requested URL was rejected. Please consult with your administrator.<br><br>Your support ID is: 16327421591402261030<br><br><a href='javascript:history.back();'>[Go Back]</a></body></html>"
>>
>> getrecordbyid works fine.
>>
>> Do you know if this issue is related to the server I'm querying, or if this is a bug ? Do you have any idea which direction I should try looking into ?
>>
>> Thanks a lot,
>>
>> Julie.
>>
>>
>> ________________________________
>>    This message (and any attachments) is for the recipient only. NERC is subject to the Freedom of Information Act 2000 and the contents of this email and any reply you make may be disclosed by NERC unless it is exempt from release under the Act. Any material supplied to NERC may be stored in an electronic records management system.
>> ________________________________
> ________________________________
>   This message (and any attachments) is for the recipient only. NERC is subject to the Freedom of Information Act 2000 and the contents of this email and any reply you make may be disclosed by NERC unless it is exempt from release under the Act. Any material supplied to NERC may be stored in an electronic records management system.
> ________________________________