[OWSLib-users] CatalogueServiceWeb and data.gov exception

[Davide Muraro]

>  Did you notice you asked for 20 records and only get 10 back?

Yes, I did and I read it could be a default limitation in the CSW server, but I didn't think about a possibile workaround:

http://nbviewer.jupyter.org/gist/rsignell-usgs/ade82853eebb2f0c16f222f0c57d94b2

It is very appreciated, thanks Richard.

Davide

________________________________
Da: Signell, Richard <rsignell at usgs.gov>
Inviato: domenica 4 giugno 2017 23.22
A: Davide Muraro
Cc: Passmore, James H.; owslib-users at lists.osgeo.org
Oggetto: Re: [OWSLib-users] CatalogueServiceWeb and data.gov exception

Davide

> it works like a charm.

Actually, not quite.  Did you notice you asked for 20 records and only
get 10 back?

It's annoying, but with OWSlib "getrecords" or "getrecords2", no
matter what your maxrecords is, you will only get 10 back.

So you need to iterate like this:

http://nbviewer.jupyter.org/gist/rsignell-usgs/ade82853eebb2f0c16f222f0c57d94b2

-Rich

>
> Davide
>
> ________________________________
> Da: Passmore, James H. <jpass at bgs.ac.uk>
> Inviato: venerdì 2 giugno 2017 10.48
> A: Davide Muraro; owslib-users at lists.osgeo.org
> Oggetto: RE: CatalogueServiceWeb and data.gov exception
>
>
> The line
>
>
>
> csw = CatalogueServiceWeb('http://catalog.data.gov/csw?service=CSW&version=2.0.2&request=GetCapabilities')
catalog.data.gov<http://catalog.data.gov/csw?service=CSW&version=2.0.2&request=GetCapabilities>
catalog.data.gov
CSW interface for catalog.data.gov This catalog contains metadata for all first-order data, services, and applications harvested from registered metadata collections ...



>
>
>
> should be:
>
>
>
> csw = CatalogueServiceWeb('http://catalog.data.gov/csw')
>
>
>
> or (as you’re getting redirected to an https service):
>
>
>
> csw = CatalogueServiceWeb('https://catalog.data.gov/csw')
>
>
>
> If I use (Python 3.6):
>
>
>
> from owslib import csw as csw
>
> c = csw.CatalogueServiceWeb('https://catalog.data.gov/csw')
> c.getrecords2(maxrecords=20)
> print(c.records)
>
>
>
> I get a result (eventually ~ the service seems quite slow).
>
>
>
> I’ve tried running a query but it seems to be taking forever to return
>
>
>
> James
>
>
>
> From: OWSLib-users [mailto:owslib-users-bounces at lists.osgeo.org] On Behalf Of Davide Muraro
> Sent: 01 June 2017 12:44
> To: owslib-users at lists.osgeo.org
> Subject: [OWSLib-users] CatalogueServiceWeb and data.gov exception
>
>
>
> Dear All,
>
>
>
> I was doing some tests using CatalogueServiceWeb and the data.gov CSW.
>
>
>
> Basically doing something like:
>
>
>
> from owslib.csw import CatalogueServiceWeb
>
> from owslib.fes import PropertyIsEqualTo, PropertyIsLike, BBox
>
> csw = CatalogueServiceWeb('http://catalog.data.gov/csw?service=CSW&version=2.0.2&request=GetCapabilities')
>
> query = PropertyIsEqualTo('csw:AnyText', 'map')
>
> csw.getrecords2(constraints=[query], maxrecords=20)
>
> print(csw.records)
>
>
>
> i got:
>
>
>
> *python3.5/site-packages/owslib/csw.py in getrecords2(self, constraints, sortby, typenames, esn, outputschema, format, startposition, maxrecords, cql, xml, resulttype)
>
>     380
>
>     381             # process search results attributes
>
> --> 382             val = self._exml.find(util.nspath_eval('csw:SearchResults', namespaces)).attrib.get('numberOfRecordsMatched')
>
>     383             self.results['matches'] = int(util.testXMLValue(val, True))
>
>     384             val = self._exml.find(util.nspath_eval('csw:SearchResults', namespaces)).attrib.get('numberOfRecordsReturned')
>
>
>
> AttributeError: 'NoneType' object has no attribute 'attrib'
>
>
>
>
>
> Ideas?
>
>
>
> D
>
>
>
>
>
> ________________________________
> This message (and any attachments) is for the recipient only. NERC is subject to the Freedom of Information Act 2000 and the contents of this email and any reply you make may be disclosed by NERC unless it is exempt from release under the Act. Any material supplied to NERC may be stored in an electronic records management system.
> ________________________________
>
> _______________________________________________
> OWSLib-users mailing list
> OWSLib-users at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/owslib-users




--
Dr. Richard P. Signell   (508) 457-2229
USGS, 384 Woods Hole Rd.
Woods Hole, MA 02543-1598
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/owslib-users/attachments/20170605/3a0f8303/attachment.html>