[pycsw-devel] SOS Harvesting Error

[Tom Kralidis]

Dan: thanks for the info.  I'm guessing the index value is so big it's
causing a PostgreSQL.  If you comment out
https://github.com/geopython/pycsw/blob/master/pycsw/metadata.py#L678,
this will suppress the problem for the time being.

Can you open a GitHub issue
(https://github.com/geopython/pycsw/issues/new)? There are many big
SOS endpoints out there, and I think it would be valuable to assess
this more closely.

On Mon, Oct 27, 2014 at 12:48 PM,  <dan at inlet.geol.sc.edu> wrote:
> More info: The database backend when this is happening is PostGRES 9.2.
>
> When I switch over to SQLITE, the processing appears to work.
> I'm assuming SQLITE does not have the index limitation.
>
> Dan
>
>
>> The exception traceback on the local server:
>>
>> Exception happened during processing of request from ('127.0.0.1', 45798)
>> Traceback (most recent call last):
>>   File "/usr/local/src/python/lib/python2.7/SocketServer.py", line 295, in
>> _handle_request_noblock
>>     self.process_request(request, client_address)
>>   File "/usr/local/src/python/lib/python2.7/SocketServer.py", line 321, in
>> process_request
>>     self.finish_request(request, client_address)
>>   File "/usr/local/src/python/lib/python2.7/SocketServer.py", line 334, in
>> finish_request
>>     self.RequestHandlerClass(request, client_address, self)
>>   File "/usr/local/src/python/lib/python2.7/SocketServer.py", line 653, in
>> __init__
>>     self.finish()
>>   File "/usr/local/src/python/lib/python2.7/SocketServer.py", line 712, in
>> finish
>>     self.wfile.close()
>>   File "/usr/local/src/python/lib/python2.7/socket.py", line 279, in close
>>     self.flush()
>>   File "/usr/local/src/python/lib/python2.7/socket.py", line 303, in flush
>>     self._sock.sendall(view[write_offset:write_offset+buffer_size])
>> error: [Errno 32] Broken pipe
>>
>>
>>> I am attempting to populate a test pycsw server with the offerings from
>>> NDBC's SOS server. I am getting an exception of:
>>> <ows:Exception exceptionCode="NoApplicableCode" locator="source">
>>>     <ows:ExceptionText>Harvest (insert) failed: ERROR: index row size
>>> 3956
>>> exceeds maximum 2712 for index "ix_records_operateson"
>>> HINT:  Values larger than 1/3 of a buffer page cannot be indexed.
>>> Consider a function index of an MD5 hash of the value, or use full text
>>> indexing.
>>> .</ows:ExceptionText>
>>>   </ows:Exception>
>>> </ows:ExceptionReport>
>>>
>>> I am not sure what this is telling me. The xml file I am using to direct
>>> the pycsw-admin is:
>>> <?xml version="1.0" encoding="UTF-8"?>
>>> <Harvest xmlns="http://www.opengis.net/cat/csw/2.0.2"
>>> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>>> xsi:schemaLocation="http://www.opengis.net/cat/csw/2.0.2
>>> http://schemas.opengis.net/csw/2.0.2/CSW-publication.xsd" service="CSW"
>>> version="2.0.2">
>>>   <Source>http://sdf.ndbc.noaa.gov/sos/server.php</Source>
>>>   <ResourceType>http://www.opengis.net/sos/1.0</ResourceType>
>>> </Harvest>
>>>
>>> Any pointers would be great.
>>>
>>> Dan
>>>
>>> _______________________________________________
>>> pycsw-devel mailing list
>>> pycsw-devel at lists.osgeo.org
>>> http://lists.osgeo.org/mailman/listinfo/pycsw-devel
>>>
>>
>>
>> _______________________________________________
>> pycsw-devel mailing list
>> pycsw-devel at lists.osgeo.org
>> http://lists.osgeo.org/mailman/listinfo/pycsw-devel
>>
>
>
> _______________________________________________
> pycsw-devel mailing list
> pycsw-devel at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/pycsw-devel