[GeoNode-users] geonode importlayers error when metadata XML is present

Wed May 4 13:45:01 PDT 2016

Thanks for the confirmation -- glad to hear!

On Wed, 4 May 2016, Daniel Victoria wrote:

> Date: Wed, 4 May 2016 16:26:39 -0300
> From: Daniel Victoria <daniel.victoria at gmail.com>
> To: Tom Kralidis <tomkralidis at gmail.com>
> Cc: "geonode-users at lists.osgeo.org" <geonode-users at lists.osgeo.org>
> Subject: Re: [GeoNode-users] geonode importlayers error when metadata XML is
>     present
> 
> Oops, I feel embarrassed now! The metadata XML file that I though was in
> ISO format was in fact, Dublin Core. I blame it on the intern that
> downloaded the data. But it's my fault for not checking...
>
> Anyway. I downloaded the (correct) ISO XML file from a GeoNode (v2.4),
> placed it with the shapefile and imported it in our new geonode server
> (v2.4). All the metadata came along. So this is basically what we are going
> to do to move data around from one geonode instance to another
> 1) download the shapefile
> 2) download the ISO XML
> 3) Place them together
> 4) upload in the new server
> 5) Profit!
>
> Cheers
> Daniel
>
>
>
> On Wed, May 4, 2016 at 11:12 AM, Daniel Victoria <daniel.victoria at gmail.com>
> wrote:
>
>> Hi Tom,
>>
>> Sorry for the late response. I was traveling. Continuing on this issue, I
>> edited the geonode/layers/metadata.py according to your commit [1]. I then
>> tried to add the shapefile which had the offending metadata xml file and I
>> received the error "Cannot use None as a query value"
>>
>> My workflow is the following:
>> 1) Download the shapefile and the metadata (ISO XML) from an old geonode
>> (running v2.4)
>> 2) Rename the metadata xml file to be the same as the shapefile
>> 3) Upload the shapefile+metadata (xml) in the new geonode server (v2.4).
>>
>> I'm attaching the error I get when running geonode updatelayers
>>
>> Thanks for the help
>> Daniel
>>
>> [1]
>> https://github.com/GeoNode/geonode/commit/562af5f1b76cd785ecfb9ae996522a701c3f6dd5
>>
>> On Mon, Apr 25, 2016 at 10:02 PM, Tom Kralidis <tomkralidis at gmail.com>
>> wrote:
>>
>>> Hi Daniel:
>>>
>>> On Wed, Apr 13, 2016 at 2:43 PM, Daniel Victoria
>>> <daniel.victoria at gmail.com> wrote:
>>>> Hi Tom et al.,
>>>>
>>>> I saw that you closed the issue. Great. Two questions though
>>>> 1) Was the problem related to how the metadata was produced
>>> (incomplete)? Or
>>>> related to geonode not accepting incomplete data?
>>>>
>>>
>>> I would say both. Was the metadata produced in GeoNode?  If you can
>>> reproduce
>>> a workflow, this will help us validate it's a GeoNode metadata
>>> handling issue that
>>> we can then fix.
>>>
>>> Thanks
>>>
>>> ..Tom
>>>
>>>> 2) I'm a newbie when it comes to updating GeoNode. I have it installed
>>> from
>>>> the ppa repositories. How can I apply the patch you just comited?
>>> Should I
>>>> just convert the GeoNode install folder from my machine into a git
>>>> repository and pull from Github? Are there any tutorials on how to do
>>> that?
>>>>
>>>> Thanks
>>>> Daniel
>>>>
>>>>
>>>> On Tue, Apr 12, 2016 at 10:40 PM, Daniel Victoria
>>>> <daniel.victoria at gmail.com> wrote:
>>>>>
>>>>> Thanks Tom, I just opened an issue[1].
>>>>>
>>>>> Now, getting back to my problem of getting the metadata from one
>>> GeoNode
>>>>> server to another. What would be the best option? I tried downloading
>>> the
>>>>> XML and putting it along with the shapefile since I'd rather not have
>>> to
>>>>> fill 100+ metadata records by hand.
>>>>>
>>>>> What about the migration scripts from afabiani. Will that move both the
>>>>> data (vector data in postgis or imagens in GeoServer) along with the
>>>>> metadata?
>>>>>
>>>>> Cheers
>>>>> Daniel
>>>>>
>>>>> [1] https://github.com/GeoNode/geonode/issues/2469
>>>>>
>>>>> On Tue, Apr 12, 2016 at 5:27 PM, Tom Kralidis <tomkralidis at gmail.com>
>>>>> wrote:
>>>>>>
>>>>>> On Tue, Apr 12, 2016 at 12:39 PM, Daniel Victoria
>>>>>> <daniel.victoria at gmail.com> wrote:
>>>>>>> Hi all,
>>>>>>>
>>>>>>> I'm trying to populate a new geonode instance (v2.4) with some
>>>>>>> shapefiles
>>>>>>> from another geonode (v2.4).
>>>>>>>
>>>>>>> I have all the shapefiles in a directory so I'm inclined to use the
>>>>>>> importlayers command. And to make things easier, I've downloaded the
>>>>>>> metadata (ISO format, XML file) from the former geonode and placed
>>> it
>>>>>>> with
>>>>>>> the shapefiles that will be imported.
>>>>>>>
>>>>>>> The importlayers command fails with the following message
>>>>>>>
>>>>>>> ./par_a01_2013_inventoryplotsubplot.shp
>>>>>>> ================
>>>>>>> Traceback (most recent call last):
>>>>>>>   File
>>>>>>> "/usr/local/lib/python2.7/dist-packages/geonode/layers/utils.py",
>>>>>>> line 588, in upload
>>>>>>>     title=title
>>>>>>>   File
>>>>>>> "/usr/local/lib/python2.7/dist-packages/geonode/layers/utils.py",
>>>>>>> line 429, in file_upload
>>>>>>>     vals, regions, keywords = set_metadata(xml_file.read())
>>>>>>>   File
>>>>>>> "/usr/local/lib/python2.7/dist-packages/geonode/layers/metadata.py",
>>>>>>> line 62, in set_metadata
>>>>>>>     vals, regions, keywords = dc2dict(exml)
>>>>>>>   File
>>>>>>> "/usr/local/lib/python2.7/dist-packages/geonode/layers/metadata.py",
>>>>>>> line 198, in dc2dict
>>>>>>>     vals['date'] = sniff_date(mdata.modified)
>>>>>>>   File
>>>>>>> "/usr/local/lib/python2.7/dist-packages/geonode/layers/metadata.py",
>>>>>>> line 222, in sniff_date
>>>>>>>     return datetime.datetime.strptime(datestr.strip(), dfmt)
>>>>>>> AttributeError: 'NoneType' object has no attribute 'strip'
>>>>>>>
>>>>>>> If I remove the XML file from the shapefile directory, the
>>> importlayers
>>>>>>> command works OK.
>>>>>>>
>>>>>>> Any idea what is wrong? Attached is the offending metadata file,
>>> just
>>>>>>> in
>>>>>>> case.
>>>>>>> Cheers
>>>>>>> Daniel
>>>>>>
>>>>>> Daniel: looks like missing metadata.  Having said this, we should
>>> have a
>>>>>> more
>>>>>> graceful error.  If you can open a GitHub issue [1], I will fix.
>>>>>>
>>>>>> Thanks
>>>>>>
>>>>>> ..Tom
>>>>>>
>>>>>> [1] https://github.com/GeoNode/geonode/issues/new
>>>>>
>>>>>
>>>>
>>>
>>
>>
>