[GeoNode-users] geonode importlayers error when metadata XML is present

Wed May 4 07:12:13 PDT 2016

Hi Tom,

Sorry for the late response. I was traveling. Continuing on this issue, I
edited the geonode/layers/metadata.py according to your commit [1]. I then
tried to add the shapefile which had the offending metadata xml file and I
received the error "Cannot use None as a query value"

My workflow is the following:
1) Download the shapefile and the metadata (ISO XML) from an old geonode
(running v2.4)
2) Rename the metadata xml file to be the same as the shapefile
3) Upload the shapefile+metadata (xml) in the new geonode server (v2.4).

I'm attaching the error I get when running geonode updatelayers

Thanks for the help
Daniel

[1]
https://github.com/GeoNode/geonode/commit/562af5f1b76cd785ecfb9ae996522a701c3f6dd5

On Mon, Apr 25, 2016 at 10:02 PM, Tom Kralidis <tomkralidis at gmail.com>
wrote:

> Hi Daniel:
>
> On Wed, Apr 13, 2016 at 2:43 PM, Daniel Victoria
> <daniel.victoria at gmail.com> wrote:
> > Hi Tom et al.,
> >
> > I saw that you closed the issue. Great. Two questions though
> > 1) Was the problem related to how the metadata was produced
> (incomplete)? Or
> > related to geonode not accepting incomplete data?
> >
>
> I would say both. Was the metadata produced in GeoNode?  If you can
> reproduce
> a workflow, this will help us validate it's a GeoNode metadata
> handling issue that
> we can then fix.
>
> Thanks
>
> ..Tom
>
> > 2) I'm a newbie when it comes to updating GeoNode. I have it installed
> from
> > the ppa repositories. How can I apply the patch you just comited? Should
> I
> > just convert the GeoNode install folder from my machine into a git
> > repository and pull from Github? Are there any tutorials on how to do
> that?
> >
> > Thanks
> > Daniel
> >
> >
> > On Tue, Apr 12, 2016 at 10:40 PM, Daniel Victoria
> > <daniel.victoria at gmail.com> wrote:
> >>
> >> Thanks Tom, I just opened an issue[1].
> >>
> >> Now, getting back to my problem of getting the metadata from one GeoNode
> >> server to another. What would be the best option? I tried downloading
> the
> >> XML and putting it along with the shapefile since I'd rather not have to
> >> fill 100+ metadata records by hand.
> >>
> >> What about the migration scripts from afabiani. Will that move both the
> >> data (vector data in postgis or imagens in GeoServer) along with the
> >> metadata?
> >>
> >> Cheers
> >> Daniel
> >>
> >> [1] https://github.com/GeoNode/geonode/issues/2469
> >>
> >> On Tue, Apr 12, 2016 at 5:27 PM, Tom Kralidis <tomkralidis at gmail.com>
> >> wrote:
> >>>
> >>> On Tue, Apr 12, 2016 at 12:39 PM, Daniel Victoria
> >>> <daniel.victoria at gmail.com> wrote:
> >>> > Hi all,
> >>> >
> >>> > I'm trying to populate a new geonode instance (v2.4) with some
> >>> > shapefiles
> >>> > from another geonode (v2.4).
> >>> >
> >>> > I have all the shapefiles in a directory so I'm inclined to use the
> >>> > importlayers command. And to make things easier, I've downloaded the
> >>> > metadata (ISO format, XML file) from the former geonode and placed it
> >>> > with
> >>> > the shapefiles that will be imported.
> >>> >
> >>> > The importlayers command fails with the following message
> >>> >
> >>> > ./par_a01_2013_inventoryplotsubplot.shp
> >>> > ================
> >>> > Traceback (most recent call last):
> >>> >   File
> >>> > "/usr/local/lib/python2.7/dist-packages/geonode/layers/utils.py",
> >>> > line 588, in upload
> >>> >     title=title
> >>> >   File
> >>> > "/usr/local/lib/python2.7/dist-packages/geonode/layers/utils.py",
> >>> > line 429, in file_upload
> >>> >     vals, regions, keywords = set_metadata(xml_file.read())
> >>> >   File
> >>> > "/usr/local/lib/python2.7/dist-packages/geonode/layers/metadata.py",
> >>> > line 62, in set_metadata
> >>> >     vals, regions, keywords = dc2dict(exml)
> >>> >   File
> >>> > "/usr/local/lib/python2.7/dist-packages/geonode/layers/metadata.py",
> >>> > line 198, in dc2dict
> >>> >     vals['date'] = sniff_date(mdata.modified)
> >>> >   File
> >>> > "/usr/local/lib/python2.7/dist-packages/geonode/layers/metadata.py",
> >>> > line 222, in sniff_date
> >>> >     return datetime.datetime.strptime(datestr.strip(), dfmt)
> >>> > AttributeError: 'NoneType' object has no attribute 'strip'
> >>> >
> >>> > If I remove the XML file from the shapefile directory, the
> importlayers
> >>> > command works OK.
> >>> >
> >>> > Any idea what is wrong? Attached is the offending metadata file, just
> >>> > in
> >>> > case.
> >>> > Cheers
> >>> > Daniel
> >>>
> >>> Daniel: looks like missing metadata.  Having said this, we should have
> a
> >>> more
> >>> graceful error.  If you can open a GitHub issue [1], I will fix.
> >>>
> >>> Thanks
> >>>
> >>> ..Tom
> >>>
> >>> [1] https://github.com/GeoNode/geonode/issues/new
> >>
> >>
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/geonode-users/attachments/20160504/a1f1f8e3/attachment-0001.html>
-------------- next part --------------

% geonode importlayers -v 2 .
Not enabling BingMaps base layer as a BING_API_KEY is not defined in local_settings.py file.
Verifying that GeoNode is running ...

Found 1 potential layers.

[failed] Layer for './par_a01_2013_las_boundary.shp' (1/1)

Detailed report of failures:

./par_a01_2013_las_boundary.shp
================
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/geonode/layers/utils.py", line 588, in upload
    title=title
  File "/usr/local/lib/python2.7/dist-packages/geonode/layers/utils.py", line 443, in file_upload
    regions_resolved, regions_unresolved = resolve_regions(regions)
  File "/usr/local/lib/python2.7/dist-packages/geonode/layers/utils.py", line 83, in resolve_regions
    region_resolved = Region.objects.get(Q(name__iexact=region) | Q(code__iexact=region))
  File "/usr/lib/python2.7/dist-packages/django/db/models/manager.py", line 151, in get
    return self.get_queryset().get(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/django/db/models/query.py", line 301, in get
    clone = self.filter(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/django/db/models/query.py", line 593, in filter
    return self._filter_or_exclude(False, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/modeltranslation/manager.py", line 292, in _filter_or_exclude
    return super(MultilingualQuerySet, self)._filter_or_exclude(negate, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/django/db/models/query.py", line 611, in _filter_or_exclude
    clone.query.add_q(Q(*args, **kwargs))
  File "/usr/lib/python2.7/dist-packages/django/db/models/sql/query.py", line 1204, in add_q
    clause = self._add_q(where_part, used_aliases)
  File "/usr/lib/python2.7/dist-packages/django/db/models/sql/query.py", line 1236, in _add_q
    current_negated)
  File "/usr/lib/python2.7/dist-packages/django/db/models/sql/query.py", line 1240, in _add_q
    current_negated=current_negated)
  File "/usr/lib/python2.7/dist-packages/django/db/models/sql/query.py", line 1074, in build_filter
    raise ValueError("Cannot use None as a query value")
ValueError: Cannot use None as a query value

Finished processing 1 layers in 0.0 seconds.

0 Created layers
0 Updated layers
0 Skipped layers
1 Failed layers
0.000000 seconds per layer