[GeoNode-devel] Table Join - Field Format Conversion

Ariel Nunez ingenieroariel at gmail.com
Tue Mar 3 14:42:02 PST 2015

I really like where this is going, some general feedback about the module
before and then my comments about the question you posed:

1) I would like to make this an optional app, the kind of dependencies it
pulls in (sqlalchemy) should be optional and not mandatory.
2) That said, I do think we should keep it under the GeoNode github org
umbrella if possible and make the installation painless via packages (sudo
apt-get install geonode-csv)
3) I see a lot of try/except blocks without specific exceptions [1]. This
wrapping can make the code harder to debug if something goes wrong, because
it will trap all errors and reduce the stacktraces to simple messages.

About JoinTargetFormatType:

I imagine the best place to do this is at the database (postgis level),
having the output be a view that is hooked up to the Django ORM for ease of
querying or can be directly enabled in GeoServer. If you go the Python way,
I assume you would like to store things back to the database after the
cleanup process and that back and forth seems like something we should

Happy birthday,


On Mon, Feb 23, 2015 at 4:33 PM, Jeffrey Johnson <ortelius at gmail.com> wrote:

> Hi Gents,
> As mentioned in a previous post and in GNIP #1915, I've been working
> on adding a new 'datatables' contrib module for GeoNode that allows
> for uploading of CSV or other delimited files and loading these into
> PostgreSQL and joining a existing geometry to them. You can see the
> implementation here
> https://github.com/jj0hns0n/geonode/compare/table-join?expand=1 ...
> Everything is mostly working and functional at an API level. You can
> see the test setup we are using in tabular-api/code/tabular_test.py
> (this will get converted into a more complete integration test suite).
> The largest issue we have run into in our testing is field type
> mismatches. The simplest case is an int field in either the table or
> the layer and a string field in the other. There are also issues with
> the field being specified with leading zeroes or other characters in
> one of the 2 join components and not in the other. The folks at
> harvard have found other instances where they need to actually join
> the contents of 2 fields together before joining.
> Some tickets in my own fork for background/reference.
> https://github.com/jj0hns0n/geonode/issues/34
> https://github.com/jj0hns0n/geonode/issues/30
> https://github.com/jj0hns0n/geonode/issues/40
> https://github.com/jj0hns0n/geonode/issues/50
> https://github.com/jj0hns0n/geonode/issues/49 (the ticket Im dealing
> with _right now_)
> Raman (@raprasad) and I have discussed several options for dealing
> with this kind of problem and we decided to add a class
> JoinTargetFormatType that could store information necessary to do
> these kinds of conversions. You can see that class here
> https://github.com/jj0hns0n/geonode/blob/table-join/geonode/contrib/datatables/models.py#L51
> which is used in the JoinTarget class here
> https://github.com/jj0hns0n/geonode/blob/table-join/geonode/contrib/datatables/models.py#L71
> ... The basic idea is that you could have a set of pre-defined
> transformation functions or include some regex or a python function to
> do the conversion. None of this is hooked up yet, but thats the basic
> idea.
> So, we wanted to consult with all of you to see if anyone had any
> ideas about how best to hook this kind of thing up ... or perhaps some
> words of caution about how NOT to do this.
> Happy to discuss further if anyone has any questions or is unclear on
> what we are trying to do.
> Thanks in advance,
> Jeff
> To unsubscribe from this group and stop receiving emails from it, send an
> email to geonode-dev+unsubscribe at boundlessgeo.com.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/geonode-devel/attachments/20150303/f973662b/attachment.html>

More information about the geonode-devel mailing list