[GeoNode-devel] Table Join - Field Format Conversion

[Ariel Nunez]

I really like where this is going, some general feedback about the module
before and then my comments about the question you posed:

1) I would like to make this an optional app, the kind of dependencies it
pulls in (sqlalchemy) should be optional and not mandatory.
2) That said, I do think we should keep it under the GeoNode github org
umbrella if possible and make the installation painless via packages (sudo
apt-get install geonode-csv)
3) I see a lot of try/except blocks without specific exceptions [1]. This
wrapping can make the code harder to debug if something goes wrong, because
it will trap all errors and reduce the stacktraces to simple messages.

About JoinTargetFormatType:

I imagine the best place to do this is at the database (postgis level),
having the output be a view that is hooked up to the Django ORM for ease of
querying or can be directly enabled in GeoServer. If you go the Python way,
I assume you would like to store things back to the database after the
cleanup process and that back and forth seems like something we should
avoid.

Happy birthday,
Ariel

[1]
https://github.com/jj0hns0n/geonode/blob/table-join/geonode/contrib/datatables/utils.py#L257

On Mon, Feb 23, 2015 at 4:33 PM, Jeffrey Johnson <ortelius at gmail.com> wrote:

> Hi Gents,
>
> As mentioned in a previous post and in GNIP #1915, I've been working
> on adding a new 'datatables' contrib module for GeoNode that allows
> for uploading of CSV or other delimited files and loading these into
> PostgreSQL and joining a existing geometry to them. You can see the
> implementation here
> https://github.com/jj0hns0n/geonode/compare/table-join?expand=1 ...
> Everything is mostly working and functional at an API level. You can
> see the test setup we are using in tabular-api/code/tabular_test.py
> (this will get converted into a more complete integration test suite).
>
> The largest issue we have run into in our testing is field type
> mismatches. The simplest case is an int field in either the table or
> the layer and a string field in the other. There are also issues with
> the field being specified with leading zeroes or other characters in
> one of the 2 join components and not in the other. The folks at
> harvard have found other instances where they need to actually join
> the contents of 2 fields together before joining.
>
> Some tickets in my own fork for background/reference.
>
> https://github.com/jj0hns0n/geonode/issues/34
> https://github.com/jj0hns0n/geonode/issues/30
> https://github.com/jj0hns0n/geonode/issues/40
> https://github.com/jj0hns0n/geonode/issues/50
> https://github.com/jj0hns0n/geonode/issues/49 (the ticket Im dealing
> with _right now_)
>
> Raman (@raprasad) and I have discussed several options for dealing
> with this kind of problem and we decided to add a class
> JoinTargetFormatType that could store information necessary to do
> these kinds of conversions. You can see that class here
>
> https://github.com/jj0hns0n/geonode/blob/table-join/geonode/contrib/datatables/models.py#L51
> which is used in the JoinTarget class here
>
> https://github.com/jj0hns0n/geonode/blob/table-join/geonode/contrib/datatables/models.py#L71
> ... The basic idea is that you could have a set of pre-defined
> transformation functions or include some regex or a python function to
> do the conversion. None of this is hooked up yet, but thats the basic
> idea.
>
> So, we wanted to consult with all of you to see if anyone had any
> ideas about how best to hook this kind of thing up ... or perhaps some
> words of caution about how NOT to do this.
>
> Happy to discuss further if anyone has any questions or is unclear on
> what we are trying to do.
>
> Thanks in advance,
>
> Jeff
>
> To unsubscribe from this group and stop receiving emails from it, send an
> email to geonode-dev+unsubscribe at boundlessgeo.com.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/geonode-devel/attachments/20150303/f973662b/attachment.html>