[GeoNode-devel] download original datasets feature randomly fails

Mon May 20 01:24:00 PDT 2019

Hello Paolo,
yes it was tested... not recently to be honest.
Maybe we can improve it in order to look first for the original dataset and
advertise the user suddenly.
Moreover in the latest versions the management command
"set_all_layers_metadata" checks and fixes the links, and refreshes the
metadata accordingly.
We could envisage the code creating "Original Dataset" links to check for
the original resource before creating it.

Il giorno ven 17 mag 2019 alle ore 21:49 Paolo Corti <pcorti at gmail.com> ha
scritto:

> Dear Alessio
> thanks a lot for the detailed explanation.
> The reason in our instances could be that for many layers we haven't
> anymore the original shapefile but just the data in PostGIS.
> Which means that the best option for us is to disable that link.
> ps: by the way, was this tested both with rest and the importer
> uploader? (we are using the latter)
> Have a great weekend
> Paolo
>
> On Fri, May 17, 2019 at 3:47 AM Alessio Fabiani
> <alessio.fabiani at geo-solutions.it> wrote:
> >
> > Hello Paolo,
> > don't know exactly why is failing in your case, it is usually working,
> at least on my experience.
> >
> > However, the idea is the following.
> > All the download methods exposed on GeoNode, relies on OGC calls. For
> vectors WFS, raster WCS and other formats WMS.
> > That means that you overload GeoServer and potentially, especially for
> huge datasets, a single download can stuck your server by consuming all the
> machine resources available.
> > Notice also that GeoServer always does a format conversion when you ask
> for a raw data format.
> >
> > The latest versions of GeoServer have some protection mechanisms
> allowing you to configure the cap of resources to be consumed.
> > That means that for big datasets the download won't be allowed at all.
> >
> > The intention of "Original Dataset" download link, is to allow GeoNode
> to package the uploaded files into the filesystem along with all the SLDs
> defined and metadata formats, and send back to the user
> > without making OGC requests.
> > Of course this will successfully work only if:
> >
> > 1. The original uploaded file still exists and it is accessible on the
> filesystem
> > 2. The layer is not a pure table on a DB but has a file associated with
> it
> >
> > Other than this, and this could be the cause why it is failing in your
> case, the "upload session" metadata stored on GeoNode database should point
> to the correct original files uploaded into GeoNode.
> >
> > That said, if you have the chance to investigate a bit more on the cause
> of the failures, we can try to make it a bit more reliable.
> >
> > Cheers,
> > A.
> >
> >
> >
> >
> > Il giorno ven 17 mag 2019 alle ore 00:41 Paolo Corti <pcorti at gmail.com>
> ha scritto:
> >>
> >> Hi devs
> >> when downloading a layer, "Zipped Shapefile" works as expected, while
> >> "Original Datasets" randomly fails. Also, I am not sure what is the
> >> latter for: when I am able to download it successfully, I see a zip
> >> file containing a bunch of .sld and geojson file.
> >> Would be glad to learn more about this
> >> thanks
> >> Paolo
> >>
> >> --
> >> Paolo Corti
> >> Geospatial software developer
> >> web: http://www.paolocorti.net
> >> twitter: @capooti
> >> skype: capooti
> >> #drt3jc1
> >> _______________________________________________
> >> geonode-devel mailing list
> >> geonode-devel at lists.osgeo.org
> >> https://lists.osgeo.org/mailman/listinfo/geonode-devel
> >
> >
> >
> > --
> >
> > ==
> >
> > GeoServer Professional Services from the experts! Visit
> http://goo.gl/it488V for more information.
> > ==
> > Ing. Alessio Fabiani
> >
> > @alfa7691
> > Founder/Technical Lead
> >
> >
> > GeoSolutions S.A.S.
> > Via di Montramito 3/A - 55054  Massarosa (LU) - Italy
> > phone: +39 0584 962313
> > fax:     +39 0584 1660272
> > mob:   +39 331 6233686
> >
> >
> > http://www.geo-solutions.it
> > http://twitter.com/geosolutions_it
> > -------------------------------------------------------
> >
> > Con riferimento alla normativa sul trattamento dei dati personali (Reg.
> UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si
> precisa che ogni circostanza inerente alla presente email (il suo
> contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è
> riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il
> messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra
> operazione è illecita. Le sarei comunque grato se potesse darmene notizia.
> >
> >
> > This email is intended only for the person or entity to which it is
> addressed and may contain information that is privileged, confidential or
> otherwise protected from disclosure. We remind that - as provided by
> European Regulation 2016/679 “GDPR” - copying, dissemination or use of this
> e-mail or the information herein by anyone other than the intended
> recipient is prohibited. If you have received this email by mistake, please
> notify us immediately by telephone or e-mail.
>
>
>
> --
> Paolo Corti
> Geospatial software developer
> web: http://www.paolocorti.net
> twitter: @capooti
> skype: capooti
> #drt3jc1
>

-- 

==

GeoServer Professional Services from the experts! Visit http://goo.gl/it488V
for more information.
==
Ing. Alessio Fabiani

@alfa7691
Founder/Technical Lead

GeoSolutions S.A.S.
Via di Montramito 3/A - 55054  Massarosa (LU) - Italy
phone: +39 0584 962313
fax:     +39 0584 1660272
mob:   +39 331 6233686

http://www.geo-solutions.it
http://twitter.com/geosolutions_it
-------------------------------------------------------

Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE
2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si
precisa che ogni circostanza inerente alla presente email (il suo
contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è
riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il
messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra
operazione è illecita. Le sarei comunque grato se potesse darmene notizia.

This email is intended only for the person or entity to which it is
addressed and may contain information that is privileged, confidential or
otherwise protected from disclosure. We remind that - as provided by
European Regulation 2016/679 “GDPR” - copying, dissemination or use of this
e-mail or the information herein by anyone other than the intended
recipient is prohibited. If you have received this email by mistake, please
notify us immediately by telephone or e-mail.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/geonode-devel/attachments/20190520/db6961e6/attachment.html>