[GeoNode-devel] errors with style creation during layer upload to PostGIS

Thu Sep 5 07:41:28 PDT 2019

Hi Simone
I tried the reset call from API, it is much faster but unfortunately
it doesn't fix the issue as a reload does.
So at this point the only option is to adopt a more sophisticated
layer naming mechanism, unless this can be fixed at GeoServer level.
I will open a ticket now, thanks for helping with this
Paolo

On Thu, Sep 5, 2019 at 10:21 AM Paolo Corti <pcorti at gmail.com> wrote:
>
> Hi Simone
> I will try the reset and let you know.
> I just checked the approach of using a more sophisticated approach to
> name a layer, and this is solving the problem.
> Right now when uploading a new layer, GeoNode checks if there is
> already a layer with the same name in the prefix. If this is the case,
> it appends a +1 to the layer name. This create a problem when removing
> the layer and reloading it a second time, because for some reason
> GeoServer still has that name in same part of the catalog (I believe
> the problem is connected with styles catalog, as it raises when
> GeoNode tries to create a style for the layer).
> If we use a more complex naming mechanism this will be sorted - I just
> tried a primitive approach to append a random number to the layer name
> and it works.
> So, definitely I can create a ticket: we can discuss if we should fix
> this at GeoServer lever (cleaner for sure), or at GeoNode level using
> a more complex layer naming mechanism
> thanks to you, Alessio and GeoSolutions for helping with this!
> Paolo
>
> On Thu, Sep 5, 2019 at 10:10 AM Simone Giannecchini
> <simone.giannecchini at geo-solutions.it> wrote:
> >
> > Ciao Paolo,
> > doing a full reload for each layer removal is out of question as it becomes exponentially slower with the number of layers.
> >
> > A reset ( https://docs.geoserver.org/stable/en/user/rest/api/reset.html ) can be a quick solution to try.
> > However, I would want to investigate this further next week with our GeoServer devs.
> > Do we have an issue for this yet?
> >
> > Regards,
> > Simone Giannecchini
> > ==
> > GeoServer Professional Services from the experts!
> > Visit http://bit.ly/gs-services for more information.
> > ==
> > Ing. Simone Giannecchini
> > @simogeo
> > Founder/Director
> >
> > GeoSolutions S.A.S.
> > Via di Montramito 3/A
> > 55054  Massarosa (LU)
> > Italy
> > phone: +39 0584 962313
> > fax:     +39 0584 1660272
> > mob:   +39  333 8128928
> >
> > http://www.geo-solutions.it
> > http://twitter.com/geosolutions_it
> >
> > -------------------------------------------------------
> > Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia.
> >
> > This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.
> >
> >
> > On Thu, Sep 5, 2019 at 4:02 PM Paolo Corti <pcorti at gmail.com> wrote:
> >>
> >> can I do that just from API or is there a tool in GS admin site like for reload?
> >>
> >> On Thu, Sep 5, 2019 at 9:55 AM Alessio Fabiani
> >> <alessio.fabiani at geo-solutions.it> wrote:
> >> >
> >> > Also, maybe just try a reset instead of a reload.
> >> >
> >> > Moreover, is this problem related to raster data too?
> >> >
> >> > Il giorno gio 5 set 2019 alle ore 15:49 Alessio Fabiani <alessio.fabiani at geo-solutions.it> ha scritto:
> >> >>
> >> >> Uhm, might be a problem if you have thousands of layers on GeoServer. We need to be careful hitting the catalog reload everytime.
> >> >>
> >> >> Do you have the chance to try a reload with a huge GeoServer catalog?
> >> >>
> >> >> Il giorno gio 5 set 2019 alle ore 15:45 Paolo Corti <pcorti at gmail.com> ha scritto:
> >> >>>
> >> >>> Hi Alessio
> >> >>> reloading the catalog seems to have fixed it. Maybe we should reload
> >> >>> the catalog after a layer remove? (I thought it is already the case)
> >> >>> thanks
> >> >>> Paolo
> >> >>>
> >> >>> On Thu, Sep 5, 2019 at 9:35 AM Alessio Fabiani
> >> >>> <alessio.fabiani at geo-solutions.it> wrote:
> >> >>> >
> >> >>> > It looks something related to GeoServer catalog.
> >> >>> >
> >> >>> > Could you try if just reloading it, instead of restarting GeoServer, also works?
> >> >>> >
> >> >>> > Il giorno gio 5 set 2019 alle ore 15:31 Paolo Corti <pcorti at gmail.com> ha scritto:
> >> >>> >>
> >> >>> >> Dear Alessio
> >> >>> >>
> >> >>> >> unfortunately this doesn't appear to be helpful. My idea is that there
> >> >>> >> is something wrong in authorization from GeoFence.
> >> >>> >>
> >> >>> >> Here is what happening:
> >> >>> >> * Restart GeoServer
> >> >>> >> * First layer upload works correctly
> >> >>> >> * I then remove the layer and I upload it again
> >> >>> >> * I get the error. Checkin in GeoServer log I see:
> >> >>> >>
> >> >>> >> 2019-09-05 13:25:39,175 ERROR [geoserver.rest] - No such datastore:
> >> >>> >> geonode,rivers
> >> >>> >> org.geoserver.rest.ResourceNotFoundException 404 NOT_FOUND: No such
> >> >>> >> datastore: geonode,rivers
> >> >>> >>
> >> >>> >> Restarting GeoServer will fix this.
> >> >>> >> This is happening in a GeoNode project, let me see if I can experience
> >> >>> >> the same locally.
> >> >>> >>
> >> >>> >> ps: I have seen your PR with the changes in gsconfig, I believe anyway
> >> >>> >> that they are useful, will merge it as soon as build is finalized.
> >> >>> >> thanks for helping with this
> >> >>> >> Paolo
> >> >>> >>
> >> >>> >> On Thu, Sep 5, 2019 at 8:56 AM Alessio Fabiani
> >> >>> >> <alessio.fabiani at geo-solutions.it> wrote:
> >> >>> >> >
> >> >>> >> > @Paolo any chanche to test the update of the library
> >> >>> >> >
> >> >>> >> > (maybe be sure you first removed completely the old one, gsconfig)
> >> >>> >> >
> >> >>> >> > pip install geoserver-restconfig==1.0.3
> >> >>> >> >
> >> >>> >> >
> >> >>> >> >
> >> >>> >> > Il giorno mer 4 set 2019 alle ore 23:11 Paolo Corti <pcorti at gmail.com> ha scritto:
> >> >>> >> >>
> >> >>> >> >> Looking in the GeoServer log, it appears that GeoNode upload fails to
> >> >>> >> >> create the style (and then the layer) in step 7 because it cannot find
> >> >>> >> >> the newly created GeoServer layer (even if this was already created at
> >> >>> >> >> step 4 in the upload process). This is the error I see in GeoServer:
> >> >>> >> >>
> >> >>> >> >> 2019-09-04 21:05:42,632 ERROR [geoserver.rest] - No such layer: geonode:rivers
> >> >>> >> >> org.geoserver.rest.ResourceNotFoundException 404 NOT_FOUND: No such
> >> >>> >> >> layer: geonode:rivers
> >> >>> >> >>
> >> >>> >> >> That looks like a regression? Or some configuration which is not correct?
> >> >>> >> >> In my case I am using DELAYED_SECURITY_SIGNAL set to True.
> >> >>> >> >>
> >> >>> >> >> thanks
> >> >>> >> >> Paolo
> >> >>> >> >>
> >> >>> >> >> On Wed, Sep 4, 2019 at 5:01 PM Paolo Corti <pcorti at gmail.com> wrote:
> >> >>> >> >> >
> >> >>> >> >> > Hi devs
> >> >>> >> >> >
> >> >>> >> >> > I am experiencing the following issue on an instance using GeoNode
> >> >>> >> >> > master from a GeoNode project and PostGIS as a datastore. When
> >> >>> >> >> > uploading a layer, I see the following
> >> >>> >> >> > error message:
> >> >>> >> >> >
> >> >>> >> >> > >>> Step 7. Creating style for [rivers]
> >> >>> >> >> > Traceback (most recent call last):
> >> >>> >> >> >   File "/home/ubuntu/geonode/geonode/geoserver/helpers.py", line 246,
> >> >>> >> >> > in get_sld_for
> >> >>> >> >> >     _default_style = layer.default_style
> >> >>> >> >> > AttributeError: 'FeatureType' object has no attribute 'default_style'
> >> >>> >> >> > Traceback (most recent call last):
> >> >>> >> >> >   File "/home/ubuntu/geonode/geonode/geoserver/helpers.py", line 255,
> >> >>> >> >> > in get_sld_for
> >> >>> >> >> >     name = gs_layer.default_style.name if gs_layer.default_style is
> >> >>> >> >> > not None else "raster"
> >> >>> >> >> > AttributeError: 'NoneType' object has no attribute 'default_style'
> >> >>> >> >> >
> >> >>> >> >> > after that, the layer is created in Django but not in GeoServer. Any
> >> >>> >> >> > hint why this could be happening before I debug it on my own?
> >> >>> >> >> >
> >> >>> >> >> > I noticed that after restarting GeoServer the layer is again uploaded correctly.
> >> >>> >> >> > Paolo
> >> >>> >> >> >
> >> >>> >> >> > --
> >> >>> >> >> > Paolo Corti
> >> >>> >> >> > Geospatial software developer
> >> >>> >> >> > web: http://www.paolocorti.net
> >> >>> >> >> > twitter: @capooti
> >> >>> >> >> > skype: capooti
> >> >>> >> >> > #drt3jc1
> >> >>> >> >>
> >> >>> >> >>
> >> >>> >> >>
> >> >>> >> >> --
> >> >>> >> >> Paolo Corti
> >> >>> >> >> Geospatial software developer
> >> >>> >> >> web: http://www.paolocorti.net
> >> >>> >> >> twitter: @capooti
> >> >>> >> >> skype: capooti
> >> >>> >> >> #drt3jc1
> >> >>> >> >> _______________________________________________
> >> >>> >> >> geonode-devel mailing list
> >> >>> >> >> geonode-devel at lists.osgeo.org
> >> >>> >> >> https://lists.osgeo.org/mailman/listinfo/geonode-devel
> >> >>> >> >
> >> >>> >> >
> >> >>> >> >
> >> >>> >> > --
> >> >>> >> >
> >> >>> >> > ==
> >> >>> >> >
> >> >>> >> > GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information.
> >> >>> >> > ==
> >> >>> >> > Ing. Alessio Fabiani
> >> >>> >> >
> >> >>> >> > @alfa7691
> >> >>> >> > Founder/Technical Lead
> >> >>> >> >
> >> >>> >> >
> >> >>> >> > GeoSolutions S.A.S.
> >> >>> >> > Via di Montramito 3/A - 55054  Massarosa (LU) - Italy
> >> >>> >> > phone: +39 0584 962313
> >> >>> >> > fax:     +39 0584 1660272
> >> >>> >> > mob:   +39 331 6233686
> >> >>> >> >
> >> >>> >> >
> >> >>> >> > http://www.geo-solutions.it
> >> >>> >> > http://twitter.com/geosolutions_it
> >> >>> >> > -------------------------------------------------------
> >> >>> >> >
> >> >>> >> > Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia.
> >> >>> >> >
> >> >>> >> >
> >> >>> >> > This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.
> >> >>> >>
> >> >>> >>
> >> >>> >>
> >> >>> >> --
> >> >>> >> Paolo Corti
> >> >>> >> Geospatial software developer
> >> >>> >> web: http://www.paolocorti.net
> >> >>> >> twitter: @capooti
> >> >>> >> skype: capooti
> >> >>> >> #drt3jc1
> >> >>> >
> >> >>> >
> >> >>> >
> >> >>> > --
> >> >>> >
> >> >>> > ==
> >> >>> >
> >> >>> > GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information.
> >> >>> > ==
> >> >>> > Ing. Alessio Fabiani
> >> >>> >
> >> >>> > @alfa7691
> >> >>> > Founder/Technical Lead
> >> >>> >
> >> >>> >
> >> >>> > GeoSolutions S.A.S.
> >> >>> > Via di Montramito 3/A - 55054  Massarosa (LU) - Italy
> >> >>> > phone: +39 0584 962313
> >> >>> > fax:     +39 0584 1660272
> >> >>> > mob:   +39 331 6233686
> >> >>> >
> >> >>> >
> >> >>> > http://www.geo-solutions.it
> >> >>> > http://twitter.com/geosolutions_it
> >> >>> > -------------------------------------------------------
> >> >>> >
> >> >>> > Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia.
> >> >>> >
> >> >>> >
> >> >>> > This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.
> >> >>>
> >> >>>
> >> >>>
> >> >>> --
> >> >>> Paolo Corti
> >> >>> Geospatial software developer
> >> >>> web: http://www.paolocorti.net
> >> >>> twitter: @capooti
> >> >>> skype: capooti
> >> >>> #drt3jc1
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >>
> >> >> ==
> >> >>
> >> >> GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information.
> >> >> ==
> >> >> Ing. Alessio Fabiani
> >> >>
> >> >> @alfa7691
> >> >> Founder/Technical Lead
> >> >>
> >> >>
> >> >> GeoSolutions S.A.S.
> >> >> Via di Montramito 3/A - 55054  Massarosa (LU) - Italy
> >> >> phone: +39 0584 962313
> >> >> fax:     +39 0584 1660272
> >> >> mob:   +39 331 6233686
> >> >>
> >> >>
> >> >> http://www.geo-solutions.it
> >> >> http://twitter.com/geosolutions_it
> >> >> -------------------------------------------------------
> >> >>
> >> >> Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia.
> >> >>
> >> >>
> >> >> This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.
> >> >
> >> >
> >> >
> >> > --
> >> >
> >> > ==
> >> >
> >> > GeoServer Professional Services from the experts! Visit http://goo.gl/it488V for more information.
> >> > ==
> >> > Ing. Alessio Fabiani
> >> >
> >> > @alfa7691
> >> > Founder/Technical Lead
> >> >
> >> >
> >> > GeoSolutions S.A.S.
> >> > Via di Montramito 3/A - 55054  Massarosa (LU) - Italy
> >> > phone: +39 0584 962313
> >> > fax:     +39 0584 1660272
> >> > mob:   +39 331 6233686
> >> >
> >> >
> >> > http://www.geo-solutions.it
> >> > http://twitter.com/geosolutions_it
> >> > -------------------------------------------------------
> >> >
> >> > Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia.
> >> >
> >> >
> >> > This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.
> >>
> >>
> >>
> >> --
> >> Paolo Corti
> >> Geospatial software developer
> >> web: http://www.paolocorti.net
> >> twitter: @capooti
> >> skype: capooti
> >> #drt3jc1
> >> _______________________________________________
> >> geonode-devel mailing list
> >> geonode-devel at lists.osgeo.org
> >> https://lists.osgeo.org/mailman/listinfo/geonode-devel
>
>
>
> --
> Paolo Corti
> Geospatial software developer
> web: http://www.paolocorti.net
> twitter: @capooti
> skype: capooti
> #drt3jc1

-- 
Paolo Corti
Geospatial software developer
web: http://www.paolocorti.net
twitter: @capooti
skype: capooti
#drt3jc1