[QGIS-Developer] QGIS Server 3 status
Anita Graser
anitagraser at gmx.at
Tue Oct 17 12:19:22 PDT 2017
I've extended Régis' draft with some of the more recent information. Please
let me know what you think. I'll try to publish in a couple of days.
-- start --
*QGIS server refactoring is done!*
As you may know, QGIS is jumping to a new major version (yes!). Doing so
was made necessary because of the need to switch to python 3, QT5, but also
because we needed to break the QGIS API in several places. (
http://blog.qgis.org/2016/02/10/qgis-3-0-plans/)
A year ago, a tiny troll <https://github.com/qgis/qgis3.0_api/issues/63>from
switzerland rang a bell about the strong need for love server code base
required. Indeed, the API was locked by some old methods of QGIS server. In
short, QGIS server was reparsing the qgs project file in his own way, and
created dependencies to part of QGIS we needed to drop.
As outsourcing the server code base was not an option, so we had to
refactor it. The involved parties decided to get engaged in a code sprint
<https://github.com/qgis/QGIS/wiki/QGIS3---QGIS-Server-code-sprint-Notes> in
the city of Lyon , France dedicated to sharing their vision, planning the
work and finally making all the following
<https://github.com/qgis/QGIS-Enhancement-Proposals/issues/74> happen:
*Higher level refactoring*
All services (WMS GetMap, WFS GetFeature, GetLegendGraphics, WCS, GetPrint
etc..) have been rewritten. Some like WMS were entirely rewritten. Kudos to
the devs!
*New features*
- Multi-thread rendering <https://github.com/qgis/QGIS/pull/3886> like in
the desktop
- A new option to trust layer metadata
<https://github.com/qgis/QGIS/pull/5094> and thus speed up project loading
- WFS 1.1 support https://github.com/qgis/QGIS/pull/5297
- Full Python bindings for the server API
- Server Services as plugins like providers
[image: Inline image 1]
*Deep, complex and unrewarding tasks*
- Remove all singleton calls
- Cut all the dependencies to the old QGIS project file parser
- Minimize dependencies to GUI library. Since fonts are necessary to render
maps, totally removing them was not feasable.
*Infrastructure tasks*
- Build a OGC compliancy platform
<http://oslandia.com/en/2017/06/16/qgis-server-ogc-cite-compliance-testing/>
and
integrate it to a continuous integration platform. Conformity reports are
now pushed to tests.qgis.org
- Add unit tests ... and again more unit tests
- Stress QGIS server against security leaks
<http://oslandia.com/en/2017/06/14/qgis-server-security-aspect/> (SQL
injections and other malicious attacks)
- Start profiling and bench marking performances. This work still need some
love - and funding - to be achieved
A presentation
<https://git.osgeo.org/gogs/foss4g-europe/foss4g-europe-2017-paris/src/master/presentations/2017-07-20/general_track/foss4g-europe-2017-QGIS_3_Refactoring_and_enhancement-DMarteau-PBlottiere.pdf>
was
given at FOSS4G-EU in July.
*Congratulations* to the developers who worked hard on that work.
Now this deserves to be well tested, please report back any issues!
On Fri, Oct 13, 2017 at 9:50 AM, René-Luc Dhont <rldhont at gmail.com> wrote:
> And I'm waiting to add WFS 1.1.0 support:
> https://github.com/qgis/QGIS/pull/5297
>
> Regards,
> René-Luc
>
>
> Le 13/10/2017 à 08:58, Régis Haubourg a écrit :
>
> Hi Anita,
>
> among the news, Paul is currently tackling OGC conformity issues.
> see :
> https://github.com/qgis/QGIS/pull/5044
> https://github.com/qgis/QGIS/pull/5084
> https://github.com/qgis/QGIS/pull/5145
> https://github.com/qgis/QGIS/pull/5150
> https://github.com/qgis/QGIS/pull/5157
> https://github.com/qgis/QGIS/pull/5303
>
> We still have some harder issues due to crashes of the OGC testing
> platform.
>
> All tests are made using the "basic" level test (see http://cite.
> opengeospatial.org/teamengine/about/wms/1.3.0/site/)
>
> When all will be green at this level, we'll run the platform with the
> other testing levels so that we have a good overview of our path for a full
> conformity of WMS 1.3.
>
> We'll prepare a dedicated blog post toi describe what were the issues and
> what we did to fix them.
>
> Cheers!
> Régis
>
> 2017-10-12 22:56 GMT+02:00 Anita Graser <anitagraser at gmx.at>:
>
>> Thanks everyone so far! Please keep the info coming. I'll draft a blog
>> post on the weekend.
>>
>> Regards,
>> Anita
>>
>> On Thu, Oct 5, 2017 at 9:34 AM, Alessandro Pasotti <apasotti at gmail.com>
>> wrote:
>>
>>> Hi Règis,
>>>
>>> thanks for the nice summary!
>>>
>>> Maybe it would be interesting for python devs to stress out how QGIS
>>> server 3 brings the bindings and the plugins to a new level: while the
>>> original filter-style python plugins, are still available, the new
>>> service-oriented plugin-based architecture allows Python developers to
>>> create new services (for example WPS) by implementing a new `QgsService`
>>> directly in Python. Overriding existing services is also possible.
>>>
>>> The server API has now full python bindings and a QGIS server can be
>>> easily implemented directly in Python and wrapped into your favourite
>>> framework (see this simple wsgi experiment:
>>> https://gist.github.com/elpaso/0cd7fb6a0518dc46c45557d9133438b8 ).
>>>
>>> For more examples of the API, I think there are some basic examples in
>>> the official Python tests: https://github.com/qgis/QGIS/b
>>> lob/master/tests/src/python/test_qgsserver_services.py#L79
>>>
>>>
>>> Cheers
>>>
>>>
>>>
>>>
>>>
>>> On Wed, Oct 4, 2017 at 10:49 PM, Régis Haubourg <
>>> regis.haubourg at gmail.com> wrote:
>>>
>>>> Well, from a user point of view, that's a bit hard since all this is 90
>>>> % refactoring.. which is supposed to be transparent for users.
>>>>
>>>> We have the trust and multithreaded features, that should (and early
>>>> testers today found it slower today) and the ability to code new services
>>>> using python.
>>>> We can also show some numbers about OGC conformity tests and the goal
>>>> of reaching OGC certification (WMS only by now).
>>>>
>>>> Alessandro, René-Luc and David, do you have some user oriented examples
>>>> for the new modular API ?
>>>>
>>>>
>>>>
>>>> 2017-10-04 22:18 GMT+02:00 Anita Graser <anitagraser at gmx.at>:
>>>>
>>>>> Great overview, thanks for drafting Régis!
>>>>>
>>>>> Some additional ideas: Could we add a section with the key
>>>>> improvements from an user perspective?
>>>>>
>>>>> I'm particularly intrigued by the services / plugins part. Could you
>>>>> elaborate?
>>>>>
>>>>> Regards,
>>>>> Anita
>>>>>
>>>>>
>>>>>
>>>>> On Wed, Oct 4, 2017 at 9:42 PM, Régis Haubourg <
>>>>> regis.haubourg at gmail.com> wrote:
>>>>>
>>>>>> Hi again,
>>>>>> here is my very first draft:
>>>>>>
>>>>>> Several images can be pulled from the joined slides given at
>>>>>> Foss4G-EU here https://drive.google.com/file/
>>>>>> d/0BzatQbprJ-hjWF95VzdHbGRsVEk/view?usp=sharing
>>>>>>
>>>>>> René-Luc, David, Paul, don't hesitate to amend that!
>>>>>>
>>>>>> ------------------------------------------------------------
>>>>>> --------------------------------
>>>>>>
>>>>>>
>>>>>> *Qgis server refactoring is done! *
>>>>>> As you may know, QGIS is jumping to a new major version (yes!).
>>>>>> Doing so was made necessary because of the need to switch to python
>>>>>> 3, QT5, but also because we needed to break the QGIS API in several places.
>>>>>>
>>>>>> A year ago, a tiny troll
>>>>>> <https://github.com/qgis/qgis3.0_api/issues/63>from switzerland rang
>>>>>> a bell about the strong need for love server code base required.
>>>>>>
>>>>>> Indeed, the API was locked by some old methods of QGIS server. In
>>>>>> short, QGIS server was reparsing the qgs project file in his own way, and
>>>>>> created dependencies to part of QGIS we needed to drop.
>>>>>>
>>>>>> As outsourcing the server code base was not an option, so we had to
>>>>>> refactor it.
>>>>>>
>>>>>> The involved parties decided to get engaged in a code sprint
>>>>>> <https://github.com/qgis/QGIS/wiki/QGIS3---QGIS-Server-code-sprint-Notes> in
>>>>>> the city of Lyon , France dedicated to sharing their vision, planning the
>>>>>> work and finally making all the following
>>>>>> <https://github.com/qgis/QGIS-Enhancement-Proposals/issues/74> happen
>>>>>> :
>>>>>>
>>>>>>
>>>>>>
>>>>>> *# Deep, complex and unrewarding tasks: *
>>>>>>
>>>>>> - Remove all singleton calls
>>>>>>
>>>>>> - Cut all the dependencies to the old qgis project parser
>>>>>>
>>>>>> - Minimize dependencies to GUI library. Since fonts are necessary to
>>>>>> render maps, totally removing them was not feasable.
>>>>>>
>>>>>> *# Higher level refactoring*
>>>>>>
>>>>>> All services (WMS GetMap, WFS GetFeature, GetLegendGraphics, WCS,
>>>>>> GetPrint etc..) have been rewritten. Some like WMS were entirely rewritten.
>>>>>> Kudos to the dev's
>>>>>>
>>>>>>
>>>>>> *# New features and architecture stuff*
>>>>>>
>>>>>> - QGIS Server Services as plugins like providers. (slide 10 would be
>>>>>> a nice figure here)
>>>>>>
>>>>>> - Multi-thread rendering <https://github.com/qgis/QGIS/pull/3886>-
>>>>>> like in the desktop
>>>>>>
>>>>>> - A new option to trust layer metadata
>>>>>> <https://github.com/qgis/QGIS/pull/5094> and speed up project loading
>>>>>>
>>>>>>
>>>>>> *# Infrastructure tasks*
>>>>>>
>>>>>> - Build a OGC compliancy platform
>>>>>> <http://oslandia.com/en/2017/06/16/qgis-server-ogc-cite-compliance-testing/> and
>>>>>> integrate it to a continuous integration platform. Conformity reports are
>>>>>> now pushed to tests.qgis.org *XXX waiting for Richard to expose them
>>>>>> in the website*
>>>>>>
>>>>>> - Add unit tests... and again more unit tests
>>>>>>
>>>>>> - Stress QGIS server against security leaks
>>>>>> <http://oslandia.com/en/2017/06/14/qgis-server-security-aspect/> (SQL
>>>>>> injections and other malicious attacks)
>>>>>>
>>>>>> - Start profiling and bench marking performances. This work still
>>>>>> need some love - and funding - to be achieved
>>>>>>
>>>>>>
>>>>>> A presentation
>>>>>> <https://git.osgeo.org/gogs/foss4g-europe/foss4g-europe-2017-paris/src/master/presentations/2017-07-20/general_track/foss4g-europe-2017-QGIS_3_Refactoring_and_enhancement-DMarteau-PBlottiere.pdf> was
>>>>>> given at FOSS4G-EU in July.
>>>>>>
>>>>>> *Congratulations* to the developers who worked hard on that work.
>>>>>>
>>>>>> Now this deserves to be well tested, please report back any issues!
>>>>>>
>>>>>>
>>>>>>
>>>>>> 2017-10-04 15:17 GMT+02:00 Anita Graser <anitagraser at gmx.at>:
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Oct 4, 2017 at 1:23 PM, Régis Haubourg <
>>>>>>> regis.haubourg at gmail.com> wrote:
>>>>>>>
>>>>>>>> Hi
>>>>>>>>
>>>>>>>> Yes, refactoring is now complete! see :
>>>>>>>> https://github.com/qgis/QGIS-Enhancement-Proposals/issues/74
>>>>>>>> #issuecomment-334123595
>>>>>>>>
>>>>>>>> Anita, I'll draft an article tonight if you still agree to review
>>>>>>>> it, that would be great!
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Sure!
>>>>>>>
>>>>>>> Anita
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> QGIS-Developer mailing list
>>>> QGIS-Developer at lists.osgeo.org
>>>> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>>>> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>>>>
>>>
>>>
>>>
>>> --
>>> Alessandro Pasotti
>>> w3: www.itopen.it
>>>
>>
>>
>
>
> _______________________________________________
> QGIS-Developer mailing listQGIS-Developer at lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>
>
>
> _______________________________________________
> QGIS-Developer mailing list
> QGIS-Developer at lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-developer/attachments/20171017/e0d1f330/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 221578 bytes
Desc: not available
URL: <http://lists.osgeo.org/pipermail/qgis-developer/attachments/20171017/e0d1f330/attachment-0001.png>
More information about the QGIS-Developer
mailing list