[Qgis-psc] Crowd source review of blog post for Gerona Hackfest
Tim Sutton
tim at kartoza.com
Wed Jun 29 04:06:15 PDT 2016
Hi All
Please find below my draft blog post for the Gerona hackfest. Sorry it has
taken so long for me to write - it is rather long and there is a lot going
on at the moment so I've struggled to find the spare time to work on it.
Please let me know if you would like to add/remove/amend anything. I'd like
to post it tomorrow morning.
---
Time flies when you are having fun! It seems like only yesterday that
I was writing
about
<http://blog.qgis.org/2015/11/15/a-word-of-thanks-to-the-hosts-of-the-14th-qgis-hackfest-in-gran-canaria/>
the
14th Hackfest in Gran Canaria. At the end of May 2016, we held the 15th
QGIS hackfest! QGIS has been on an incredible journey since the project was
started by Gary Sherman 14 years ago, and the fact that the hackfest was
held in tandem with the 2nd QGIS International User conference is a
testament to the growth and strength of the project.
2nd International QGIS Conference
Isn't it amazing - we just held the *second *international QGIS User's
Conference! We really need to give credit to the amazing team who ran a
totally seamless operation to organise the event: Gemma Pons, Toni
Hernández, Josep Sitjar, Alexandre Busquets, Ferran Orduña, Rosa Olivella,
Laura Olivas, & Lluís Vicens
[image: Girona Organising Team]
<https://qgisblog.files.wordpress.com/2016/06/screen-shot-2016-06-20-at-00-21-31.png>
We would like to give a special thank you to the University of
Gerona's Director of GIS Service (SIGTE) - Gemma Boix who helped to
organised the event as well as ensuring the institutional support for the
event. In case I have missed mentioning someone by name, our thanks to all
the other volunteers and the sponsors of the conference. OSGeo
<http://osgeo.org> also supported the hack fest for which we are extremely
grateful. Not only did the conference team host the conference event, they
also covered a large part of the costs of the hackfest that followed the
conference - for which we can't thank them enough!
The conference team also extends their thanks to the attendees and
presenters, instructors and developers who also actively participated on
the event! For those interested in viewing the various talks at the
conference, here are some handy links:
- Sides from all talks
<http://www.sigte.udg.edu/jornadassiglibre/en/international-qgis-user-and-developer-conference/conferencia-qgis/>
- Videos for conference presentations <https://vimeo.com/sigteudg/albums>
State of QGIS
The QGIS.org project is in a very healthy state right now. For my talk
<https://qgisblog.files.wordpress.com/2016/06/qgisgironahackfesttalk.pdf> at
the user conference (video here <http://diobma.udg.edu/handle/10256.1/4292>)
I got some fresh download stats from our servers and the numbers are quite
astounding: QGIS 2.8 LTR (release Feb 2015) has been downloaded over 679,
000 times for the Windows Standalone installer (which includes all bug fix
releases). Even after controlling for overestimates (from bot downloads,
multiple downloads per user, "try and don't use") and underestimates
(single downloads being distributed to many users) it is clear that we have
an extremely large and constantly growing user base. My apologies for not
mentioning Gary Sherman (our project founder) by name at the start of my
talk - that was totally unintentional!
Something else that is clear from the make up of the talks, workshop topics
and attendees at the user conference and hackfest:* QGIS is increasingly
moving from single user environments to large multi-user deployments.* This
'edge of network effect' is a common phenomenon in FOSS and is largely how
Linux came to be such a lynchpin in the dev-ops world. Sys admins and (in
the case of QGIS) GIS power users, test out the software on their own
devices, see the potential for it in their workplace and start integrating
it into their workflows in the office until eventually it has become a
mission critical piece of software for an organisation.
One thing my slides probably do not make clear is that there is a huge
amount of investment being made into QGIS and plugins for QGIS to fulfil a
wide variety of needs. *These investments are largely external to the
project* (i.e. not factored into the financial figures I mentioned in the
talk) and happen in direct client-to-developer relationships completely
bypassing (from a financial sense) the upstream QGIS.ORG project. This is
a really good model since we do not need to deal with contract delivery,
competing interests etc.
If we do a simple calculation based on direct QGIS.org revenue for
2015/2016 (around EUR 69, 000) to downloads, the average revenue per
download of QGIS 2.8.x was around EUR 0.10. All of the money we receive
into the project goes into improving the QGIS, maintaining infrastructure
and funding travel and accommodation for hackfests. I mention these numbers
both because they are interesting and because it is good to emphasise how
incredibly grateful we are to each and every one of our sponsors and donors
that support the project. We really do run on a shoe-string budget and we
have audacious goals and a vision to put spatial decision making tools into
the hands of everyone on the planet who wants to use them. Your sponsorship
and donations are a key enabler to making this vision a reality!
Ok you probably are more interested in the part below than the part above
right? What actually happened at the hackfest? Lets dig in and find out!
Cool stuff from the QGIS Hackfest
I am only going to focus on the hackfest here because the videos from the
talks at the QGIS User Conference have been posted online (see above), so
you can take in all the QGIS goodness you like from those. There was a lot
going on at the hackfest, so these are only the nuggets I managed to
cherry-pick from the talks.
Give processing some love[image: Screen Shot 2016-06-12 at 22.13.15]
During the hackfest and user conference, Victor Olaya (lead developer for
the QGIS processing framework) really did a great job of promoting the idea
of writing processing plugins rather than 'normal' plugins. His argument is
that most plugins that are intended to provide single purpose analytical
capabilities (we are looking at you geeks about to write the 300th buffer
plugin!) would be better off implemented as processing plugins:
- the plugin author would not need to spend valuable time writing user
interfaces, input validators and so on.
- users of the plugins could chain the tool into complex workflows
easily, rather than only being able to use it on a once off basis.
- we would grow the amount of options available in the processing
toolbox while at the same time reducing the sometimes overwhelming amount
of choice in the QGIS Plugin Manager.
ILWIS processing tools coming soon
Also on the topic of plugins, Bas Restsios from the ILWIS Project
<http://www.ilwis.org> gave a demo of the ILWIS software in order for QGIS
developers to be more aware of its capabilities. Although currently Windows
only, the ILWIS developers are in the process of porting the software to be
based on the Qt5 framework, which means us Linux and OSX users will get to
enjoy using it too soon. ILWIS is Open Source and does its rendering using
OpenGL. Because of their smart rendering system, everything draws lightning
fast. ILWIS packs in many remote sensing tools and raster analysis tools
and should be on anyone's radar if they are interested in FOSSGIS. The best
part (from my point of view) is that Bas and his team members are also busy
creating a set of processing plugins for QGIS that call out to ILWIS's
command line tools. This means you can expect a huge leap forward in the
number of raster based analysis functions you can do with QGIS in the near
future.
GeoPackage, JPEG2000, WFS improvements
It was really great to have Even Rouault (maintainer of GDAL/OGR) present
at the hackfest. Even recently received core committer rights to the QGIS
code repository and has been making great contributions by adding support
for the OGC Geopackage <http://www.geopackage.org> format (death to
shapefiles!). The shapefile format is long in the tooth and doesn't serve
the GIS community well as a *de facto* standard for GIS data interchange.
It also doesn't make a good format for intermediate representation of data
processing outputs (e.g. when using multi algorithm processing models).
Even has also been working on improvements to WFS
<https://github.com/qgis/QGIS-Enhancement-Proposals/issues/53> support in
QGIS which many will appreciate.
Not directly related to QGIS, but Even also mentioned he has been giving
the GDAL driver <http://www.gdal.org/frmt_jp2openjpeg.html> for JPEG2000
<http://www.opengeospatial.org/standards/gmljp2> some love - gaining good
performance increases. This gives me some hope that there will be a viable
open format alternative to ECW and MrSid in the future - something which
QGIS will benefit from greatly. One of the interesting things that GDAL
supports is embedded vectors - you can write the raster with gdal_translate
and then during creation pass a shapefile or GML stream to the
gdal_translate. I look forward to the day (not currently on the roadmap)
where we can ship JP2 images with embedded vector masks and use them
seamlessly in QGIS.
Planning for 3.0 release
We did some planning for the 3.0 release of QGIS and in particular fine
tuned the plans for how we will manage the transition from QGIS 2.x to 3.x.
In February 2016 I posted an outline
<http://blog.qgis.org/2016/02/10/qgis-3-0-plans/> of the general approach
we planned to follow. Some developers felt that we would be better off
having the core of the 3.0 transition work happening on the master branch
of QGIS so that it has more attention and testing focussed on it. I guess
there are two main interest groups to consider here, so I will break down
the outcome between take-home points for developers and general users:
*For developers:*
- 2.16 gets released off master.
- After 2.16 we create a 2x branch and the 2x branch will be in
caretaker mode.
- Support for Qt4 and Py2 will be discontinued in master and we do
packaging only against Qt5 and Py3.
- Packages may not be immediately available on master as we need to get
the packaging systems update.
- After 2.16 is released API breaking changes are allowed in master (but
code must build and packaging not broken, plugins will be broken).
- All API breaking changes should be annotated by means of a Doxygen
page patch to indicate what changes were made.
- We target the API change window to end in Feb 2017 so that we can have
a 3.0 release in March 2017.
- We may do a 2.18 release off the 2x branch if there have been
substantial changes in the 2x branch. If there are no substantial changes
in the 2x branch, we will not do a 2.18 release.
- In January 2017 we will have a review to establish if all the API
breaking changes are complete. If someone has specific implementation plans
that are in progress and they need more time, we will (under agreement from
the dev community) extend the API breaking window and push out the release
date.
- Following the release of 3.0 we will implement API freeze again and
polish up the codebase for an LTR release based on 3.2
- 2.14 LTR support will be extended until 3.2 is ready
- *Note:* for QGIS 3.x, a minimum of Qt 5.5 is recommended
- *Note:* for QGIS 3.x a minimum of Python 3.4 is recommended
- *Note:* We are looking for a volunteer to set up a windows build
system using vagrant on win 7 so that we can test and build automaticallyl
*Public messaging*
- 2.18 release is not guaranteed
- We are planning a 3.0 release in March 2017
- *This release date may be postponed* - we will do a review in January
to establish whether we are on track for the release date
- The 3.0 release will break your plugins - we will publish a migration
guide and tools to help you migrate your plugin to the new platform
- If you have any queries about the process please feel free to contact
us
Style repository
Akbar Gumbira, our Google Summer of Code (GSOC) student joined us at the
hackfest. For his GSOC project, Akbar is working on a unified way to share
styles <https://github.com/qgis/QGIS-Enhancement-Proposals/issues/58>,
symbols, ramps and markers in QGIS. The idea will be very similar to the
QGIS plugin repository, where users can host their favourite cartographic
elements for everyone else to enjoy. There was quite a lot of technical
discussion about the exact mechanisms to use for hosting shared symbology
with the key elements of the debate about whether to use git as a hosting
system, simple zip files or some other mechanism. If you wish to chat to
Akbar about his work, visit this chat room
<https://gitter.im/akbargumbira/qgis_symbology_sharing> or comment on the
QEP <https://github.com/qgis/QGIS-Enhancement-Proposals/issues/58>.
*OSX Packaging and Building*
QGIS is an attractive proposition for OSX users since the big commercial
vendors typically don't support OSX. Since QGIS is built on cross platform
technologies, this does not pose a huge limitation for us, but there are
few OSX developers in the QGIS developer team and the platform requirements
and constraints are pretty complex. Larry Shaffer has been doing awesome
work to cut through the various issues and simplify the installation
process - both for end users and for developers. It's finally pretty easy
to get a development environment up and running on a Mac now. Using brew (a
package manager for OSX) you can install all the needed dependencies and
get QGIS compiling in Qt Creator with full debugging support. The next
challenge is going to be supporting this under Qt5 for QGIS 3.0 and Larry
has been doing a bunch of great work to make that happen.
Lizmap!
There are several web client frontends for QGIS out there. Lizmap
<http://www.3liz.com/en/lizmap.html> is probably the most feature rich of
them. Michaël Douchin (@kimaidou on twitter) showed off the latest version
of Lizmap. If you are doing any web mapping, you really should check it out!
QField
Marco Bernasocchi showed off the work they have been doing on QField
<http://www.opengis.ch/android-gis/qfield/> - an Android app for your
mobile device. The workflow for using QField is simple: Create a QGIS
project on your desktop, copy it over to your device's SD Card or internal
storage, then open the same project on your device. Since it uses the same
QGIS 2.14 backend as you are probably running on your desktop, all the
cartographic elements from your desktop are supported - including the new
2.5D rendering. The main use case for QField is field data collection and
new in QField is the ability to capture point data and edit / update
feature attributes. This opens many possibilities. During the hackfest I
did some testing using a PostGIS layer in a project with the data coming
from a remote QGIS server - and it works (assuming you have internet
connectivity). I also did some testing using BTSync to create a
synchronised file system between my mobile device and my desktop. Edits to
layers on the desktop and the mobile device can easily be pushed back and
forth, making it very very easy to push out maps and new data to workers in
the field. Marco also shared some roadmap plans for QField including
broader support for form elements (especially value maps) and support for
digitising lines and polygons. I am eagerly looking forward to seeing how
QField develops!
Underwater autonomous vehicles & QGIS
[image: Natalia]
(With apologies for using the colloquial term 'drone' which I know Natália
does not like). One of the really cool things we did at the hackfest was
take a little side trip to visit the Computer Vision and Robotics Institute
<http://cirs.udg.edu>. Natália Hurtós (who works for the institute) kindly
gave a bunch of us QGIS geeks a tour. The work they are doing building
[relatively] cheap underwater autonomous vehicles is really awesome and
inspiring - all the more so because Natalia is planning to build the
mission planning tool in using the QGIS libraries!
Testing
There were lots of interesting things going on for those interested in
testing. Alessandro Pasotti showed off some really cool stuff he has been
working on for running python tests directly in QGIS instead of using a
mock QGIS iface object. There are huge benefits to doing this since your
tests run in a 'real' QGIS environment. He also showed off how they are
testing QGIS plugins in Docker <http://docker.com> using the above
mentioned technique.
If you do need / want to use an mock iface object, Matthias Kuhn has been
promoting the use of the new qgis.testing python module improvements which
includes an iface object which is comprehensive in terms of API stubs.
*from qgis.testing import start_app, unittest *
He has also created a unittest subclass that you can use which includes
nice goodies like letting you do asserts that geometries match.
Matthias Kuhn, Nyall Dawson and others have really been leading the charge
to build a more comprehensive test suite in QGIS and there were lots of
other interesting tips and tricks been shown like how to make your travis
tests run against multiple versions of QGIS - which can be very handy for
plugin authors.
Victor Olaya also showed off the tester plugin
<https://github.com/boundlessgeo/qgis-tester-plugin> - for automated GUI
testing based on recorded interaction sequences.
*Nice things for developers*
Martin Dobias showed off some of the tools they have been developing
including:
- The report plugin <http://plugins.qgis.org/plugins/report/> (see
http://plugins.qgis.org/plugins/report/
<https://github.com/lutraconsulting/qgis-report-plugin>) that lets you
trap python exceptions and send a bug report directly to a github issue
tracker.
- Martin also showed of the First Aid plugin
<http://plugins.qgis.org/plugins/firstaid/> which lets you capture
tracebacks in a more elegant way and also debug your plugin, stepping
through the code and execute python instructions in the current run
context. The First AID plugin makes a great alternative to using remote
debugging from an IDE which is time consuming to set up and technical for
less experienced developers to do.
*Training plugin*
Victor Olaya showed of a plugin (the Lessons plugin) they are working on to
facilitate interactive training in QGIS. It uses a simple dock interface to
guide the user through a series of activities. Interestingly you can ask it
to play recorded macros (saved as python code) of the active being
explained so that you can see how it is done if you can't figure it out
yourself. Their authoring system also allows you to record these macros.
*Docker*
Docker is increasingly becoming a useful building block for those wishing
to deploy QGIS in server side contexts. Patrick Valescc and Stéphane
Brunner did some really awesome work refactoring the QGIS Server docker
image - you can get a preview here
https://github.com/pvalsecc/QGIS/tree/docker (read the README
<https://github.com/pvalsecc/QGIS/tree/docker/docker> for usage notes). We
would like to eventually merge this into the upstream QGIS repo so that we
have a versioned docker build set up for each QGIS release.
*Crayfish*
Martin Dobias showed off the Crayfish plugin
<http://plugins.qgis.org/plugins/crayfish/>. Crayfish includes some really
awesome multi temporal visualisation tools - really useful if you have time
slice data in NetCDF or similar formats and you want to view layers
sequentially as animations. It also has cool symbology additions to show
flow / directionality arrows over your raster.
*Lightning talks*
Here is also a quick-fire list of features that were shown off -that are
coming down the QGIS conveyor belt - mostly for QGIS 2.16 but also new
plugins and other efforts:
- *Style dock* in QGIS 2.16 with undo / redo - Nathan Woodrow
- *Dynamic hillside rendering* - Nathan Woodrow
- Awesome new *gradient editor* - Nyall Dawson
- *D3 & Plotly* charting with chart interaction showing on the layer -
Matteo Ghetta and Michael Douchin
- *Forms improvements*: the 'drag and drop' editor will support multiple
columns - Matthias Kuhn
- Table view will be able to filter which columns to show (to let you *hide
unwanted columns*) - Matthias Kuhn
- *Action widget columns in the attribute table*: you can place one or
more action widgets (and icons) into the attribute table making it very
easy to fire off an action (e.g. python script) for a particular record -
Matthias Kuhn
- Support for *changing column order* display in attribute table -
Matthias Kuhn
- In composer you can get have* JSON of all layer relations* that you
can use in your html widget - Nyall Dawson
- *Aggregation in expressions*. We will explain this more in the visual
changeling for the next release, but you will be able to compute attribute
aggregates (e.g. sum of areas) in your reports - Nyall Dawson
- *Default values for fields* - Matthias Kuhn showed upcoming work to
add support for field defaults in when adding a new record to a vector
layer.
- *Transaction groups / cascaded editing mode* when you are editing a
feature so that related tables get put into edit mode (project property
needs to be enabled) - again we will explain this more in the upcoming 2.16
changelog - Matthias Kuhn
- *SectorPlot plugin:* a plugin to let you make 'pizza plots' of your
data - Raymond Nijssen and Richard Duivenvoorde
- Denis Rouzaud showed off work he has been doing to create a *standard
settings dialog* that you can use in your plugins. It has support for
different input types and layer chooser etc. They provide a settings dialog
base class that you can derive from in your plugin which will take care of
reading and writing your settings to Settings.
- *User profiles plugin* - lets you create customised user interface for
different user categories - Alex Bruy
- *Georeferenced PDFs and outputs from composer* (sorry not yet GeoPDF)
but you can use composer to create a PDF - and then add that into QGIS
just like any other raster layer. Nyall Dawson
- Nyall Dawson chatted about his work on *QGIS Task manager* - a tool to
let you easily create concurrent background tasks and show the progress of
each task in a simple UI. I believe he is looking for funding to help him
get this finished, so if you are interested please contact him.
- *Bivariate legends plugin* (not published yet). Thomas Gratier showed
off work he is doing to support the creation of bivariate legends.
*Albireo QGIS Fork*
[image: albireo]
Sandro Mani showed off an awesome new QGIS front end they have been working
on at Sourcepole. Their implementation creates a brand dew ribbon based UI
paradigm for QGIS (don't scoff, it's actually pretty cool!). They have
pared down the number of features and grouped functionality into discrete
areas, both the menus and ribbon icons changing based on the context of
what you are doing. You can see the source code here:
https://github.com/sourcepole/kadas-albireo. Here are some bullet notes I
made while watching his presentation:
- 3D Globe updates
- Redlining - you can sketch onto the map without saving those
geometries in a specific layer.
- Measuring & profile tools
- 3d feature extrusion onto the globe (and they have been doing a lot of
work to get the QGIS Globe implementation ready for production use)
- Shift-drag for zoom - just like you can do in OpenLayers and Leaflet
(Nyall then went and added this to QGIS for 2.16 - yay!)
- Switch coordinate reference systems used for cursor position display
easily from the status bar
There were many other cool features - some of all may make their way into
the mainstream QGIS desktop. If you are looking to support users who need a
simplified QGIS user interface, Albireo QGIS spin off is something to watch.
Conclusion
Girona was an absolutely awesome venue for the QGIS Conference and
Hackfest. I hope the participants of the conference gained good benefit
from the experience. The hackfest had around 50 participants and beyond the
above notes, there was such a lot going on including work on the
documentation system, discussions on the proposed community voter system
(more on that in a follow up blog post) and many other things. I really
encourage you to attend these events if you want to keep track of the
leading edge of QGIS developers.
--
------------------------------------------------------------------------------------------
Tim Sutton
Visit http://kartoza.com to find out about open source:
* Desktop GIS programming services
* Geospatial web development
* GIS Training
* Consulting Services
Skype: timlinux Irc: timlinux on #qgis at freenode.net
Tim is a member of the QGIS Project Steering Committee
-------------------------------------------------------------------------------------------
Kartoza is a merger between Linfiniti and Afrispatial
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-psc/attachments/20160629/a314b03d/attachment.html>
More information about the Qgis-psc
mailing list