[Qgis-psc] Crowd source review of blog post for Gerona Hackfest

[Yves Jacolin]

Hi Tim,

Just two feedbacks:

* Patrick's lastname is Valsecchi not Valsecc :)

* Any way to add a few notes on documentation? We really need more 
contributors and so this is important to show that there is some work done.

FYI, we have only ~25 issues still open (more than 45 last week). We are 
working hard to close most of the ticket this week and the next one.

Thanks,

Y.
On Wednesday, June 29, 2016 13:06:15 Tim Sutton wrote:
> Hi All
> 
> Please find below my draft blog post for the Gerona hackfest. Sorry it has
> taken so long for me to write - it is rather long and there is a lot going
> on at the moment so I've struggled to find the spare time to work on it.
> Please let me know if you would like to add/remove/amend anything. I'd like
> to post it tomorrow morning.
> 
> ---
> 
> Time flies when you are having fun! It seems like only yesterday that
> I was writing
> about
> <http://blog.qgis.org/2015/11/15/a-word-of-thanks-to-the-hosts-of-the-14th-q
> gis-hackfest-in-gran-canaria/> the
> 14th Hackfest in Gran Canaria. At the end of May 2016, we held the 15th
> QGIS hackfest! QGIS has been on an incredible journey since the project was
> started by Gary Sherman 14 years ago, and the fact that the hackfest was
> held in tandem with the 2nd QGIS International User conference is a
> testament to the growth and strength of the project.
> 2nd International QGIS Conference
> 
> Isn't it amazing - we just held the *second *international QGIS User's
> Conference! We really need to give credit to the amazing team who ran a
> totally seamless operation to organise the event: Gemma Pons, Toni
> Hernández, Josep Sitjar, Alexandre Busquets, Ferran Orduña, Rosa Olivella,
> Laura Olivas, & Lluís Vicens
> 
> [image: Girona Organising Team]
> <https://qgisblog.files.wordpress.com/2016/06/screen-shot-2016-06-20-at-00-2
> 1-31.png>
> 
> We would like to give a special thank you to the University of
> Gerona's Director of GIS Service (SIGTE) - Gemma Boix who helped to
> organised the event as well as ensuring the institutional support for the
> event. In case I have missed mentioning someone by name, our thanks to all
> the other volunteers and the sponsors of the conference. OSGeo
> <http://osgeo.org> also supported the hack fest for which we are extremely
> grateful. Not only did the conference team host the conference event, they
> also covered a large part of the costs of the hackfest that followed the
> conference - for which we can't thank them enough!
> 
> The conference team also extends their thanks to the attendees and
> presenters, instructors and developers who also actively participated on
> the event!  For those interested in viewing the various talks at the
> conference, here are some handy links:
> 
>    - Sides from all talks
>   
> <http://www.sigte.udg.edu/jornadassiglibre/en/international-qgis-user-and-d
> eveloper-conference/conferencia-qgis/> - Videos for conference presentations
> <https://vimeo.com/sigteudg/albums>
> 
> 
> State of QGIS
> 
> The QGIS.org project is in a very healthy state right now. For my talk
> <https://qgisblog.files.wordpress.com/2016/06/qgisgironahackfesttalk.pdf> at
> the user conference (video here
> <http://diobma.udg.edu/handle/10256.1/4292>) I got some fresh download
> stats from our servers and the numbers are quite astounding: QGIS 2.8 LTR
> (release Feb 2015) has been downloaded over 679, 000 times for the Windows
> Standalone installer (which includes all bug fix releases). Even after
> controlling for overestimates (from bot downloads, multiple downloads per
> user, "try and don't use") and underestimates (single downloads being
> distributed to many users) it is clear that we have an extremely large and
> constantly growing user base. My apologies for not mentioning Gary Sherman
> (our project founder) by name at the start of my talk - that was totally
> unintentional!
> 
> Something else that is clear from the make up of the talks, workshop topics
> and attendees at the user conference and hackfest:* QGIS is increasingly
> moving from single user environments to large multi-user deployments.* This
> 'edge of network effect' is a common phenomenon in FOSS and is largely how
> Linux came to be such a lynchpin in the dev-ops world. Sys admins and (in
> the case of QGIS) GIS power users, test out the software on their own
> devices, see the potential for it in their workplace and start integrating
> it into their workflows in the office until eventually it has become a
> mission critical piece of software for an organisation.
> 
> One thing my slides probably do not make clear is that there is a huge
> amount of investment being made into QGIS and plugins for QGIS to fulfil a
> wide variety of needs. *These investments are largely external to the
> project* (i.e. not factored into the financial figures I mentioned in the
> talk) and happen in direct client-to-developer relationships completely
> bypassing (from a financial sense) the upstream  QGIS.ORG project. This is
> a really good model since we do not need to deal with contract delivery,
> competing interests etc.
> 
> If we do a simple calculation based on direct QGIS.org revenue for
> 2015/2016 (around EUR 69, 000) to downloads, the average revenue per
> download of QGIS 2.8.x was around EUR 0.10. All of the money we receive
> into the project goes into improving the QGIS, maintaining infrastructure
> and funding travel and accommodation for hackfests. I mention these numbers
> both because they are interesting and because it is good to emphasise how
> incredibly grateful we are to each and every one of our sponsors and donors
> that support the project. We really do run on a shoe-string budget and we
> have audacious goals and a vision to put spatial decision making tools into
> the hands of everyone on the planet who wants to use them. Your sponsorship
> and donations are a key enabler to making this vision a reality!
> 
> Ok you probably are more interested in the part below than the part above
> right? What actually happened at the hackfest? Lets dig in and find out!
> Cool stuff from the QGIS Hackfest
> 
> I am only going to focus on the hackfest here because the videos from the
> talks at the QGIS User Conference have been posted online (see above), so
> you can take in all the QGIS goodness you like from those. There was a lot
> going on at the hackfest, so these are only the nuggets I managed to
> cherry-pick from the talks.
> Give processing some love[image: Screen Shot 2016-06-12 at 22.13.15]
> 
> During the hackfest and user conference, Victor Olaya (lead developer for
> the QGIS processing framework) really did a great job of promoting the idea
> of writing processing plugins rather than 'normal' plugins. His argument is
> that most plugins that are intended to provide single purpose analytical
> capabilities (we are looking at you geeks about to write the 300th buffer
> plugin!) would  be better off implemented as processing plugins:
> 
>    - the plugin author would not need to spend valuable time writing user
>    interfaces, input validators and so on.
>    - users of the plugins could chain the tool into complex workflows
>    easily, rather than only being able to use it on a once off basis.
>    - we would grow the amount of options available in the processing
>    toolbox while at the same time reducing the sometimes overwhelming amount
> of choice in the QGIS Plugin Manager.
> 
> ILWIS processing tools coming soon
> 
> Also on the topic of plugins, Bas Restsios from the ILWIS Project
> <http://www.ilwis.org> gave a demo of the ILWIS software in order for QGIS
> developers to be more aware of its capabilities. Although currently Windows
> only, the ILWIS developers are in the process of porting the software to be
> based on the Qt5 framework, which means us Linux and OSX users will get to
> enjoy using it too soon. ILWIS is Open Source and does its rendering using
> OpenGL. Because of their smart rendering system, everything draws lightning
> fast. ILWIS packs in many remote sensing tools and raster analysis tools
> and should be on anyone's radar if they are interested in FOSSGIS. The best
> part (from my point of view) is that Bas and his team members are also busy
> creating a set of processing plugins for QGIS that call out to ILWIS's
> command line tools. This means you can expect a huge leap forward in the
> number of raster based analysis functions you can do with QGIS in the near
> future.
> GeoPackage, JPEG2000, WFS improvements
> 
> It was really great to have Even Rouault (maintainer of GDAL/OGR) present
> at the hackfest. Even recently received core committer rights to the QGIS
> code repository and has been making great contributions by adding support
> for the OGC Geopackage <http://www.geopackage.org> format (death to
> shapefiles!). The shapefile format is long in the tooth and doesn't serve
> the GIS community well as a *de facto* standard for GIS data interchange.
> It also doesn't make a good format for intermediate  representation of data
> processing outputs (e.g. when using multi algorithm processing models).
> Even has also been working on improvements to WFS
> <https://github.com/qgis/QGIS-Enhancement-Proposals/issues/53> support in
> QGIS which many will appreciate.
> 
> Not directly related to QGIS, but Even also mentioned he has been giving
> the GDAL driver <http://www.gdal.org/frmt_jp2openjpeg.html> for JPEG2000
> <http://www.opengeospatial.org/standards/gmljp2> some love - gaining good
> performance increases. This gives me some hope that there will be a viable
> open format alternative to ECW and MrSid in the future - something which
> QGIS will benefit from greatly. One of the interesting things that GDAL
> supports is embedded vectors - you can write the raster with gdal_translate
> and then during creation pass a shapefile or GML stream to the
> gdal_translate. I look forward to the day (not currently on the roadmap)
> where we can ship JP2 images with embedded vector masks and use them
> seamlessly in QGIS.
> Planning for 3.0 release
> 
> We did some planning for the 3.0 release of QGIS and in particular fine
> tuned the plans for how we will manage the transition from QGIS 2.x to 3.x.
> In February 2016 I posted an outline
> <http://blog.qgis.org/2016/02/10/qgis-3-0-plans/> of the general approach
> we planned to follow. Some developers felt that we would be better off
> having the core of the 3.0 transition work happening on the master branch
> of QGIS so that it has more attention and testing focussed on it. I guess
> there are two main interest groups to consider here, so I will break down
> the outcome between take-home points for developers and general users:
> 
> *For developers:*
> 
>    - 2.16 gets released off master.
>    - After 2.16 we create a 2x branch and the 2x branch will be in
>    caretaker mode.
>    - Support for Qt4 and Py2 will be discontinued in master and we do
>    packaging only against Qt5 and Py3.
>    - Packages may not be immediately available on master as we need to get
>    the packaging systems update.
>    - After 2.16 is released API breaking changes are allowed in master (but
>    code must build and packaging not broken, plugins will be broken).
>    - All API breaking changes should be annotated by means of a Doxygen
>    page patch to indicate what changes were made.
>    - We target the API change window to end in Feb 2017 so that we can have
>    a 3.0 release in March 2017.
>    - We may do a 2.18 release off the 2x branch  if there have been
>    substantial changes in the 2x branch. If there are no substantial changes
> in the 2x branch, we will not do a 2.18 release.
>    - In January 2017 we will have a review to establish if all the API
>    breaking changes are complete. If someone has specific implementation
> plans that are in progress and they need more time, we will (under
> agreement from the dev community) extend the API breaking window and push
> out the release date.
>    - Following the release of 3.0 we will implement API freeze again and
>    polish up the codebase for an LTR release based on 3.2
>    - 2.14 LTR support will be extended until 3.2 is ready
>    - *Note:* for QGIS 3.x, a minimum of Qt 5.5 is recommended
>    - *Note:* for QGIS 3.x a minimum of Python 3.4 is recommended
>    - *Note:* We are looking for a volunteer to set up a  windows build
>    system using vagrant on win 7 so that we can test and build
> automaticallyl
> 
> *Public messaging*
> 
>    - 2.18 release is not guaranteed
>    - We are planning a 3.0 release in March 2017
>    - *This release date may be postponed* - we will do a review in January
>    to establish whether we are on track for the release date
>    - The 3.0 release will break your plugins - we will publish a migration
>    guide and tools to help you migrate your plugin to the new platform
>    - If you have any queries about the process please feel free to contact
>    us
> 
> Style repository
> 
> Akbar Gumbira, our Google Summer of Code (GSOC) student joined us at the
> hackfest. For his GSOC project, Akbar is working on a unified way to share
> styles <https://github.com/qgis/QGIS-Enhancement-Proposals/issues/58>,
> symbols, ramps and markers in QGIS. The idea will be very similar to the
> QGIS plugin repository, where users can host their favourite cartographic
> elements for everyone else to enjoy. There was quite a lot of technical
> discussion about the exact mechanisms to use for hosting shared symbology
> with the key elements of the debate about whether to use git as a hosting
> system, simple zip files or some other mechanism. If you wish to chat to
> Akbar about his work, visit this chat room
> <https://gitter.im/akbargumbira/qgis_symbology_sharing> or comment on the
> QEP <https://github.com/qgis/QGIS-Enhancement-Proposals/issues/58>.
> 
> *OSX Packaging and Building*
> 
> QGIS is an attractive proposition for OSX users since the big commercial
> vendors typically don't support OSX. Since QGIS is built on cross platform
> technologies, this does not pose a huge limitation for us, but there are
> few OSX developers in the QGIS developer team and the platform requirements
> and constraints are pretty complex. Larry Shaffer has been doing awesome
> work to cut through the various issues and simplify the installation
> process - both for end users and for developers. It's finally pretty easy
> to get a development environment up and running on a Mac now. Using brew (a
> package manager for OSX) you can install all the needed dependencies and
> get  QGIS compiling in Qt Creator with full debugging support. The next
> challenge is going to be supporting this under Qt5 for QGIS 3.0 and Larry
> has been doing a bunch of great work to make that happen.
> Lizmap!
> 
> There are several web client frontends for QGIS out there. Lizmap
> <http://www.3liz.com/en/lizmap.html> is probably the most feature rich of
> them. Michaël Douchin (@kimaidou on twitter) showed off the latest version
> of Lizmap. If you are doing any web mapping, you really should check it out!
> QField
> 
> Marco Bernasocchi showed off the work they have been doing on QField
> <http://www.opengis.ch/android-gis/qfield/> - an Android app for your
> mobile device. The workflow for using QField is simple: Create a QGIS
> project on your desktop, copy it over to your device's SD Card or internal
> storage, then open the same project on your device. Since it uses the same
> QGIS 2.14 backend as you are probably running on your desktop, all the
> cartographic elements from your desktop are supported - including the new
> 2.5D rendering. The main use case for QField is field data collection and
> new in QField is the ability to capture point data and edit / update
> feature attributes. This opens many possibilities. During the hackfest I
> did some testing using a PostGIS layer in a project with the data coming
> from a remote QGIS server - and it works (assuming you have internet
> connectivity). I also did some testing using BTSync to create a
> synchronised file system between my mobile device and my desktop. Edits to
> layers on the desktop and the mobile device can easily be pushed back and
> forth, making it very very easy to push out maps and new data to workers in
> the field. Marco also shared some roadmap plans for QField including
> broader support for form elements (especially value maps) and support for
> digitising lines and polygons. I am eagerly looking forward to seeing how
> QField develops!
> Underwater autonomous vehicles & QGIS
> 
> [image: Natalia]
> 
> (With apologies for using the colloquial term 'drone' which I know Natália
> does not like). One of the really cool things we did at the hackfest was
> take a little side trip to visit the Computer Vision and Robotics Institute
> <http://cirs.udg.edu>. Natália Hurtós (who works for the institute) kindly
> gave a bunch of us QGIS geeks a tour. The work they are doing building
> [relatively] cheap underwater autonomous vehicles is really awesome and
> inspiring - all the more so because Natalia is planning to build the
> mission planning tool in using the QGIS libraries!
> Testing
> 
> There were lots of interesting things going on for those interested in
> testing. Alessandro Pasotti showed off some really cool stuff he has been
> working on for running python tests directly in QGIS instead of using a
> mock QGIS iface object. There are huge benefits to doing this since your
> tests run in  a 'real' QGIS environment. He also showed off how they are
> testing QGIS plugins in Docker <http://docker.com> using the above
> mentioned technique.
> 
> If you do need / want to use an mock iface object, Matthias Kuhn has been
> promoting the use of the new qgis.testing python module improvements which
> includes an iface object which is comprehensive in terms of API stubs.
> 
> *from qgis.testing import start_app, unittest *
> 
> He has also created a unittest subclass that you can use which includes
> nice goodies like letting you do asserts that geometries match.
> 
> Matthias Kuhn, Nyall Dawson and others have really been leading the charge
> to build a more comprehensive test suite in QGIS and there were lots of
> other interesting tips and tricks been shown like how to make your travis
> tests run against multiple versions of QGIS - which can be very handy for
> plugin authors.
> 
> Victor Olaya also showed off the tester plugin
> <https://github.com/boundlessgeo/qgis-tester-plugin> - for automated GUI
> testing based on recorded interaction sequences.
> 
> *Nice things for developers*
> 
> Martin Dobias showed off some of the tools they have been developing
> including:
> 
>    - The report plugin <http://plugins.qgis.org/plugins/report/> (see
>    http://plugins.qgis.org/plugins/report/
>    <https://github.com/lutraconsulting/qgis-report-plugin>) that lets you
>    trap python exceptions and send a bug report directly to a github issue
>    tracker.
>    - Martin also showed of the First Aid plugin
>    <http://plugins.qgis.org/plugins/firstaid/> which lets you capture
>    tracebacks in a more elegant way and also debug your plugin, stepping
>    through the code and execute python instructions in the current run
>    context. The First AID plugin makes a great alternative to using remote
>    debugging from an IDE which is time consuming to set up and technical for
> less experienced developers to do.
> 
> *Training plugin*
> 
> Victor Olaya showed of a plugin (the Lessons plugin) they are working on to
> facilitate interactive training in QGIS. It uses a simple dock interface to
> guide the user through a series of activities. Interestingly you can ask it
> to play recorded macros (saved as python code) of the active being
> explained so that you can see how it is done if you can't figure it out
> yourself. Their authoring system also allows you to record these  macros.
> 
> *Docker*
> 
> Docker is increasingly becoming a useful building block for those wishing
> to deploy QGIS in server side contexts. Patrick Valescc and Stéphane
> Brunner did some really awesome work refactoring the QGIS Server docker
> image - you can get a preview here
> https://github.com/pvalsecc/QGIS/tree/docker (read the README
> <https://github.com/pvalsecc/QGIS/tree/docker/docker> for usage notes). We
> would like to eventually merge this into the upstream QGIS repo so that we
> have a versioned docker build set up for each QGIS release.
> 
> *Crayfish*
> 
> Martin Dobias showed off the Crayfish plugin
> <http://plugins.qgis.org/plugins/crayfish/>. Crayfish includes some really
> awesome multi temporal visualisation tools - really useful if you have time
> slice data in NetCDF or similar formats and you want to view layers
> sequentially as animations. It also has cool symbology additions to show
> flow / directionality arrows over your raster.
> 
> 
> *Lightning talks*
> 
> Here is also a quick-fire list of features that were shown off -that are
> coming down the QGIS conveyor belt - mostly for QGIS 2.16 but also new
> plugins and other efforts:
> 
>    - *Style dock* in QGIS 2.16 with undo / redo - Nathan Woodrow
>    - *Dynamic hillside rendering* - Nathan Woodrow
>    - Awesome new *gradient editor* - Nyall Dawson
>    - *D3 & Plotly* charting with chart interaction showing on the layer -
>    Matteo Ghetta and Michael Douchin
>    - *Forms improvements*: the 'drag and drop' editor will support multiple
>    columns - Matthias Kuhn
>    - Table view will be able to filter which columns to show (to let you
> *hide unwanted columns*) - Matthias Kuhn
>    - *Action widget columns in the attribute table*: you can place one or
>    more action widgets (and icons) into the attribute table making it very
>    easy to fire off an action (e.g. python script) for a particular record -
> Matthias Kuhn
>    - Support for *changing column order* display in attribute table -
>    Matthias Kuhn
>    - In composer you can get have* JSON of all layer relations* that you
>    can use in your html widget - Nyall Dawson
>    - *Aggregation in expressions*. We will explain this more in the visual
>    changeling for the next release, but you will be able to compute
> attribute aggregates (e.g. sum of areas) in your reports - Nyall Dawson
>    - *Default values for fields* - Matthias Kuhn showed upcoming work to
>    add support for field defaults in when adding a new record to a vector
>    layer.
>    - *Transaction groups / cascaded editing mode* when you are editing a
>    feature so that related tables get put into edit mode (project property
>    needs to be enabled) - again we will explain this more in the upcoming
> 2.16 changelog - Matthias Kuhn
>    - *SectorPlot plugin:* a plugin to let you make 'pizza plots' of your
>    data - Raymond Nijssen and Richard Duivenvoorde
>    - Denis Rouzaud showed off work he has been doing to create a *standard
>    settings dialog* that you can use in your plugins. It has support for
>    different input types and layer chooser etc. They provide a settings
> dialog base class that you can derive from in your plugin which will take
> care of reading and writing your settings to Settings.
>    - *User profiles plugin* - lets you create customised user interface for
>    different user categories - Alex Bruy
>    - *Georeferenced PDFs and outputs from composer* (sorry not yet GeoPDF)
>    but you can use composer to create a  PDF - and then add that into QGIS
>    just like any other raster layer. Nyall Dawson
>    - Nyall Dawson chatted about his work on *QGIS Task manager* - a tool to
>    let you easily create concurrent background tasks and show the progress
> of each task in a simple UI. I believe he is looking for funding to help
> him get this finished, so if you are interested please contact him. -
> *Bivariate legends plugin* (not published yet).  Thomas Gratier showed off
> work he is doing to support the creation of bivariate legends.
> 
> *Albireo QGIS Fork*
> 
> [image: albireo]
> 
> Sandro Mani showed off an awesome new QGIS front end they have been working
> on at Sourcepole. Their implementation creates a brand dew ribbon based UI
> paradigm for QGIS (don't scoff, it's actually pretty cool!). They have
> pared down the number of features and grouped functionality into discrete
> areas, both the menus and ribbon icons changing based on the context of
> what you are doing. You can see the source code here:
> https://github.com/sourcepole/kadas-albireo.  Here are some bullet notes I
> made while watching his presentation:
> 
>    - 3D Globe updates
>    - Redlining - you can sketch onto the map without saving those
>    geometries in a specific layer.
>    - Measuring & profile tools
>    - 3d feature extrusion onto the globe (and they have been doing a lot of
>    work to get the QGIS Globe implementation ready for production use)
>    - Shift-drag for zoom - just like you can do in OpenLayers and Leaflet
>    (Nyall then went and added this to QGIS for 2.16 - yay!)
>    - Switch coordinate reference systems used for cursor position display
>    easily from the status bar
> 
> There were many other cool features - some of all may make their way into
> the mainstream QGIS desktop. If you are looking to support users who need a
> simplified QGIS user interface, Albireo QGIS spin off is something to watch.
> Conclusion
> 
> Girona was an absolutely awesome venue for the QGIS Conference and
> Hackfest. I hope the participants of the conference gained good benefit
> from the experience. The hackfest had around 50 participants and beyond the
> above notes, there was such a lot going on including work on the
> documentation system, discussions on the proposed community voter system
> (more on that in a follow up blog post) and many other things. I really
> encourage you to attend these events if you want to keep track of the
> leading edge of QGIS developers.