[Qgis-user] Best practice, database vs WFS

Wed Jan 6 08:10:39 PST 2021

Hi,

I want to add my 2 cents (all AFAIK):
WFS is a standard for data exchange as has been pointed out by 
Alessandro already. The data are in an XML format (GML) which is _slow_ 
compared to data from a database for several reasons. QGIS downloads the 
complete layer as GML which, depending on the amount of datasets, can be 
huge. There is no spatial index helping QGIS in rendering. Using a layer 
from a database is fast because - assuming you created a spatial index - 
QGIS only loads the data needed for the particular part of the map you 
are currently viewing.

AFAIU you are running different database systems in you network (or 
different instances of the same system or different databases on one 
system). Depending on the amount of layers e.g. PostreSQL's concept of 
foreign data wrappers might be a solution (I do not know anything about 
performance though) enabling you to bundle all your layers in one 
PostgreSQL database.

Apart from the technical perspective: If you have e.g. several hundred 
layers you will always face the problem of how users know where to find 
a particular layer, even if theses layers are all in the same database 
(same would be true for several hundred WFS layers on one server btw). 
In this case you will always need some kind of metadata system where 
users can search for data and that tells them how they can access them.

regards
Bernhard

Am 06.01.2021 um 16:48 schrieb Paul Wittle:
> Hi Alessandro,
> 
> I guess the key is ease of data discovery. So if all your geodata is on one database then it is all easy enough but if you have multiple database types / instances then you are relying a lot on users being able to find the right database / instance and then having potentially different login mechanisms for each database.
> 
> This is why the idea of WFS seems appealing because we can authenticate at access to the WFS server using a single method; expose all the accessible datasets in a single list and in theory potentially employ greater consistency to the method of updates.
> 
> We have found that the SQL issued by each database driver can vary in terms of the SQL optimisation because each one is developed independently of others (i.e. the Oracle data access is not necessary developed with much reference to the Postgres or SQL database data access clients). Whilst fundamentally the SQL statements are still in effect translated into the appropriate SQL for each database type the original statement should in theory be more consistent?
> 
> In terms of speed; I suspect you are definitely correct hence my asking the question really.
> 
> The comments above are really to simply flesh out the question as to whether or not using a database is really simpler though.
> 
> That said; we may also use something like a custom plugin or GeoNetwork as a data discovery tool which is another way of helping users add the right layer without needing to know which database it comes from.
> 
> I hope that clarifies the question a little better?
> 
> Cheers,
> Paul
> 
> -----Original Message-----
> From: Alessandro Pasotti <apasotti at gmail.com>
> Sent: 06 January 2021 15:11
> To: Paul Wittle <P.Wittle at dorsetcc.gov.uk>
> Cc: qgis-user at lists.osgeo.org
> Subject: Re: [Qgis-user] Best practice, database vs WFS
> 
> Hi Paul,
> 
> if you want to share geodata within your organization on a private network a database is the best solution: faster and simpler. The constraint is that you will need an application (like QGIS) to access your data..
> 
> WFS is a web service standard for interoperability, it is ideal for sharing data on the internet over HTTP, there is no need for a particular application to use the service: any HTTP client is able to do that.
> 
> Regards
> 
> 
> On Wed, Jan 6, 2021 at 3:43 PM Paul Wittle <paul.wittle at dorsetcouncil.gov.uk> wrote:
>>
>> Hi,
>>
>>
>>
>> As I’m sure is clear from the number of posts I’ve done of late we are currently looking at how we use QGIS within our business. I thought I’d ask a question here to see if others are considering it as I can’t find too much chat online about it but I wondered if perhaps there should be some.
>>
>>
>>
>> We have concluded that in theory the WFS and WFS-T protocols are an OGC standard (https://www.ogc.org/standards/wfs) and that using an OGC compliant server they can be used to front various data source formats; i.e. Postgres, Oracle, SQL Server etc. In theory that means that if you use WFS and WFS-T in QGIS it should mean that user experience becomes more consistent for the people using those layers in QGIS.
>>
>>
>>
>> That all sounds great, but I can’t seem to even get my WFS to load correctly in QGIS at present and it doesn’t seem to be something that is recommended online. Given that both WFS and direct database access both return full details (vector geometries and attributes) to QGIS; would you expect performance of WFS to be similar or significantly slower?
>>
>>
>>
>> Is the use of OGC compliant WFS something that you personally feel is something we should be aspiring to use more widely at the local / network level in QGIS or do you favour just loading directly from databases?
>>
>>
>>
>> I’m honestly very interested to hear what others think on this as theoretically you would think the creation of an OGC standard would have this sort of aspiration but I’m increasingly concluding that this kind of use of WFS is very limited. It seems to me that the most common use case is just for occasional layers where you need to work with others over the internet.
>>
>>
>>
>> To ensure we are talking about the same thing; I’m thinking that the access to WFS in this context would be locally within your own network or device as clearly going over the internet will add a significant overhead and potential for delay.
>>
>>
>>
>> Feel free to message me back directly or message the group if you think it is a worthwhile discussion but as I say I’d love to hear what others think.
>>
>>
>>
>> Cheers,
>>
>> Paul
>>
>>
>>
>>
>>
>> This e-mail and any files transmitted with it are intended solely for
>> the use of the individual or entity to whom they are addressed. It may
>> contain unclassified but sensitive or protectively marked material and
>> should be handled accordingly. Unless you are the named addressee (or
>> authorised to receive it for the addressee) you may not copy or use
>> it, or disclose it to anyone else. If you have received this
>> transmission in error please notify the sender immediately. All
>> traffic may be subject to recording and/or monitoring in accordance
>> with relevant legislation. Any views expressed in this message are
>> those of the individual sender, except where the sender specifies and
>> with authority, states them to be the views of Dorset Council. Dorset
>> Council does not accept service of documents by fax or other
>> electronic means. Virus checking: Whilst all reasonable steps have
>> been taken to ensure that this electronic communication and its
>> attachments whether encoded, encrypted or otherwise supplied are free
>> from computer viruses, Dorset Council accepts no liability in respect
>> of any loss, cost, damage or expense suffered as a result of accessing
>> this message or any of its attachments. For information on how Dorset
>> Council processes your information, please see
>> www.dorsetcouncil.gov.uk/416433
>> _______________________________________________
>> Qgis-user mailing list
>> Qgis-user at lists.osgeo.org
>> List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
>> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user
> 
> 
> 
> --
> Alessandro Pasotti
> QCooperative:  www.qcooperative.net
> ItOpen:   www.itopen.it
> This e-mail and any files transmitted with it are intended solely for the use of the individual or entity to whom they are addressed. It may contain unclassified but sensitive or protectively marked material and should be handled accordingly. Unless you are the named addressee (or authorised to receive it for the addressee) you may not copy or use it, or disclose it to anyone else. If you have received this transmission in error please notify the sender immediately. All traffic may be subject to recording and/or monitoring in accordance with relevant legislation. Any views expressed in this message are those of the individual sender, except where the sender specifies and with authority, states them to be the views of Dorset Council. Dorset Council does not accept service of documents by fax or other electronic means. Virus checking: Whilst all reasonable steps have been taken to ensure that this electronic communication and its attachments whether encoded, encrypted or otherwise supplied are free from computer viruses, Dorset Council accepts no liability in respect of any loss, cost, damage or expense suffered as a result of accessing this message or any of its attachments. For information on how Dorset Council processes your information, please see www.dorsetcouncil.gov.uk/416433
> _______________________________________________
> Qgis-user mailing list
> Qgis-user at lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user
> 