[Qgis-user] Search through folders and combine SHP files

Sun May 12 14:41:03 PDT 2024

Hi Nigel

Thank you, that works a treat, although it does fail if I try to merge a line feature at the same time as a polygon.  That is not a big deal and can get around that.

Your comment, “Then parse these through to the merge vector layer”.  I can’t see a way of parsing the filelist.txt to the tool. I am not experienced in using python.

Cheers

TIby

From: Nigel Berjak - S3 Technologies <nigel at s3.co.za <mailto:nigel at s3.co.za> > 
Sent: Friday, May 10, 2024 6:03 PM
To: Nyall Dawson <nyall.dawson at gmail.com <mailto:nyall.dawson at gmail.com> >
Cc: Tony Shepherd (FarmMaps NZ) <tony at farmmaps.nz <mailto:tony at farmmaps.nz> >; qgis-user at lists.osgeo.org <mailto:qgis-user at lists.osgeo.org> 
Subject: Re: [Qgis-user] Search through folders and combine SHP files

Hi

For your listing, although it would not provide the file type i.e. point, line, polygon etc, you could simply create a DOS batch command (dir/s *.shp > shapefilelist.txt) to locate all the file and generate a list of their locations. Then parse these through to the merge vector layer. If you are merging them all into a Geopackage, then file type would not be relevant and you could then sort them once they are in a single location. I don't think the Geopackage would have a limitation on the number of files and records in it.

Perhaps this can assist.

---

Regards,

Nigel Berjak
S3 Technologies
Geographic Information Systems & Large Format Printing specialists
T: +27 33 3423681
F: +27 86 6721242
E:  <mailto:nigel at S3.co.za> nigel at S3.co.za
Website:  <http://www.S3.co.za> http://www.S3.co.za

Please consider the environment before printing this email.

On 2024-05-10 04:06, Nyall Dawson via QGIS-User wrote:

On Fri, 10 May 2024 at 09:06, Tony Shepherd (FarmMaps NZ) via QGIS-User < <mailto:qgis-user at lists.osgeo.org> qgis-user at lists.osgeo.org> wrote:

Hi

I have loads, literally thousands, of SHP, GPKG, TAB files located in loads of different folders and sub-folders.

Is there a plugin that will trawl through a set of folders, copy say just the polygons from each file into a new file, and populate a field with say the filename and folder the polygon(s) came from?

Essentially combining a heap of files into one with some metadata about where the data came from?

Why?  I have files from clients dating back about 20 years.  Clients sell properties.  Properties get subdivided.  New clients come and see us, and often we don't realise we have GIS info on file for that property from many years ago, usually under a different name.  It would be handy to see a polygon on the screen and easily go back to some old GIS data.

Maybe I am missing an easy tool to do this, but I can't see anything obvious.

You could use the "Merge Vector Layers" tool from the Processing toolbox, that does everything you want. The trickiest bit would be getting a master list of ALL your shapefiles/etc you want to combine and passing this to the tool. It's not going to be easy from the GUI itself, as that only allows you to add files from a single folder at once.

I'd run the tool on the files from a single folder, then checkout the processing history dialog and copy the python command which corresponds to what you just did. You could then adapt this command to pass a complete list of all the files you want to combine. (It's quite straightforward in Python to build a list of files which recursively match a file pattern -- see eg  <https://stackoverflow.com/a/2186565> https://stackoverflow.com/a/2186565 )

Hope that helps!

Nyall

Cheers

Tony

------------------------------------------------------------------------------------------------------------------

Tony Shepherd

  GeoSpatial Manager @ FarmMaps NZ & Photographer @ Shepherd Photos

Phone – 027 435 6193  | Website –  <http://shepherdphotos.co.nz/> shepherdphotos.co.nz

Email – Maps  <mailto:tony at farmmaps.nz> tony at farmmaps.nz  |  Email – Photos   <mailto:tony at shepherdphotos.co.nz> tony at shepherdphotos.co.nz

Facebook  <https://www.facebook.com/TonyShepherdPhotos> TonyShepherdPhotos

_______________________________________________
QGIS-User mailing list
 <mailto:QGIS-User at lists.osgeo.org> QGIS-User at lists.osgeo.org
List info:  <https://lists.osgeo.org/mailman/listinfo/qgis-user> https://lists.osgeo.org/mailman/listinfo/qgis-user
Unsubscribe:  <https://lists.osgeo.org/mailman/listinfo/qgis-user> https://lists.osgeo.org/mailman/listinfo/qgis-user

_______________________________________________
QGIS-User mailing list
 <mailto:QGIS-User at lists.osgeo.org> QGIS-User at lists.osgeo.org
List info:  <https://lists.osgeo.org/mailman/listinfo/qgis-user> https://lists.osgeo.org/mailman/listinfo/qgis-user
Unsubscribe:  <https://lists.osgeo.org/mailman/listinfo/qgis-user> https://lists.osgeo.org/mailman/listinfo/qgis-user

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-user/attachments/20240513/f9b00b2b/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 3062 bytes
Desc: not available
URL: <http://lists.osgeo.org/pipermail/qgis-user/attachments/20240513/f9b00b2b/attachment.jpg>