[Qgis-user] Search through folders and combine SHP files

[Nigel Berjak - S3 Technologies]

Hi

For your listing, although it would not provide the file type i.e. 
point, line, polygon etc, you could simply create a DOS batch command 
(dir/s *.shp > shapefilelist.txt) to locate all the file and generate a 
list of their locations. Then parse these through to the merge vector 
layer. If you are merging them all into a Geopackage, then file type 
would not be relevant and you could then sort them once they are in a 
single location. I don't think the Geopackage would have a limitation on 
the number of files and records in it.

Perhaps this can assist.

---
Regards,

Nigel Berjak
S3 Technologies
Geographic Information Systems & Large Format Printing specialists
T: +27 33 3423681
F: +27 86 6721242
E: nigel at S3.co.za
Website: http://www.S3.co.za

Please consider the environment before printing this email.

On 2024-05-10 04:06, Nyall Dawson via QGIS-User wrote:

> On Fri, 10 May 2024 at 09:06, Tony Shepherd (FarmMaps NZ) via QGIS-User 
> <qgis-user at lists.osgeo.org> wrote:
> 
>> Hi
>> 
>> I have loads, literally thousands, of SHP, GPKG, TAB files located in 
>> loads of different folders and sub-folders.
>> 
>> Is there a plugin that will trawl through a set of folders, copy say 
>> just the polygons from each file into a new file, and populate a field 
>> with say the filename and folder the polygon(s) came from?
>> 
>> Essentially combining a heap of files into one with some metadata 
>> about where the data came from?
>> 
>> Why?  I have files from clients dating back about 20 years.  Clients 
>> sell properties.  Properties get subdivided.  New clients come and see 
>> us, and often we don't realise we have GIS info on file for that 
>> property from many years ago, usually under a different name.  It 
>> would be handy to see a polygon on the screen and easily go back to 
>> some old GIS data.
>> 
>> Maybe I am missing an easy tool to do this, but I can't see anything 
>> obvious.
> 
> You could use the "Merge Vector Layers" tool from the Processing 
> toolbox, that does everything you want. The trickiest bit would be 
> getting a master list of ALL your shapefiles/etc you want to combine 
> and passing this to the tool. It's not going to be easy from the GUI 
> itself, as that only allows you to add files from a single folder at 
> once.
> 
> I'd run the tool on the files from a single folder, then checkout the 
> processing history dialog and copy the python command which corresponds 
> to what you just did. You could then adapt this command to pass a 
> complete list of all the files you want to combine. (It's quite 
> straightforward in Python to build a list of files which recursively 
> match a file pattern -- see eg https://stackoverflow.com/a/2186565 )
> 
> Hope that helps!
> Nyall
> 
>> Cheers
>> 
>> Tony
>> 
>> ------------------------------------------------------------------------------------------------------------------
>> 
>> Tony Shepherd
>> 
>> _  GeoSpatial Manager @ FarmMaps NZ & Photographer @ Shepherd Photos_
>> 
>> _Phone_ - 027 435 6193  | _Website_ - shepherdphotos.co.nz [1]
>> 
>> _Email_ - Maps tony at farmmaps.nz  |  _Email_ - Photos  
>> tony at shepherdphotos.co.nz
>> 
>> _Facebook_ TonyShepherdPhotos [2]
>> 
>> _______________________________________________
>> QGIS-User mailing list
>> QGIS-User at lists.osgeo.org
>> List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
>> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user
> 
> _______________________________________________
> QGIS-User mailing list
> QGIS-User at lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user


Links:
------
[1] http://shepherdphotos.co.nz/
[2] https://www.facebook.com/TonyShepherdPhotos
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-user/attachments/20240510/b10ec29c/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 3062 bytes
Desc: not available
URL: <http://lists.osgeo.org/pipermail/qgis-user/attachments/20240510/b10ec29c/attachment.jpg>