[Qgis-user] Convert huge shapefile to small size

[Richard Greenwood]

You could export the shapefile and use *Layer Options > RESIZE=YES*. This
is the same as the ogr2ogr switch *-lco RESIZE=YES*, which reduces the
attribute column widths down to their minimum width required to store
the longest actual value in the data. A dbf file allocates space for the
full column width even if it's empty. So for example if you have a
column width of 254 characters but the longest string is only 12
characters, your dbf will have a lot of wasted space in it. Resizing
the column would write out a new dbf with a column width of 12 characters.

On Thu, Jul 28, 2022 at 8:15 AM jhubbslist--- via Qgis-user <
qgis-user at lists.osgeo.org> wrote:

> Past a point, trying to run a GIS app with very large data structures just
> doesn't work well. If you really need all the information you're carrying
> around for your analysis or whatever, you may need to reach for different
> tools or use the tools you have differently.
>
> I assume it's not so much the >5GiB of disk space that's the issue and
> that you've maxxed out the CPU, graphics, and disk I/O rate as much as is
> practical so mostly it's a matter of how long it takes maps etc. to paint
> onscreen. It may help you to move the heavy-lift onto PostgreSQL/PostGIS
> where you can make use of spatial indexing. Or, you can craft your
> operations the way you want them in QGIS but do the actual work with e.g.
> GDAL calls in Python. I spoke with someone a couple weeks ago whose
> particular GIS process worked better in GRASS than in QGIS, so that's
> something you might look into as well.
>
> On 7/28/22 7:52 AM, krishna Ayyala via Qgis-user wrote:
>
> dbf file itself is 5.1GB. Rest all other files are less than 500MB. It is
> the number of records which is huge. It has about 117,2100 points.
>
> On Thu, Jul 28, 2022 at 12:02 AM Bernd Vogelgesang via Qgis-user <
> qgis-user at lists.osgeo.org> wrote:
>
>> how big is the dbf file of that shape? Maybe you can also drop some
>> attributes.
>>
>> Am 28.07.22 um 03:46 schrieb krishna Ayyala via Qgis-user:
>> > Hello,
>> > I have a shapefile of 5GB in size. Is it possible to convert this
>> > shapefile to a smaller size file? It can be any format, not
>> > necessarily a shapefile. But, preferably a vector format. I tried to
>> > convert it into tiles, but that didn't work as it was losing the
>> > resolution. I am looking to convert this 5GB size file to about 500MB.
>> >
>> > Regards.
>> >
>> > _______________________________________________
>> > Qgis-user mailing list
>> > Qgis-user at lists.osgeo.org
>> > List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
>> > Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user
>> _______________________________________________
>> Qgis-user mailing list
>> Qgis-user at lists.osgeo.org
>> List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
>> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user
>>
>
> _______________________________________________
> Qgis-user mailing listQgis-user at lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user
>
>
> _______________________________________________
> Qgis-user mailing list
> Qgis-user at lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user
>


-- 
Richard W. Greenwood
www.greenwoodmap.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-user/attachments/20220728/b8a03ca1/attachment.htm>