[gdal-dev] Read a /vsigzip/ csv.gz all at once

[Joaquim Manuel Freire Luís]

Hi,

Im trying to read files like
https://bulk.meteostat.net/v2/hourly/2022/08554.csv.gz

in my Julia wrapper. The point is that, although I'm kind off succeeding, the hole operation is very slow.
What I'm doing (code not committed yet so can't post a link) is to read like this

layer = getlayer(dataset, 0)
for f in layer
               for k = 1: Gdal.nfield(f)
                              Gdal.getfield(f, k-1)
...

This works but it's extremely slow because each "getfield" takes about 1e-4 seconds and the file has ~8 k rows, each with 13 fields. That amounts to > 10 sec.

I've searched but couldn't find a way to read the entire file at once (which takes 1e-2 seconds if I read it, locally, with a gzip wrapper) and return it as a single string array that I could parse later.

Is that possible?

Thanks

Joaquim
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20250624/5bd26729/attachment.htm>