<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Aptos;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:11.0pt;
font-family:"Aptos",sans-serif;
mso-ligatures:standardcontextual;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#467886;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Aptos",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:11.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="#467886" vlink="#96607D" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal"><span lang="PT">Hi,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="PT"><o:p> </o:p></span></p>
<p class="MsoNormal">Im trying to read files like <o:p></o:p></p>
<p class="MsoNormal"><a href="https://bulk.meteostat.net/v2/hourly/2022/08554.csv.gz">https://bulk.meteostat.net/v2/hourly/2022/08554.csv.gz</a><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">in my Julia wrapper. The point is that, although I’m kind off succeeding, the hole operation is very slow.<o:p></o:p></p>
<p class="MsoNormal">What I’m doing (code not committed yet so can’t post a link) is to read like this<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">layer = getlayer(dataset, 0)<o:p></o:p></p>
<p class="MsoNormal">for f in layer<o:p></o:p></p>
<p class="MsoNormal"> for k = 1: Gdal.nfield(f)<o:p></o:p></p>
<p class="MsoNormal"> Gdal.getfield(f, k-1)<o:p></o:p></p>
<p class="MsoNormal">…<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">This works but it’s extremely slow because each “getfield” takes about 1e-4 seconds and the file has ~8 k rows, each with 13 fields. That amounts to > 10 sec.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I’ve searched but couldn’t find a way to read the entire file at once (which takes 1e-2 seconds if I read it, locally, with a gzip wrapper) and return it as a single string array that I could parse later.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Is that possible? <o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Thanks<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Joaquim<o:p></o:p></p>
</div>
</body>
</html>