<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p
{mso-style-priority:99;
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
pre
{mso-style-priority:99;
mso-style-link:"HTML - wstępnie sformatowany Znak";
margin:0cm;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";}
span.HTML-wstpniesformatowanyZnak
{mso-style-name:"HTML - wstępnie sformatowany Znak";
mso-style-priority:99;
mso-style-link:"HTML - wstępnie sformatowany";
font-family:Consolas;}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-style-priority:99;
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
span.Stylwiadomocie-mail21
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:70.85pt 70.85pt 70.85pt 70.85pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="PL" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US">Hi Even,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US">Thanks for your replay. More info about the problem:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US">Error:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US">
<br>
<i>Save error:<o:p></o:p></i></span></p>
<p class="MsoNormal"><i><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US">Export to vector file failed.<o:p></o:p></span></i></p>
<p class="MsoNormal"><i><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US">Error: Feature write errors:<o:p></o:p></span></i></p>
<p class="MsoNormal"><i><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US">Feature creation error (OGR error: negative FID are not supported)<o:p></o:p></span></i></p>
<p class="MsoNormal"><i><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US"><o:p> </o:p></span></i></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US">The database has more than 2 bilons of records (record – I mean record in the database table). The error occurs when I export a single
table witout the primary key to .xlsx. <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US">I suppose that when the table doesn’t have the primary key it use tid – the tuple identifier, and the size of tid may exceed the size
of a 32-bit integer. It cause the problem to export this table to .xlsx format by OGR library.
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US">Regards,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US">Marta Kopszak<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> gdal-dev <gdal-dev-bounces@lists.osgeo.org>
<b>On Behalf Of </b>Even Rouault<br>
<b>Sent:</b> Monday, January 10, 2022 11:42 AM<br>
<b>To:</b> gdal-dev@lists.osgeo.org<br>
<b>Subject:</b> Re: [gdal-dev] Bug in OGR library<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p>Marta,<o:p></o:p></p>
<p>What is the bug exactly ?<o:p></o:p></p>
<p>"2 billion records" : by record, you mean cells, like a sheet with 2048 columns and 1048576 rows ? By quickly looking at the code I can't see an obvious limitation, but that doesn't mean anything. Closer investigation would be needed. Please file an issue
at <a href="https://github.com/OSGeo/gdal/issues/new">https://github.com/OSGeo/gdal/issues/new</a> with exact details how to reproduce (number of rows/features, and columns/fields in particular).<o:p></o:p></p>
<p>But even if the writer side would work, the OGR reader side would probably have issues as the whole spreadsheet will be loaded into RAM and for 2 billion cells, I would expect at least 32 or more GB of RAM to be needed. Perhaps spreadsheet software are able
to cope with that in a smarter way.<o:p></o:p></p>
<p>Even<o:p></o:p></p>
<div>
<p class="MsoNormal">Le 10/01/2022 à 11:27, Kopszak Marta - Partner Hurt a écrit :<o:p></o:p></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Dear Sirs,<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">I ran into an error when exporting data through the OGR library to the .xlsx and .ods format. The bug occurs when there are more than 2 billion records in the database. I know the
problem was solved for other formats, e.g. csv. Will the problem be solved also for other formats (e.g. xlsx, .ods)? If this is not the correct e-mail, please forward this e-mail to the appropriate person.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Kind regards,<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Marta Kopszak<o:p></o:p></p>
<table class="MsoNormalTable" border="0" cellspacing="3" cellpadding="0">
<tbody>
<tr>
<td style="padding:.75pt .75pt .75pt .75pt"></td>
</tr>
<tr>
<td style="padding:.75pt .75pt .75pt .75pt">
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><a href="https://www.orange.pl/" target="_blank"><b><span style="color:windowtext;text-decoration:none">Marta Kopszak</span></b><span style="color:windowtext;text-decoration:none"><br>
Współpracownik</span></a><span class="MsoHyperlink"><span style="color:windowtext;text-decoration:none"><o:p></o:p></span></span></p>
</td>
</tr>
<tr>
<td style="padding:.75pt .75pt .75pt .75pt">
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><a href="https://www.orange.pl/" target="_blank"><span style="color:windowtext;text-decoration:none">Rozwój Sieci Stacjonarnej, Wydział Rozwoju Planowania<br>
Orange Polska, gen. Romualda Traugutta 55, 50-416 Wrocław | <u><span style="color:blue">RODO - informacja o danych</span></u>
</span></a><span class="MsoHyperlink"><span style="color:windowtext;text-decoration:none"><o:p></o:p></span></span></p>
</td>
</tr>
</tbody>
</table>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><a href="https://www.orange.pl/" target="_blank"><span style="color:windowtext;text-decoration:none"> </span></a><span class="MsoHyperlink"><span style="color:windowtext;text-decoration:none"><o:p></o:p></span></span></p>
</div>
<p class="MsoNormal"><a href="https://www.orange.pl/" target="_blank"><span style="color:windowtext;text-decoration:none"><br>
</span></a><span class="MsoHyperlink"><span style="color:windowtext;text-decoration:none"><o:p></o:p></span></span></p>
<pre><a href="https://www.orange.pl/" target="_blank"><span style="color:windowtext;text-decoration:none">_______________________________________________</span></a><span class="MsoHyperlink"><span style="color:windowtext;text-decoration:none"><o:p></o:p></span></span></pre>
<pre><a href="https://www.orange.pl/" target="_blank"><span style="color:windowtext;text-decoration:none">gdal-dev mailing list</span></a><span class="MsoHyperlink"><span style="color:windowtext;text-decoration:none"><o:p></o:p></span></span></pre>
<pre><a href="https://www.orange.pl/" target="_blank"><span style="color:windowtext;text-decoration:none"><u><span style="color:blue">gdal-dev@lists.osgeo.org</span></u></span></a><span class="MsoHyperlink"><span style="color:windowtext;text-decoration:none"><o:p></o:p></span></span></pre>
<pre><a href="https://www.orange.pl/" target="_blank"><span style="color:windowtext;text-decoration:none"><u><span style="color:blue">https://lists.osgeo.org/mailman/listinfo/gdal-dev</span></u></span></a><span class="MsoHyperlink"><span style="color:windowtext;text-decoration:none"><o:p></o:p></span></span></pre>
</blockquote>
<pre><a href="https://www.orange.pl/" target="_blank"><span style="color:windowtext;text-decoration:none">-- </span></a><span class="MsoHyperlink"><span style="color:windowtext;text-decoration:none"><o:p></o:p></span></span></pre>
<pre><a href="https://www.orange.pl/" target="_blank"><span style="color:windowtext;text-decoration:none"><u><span style="color:blue">http://www.spatialys.com</span></u></span></a><span class="MsoHyperlink"><span style="color:windowtext;text-decoration:none"><o:p></o:p></span></span></pre>
<pre><a href="https://www.orange.pl/" target="_blank"><span style="color:windowtext;text-decoration:none">My software is free, but my time generally not.</span></a><span class="MsoHyperlink"><span style="color:windowtext;text-decoration:none"><o:p></o:p></span></span></pre>
</div>
</body>
</html>