<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<br>
Thank you.<br>
<br>
These building outlines don't produce fields that are extremely wide
<br>
(lots of characters on a single line). But I tend to store my
geoj's <br>
with each point on a single line, to prevent excessively wide lines.<br>
It makes a BIG difference for QGIS with the geoj's I work with. <br>
You might try that just to see if it makes a difference for you.<br>
<br>
So you could replace "],[" with "],\n[" (\n being a newline). The
file <br>
might get even bigger, but QGIS is much happier with the newlines
than <br>
with the wide rows.<br>
<br>
<br>
Mike<br>
<br>
<br>
<br>
<div class="moz-cite-prefix">On 7/29/21 9:26 AM, Newcomb, Doug
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:BLAPR09MB67863EA3A999B2449D93657DF1EB9@BLAPR09MB6786.namprd09.prod.outlook.com">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<style type="text/css" style="display:none;">P {margin-top:0;margin-bottom:0;}</style>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255,
255, 255);">
<a href="https://github.com/microsoft/USBuildingFootprints"
id="LPlnk" moz-do-not-send="true">https://github.com/microsoft/USBuildingFootprints</a><br>
</div>
<div class="_Entity _EType_OWALinkPreview _EId_OWALinkPreview
_EReadonly_1">
<div
id="LPBorder_GTaHR0cHM6Ly9naXRodWIuY29tL21pY3Jvc29mdC9VU0J1aWxkaW5nRm9vdHByaW50cw.."
class="LPBorder592208" style="width: 100%; margin-top: 16px;
margin-bottom: 16px; position: relative; max-width: 800px;
min-width: 424px;">
<table id="LPContainer592208" role="presentation"
style="padding: 12px 36px 12px 12px; width: 100%;
border-width: 1px; border-style: solid; border-color:
rgb(200, 200, 200); border-radius: 2px;">
<tbody>
<tr style="border-spacing: 0px;" valign="top">
<td>
<div id="LPImageContainer592208" style="position:
relative; margin-right: 12px; height: 120px;
overflow: hidden; width: 240px;">
<a target="_blank" id="LPImageAnchor592208"
href="https://github.com/microsoft/USBuildingFootprints"
moz-do-not-send="true"><img
id="LPThumbnailImageId592208" alt=""
style="display: block;"
src="https://opengraph.githubassets.com/225fd5de0d4b915c82edf14205ec82909692b31f97fa3453012808d51a8fd323/microsoft/USBuildingFootprints"
moz-do-not-send="true" width="240" height="120"></a></div>
</td>
<td style="width: 100%;">
<div id="LPTitle592208" style="font-size: 21px;
font-weight: 300; margin-right: 8px; font-family:
wf_segoe-ui_light, "Segoe UI Light",
"Segoe WP Light", "Segoe UI",
"Segoe WP", Tahoma, Arial, sans-serif;
margin-bottom: 12px;">
<a target="_blank" id="LPUrlAnchor592208"
href="https://github.com/microsoft/USBuildingFootprints"
style="text-decoration: none; color:
var(--themePrimary);" moz-do-not-send="true">GitHub
- microsoft/USBuildingFootprints: Computer
generated building footprints for the United
States</a></div>
<div id="LPDescription592208" style="font-size: 14px;
max-height: 100px; color: rgb(102, 102, 102);
font-family: wf_segoe-ui_normal, "Segoe
UI", "Segoe WP", Tahoma, Arial,
sans-serif; margin-bottom: 12px; margin-right: 8px;
overflow: hidden;">
Introduction. Microsoft Maps is releasing country
wide open building footprints datasets in United
States. This dataset contains 129,591,852 computer
generated building footprints derived using our
computer vision algorithms on satellite imagery.</div>
<div id="LPMetadata592208" style="font-size: 14px;
font-weight: 400; color: rgb(166, 166, 166);
font-family: wf_segoe-ui_normal, "Segoe
UI", "Segoe WP", Tahoma, Arial,
sans-serif;">
github.com</div>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255,
255, 255);">
Texas is 2.38 GB , but you get a variety of other sizes</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255,
255, 255);">
<br>
</div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font style="font-size:11pt"
face="Calibri, sans-serif" color="#000000"><b>From:</b>
gdal-dev <a class="moz-txt-link-rfc2396E" href="mailto:gdal-dev-bounces@lists.osgeo.org"><gdal-dev-bounces@lists.osgeo.org></a> on behalf of
Mike Flannigan <a class="moz-txt-link-rfc2396E" href="mailto:mflan@mflan.com"><mflan@mflan.com></a><br>
<b>Sent:</b> Thursday, July 29, 2021 9:49 AM<br>
<b>To:</b> <a class="moz-txt-link-abbreviated" href="mailto:gdal-dev@lists.osgeo.org">gdal-dev@lists.osgeo.org</a>
<a class="moz-txt-link-rfc2396E" href="mailto:gdal-dev@lists.osgeo.org"><gdal-dev@lists.osgeo.org></a><br>
<b>Subject:</b> [EXTERNAL] Re: [gdal-dev] Large GeoJSONs and
aborting file opening</font>
<div> </div>
</div>
<div class="BodyFragment"><font size="2"><span
style="font-size:11pt;">
<div class="PlainText"><br>
<br>
This email has been received from outside of DOI - Use
caution before clicking on links, opening attachments, or
responding.<br>
<br>
<br>
<br>
I would like to hear more about large GeoJSON files.<br>
How large are they?<br>
<br>
My GeoJSON files contain linear features only. My<br>
largest one is 50.2 MB with 1,230,000 newlines in it.<br>
Next biggest one is 12 MB with 280,000 newlines. These<br>
and about 140 other geojsons are open in the same project<br>
and I have no problems. In fact I converted from<br>
SHP to geojson 2 years ago because I used to have problems<br>
with SHP linear files.<br>
<br>
I use QGIS 3.16.8 on Linux Mint.<br>
<br>
<br>
Mike<br>
<br>
<br>
On 7/28/21 2:36 PM, <a class="moz-txt-link-abbreviated" href="mailto:gdal-dev-request@lists.osgeo.org">gdal-dev-request@lists.osgeo.org</a>
wrote:<br>
> Date: Wed, 28 Jul 2021 12:22:12 -0700<br>
> From: Simon Eves<a class="moz-txt-link-rfc2396E" href="mailto:simon.eves@omnisci.com"><simon.eves@omnisci.com></a><br>
> <a class="moz-txt-link-abbreviated" href="mailto:To:gdal-dev@lists.osgeo.org">To:gdal-dev@lists.osgeo.org</a><br>
> Subject: [gdal-dev] Large GeoJSONs and aborting file
opening<br>
> Message-ID:<br>
>
<a class="moz-txt-link-rfc2396E" href="mailto:CAJf0KTRsaskSOsPv8tbA+iTB+TqL_ui5y4n05WGLDw_3guRs4w@mail.gmail.com"><CAJf0KTRsaskSOsPv8tbA+iTB+TqL_ui5y4n05WGLDw_3guRs4w@mail.gmail.com></a><br>
> Content-Type: text/plain; charset="utf-8"<br>
><br>
> Dear All,<br>
><br>
> I am aware that some improvements were made in the
2.3 timeframe with<br>
> regards to dealing with large GeoJSON files, although
even in 3.2, it's<br>
> still very slow and memory hungry.<br>
><br>
> Our system allows for aborting imports, but this only
works reliably once<br>
> it has actually got to the stage of reading features
from the file. With<br>
> the GeoJSON, it just sits in the GDALOpenEx call for
ages.<br>
><br>
> My question, therefore, is whether it might be
practical to run the<br>
> GDALOpenEx in a separate thread with a future to
return the resulting<br>
> handle, such that it could be monitored and killed if
necessary?<br>
><br>
> Mainly I would be concerned that killing the thread
might trash some global<br>
> GDAL state that might then not be recoverable, or
that the open relies on<br>
> some TLS for the process thread and therefore might
not work properly.<br>
><br>
> We're going to try it anyway, but opinions welcomed,
thanks!<br>
><br>
> Simon<br>
<br>
<br>
_______________________________________________<br>
gdal-dev mailing list<br>
<a class="moz-txt-link-abbreviated" href="mailto:gdal-dev@lists.osgeo.org">gdal-dev@lists.osgeo.org</a><br>
<a
href="https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.osgeo.org%2Fmailman%2Flistinfo%2Fgdal-dev&data=04%7C01%7Cdoug_newcomb%40fws.gov%7C4a700f32558543c33b2308d9529c3a35%7C0693b5ba4b184d7b9341f32f400a5494%7C0%7C0%7C637631653236424668%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=86Xw3QEVbY0bqUurcBL8iLFi9IVjl0vGyErJUclKFFg%3D&reserved=0"
moz-do-not-send="true">https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.osgeo.org%2Fmailman%2Flistinfo%2Fgdal-dev&data=04%7C01%7Cdoug_newcomb%40fws.gov%7C4a700f32558543c33b2308d9529c3a35%7C0693b5ba4b184d7b9341f32f400a5494%7C0%7C0%7C637631653236424668%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=86Xw3QEVbY0bqUurcBL8iLFi9IVjl0vGyErJUclKFFg%3D&reserved=0</a><br>
</div>
</span></font></div>
</blockquote>
<br>
</body>
</html>