<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
<a href="https://github.com/microsoft/USBuildingFootprints" id="LPlnk">https://github.com/microsoft/USBuildingFootprints</a><br>
</div>
<div class="_Entity _EType_OWALinkPreview _EId_OWALinkPreview _EReadonly_1">
<div id="LPBorder_GTaHR0cHM6Ly9naXRodWIuY29tL21pY3Jvc29mdC9VU0J1aWxkaW5nRm9vdHByaW50cw.." class="LPBorder592208" style="width: 100%; margin-top: 16px; margin-bottom: 16px; position: relative; max-width: 800px; min-width: 424px;">
<table id="LPContainer592208" role="presentation" style="padding: 12px 36px 12px 12px; width: 100%; border-width: 1px; border-style: solid; border-color: rgb(200, 200, 200); border-radius: 2px;">
<tbody>
<tr valign="top" style="border-spacing: 0px;">
<td>
<div id="LPImageContainer592208" style="position: relative; margin-right: 12px; height: 120px; overflow: hidden; width: 240px;">
<a target="_blank" id="LPImageAnchor592208" href="https://github.com/microsoft/USBuildingFootprints"><img id="LPThumbnailImageId592208" alt="" height="120" style="display: block;" width="240" src="https://opengraph.githubassets.com/225fd5de0d4b915c82edf14205ec82909692b31f97fa3453012808d51a8fd323/microsoft/USBuildingFootprints"></a></div>
</td>
<td style="width: 100%;">
<div id="LPTitle592208" style="font-size: 21px; font-weight: 300; margin-right: 8px; font-family: wf_segoe-ui_light, "Segoe UI Light", "Segoe WP Light", "Segoe UI", "Segoe WP", Tahoma, Arial, sans-serif; margin-bottom: 12px;">
<a target="_blank" id="LPUrlAnchor592208" href="https://github.com/microsoft/USBuildingFootprints" style="text-decoration: none; color: var(--themePrimary);">GitHub - microsoft/USBuildingFootprints: Computer generated building footprints for the United States</a></div>
<div id="LPDescription592208" style="font-size: 14px; max-height: 100px; color: rgb(102, 102, 102); font-family: wf_segoe-ui_normal, "Segoe UI", "Segoe WP", Tahoma, Arial, sans-serif; margin-bottom: 12px; margin-right: 8px; overflow: hidden;">
Introduction. Microsoft Maps is releasing country wide open building footprints datasets in United States. This dataset contains 129,591,852 computer generated building footprints derived using our computer vision algorithms on satellite imagery.</div>
<div id="LPMetadata592208" style="font-size: 14px; font-weight: 400; color: rgb(166, 166, 166); font-family: wf_segoe-ui_normal, "Segoe UI", "Segoe WP", Tahoma, Arial, sans-serif;">
github.com</div>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
Texas is 2.38 GB , but you get a variety of other sizes</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
<br>
</div>
<div id="appendonsend"></div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> gdal-dev <gdal-dev-bounces@lists.osgeo.org> on behalf of Mike Flannigan <mflan@mflan.com><br>
<b>Sent:</b> Thursday, July 29, 2021 9:49 AM<br>
<b>To:</b> gdal-dev@lists.osgeo.org <gdal-dev@lists.osgeo.org><br>
<b>Subject:</b> [EXTERNAL] Re: [gdal-dev] Large GeoJSONs and aborting file opening</font>
<div> </div>
</div>
<div class="BodyFragment"><font size="2"><span style="font-size:11pt;">
<div class="PlainText"><br>
<br>
This email has been received from outside of DOI - Use caution before clicking on links, opening attachments, or responding.<br>
<br>
<br>
<br>
I would like to hear more about large GeoJSON files.<br>
How large are they?<br>
<br>
My GeoJSON files contain linear features only. My<br>
largest one is 50.2 MB with 1,230,000 newlines in it.<br>
Next biggest one is 12 MB with 280,000 newlines. These<br>
and about 140 other geojsons are open in the same project<br>
and I have no problems. In fact I converted from<br>
SHP to geojson 2 years ago because I used to have problems<br>
with SHP linear files.<br>
<br>
I use QGIS 3.16.8 on Linux Mint.<br>
<br>
<br>
Mike<br>
<br>
<br>
On 7/28/21 2:36 PM, gdal-dev-request@lists.osgeo.org wrote:<br>
> Date: Wed, 28 Jul 2021 12:22:12 -0700<br>
> From: Simon Eves<simon.eves@omnisci.com><br>
> To:gdal-dev@lists.osgeo.org<br>
> Subject: [gdal-dev] Large GeoJSONs and aborting file opening<br>
> Message-ID:<br>
> <CAJf0KTRsaskSOsPv8tbA+iTB+TqL_ui5y4n05WGLDw_3guRs4w@mail.gmail.com><br>
> Content-Type: text/plain; charset="utf-8"<br>
><br>
> Dear All,<br>
><br>
> I am aware that some improvements were made in the 2.3 timeframe with<br>
> regards to dealing with large GeoJSON files, although even in 3.2, it's<br>
> still very slow and memory hungry.<br>
><br>
> Our system allows for aborting imports, but this only works reliably once<br>
> it has actually got to the stage of reading features from the file. With<br>
> the GeoJSON, it just sits in the GDALOpenEx call for ages.<br>
><br>
> My question, therefore, is whether it might be practical to run the<br>
> GDALOpenEx in a separate thread with a future to return the resulting<br>
> handle, such that it could be monitored and killed if necessary?<br>
><br>
> Mainly I would be concerned that killing the thread might trash some global<br>
> GDAL state that might then not be recoverable, or that the open relies on<br>
> some TLS for the process thread and therefore might not work properly.<br>
><br>
> We're going to try it anyway, but opinions welcomed, thanks!<br>
><br>
> Simon<br>
<br>
<br>
_______________________________________________<br>
gdal-dev mailing list<br>
gdal-dev@lists.osgeo.org<br>
<a href="https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.osgeo.org%2Fmailman%2Flistinfo%2Fgdal-dev&data=04%7C01%7Cdoug_newcomb%40fws.gov%7C4a700f32558543c33b2308d9529c3a35%7C0693b5ba4b184d7b9341f32f400a5494%7C0%7C0%7C637631653236424668%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=86Xw3QEVbY0bqUurcBL8iLFi9IVjl0vGyErJUclKFFg%3D&reserved=0">https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.osgeo.org%2Fmailman%2Flistinfo%2Fgdal-dev&data=04%7C01%7Cdoug_newcomb%40fws.gov%7C4a700f32558543c33b2308d9529c3a35%7C0693b5ba4b184d7b9341f32f400a5494%7C0%7C0%7C637631653236424668%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=86Xw3QEVbY0bqUurcBL8iLFi9IVjl0vGyErJUclKFFg%3D&reserved=0</a><br>
</div>
</span></font></div>
</body>
</html>