[gdal-dev] CAD (DWG) Driver

Dmitry Baryshnikov bishop.dev at gmail.com
Tue Nov 22 01:45:17 PST 2016


Hi Nigel,

Thank you for information. I think enhancing OpenCAD library with 
support of additional DWG versions is right way. I'll study your code.


Best regards,
     Dmitry

21.11.16 13:10, Nigel Westbury пишет:
> The DWG driver contribution is much appreciated.  It currently supports older versions of files up to R1015 (AutoCAD 2000).  The DWG format was relatively straight forward up to that version.  In 2004 the format became rather more complex, including simple xor encryption and LZ77 compression of most of the records.  In 2007 the format became considerably more complex again, including Reed-Solomon encoding.  In 2010 the format became simpler again, going back to a format more similar to the 2004 format.  Our experience here at 1Spatial is that customers are on versions 2010 or 2013.  I have not received a DWG file from an earlier version from a customer.  Therefore we have implemented a library for reading versions 2010 and later (https://github.com/westbury/dwg-lib).  This has been implemented in Java and could readily be ported to C or otherwise compiled to a DLL, though as it is currently under active development this would need some thought to avoid maintaining two copies.  Alternatively we could leave it in Java and call it much as the GDAL GeoMedia driver does for Jackcess.   We have put the code under the MIT licence so that it would be available to GDAL.  The work was implemented by reference to the Open Design Specification for .dwg files Version 5.3.  We do plan on limited writing too, but probably updates to certain elements only, no support for outputting a complete DWG file from scratch.
>
> Nigel Westbury
>
> -----Original Message-----
> From: gdal-dev [mailto:gdal-dev-bounces at lists.osgeo.org] On Behalf Of gdal-dev-request at lists.osgeo.org
> Sent: 19 November 2016 19:49
> To: gdal-dev at lists.osgeo.org
> Subject: gdal-dev Digest, Vol 150, Issue 33
>
> Send gdal-dev mailing list submissions to
> gdal-dev at lists.osgeo.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
> http://lists.osgeo.org/mailman/listinfo/gdal-dev
> or, via email, send a message with subject or body 'help' to
> gdal-dev-request at lists.osgeo.org
>
> You can reach the person managing the list at
> gdal-dev-owner at lists.osgeo.org
>
> When replying, please edit your Subject line so it is more specific than "Re: Contents of gdal-dev digest..."
>
>
> Today's Topics:
>
>     1. Re: CAD (DWG) Driver (Even Rouault)
>     2. Re: CAD (DWG) Driver (Kurt Schwehr)
>     3. Re: CAD (DWG) Driver (Even Rouault)
>     4. Re: CAD (DWG) Driver (Dmitry Baryshnikov)
>     5. Re: identifying the fields from the .osm file (Djordje Spasic)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Sat, 19 Nov 2016 17:49:25 +0100
> From: Even Rouault <even.rouault at spatialys.com>
> To: Dmitry Baryshnikov <bishop.dev at gmail.com>, schwehr at gmail.com,
> gdal-dev at lists.osgeo.org
> Subject: Re: [gdal-dev] CAD (DWG) Driver
> Message-ID: <7873061.FYxrsrM0Qa at even-i700>
> Content-Type: text/plain; charset="us-ascii"
>
> On samedi 19 novembre 2016 19:34:02 CET Dmitry Baryshnikov wrote:
>> Hi Even,
>>
>> The big_endian test now is ok. But osx crash at the exiting of python
>> after all test finished successfully.
>>
>> This is error message:
>>
>> libc++abi.dylib: terminating with uncaught exception of type
>> std::__1::system_error: mutex lock failed: Invalid argument
>> ./gdal/ci/travis/osx/script.sh: line 13: 56363 Abort trap: 6
>> python run_all.py
>>
>> I find out that this may be the result of resources free order:
>> https://github.com/dmlc/mxnet/issues/309
>> https://github.com/google/benchmark/issues/52
>>
>> No idea why this occurred then C++11 used. I'll try to use osx CI test
>> disabling the CAD driver to exclude this possible issue.
>>
> Hum, I think this might be indeed an issue with resource free order between the GDALDestroy() destructor function (in gcore/gdaldllmain.cpp) and the static C++11 mutex Kurt introduced in
> https://github.com/OSGeo/gdal/commit/a9b947d6850d496f09e668f4cd148826d45d9fa9
>
> If this mutex gets destroyed before GDALDestroy() is called, then things might indeed crash.
>
> I think we should remove this static C++11 mutex. Kurt ?
>
> (Another option would be to remove the library destructor function. I've the feeling they cause more harm than good.)
>
> --
> Spatialys - Geospatial professional services http://www.spatialys.com
>
>
> ------------------------------
>
> Message: 2
> Date: Sat, 19 Nov 2016 10:06:26 -0800
> From: Kurt Schwehr <schwehr at gmail.com>
> To: Even Rouault <even.rouault at spatialys.com>
> Cc: Dmitry Baryshnikov <bishop.dev at gmail.com>, gdal dev
> <gdal-dev at lists.osgeo.org>
> Subject: Re: [gdal-dev] CAD (DWG) Driver
> Message-ID:
> <CACmBxysE_yyRtWmOCc=A6kEPLiwdkpi4k6=HEk8jJVpAuM_9ww at mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> I'm really not sure.
>
> You can certainly try removing that C++11 mutex.  It was added based on the
> comment at that point in the code.  We'll just have to keep an eye out for
> possible occasional flakes.  If that's the case, we'll need to carefully
> work through the logic and try to figure out a solution that works for
> everything.
>
> +1 for getting the driver into the tree.  It will be easier to make
> additional changes to either it or GDAL with it in the same tree.
>
> On Sat, Nov 19, 2016 at 8:49 AM, Even Rouault <even.rouault at spatialys.com>
> wrote:
>
>> On samedi 19 novembre 2016 19:34:02 CET Dmitry Baryshnikov wrote:
>>> Hi Even,
>>>
>>> The big_endian test now is ok. But osx crash at the exiting of python
>>> after all test finished successfully.
>>>
>>> This is error message:
>>>
>>> libc++abi.dylib: terminating with uncaught exception of type
>>> std::__1::system_error: mutex lock failed: Invalid argument
>>> ./gdal/ci/travis/osx/script.sh: line 13: 56363 Abort trap: 6
>>> python run_all.py
>>>
>>> I find out that this may be the result of resources free order:
>>> https://github.com/dmlc/mxnet/issues/309
>>> https://github.com/google/benchmark/issues/52
>>>
>>> No idea why this occurred then C++11 used. I'll try to use osx CI test
>>> disabling the CAD driver to exclude this possible issue.
>>>
>> Hum, I think this might be indeed an issue with resource free order between
>> the GDALDestroy() destructor function (in gcore/gdaldllmain.cpp) and the
>> static C++11 mutex Kurt introduced in
>> https://github.com/OSGeo/gdal/commit/a9b947d6850d496f09e668f4cd1488
>> 26d45d9fa9
>>
>> If this mutex gets destroyed before GDALDestroy() is called, then things
>> might
>> indeed crash.
>>
>> I think we should remove this static C++11 mutex. Kurt ?
>>
>> (Another option would be to remove the library destructor function. I've
>> the
>> feeling they cause more harm than good.)
>>
>> --
>> Spatialys - Geospatial professional services
>> http://www.spatialys.com
>>
>
>
> --
> --
> http://schwehr.org
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20161119/da36d65e/attachment-0001.html>
>
> ------------------------------
>
> Message: 3
> Date: Sat, 19 Nov 2016 19:24:16 +0100
> From: Even Rouault <even.rouault at spatialys.com>
> To: Kurt Schwehr <schwehr at gmail.com>
> Cc: Dmitry Baryshnikov <bishop.dev at gmail.com>, gdal dev
> <gdal-dev at lists.osgeo.org>
> Subject: Re: [gdal-dev] CAD (DWG) Driver
> Message-ID: <1731467.ZNQqGo50XJ at even-i700>
> Content-Type: text/plain; charset="us-ascii"
>
> On samedi 19 novembre 2016 10:06:26 CET Kurt Schwehr wrote:
>> I'm really not sure.
> Not sure that the crash mentionned by Dmitry is related to that ? It looks
> like a really good canditate to me. That's the only C++11 mutex in GDAL, with
> 2  other ones in the GeoTIFF driver (and those are likely not triggered at
> process termination)
>
>> You can certainly try removing that C++11 mutex.  It was added based on the
>> comment at that point in the code.  We'll just have to keep an eye out for
>> possible occasional flakes.  If that's the case, we'll need to carefully
>> work through the logic and try to figure out a solution that works for
>> everything.
> I doubt we will see flakes related to that. The mutex was added to prevent
> races in GDALDestroyDriverManager(), but I'm not aware of code in GDAL and its
> utilities that would call it from several threads. And the function has always
> been documented as thread-unsafe.
>
>> +1 for getting the driver into the tree.  It will be easier to make
>> additional changes to either it or GDAL with it in the same tree.
>>
>> On Sat, Nov 19, 2016 at 8:49 AM, Even Rouault <even.rouault at spatialys.com>
>>
>> wrote:
>>> On samedi 19 novembre 2016 19:34:02 CET Dmitry Baryshnikov wrote:
>>>> Hi Even,
>>>>
>>>> The big_endian test now is ok. But osx crash at the exiting of python
>>>> after all test finished successfully.
>>>>
>>>> This is error message:
>>>>
>>>> libc++abi.dylib: terminating with uncaught exception of type
>>>> std::__1::system_error: mutex lock failed: Invalid argument
>>>> ./gdal/ci/travis/osx/script.sh: line 13: 56363 Abort trap: 6
>>>> python run_all.py
>>>>
>>>> I find out that this may be the result of resources free order:
>>>> https://github.com/dmlc/mxnet/issues/309
>>>> https://github.com/google/benchmark/issues/52
>>>>
>>>> No idea why this occurred then C++11 used. I'll try to use osx CI test
>>>> disabling the CAD driver to exclude this possible issue.
>>> Hum, I think this might be indeed an issue with resource free order
>>> between
>>> the GDALDestroy() destructor function (in gcore/gdaldllmain.cpp) and the
>>> static C++11 mutex Kurt introduced in
>>> https://github.com/OSGeo/gdal/commit/a9b947d6850d496f09e668f4cd1488
>>> 26d45d9fa9
>>>
>>> If this mutex gets destroyed before GDALDestroy() is called, then things
>>> might
>>> indeed crash.
>>>
>>> I think we should remove this static C++11 mutex. Kurt ?
>>>
>>> (Another option would be to remove the library destructor function. I've
>>> the
>>> feeling they cause more harm than good.)
>>>
>>> --
>>> Spatialys - Geospatial professional services
>>> http://www.spatialys.com
>
> --
> Spatialys - Geospatial professional services
> http://www.spatialys.com
>
>
> ------------------------------
>
> Message: 4
> Date: Sat, 19 Nov 2016 22:22:13 +0300
> From: Dmitry Baryshnikov <bishop.dev at gmail.com>
> To: Even Rouault <even.rouault at spatialys.com>, Kurt Schwehr
> <schwehr at gmail.com>
> Cc: gdal dev <gdal-dev at lists.osgeo.org>
> Subject: Re: [gdal-dev] CAD (DWG) Driver
> Message-ID: <69e67501-5af0-d09a-bf2a-4faed559ad6f at gmail.com>
> Content-Type: text/plain; charset="utf-8"; Format="flowed"
>
> I commented out the C++11 mutex in gdaldrivermanager.cpp
> <https://github.com/OSGeo/gdal/pull/164/commits/d1449e2eb5a85a6ec2520ad77ee579015fe14197#diff-e3e90e0089f1b70ddf36ebe89a73a733>
> - this helps. OSX CI test passed successfully.
>
> https://travis-ci.org/OSGeo/gdal/jobs/177297023
>
> Best regards,
>       Dmitry
>
> 19.11.16 21:24, Even Rouault пишет:
>> On samedi 19 novembre 2016 10:06:26 CET Kurt Schwehr wrote:
>>> I'm really not sure.
>> Not sure that the crash mentionned by Dmitry is related to that ? It looks
>> like a really good canditate to me. That's the only C++11 mutex in GDAL, with
>> 2  other ones in the GeoTIFF driver (and those are likely not triggered at
>> process termination)
>>
>>> You can certainly try removing that C++11 mutex.  It was added based on the
>>> comment at that point in the code.  We'll just have to keep an eye out for
>>> possible occasional flakes.  If that's the case, we'll need to carefully
>>> work through the logic and try to figure out a solution that works for
>>> everything.
>> I doubt we will see flakes related to that. The mutex was added to prevent
>> races in GDALDestroyDriverManager(), but I'm not aware of code in GDAL and its
>> utilities that would call it from several threads. And the function has always
>> been documented as thread-unsafe.
>>
>>> +1 for getting the driver into the tree.  It will be easier to make
>>> additional changes to either it or GDAL with it in the same tree.
>>>
>>> On Sat, Nov 19, 2016 at 8:49 AM, Even Rouault <even.rouault at spatialys.com>
>>>
>>> wrote:
>>>> On samedi 19 novembre 2016 19:34:02 CET Dmitry Baryshnikov wrote:
>>>>> Hi Even,
>>>>>
>>>>> The big_endian test now is ok. But osx crash at the exiting of python
>>>>> after all test finished successfully.
>>>>>
>>>>> This is error message:
>>>>>
>>>>> libc++abi.dylib: terminating with uncaught exception of type
>>>>> std::__1::system_error: mutex lock failed: Invalid argument
>>>>> ./gdal/ci/travis/osx/script.sh: line 13: 56363 Abort trap: 6
>>>>> python run_all.py
>>>>>
>>>>> I find out that this may be the result of resources free order:
>>>>> https://github.com/dmlc/mxnet/issues/309
>>>>> https://github.com/google/benchmark/issues/52
>>>>>
>>>>> No idea why this occurred then C++11 used. I'll try to use osx CI test
>>>>> disabling the CAD driver to exclude this possible issue.
>>>> Hum, I think this might be indeed an issue with resource free order
>>>> between
>>>> the GDALDestroy() destructor function (in gcore/gdaldllmain.cpp) and the
>>>> static C++11 mutex Kurt introduced in
>>>> https://github.com/OSGeo/gdal/commit/a9b947d6850d496f09e668f4cd1488
>>>> 26d45d9fa9
>>>>
>>>> If this mutex gets destroyed before GDALDestroy() is called, then things
>>>> might
>>>> indeed crash.
>>>>
>>>> I think we should remove this static C++11 mutex. Kurt ?
>>>>
>>>> (Another option would be to remove the library destructor function. I've
>>>> the
>>>> feeling they cause more harm than good.)
>>>>
>>>> --
>>>> Spatialys - Geospatial professional services
>>>> http://www.spatialys.com
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20161119/78087c6b/attachment-0001.html>
>
> ------------------------------
>
> Message: 5
> Date: Sat, 19 Nov 2016 19:45:40 +0000 (UTC)
> From: Djordje Spasic <issworld2000 at yahoo.com>
> To: Etienne Trimaille <etienne.trimaille at gmail.com>
> Cc: "gdal-dev at lists.osgeo.org" <gdal-dev at lists.osgeo.org>
> Subject: Re: [gdal-dev] identifying the fields from the .osm file
> Message-ID: <840769665.311796.1479584740697 at mail.yahoo.com>
> Content-Type: text/plain; charset="utf-8"
>
> Thank you Etienne.
> I am very interested in your reply, but I haven't quite understand you.
> The ogr2ogr.exe file is using a particular python script to read the .osm file and identify its unique fields?
> Can you be a bit more specific please? Solving this issue will mean a lot to me.
> Sorry for disturbance, and thank you for the clarification in advance.Regards,
> Djordje
>
>
>      On Saturday, November 19, 2016 5:09 PM, Etienne Trimaille <etienne.trimaille at gmail.com> wrote:
>
>
>   The parsing is done using python in the plugin. We can just use the parser by calling the python file. Python is included when you install osgeo4w.Yes you can set the osmconf.ini or directly use the goal ogr api to write the output file.
> On 19 Nov 2016 18:00, "Djordje Spasic" <issworld2000 at yahoo.com> wrote:
>
> Thank you for the quick reply Etienne!
>
> I am stuck with ogr2ogr.exe, or GDAL C# libraries. I can not use QGIS or other applications.
> Indeed Even told me that identifying all fields in the .osm file, would require doing 2 read passes of the .osm file.
> I thought that maybe there is a way of doing exactly that, where in the first read I would get all the fields, find only those that are unique, and then perform a second read by setting these fields in the osmconf.ini file?
>
>      On Saturday, November 19, 2016 2:40 AM, Etienne Trimaille <etienne.trimaille at gmail.com> wrote:
>
>
>   Hi,
> With ogr2ogr, you can't read all keys from an OSM file.
> As Even said, you have to use another loop to read all available keys in the file. You can have a look at the QGIS QuickOSM plugin. You can open a OSM file and ask the plugin to read all keys available in the file.
> 2016-11-19 8:30 GMT+07:00 Djordje Spasic <issworld2000 at yahoo.com>:
>
> During the last couple of days, both Jukka and Even helped me with some .osm file issues. And I am grateful on that.
>
> I would like to ask one more question, if that's ok:
> Is it possible to open an .osm file through ogr2ogr.exe and somehow read all the fields (keys) used in that .osm file? The final list of those fields do not have to be unique, they can repeat.
> Is it possible to somehow save all those fields to a .shp or .txt file through ogr2ogr.exe?
>
> Thank you for the reply.
>
> ______________________________ _________________
> gdal-dev mailing list
> gdal-dev at lists.osgeo.org
> http://lists.osgeo.org/mailman /listinfo/gdal-dev
>
>
>
>
>
>
>
>
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20161119/ebd0edd4/attachment.html>
>
> ------------------------------
>
> Subject: Digest Footer
>
> _______________________________________________
> gdal-dev mailing list
> gdal-dev at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/gdal-dev
>
> ------------------------------
>
> End of gdal-dev Digest, Vol 150, Issue 33
> *****************************************
>   [http://www.1spatial.com/signatures/1SpatialLogo_Small.jpg] <http://www.1spatial.com>
>
>
>
>
>
> [http://www.1spatial.com/wp-content/themes/1spatial/images/youtube.png]<http://www.youtube.com/user/1SpatialGroup> [http://www.1spatial.com/wp-content/themes/1spatial/images/facebook.png] <https://www.facebook.com/1Spatial>  [http://www.1spatial.com/wp-content/themes/1spatial/images/linkedIn.png] <http://www.linkedin.com/company/1spatial>  [http://www.1spatial.com/wp-content/themes/1spatial/images/twitter.png] <http://www.twitter.com/1spatial>
>
> ________________________________
>
> The information in this e-mail is confidential and may be legally privileged. It is intended solely for the addressee. Access to this email by anyone else is unauthorised. Any views or opinions expressed in this e-mail may be solely those of the author and are not necessarily those of 1Spatial Plc. This email has been scanned for viruses and message content. 1Spatial Group Limited is a company registered in England and Wales No.4785688. Registered Office: Tennyson House, Cambridge Business Park, Cambridge CB4 0WZ
> _______________________________________________
> gdal-dev mailing list
> gdal-dev at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/gdal-dev




More information about the gdal-dev mailing list