[gdal-dev] Creation of a new driver from scratch

Wed Aug 30 07:37:04 PDT 2023

Hi Javier,
thanks for your answer. It’s very similar, yes. I’ll try to use it in my driver.

While fetching features I have some information in memory (always 64bits variables) and some in memory-disk (it’s a buffer that writes everything in disk when it’s full as many times as needed).
If, in the middle of writing I find that user wants AUTO and I was writing in 32bits but it’s no possible anymore then I translate the written part from 32 to 64 (and I do nothing with the memory part).
At the end when I write the memory part I take care of what option is the good one.
I could always write 64bits format but if it’s not needed it’s a pity to waste disk memory for that.
This consumes time but if you can’t know the number of features (or vertices in 3D, which they also need 64bits offsets) a priori I think that’s the only way to do that or don’t implement AUTO and let user find out that user cannot write 32bits option for a large translation. And make user begin again. I think that if I can save some time it’s better than start again.

For me the best option is knowing at the beginning what option is enough to write every file.

Sorry for the extension of my words.

It remains knowing how to inform user (when I use –progress) that I am starting again... I suppose that a message in the middle of the % points.

And by the way... how can GDAL know the % if the number of features is not reachable?

Thanks!

De: Javier Jimenez Shaw <j1 at jimenezshaw.com>
Enviado el: dimecres, 30 d’agost de 2023 15:39
Para: Abel Pau <a.pau at creaf.uab.cat>
CC: gdal-dev at lists.osgeo.org
Asunto: Re: [gdal-dev] Creation of a new driver from scratch

That sounds very similar to BIGTIFF option in GeoTIFF driver.
also a bit like GEOTIFF_VERSION option.
https://gdal.org/drivers/raster/gtiff.html

You can change that in the middle of the writing? wow.

On Wed, 30 Aug 2023 at 06:21, Abel Pau <a.pau at creaf.uab.cat<mailto:a.pau at creaf.uab.cat>> wrote:
Hi again,
I hope you have had nice holidays!

In my journey to writing a new driver now I have a doubt.

Our format can be written in two versions:
v1.1: internal offsets and FID are only saved in 4 bytes. So, there is a limited number of FID you can read/write.
v2.0: internal offsets and FID are only saved in 8 bytes. So, there is a huge number of FID you can read/write.

I am interested in proportionate 3 ways of proceeding when a used calls translation from one format to our format.
way 1: force to write V1.1 and stop writing features if the limit is reached.
way 2: force to write V2.0 (and stop writing features if the limit is reached (it will no happen in a near future, I think)).
way 3: leave that the program decides if V1.1 is not enough and then it changes automatically to V2.0. It does it in the middle of the process, when FID or some used internal offset gets the limit.

So, my questions:
1.- Which ogr2ogr parameter is the best for doing that?
        -f? specificating "MiraMon V1.1" or "MiraMonV2.0" or "MiraMon" (automatic way).
      Or there is another way I haven't seen (I've read all options in https://gdal.org/programs/ogr2ogr.html#ogr2ogr)

2.- When using -progress some points appear on screen informing the progress. At the end of the progress (or sometimes driver requires) there is a gap of time that driver is writing some pendant information.
There is any way to inform the user what the process is doing?

Thanks in advance. I appreciate all your always kind answers.
Abel.

-----Mensaje original-----
De: gdal-dev <gdal-dev-bounces at lists.osgeo.org<mailto:gdal-dev-bounces at lists.osgeo.org>> En nombre de Abel Pau
Enviado el: divendres, 9 de juny de 2023 12:11
Para: Even Rouault <even.rouault at spatialys.com<mailto:even.rouault at spatialys.com>>
CC: gdal-dev at lists.osgeo.org<mailto:gdal-dev at lists.osgeo.org>
Asunto: Re: [gdal-dev] Creation of a new driver from scratch

Hi again.

Finally I managed to use CMake to create a Visual Studio sln. And it's beautiful see all drivers there.
But it's not crear for me what I have to modify (Cmakelist.txt,.. or something else) to include the new driver. I need it to be there to be able to debug.
I created a copy of, let's say, Shape and modified the Cmakelists.txt in an apropriate way... but I miss something because recompiling and rebuilding doesn't change anything and my driver is not in the project.

Anyone can make me a simple list of which files I have to modify?
THANKS in advance!!!

Abel.

-----Mensaje original-----
De: Even Rouault <even.rouault at spatialys.com<mailto:even.rouault at spatialys.com>> Enviado el: divendres, 2 de juny de 2023 20:34
Para: Abel Pau <a.pau at creaf.uab.cat<mailto:a.pau at creaf.uab.cat>>
CC: gdal-dev at lists.osgeo.org<mailto:gdal-dev at lists.osgeo.org>; Howard Butler <howard at hobu.co<mailto:howard at hobu.co>>
Asunto: Re: [gdal-dev] Creation of a new driver from scratch

> It is indeed desired that the code style of the contribution matches GDAL's style to some degree. GDAL provides a .clang-format file that might be helpful when you are editing to provide some automated conformance.

Cf https://gdal.org/development/dev_practices.html#commit-hooks also to install pre-commit hooks to ensure the formatting is OK & fix it if not.

https://gdal.org/development/rfc/rfc8_devguide.html also gives some hints on the general development practices. Generally try to stick with the conventions you observe in source code of drivers you take inspiration from.

>
> You might be able to build a sln file using GDAL's CMake configuration, but there is no standard documentation about how to do this. Do know that most any configuration you provide must build and work with the standard GDAL CMake configuration which is typically invoked through the command line.

The source of "truth" will be the CMakeLists.txt file.

sln files are just a possible by-product of CMake that isn't stored in git.

You may just google "Cmake visual studio" as there's nothing GDAL specific regarding this topic. ==>
https://learn.microsoft.com/en-us/cpp/build/cmake-projects-in-visual-studio?view=msvc-170
among other links that you'll get

>
>> ·         Once I have a Visual Studio solution (sln) to inspire myself, how should I proceed to create the driver? Is there any pre-existing pattern or template to follow?
> Find an existing raster or vector driver in the project that matches MiraMon's vector or raster format and use it as a skeleton.
>
>> I would like to be able to debug it.
> Familiarize yourself with GDAL's tests (a good place to learn is to look at GDAL's CI configuration), get tests running locally on your machine, and then write tests that provide test coverage for your raster or vector driver.
For vector drivers, the test_ogrsf utility that is built with GDAL is used by most unit tests of vector drivers to check their compliance w.r.t expectations of what a driver should do/not do. Not that of course this is only generic testing. You also need to add tests that check that the content you read from a test file is the one you expect.

--
http://www.spatialys.com
My software is free, but my time generally not.

_______________________________________________
gdal-dev mailing list
gdal-dev at lists.osgeo.org<mailto:gdal-dev at lists.osgeo.org>
https://lists.osgeo.org/mailman/listinfo/gdal-dev
_______________________________________________
gdal-dev mailing list
gdal-dev at lists.osgeo.org<mailto:gdal-dev at lists.osgeo.org>
https://lists.osgeo.org/mailman/listinfo/gdal-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20230830/6433101c/attachment-0001.htm>