[gdal-dev] Issues with shapefiles with Z values?

[Rahkonen Jukka]

Hi,

The ESRI Shapefile Technical Description is not the most exact document that I have seen, and as far as I know ESRI has never published any clarifications or corrigendum to the original paper from 1998.

The interpretation that measures are optional and there is no need to fill the M array with no data like -10^38 comes from the tables, for example table 14 "PolyLineZ Record Contents". The last lines of the table are (notice asterisks):
Byte Z*
Byte Z+8*
Byte Z+16*

The meaning of the asterisk is explained in the endnote:
* optional

There is also this sentense: "Shapes of this type [Shape Types in X,Y,Z Space] have an optional coordinate M. Note that "no data" value can be specified as a value for M (see Numeric Types on page 2)."
It says "can be", not "must be"

Nothing more can be found from the specification. That ESRI software reads correctly shapefiles which do not have a dummy M array suggests that the dummy array is not needed.

-Jukka Rahkonen-


________________________________________
Lähettäjä: gdal-dev <gdal-dev-bounces at lists.osgeo.org> käyttäjän David Strip via gdal-dev <gdal-dev at lists.osgeo.org> puolesta
Lähetetty: Keskiviikko 15. lokakuuta 2025 23.33
Vastaanottaja: gdal dev <gdal-dev at lists.osgeo.org>; Jukka Rahkonen <jukka.rahkonen at mmmtike.fi>; Andrea Giudiceandrea <andreaerdna at libero.it>
Aihe: Re: [gdal-dev] Issues with shapefiles with Z values?

I sent Jukka and Andrea's comments to the developer. Here is the response I received from him:Thank you for you email and taking the time to actually get in touch with GDAL maintainers and forward their replies to me.That's an interesting statement from them. I have been working with shapefiles for the last 20 years and my personal interpretation of M array being "optional" has always been that in case no M dimensions are needed, some "no data" values can be stored in the binary file.For reference, on page 10 of the specs under "Measured Shape Types in X,Y Space" paragraph it reads:    Shapes of this type have an additional coordinate M. Note that "no data" value can be specified as a value for M (see Numeric Types on page 2).And on page 15 under "Shape Types in X,Y,Z Space" paragraph, again:    Shapes of this type have an optional coordinate M. Note that "no data" value can be specified as a value for M (see Numeric Types on page 2).On page 2, "Numeric Types" paragraph states:    [...] shapefiles support the concept of "no data" values, but they are currently used only for measures.    Any floating point number smaller than –1038 is considered by a shapefile reader to represent a "no data" value.Thus, my interpretation has always been something like that in case M array is not needed, I must fill it with "no data" values.Probably GDAL devs know more than me, but it seems odd that in order to establish whether the M array is present or not in the binary SHP file, one needs to check the SHX index file for the binary address of the next record. Surely it could be, but it feels cumbersome to me (not that it would be the only cumbersome thing about shapefile specs :)) )About the vast majority of GIS software being able to read those shapefiles with missing M arrays, I have always thought they chose to silently ignore the missing M array of the last record as a favor to the user (yep, I know technically it's not just the last record...).Anyways, I am open to admit I have always been wrong with my interpretation of the specs (maybe even relieved), it would be great to have a confirmation from GDAL devs.I think the source of the problem is confusion between "no data" and "optional". I interpret "no data" as a null indicator,  to fill in required values but indicate that the value is meaningless, while "optional" means the data need not appear at all. The developer is not a native English speaker, which may have contributed to the confusion.