[SCM] PostGIS branch master updated. 3.7.0beta1-260-g9e46247541
git at osgeo.org
git at osgeo.org
Sun Aug 9 03:39:15 PDT 2026
This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "PostGIS".
The branch, master has been updated
via 9e462475410a58876992c2f7ea61fda4ac403ca8 (commit)
via 0a7c72a9102033c8b6ddb4f3875cc0741d3baa0a (commit)
from 62492292e2224da63e13bc18be1d01b5b10eed1a (commit)
Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.
- Log -----------------------------------------------------------------
commit 9e462475410a58876992c2f7ea61fda4ac403ca8
Merge: 62492292e2 0a7c72a910
Author: Darafei Praliaskouski <komzpa at gmail.com>
Date: Sun Aug 9 03:39:14 2026 -0700
Merge pull request 'flatgeobuf: validate input buffers before decoding' (!669) from Komzpa/postgis:fix/flatgeobuf-input-validation-20260808 into master
## Summary
- validate FlatGeobuf size-prefixed header and feature buffers against the remaining `bytea` before FlatBuffers decodes them
- reject malformed variable-length string and datetime property values before copying them into PostgreSQL datums
- add adjacent guards for packed RTree metadata, geometry coordinate vectors, feature-loop advancement, and `NULL` FlatGeobuf input
- add regression coverage for truncated and oversized FlatGeobuf input, and credit the reporter in `NEWS`/manual credits
## Validation
- `git diff --check`
- `utils/docs/check_news.sh --base-ref gitea/master .`
- `make check-contributor-credits`
- `make -j8`
- `./config.status regress/core/tests.mk`
- `make -C regress/core check RUNTESTFLAGS="" TESTS="./flatgeobuf"` (normal and upgrade regression passes)
Reviewed-on: https://gitea.osgeo.org/postgis/postgis/pulls/669
commit 0a7c72a9102033c8b6ddb4f3875cc0741d3baa0a
Author: Darafei Praliaskouski <me at komzpa.net>
Date: Sat Aug 8 15:23:44 2026 +0400
flatgeobuf: validate input buffers before decoding
Validate FlatGeobuf size-prefixed header and feature buffers against the remaining bytea before FlatBuffers decodes them. Reject malformed variable-length property values and adjacent malformed geometry/index metadata before copying or advancing through decoder state.
Also handle NULL bytea input to ST_FromFlatGeobuf explicitly, preserving its existing empty-set behavior for NULL input.
diff --git a/NEWS b/NEWS
index b7eae91bb3..e2db01d0d2 100644
--- a/NEWS
+++ b/NEWS
@@ -14,6 +14,10 @@ These are only changes since 3.7.0beta1.
fixed by Darafei Praliaskouski)
- OSSFuzz 6302177757560832, account for GSERIALIZED float bbox rounding
in fuzzer round trips (Darafei Praliaskouski)
+ - GT-669, Validate FlatGeobuf size-prefixed buffers and variable-length
+ property values before decoding
+ (reported by Sarath Kumar, IITM Pravartak Security Team;
+ fixed by Darafei Praliaskouski)
- Stop the extension upgrade script running ANALYZE inside its transaction,
where it could deadlock with autovacuum (Darafei Praliaskouski)
- [liblwgeom] Reject malformed GSERIALIZED NURBS before curve
diff --git a/deps/flatgeobuf/flatgeobuf_c.cpp b/deps/flatgeobuf/flatgeobuf_c.cpp
index 16a259375b..a5b121f942 100644
--- a/deps/flatgeobuf/flatgeobuf_c.cpp
+++ b/deps/flatgeobuf/flatgeobuf_c.cpp
@@ -28,12 +28,14 @@
#include "geometryreader.h"
#include "packedrtree.h"
+#include <exception>
+
using namespace flatbuffers;
using namespace FlatGeobuf;
typedef flatgeobuf_ctx ctx;
-uint8_t flatgeobuf_magicbytes[] = { 0x66, 0x67, 0x62, 0x03, 0x66, 0x67, 0x62, 0x01 };
+uint8_t flatgeobuf_magicbytes[] = {0x66, 0x67, 0x62, 0x03, 0x66, 0x67, 0x62, 0x01};
uint8_t FLATGEOBUF_MAGICBYTES_SIZE = sizeof(flatgeobuf_magicbytes);
struct FeatureItem : FlatGeobuf::Item {
@@ -41,20 +43,30 @@ struct FeatureItem : FlatGeobuf::Item {
uint64_t offset;
};
-int flatgeobuf_encode_header(ctx *ctx)
+static size_t
+flatgeobuf_size_prefixed_verifier_length(uoffset_t size)
+{
+ return (size_t)size + sizeof(uoffset_t);
+}
+
+int
+flatgeobuf_encode_header(ctx *ctx)
{
FlatBufferBuilder fbb;
fbb.TrackMinAlign(8);
// inspect first geometry
- if (ctx->lwgeom != NULL) {
+ if (ctx->lwgeom != NULL)
+ {
if (lwgeom_has_srid(ctx->lwgeom))
ctx->srid = ctx->lwgeom->srid;
ctx->has_z = lwgeom_has_z(ctx->lwgeom);
ctx->has_m = lwgeom_has_m(ctx->lwgeom);
ctx->lwgeom_type = ctx->lwgeom->type;
- ctx->geometry_type = (uint8_t) GeometryWriter::get_geometrytype(ctx->lwgeom);
- } else {
+ ctx->geometry_type = (uint8_t)GeometryWriter::get_geometrytype(ctx->lwgeom);
+ }
+ else
+ {
LWDEBUG(2, "ctx->lwgeom is null");
ctx->geometry_type = 0;
}
@@ -64,10 +76,12 @@ int flatgeobuf_encode_header(ctx *ctx)
std::vector<flatbuffers::Offset<FlatGeobuf::Column>> columns;
std::vector<flatbuffers::Offset<FlatGeobuf::Column>> *pColumns = nullptr;
- if (ctx->columns_size > 0) {
- for (uint16_t i = 0; i < ctx->columns_size; i++) {
+ if (ctx->columns_size > 0)
+ {
+ for (uint16_t i = 0; i < ctx->columns_size; i++)
+ {
auto c = ctx->columns[i];
- columns.push_back(CreateColumnDirect(fbb, c->name, (ColumnType) c->type));
+ columns.push_back(CreateColumnDirect(fbb, c->name, (ColumnType)c->type));
}
}
if (columns.size() > 0)
@@ -79,7 +93,8 @@ int flatgeobuf_encode_header(ctx *ctx)
std::vector<double> envelope;
std::vector<double> *pEnvelope = nullptr;
- if (ctx->has_extent) {
+ if (ctx->has_extent)
+ {
envelope.push_back(ctx->xmin);
envelope.push_back(ctx->ymin);
envelope.push_back(ctx->xmax);
@@ -88,21 +103,32 @@ int flatgeobuf_encode_header(ctx *ctx)
if (envelope.size() > 0)
pEnvelope = &envelope;
- const auto header = CreateHeaderDirect(
- fbb, ctx->name, pEnvelope, (GeometryType) ctx->geometry_type, ctx->has_z, ctx->has_m, ctx->has_t, ctx->has_tm, pColumns, ctx->features_count, ctx->index_node_size, crs);
+ const auto header = CreateHeaderDirect(fbb,
+ ctx->name,
+ pEnvelope,
+ (GeometryType)ctx->geometry_type,
+ ctx->has_z,
+ ctx->has_m,
+ ctx->has_t,
+ ctx->has_tm,
+ pColumns,
+ ctx->features_count,
+ ctx->index_node_size,
+ crs);
fbb.FinishSizePrefixed(header);
const auto buffer = fbb.GetBufferPointer();
const auto size = fbb.GetSize();
LWDEBUGF(2, "header size %d (with size prefix)", size);
- Verifier verifier(buffer, size - sizeof(uoffset_t));
- if (VerifySizePrefixedHeaderBuffer(verifier)) {
+ Verifier verifier(buffer, size);
+ if (!VerifySizePrefixedHeaderBuffer(verifier))
+ {
lwerror("buffer did not pass verification");
return -1;
}
- ctx->buf = (uint8_t *) lwrealloc(ctx->buf, ctx->offset + size);
+ ctx->buf = (uint8_t *)lwrealloc(ctx->buf, ctx->offset + size);
LWDEBUGF(2, "copying to ctx->buf at offset %llu", ctx->offset);
memcpy(ctx->buf + ctx->offset, buffer, size);
@@ -111,7 +137,8 @@ int flatgeobuf_encode_header(ctx *ctx)
return 0;
}
-int flatgeobuf_encode_feature(ctx *ctx)
+int
+flatgeobuf_encode_feature(ctx *ctx)
{
FlatBufferBuilder fbb;
Offset<Geometry> geometry = 0;
@@ -119,13 +146,15 @@ int flatgeobuf_encode_feature(ctx *ctx)
fbb.TrackMinAlign(8);
- if (ctx->lwgeom != NULL && !lwgeom_is_empty(ctx->lwgeom)) {
+ if (ctx->lwgeom != NULL && !lwgeom_is_empty(ctx->lwgeom))
+ {
LWDEBUGG(3, ctx->lwgeom, "GeometryWriter input LWGEOM");
- if (ctx->lwgeom_type != ctx->lwgeom->type) {
+ if (ctx->lwgeom_type != ctx->lwgeom->type)
+ {
lwerror("mixed geometry type is not supported");
return -1;
}
- GeometryWriter writer(fbb, ctx->lwgeom, (GeometryType) ctx->geometry_type, ctx->has_z, ctx->has_m);
+ GeometryWriter writer(fbb, ctx->lwgeom, (GeometryType)ctx->geometry_type, ctx->has_z, ctx->has_m);
geometry = writer.write(0);
}
if (ctx->properties_len > 0)
@@ -140,21 +169,24 @@ int flatgeobuf_encode_feature(ctx *ctx)
LWDEBUGF(3, "encode_feature size %u", size);
- Verifier verifier(buffer, size - sizeof(uoffset_t));
- if (VerifySizePrefixedFeatureBuffer(verifier)) {
+ Verifier verifier(buffer, size);
+ if (!VerifySizePrefixedFeatureBuffer(verifier))
+ {
lwerror("buffer did not pass verification");
return -1;
}
LWDEBUGF(3, "reallocating ctx->buf to size %llu", ctx->offset + size);
- ctx->buf = (uint8_t * ) lwrealloc(ctx->buf, ctx->offset + size);
+ ctx->buf = (uint8_t *)lwrealloc(ctx->buf, ctx->offset + size);
LWDEBUGF(3, "copying feature to ctx->buf at offset %llu", ctx->offset);
memcpy(ctx->buf + ctx->offset, buffer, size);
- if (ctx->create_index) {
- auto item = (flatgeobuf_item *) lwalloc(sizeof(flatgeobuf_item));
+ if (ctx->create_index)
+ {
+ auto item = (flatgeobuf_item *)lwalloc(sizeof(flatgeobuf_item));
memset(item, 0, sizeof(flatgeobuf_item));
- if (ctx->lwgeom != NULL && !lwgeom_is_empty(ctx->lwgeom)) {
+ if (ctx->lwgeom != NULL && !lwgeom_is_empty(ctx->lwgeom))
+ {
auto gbox = lwgeom_get_bbox(ctx->lwgeom);
item->xmin = gbox->xmin;
item->xmax = gbox->xmax;
@@ -171,15 +203,15 @@ int flatgeobuf_encode_feature(ctx *ctx)
return 0;
}
-void flatgeobuf_create_index(ctx *ctx)
+void
+flatgeobuf_create_index(ctx *ctx)
{
// convert to structure expected by packedrtree
std::vector<std::shared_ptr<Item>> items;
- for (uint64_t i = 0; i < ctx->features_count; i++) {
+ for (uint64_t i = 0; i < ctx->features_count; i++)
+ {
const auto item = std::make_shared<FeatureItem>();
- item->nodeItem = {
- ctx->items[i]->xmin, ctx->items[i]->ymin, ctx->items[i]->xmax, ctx->items[i]->ymax
- };
+ item->nodeItem = {ctx->items[i]->xmin, ctx->items[i]->ymin, ctx->items[i]->xmax, ctx->items[i]->ymax};
item->offset = ctx->items[i]->offset;
item->size = ctx->items[i]->size;
items.push_back(item);
@@ -196,30 +228,32 @@ void flatgeobuf_create_index(ctx *ctx)
// allocate new buffer and write magicbytes
auto oldbuf = ctx->buf;
auto oldoffset = ctx->offset;
- ctx->buf = (uint8_t *) lwalloc(sizeof(signed int) + FLATGEOBUF_MAGICBYTES_SIZE);
+ ctx->buf = (uint8_t *)lwalloc(sizeof(signed int) + FLATGEOBUF_MAGICBYTES_SIZE);
memcpy(ctx->buf + sizeof(signed int), flatgeobuf_magicbytes, FLATGEOBUF_MAGICBYTES_SIZE);
ctx->offset = sizeof(signed int) + FLATGEOBUF_MAGICBYTES_SIZE;
// write new header
flatgeobuf_encode_header(ctx);
// calculate new offsets
uint64_t featureOffset = 0;
- for (auto item : items) {
+ for (auto item : items)
+ {
auto featureItem = std::static_pointer_cast<FeatureItem>(item);
featureItem->nodeItem.offset = featureOffset;
featureOffset += featureItem->size;
}
// create and write index
PackedRTree tree(items, extent, ctx->index_node_size);
- const auto writeData = [&ctx] (const void *data, const size_t size) {
- ctx->buf = (uint8_t *) lwrealloc(ctx->buf, ctx->offset + size);
+ const auto writeData = [&ctx](const void *data, const size_t size) {
+ ctx->buf = (uint8_t *)lwrealloc(ctx->buf, ctx->offset + size);
memcpy(ctx->buf + ctx->offset, data, size);
ctx->offset += size;
};
tree.streamWrite(writeData);
// read items and write in sorted order
- for (auto item : items) {
+ for (auto item : items)
+ {
auto featureItem = std::static_pointer_cast<FeatureItem>(item);
- ctx->buf = (uint8_t *) lwrealloc(ctx->buf, ctx->offset + featureItem->size);
+ ctx->buf = (uint8_t *)lwrealloc(ctx->buf, ctx->offset + featureItem->size);
LWDEBUGF(2, "copy from offset %llu", featureItem->offset);
memcpy(ctx->buf + ctx->offset, oldbuf + featureItem->offset, featureItem->size);
ctx->offset += featureItem->size;
@@ -227,14 +261,16 @@ void flatgeobuf_create_index(ctx *ctx)
lwfree(oldbuf);
}
-int flatgeobuf_decode_feature(ctx *ctx)
+int
+flatgeobuf_decode_feature(ctx *ctx)
{
LWDEBUGF(2, "reading size prefix at %llu", ctx->offset);
auto size = flatbuffers::GetPrefixedSize(ctx->buf + ctx->offset);
LWDEBUGF(2, "size is %u (without size prefix)", size);
- Verifier verifier(ctx->buf + ctx->offset, size);
- if (VerifySizePrefixedFeatureBuffer(verifier)) {
+ Verifier verifier(ctx->buf + ctx->offset, flatgeobuf_size_prefixed_verifier_length(size));
+ if (!VerifySizePrefixedFeatureBuffer(verifier))
+ {
lwerror("buffer did not pass verification");
return -1;
}
@@ -245,34 +281,46 @@ int flatgeobuf_decode_feature(ctx *ctx)
ctx->offset += size;
const auto geometry = feature->geometry();
- if (geometry != nullptr) {
- LWDEBUGF(3, "Constructing GeometryReader with geometry_type %d has_z %d haz_m %d", ctx->geometry_type, ctx->has_z, ctx->has_m);
- GeometryReader reader(geometry, (GeometryType) ctx->geometry_type, ctx->has_z, ctx->has_m);
+ if (geometry != nullptr)
+ {
+ LWDEBUGF(3,
+ "Constructing GeometryReader with geometry_type %d has_z %d haz_m %d",
+ ctx->geometry_type,
+ ctx->has_z,
+ ctx->has_m);
+ GeometryReader reader(geometry, (GeometryType)ctx->geometry_type, ctx->has_z, ctx->has_m);
ctx->lwgeom = reader.read();
if (ctx->srid > 0)
lwgeom_set_srid(ctx->lwgeom, ctx->srid);
LWDEBUGG(3, ctx->lwgeom, "GeometryReader output LWGEOM");
- } else {
+ }
+ else
+ {
ctx->lwgeom = NULL;
}
- if (feature->properties() != nullptr && feature->properties()->size() != 0) {
- ctx->properties = (uint8_t *) feature->properties()->data();
+ if (feature->properties() != nullptr && feature->properties()->size() != 0)
+ {
+ ctx->properties = (uint8_t *)feature->properties()->data();
ctx->properties_len = feature->properties()->size();
- } else {
+ }
+ else
+ {
ctx->properties_len = 0;
}
return 0;
}
-int flatgeobuf_decode_header(ctx *ctx)
+int
+flatgeobuf_decode_header(ctx *ctx)
{
LWDEBUGF(2, "reading size prefix at %llu", ctx->offset);
auto size = flatbuffers::GetPrefixedSize(ctx->buf + ctx->offset);
LWDEBUGF(2, "size is %u (without size prefix)", size);
- Verifier verifier(ctx->buf + ctx->offset, size);
- if (VerifySizePrefixedHeaderBuffer(verifier)) {
+ Verifier verifier(ctx->buf + ctx->offset, flatgeobuf_size_prefixed_verifier_length(size));
+ if (!VerifySizePrefixedHeaderBuffer(verifier))
+ {
lwerror("buffer did not pass verification");
return -1;
}
@@ -283,7 +331,7 @@ int flatgeobuf_decode_header(ctx *ctx)
auto header = GetHeader(ctx->buf + ctx->offset);
ctx->offset += size;
- ctx->geometry_type = (uint8_t) header->geometry_type();
+ ctx->geometry_type = (uint8_t)header->geometry_type();
ctx->features_count = header->features_count();
ctx->has_z = header->has_z();
ctx->has_m = header->has_m();
@@ -294,24 +342,41 @@ int flatgeobuf_decode_header(ctx *ctx)
if (crs != nullptr)
ctx->srid = crs->code();
auto columns = header->columns();
- if (columns != nullptr) {
+ if (columns != nullptr)
+ {
auto size = columns->size();
- ctx->columns = (flatgeobuf_column **) lwalloc(sizeof(flatgeobuf_column *) * size);
+ ctx->columns = (flatgeobuf_column **)lwalloc(sizeof(flatgeobuf_column *) * size);
ctx->columns_size = size;
- for (uint32_t i = 0; i < size; i++) {
+ for (uint32_t i = 0; i < size; i++)
+ {
auto column = columns->Get(i);
- ctx->columns[i] = (flatgeobuf_column *) lwalloc(sizeof(flatgeobuf_column));
+ ctx->columns[i] = (flatgeobuf_column *)lwalloc(sizeof(flatgeobuf_column));
memset(ctx->columns[i], 0, sizeof(flatgeobuf_column));
ctx->columns[i]->name = column->name()->c_str();
- ctx->columns[i]->type = (uint8_t) column->type();
+ ctx->columns[i]->type = (uint8_t)column->type();
}
}
LWDEBUGF(2, "ctx->geometry_type: %d", ctx->geometry_type);
LWDEBUGF(2, "ctx->columns_len: %d", ctx->columns_size);
- if (ctx->index_node_size > 0 && ctx->features_count > 0) {
- auto treeSize = PackedRTree::size(ctx->features_count, ctx->index_node_size);
+ if (ctx->index_node_size > 0 && ctx->features_count > 0)
+ {
+ uint64_t treeSize;
+ try
+ {
+ treeSize = PackedRTree::size(ctx->features_count, ctx->index_node_size);
+ }
+ catch (const std::exception &e)
+ {
+ lwerror("flatgeobuf: invalid packed rtree metadata: %s", e.what());
+ return -1;
+ }
+ if (treeSize > ctx->size - ctx->offset)
+ {
+ lwerror("flatgeobuf: packed rtree exceeds remaining input");
+ return -1;
+ }
LWDEBUGF(2, "Adding tree size %llu to offset", treeSize);
ctx->offset += treeSize;
}
diff --git a/deps/flatgeobuf/geometryreader.cpp b/deps/flatgeobuf/geometryreader.cpp
index 8734c22255..413e56257c 100644
--- a/deps/flatgeobuf/geometryreader.cpp
+++ b/deps/flatgeobuf/geometryreader.cpp
@@ -27,14 +27,16 @@
using namespace flatbuffers;
using namespace FlatGeobuf;
-LWPOINT *GeometryReader::readPoint()
+LWPOINT *
+GeometryReader::readPoint()
{
POINTARRAY *pa;
POINT4D pt;
pa = ptarray_construct_empty(m_has_z, m_has_m, 1);
- if (m_geometry->xy() == nullptr || m_geometry->xy()->size() == 0) {
+ if (m_geometry->xy() == nullptr || m_geometry->xy()->size() == 0)
+ {
return lwpoint_construct(0, NULL, pa);
}
@@ -50,16 +52,16 @@ LWPOINT *GeometryReader::readPoint()
if (m_has_m)
m = m_geometry->m()->Get(m_offset);
- pt = (POINT4D) { x, y, z, m };
+ pt = (POINT4D){x, y, z, m};
ptarray_append_point(pa, &pt, LW_TRUE);
return lwpoint_construct(0, NULL, pa);
}
-POINTARRAY *GeometryReader::readPA()
+POINTARRAY *
+GeometryReader::readPA()
{
POINTARRAY *pa;
POINT4D pt;
- uint32_t npoints;
const auto xy = m_geometry->xy();
const auto z = m_has_z ? m_geometry->z() : nullptr;
@@ -67,7 +69,8 @@ POINTARRAY *GeometryReader::readPA()
pa = ptarray_construct_empty(m_has_z, m_has_m, m_length);
- for (uint32_t i = m_offset; i < m_offset + m_length; i++) {
+ for (uint32_t i = m_offset; i < m_offset + m_length; i++)
+ {
double xv = xy->Get(i * 2 + 0);
double yv = xy->Get(i * 2 + 1);
double zv = 0;
@@ -76,43 +79,53 @@ POINTARRAY *GeometryReader::readPA()
zv = z->Get(i);
if (m_has_m)
mv = m->Get(i);
- pt = (POINT4D) { xv, yv, zv, mv };
+ pt = (POINT4D){xv, yv, zv, mv};
ptarray_append_point(pa, &pt, LW_TRUE);
}
return pa;
}
-LWMPOINT *GeometryReader::readMultiPoint()
+LWMPOINT *
+GeometryReader::readMultiPoint()
{
POINTARRAY *pa = readPA();
return lwmpoint_construct(0, pa);
}
-LWLINE *GeometryReader::readLineString()
+LWLINE *
+GeometryReader::readLineString()
{
POINTARRAY *pa = readPA();
return lwline_construct(0, NULL, pa);
}
-LWMLINE *GeometryReader::readMultiLineString()
+LWMLINE *
+GeometryReader::readMultiLineString()
{
auto ends = m_geometry->ends();
+ const uint32_t totalPoints = m_length;
uint32_t ngeoms = 1;
if (ends != nullptr && ends->size() > 1)
ngeoms = ends->size();
auto *lwmline = lwmline_construct_empty(0, m_has_z, m_has_m);
- if (ngeoms > 1) {
- for (uint32_t i = 0; i < ngeoms; i++) {
+ if (ngeoms > 1)
+ {
+ for (uint32_t i = 0; i < ngeoms; i++)
+ {
const auto e = ends->Get(i);
+ if (e < m_offset || e > totalPoints)
+ lwerror("flatgeobuf: invalid geometry ends");
m_length = e - m_offset;
POINTARRAY *pa = readPA();
lwmline_add_lwline(lwmline, lwline_construct(0, NULL, pa));
m_offset = e;
}
- } else {
+ }
+ else
+ {
POINTARRAY *pa = readPA();
lwmline_add_lwline(lwmline, lwline_construct(0, NULL, pa));
}
@@ -120,86 +133,125 @@ LWMLINE *GeometryReader::readMultiLineString()
return lwmline;
}
-LWPOLY *GeometryReader::readPolygon()
+LWPOLY *
+GeometryReader::readPolygon()
{
const auto ends = m_geometry->ends();
+ const uint32_t totalPoints = m_length;
uint32_t nrings = 1;
if (ends != nullptr && ends->size() > 1)
nrings = ends->size();
- auto **ppa = (POINTARRAY **) lwalloc(sizeof(POINTARRAY *) * nrings);
- if (nrings > 1) {
- for (uint32_t i = 0; i < nrings; i++) {
+ auto **ppa = (POINTARRAY **)lwalloc(sizeof(POINTARRAY *) * nrings);
+ if (nrings > 1)
+ {
+ for (uint32_t i = 0; i < nrings; i++)
+ {
const auto e = ends->Get(i);
+ if (e < m_offset || e > totalPoints)
+ lwerror("flatgeobuf: invalid geometry ends");
m_length = e - m_offset;
ppa[i] = readPA();
m_offset = e;
}
- } else {
+ }
+ else
+ {
ppa[0] = readPA();
}
return lwpoly_construct(0, NULL, nrings, ppa);
}
-LWMPOLY *GeometryReader::readMultiPolygon()
+LWMPOLY *
+GeometryReader::readMultiPolygon()
{
auto parts = m_geometry->parts();
+ if (parts == nullptr)
+ lwerror("flatgeobuf: missing geometry parts");
+
auto *mp = lwmpoly_construct_empty(0, m_has_z, m_has_m);
- for (uoffset_t i = 0; i < parts->size(); i++) {
- GeometryReader reader { parts->Get(i), GeometryType::Polygon, m_has_z, m_has_m };
- const auto p = (LWPOLY *) reader.read();
+ for (uoffset_t i = 0; i < parts->size(); i++)
+ {
+ GeometryReader reader{parts->Get(i), GeometryType::Polygon, m_has_z, m_has_m};
+ const auto p = (LWPOLY *)reader.read();
lwmpoly_add_lwpoly(mp, p);
}
return mp;
}
-LWCOLLECTION *GeometryReader::readGeometryCollection()
+LWCOLLECTION *
+GeometryReader::readGeometryCollection()
{
auto parts = m_geometry->parts();
+ if (parts == nullptr)
+ lwerror("flatgeobuf: missing geometry parts");
+
auto *gc = lwcollection_construct_empty(COLLECTIONTYPE, 0, m_has_z, m_has_m);
- for (uoffset_t i = 0; i < parts->size(); i++) {
+ for (uoffset_t i = 0; i < parts->size(); i++)
+ {
auto part = parts->Get(i);
- GeometryReader reader { part, part->type(), m_has_z, m_has_m };
+ GeometryReader reader{part, part->type(), m_has_z, m_has_m};
const auto g = reader.read();
lwcollection_add_lwgeom(gc, g);
}
return gc;
}
-LWGEOM *GeometryReader::read()
+LWGEOM *
+GeometryReader::read()
{
// nested types
- switch (m_geometry_type) {
- case GeometryType::GeometryCollection: return (LWGEOM *) readGeometryCollection();
- case GeometryType::MultiPolygon: return (LWGEOM *) readMultiPolygon();
- /*case GeometryType::CompoundCurve: return readCompoundCurve();
- case GeometryType::CurvePolygon: return readCurvePolygon();
- case GeometryType::MultiCurve: return readMultiCurve();
- case GeometryType::MultiSurface: return readMultiSurface();
- case GeometryType::PolyhedralSurface: return readPolyhedralSurface();*/
- default: break;
+ switch (m_geometry_type)
+ {
+ case GeometryType::GeometryCollection:
+ return (LWGEOM *)readGeometryCollection();
+ case GeometryType::MultiPolygon:
+ return (LWGEOM *)readMultiPolygon();
+ /*case GeometryType::CompoundCurve: return readCompoundCurve();
+ case GeometryType::CurvePolygon: return readCurvePolygon();
+ case GeometryType::MultiCurve: return readMultiCurve();
+ case GeometryType::MultiSurface: return readMultiSurface();
+ case GeometryType::PolyhedralSurface: return readPolyhedralSurface();*/
+ default:
+ break;
}
// if not nested must have geometry data
const auto pXy = m_geometry->xy();
- const auto xySize = pXy->size();
- m_length = xySize / 2;
+ if (pXy == nullptr)
+ lwerror("flatgeobuf: missing geometry coordinates");
- switch (m_geometry_type) {
- case GeometryType::Point: return (LWGEOM *) readPoint();
- case GeometryType::MultiPoint: return (LWGEOM *) readMultiPoint();
- case GeometryType::LineString: return (LWGEOM *) readLineString();
- case GeometryType::MultiLineString: return (LWGEOM *) readMultiLineString();
- case GeometryType::Polygon: return (LWGEOM *) readPolygon();
- /*
- case GeometryType::CircularString: return readSimpleCurve<OGRCircularString>(true);
- case GeometryType::Triangle: return readTriangle();
- case GeometryType::TIN: return readTIN();
- */
- default:
- lwerror("flatgeobuf: GeometryReader::read: Unknown type %d", (int) m_geometry_type);
+ const auto xySize = pXy->size();
+ if (xySize % 2 != 0)
+ lwerror("flatgeobuf: invalid xy coordinate count");
+
+ m_length = xySize / 2;
+ if (m_has_z && (m_geometry->z() == nullptr || m_geometry->z()->size() < m_length))
+ lwerror("flatgeobuf: invalid z coordinate count");
+ if (m_has_m && (m_geometry->m() == nullptr || m_geometry->m()->size() < m_length))
+ lwerror("flatgeobuf: invalid m coordinate count");
+
+ switch (m_geometry_type)
+ {
+ case GeometryType::Point:
+ return (LWGEOM *)readPoint();
+ case GeometryType::MultiPoint:
+ return (LWGEOM *)readMultiPoint();
+ case GeometryType::LineString:
+ return (LWGEOM *)readLineString();
+ case GeometryType::MultiLineString:
+ return (LWGEOM *)readMultiLineString();
+ case GeometryType::Polygon:
+ return (LWGEOM *)readPolygon();
+ /*
+ case GeometryType::CircularString: return readSimpleCurve<OGRCircularString>(true);
+ case GeometryType::Triangle: return readTriangle();
+ case GeometryType::TIN: return readTIN();
+ */
+ default:
+ lwerror("flatgeobuf: GeometryReader::read: Unknown type %d", (int)m_geometry_type);
}
return nullptr;
}
diff --git a/doc/credits.xml b/doc/credits.xml
index 22416b412b..baf46ede69 100644
--- a/doc/credits.xml
+++ b/doc/credits.xml
@@ -422,6 +422,7 @@
<member>Roger Crew</member>
<member>Ron Mayer</member>
<member>Sam Peters</member>
+ <member>Sarath Kumar</member>
<member>Sergei Shoulbakov</member>
<member>Sergey Fedoseev</member>
<member>Shinichi Sugiyama</member>
@@ -485,6 +486,7 @@
<listitem><simpara><link xlink:href="https://www.google.com">Google</link></simpara></listitem>
<listitem><simpara><link xlink:href="https://www.highgo.com">HighGo</link></simpara></listitem>
<listitem><simpara>Hunter Systems Group</simpara></listitem>
+ <listitem><simpara><link xlink:href="https://iitmpravartak.org.in/">IITM Pravartak Security Team</link></simpara></listitem>
<listitem><simpara><link xlink:href="https://pti-agriambio.csic.es">INIA-CSIC</link></simpara></listitem>
<listitem><simpara><link xlink:href="https://www.isciences.com">ISciences, LLC</link></simpara></listitem>
<listitem><simpara><link xlink:href="https://www.jirotech.com">Jirotech (formerly LISAsoft)</link></simpara></listitem>
diff --git a/postgis/flatgeobuf.c b/postgis/flatgeobuf.c
index 37411e9f36..ee0ce6b50c 100644
--- a/postgis/flatgeobuf.c
+++ b/postgis/flatgeobuf.c
@@ -146,7 +146,9 @@ flatgeobuf_write_double_le(uint8_t *dst, double value)
flatgeobuf_write_le64(dst, bits);
}
-static uint8_t get_column_type(Oid typoid) {
+static uint8_t
+get_column_type(Oid typoid)
+{
switch (typoid)
{
case BOOLOID:
@@ -174,11 +176,11 @@ static uint8_t get_column_type(Oid typoid) {
case TIMESTAMPTZOID:
return flatgeobuf_column_type_datetime;
}
- elog(ERROR, "flatgeobuf: get_column_type: '%d' column type not supported",
- typoid);
+ elog(ERROR, "flatgeobuf: get_column_type: '%d' column type not supported", typoid);
}
-static void inspect_table(struct flatgeobuf_agg_ctx *ctx)
+static void
+inspect_table(struct flatgeobuf_agg_ctx *ctx)
{
flatgeobuf_column *c;
flatgeobuf_column **columns;
@@ -196,18 +198,24 @@ static void inspect_table(struct flatgeobuf_agg_ctx *ctx)
// inspect columns
// NOTE: last element will be unused if geom attr is found
- for (int i = 0; i < natts; i++) {
+ for (int i = 0; i < natts; i++)
+ {
Oid typoid = getBaseType(TupleDescAttr(tupdesc, i)->atttypid);
const char *key = TupleDescAttr(tupdesc, i)->attname.data;
POSTGIS_DEBUGF(2, "inspecting column definition for %s with oid %d", key, typoid);
- if (ctx->geom_name == NULL) {
- if (!geom_found && typoid == postgis_oid(GEOMETRYOID)) {
+ if (ctx->geom_name == NULL)
+ {
+ if (!geom_found && typoid == postgis_oid(GEOMETRYOID))
+ {
ctx->geom_index = i;
geom_found = true;
continue;
}
- } else {
- if (!geom_found && strcmp(key, ctx->geom_name) == 0) {
+ }
+ else
+ {
+ if (!geom_found && strcmp(key, ctx->geom_name) == 0)
+ {
ctx->geom_index = i;
geom_found = true;
continue;
@@ -215,7 +223,7 @@ static void inspect_table(struct flatgeobuf_agg_ctx *ctx)
}
POSTGIS_DEBUGF(2, "creating column definition for %s with oid %d", key, typoid);
- c = (flatgeobuf_column *) palloc0(sizeof(flatgeobuf_column));
+ c = (flatgeobuf_column *)palloc0(sizeof(flatgeobuf_column));
c->name = pstrdup(key);
c->type = get_column_type(typoid);
columns[columns_size] = c;
@@ -225,21 +233,25 @@ static void inspect_table(struct flatgeobuf_agg_ctx *ctx)
if (!geom_found)
elog(ERROR, "no geom column found");
- if (columns_size > 0) {
+ if (columns_size > 0)
+ {
ctx->ctx->columns = columns;
ctx->ctx->columns_size = columns_size;
}
}
// ensure properties has room for at least size
-static void ensure_properties_size(struct flatgeobuf_agg_ctx *ctx, size_t size)
+static void
+ensure_properties_size(struct flatgeobuf_agg_ctx *ctx, size_t size)
{
- if (ctx->ctx->properties_size == 0) {
+ if (ctx->ctx->properties_size == 0)
+ {
ctx->ctx->properties_size = 1024 * 4;
POSTGIS_DEBUGF(2, "flatgeobuf: properties buffer to size %d", ctx->ctx->properties_size);
ctx->ctx->properties = palloc(ctx->ctx->properties_size);
}
- if (ctx->ctx->properties_size < size) {
+ if (ctx->ctx->properties_size < size)
+ {
ctx->ctx->properties_size = ctx->ctx->properties_size * 2;
POSTGIS_DEBUGF(2, "flatgeobuf: reallocating properties buffer to size %d", ctx->ctx->properties_size);
ctx->ctx->properties = repalloc(ctx->ctx->properties, ctx->ctx->properties_size);
@@ -248,13 +260,16 @@ static void ensure_properties_size(struct flatgeobuf_agg_ctx *ctx, size_t size)
}
// ensure items have room for at least ctx->ctx->features_count + 1
-static void ensure_items_len(struct flatgeobuf_agg_ctx *ctx)
+static void
+ensure_items_len(struct flatgeobuf_agg_ctx *ctx)
{
- if (ctx->ctx->features_count == 0) {
+ if (ctx->ctx->features_count == 0)
+ {
ctx->ctx->items_len = 32;
ctx->ctx->items = palloc(sizeof(flatgeobuf_item *) * ctx->ctx->items_len);
}
- if (ctx->ctx->items_len < (ctx->ctx->features_count + 1)) {
+ if (ctx->ctx->items_len < (ctx->ctx->features_count + 1))
+ {
ctx->ctx->items_len = ctx->ctx->items_len * 2;
POSTGIS_DEBUGF(2, "flatgeobuf: reallocating items to len %lld", ctx->ctx->items_len);
ctx->ctx->items = repalloc(ctx->ctx->items, sizeof(flatgeobuf_item *) * ctx->ctx->items_len);
@@ -262,7 +277,8 @@ static void ensure_items_len(struct flatgeobuf_agg_ctx *ctx)
}
}
-static void encode_properties(flatgeobuf_agg_ctx *ctx)
+static void
+encode_properties(flatgeobuf_agg_ctx *ctx)
{
uint16_t ci = 0;
size_t offset = 0;
@@ -280,9 +296,10 @@ static void encode_properties(flatgeobuf_agg_ctx *ctx)
double double_value;
char *string_value;
- //Jsonb *jb;
+ // Jsonb *jb;
- for (i = 0; i < (uint32_t) ctx->tupdesc->natts; i++) {
+ for (i = 0; i < (uint32_t)ctx->tupdesc->natts; i++)
+ {
if (ctx->geom_index == i)
continue;
datum = GetAttributeByNum(ctx->row, i + 1, &isnull);
@@ -292,7 +309,8 @@ static void encode_properties(flatgeobuf_agg_ctx *ctx)
flatgeobuf_write_le16(ctx->ctx->properties + offset, ci);
offset += sizeof(ci);
typoid = getBaseType(TupleDescAttr(ctx->tupdesc, i)->atttypid);
- switch (typoid) {
+ switch (typoid)
+ {
case BOOLOID:
byte_value = DatumGetBool(datum) ? 1 : 0;
ensure_properties_size(ctx, offset + sizeof(byte_value));
@@ -358,38 +376,63 @@ static void encode_properties(flatgeobuf_agg_ctx *ctx)
offset += len;
break;
}
- // TODO: handle date/time types
- // case JSONBOID:
- // jb = DatumGetJsonbP(datum);
- // string_value = JsonbToCString(NULL, &jb->root, VARSIZE(jb));
- // len = strlen(string_value);
- // memcpy(data + offset, &len, sizeof(len));
- // offset += sizeof(len);
- // memcpy(data + offset, string_value, len);
- // offset += len;
- // break;
+ // TODO: handle date/time types
+ // case JSONBOID:
+ // jb = DatumGetJsonbP(datum);
+ // string_value = JsonbToCString(NULL, &jb->root, VARSIZE(jb));
+ // len = strlen(string_value);
+ // memcpy(data + offset, &len, sizeof(len));
+ // offset += sizeof(len);
+ // memcpy(data + offset, string_value, len);
+ // offset += len;
+ // break;
}
ci++;
}
- if (offset > 0) {
+ if (offset > 0)
+ {
POSTGIS_DEBUGF(3, "offset %ld", offset);
ctx->ctx->properties_len = offset;
}
}
-void flatgeobuf_check_magicbytes(struct flatgeobuf_decode_ctx *ctx)
+void
+flatgeobuf_check_magicbytes(struct flatgeobuf_decode_ctx *ctx)
{
- uint8_t *buf = ctx->ctx->buf + ctx->ctx->offset;
+ uint8_t *buf;
uint32_t i;
+ if (ctx->ctx->offset > ctx->ctx->size || FLATGEOBUF_MAGICBYTES_SIZE > ctx->ctx->size - ctx->ctx->offset)
+ elog(ERROR, "Data is not FlatGeobuf");
+
+ buf = ctx->ctx->buf + ctx->ctx->offset;
for (i = 0; i < FLATGEOBUF_MAGICBYTES_SIZE / 2; i++)
if (buf[i] != flatgeobuf_magicbytes[i])
elog(ERROR, "Data is not FlatGeobuf");
ctx->ctx->offset += FLATGEOBUF_MAGICBYTES_SIZE;
}
-static void decode_properties(struct flatgeobuf_decode_ctx *ctx, Datum *values, bool *isnull)
+void
+flatgeobuf_check_sizeprefix(flatgeobuf_ctx *ctx)
+{
+ uint64_t remaining;
+ uint32_t size;
+
+ if (ctx->offset > ctx->size)
+ elog(ERROR, "flatgeobuf: read past end of input");
+
+ remaining = ctx->size - ctx->offset;
+ if (remaining < sizeof(uint32_t))
+ elog(ERROR, "flatgeobuf: truncated size prefix");
+
+ size = flatgeobuf_read_le32(ctx->buf + ctx->offset);
+ if ((uint64_t)size > remaining - sizeof(uint32_t))
+ elog(ERROR, "flatgeobuf: size prefix exceeds remaining input");
+}
+
+static void
+decode_properties(struct flatgeobuf_decode_ctx *ctx, Datum *values, bool *isnull)
{
uint16_t i, ci;
flatgeobuf_column *column;
@@ -402,7 +445,8 @@ static void decode_properties(struct flatgeobuf_decode_ctx *ctx, Datum *values,
if (size > 0 && size < (sizeof(uint16_t) + sizeof(uint8_t)))
elog(ERROR, "flatgeobuf: decode_properties: Unexpected properties data size %d", size);
- while (offset + 1 < size) {
+ while (offset + 1 < size)
+ {
if (offset + sizeof(uint16_t) > size)
elog(ERROR, "flatgeobuf: decode_properties: Unexpected offset %d", offset);
i = flatgeobuf_read_le16(data + offset);
@@ -413,7 +457,8 @@ static void decode_properties(struct flatgeobuf_decode_ctx *ctx, Datum *values,
column = ctx->ctx->columns[i];
type = column->type;
isnull[ci] = false;
- switch (type) {
+ switch (type)
+ {
case flatgeobuf_column_type_bool: {
uint8_t value;
if (offset + sizeof(uint8_t) > size)
@@ -473,7 +518,7 @@ static void decode_properties(struct flatgeobuf_decode_ctx *ctx, Datum *values,
if (offset + sizeof(uint32_t) > size)
elog(ERROR, "flatgeobuf: decode_properties: Invalid size for uint value");
value = flatgeobuf_read_le32(data + offset);
- values[ci] = Int64GetDatum((int64_t)(uint64_t) value);
+ values[ci] = Int64GetDatum((int64_t)(uint64_t)value);
offset += sizeof(uint32_t);
break;
}
@@ -519,7 +564,9 @@ static void decode_properties(struct flatgeobuf_decode_ctx *ctx, Datum *values,
elog(ERROR, "flatgeobuf: decode_properties: Invalid size for string value");
len = flatgeobuf_read_le32(data + offset);
offset += sizeof(len);
- values[ci] = PointerGetDatum(cstring_to_text_with_len((const char *) data + offset, len));
+ if (len > size - offset)
+ elog(ERROR, "flatgeobuf: decode_properties: string length exceeds buffer");
+ values[ci] = PointerGetDatum(cstring_to_text_with_len((const char *)data + offset, len));
offset += len;
break;
}
@@ -539,12 +586,14 @@ static void decode_properties(struct flatgeobuf_decode_ctx *ctx, Datum *values,
DateTimeErrorExtra extra;
#endif
if (offset + sizeof(len) > size)
- elog(ERROR, "flatgeobuf: decode_properties: Invalid size for string value");
+ elog(ERROR, "flatgeobuf: decode_properties: Invalid size for datetime value");
len = flatgeobuf_read_le32(data + offset);
offset += sizeof(len);
- buf = palloc0(len + 1);
- memcpy(buf, (const char *) data + offset, len);
- ParseDateTime((const char *) buf, workbuf, sizeof(workbuf), field, ftype, MAXDATEFIELDS, &nf);
+ if (len > size - offset)
+ elog(ERROR, "flatgeobuf: decode_properties: datetime length exceeds buffer");
+ buf = palloc0((Size)len + 1);
+ memcpy(buf, (const char *)data + offset, len);
+ ParseDateTime((const char *)buf, workbuf, sizeof(workbuf), field, ftype, MAXDATEFIELDS, &nf);
#if POSTGIS_PGSQL_VERSION >= 160
DecodeDateTime(field, ftype, nf, &dtype, tm, &fsec, &tzp, &extra);
@@ -569,28 +618,33 @@ static void decode_properties(struct flatgeobuf_decode_ctx *ctx, Datum *values,
elog(ERROR, "flatgeobuf: decode_properties: Unknown type %d", type);
}
}
-
}
-void flatgeobuf_decode_row(struct flatgeobuf_decode_ctx *ctx)
+void
+flatgeobuf_decode_row(struct flatgeobuf_decode_ctx *ctx)
{
HeapTuple heapTuple;
uint32_t natts = ctx->tupdesc->natts;
Datum *values = palloc0(natts * sizeof(Datum));
bool *isnull = palloc(natts * sizeof(bool));
- for (uint32_t j = 0; j < natts; j++) isnull[j] = true;
+ for (uint32_t j = 0; j < natts; j++)
+ isnull[j] = true;
isnull[0] = false;
values[0] = Int32GetDatum(ctx->fid);
+ flatgeobuf_check_sizeprefix(ctx->ctx);
if (flatgeobuf_decode_feature(ctx->ctx))
elog(ERROR, "flatgeobuf_decode_feature: unsuccessful");
- if (ctx->ctx->lwgeom != NULL) {
+ if (ctx->ctx->lwgeom != NULL)
+ {
values[1] = PointerGetDatum(geometry_serialize(ctx->ctx->lwgeom));
isnull[1] = false;
- } else {
+ }
+ else
+ {
POSTGIS_DEBUG(3, "geometry is null");
isnull[1] = true;
}
@@ -604,7 +658,11 @@ void flatgeobuf_decode_row(struct flatgeobuf_decode_ctx *ctx)
POSTGIS_DEBUGF(3, "fid now %d", ctx->fid);
- if (ctx->ctx->offset == ctx->ctx->size) {
+ if (ctx->ctx->offset > ctx->ctx->size)
+ elog(ERROR, "flatgeobuf_decode_row: read past end of input");
+
+ if (ctx->ctx->offset == ctx->ctx->size)
+ {
POSTGIS_DEBUGF(3, "reached end at %lld", ctx->ctx->offset);
ctx->done = true;
}
@@ -613,7 +671,8 @@ void flatgeobuf_decode_row(struct flatgeobuf_decode_ctx *ctx)
/**
* Initialize aggregation context.
*/
-struct flatgeobuf_agg_ctx *flatgeobuf_agg_ctx_init(const char *geom_name, const bool create_index)
+struct flatgeobuf_agg_ctx *
+flatgeobuf_agg_ctx_init(const char *geom_name, const bool create_index)
{
struct flatgeobuf_agg_ctx *ctx;
size_t size = VARHDRSZ + FLATGEOBUF_MAGICBYTES_SIZE;
@@ -637,7 +696,8 @@ struct flatgeobuf_agg_ctx *flatgeobuf_agg_ctx_init(const char *geom_name, const
* Allocates a new feature, increment feature counter and
* encode properties into it.
*/
-void flatgeobuf_agg_transfn(struct flatgeobuf_agg_ctx *ctx)
+void
+flatgeobuf_agg_transfn(struct flatgeobuf_agg_ctx *ctx)
{
LWGEOM *lwgeom = NULL;
bool isnull = false;
@@ -648,8 +708,9 @@ void flatgeobuf_agg_transfn(struct flatgeobuf_agg_ctx *ctx)
inspect_table(ctx);
datum = GetAttributeByNum(ctx->row, ctx->geom_index + 1, &isnull);
- if (!isnull) {
- gs = (GSERIALIZED *) PG_DETOAST_DATUM_COPY(datum);
+ if (!isnull)
+ {
+ gs = (GSERIALIZED *)PG_DETOAST_DATUM_COPY(datum);
lwgeom = lwgeom_from_gserialized(gs);
}
ctx->ctx->lwgeom = lwgeom;
@@ -668,15 +729,19 @@ void flatgeobuf_agg_transfn(struct flatgeobuf_agg_ctx *ctx)
*
* Encode into Data message and return it packed as a bytea.
*/
-uint8_t *flatgeobuf_agg_finalfn(struct flatgeobuf_agg_ctx *ctx)
+uint8_t *
+flatgeobuf_agg_finalfn(struct flatgeobuf_agg_ctx *ctx)
{
POSTGIS_DEBUGF(3, "called at offset %lld", ctx->ctx->offset);
if (ctx == NULL)
flatgeobuf_agg_ctx_init(NULL, false);
// header only result
- if (ctx->ctx->features_count == 0) {
+ if (ctx->ctx->features_count == 0)
+ {
flatgeobuf_encode_header(ctx->ctx);
- } else if (ctx->ctx->create_index) {
+ }
+ else if (ctx->ctx->create_index)
+ {
ctx->ctx->index_node_size = 16;
flatgeobuf_create_index(ctx->ctx);
}
diff --git a/postgis/flatgeobuf.h b/postgis/flatgeobuf.h
index c3b13cb6cc..ddad548556 100644
--- a/postgis/flatgeobuf.h
+++ b/postgis/flatgeobuf.h
@@ -43,8 +43,7 @@
#include "lwgeom_log.h"
#include "flatgeobuf_c.h"
-typedef struct flatgeobuf_agg_ctx
-{
+typedef struct flatgeobuf_agg_ctx {
flatgeobuf_ctx *ctx;
const char *geom_name;
uint32_t geom_index;
@@ -52,13 +51,11 @@ typedef struct flatgeobuf_agg_ctx
HeapTupleHeader row;
} flatgeobuf_agg_ctx;
-
flatgeobuf_agg_ctx *flatgeobuf_agg_ctx_init(const char *geom_name, const bool create_index);
void flatgeobuf_agg_transfn(flatgeobuf_agg_ctx *ctx);
uint8_t *flatgeobuf_agg_finalfn(flatgeobuf_agg_ctx *ctx);
-typedef struct flatgeobuf_decode_ctx
-{
+typedef struct flatgeobuf_decode_ctx {
flatgeobuf_ctx *ctx;
TupleDesc tupdesc;
Datum result;
@@ -68,6 +65,7 @@ typedef struct flatgeobuf_decode_ctx
} flatgeobuf_decode_ctx;
void flatgeobuf_check_magicbytes(struct flatgeobuf_decode_ctx *ctx);
+void flatgeobuf_check_sizeprefix(flatgeobuf_ctx *ctx);
void flatgeobuf_decode_row(struct flatgeobuf_decode_ctx *ctx);
#endif
diff --git a/postgis/lwgeom_in_flatgeobuf.c b/postgis/lwgeom_in_flatgeobuf.c
index 235fbbeed8..a71a30abbc 100644
--- a/postgis/lwgeom_in_flatgeobuf.c
+++ b/postgis/lwgeom_in_flatgeobuf.c
@@ -22,7 +22,6 @@
*
**********************************************************************/
-
#include <assert.h>
#include "postgres.h"
@@ -36,8 +35,11 @@
#include <utils/builtins.h>
#include "flatgeobuf.h"
-static char *get_pgtype(uint8_t column_type) {
- switch (column_type) {
+static char *
+get_pgtype(uint8_t column_type)
+{
+ switch (column_type)
+ {
case flatgeobuf_column_type_bool:
return "boolean";
case flatgeobuf_column_type_byte:
@@ -71,11 +73,21 @@ static const char *
flatgeobuf_type_name(uint8_t fgb_type)
{
/* Names match FlatGeobuf::EnumNamesColumnType() in header_generated.h */
- static const char * const names[] = {
- "Byte", "UByte", "Bool", "Short", "UShort",
- "Int", "UInt", "Long", "ULong",
- "Float", "Double", "String", "Json", "DateTime", "Binary"
- };
+ static const char *const names[] = {"Byte",
+ "UByte",
+ "Bool",
+ "Short",
+ "UShort",
+ "Int",
+ "UInt",
+ "Long",
+ "ULong",
+ "Float",
+ "Double",
+ "String",
+ "Json",
+ "DateTime",
+ "Binary"};
if (fgb_type >= sizeof(names) / sizeof(names[0]))
return "unknown";
return names[fgb_type];
@@ -106,8 +118,7 @@ flatgeobuf_type_compatible(uint8_t fgb_type, Oid pgtype)
case flatgeobuf_column_type_string:
return pgtype == TEXTOID || pgtype == VARCHAROID;
case flatgeobuf_column_type_datetime:
- return pgtype == DATEOID || pgtype == TIMEOID ||
- pgtype == TIMESTAMPOID || pgtype == TIMESTAMPTZOID;
+ return pgtype == DATEOID || pgtype == TIMEOID || pgtype == TIMESTAMPOID || pgtype == TIMESTAMPTZOID;
case flatgeobuf_column_type_json:
return pgtype == JSONBOID;
case flatgeobuf_column_type_binary:
@@ -117,7 +128,8 @@ flatgeobuf_type_compatible(uint8_t fgb_type, Oid pgtype)
}
PG_FUNCTION_INFO_V1(pgis_tablefromflatgeobuf);
-Datum pgis_tablefromflatgeobuf(PG_FUNCTION_ARGS)
+Datum
+pgis_tablefromflatgeobuf(PG_FUNCTION_ARGS)
{
struct flatgeobuf_decode_ctx *ctx;
text *schema_input;
@@ -150,14 +162,16 @@ Datum pgis_tablefromflatgeobuf(PG_FUNCTION_ARGS)
ctx->ctx->offset = 0;
flatgeobuf_check_magicbytes(ctx);
+ flatgeobuf_check_sizeprefix(ctx->ctx);
flatgeobuf_decode_header(ctx->ctx);
initStringInfo(&sql);
- appendStringInfo(&sql, "create table %s.%s (id int, geom geometry",
- quote_identifier(schema), quote_identifier(table));
+ appendStringInfo(
+ &sql, "create table %s.%s (id int, geom geometry", quote_identifier(schema), quote_identifier(table));
POSTGIS_DEBUGF(2, "found %d columns", ctx->ctx->columns_size);
- for (i = 0; i < ctx->ctx->columns_size; i++) {
+ for (i = 0; i < ctx->ctx->columns_size; i++)
+ {
flatgeobuf_column *column = ctx->ctx->columns[i];
const char *name = column->name;
uint8_t column_type = column->type;
@@ -187,7 +201,8 @@ Datum pgis_tablefromflatgeobuf(PG_FUNCTION_ARGS)
// https://stackoverflow.com/questions/11740256/refactor-a-pl-pgsql-function-to-return-the-output-of-various-select-queries
PG_FUNCTION_INFO_V1(pgis_fromflatgeobuf);
-Datum pgis_fromflatgeobuf(PG_FUNCTION_ARGS)
+Datum
+pgis_fromflatgeobuf(PG_FUNCTION_ARGS)
{
FuncCallContext *funcctx;
@@ -197,7 +212,8 @@ Datum pgis_fromflatgeobuf(PG_FUNCTION_ARGS)
struct flatgeobuf_decode_ctx *ctx;
- if (SRF_IS_FIRSTCALL()) {
+ if (SRF_IS_FIRSTCALL())
+ {
funcctx = SRF_FIRSTCALL_INIT();
oldcontext = MemoryContextSwitchTo(funcctx->multi_call_memory_ctx);
@@ -205,8 +221,14 @@ Datum pgis_fromflatgeobuf(PG_FUNCTION_ARGS)
if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("first argument of function must be composite type")));
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("first argument of function must be composite type")));
+
+ if (PG_ARGISNULL(1))
+ {
+ MemoryContextSwitchTo(oldcontext);
+ SRF_RETURN_DONE(funcctx);
+ }
data = PG_GETARG_BYTEA_PP(1);
@@ -223,18 +245,21 @@ Datum pgis_fromflatgeobuf(PG_FUNCTION_ARGS)
funcctx->user_fctx = ctx;
- if (ctx->ctx->size == 0) {
+ if (ctx->ctx->size == 0)
+ {
POSTGIS_DEBUG(2, "no data");
MemoryContextSwitchTo(oldcontext);
SRF_RETURN_DONE(funcctx);
}
flatgeobuf_check_magicbytes(ctx);
+ flatgeobuf_check_sizeprefix(ctx->ctx);
flatgeobuf_decode_header(ctx->ctx);
POSTGIS_DEBUGF(2, "header decoded now at offset %lld", ctx->ctx->offset);
- if (ctx->ctx->size == ctx->ctx->offset) {
+ if (ctx->ctx->size == ctx->ctx->offset)
+ {
POSTGIS_DEBUGF(2, "no feature data offset %lld", ctx->ctx->offset);
MemoryContextSwitchTo(oldcontext);
SRF_RETURN_DONE(funcctx);
@@ -246,8 +271,9 @@ Datum pgis_fromflatgeobuf(PG_FUNCTION_ARGS)
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
errmsg("flatgeobuf: column count mismatch: "
- "file has %u columns, target type has %d",
- ctx->ctx->columns_size, tupdesc->natts - 2)));
+ "file has %u columns, target type has %d",
+ ctx->ctx->columns_size,
+ tupdesc->natts - 2)));
for (uint16_t col_i = 0; col_i < ctx->ctx->columns_size; col_i++)
{
@@ -257,10 +283,10 @@ Datum pgis_fromflatgeobuf(PG_FUNCTION_ARGS)
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
errmsg("flatgeobuf: column \"%s\" type mismatch: "
- "file type \"%s\" is not compatible with PostgreSQL type %s",
- col->name,
- flatgeobuf_type_name(col->type),
- format_type_be(pgtype))));
+ "file type \"%s\" is not compatible with PostgreSQL type %s",
+ col->name,
+ flatgeobuf_type_name(col->type),
+ format_type_be(pgtype))));
}
MemoryContextSwitchTo(oldcontext);
@@ -269,11 +295,14 @@ Datum pgis_fromflatgeobuf(PG_FUNCTION_ARGS)
funcctx = SRF_PERCALL_SETUP();
ctx = funcctx->user_fctx;
- if (!ctx->done) {
+ if (!ctx->done)
+ {
flatgeobuf_decode_row(ctx);
POSTGIS_DEBUG(2, "Calling SRF_RETURN_NEXT");
SRF_RETURN_NEXT(funcctx, ctx->result);
- } else {
+ }
+ else
+ {
POSTGIS_DEBUG(2, "Calling SRF_RETURN_DONE");
SRF_RETURN_DONE(funcctx);
}
diff --git a/regress/core/flatgeobuf.sql b/regress/core/flatgeobuf.sql
index f5390a258b..6d8ba9148e 100644
--- a/regress/core/flatgeobuf.sql
+++ b/regress/core/flatgeobuf.sql
@@ -176,6 +176,17 @@ select 'MM1' from ST_FromFlatGeobuf(null::flatgeobuf_mm_text, (
select 'MM2' from ST_FromFlatGeobuf(null::flatgeobuf_mm_long, (
select ST_AsFlatGeobuf(q) fgb from (select null::geometry, 42::bigint as val1, 43::bigint as val2) q));
+select '--- Malformed input detection ---';
+
+-- Magic bytes plus a truncated size prefix.
+select 'MI1' from ST_FromFlatGeobuf(null::flatgeobuf_t1, '\x6667620366676201010203'::bytea);
+
+-- Valid magic bytes plus a header size prefix that exceeds the remaining input.
+select 'MI2' from ST_FromFlatGeobuf(null::flatgeobuf_t1, '\x6667620366676201ffffffff'::bytea);
+
+-- NULL bytea input should be handled by STRICT rather than reaching the C decoder.
+select 'MI3', count(*) from ST_FromFlatGeobuf(null::flatgeobuf_t1, null::bytea);
+
select '--- Quoted identifiers ---';
-- Verify that special characters in column names are properly quoted
diff --git a/regress/core/flatgeobuf_expected b/regress/core/flatgeobuf_expected
index 6e97ee4250..b9ef9de1be 100644
--- a/regress/core/flatgeobuf_expected
+++ b/regress/core/flatgeobuf_expected
@@ -27,6 +27,10 @@ E1|0|t|POINT(1.1 2.1)|f
--- Type mismatch detection ---
ERROR: flatgeobuf: column "val" type mismatch: file type "Long" is not compatible with PostgreSQL type text
ERROR: flatgeobuf: column count mismatch: file has 2 columns, target type has 1
+--- Malformed input detection ---
+ERROR: flatgeobuf: truncated size prefix
+ERROR: flatgeobuf: size prefix exceeds remaining input
+MI3|0
--- Quoted identifiers ---
QI1
QI2
-----------------------------------------------------------------------
Summary of changes:
NEWS | 4 +
deps/flatgeobuf/flatgeobuf_c.cpp | 177 +++++++++++++++++++++++++------------
deps/flatgeobuf/geometryreader.cpp | 148 +++++++++++++++++++++----------
doc/credits.xml | 2 +
postgis/flatgeobuf.c | 177 +++++++++++++++++++++++++------------
postgis/flatgeobuf.h | 8 +-
postgis/lwgeom_in_flatgeobuf.c | 85 ++++++++++++------
regress/core/flatgeobuf.sql | 11 +++
regress/core/flatgeobuf_expected | 4 +
9 files changed, 423 insertions(+), 193 deletions(-)
hooks/post-receive
--
PostGIS
More information about the postgis-tickets
mailing list