[PostGIS] #5910: ST_AsBinary, ST_AsEWKB, and ST_AsHEXEWKB return results even when given invalid byte order strings

[PostGIS]

#5910: ST_AsBinary, ST_AsEWKB, and ST_AsHEXEWKB return results even when given
invalid byte order strings
------------------------------------------------+--------------------------
 Reporter:  dmyzl                               |      Owner:  pramsey
     Type:  defect                              |     Status:  new
 Priority:  medium                              |  Milestone:  PostGIS
                                                |  3.5.4
Component:  postgis                             |    Version:  3.5.x
 Keywords:  ST_AsBinary ST_AsEWKB ST_AsHEXEWKB  |
------------------------------------------------+--------------------------
 The functions ST_AsEWKB, ST_AsBinary, and ST_AsHEXEWKB accept an optional
 second argument that specifies the byte order of the output. According to
 the documentation, the valid values are 'XDR' (big endian) and 'NDR'
 (little endian).

 However, I noticed that if other arbitrary strings are passed as the
 second argument, the functions still return a result — and always in
 little-endian order. For example:

 {{{
 select ST_AsEWKB(ST_GeomFromText( 'point empty' ),'aaaa');
 select ST_AsBinary(ST_GeomFromText( 'point empty' ),'aaaa');
 select ST_AsHEXEWKB(ST_GeomFromText( 'point empty' ),'aaaa');
 }}}

 I'm not sure whether this behavior is intended, but the documentation
 (https://postgis.net/docs/manual-3.5/ST_AsEWKB.html) does not mention that
 unrecognized values will default to little endian.

 postgis_full_version:

 {{{
 POSTGIS="3.5.3 0" [EXTENSION] PGSQL="170" GEOS="3.13.1-CAPI-1.19.2"
 SFCGAL="SFCGAL 1.5.0, CGAL 5.6, BOOST 1.74.0" PROJ="9.3.1
 NETWORK_ENABLED=OFF URL_ENDPOINT=https://cdn.proj.org
 USER_WRITABLE_DIRECTORY=/home/yzl/.local/share/proj
 DATABASE_PATH=/home/yzl/project/build_pg_postgis/proj-9.3.1/tmp_install/share/proj/proj.db"
 (compiled against PROJ 9.3.1) GDAL="GDAL 3.7.2, released 2023/09/05"
 LIBXML="2.9.13" LIBJSON="0.17" LIBPROTOBUF="1.3.3" WAGYU="0.5.0
 (Internal)" (core procs from "3.5.2 dea6d0a" need upgrade) TOPOLOGY
 (topology procs from "3.5.2 dea6d0a" need upgrade) RASTER (raster procs
 from "3.5.2 dea6d0a" need upgrade) (sfcgal procs from "3.5.2 dea6d0a" need
 upgrade)
 }}}
-- 
Ticket URL: <https://trac.osgeo.org/postgis/ticket/5910>
PostGIS <http://trac.osgeo.org/postgis/>
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.