[mapserver-users] IMS Emulator

[Paul Ramsey]

Mapserverers:

  The perl-based IMS emulator is available for download at
http://postgis.refractions.net/imsemu-0.1.tar.gz 
  
  It is pretty (very) rough, but it works. On the to do list are such
unimportant features as querying. It turns out that doing querying for
ArcGIS clients is somewhat harder than one might hope, because the
transfer format is an unspecified binary format. Attached is a
preliminary spec for that format. If anyone has additions, I would love
to hear them.


PostGISers:

  The IMS emulator, once extended to do feature serving using the binary
format, will allow one to use a PostGIS datasource as if it were an
ArcIMS feature server within ArcGIS 8. This will give us a nice (read
only) client to PostGIS in addition to Mapserver.
-------------- next part --------------
ArcIMS GET_FEATURES "Compressed Binary Format"

GENERAL PRINCIPLES

When a client calls GET_FEATURES on ArcIMS without specifying an outputmode,
the default mode is to return "Compressed Binary Format" (CBF). CBF appears to
be completely undocumented. I have attempted to understand what makes up a CBF
file.

CBF is not OpenGIS Well-Known Binary, it is not the ESRI Shape File format, and
it is not the "compressed binary" format used by ArcSDE. It is an odd hybrid all
its own, combining storage for attribute information alongside coordinate
information.

This specification still has some holes, where "mystery bytes" live whose
purpose has not been determined yet.

The CBF file has the following general format:

  FILE HEADER
  [ATTRIBUTE FIELD DESCRIPTION 1]
  [ATTRIBUTE FIELD DESCRIPTION 2]
  [ATTRIBUTE FIELD DESCRIPTION 3]
  ...
  [ATTRIBUTE FIELD DESCRIPTION N]
  ATTRIBUTE FOOTER
  [SHAPE HEADER 1]
  [SHAPE COORDINATES 1]
  [SHAPE ATTRIBUTE VALUES 1]
  [SHAPE HEADER 2]
  [SHAPE COORDINATES 2]
  [SHAPE ATTRIBUTE VALUES 2]
  ...
  [SHAPE HEADER X]
  [SHAPE COORDINATES X]
  [SHAPE ATTRIBUTE VALUES X]
  FILE END


FILE HEADER

Bytes 1-4

  Standard header values. The first four bytes do not appear to be the same all
  the time. The standard values for the header (in hex) are: 05 18 01 00

Bytes 5-8

  Unknown header bytes. These four bytes vary for every sample I have, but I
  have not figured out what they mean yet.

ATTRIBUTE FIELD DESCRIPTION

This gives information about the attribute fields. The type is important,
because type is necessary in order to read the attribute values farther below.
Integer types will always have values stored in 4-byte integers. Floating point
types will always have values stored in 8-byte doubles. Strings will be stored
as variable length, with null-termination to indicate the end of the value.

Shape fields (-98) always have 0 precision and size.
Object ID fields always have 0 precision and size 10.

Bytes 1-2

  Field type number. Signed two-byte integer.

    4 - Integer
    8 - Floating Point Double
   12 - Variable Length String
  -98 - Shape
  -99 - Object ID

Byte 3

  Field precision. Unsigned, one byte. Corresponds to usual DBF file description
  of numeric precision. e.g. number(12,5) where 5 is the precision.

Bytes 4-5

  Field size. Unsigned, two bytes. Corresponds to usual DBF file description
  of numeric size. e.g. number(12,5) where 12 is the size.

Bytes 6-N

  Field name. Null-terminated character array. Read until you hit the null
  value (00).


ATTRIBUTE FOOTER

Bytes 1-9

After the attribute field information, there always seems to be a constant set
of nine bytes: 08 43 41 C3 79 37 E0 80 00


SHAPE HEADER

I do not competely understand the shape header right now. In addition to the
information given below about the basic structure (with types 1, 2 and 3)
there is another type, 9, which occurs only sometimes, and is followed by a
variable number of bytes of mostly null values. I am guessing for now that it is
possible to write valid CBF files completely ignoring the 9 type and just using
the 1, 2 and 3 types, which are at least self-consistent.

Byte 1

  Shape type. One-byte value.

  1 - MultiPoint
  2 - MultiLine
  3 - MultiPolygon

Bytes 2-5

  Number of parts in the shape (points in multipoint, lines in multiline, rings
  plus holes in multipolygon). Signed four-byte integer.

SHAPE COORDINATES

Each part of the shape consists of a length, and then the double-value
vertices. The vertices are stored as offsets relative to the previous vertex.
So, the first vertex is a spatial coordinate, and all the rest are relative
offsets from the previous vertex.

Bytes 1-4

  Number of vertices in the part. Signed four-byte integer.

Bytes 5-N

  Vertices as eight-byte doubles, calculated in the following manner:

  Coordinate 1: X1, Y1
  Coordinate 2: (X2-X1), (Y2-Y1)
  Coordinate 3: (X3-X2), (Y3-Y2)
  ...
  Coordinate N: (XN - X(N-1)), (YN - Y(N-1))

Example:

  Spatial coordinates:

  (2,3) (2,4) (5,6) (8,3)

  Corresponding ESRI CBF coordinates:

  (2,3) (0,1) (3,2) (3,-3)

SHAPE ATTRIBUTE VALUES

Attribute values corresponding to the attribute fields defined in the ATTRIBUTE
FIELD DESCRIPTIONGS. Integers are encoded as 4-byte signed integers. Floats are
encoded as 8-byte doubles. Strings are null-terminated variable-length arrays of
characters (you know you're at the end when you hit the null).

The "SHAPE" attribute field is *ignored* in the attribute values. No value is
entered for it at all, just skip right over.


FILE END

The CBF file always ends with a single end-of-file hex byte: FF