[mapserver-dev] encodings

[Frank Warmerdam]

Paul Ramsey wrote:
> Agree. Step one, a proposal that specifies the rules of the game. The
> problem right now is the rules are pretty unclear. What is the
> internal encoding for Mapserver? etc. Incidentally, making all
> internal string handling UTF8 and then setting the MAP and LAYER flags
> to indicate what the inputs are would be a nice touch.

Folks,

My personal opinion is that we should work towards making UTF8 the internal
representation.  It would be up to data sources to convert on the fly to
UTF-8, and when needed we could convert on output.

Interesting strings in the mapfile could also be provided in utf-8.

It should not be necessary to specify encoding anywhere except if some
input datasources have no way of knowing their encoding (accurately)
in which case perhaps there should be a mechanism to set it as a user.

This was the approach taken in OGR.

Thomas has noted in IRC that rigerously handling UTF-8 strings requires
careful handling of where the character boundaries are in complex strings.
My experience in OGR has been that this is seldom an issue, but then i'm
a bit of a hack and willing to take risks that others are sometimes not
willing to.

Best regards,
-- 
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam, warmerdam at pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | Geospatial Programmer for Rent