[mapserver-dev] Text editors adding UTF-8 BOM to mapfiles?

[Norman Vine]

I agree that this can be very annoying

+1 for ignoring a leading BOM code

Note I suggest advising clients to switch to Notepad++ or another editor that
will not automatically add this

Norman

On Jan 6, 2015, at 4:39 PM, Daniel Morissette <dmorissette at mapgears.com> wrote:

> I never encountered this, but after reading about the BOM character for the first time [1], I am thinking that perhaps it would be worth modifying the MapServer mapfile parser to simply ignore the UTF-8 BOM code (EF BB BF) if it is present at the beginning of a map file since it should be safe to do so.
> 
> However the other BOM values (for UTF-16 and UTF-32) should proably not be skipped as they are indicating that the file is binary encoded, and in this case we really need to care about the BOM and read the file as binary which would involve much more work in the parser.
> 
> My 0.02$
> 
> Daniel
> 
> 
> [1] http://www.unicode.org/faq/utf_bom.html#bom5
> 
> 
> On 2015-01-06 3:18 PM, Stephen Woodbridge wrote:
>> Hi all,
>> 
>> I have run into a bunch of clients that edit their mapfiles (mostly on
>> windows) and then get a mapserver error: about the mapfile not starting
>> with the token MAP.
>> 
>> This is very hard to run down because you can not see these garbage
>> characters in most editors. Doing a hex dump I get:
>> 
>> 00000000  ef bb bf 4d 41 50 0a 20  20 4e 41 4d 45 20 22 4a  |...MAP.
>> NAME "J|
>> 
>> In vim, I just found out that I can:
>> 
>> :set nobomb
>> :w
>> 
>> And that will remove it. Did I mention that this is really annoying
>> because it is not obvious what the problem is!
>> 
>> Questions:
>> 
>> 1) anyone else run into this?
>> 2) would it be a major security risk to ignore the BOM if it exists?
>> 3) should I open a bug for this?
>> 4) this probably happens on other text files also, but my guess is that
>> they are less likely to be edited by the client so I'm not seeing it as
>> a problem.
>> 
>> Thoughts?
>> 
>> -Steve W
>> _______________________________________________
>> mapserver-dev mailing list
>> mapserver-dev at lists.osgeo.org
>> http://lists.osgeo.org/mailman/listinfo/mapserver-dev
> 
> 
> -- 
> Daniel Morissette
> T: +1 418-696-5056 #201
> http://www.mapgears.com/
> Provider of Professional MapServer Support since 2000
> _______________________________________________
> mapserver-dev mailing list
> mapserver-dev at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/mapserver-dev