[Live-demo] Converting docs to UTF encoding

[Hamish]

Zhengfan wrote:
> One thing that does worry me is line encoding. Most En docs I
> found use MS Win style EOL (0x0D0A). HTML conversion seems to
> work fine with that, but Unix style (0x0A) could be more
> secure. Should we convert EOL ? Besides, is absence of last
> line EOL a problem ?

Jorge wrote:
> If HTML docs are being rendered well it's not a huge problem,
> dos2unix should solve tho problem. You can concatenate the find
> command I sent with dos2unix to exectue it in all files with
> windows EOL.
> If you want me to do it and commit the files I can do it also.

Zhengfan wrote:
> Thanks for the dos2unix suggestion.
> I decided not to do anything for now. We are very close to 6.0
> and I can't confirm DOS/Win EOL is actually causing any
> problem. (At least the HTMLs are fine.) It should be a good
> idea to keep things as they are as long as they are not broken
> so we can save some possible trouble fixing new issues in a
> hurry. I'll try to convert the EOLs after 6.0 release.


Hi,

don't worry about it, there is no problem & nothing to fix.

for each file added to svn we (manually) run a script called
module_svn_propset, it's listed in the wiki if you want to run
it yourself. It does a number of things based on file types,
for ReST documents it removes any executable flag, then:

Added: svn:mime-type
   + text/plain
Added: svn:keywords
   + Author Date Id
Added: svn:eol-style
   + native


the "svn:eol-style native" means that the EOL chars are
automatically translated to be correct on the current platform,
both uploading and downloading. The plain text mime type (and
some heuristics I think) tell it not to try and swap the chars
when they appear in a binary file like an image (CVS was pretty
bad for that). The keywords svn prop we don't really use, but
it is there so you can add a comment in the file like:

.. last edited in rev $Id$ by $Author$, $Date$.


if translators use 'svn copy' from the English version of the
text, these file properties will be automatically copied too.
Otherwise there may be a few days after someone 'svn add's the
translated document before the module_svn_propset is run. (I
usually do that after every full 'svn up')
It's better to use 'svn copy anyway, since that preserves the
full change history of the document, which may be important for
future provenience reviews.


cheers,
Hamish