[mapserver-dev] Fwd: RE: [FriBidi] Incorrect Arabic text rendering in maperver

[Stephen Woodbridge]

Ok, I have attached a patch applying Ian changes to trunk. But I have 
not tested this as I don't have a working 6.0 setup yet.

Ian, if you can pull 6.0 and apply this patch, test and report that 
would probably be very helpful.

-Steve W

On 3/21/2011 9:26 PM, Stephen Woodbridge wrote:
> Hmmm, I just realized this never make it to the dev list.
>
> -Steve W
>
> -------- Original Message --------
> Subject: RE: [FriBidi] Incorrect Arabic text rendering in maperver
> Date: Sun, 13 Mar 2011 03:40:52 -0400
> From: Ian Walberg <ian.walberg at airborne.aero>
> To: Stephen Woodbridge <woodbri at swoodbridge.com>
> CC: Behdad Esfahbod <behdad.esfahbod at gmail.com>,
> <mapserver-devs at lists.osgeo.org>
>
> Stephen,
>
> Attached is the modified mapserver mapstring.c file which includes
> changes based on your code snippet.
>
> We have tested this today in our configuration and this solves the issue
> we had.
>
> I have not previous submitted a change so I am not sure what the process
> is.
>
> Regards
>
> Ian
>
> -----Original Message-----
> From: Stephen Woodbridge [mailto:woodbri at swoodbridge.com]
> Sent: Saturday, March 12, 2011 7:35 PM
> To: Ian Walberg
> Subject: Re: [FriBidi] Incorrect Arabic text rendering in maperver
>
> Ian,
>
> I'm have not followed all of this thread, but from what I have gathered,
> the mapserver code needs to walk over the string and remove these
> characters which seems reasonable.
>
> So if you are up to making a patch to mapserver and attaching that to a
> the bug you opened, we can see if we can get it reviewed and added to
> 6.0 which just went to beta.
>
> FRIBIDI_CHAR_FILL is evidently a three byte sequence if I recall from an
> earlier post on this thread.
>
>  > Also does FRIBIDI_CHAR_FILL get converted to a single utf8 character?
>  > What is its value?
> \xef\xbb\xbf.
>
> This is untested, but I think something like this should work to remove
> the FRIBIDI_CHAR_FILL character sequences. You will need to adapt it to
> the mapserver code.
>
> /* scan str and compress out FRIBIDI_CHAR_FILL UTF8 characters */
> if (utf8) {
> len = strlen(str);
> for (i=0, j=0; i<len; i++, j++) {
> if (str[i] == '\xef' && str[i+1] == '\xbb' && str[i+2] == '\xbf')
> i += 3;
> if (i != j) str[j] = str[i];
> }
> str[j] = '\0';
> }
>
> -Steve W
>
> On 3/12/2011 7:04 PM, Behdad Esfahbod wrote:
>> On 03/09/11 14:08, Ian Walberg wrote:
>>> Behdad,
>>
>> Hello,
>>
>>> We are back looking at this again and have tried a few combinations
>>> and none seem to work.
>>>
>>> Here is the test change we made to fribidi-arabic.c
>>>
>>> for (i = 0; i< len - 1; i++) {
>>> register FriBidiChar c;
>>> if (FRIBIDI_LEVEL_IS_RTL(embedding_levels[i])&&
>>> embedding_levels[i] == embedding_levels[i+1]&&
>>> (c = PAIR_MATCH(table, size, str[i], str[i+1])))
>>> {
>>> /* str[i] = FRIBIDI_CHAR_FILL; */
>>> // Do we need to remove this next line as well?
>>> FRIBIDI_SET_BITS(ar_props[i], FRIBIDI_MASK_LIGATURED);
>>> /* str[i+1] = c; */
>>> str[i] = c;
>>> }
>>> }
>>
>> I suggest instead of changing the FriBidi code you simply walk over
>> the output
>> string and remove the filler character.
>>
>> behdad
>>
>>
>>> The assumption is that we would make the 'real' change to mapserver
>>> and not here.
>>
>>> Thanks
>>>
>>> Ian
>
>
>
> _______________________________________________
> mapserver-dev mailing list
> mapserver-dev at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/mapserver-dev

-------------- next part --------------
--- mapstring.c (revision 11298)
+++ mapstring.c (working copy)
@@ -1593,6 +1593,7 @@
     FriBidiLevel *levels;
     FriBidiStrIndex new_len;
     fribidi_boolean log2vis;
+    int i, j;

     visual = (FriBidiChar *) msSmallMalloc (sizeof (FriBidiChar) * (len + 1));
     ltov = NULL;
@@ -1626,6 +1627,22 @@
      new_len =
        fribidi_unicode_to_charset (from_char_set_num,
            visual, len, outstring);
+
+    /* scan str and compress out FRIBIDI_CHAR_FILL UTF8 characters */
+
+    for (i=0, j=0; i<new_len; i++, j++)
+    {
+      if (outstring[i] == '\xef' && outstring[i+1] == '\xbb' && outstring[i+2] == '\xbf')
+      {
+         i += 3;
+      }
+      if (i != j)
+      {
+        outstring[j] = outstring[i];
+      }
+    }
+    outstring[j] = '\0';
+
 #endif
      return msStrdup(outstring);
   }