Hi,<br><br>I don't know much about the hindi character sets.<br>I guess you could extent that byte array to string copy function with arbitrary character sizes, like for double bytes something like:<br><br> for (int i = 0; i < bytes.Length; i=i+2)<br>
s.Append(Convert.ToChar(bytes[i] + 256*bytes[i+1]));<br><br>Best regards,<br><br>Tamas<br><br><br><br><div class="gmail_quote">2009/3/4 Murty Maganti <span dir="ltr"><<a href="mailto:MMaganti@oriongis.com">MMaganti@oriongis.com</a>></span><br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div link="blue" vlink="purple" lang="EN-US">
<div>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);">Hi Tamas </span></p>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);"> </span></p>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);">This is still not working for some of the Asian languages. </span></p>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);"> </span></p>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);">I suspect the issue could be in this line of your sample code
below</span></p><div class="im">
<p>s.Append(Convert.ToChar(bytes[i])); </p>
<p> </p>
</div><p><span style="font-size: 11pt; color: rgb(31, 73, 125);">Here, one single byte is used to convert to a character. But my
understanding is that UTF-8 can consume from 1 to 4 bytes to represent one character
code point. It worked fine for Arabic may be because all Arabic characters can
be represented using a single byte. </span></p>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);"> </span></p>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);">When I tried the same code below with Hindi, an Indian language,
some of the characters are shown junk (but not all characters). I guess those
characters which occupy more than a byte turned out to be junk.</span></p>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);"> </span></p>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);">I am also trying the opposite of the sample code below i.e. read
a field value from map server (shapeObj.values), which is in Hindi, and show on
a web page, again it turns out to be junk. I tried to look at the byte values
of the string in VS by using </span></p>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);"> </span></p>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);">Byte[] bites = Encoding.Unicode.GetBytes(shapeObj.values[0])</span></p>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);"> </span></p>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);">I notice that they are actually code point of UTF-8 but
interpreted as UTF-16 and may be the reason I see junk characters on web page.
But I don’t know how to extract those UTF-8 byte values from UTF-16.</span></p>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);"> </span></p>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);">I am just giving sample code here to explain</span></p>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);"> </span></p>
<p style=""><span style="font-size: 10pt; font-family: "Courier New";"> <span style="color: blue;">byte</span>[]
utf16 = <span style="color: rgb(43, 145, 175);">Encoding</span>.Unicode.GetBytes(<span style="color: rgb(163, 21, 21);">"</span></span><span style="font-size: 10pt; font-family: Mangal; color: rgb(163, 21, 21);" lang="HI">कीचनर</span><span style="font-size: 10pt; font-family: "Courier New"; color: rgb(163, 21, 21);">"</span><span style="font-size: 10pt; font-family: "Courier New";">); //The text is in Hindi, an Indian language</span></p>
<p style=""><span style="font-size: 10pt; font-family: "Courier New";"> <span style="color: blue;">byte</span>[]
utf8 = <span style="color: rgb(43, 145, 175);">Encoding</span>.UTF8.GetBytes(<span style="color: rgb(163, 21, 21);">"</span></span><span style="font-size: 10pt; font-family: Mangal; color: rgb(163, 21, 21);" lang="HI">कीचनर</span><span style="font-size: 10pt; font-family: "Courier New"; color: rgb(163, 21, 21);">"</span><span style="font-size: 10pt; font-family: "Courier New";">);</span></p>
<p style=""><span style="font-size: 10pt; font-family: "Courier New";"> </span></p>
<p style=""><span style="font-size: 10pt; font-family: "Courier New";"> <span style="color: rgb(43, 145, 175);">shapeObj</span>
shape = layer.getFeature(result.shapeindex, result.tileindex);</span></p>
<p style=""><span style="font-size: 10pt; font-family: "Courier New";"> <span style="color: blue;">string</span>
value = shape.values[1]; //This contains the same text (in Hindi) as above in
the shape file.</span></p>
<p style=""><span style="font-size: 10pt; font-family: "Courier New";"> </span></p>
<p><span style="font-size: 10pt; font-family: "Courier New";">
<span style="color: blue;">byte</span>[] bytes = <span style="color: rgb(43, 145, 175);">Encoding</span>.Unicode.GetBytes(value);
//There are byte values of characters decoded from UTF-16. .Net internally
stores all strings in UTF-16 </span></p>
<p><span style="font-size: 10pt; font-family: "Courier New";"> </span></p>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);">Now if I examine the values of ‘utf8’ and ‘bytes’ arrays</span></p>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);"> </span></p>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);">utf8 – 224,164,149,224,165,128,224,164,154,224,164,168,224,164,176</span></p>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);">bytes – <b>224</b>,0,<b>164</b>,0,34,32,<b>224</b>,0,<b>165</b>,0,172,32,<b>224</b>,0,<b>164</b>,0,97,1,<b>224</b>,0,<b>164</b>,0,<b>168</b>,0,<b>224</b>,0,<b>164</b>,0,<b>176</b>,0</span></p>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);">utf16 – 21,9,64,9,26,9,40,9,48,9</span></p>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);"> </span></p>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);">The first byte value is same as UTF-8. Second byte value is 0 as
UTF-16 takes atleast 2 bytes for a character. This gives me impression that the
byte values are in UTF-8 and are not converted to UTF-16 to by .Net.</span></p>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);"> </span></p>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);">Appreciate if you see any solution for this and let me know. </span></p>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);"> </span></p>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);">Thanks</span></p>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);">Murty<br>
<br>
</span></p>
<div style="border-style: solid none none; border-color: rgb(181, 196, 223) -moz-use-text-color -moz-use-text-color; border-width: 1pt medium medium; padding: 3pt 0in 0in;">
<p><b><span style="font-size: 10pt;">From:</span></b><span style="font-size: 10pt;"> Tamas Szekeres
[mailto:<a href="mailto:szekerest@gmail.com" target="_blank">szekerest@gmail.com</a>] <br>
<b>Sent:</b> Friday, February 06, 2009 6:59 PM<div><div></div><div class="h5"><br>
<b>To:</b> Murty Maganti<br>
<b>Cc:</b> <a href="mailto:mapserver-users@lists.osgeo.org" target="_blank">mapserver-users@lists.osgeo.org</a><br>
<b>Subject:</b> Re: [mapserver-users] Encoding issues</div></div></span></p>
</div><div><div></div><div class="h5">
<p> </p>
<p style="margin-bottom: 12pt;">You might have to make explicit
conversion maually something like:<br>
<br>
string value
= "<span dir="rtl" lang="AR-SA">لققافعععىىةةونه</span><span dir="ltr"></span><span dir="ltr"></span>"; //I actually get this (in arabic) through user input<br>
byte[] bytes
= Encoding.Convert(Encoding.Unicode, Encoding.GetEncoding(1256),
Encoding.Unicode.GetBytes(value));<br>
StringBuilder s = new StringBuilder();<br>
for (int i =
0; i < bytes.Length; i++)<br>
s.Append(Convert.ToChar(bytes[i]));<br>
shpObj.text
= s.ToString();<br>
<br>
Best regards,<br>
<br>
Tamas<br>
<br>
<br>
</p>
<div>
<p>2009/2/6 Murty Maganti <<a href="mailto:MMaganti@oriongis.com" target="_blank">MMaganti@oriongis.com</a>></p>
<div>
<div>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);">HI </span></p>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);"> </span></p>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);">I am doing a simple thing. I
have a map file and trying to show some static text in Arabic on map. You can
try this with any map file as it is nothing to do with layers from map.</span></p>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);"> </span></p>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);">At run time (like on a button
click), please add this</span></p>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);"> </span></p>
<p><span style="font-size: 10pt; font-family: "Courier New";">
<span style="color: rgb(43, 145, 175);">layerObj</span> lyr = <span style="color: blue;">new</span>
<span style="color: rgb(43, 145, 175);">layerObj</span>(mapObj);</span></p>
<p><span style="font-size: 10pt; font-family: "Courier New";">
<a href="http://lyr.name" target="_blank">lyr.name</a> = <span style="color: rgb(163, 21, 21);">"TextAcetate"</span>;</span></p>
<p><span style="font-size: 10pt; font-family: "Courier New";">
lyr.status = <span style="color: rgb(43, 145, 175);">mapscript</span>.MS_ON;</span></p>
<p><span style="font-size: 10pt; font-family: "Courier New";">
lyr.type = <span style="color: rgb(43, 145, 175);">MS_LAYER_TYPE</span>.MS_LAYER_ANNOTATION;</span></p>
<p><span style="font-size: 10pt; font-family: "Courier New";">
lyr.labelcache = <span style="color: rgb(43, 145, 175);">mapscript</span>.MS_TRUE;</span></p>
<p><span style="font-size: 10pt; font-family: "Courier New";"> </span></p>
<p><span style="font-size: 10pt; font-family: "Courier New";">
<span style="color: blue;">double</span> locationX = 50;</span></p>
<p><span style="font-size: 10pt; font-family: "Courier New";">
<span style="color: blue;">double</span> locationY = 50;</span></p>
<p><span style="font-size: 10pt; font-family: "Courier New";"> </span></p>
<p style="margin-left: 1in; text-indent: 0.5in;"><span style="font-size: 10pt; font-family: "Courier New";">lyr.transform = (<span style="color: blue;">int</span>)<span style="color: rgb(43, 145, 175);">mapscript</span>.MS_FALSE;</span></p>
<p style="margin-left: 1in; text-indent: 0.5in;"><span style="font-size: 10pt; font-family: "Courier New";"> </span></p>
<p style="margin-left: 1in; text-indent: 0.5in;"><span style="font-size: 10pt; font-family: "Courier New"; color: rgb(43, 145, 175);">classObj</span><span style="font-size: 10pt; font-family: "Courier New";"> layerClass = <span style="color: blue;">new</span>
<span style="color: rgb(43, 145, 175);">classObj</span>(lyr);</span></p>
<p style="margin-left: 1in; text-indent: 0.5in;"><span style="font-size: 11pt; color: rgb(31, 73, 125);"> </span></p>
<p style="margin-left: 1in; text-indent: 0.5in;"><span style="font-size: 11pt; color: rgb(31, 73, 125);">//All label properties</span></p>
<p style="margin-left: 1in; text-indent: 0.5in;"><span style="font-size: 10pt; font-family: "Courier New";">layerClass.label.size = 15;</span></p>
<p style="margin-left: 1in; text-indent: 0.5in;"><span style="font-size: 10pt; font-family: "Courier New";">layerClass.label.type =<span style="color: rgb(43, 145, 175);">
MS_FONT_TYPE</span>.MS_TRUETYPE;</span></p>
<p style="margin-left: 1in; text-indent: 0.5in;"><span style="font-size: 10pt; font-family: "Courier New";">…</span></p>
<p style="margin-left: 1in; text-indent: 0.5in;"><span style="font-size: 10pt; font-family: "Courier New";">…</span></p>
<p style="margin-left: 1in; text-indent: 0.5in;"><span style="font-size: 10pt; font-family: "Courier New";">layerClass.label.encoding = "CP1256";</span></p>
<p style="margin-left: 1in; text-indent: 0.5in;"><span style="font-size: 10pt; font-family: "Courier New";"> </span></p>
<p style="margin-left: 1in; text-indent: 0.5in;"><span style="font-size: 11pt; color: rgb(31, 73, 125);"> </span></p>
<p><span style="font-size: 10pt; font-family: "Courier New";">
<span style="color: rgb(43, 145, 175);">shapeObj</span> shpObj = <span style="color: blue;">new</span>
<span style="color: rgb(43, 145, 175);">shapeObj</span>((<span style="color: blue;">int</span>)<span style="color: rgb(43, 145, 175);">MS_SHAPE_TYPE</span>.MS_SHAPE_POINT);</span></p>
<p><span style="font-size: 10pt; font-family: "Courier New";">
<span style="color: rgb(43, 145, 175);">lineObj</span> lnObj = <span style="color: blue;">new</span>
<span style="color: rgb(43, 145, 175);">lineObj</span>();</span></p>
<p><span style="font-size: 10pt; font-family: "Courier New";"> </span></p>
<p><span style="font-size: 10pt; font-family: "Courier New";">
<span style="color: rgb(43, 145, 175);">pointObj</span> pt = <span style="color: blue;">new</span>
<span style="color: rgb(43, 145, 175);">pointObj</span>(locationX, locationY, 0, 0);</span></p>
<p><span style="font-size: 10pt; font-family: "Courier New";">
lnObj.add(pt);</span></p>
<p><span style="font-size: 10pt; font-family: "Courier New";"> </span></p>
<p><span style="font-size: 10pt; font-family: "Courier New";">
shpObj.add(lnObj);</span></p>
<p><span style="font-size: 10pt; font-family: "Courier New";"> </span></p>
<p><span style="font-size: 10pt; font-family: "Courier New";">
shpObj.text = "</span><span dir="rtl" style="font-size: 10pt; font-family: "Courier New";" lang="AR-AE">لققافعععىىةةونه</span><span dir="ltr"></span><span style="font-size: 10pt; font-family: "Courier New";"><span dir="ltr"></span>";
//I actually get this (in arabic) through user input</span></p>
<p><span style="font-size: 10pt; font-family: "Courier New";"> </span></p>
<p><span style="font-size: 10pt; font-family: "Courier New";">
lyr.addFeature(shpObj);</span></p>
<p><span style="font-size: 10pt; font-family: "Courier New";"> </span></p>
<p><span style="font-size: 10pt; font-family: "Courier New";">mapObj.draw();
//Onto a picture box or save as file</span></p>
<p><span style="font-size: 10pt; font-family: "Courier New";"> </span></p>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);">(In the map file, my output
format is set to GD/PNG)</span></p>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);"> </span></p>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);">Please let me know if you need
more information.</span></p>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);"> </span></p>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);">Thanks</span></p>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);">Murty</span></p>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);"> </span></p>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);"> </span></p>
<div style="border-style: solid none none; border-color: -moz-use-text-color; border-width: 1pt medium medium; padding: 3pt 0in 0in;">
<p><b><span style="font-size: 10pt;">From:</span></b><span style="font-size: 10pt;"> <a href="mailto:mapserver-users-bounces@lists.osgeo.org" target="_blank">mapserver-users-bounces@lists.osgeo.org</a> [mailto:<a href="mailto:mapserver-users-bounces@lists.osgeo.org" target="_blank">mapserver-users-bounces@lists.osgeo.org</a>]
<b>On Behalf Of </b>Tamas Szekeres<br>
<b>Sent:</b> Friday, February 06, 2009 4:12 PM</span></p>
<div>
<div>
<p><span style="font-size: 10pt;"><br>
<b>To:</b> Murty Maganti<br>
<b>Cc:</b> <a href="mailto:mapserver-users@lists.osgeo.org" target="_blank">mapserver-users@lists.osgeo.org</a><br>
<b>Subject:</b> Re: [mapserver-users] Encoding issues</span></p>
</div>
</div>
</div>
<div>
<div>
<p> </p>
<p style="margin-bottom: 12pt;">Please send me your example so that I could
examine what's going on.<br>
<br>
Best regards,<br>
<br>
Tamas<br>
<br>
</p>
<div>
<p>2009/2/6 Murty Maganti <<a href="mailto:MMaganti@oriongis.com" target="_blank">MMaganti@oriongis.com</a>></p>
<div>
<div>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);">Hi</span></p>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);"> </span></p>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);">I tried with the suggested
encoding but still no success.</span></p>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);">From the output below, I guess
ICONV support is included.</span></p>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);"> </span></p>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);">E:\Utils\MapServer\Map Server
5.2 RC\ms4w\Apache\cgi-bin>mapserv -v</span></p>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);">MapServer version 5.2.0
OUTPUT=GIF OUTPUT=PNG OUTPUT=JPEG OUTPUT=WBMP OUTPUT=PDF</span></p>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);"> OUTPUT=SWF OUTPUT=SVG
SUPPORTS=PROJ SUPPORTS=AGG SUPPORTS=FREETYPE <b>SUPPORTS=ICO</b></span></p>
<p><b><span style="font-size: 11pt; color: rgb(31, 73, 125);">NV</span></b><span style="font-size: 11pt; color: rgb(31, 73, 125);"> SUPPORTS=FRIBIDI SUPPORTS=WMS_SERVER
SUPPORTS=WMS_CLIENT SUPPORTS=WFS_SERVER</span></p>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);">SUPPORTS=WFS_CLIENT
SUPPORTS=WCS_SERVER SUPPORTS=SOS_SERVER SUPPORTS=FASTCGI SUP</span></p>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);">PORTS=THREADS SUPPORTS=GEOS
SUPPORTS=RGBA_PNG INPUT=JPEG INPUT=POSTGIS INPUT=OGR</span></p>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);"> INPUT=GDAL
INPUT=SHAPEFILE</span></p>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);"> </span></p>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);">Where can get some details on how
to build the C# mapscript (Managed assembly only) from Visual Studio, keeping
all unmanaged dlls from binaries from ms4w. I just want to give a try using
MarshalAsAttribute.</span></p>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);"> </span></p>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);">Thanks</span></p>
<p><span style="font-size: 11pt; color: rgb(31, 73, 125);">Murty</span></p>
<div style="border-style: solid none none; border-color: -moz-use-text-color; border-width: 1pt medium medium; padding: 3pt 0in 0in;">
<p><b><span style="font-size: 10pt;">From:</span></b><span style="font-size: 10pt;"> Tamas Szekeres [mailto:<a href="mailto:szekerest@gmail.com" target="_blank">szekerest@gmail.com</a>] <br>
<b>Sent:</b> Friday, February 06, 2009 3:02 PM<br>
<b>To:</b> Murty Maganti<br>
<b>Cc:</b> <a href="mailto:mapserver-users@lists.osgeo.org" target="_blank">mapserver-users@lists.osgeo.org</a><br>
<b>Subject:</b> Re: [mapserver-users] Encoding issues</span></p>
</div>
<div>
<div>
<p> </p>
<p style="margin-bottom: 12pt;">Hi,<br>
<br>
You might want to try with encoding="ISO-8859-6" assuming you have
libiconv compiled in.<br>
The c# mapscript doesn't specify explicit conversion during the marshaling. In
this case I assume an unicode to Charset.Ansi conversion will automatically
takes place by default.<br>
<br>
Best regards,<br>
<br>
Tamas<br>
<br>
<br>
</p>
<div>
<p>2009/2/6 Murty Maganti <<a href="mailto:MMaganti@oriongis.com" target="_blank">MMaganti@oriongis.com</a>></p>
<div>
<div>
<p>Hello </p>
<p> </p>
<p>I am having some issues using Arabic text as labels. I am using C# map
script. I am setting the following at runtime</p>
<p> </p>
<p>labelObj label = classObj.label;</p>
<p>label.encoding = "CP1256";</p>
<p>label.text = "some text in Arabic"; (At rune time in VS, I can see
the text is actually in Arabic)</p>
<p> </p>
<p>But labels are displayed as '?????'.</p>
<p> </p>
<p> Is there any conversion I need to do before setting the text value.
How are the string represented in the underlying mapscript dll (ASCII or
Unicode?). As I was reading in the MSDN, the default marshalling uses LPStr
which is a single byte of ASCII. Does it mean that first I need to convert from
Unicode to ASCII in C# before setting the value.</p>
<p> </p>
<p>Appreciate any help.</p>
<p> </p>
<p>Thanks</p>
<p>Murty</p>
<p> </p>
</div>
</div>
<p style="margin-bottom: 12pt;"><br>
_______________________________________________<br>
mapserver-users mailing list<br>
<a href="mailto:mapserver-users@lists.osgeo.org" target="_blank">mapserver-users@lists.osgeo.org</a><br>
<a href="http://lists.osgeo.org/mailman/listinfo/mapserver-users" target="_blank">http://lists.osgeo.org/mailman/listinfo/mapserver-users</a></p>
</div>
<p> </p>
</div>
</div>
</div>
</div>
</div>
<p> </p>
</div>
</div>
</div>
</div>
</div>
<p> </p>
</div></div></div>
</div>
</blockquote></div><br>