[gdal-dev] UTF-8 String Support in GDALOpen() and OGRSFDriverRegistrar::Open()

[Even Rouault]

Louis, Chaintanya,

I just wanted to mention that the topic of encoding for filenames dealt by GDAL
or OGR is a known issue that has not been addressed yet. You can read
http://trac.osgeo.org/gdal/wiki/rfc5_unicode which was a proposal but has not
been implemented. Some infrastructure for re-encoding has been introduced during
the implementation of http://trac.osgeo.org/gdal/wiki/rfc23_ogr_unicode (but
RFC23 only addresses the issue of encoding in OGR field values, not for
filenames)

My understanding is that :
* on Windows the current API used by GDAL/OGR does not expect UTF8 or Unicode
but ANSI.
* on Linux systems, UTF-8 is now assumed

Best regards,

Even

Selon Lodewijk Pool <louis.pool at gmail.com>:

> Hi Chaitanya,
>
> I appreciate you taking the time to check. The TAB extension is MapInfo's
> vector file format. The odd thing is that I did exactly the same test as you
> did, I renamed a GeoTiff file to the offending filename and tried the normal
> Raster Driver and got the same problem. Still, as far as you aware these
> functions should support UTF-8 encoded strings?  There could possibly be a
> peculiarity in the way I pack UTF-8 strings, though I am reasonably certain
> that they are encoded correctly.
>
> Could you perhaps send me the code snippet you used to test the
> functionality (the part where you pass the string to GDALOpen). Do you think
> there is a chance that my compiled version may differ from your own, i.e. is
> it possible that I compiled a version of GDAL without UTF support?
>
> Best Regards,
> Louis.
>
> On Mon, Aug 31, 2009 at 6:35 PM, Chaitanya kumar CH
> <chaitanya.ch at gmail.com>wrote:
>
> > Louis,
> >
> > I couldn't reproduce the problem on my WinXP-32 system with vc8 with locale
> > set to uk english. However, I used the filename on a GeoTiff file. I
> > couldn't identify the .TAB extension. I am not sure that is a problem.
> >
> > Some of the drivers may not handle non-ascii data but file names should not
> > be a problem.
> >
> > If you don't find any problem at your application side, submit a bug report
> > at http://trac.osgeo.org/gdal/
> >
> >
> > On Mon, Aug 31, 2009 at 8:02 PM, Lodewijk Pool <louis.pool at gmail.com>wrote:
> >
> >> Hi Chaitanya,
> >>
> >> Yes, this is using the C/C++ API, the functions I am using are declared in
> >> *gdal.h* and *ogrsf_frmts.h* respectively. I am using WinXP 32bit (UK
> >> English locale) and a version of GDAL 1.6.2 that I compiled for Win32
> using
> >> the supplied nmake script files for VC8. The specific filename that is
> >> causing me problems is this one; *"découpage_geographique.TAB"*. If I
> >> remove the 'é' character in that string and replace it with a normal 'e'
> the
> >> file opens without any problems.
> >>
> >> Any help would be appreciated.
> >>
> >> Best Regards,
> >> Louis.
> >>
> >>
> >>
> >>
> >>
> >> On Mon, Aug 31, 2009 at 4:10 PM, Chaitanya kumar CH <chaitanya.ch@
> >> gmail.com> wrote:
> >>
> >>> Louis,
> >>>
> >>> GDAL/OGR usually supports utf-8 encoding. I just don't know where it
> >>> doesn't support.
> >>> Can you provide the details of the OS you are working on? Also, some
> >>> sample file names that caused you problems will come handy.
> >>> I presume you are working in C/C++.
> >>>
> >>> On Mon, Aug 31, 2009 at 6:37 PM, Lodewijk Pool
> <louis.pool at gmail.com>wrote:
> >>>
> >>>> Hi All,
> >>>>
> >>>> I'm having problems opening Raster and Vector Datasources that have
> >>>> filenames and paths with special characters. I'm using GDALOpen for
> Raster
> >>>> sources and OGRSFDriverRegistrar::Open() for Vector sources, the strings
> I
> >>>> pass for the filenames are UTF-8 encoded. Does anyone know whether these
> >>>> functions support UTF-8 encoding, and if not, whether there are any
> other
> >>>> API entry points that do support UTF-8 and/or UTF-16?
> >>>>
> >>>> Thank you in advance,
> >>>> Louis.
> >>>>
> >>>>  _______________________________________________
> >>>> gdal-dev mailing list
> >>>> gdal-dev at lists.osgeo.org
> >>>> http://lists.osgeo.org/mailman/listinfo/gdal-dev
> >>>>
> >>>
> >>>
> >>>
> >>> Best regards,
> >>> --
> >>> Chaitanya kumar CH.
> >>>
> >>
> >>
> >
> >
> > Best regards,
> > --
> > Chaitanya kumar CH.
> >
>