[Qgis-user] QGIS 2.0.1: Missing source encoding codecs

[Akihiro Takemura]

Hi,

I need to load Shift-JIS encoded .dbf file but I cannot seem to find an option to set proper source encoding.
Normally, I would expect to find Shift_JIS or CP932 in Source encoding at QGIS 2.0.1 > Layer > Add Vector Layer, 
but I cannot find SJIS, Shift_JIS, CP932, EUC-JP, JIS7 or anything else that remotely resembles Japanese encoding.

I did a bit of source code digging as well just to have a look, and found that Shift-JIS is included in src/core/qgsvectordataprovider.cpp (#423) as a compiler option,
and if that's not compiled, QTextCodec::availableCodecs() also includes Shift-JIS http://qt-project.org/doc/qt-4.8/qtextcodec.html#details *1
I can see codecs included in QTextCodec like Iscii-Bng, Dev, Gjr… in my source encoding list, but no Shift-JIS.
So my guesses are either availableCodecs() returning an erratic list, 
or something is happening to it after it has been fed through the dataprovider, but I don't know how to trace that with my limited skills.

I have used Homebrew and tap from dakcarto https://github.com/dakcarto/homebrew-osgeo4mac (many thanks to dakcaro for sharing this)
MacOSX 10.8.5
QGIS: 2.0.1
GDAL: 1.10.1 (brewed)

As alternatives, I can currently a) convert every shapefiles into UTF-8 and load them b) use Shapefile Encoding Fixer to correct encoding every time.
Also note that when "Ignore shapefile declaration" is checked, Shapefile Encoding Fixer plugin will not work (LDID/CPG) and data inside attribute table is incorrectly displayed, but when it is not checked, correct attributes are displayed by setting LDID=SJIS.

Does anyone have any ideas as to how to fix this issue, or have had similar experiences while using non-UTF8 encoded shapefiles?

*1 just to make sure, and I've got the same results from the Python console inside QGIS.
Python 2.7.5 (v2.7.5:ab05e7dd2788, May 13 2013, 13:18:45) 
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from PyQt4 import QtCore
>>> QtCore.QTextCodec.availableCodecs()
[PyQt4.QtCore.QByteArray('System'), PyQt4.QtCore.QByteArray('UTF-8'),
/*snip*/
PyQt4.QtCore.QByteArray('EUC-JP'), PyQt4.QtCore.QByteArray('ISO-2022-JP'), 
PyQt4.QtCore.QByteArray('Shift_JIS'), PyQt4.QtCore.QByteArray('JIS7'), PyQt4.QtCore.QByteArray('SJIS')
/*snip*/

Thanks,
Akihiro
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-user/attachments/20140105/4c569f6e/attachment.html>