[pdal] Non-English characters

[pidgeon13 at googlemail.com]

Hi,

The user browses for a file in a dialog file browser, and I convert this string to UTF-8. I then pass this to pdal using the API as shown in the attached text file. I am using /utf-8.

I have also run:

“C:\Users\stephenp\PDAL\bin>pdal.exe info C:\Users\stephenp\PDALe\рфддвбс.ply” which gave the following:

PDAL: File not found: C:\Users\stephenp\PDALe\???????.ply

Regards,

Stephen

Sent from Mail for Windows 10

From: Andrew Bell
Sent: 17 January 2018 02:24
To: Stephen Pidgeon
Cc: pdal at lists.osgeo.org
Subject: Re: [pdal] Non-English characters

This works fine on OSX:

master $ bin/pdal info ~/Downloads/рфддвбс.ply

How, exactly, are you invoking pdal or using the API such that it fails?

On Tue, Jan 16, 2018 at 12:09 PM, <pidgeon13 at googlemail.com> wrote:
Hi,
 
Attached is such a file. I am developing on Windows, specifically in VS2015.
 
Stephen
 
Sent from Mail for Windows 10
 
From: Andrew Bell
Sent: 16 January 2018 13:05

To: Stephen Pidgeon
Cc: pdal at lists.osgeo.org
Subject: Re: [pdal] Non-English characters
 
Also, can you provide the platform you're using on which you're having problems?
 
On Tue, Jan 16, 2018 at 7:53 AM, Andrew Bell <andrew.bell.ia at gmail.com> wrote:
In your example, the filenames don't have non-ascii characters, just the directory names. Do you have an example where the actual filenames are non-ascii that fails?
 
On Tue, Jan 16, 2018 at 7:48 AM, <pidgeon13 at googlemail.com> wrote:
Hi Andrew,
 
It seems to me that passing UTF-8 std::strings to the readers doesn’t seem to work. I created a folder with foreign characters and placed a few different points files inside, but am not getting a useful PointView when executing the table (there’s either an exception thrown or there are no points). These same files are read fine when they are in a purely English folder path.
 
I've attached a code snippet and the information from the strings that are being passed to readerOptions.add(), if you want some more information I can try to provide it. 
 
Regards,
 
Stephen
 
Sent from Mail for Windows 10
 
From: Andrew Bell
Sent: 15 January 2018 17:24
To: Stephen Pidgeon
Cc: pdal at lists.osgeo.org
Subject: Re: [pdal] Non-English characters
 
On Mon, Jan 15, 2018 at 12:13 PM, <pidgeon13 at googlemail.com> wrote:
Hi,
 
I am looking to allow non-English characters in pathnames when passing them to PDAL readers, however inferReaderDriver and add(“filename”, .) both accept std::strings as arguments. Does PDAL support using file paths that include Unicode characters? Will I need to implement something with UTF-8?
 
I believe that UTF-8 should work fine provided the drivers that open the files can deal with it.  It's possible that we're doing some parsing of paths that may not take this into account, but I'm not sure.  If you have examples of this not working, let me know and I'll take a look.
 
-- 
Andrew Bell
andrew.bell.ia at gmail.com
 



 
-- 
Andrew Bell
andrew.bell.ia at gmail.com



 
-- 
Andrew Bell
andrew.bell.ia at gmail.com
 




-- 
Andrew Bell
andrew.bell.ia at gmail.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/pdal/attachments/20180117/500bf095/attachment-0001.html>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: Pdal code snippet.txt
URL: <http://lists.osgeo.org/pipermail/pdal/attachments/20180117/500bf095/attachment-0001.txt>