WXGUI vector editing fails with python unicode error

[marisn]

Steps to reproduce:

• Create a new GRASS vector data set;
• In "Attribute table" dialog choose "Manage table" and add a single text attribute;
• Rightclick layer and choose "Start editing";
• Add a feature and fill text field with: "āšņļ";
• Click "Submit" and observe infamous unicode error in WXGUI console.

Traceback (most recent call last):
  File "/home/maris/soft/grass_trunk/dist.x86_64-unknown-
linux-gnu/etc/gui/wxpython/dbmgr/dialogs.py", line 318, in
OnSubmit

for sql in self.GetSQLString(updateValues = True):
  File "/home/maris/soft/grass_trunk/dist.x86_64-unknown-
linux-gnu/etc/gui/wxpython/dbmgr/dialogs.py", line 241, in
GetSQLString

updatedValues.append("'" + str(newvalue) + "'")
UnicodeEncodeError
:
'ascii' codec can't encode characters in position 3-4:
ordinal not in range(128)

"Data browser" encoding is set to "utf-8"

GRASS 7.0.svn (Igaunija):~/soft/grass_trunk > locale
LANG=lv_LV
LC_CTYPE="lv_LV.utf8"
LC_NUMERIC="lv_LV.utf8"
LC_TIME="lv_LV.utf8"
LC_COLLATE="lv_LV.utf8"
LC_MONETARY="lv_LV.utf8"
LC_MESSAGES="lv_LV.utf8"
LC_PAPER="lv_LV.utf8"
LC_NAME="lv_LV.utf8"
LC_ADDRESS="lv_LV.utf8"
LC_TELEPHONE="lv_LV.utf8"
LC_MEASUREMENT="lv_LV.utf8"
LC_IDENTIFICATION="lv_LV.utf8"
LC_ALL=lv_LV.utf8

Notice to all developers - if You do anything user visible or provided text related, keep "āšņļ" and "йцукен" at Your hand all the time, please.

[mlennert]

I can confirm this bug.

Setting utf-8 in the GUI preferences Attribut tab, I can edit the attribute table and add 'āšņļ', but even with that setting, I cannot edit the attributes in the digitizer. I get the same error message as Maris.

I think it might be helpful to try to develop guidelines for developers on how to handle non-ascii. Especially in the wxGUI this issue comes up very often.

Moritz

[marisn]

It is still an issue with current trunk. As it prevents users from creating new vector data sets, I'm setting this as a blocker. Unless GRASS 7.0 ships without wxGUI, UnicodeEncodeError needs to be eliminated in it.

Some reading for those, who have problems with strings in Python: ​http://blog.notdot.net/2010/07/Getting-unicode-right-in-Python

[annakrat]

Replying to marisn:

It is still an issue with current trunk. As it prevents users from creating new vector data sets, I'm setting this as a blocker. Unless GRASS 7.0 ships without wxGUI, UnicodeEncodeError needs to be eliminated in it.

Some reading for those, who have problems with strings in Python: ​http://blog.notdot.net/2010/07/Getting-unicode-right-in-Python

I attached a diff which solves some of the issues. I didn't commit it because it's not complete (I don't have much time for it). So feel free to improve it.

[annakrat]

Replying to annakrat:

I attached a diff which solves some of the issues. I didn't commit it because it's not complete (I don't have much time for it). So feel free to improve it.

I extended the diff and it seems to work somehow so I committed it in r61897. Needs testing.

[neteler]

Replying to annakrat:

I extended the diff and it seems to work somehow so I committed it in r61897. Needs testing.

Can this be backported?

[mlennert]

Replying to neteler:

Replying to annakrat:

I extended the diff and it seems to work somehow so I committed it in r61897. Needs testing.

Can this be backported?

It works for me in trunk at least. So I would say +1.

Moritz

[annakrat]

Replying to mlennert:

Replying to neteler:

Replying to annakrat:

I extended the diff and it seems to work somehow so I committed it in r61897. Needs testing.

Can this be backported?

It works for me in trunk at least. So I would say +1.

Thanks for testing, backported in r62378. I will close it for now, however encoding issues generally are not solved properly yet.