change language to Turkish, machine keeps rebooting

[chihyu]

build: 653, running continuously for less than 2 days

serial no.: CSN74400083

Tried changing the language to Turkish by following http://wiki.laptop.org/go/Sugar_Control_Panel#Languages but the machine kept rebooting afterwards.

steps of reproduction

* go to Terminal activity

* sugar-control-panel -s language Turkish

* Ctrl-Alt-erase to restart sugar

[chihyu]

Restarted the machine by holding down the power button, but it still kept rebooting. This was stopped by

* Ctrl-Alt-F1

* login as root

* type init 3

Tried setting the language back to English by typing

sugar-control-panel -s language English/USA

and the error message showed up:

-bash-3.2# sugar-control-panel -s language English/USA
Traceback (most recent call last):
File "/usr/bin/sugar-control-panel", line 26, in <module>
sys.path.insert(0, env.get_shell_path())
File "/usr/lib/python2.5/site-packages/sugar/env.py", line 90, in get_shell_path
return _get_sugar_path('shell', path)
File "/usr/lib/python2.5/site-packages/sugar/env.py", line 36, in _get_sugar_path
raise RuntimeError("The SUGAR_PATH environment variable is not set.")
RuntimeError: The SUGAR_PATH environment variable is not set.

Replying to chihyu:

build: 653, running continuously for less than 2 days serial no.: CSN74400083 Tried changing the language to Turkish by following http://wiki.laptop.org/go/Sugar_Control_Panel#Languages but the machine kept rebooting afterwards. == steps of reproduction == * go to Terminal activity * sugar-control-panel -s language Turkish * Ctrl-Alt-erase to restart sugar

[marco]

This is the relevant traceback:

Traceback (most recent call last):

File "/usr/bin/sugar-shell", line 35, in <module>

from view.Shell import Shell

File "/usr/share/sugar/shell/view/Shell.py", line 39, in <module>

from view.home.HomeWindow import HomeWindow

File "/usr/share/sugar/shell/view/home/HomeWindow.py", line 23, in <module>

from view.home.MeshBox import MeshBox

File "/usr/share/sugar/shell/view/home/MeshBox.py", line 41, in <module>

from view.home.spreadlayout import SpreadLayout

File "/usr/share/sugar/shell/view/home/spreadlayout.py", line 18, in <module>

from numpy import array

File "/usr/lib/python2.5/site-packages/numpy/init.py", line 39, in <module>

import core

File "/usr/lib/python2.5/site-packages/numpy/core/init.py", line 8, in <module>

import numerictypes as nt

File "/usr/lib/python2.5/site-packages/numpy/core/numerictypes.py", line 241, in <module>

void = allTypesvoid?

KeyError: 'void'

[chihyu]

steps to revive the laptop

* Ctrl-Alt-F2

* login as root

* su olpc

* export SUGAR_PATH=/usr/share/sugar

* sugar-control-panel -s language English/USA

* restart the machine by pressing the power button, while holding down the right mouse button (o)

Replying to chihyu:

Restarted the machine by holding down the power button, but it still kept rebooting. This was stopped by * Ctrl-Alt-F1 * login as root * type init 3 Tried setting the language back to English by typing sugar-control-panel -s language English/USA and the error message showed up: -bash-3.2# sugar-control-panel -s language English/USA
Traceback (most recent call last):
File "/usr/bin/sugar-control-panel", line 26, in <module>
sys.path.insert(0, env.get_shell_path())
File "/usr/lib/python2.5/site-packages/sugar/env.py", line 90, in get_shell_path
return _get_sugar_path('shell', path)
File "/usr/lib/python2.5/site-packages/sugar/env.py", line 36, in _get_sugar_path
raise RuntimeError("The SUGAR_PATH environment variable is not set.")
RuntimeError: The SUGAR_PATH environment variable is not set. Replying to chihyu:

build: 653, running continuously for less than 2 days serial no.: CSN74400083 Tried changing the language to Turkish by following http://wiki.laptop.org/go/Sugar_Control_Panel#Languages but the machine kept rebooting afterwards. == steps of reproduction == * go to Terminal activity * sugar-control-panel -s language Turkish * Ctrl-Alt-erase to restart sugar

[jg]

I updated http://wiki.laptop.org/index.php?title=Sugar_Control_Panel&action=submit to reflect the previous information.

Marco, is there some easy way to get Sugar to not lose its cookies entirely?

[marco]

Unfortunately I don't have a theory on what could be causing this issue. Answering that question would require to spend a bit of time to track down the issue. I can do it if you think it's worth for Update.1

[jg]

I suspect we should look into it a bit: translations/languages are important, and the failure is catastrophic.

Please bound your effort, however, if you haven't found it in a couple hours, we'll just release note it and hope the support burden isn't too bad....

[marco]

Easy to repro on both Fedora and Ubuntu with:

LANG=tr_TR.UTF-8 python -c "import gtk, numpy"

I suspect this is related to the fact that gtk changes the default encoding to utf-8.

[marco]

Work around attached. erikos is opening a ticket upstream.

[rwh]

ugh. We don't have a choice; hopefully gets fixed upstream quickly!

[marco]

Upstream ticket: http://scipy.org/scipy/numpy/ticket/643

[marco]

Work around checked in, will be in tomorrow release.

[rkern]

Here is a minimal replication of the precise problem:

$ LANG=tr_TR.UTF-8 python -c "import gtk;print 'VOID'.lower()"
voId
$ LANG=tr_TR.UTF-8 python -c "print 'VOID'.lower()"
void
$ LANG=tr_TR.UTF-8 python -c "import sys;reload(sys);sys.setdefaultencoding('utf-8');print 'VOID'.lower()"
void

I only get this on my XO. I cannot replicate this on Ubuntu Feisty (python-gtk2 2.10.4-0ubuntu3). I'll upgrade to Gutsy and see if the culprit is a newer version of GTK/PyGTK. Since marco does see it on Ubuntu, I suspect this is the case.

[AlbertCahalan]

In all 3 cases, you're getting the wrong result for Turkish. You should get U+0131, the lowercase undotted I.

In the first case, you're also getting the wrong result for the C and POSIX locales. The first case is really broken; it is not correct for any locale. The source of this mess, probably glibc, is buggy. You should be getting C/POSIX behavior until you enable the locale, after which you should get Turkish behavior. Bastard hybrid locales are not useful for anything.

http://www.i18nguy.com/unicode/turkish-i18n.html

[rkern]

What should u'VOID'.lower() give in a Turkish locale?

[AlbertCahalan]

Assuming trac supports the characters: voıd

The bytes are: 0x76,0x6f,0xc4,0xb1,0x64

The above assumes that the program has enabled the locale. One should be able to switch the locale on and off as needed.

[rkern]

Hmm, the Python documentation says "For 8-bit strings, this method is locale-dependent." This implies (to me) that for unicode strings the method isn't locale-dependent. None of the systems I have available to me behave as you describe, but I may not have Turkish locale support installed.

The problem I have is that numpy is a library, not an application. I can't really control the locale without being very unneighborly to the application using numpy and any other library in the process. The Python documentation explicitly recommends against it. It would be easier to use unicode strings with ASCII characters (if the locale doesn't interfere with that as you claim), or replace all of our calls to ''.lower() with a utility function that uses English rules.

[rkern]

Okay, I figured out how to replicate this without GTK or the XO. GTK isn't the problem; numpy is. In Python, the locale will be 'C' unless if the user's locale is specifically requested. Apparently, gtk does this.

$ LANG=tr_TR.UTF-8 python -c "import locale;locale.setlocale(locale.LC_ALL, '');print repr('VOID'.lower())"
'voId'
$ LANG=tr_TR.UTF-8 python -c "import locale;locale.setlocale(locale.LC_ALL, '');print repr(u'VOID'.lower())"
u'void'

It appears that the result of u'VOID'.lower() is not locale-dependent.

[AlbertCahalan]

test code and Fedora 8 results

[AlbertCahalan]

If the result of u'VOID'.lower() is not locale-dependent, then Python is buggy.

I just tested the locale itself on Fedora 8, directly via C. Somehow I forgot to test Python, but that will match up with C unless it is buggy.

The tr_TR.utf8 locale (and tr_TR.UTF-8 alias) treat the tittle (dot) like any random accent mark. The dotted/undotted distinction is fully preserved when converting between uppercase and lowercase.

The en_US.utf8 locale (and en_US.UTF-8 alias) will mangle Turkish letters on case changes. A round trip through towupper and towlower will turn a dotted capital I into an undotted one, and will turn an undotted lowercase I into a dotted one. (you always end up with ASCII)

The C locale refuses to make case changes to any non-ASCII character. All non-ASCII passes through.

[rkern]

I have been informed by Fredrik Lundh and Martin v. Löwis that u'VOID'.lower() is not locale-dependent. You can take it up with them if you think that behavior is buggy. Personally, I do not believe that every string manipulation must depend on the user's locale; like in this case, not every string comes from the user.

[AlbertCahalan]

Replying to rkern:

I have been informed by Fredrik Lundh and Martin v. Löwis that u'VOID'.lower() is not locale-dependent. You can take it up with them if you think that behavior is buggy. Personally, I do not believe that every string manipulation must depend on the user's locale; like in this case, not every string comes from the user.

Well OK then: the Python language does not support any Turkish locale.

The ramifications of this for OLPC are... interesting. You must abandon either Turkish or Python. Either way, you may as well close this bug.

[marco]

Work around is in, correct fix will go in when we upgrade to latest upstream. rkern, thanks so much for looking into it!

[mtd]

Ticket now closed upstream -- and this means it's fixed in joyride, now, IIUC:

bash-3.2$ cat /boot/olpc_build
joyride 2146
bash-3.2$ LANG=tr_TR.UTF-8 python -c "import gtk, numpy; print 'ok'" 
ok

...and numpy's taking a fair amount of time to import:

Jul 20 02:55:57 xo logger: 7212_SUGAR_SHELL_MAIN_IMPORTING
Jul 20 02:55:57 xo logger: 7212_SUGAR_SHELL_MAIN_numpy_start
Jul 20 02:56:03 xo logger: 7212_SUGAR_SHELL_MAIN_numpy_end
Jul 20 02:56:07 xo logger: 7212_SUGAR_SHELL_MAIN_IMPORTING_DONE

...so shall we remove the workaround?

[mtd]

The import time isn't numpy's fault, so sorry for the noise.