[SVN] Re: Python unicode error

[Jeroen van der Ham]

Allan Odgaard wrote:
> For example a script like this will error:
> 
>     # coding: utf-8
>     print("æble")
> 
> Is r8839 supposed to allow the above?

I've spent some time working with Allan trying to tackle this bug, but 
we've been unable to come to a solution.

The problem seems to be that the print function detects the output 
encoding and uses that to encode the string to output.

This seems to point to lines 30 & 31 in tmhooks.py:
> sys.stdout = codecs.getwriter('utf-8')(sys.stdout)
> sys.stderr = codecs.getwriter('utf-8')(sys.stderr)

This codec will convert to utf-8, and assumes input to be ascii.
While commenting those lines out will make the above example work, it 
will break when printing a unicode string:

	print u"æble"

	UnicodeEncodeError: ascii, ascii, æble, 0, 1, ordinal not in range(128)

I believe that this is because somewhere along the line the outputstream 
is redirected and it is detected as ascii. Perhaps someone with more 
knowledge of Ruby can have a look at this and figure out what might be 
going wrong.

Jeroen.