[SVN] Re: Python unicode error

[Alexander Ross]

On Wed, Apr 30, 2008 at 9:40 AM, Jeroen van der Ham <jeroen at je-ju.net> wrote:
> Allan Odgaard wrote:
>
> > For example a script like this will error:
> >
> >    # coding: utf-8
> >    print("æble")
> >
> > Is r8839 supposed to allow the above?
> >
>
>  I've spent some time working with Allan trying to tackle this bug, but
> we've been unable to come to a solution.

Honestly, I don't know if there IS a good solution.  Python 3 will be
unicode throughout, so perhaps this problem will be alleviated then.
I don't really have time to work on this right now though...

>  The problem seems to be that the print function detects the output encoding
> and uses that to encode the string to output.
>
>  This seems to point to lines 30 & 31 in tmhooks.py:
>
> > sys.stdout = codecs.getwriter('utf-8')(sys.stdout)
> > sys.stderr = codecs.getwriter('utf-8')(sys.stderr)
> >
>
>  This codec will convert to utf-8, and assumes input to be ascii.
>  While commenting those lines out will make the above example work, it will
> break when printing a unicode string:
>
>         print u"æble"
>
>         UnicodeEncodeError: ascii, ascii, æble, 0, 1, ordinal not in
> range(128)
>
>  I believe that this is because somewhere along the line the outputstream is
> redirected and it is detected as ascii. Perhaps someone with more knowledge
> of Ruby can have a look at this and figure out what might be going wrong.