[Scilab-Dev] Test report of Chinese (or multibytes languages in general) locale mesages with Scilab 5-beta-1

[Yung-Jang Lee]

 Sylvestre:

Thank for your test under Linux, I just plan to do the same thing.

Your test means behavior of Java console under Linux and Windows is the
same. As I reported before, if OS's locale is not UTF8 compatible,  both OS
send characters to Java console in it's native encoding according to locale
setting (LANG).

Characters in  zh_TW are encoding in BIG5 (http://en.wikipedia.org/wiki/Big5)
while characters in  zh_CN  are encoding in GB2312(
http://en.wikipedia.org/wiki/GB2312). Both of them are two-bytes code which
are not subset of Unicode.

BIG5 is de facto standard in Taiwan and GB2312 is national standard in China
, they are most popular encoding in for Chinese especially under Windows.

To solve this problem, we must decide locale setting in runtime  and
transform  characters  back  to  UTF8  if  the locale setting  is not in
UTF8.

Hope these can help.

YungLee

2008/5/29 Sylvestre Ledru <sylvestre.ledru at inria.fr>:Iti

> Hello Yung-Jang,
>
> I haven't forget you. Just busy on other stuff.
>
> Note that I am working under Linux Debian with only-required-packages
> (understand: I am not sure to have all the font for the traditionnal
> chinese language).
>
> After playing with internal & external bugs, I have been able to
> reproduce your bug and to fix it (you can call it a workaround)
>
> When I launch this way:
> # LANG=zh_TW ./bin/scilab
> I get "you-failed.jpg"
>
> but when I launched it with:
> # LANG=zh_TW.UTF-8 ./bin/scilab
> I get "Working.jpg"
>
> Does it mean anything to you ?
> (for those who don't know, Java is full Unicode)
>
> Is there zh_TW charset which would not be UTF-8 ?
>
> Sylvestre
>
>
>
> > Hi, everybody :
> >
> > I am just beginning to  translate message PO files (in Scilab
> > 5-beta-1) into Chinese language (zh_TW),  and hope that will meet the
> > next release schedule of Scilab 5.
> >
> > To test my translation (only few PO file right now), I generated
> > scilab.mo and scilab.po files in loclate\zh_TW\LC_MESSAGES with
> > script  generatePoFile in directory  toos\localization after setting
> > language to 'zh_TW'.
> >
> > The attached files are screehshots  after startup  of Scilab  showing
> > banner lines of my Chinese messages by   scilex.exe and Wscilex.exe
> > respectively in Windows-XP.
> >
> > The first screehshot (scilexChinese.png generated by scilex.exe )
> > correctly displays Chinese characters , while the second  one
> > (wscilexChinese.png by Wscilex.exe) fails.
> >
> > My Windows-XP locale setting is zh_TW.BIG5 , while  messages  in
> > core.po file  are in UTF-8 encoding.
> >
> > It appears to me that  Windows  will transform messages back to  its
> > default encoding (in my case, BIG5) and DOS console (scilex) handle
> > this correctly.  But Java console expect an UTF-8  messages and the
> > received  BIG5 encoding messages cause its failure in showing Chinese
> > banner lines.
> >
> > I think this is not a specific problem associate with Chinese
> > language, any multibytes language with  locale setting not in UTF-8
> > will probable  fail in the same way.
> >
> > I try to trace the source codes of Java console module to find a way
> > to attack this problem.  Transform messages back to UTF-8 in
> > ConsolePrint.cpp ( module console) may help , for example :
> >
> > int ConsolePrintf(char *line)
> > {
> >   lineUtf8=localeToUtf8(line); // <------ transform back to utf8
> >   CallScilabBridge::display(getScilabJavaVM(), line);
> >   return 0;
> > }
> >
> > In this way, we need a function 'localeToUtf8'  which transform output
> > strings from locale to UTF-8 encoding. This function is provided in
> > Glib and may be implemented form iconv API within current Scialb
> > developement environment.
> >
> > Hope my test can help and looking forward for suggestions or
> > instructions to attack this problems.
> >
> > Best regards,
> >
> > YungLee
> >
> >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.scilab.org/pipermail/dev/attachments/20080530/aa091144/attachment.htm>