From: Markus Grueneis
Subject: Newbie: utf-8 on slime/clisp win32
Date: 
Message-ID: <42dp8bF1hh69dU1@individual.net>
MOi!


This official de-lurk after reading some dozen pages of PCL is about 
some, as it seems, configuration issues.

As a simple, personal exercise I try to make a vocabulary-trainer for 
Chinese, based on CEDICT [1], a nice free dict available in several 
encodings including utf-8.

First of all, I would want to read in the file, and for this beginning 
end of my competence level a text-interface in the slime-clisp-repl will 
have to do.

After the first test to just write the file-contents with:

(with-open-file (stream "d:/some/correct/path/to/dict.txt"
                             :external-format charset:utf-8)
   (loop for line = (read-line stream nil)
        while line do (format t "~a~%" line)))

the line

 > # CEDICT Sat Jun 11 19:41:50 2005; Copyright 2005; -*- coding: \
 > utf-8-dos -*-

printed correctly, and then I got this error:

 > Character #\u5475 cannot be represented in the character set
 > CHARSET:ISO-8859-1
 >
 >    [Condition of type EXT:SIMPLE-CHARSET-TYPE-ERROR]

If it matters, I'm working with Emacs v22 on WinXP and using lispbox-0.6 
with Clisp.

After some quick googling I came to [2], and followed the steps, 
summarized as MULE->UTF8, slime-net-coding-system 'utf-8-unix.

As it stands, copy-and-paste into my Emacs (v22) did work, and the 
characters show correctly, but still I can neither send such Chars to 
Clisp, nor can the CL format call print it.  For instance, if I enter 
"∑" or "呵" in the REPL, I get a slightly different error, seems to have 
to do with the Emacs-side, where I'm even more a newbie:

 > Debugger entered--Lisp error:
 > (error "Coding system iso-latin-1-unix not suitable for
 >        \"00004f(:emacs-rex (swank:listener-eval
 >        \\\"\\\\\\\"\x53c31\\\\\\\"\\\")
 >        \\\"COMMON-LISP-USER\\\" :repl-thread 54)\"")

Ok, I have to change the coding system, to something like utf-8.  But 
where and how?  Another search and a few clicks on the results didn't 
help me much (just the same instructions another time), and Cliki is 
also quiet about this topic.

I would be thankfull for any pointers, howtos, correct search terms etc.!


Thanks,
Markus

[1] http://www.mandarintools.com/cedict.html
[2] http://nolan.eakins.net/node/150

From: Sam Steingold
Subject: Re: Newbie: utf-8 on slime/clisp win32
Date: 
Message-ID: <uirsu85ac.fsf@gnu.org>
> * Markus Grueneis <··········@tzk.arg> [2006-01-09 01:32:10 +0100]:
>
>  > Character #\u5475 cannot be represented in the character set
>  > CHARSET:ISO-8859-1

http://clisp.cons.org/impnotes/faq.html#faq-enc-err

-- 
Sam Steingold (http://www.podval.org/~sds) running w2k
http://www.openvotingconsortium.org/ http://www.jihadwatch.org/
http://www.camera.org http://www.savegushkatif.org http://www.mideasttruth.com/
If you try to fail, and succeed, which have you done?
From: Markus Grueneis
Subject: Re: Newbie: utf-8 on slime/clisp win32
Date: 
Message-ID: <42euvkF1iju55U1@individual.net>
Sam Steingold wrote:
>> * Markus Grueneis <··········@tzk.arg> [2006-01-09 01:32:10 +0100]:
>>
>>  > Character #\u5475 cannot be represented in the character set
>>  > CHARSET:ISO-8859-1
> 
> http://clisp.cons.org/impnotes/faq.html#faq-enc-err
> 

Thank you very much for this URL, but if I understand that piece:

 > Note that this error may be signaled by the Print part of the
 > read-eval-print loop and not by the function you call. E.g., if file
 > "foo" contains non-ASCII characters, you will see such an error [...]
 >

correctly, than there is know way with Clisp to print a chinese 
character to standard output?

I didn't have much time yet and will inspect this further on evening, so 
if it's kinda silly, just ignore it please.


Best regards,
Markus
From: Pascal Bourguignon
Subject: Re: Newbie: utf-8 on slime/clisp win32
Date: 
Message-ID: <877j99wopv.fsf@thalassa.informatimago.com>
Markus Grueneis <··········@gmx.net> writes:

> Sam Steingold wrote:
>>> * Markus Grueneis <··········@tzk.arg> [2006-01-09 01:32:10 +0100]:
>>>
>>>  > Character #\u5475 cannot be represented in the character set
>>>  > CHARSET:ISO-8859-1
>> http://clisp.cons.org/impnotes/faq.html#faq-enc-err
>> 
>
> Thank you very much for this URL, but if I understand that piece:
>
>  > Note that this error may be signaled by the Print part of the
>  > read-eval-print loop and not by the function you call. E.g., if file
>  > "foo" contains non-ASCII characters, you will see such an error [...]
>  >
>
> correctly, than there is know way with Clisp to print a chinese
> character to standard output?

If you have a utf-8 standard output, you can print anything.  Of
course, on the other side,  you need to have a utf-8 terminal to be
able to _see_ anything.


-- 
__Pascal Bourguignon__                     http://www.informatimago.com/

PLEASE NOTE: Some quantum physics theories suggest that when the
consumer is not directly observing this product, it may cease to
exist or will exist only in a vague and undetermined state.
From: Markus Grueneis
Subject: Re: Newbie: utf-8 on slime/clisp win32
Date: 
Message-ID: <42fsp9F1j0u9mU1@individual.net>
Pascal Bourguignon wrote:
> [...]
> 
> If you have a utf-8 standard output, you can print anything.  Of
> course, on the other side,  you need to have a utf-8 terminal to be
> able to _see_ anything.
> 

That I know, but never mind.  I just tried it another time with exactly 
the same .emacs configuration as at the last midnight session, and 
everything worked out of the box.  Probably I'm too stupid to restart 
emacs or whatever, I can't remember any difference.

However, it just works, so OK.


Thanks for all your effort, my fault,
Markus
From: ogliomus
Subject: Re: Newbie: utf-8 on slime/clisp win32
Date: 
Message-ID: <rt28s11qb5sdovhb0v30l890vi4a08o15i@4ax.com>
Markus Grueneis <··········@gmx.net> wrote:
>> [...]
>
>That I know, but never mind.  I just tried it another time with exactly 
>the same .emacs configuration as at the last midnight session, and 
>everything worked out of the box.  Probably I'm too stupid to restart 
>emacs or whatever, I can't remember any difference.
>
>However, it just works, so OK.
>
>
>Thanks for all your effort, my fault,

Thank you beacuse I gave up about _printing_ cjk characters using
slime clisp and emacs on windows 2000. But after reading your post I
tryed again and realized that it works with emacsW32 and (I suppose,
not tested thoroughly) also with emacs and Xemacs cygwin. But doesn't
work for me with emacs (official port?) shipped with lispbox.
GC
From: Markus Grueneis
Subject: Re: Newbie: utf-8 on slime/clisp win32
Date: 
Message-ID: <43C52961.80208@gmx.net>
ogliomus wrote:
> [...]
> 
> Thank you beacuse I gave up about _printing_ cjk characters using
> slime clisp and emacs on windows 2000. But after reading your post I
> tryed again and realized that it works with emacsW32 and (I suppose,
> not tested thoroughly) also with emacs and Xemacs cygwin. But doesn't
> work for me with emacs (official port?) shipped with lispbox.
> GC

I don't know what w32-lispbox Version you are using, the recently 
announced lispbox-0.7 from Peter Seibel includes 21.3, which is the 
first stable version handling UCS in a "sane" way.

After all, with v21 you need mule-ucs installed (a compressed 1MB 
package), wich is, as I just looked up, not installed in lispbox.  You 
can get it at [3].

I'm now using v22, and at least Gentoo Doc [2] says UTF-8 support is 
complete (on the other side, Xemacs v22 not, same paragraph).


mfg
Markus


[1] http://www.cl.cam.ac.uk/~mgk25/unicode.html#apps
[2] http://www.gentoo.org/doc/en/utf-8.xml#doc_chap3, search for "emacs"
[3] 
http://dir.filewatcher.com/d/NetBSD/1.6.2/sparc64/Mule-UCS-0.84nb4.tgz.1201653.html
From: ogliomus
Subject: Re: Newbie: utf-8 on slime/clisp win32
Date: 
Message-ID: <npkas1h335vna6e25n4flmoefgt8jtfmun@4ax.com>
Markus Grueneis <··········@gmx.net> wrote:
>> [Emacs unicode support]
>
>I don't know what w32-lispbox Version you are using, the recently 
>announced lispbox-0.7 from Peter Seibel includes 21.3, which is the 
>first stable version handling UCS in a "sane" way.
>
>After all, with v21 you need mule-ucs installed (a compressed 1MB 
>package), wich is, as I just looked up, not installed in lispbox.  You 
>can get it at [3].
>

I downloaded the mule-ucs package, but still I can't paste japanese
characters in emacs 21.3 in windows 2000. The library is in the
"load-path", i wrote the few lines ((require 'un-define) etc.) in
~.emacs start file but Emacs still handle the characters as two byte. 
  
>I'm now using v22, and at least Gentoo Doc [2] says UTF-8 support is 
>complete (on the other side, Xemacs v22 not, same paragraph).

I am using too emacsW32 which is version 22, and it works, so no
problem.
GC

>
>[1] http://www.cl.cam.ac.uk/~mgk25/unicode.html#apps
>[2] http://www.gentoo.org/doc/en/utf-8.xml#doc_chap3, search for "emacs"
>[3] 
>http://dir.filewatcher.com/d/NetBSD/1.6.2/sparc64/Mule-UCS-0.84nb4.tgz.1201653.html
From: Markus Grueneis
Subject: Re: Newbie: utf-8 on slime/clisp win32
Date: 
Message-ID: <43C566E6.4090905@gmx.net>
ogliomus wrote:
 > Markus Grueneis <··········@gmx.net> wrote:
 >>> [Emacs unicode support]
 >> [mule-ucs and probs]
 >
 > I am using too emacsW32 which is version 22, and it works, so no
 > problem.
 > GC
 >

Since I'm also not the Emacs wizard, I can't help you further, sorry, 
but I think we can be happy with our running v22, and forget about the 
odd configuration problems of oldish software ;-)  SLIME works in v22 
without problems, so why would we want to switch anyway?

 >> [links]


Have a nice day,
Markus
From: ogliomus
Subject: Re: Newbie: utf-8 on slime/clisp win32
Date: 
Message-ID: <ks0bs19h0dkcuditbii4d1cloour45qkje@4ax.com>
Markus Grueneis <··········@gmx.net> wrote:
> > Markus Grueneis <··········@gmx.net> wrote:
> >>> [Emacs unicode support]
> >> [mule-ucs and probs]
> >
> > I am using too emacsW32 which is version 22, and it works, so no
> > problem.
> >
>
>Since I'm also not the Emacs wizard, I can't help you further, sorry, 
>but I think we can be happy with our running v22, and forget about the 
>odd configuration problems of oldish software ;-)  SLIME works in v22 
>without problems, so why would we want to switch anyway?
>
... and if you write some _even_very_simple_ vocabulary-trainer for
chinese, why not consider releasing it to the "pubblic domain"? May be
there is someone that would like to give it a try.
GC
From: Markus Grueneis
Subject: Re: Newbie: utf-8 on slime/clisp win32
Date: 
Message-ID: <42mnh2F1k0hvvU1@individual.net>
ogliomus wrote:
> Markus Grueneis <··········@gmx.net> wrote:
>> [Emacs]
>
> ... and if you write some _even_very_simple_ vocabulary-trainer for
> chinese, why not consider releasing it to the "pubblic domain"? May be
> there is someone that would like to give it a try.
> GC

Will be, will be.  But I'll have to learn the language first!

I consider the latter part to be a big one for me...


mfg
Markus
From: Peter Seibel
Subject: Re: Newbie: utf-8 on slime/clisp win32
Date: 
Message-ID: <m23bjp1emi.fsf@gigamonkeys.com>
Markus Grueneis <··········@gmx.net> writes:

> ogliomus wrote:
>> Markus Grueneis <··········@gmx.net> wrote:
>>> [Emacs]
>>
>> ... and if you write some _even_very_simple_ vocabulary-trainer for
>> chinese, why not consider releasing it to the "pubblic domain"? May
>> be there is someone that would like to give it a try. GC
>
> Will be, will be.  But I'll have to learn the language first!

I hope you're talking about Lisp, not Chinese.

-Peter

-- 
Peter Seibel           * ·····@gigamonkeys.com
Gigamonkeys Consulting * http://www.gigamonkeys.com/
Practical Common Lisp  * http://www.gigamonkeys.com/book/
From: Markus Grueneis
Subject: Re: Newbie: utf-8 on slime/clisp win32
Date: 
Message-ID: <42vntlF1l1q25U1@individual.net>
Peter Seibel wrote:
 > [comments and suggestions...]
>> Will be, will be.  But I'll have to learn the language first!
> 
> I hope you're talking about Lisp, not Chinese.
> 

Well, I think a ready vocabulary-trainer is more useful when learning 
chinese than afterwards...


mfg
Markus

> -Peter
>