From: Andreas Thiele
Subject: Uffi and unicode
Date: 
Message-ID: <fnj426$8bh$02$1@news.t-online.com>
Hi,

can anybody give me a quick hint? I wrote a piece of software which comunicates via Uffi to a database. It works with Lispworks and CMUCL. Now I tested sbcl and have a unicode problem while neither my app nor the db is unicode aware.

Were do I have look to fix this?

Andreas

From: Andreas Thiele
Subject: Re: Uffi and unicode
Date: 
Message-ID: <fnj9t7$ia1$02$1@news.t-online.com>
Andreas Thiele wrote:
> Hi,
> 
> can anybody give me a quick hint? I wrote a piece of software which comunicates via Uffi to a database. It works with Lispworks
> and CMUCL. Now I tested sbcl and have a unicode problem while neither my app nor the db is unicode aware. 
> 
> Were do I have look to fix this?
> 
> Andreas

I found a quick hack, just compiled a fresh sbcl without :sb-unicode - works fine.

Andreas
From: Luís Oliveira
Subject: Re: Uffi and unicode
Date: 
Message-ID: <m1zluqpqxf.fsf@deadspam.com>
"Andreas Thiele" <······@nospam.com> writes:
> can anybody give me a quick hint? I wrote a piece of software which
> comunicates via Uffi to a database. It works with Lispworks and
> CMUCL. Now I tested sbcl and have a unicode problem while neither my
> app nor the db is unicode aware.

Binding sb-alien::*default-c-string-external-format* to :iso-8859-1
might work.

-- 
Luís Oliveira
http://student.dei.uc.pt/~lmoliv/
From: Andreas Thiele
Subject: Re: Uffi and unicode
Date: 
Message-ID: <fnjm0d$dep$03$1@news.t-online.com>
"Luís" Oliveira wrote:
> "Andreas Thiele" <······@nospam.com> writes:
>> can anybody give me a quick hint? I wrote a piece of software which
>> comunicates via Uffi to a database. It works with Lispworks and
>> CMUCL. Now I tested sbcl and have a unicode problem while neither my
>> app nor the db is unicode aware.
> 
> Binding sb-alien::*default-c-string-external-format* to :iso-8859-1
> might work.

Luis,

thanks for the hint. Unfortunately this already is the default value on my machine and doesn't solve the problem. But I found a brute force hack (my previous post).

Andreas
From: Pascal J. Bourguignon
Subject: Re: Uffi and unicode
Date: 
Message-ID: <7cwspu8bws.fsf@pbourguignon.anevia.com>
"Andreas Thiele" <······@nospam.com> writes:
> can anybody give me a quick hint? I wrote a piece of software which
> comunicates via Uffi to a database. It works with Lispworks and
> CMUCL. Now I tested sbcl and have a unicode problem while neither my
> app nor the db is unicode aware.
>
> Were do I have look to fix this?

Here is what I did to tame encodings with clsql:

(defparameter *weubcwk*
  #+sbcl  :iso-8859-1
  #+clisp charset:iso-8859-1
  "Whatever Encoding Used By Clsql Who Knows")

(defun utf-8-bytes  (data)
  (let ((bytes
         #+sbcl  (sb-ext:string-to-octets     data  :external-format *weubcwk*)
         #+clisp (ext:convert-string-to-bytes data  *weubcwk*)))
    #+sbcl  (sb-ext:octets-to-string          bytes :external-format :utf-8)
    #+clisp (ext:convert-string-from-bytes    bytes charset:utf-8)))

(defun utf-8-string (string)
  (let ((bytes
         #+sbcl  (sb-ext:string-to-octets     string :external-format :utf-8)
         #+clisp (ext:convert-string-to-bytes string charset:utf-8)))
    #+sbcl  (sb-ext:octets-to-string          bytes  :external-format *weubcwk*)
    #+clisp (ext:convert-string-from-bytes    bytes  *weubcwk*)))


(clsql:def-view-class language ()
  (#|...|#
   (name         :db-kind :base
                 :db-constraints :not-null
                 :db-reader utf-8-bytes       ; <---
                 :db-writer utf-8-string      ; <---
                 :type (varchar 75)
                 :reader language-name))
  (:base-table "LANGUAGES"))


-- 
__Pascal Bourguignon__
From: Andreas Thiele
Subject: Re: Uffi and unicode
Date: 
Message-ID: <fnobvu$o54$03$1@news.t-online.com>
Pascal J. Bourguignon wrote:
> "Andreas Thiele" <······@nospam.com> writes:
>> can anybody give me a quick hint? I wrote a piece of software which
>> comunicates via Uffi to a database. It works with Lispworks and
>> CMUCL. Now I tested sbcl and have a unicode problem while neither my
>> app nor the db is unicode aware.
>> 
>> Were do I have look to fix this?
> 
> Here is what I did to tame encodings with clsql:
> 
> (defparameter *weubcwk*
>  #+sbcl  :iso-8859-1
>  #+clisp charset:iso-8859-1
>  "Whatever Encoding Used By Clsql Who Knows")
> 
> (defun utf-8-bytes  (data)
>  (let ((bytes
>         #+sbcl  (sb-ext:string-to-octets     data  :external-format *weubcwk*)
>         #+clisp (ext:convert-string-to-bytes data  *weubcwk*)))
>    #+sbcl  (sb-ext:octets-to-string          bytes :external-format :utf-8)
>    #+clisp (ext:convert-string-from-bytes    bytes charset:utf-8)))
> 
> (defun utf-8-string (string)
>  (let ((bytes
>         #+sbcl  (sb-ext:string-to-octets     string :external-format :utf-8)
>         #+clisp (ext:convert-string-to-bytes string charset:utf-8)))
>    #+sbcl  (sb-ext:octets-to-string          bytes  :external-format *weubcwk*)
>    #+clisp (ext:convert-string-from-bytes    bytes  *weubcwk*)))
> 
> 
> (clsql:def-view-class language ()
>  (#|...|#
>   (name         :db-kind :base
>                 :db-constraints :not-null
>                 :db-reader utf-8-bytes       ; <---
>                 :db-writer utf-8-string      ; <---
>                 :type (varchar 75)
>                 :reader language-name))
>  (:base-table "LANGUAGES"))

Pascal,

thanks for your hints. I use my own ODBC lib, so CLSQL is not involved.

Meanwhile I found it is an Uffi issue. When Uffi sees unicode sbcl it assumes
foreign strings are UTF-8. I will discuss this on the Uffi mailing list and suggest
a patch. It is not sure that all foreign strings are UTF-8.

By patching Uffi I succeeded using the standard sbcl distribution.

Thank you all.

Andreas