From: Anton Kazennikov
Subject: Different behavior of deftype on clisp and sbcl. How it should really work?
Date: Sat, 30 Sep 2006 13:58:55 +0000
Message-ID: <87wt7llalc.fsf@kzn.homelinux.org>

Hello,

I wrote the following piece of code:

(deftype whitespace () '(member #\Space #\Newline #\Return #\Tab))
(deftype digit () '(member #\0 #\1 #\2 #\3 #\4 #\5 #\6 #\7 #\8 #\9))
(deftype punctuation () '(member #\. #\, #\! #\? #\; #\" #\' #\] #\[ #\( #\) #\\ #\/  #\{ #\} #\:))
(deftype text-type () '(not (or digit whitespace punctuation)))

(defparameter test-string "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ.")

(defun test-text-type () 
  (loop 
     for i from 0 to (1- (length test-string)) 
     collect (typep (aref test-string i) 'text-type)))


(test-text-type) on sbcl returns
(NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL
 NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL
 NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL)

when on clisp (test-text-type) returns

(T T T T T T T T T T T T T T T T T T T T T T T T T T T T T T T T T T T T T T T
 T T T T T T T T T T T T T NIL)

The bug is in sbcl or in clisp? And how text-type should be correctly defined
if I am wrong?

-- 
With best regards,
Anton Kazennikov.  mailto:kazennikov[at]mirea.ru ICQ# 98965967

• Re: Different behavior of deftype on clisp and sbcl. How it should really work? Richard S. Hall
  • Re: Different behavior of deftype on clisp and sbcl. How it should really work? Bill Atkins
    • Re: Different behavior of deftype on clisp and sbcl. How it should really work? Richard S. Hall
  • Re: Different behavior of deftype on clisp and sbcl. How it should really work? Richard S. Hall
    • Re: Different behavior of deftype on clisp and sbcl. How it should really work? Bill Atkins
    • Re: Different behavior of deftype on clisp and sbcl. How it should really work? Paul Dietz
• Re: Different behavior of deftype on clisp and sbcl. How it should really work? Juho Snellman
• Re: Different behavior of deftype on clisp and sbcl. How it should really work? Pascal Bourguignon

From: Richard S. Hall
Subject: Re: Different behavior of deftype on clisp and sbcl. How it should really work?
Date: Sat, 30 Sep 2006 15:44:50 +0000
Message-ID: <ShwTg.3203$Y24.2387@newsread4.news.pas.earthlink.net>

Use "(deftype xxx () #'(member ... :test #'char=)"

member by default uses #'eq as the test.  Whether or not characters with 
identical print-names are actually identical objects is underspecified in 
the Common Lisp spec to be implementation specific. In SBCL they are not. 
Using char= as the test will work in any implmentation.  Also you should use 
#' instead of just ' in front of the member statement.  If you compile the 
code and use just ' the member statement will be interpreted not compiled 
where #' will ensure its compiled.  The bottom line, either will work, but 
#' (may) be faster...

"Anton Kazennikov" <···············@gmail.com> wrote in message 
···················@kzn.homelinux.org...
>
> Hello,
>
> I wrote the following piece of code:
>
> (deftype whitespace () '(member #\Space #\Newline #\Return #\Tab))
> (deftype digit () '(member #\0 #\1 #\2 #\3 #\4 #\5 #\6 #\7 #\8 #\9))
> (deftype punctuation () '(member #\. #\, #\! #\? #\; #\" #\' #\] #\[ #\( 
> #\) #\\ #\/  #\{ #\} #\:))
> (deftype text-type () '(not (or digit whitespace punctuation)))
>
> (defparameter test-string 
> "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ.")
>
> (defun test-text-type ()
>  (loop
>     for i from 0 to (1- (length test-string))
>     collect (typep (aref test-string i) 'text-type)))
>
>
> (test-text-type) on sbcl returns
> (NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL 
> NIL
> NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL 
> NIL
> NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL)
>
> when on clisp (test-text-type) returns
>
> (T T T T T T T T T T T T T T T T T T T T T T T T T T T T T T T T T T T T T 
> T T
> T T T T T T T T T T T T T NIL)
>
> The bug is in sbcl or in clisp? And how text-type should be correctly 
> defined
> if I am wrong?
>
> -- 
> With best regards,
> Anton Kazennikov.  mailto:kazennikov[at]mirea.ru ICQ# 98965967 

From: Bill Atkins
Subject: Re: Different behavior of deftype on clisp and sbcl. How it should really work?
Date: Sat, 30 Sep 2006 16:32:03 +0000
Message-ID: <m2y7s1pb7g.fsf@weedle-24.dynamic.rpi.edu>

"Richard S. Hall" <·······@earthlink.net> writes:

> Use "(deftype xxx () #'(member ... :test #'char=)"

Wrong.  Use (deftype xxx () '(member #\a #\b)) .

CL-USER 5 : 2 > (deftype xxx () '(member #\a #\b))
XXX

CL-USER 6 : 2 > (typep #\a 'xxx)
T

CL-USER 7 : 2 > (typep #\c 'xxx)
NIL


> member by default uses #'eq as the test.  

Wrong.  It uses EQL.

> Whether or not characters with 
> identical print-names are actually identical objects is underspecified in 
> the Common Lisp spec to be implementation specific. 

Characters don't have print names.  EQL is guaranteed to return T if
two identical characters are passed to it, regardless of whether or
not they are the same object.

> In SBCL they are not. 
> Using char= as the test will work in any implmentation.  

So will EQL!

> Also you should use 
> #' instead of just ' in front of the member statement.

Wrong.  DEFTYPE takes a type specifier, not a function.

>  If you compile the 
> code and use just ' the member statement will be interpreted not compiled 
> where #' will ensure its compiled. 

Huh??  In SBCL in particular, there is no interpreter; everything is
compiled.  In other implementations, your statement is still wrong.

> The bottom line, either will work, but 
> #' (may) be faster...

Don't top-post.

Giving wrong advice is worse than giving no advice at all.

From: Richard S. Hall
Subject: Re: Different behavior of deftype on clisp and sbcl. How it should really work?
Date: Sat, 30 Sep 2006 17:26:31 +0000
Message-ID: <bNxTg.3308$Y24.2364@newsread4.news.pas.earthlink.net>

"Bill Atkins" <······@rpi.edu> wrote in message 
···················@weedle-24.dynamic.rpi.edu...
> "Richard S. Hall" <·······@earthlink.net> writes:
>
>> Use "(deftype xxx () #'(member ... :test #'char=)"
>
> Wrong.  Use (deftype xxx () '(member #\a #\b)) .
>
> CL-USER 5 : 2 > (deftype xxx () '(member #\a #\b))
> XXX
>
> CL-USER 6 : 2 > (typep #\a 'xxx)
> T
>
> CL-USER 7 : 2 > (typep #\c 'xxx)
> NIL
>
>
>> member by default uses #'eq as the test.
>
> Wrong.  It uses EQL.
>
>> Whether or not characters with
>> identical print-names are actually identical objects is underspecified in
>> the Common Lisp spec to be implementation specific.
>
> Characters don't have print names.  EQL is guaranteed to return T if
> two identical characters are passed to it, regardless of whether or
> not they are the same object.
>
>> In SBCL they are not.
>> Using char= as the test will work in any implmentation.
>
> So will EQL!
>
>> Also you should use
>> #' instead of just ' in front of the member statement.
>
> Wrong.  DEFTYPE takes a type specifier, not a function.
>
>>  If you compile the
>> code and use just ' the member statement will be interpreted not compiled
>> where #' will ensure its compiled.
>
> Huh??  In SBCL in particular, there is no interpreter; everything is
> compiled.  In other implementations, your statement is still wrong.
>
>> The bottom line, either will work, but
>> #' (may) be faster...
>
> Don't top-post.
>
> Giving wrong advice is worse than giving no advice at all.

You are right, I should do my homework before trusting my foggy old memory! 
The issue with SBCL is not with character objects but with strings.  In 
SBCL:

>(member "a" '("a" "b" "c" "d"))
NIL ; (which may be non nil in many implementations)

or even
 >(eq "a" "a")
NIL

>(eql "a" "a")
NIL

however:

(member #\a '(#\a #\b #\c #\d))
(#\a #\b #\c #\d) ; (always non-nil)

I've had this issue with things that "work by coincedence" in other 
implementations "working as designed" in SBCL so I jumped to conclusions 
because its the first thing I think of when I see something that looks the 
same (OK print-name is the wrong term) being unequal in SBCL.  BTW the 
hyperspec doesn't say what the member FUNCTION defaults to, although it does 
on the TYPE-SPECIFIER so I relied on my memory. Clearly a mistake.  As a 
result I learned something that will stick better in my memory.  Hopefully 
more good than harm was ultimately done. :-)

  -- Rick

From: Richard S. Hall
Subject: Re: Different behavior of deftype on clisp and sbcl. How it should really work?
Date: Sat, 30 Sep 2006 16:31:07 +0000
Message-ID: <fZwTg.3227$Y24.1383@newsread4.news.pas.earthlink.net>

My bad, I just re-read the hyperspec - member FUNCTION uses #'eq by 
default - this is not true of member TYPE-SPECIFIER which uses #'eql, which 
should work.  Therefore, it is a bug.

"Richard S. Hall" <·······@earthlink.net> wrote in message 
························@newsread4.news.pas.earthlink.net...
> Use "(deftype xxx () #'(member ... :test #'char=)"
>
> member by default uses #'eq as the test.  Whether or not characters with 
> identical print-names are actually identical objects is underspecified in 
> the Common Lisp spec to be implementation specific. In SBCL they are not. 
> Using char= as the test will work in any implmentation.  Also you should 
> use #' instead of just ' in front of the member statement.  If you compile 
> the code and use just ' the member statement will be interpreted not 
> compiled where #' will ensure its compiled.  The bottom line, either will 
> work, but #' (may) be faster...
>
> "Anton Kazennikov" <···············@gmail.com> wrote in message 
> ···················@kzn.homelinux.org...
>>
>> Hello,
>>
>> I wrote the following piece of code:
>>
>> (deftype whitespace () '(member #\Space #\Newline #\Return #\Tab))
>> (deftype digit () '(member #\0 #\1 #\2 #\3 #\4 #\5 #\6 #\7 #\8 #\9))
>> (deftype punctuation () '(member #\. #\, #\! #\? #\; #\" #\' #\] #\[ 
>> #\( #\) #\\ #\/  #\{ #\} #\:))
>> (deftype text-type () '(not (or digit whitespace punctuation)))
>>
>> (defparameter test-string 
>> "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ.")
>>
>> (defun test-text-type ()
>>  (loop
>>     for i from 0 to (1- (length test-string))
>>     collect (typep (aref test-string i) 'text-type)))
>>
>>
>> (test-text-type) on sbcl returns
>> (NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL 
>> NIL
>> NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL 
>> NIL
>> NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL)
>>
>> when on clisp (test-text-type) returns
>>
>> (T T T T T T T T T T T T T T T T T T T T T T T T T T T T T T T T T T T T 
>> T T T
>> T T T T T T T T T T T T T NIL)
>>
>> The bug is in sbcl or in clisp? And how text-type should be correctly 
>> defined
>> if I am wrong?
>>
>> -- 
>> With best regards,
>> Anton Kazennikov.  mailto:kazennikov[at]mirea.ru ICQ# 98965967
>
> 

From: Bill Atkins
Subject: Re: Different behavior of deftype on clisp and sbcl. How it should really work?
Date: Sat, 30 Sep 2006 16:39:57 +0000
Message-ID: <m2odsxpaua.fsf@weedle-24.dynamic.rpi.edu>

"Richard S. Hall" <·······@earthlink.net> writes:

> My bad, I just re-read the hyperspec - member FUNCTION uses #'eq by 
> default 

Where do you see this?

From: Paul Dietz
Subject: Re: Different behavior of deftype on clisp and sbcl. How it should really work?
Date: Mon, 02 Oct 2006 20:31:54 +0000
Message-ID: <efrsvq$r4l$1@engnntp2.cig.mot.com>

Richard S. Hall wrote:

> My bad, I just re-read the hyperspec - member FUNCTION uses #'eq by 
> default 

No, it uses EQL by default.

	Paul

From: Juho Snellman
Subject: Re: Different behavior of deftype on clisp and sbcl. How it should really work?
Date: Sat, 30 Sep 2006 16:04:51 +0000
Message-ID: <slrneht5d3.lhl.jsnell@sbz-30.cs.Helsinki.FI>

Anton Kazennikov <···············@gmail.com> wrote:
> (deftype whitespace () '(member #\Space #\Newline #\Return #\Tab))
> (deftype digit () '(member #\0 #\1 #\2 #\3 #\4 #\5 #\6 #\7 #\8 #\9))
> (deftype punctuation () '(member #\. #\, #\! #\? #\; #\" #\' #\] #\[ #\( #\) #\\ #\/  #\{ #\} #\:))
> (deftype text-type () '(not (or digit whitespace punctuation)))
>
> (defparameter test-string "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ.")
>
> (defun test-text-type () 
>   (loop 
>      for i from 0 to (1- (length test-string)) 
>      collect (typep (aref test-string i) 'text-type)))
>
>
> (test-text-type) on sbcl returns
> (NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL
>  NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL
>  NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL)

Looks like a bug in SBCL (related to negated character set types).
Applying the following patch and recompiling SBCL should fix the
problem.

Index: src/code/late-type.lisp
===================================================================
RCS file: /cvsroot/sbcl/sbcl/src/code/late-type.lisp,v
retrieving revision 1.134
diff -u -b -r1.134 late-type.lisp
--- src/code/late-type.lisp	13 Sep 2006 15:59:33 -0000	1.134
+++ src/code/late-type.lisp	30 Sep 2006 15:40:45 -0000
@@ -3118,8 +3118,8 @@
                     (when (> (caar pairs) 0)
                       (push (cons 0 (1- (caar pairs))) not-pairs))
                     (do* ((tail pairs (cdr tail))
-                          (high1 (cdar tail))
-                          (low2 (caadr tail)))
+                           (high1 (cdar tail) (cdar tail))
+                           (low2 (caadr tail) (caadr tail)))
                          ((null (cdr tail))
                           (when (< (cdar tail) (1- sb!xc:char-code-limit))
                             (push (cons (1+ (cdar tail))

-- 
Juho Snellman

From: Pascal Bourguignon
Subject: Re: Different behavior of deftype on clisp and sbcl. How it should really work?
Date: Sat, 30 Sep 2006 15:33:04 +0000
Message-ID: <87wt7lqsi7.fsf@thalassa.informatimago.com>

Anton Kazennikov <···············@gmail.com> writes:

> Hello,
>
> I wrote the following piece of code:
>
> (deftype whitespace () '(member #\Space #\Newline #\Return #\Tab))
> (deftype digit () '(member #\0 #\1 #\2 #\3 #\4 #\5 #\6 #\7 #\8 #\9))
> (deftype punctuation () '(member #\. #\, #\! #\? #\; #\" #\' #\] #\[ #\( #\) #\\ #\/  #\{ #\} #\:))
> (deftype text-type () '(not (or digit whitespace punctuation)))
>
> (defparameter test-string "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ.")
>
> (defun test-text-type () 
>   (loop 
>      for i from 0 to (1- (length test-string)) 
>      collect (typep (aref test-string i) 'text-type)))
>
>
> (test-text-type) on sbcl returns
> (NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL
>  NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL
>  NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL NIL)
>
> when on clisp (test-text-type) returns
>
> (T T T T T T T T T T T T T T T T T T T T T T T T T T T T T T T T T T T T T T T
>  T T T T T T T T T T T T T NIL)
>
> The bug is in sbcl or in clisp? 

The bug is with sbcl.


> And how text-type should be correctly defined if I am wrong?

There is a ALPHA-CHAR-P and a ALPHANUMERICP predicate.

You're missing all these digits:

(defun numericp (ch) (and (ALPHANUMERICP ch) (not (alpha-char-p ch))))
(coerce (loop :for i :from 0 :to #x110000
              :when (numericp (code-char i)) :collect (code-char i)) 'string)

Perhaps sbcl supports better SATISFIES types than OR.

(defun spacep (ch)  ; on Mac, newline could be return!!!
    (member ch '(#\Space #\Newline #\linefeed #\Return #\Tab)))
(defun punctuationp (ch) 
   (not (or (spacep ch) (alphanumericp ch))))

(deftype whitespace  () '(satisfies spacep))
(deftype digit       () '(satisfies numericp))
(deftype alphabetic  () '(satisfies alpha-char-p))
(deftype punctuation () '(satisfies punctuationp))

(defparameter *test-string*
    "HeiHyvääpäivääΓειάσαςשלוםЗдравствуйтеสวัสดีครับสวัสดีค่ะこんにちはｺﾝﾆﾁﾊ你好안녕하세요안녕하십니까")


(defun test-alphabetic () 
   (loop :for ch :across *test-string* 
         :collect (cons (typep ch 'alphabetic) (string ch))))

Gives good results on both implementations, but there remains a few
differences  between sbcl and clisp:

(coerce (loop :for ch :across *test-string* 
      :collect (if (typep ch 'alphabetic) 1 0)) 'bit-vector)

SBCL:
#*1111111111111111111111111111111111111110110110111011010111111111111111111111111
#*1111111111111111111111111111111111111111111111111111111111111111111111111111111
CLISP:

"ั" being considered an "accent" by SBCL.

SBCL> (let ((*test-string* "สวัสดีครับ")) (test-alphabetic))
((T . "ส") (T . "ว") (NIL . "ั") (T . "ส") (T . "ด") (NIL . "ี") (T . "ค")
 (T . "ร") (NIL . "ั") (T . "บ"))


CLISP> (let ((*test-string* "สวัสดีครับ")) (test-alphabetic))
((T . "ส") (T . "ว") (T . "ั") (T . "ส") (T . "ด") (T . "ี") (T . "ค")
 (T . "ร") (T . "ั") (T . "บ"))


I don't know who splits the "วั" into "วั", perhaps it's emacs, perhaps
the inferior lisps.


(And it seems that some characters found in emacs Hello file cannot be
encoded in unicode.)

-- 
__Pascal Bourguignon__                     http://www.informatimago.com/

"This statement is false."            In Lisp: (defun Q () (eq nil (Q)))