From: John Connors
Subject: Slightly less newbie lisp questions..
Date: 
Message-ID: <cp5hcs$m0q$1@newsg4.svr.pol.co.uk>
I've got myself to the point (slowly, working for part of one day a 
week) to the point where I can write a simple software renderer in CL.

I do have a few questions, however.

1> I'm in the REPL. I know I need to use a symbol "foo", but I don't 
know what package it's in (and I won't neccessarily have it loaded) how 
do I find out where it lives? This is SBCL or CMUCL (and SLIME).

2> CMUCL & SBCL have this handy (system:vector-sap var) that lets me 
pass a pointer to a simple array of (unsigned-byte32's) to a c-function 
that takes a pointer to an unsigned int (assuming 32 bit platform). Can 
I do this in other lisps with UFFI without having to duplicate a copy of 
a 648x480 (or worse) pixel buffer array on the stack?


Cyborg Animation Programmer
http://yagc.blogspot.com
http://badbyteblues.blogspot.com

From: Kenny Tilton
Subject: Re: Slightly less newbie lisp questions..
Date: 
Message-ID: <V_rtd.66920$Vk6.50249@twister.nyc.rr.com>
John Connors wrote:
> 
> I've got myself to the point (slowly, working for part of one day a 
> week) to the point where I can write a simple software renderer in CL.
> 
> I do have a few questions, however.
> 
> 1> I'm in the REPL. I know I need to use a symbol "foo", but I don't 
> know what package it's in (and I won't neccessarily have it loaded) how 
> do I find out where it lives? This is SBCL or CMUCL (and SLIME).

(apropos 'foo) or (apropos "foo")

can be a fragment, so the above will also pickup 'fools-rush-in


> 
> 2> CMUCL & SBCL have this handy (system:vector-sap var) that lets me 
> pass a pointer to a simple array of (unsigned-byte32's) to a c-function 
> that takes a pointer to an unsigned int (assuming 32 bit platform). Can 
> I do this in other lisps with UFFI without having to duplicate a copy of 
> a 648x480 (or worse) pixel buffer array on the stack?

Yes. And UFFI comes with a nice PDF doc, so I will refer you to that for 
deets. But watch out, apparently there is some issue between UFFI and 
cmucl/sbcl which can cause massive hoggishness of some ilk... others 
better-informed will pitch in.

kenny

-- 
Cells? Cello? Celtik?: http://www.common-lisp.net/project/cells/
Why Lisp? http://alu.cliki.net/RtL%20Highlight%20Film
From: John Connors
Subject: Re: Slightly less newbie lisp questions..
Date: 
Message-ID: <cpfti0$sbi$2@newsg3.svr.pol.co.uk>
 >
 >
 > John Connors wrote:
 >
 >>
 >> 2> CMUCL & SBCL have this handy (system:vector-sap var) that lets me 
pass a pointer to a simple array of (unsigned-byte32's) to a c-function 
that takes a pointer to an unsigned int (assuming 32 bit platform). Can 
I do this in other lisps with UFFI without having to duplicate a copy of 
a 648x480 (or worse) pixel buffer array on the stack?
 >
 >
 > Kenny Tilton wrote:
 >
 > Yes. And UFFI comes with a nice PDF doc, so I will refer you to that 
for deets. But watch out, apparently there is some issue between UFFI 
and cmucl/sbcl which can cause massive hoggishness of some ilk... others 
better-informed will pitch in.
 >
 > kenny
 >

Thanks. Having RTFM I get the impression I'd have to jump through UFFI's 
hoops: to define my pixel buffer I'd have to..

(deftype pixel :unsigned-int)

(setq *pixel-buffer*
   (allocate-foreign-object pixel
     (* *pixel-buffer-width*
        *pixel-buffer-height*)))

And pixel buffer would be a pointer to the pixel buffer, which I could 
pass to/from C. However, the cost would be that in normal LISP I'd have
to read / setf the array via

(uffi:deref-array *pixel-buffer* '(:array pixel) index)

Whearas, with the vector-sap method I could just

(uffi:def-type pixel :unsigned-int)

(setq *pixel-buffer*
   (make-array 'pixel
     (* *pixel-buffer-width*
        *pixel-buffer-height*)))

..and access *pixel-buffer* the usual way, calling sb-sys:vector-sap 
only at c-call time.

Am I obsessing too much with implementaion details? Or is the cost of 
uffi:deref-array likely to be higher than a simple aref?

-- 
Cyborg Animation Programmer
http://yagc.blogspot.com
http://badbyteblues.blogspot.com


PS: Turns out I *am* obsessing over implementation details. I profiled 
my code and the LISP portion ran damn near instantaneously. The 
bottlneck appears to be at the 'C' side. Oh, I do love to be beside the 
'C' side..boom..boom..

..and anyhow I can always wrap the difference between SBCL/CMUCL & the 
rest of the CL world in some nice macros, with the handy side effect of
defining my window size at compile time, thus enabling constant folding 
on pixel buffer index calculations..

-- 
Cyborg Animation Programmer
http://yagc.blogspot.com
http://badbyteblues.blogspot.com
From: Kenny Tilton
Subject: Re: Slightly less newbie lisp questions..
Date: 
Message-ID: <SjLud.24119$Yh2.10204518@twister.nyc.rr.com>
John Connors wrote:
> 
> 
>  >
>  >
>  > John Connors wrote:
>  >
>  >>
>  >> 2> CMUCL & SBCL have this handy (system:vector-sap var) that lets me 
> pass a pointer to a simple array of (unsigned-byte32's) to a c-function 
> that takes a pointer to an unsigned int (assuming 32 bit platform). Can 
> I do this in other lisps with UFFI without having to duplicate a copy of 
> a 648x480 (or worse) pixel buffer array on the stack?
>  >
>  >
>  > Kenny Tilton wrote:
>  >
>  > Yes. And UFFI comes with a nice PDF doc, so I will refer you to that 
> for deets. But watch out, apparently there is some issue between UFFI 
> and cmucl/sbcl which can cause massive hoggishness of some ilk... others 
> better-informed will pitch in.
>  >
>  > kenny
>  >
> 
> Thanks. Having RTFM I get the impression I'd have to jump through UFFI's 
> hoops: to define my pixel buffer I'd have to..
> 
> (deftype pixel :unsigned-int)
> 
> (setq *pixel-buffer*
>   (allocate-foreign-object pixel
>     (* *pixel-buffer-width*
>        *pixel-buffer-height*)))
> 
> And pixel buffer would be a pointer to the pixel buffer, which I could 
> pass to/from C. However, the cost would be that in normal LISP I'd have
> to read / setf the array via
> 
> (uffi:deref-array *pixel-buffer* '(:array pixel) index)
> 
> Whearas, with the vector-sap method I could just
> 
> (uffi:def-type pixel :unsigned-int)
> 
> (setq *pixel-buffer*
>   (make-array 'pixel
>     (* *pixel-buffer-width*
>        *pixel-buffer-height*)))
> 
> ..and access *pixel-buffer* the usual way, calling sb-sys:vector-sap 
> only at c-call time.
> 
> Am I obsessing too much with implementaion details? Or is the cost of 
> uffi:deref-array likely to be higher than a simple aref?
> 

I do not know, and it likely varies between implementations. I suggest 
you simply code up a couple of loops (one an UFFI array, none an 
ordinary Lisp array) and use TIME to see if there is any difference. You 
might also use MACROEXPAND-1 to see into what the UFFI code expands. 
UFFI takes advantage of whatever the underlying implementation can 
handle, so I wager you get the same code. And in other Lisps, if they 
can do what CMUCL/SBCL do, so will UFFI. if not, things might go the 
other way: the array is a foreign array, but the implementation may 
offer array access just as fast as native Lisp array access. In Allegro 
CL your examples expand to:

(ff:allocate-fobject (list :array 'pixel
                        (* *pixel-buffer-width*
                          *pixel-buffer-height*)) :c)

(ff:fslot-value-typed '(:array pixel) :c *pixel-buffer* index)

Not that I have tested the latter to see how fast it is. But it looks 
fast. :)

kenny

-- 
Cells? Cello? Celtik?: http://www.common-lisp.net/project/cells/
Why Lisp? http://alu.cliki.net/RtL%20Highlight%20Film
From: John Connors
Subject: Re: Slightly less newbie lisp questions..
Date: 
Message-ID: <cq2fls$jmh$2@news7.svr.pol.co.uk>
Kenny Tilton wrote:
>>
>> Am I obsessing too much with implementaion details? Or is the cost of 
>> uffi:deref-array likely to be higher than a simple aref?
>>
> 
> I do not know, and it likely varies between implementations. I suggest 
> you simply code up a couple of loops (one an UFFI array, none an 
> ordinary Lisp array) and use TIME to see if there is any difference. You 
> might also use MACROEXPAND-1 to see into what the UFFI code expands. 
> UFFI takes advantage of whatever the underlying implementation can 
> handle, so I wager you get the same code. And in other Lisps, if they 
> can do what CMUCL/SBCL do, so will UFFI. if not, things might go the 
> other way: the array is a foreign array, but the implementation may 
> offer array access just as fast as native Lisp array access. In Allegro 
> CL your examples expand to:
> 
> (ff:allocate-fobject (list :array 'pixel
>                        (* *pixel-buffer-width*
>                          *pixel-buffer-height*)) :c)
> 
> (ff:fslot-value-typed '(:array pixel) :c *pixel-buffer* index)
> 
> Not that I have tested the latter to see how fast it is. But it looks 
> fast. :)
> 
> kenny
> 
I've been playing around with sbcl profilers, it all seems ok. The major 
bottleneck is the blit, as I suspected and not the lisp which is pretty 
damn fast. For now, I've attacked the problem the UFFI way, moving my 
frame buffer into alien land and 16 byte alinging it to get the extra 
speed from x86 iron.

However it occurs to me there is a much more LISP way to tackle what I'm 
doing. If I know the size of the window/screen I want at compile time, I 
can get SDL to list the possible pixel formats, and then pick the 
optimal one, which would amount to compiling in the "right" set of 
frambuffer/pixel operation macros. It'd get round the problems of the 
method I'm using now, which is to pretend I am addressing a 32bit ARGB 
buffer no matter what and deal with the pixel format problems at blit time.



-- 
Cyborg Animation Programmer
http://yagc.blogspot.com
http://badbyteblues.blogspot.com
From: Frode Vatvedt Fjeld
Subject: Re: Slightly less newbie lisp questions..
Date: 
Message-ID: <2h1xe1ws0e.fsf@vserver.cs.uit.no>
John Connors <·····@yagc.dot.ndo.dot.co.dot.uk> writes:

> 1> I'm in the REPL. I know I need to use a symbol "foo", but I don't 
> know what package it's in (and I won't neccessarily have it loaded)
> how do I find out where it lives? This is SBCL or CMUCL (and SLIME).

Try (cl:apropos "foo").

-- 
Frode Vatvedt Fjeld
From: Szymon
Subject: Re: Slightly less newbie lisp questions..
Date: 
Message-ID: <85d5xl34vq.fsf@eva.rplacd.net>
John Connors <·····@yagc.dot.ndo.dot.co.dot.uk> writes:

Hi.

> 1> I'm in the REPL. I know I need to use a symbol "foo", but I don't
> know what package it's in (and I won't neccessarily have it loaded)
> how do I find out where it lives? This is SBCL or CMUCL (and SLIME).

;; simple utility: 'locate-symbol'
;; btw, I'm not experienced lispnik --
;; be cautious.

(let ((kw-list '(:inherited :external :internal))
      (ht #.(make-hash-table :test #'eq :size 3)))
  (defun locate-symbol (symbol)
    (mapc (lambda (kw) (setf (gethash kw ht) nil)) kw-list)
    (mapc (lambda (package)
	    (let ((temp (nth-value 1 (find-symbol (symbol-name symbol) package))))
	      (if temp (push (package-name package) (gethash temp ht)))))
	  (list-all-packages))
    (values-list
     (pairlis kw-list (mapcar (lambda (kw) (gethash kw ht)) kw-list)))))

CL-USER> (locate-symbol 'foo)

   (:INTERNAL "SB-KERNEL" "SB-C" "COMMON-LISP-USER" "SWANK-COMPLETION-TEST")

   (:EXTERNAL "KEYWORD")

   (:INHERITED)


CL-USER> (locate-symbol 'car)

   (:INTERNAL)

   (:EXTERNAL "COMMON-LISP")

   (:INHERITED "SB-EXT" "SB-KERNEL" "SB-DI" "SB-VM" "SB-UNIX" ........ )

Regards, Szymon.
From: Szymon
Subject: Re: Slightly less newbie lisp questions..
Date: 
Message-ID: <858y8934aj.fsf@eva.rplacd.net>
I should use plist instead of hash table...
From: Pascal Bourguignon
Subject: Re: Slightly less newbie lisp questions..
Date: 
Message-ID: <871xe0yi43.fsf@thalassa.informatimago.com>
Szymon <············@o2.pl> writes:

> John Connors <·····@yagc.dot.ndo.dot.co.dot.uk> writes:
> 
> Hi.
> 
> > 1> I'm in the REPL. I know I need to use a symbol "foo", but I don't
> > know what package it's in (and I won't neccessarily have it loaded)
> > how do I find out where it lives? This is SBCL or CMUCL (and SLIME).
> 
> ;; simple utility: 'locate-symbol'
> ;; btw, I'm not experienced lispnik --
> ;; be cautious.
> 
> (let ((kw-list '(:inherited :external :internal))
>       (ht #.(make-hash-table :test #'eq :size 3)))
>   (defun locate-symbol (symbol)
>     (mapc (lambda (kw) (setf (gethash kw ht) nil)) kw-list)
>     (mapc (lambda (package)
> 	    (let ((temp (nth-value 1 (find-symbol (symbol-name symbol) package))))
> 	      (if temp (push (package-name package) (gethash temp ht)))))
> 	  (list-all-packages))
>     (values-list
>      (pairlis kw-list (mapcar (lambda (kw) (gethash kw ht)) kw-list)))))

There is no reason why you should keep garbage after leaving
locate-symbol. 


(defun locate-symbol (symbol)
  (let ((ht (make-hash-table :test (function eq) :size 3)))
    ;; you could even add:
    (declare (dynamic-extend ht))
    (mapc (lambda (package)
            (let ((temp (nth-value 1 (find-symbol
                                      (symbol-name symbol) package))))
              (if temp (push (package-name package) (gethash temp ht)))))
          (list-all-packages))
    #-pjb
    (values-list ;; do you really want several values? 
     (mapcar (lambda (kw) (cons kw (gethash kw ht)))
            '(:inherited :external :internal)))
    #+pjb ;;I would write:
    (let ((result '())) 
      (maphash (lambda (k v) (push (cons k v) result)) ht) 
      result)
    ))



-- 
__Pascal Bourguignon__                     http://www.informatimago.com/
The world will now reboot; don't bother saving your artefacts.
From: Szymon
Subject: Re: Slightly less newbie lisp questions..
Date: 
Message-ID: <851xe09219.fsf@eva.rplacd.net>
Pascal Bourguignon <····@mouse-potato.com> writes:

> [.....]

> There is no reason why you should keep garbage after leaving
> locate-symbol. 

Thanks for the advices.

> (defun locate-symbol (symbol)
>   (let ((ht (make-hash-table :test (function eq) :size 3)))

>     ;; you could even add:
>     (declare (dynamic-extend ht))

Interesting.

>     (mapc (lambda (package)
>             (let ((temp (nth-value 1 (find-symbol
>                                       (symbol-name symbol) package))))
>               (if temp (push (package-name package) (gethash temp ht)))))
>           (list-all-packages))
>     #-pjb
>     (values-list ;; do you really want several values? 

No :) but they are printed more nicely in REPL...

>      (mapcar (lambda (kw) (cons kw (gethash kw ht)))
>             '(:inherited :external :internal)))
>     #+pjb ;;I would write:
>     (let ((result '())) 
>       (maphash (lambda (k v) (push (cons k v) result)) ht) 
>       result)
>     ))

I rewrite locate-symbol -- withount hash tables:

(defun locate-symbol (symbol &aux result)
  (dolist (package (list-all-packages) result)
    (let ((temp (nth-value 1 (find-symbol (symbol-name symbol) package))))
      (when temp (push (package-name package) (getf result temp))))))

Btw, it returns plist instead of values.

Regards, Szymon.