I am having trouble using the profiling code in Practical Common Lisp.
The book provides half a page of code, a short explanatory paragraph,
and one incomplete usage example. I suspect that I am supposed to
understand how to use it by reading the code (here's a trivial but
useful profiling tool), but I haven't been able to figure it out. What
follows is my first usage attempt (later attempts were random syntax
changes that didn't help), the resulting error, followed by a
cut-and-paste of the book section.
How I thought it would work:
(with-timing "fun1" (print 0))
(show-timing-data)
--> 100% fun1: 5 ticks over 1 calls for 5 per.
How it actually worked:
(with-timing "fun1" (print 0))
--> EVAL: variable START has no value
[Condition of type SYSTEM::SIMPLE-UNBOUND-VARIABLE]
Snippet from book:
(defparameter *timing-data* ())
(defmacro with-timing (label &body body)
(with-gensyms (start)
`(let ((,start (get-internal-run-time)))
(unwind-protect (progn ,@body)
(push (list ',label ,start (get-internal-run-time))
*timing-data*)))))
(defun clear-timing-data ()
(setf *timing-data* ()))
(defun show-timing-data ()
(loop for (label time count time-per %-of-total) in
(compile-timing-data) do
(format t "~3d% ~a: ~d ticks over ~d calls for ~d per.~%"
%-of-total label time count time-per)))
(defun compile-timing-data ()
(loop with timing-table = (make-hash-table)
with count-table = (make-hash-table)
for (label start end) in *timing-data*
for time = (- end start)
summing time into total
do
(incf (gethash label timing-table 0) time)
(incf (gethash label count-table 0))
finally
(return
(sort
(loop for label being the hash-keys in timing-table collect
(let ((time (gethash label timing-table))
(count (gethash label count-table)))
(list label time count (round (/ time count)) (round
(* 100 (/ time total))))))
#'> :key #'fifth))))
This profiler lets you wrap a with-timing around any form; each time
the form is executed, the time it starts and the time it ends are
recorded, associating with a label you provide. The function
show-timing-data dumps out a table showing how much time was spent in
different labeled sections of code like this:
CL-USER> (show-timing-data)
84% BAR: 650 ticks over 2 calls for 325 per.
16% FOO: 120 ticks over 5 calls for 24 per.
NIL
"Eric Lavigne" <············@gmail.com> wrote in message
·····························@o13g2000cwo.googlegroups.com...
>I am having trouble using the profiling code in Practical Common Lisp.
> The book provides half a page of code, a short explanatory paragraph,
> and one incomplete usage example. I suspect that I am supposed to
> understand how to use it by reading the code (here's a trivial but
> useful profiling tool), but I haven't been able to figure it out. What
> follows is my first usage attempt (later attempts were random syntax
> changes that didn't help), the resulting error, followed by a
> cut-and-paste of the book section.
>
> How I thought it would work:
>
> (with-timing "fun1" (print 0))
> (show-timing-data)
> --> 100% fun1: 5 ticks over 1 calls for 5 per.
>
> How it actually worked:
>
> (with-timing "fun1" (print 0))
> --> EVAL: variable START has no value
> [Condition of type SYSTEM::SIMPLE-UNBOUND-VARIABLE]
>
It looks like you didn't enter the definition for WITH-GENSYMS. It's
shown earlier in the book.
--
Geoff
>It looks like you didn't enter the definition for
>WITH-GENSYMS. It's shown earlier in the book.
Yes I missed that. Thanks.
I pasted the with-gensyms definition into my REPL (what do the letters
stand for?) and got a new error:
(defmacro with-gensyms ((&rest names) &body body)
`(let ,(loop for n in names collect `(,n (gensym)))
,@body))
DEFUN/DEFMACRO(WITH-GENSYMS):
#<PACKAGE EXT> is locked
[Condition of type SYSTEM::SIMPLE-PACKAGE-ERROR]
My solution, since with-gensyms is a fairly simple macro that was only
needed once, was to perform the macro expansion manually and remove the
profiling code's dependency on with-gensyms. This works fine, but I am
still curious why my with-gensyms definition was rejected.
Eric Lavigne <············@gmail.com> wrote:
> REPL (what do the letters stand for?)
http://www.gigamonkeys.com/book/lather-rinse-repeat-a-tour-of-the-repl.html
"That endless cycle of reading, evaluating, and printing is why it's
called the read-eval-print loop, or REPL for short. It's also referred
to as the top-level, the top-level listener, or the Lisp listener"
"Eric Lavigne" <············@gmail.com> writes:
> >It looks like you didn't enter the definition for
> >WITH-GENSYMS. It's shown earlier in the book.
>
> Yes I missed that. Thanks.
>
> I pasted the with-gensyms definition into my REPL (what do the letters
> stand for?) and got a new error:
Read Evaluate Print Loop
> (defmacro with-gensyms ((&rest names) &body body)
> `(let ,(loop for n in names collect `(,n (gensym)))
> ,@body))
>
> DEFUN/DEFMACRO(WITH-GENSYMS):
> #<PACKAGE EXT> is locked
> [Condition of type SYSTEM::SIMPLE-PACKAGE-ERROR]
Try doing (in-package :cl-user)
> My solution, since with-gensyms is a fairly simple macro that was only
> needed once, was to perform the macro expansion manually and remove the
> profiling code's dependency on with-gensyms. This works fine, but I am
> still curious why my with-gensyms definition was rejected.
--
An ideal world is left as an excercise to the reader.
--- Paul Graham, On Lisp 8.1
No excuses. No apologies. Just do it.
--- Erik Naggum
"Eric Lavigne" <············@gmail.com> writes:
>>It looks like you didn't enter the definition for
>>WITH-GENSYMS. It's shown earlier in the book.
>
> Yes I missed that. Thanks.
>
> I pasted the with-gensyms definition into my REPL (what do the letters
> stand for?) and got a new error:
>
> (defmacro with-gensyms ((&rest names) &body body)
> `(let ,(loop for n in names collect `(,n (gensym)))
> ,@body))
>
> DEFUN/DEFMACRO(WITH-GENSYMS):
> #<PACKAGE EXT> is locked
> [Condition of type SYSTEM::SIMPLE-PACKAGE-ERROR]
>
> My solution, since with-gensyms is a fairly simple macro that was only
> needed once, was to perform the macro expansion manually and remove the
> profiling code's dependency on with-gensyms. This works fine, but I am
> still curious why my with-gensyms definition was rejected.
I'm guessing you're using CLISP, in which there's already a
WITH-GENSYMS macro defined in the EXT package which is used by
CL-USER. The Lispbox distros I provide at
<http://www.gigamonkeys.com/book/lispbox/> clean out CL-USER so it
doesn't use any implementation-specific packages (which would fix this
problem). Or you can define your own package that just uses
COMMON-LISP (a.k.a. CL) and you won't have these problems.
-Peter
--
Peter Seibel ·····@gigamonkeys.com
Lisp is the red pill. -- John Fraser, comp.lang.lisp
>I'm guessing you're using CLISP, in which there's already
>a WITH-GENSYMS macro defined in the EXT package which
>is used by CL-USER.
Yes, I'm using clisp.
>The Lispbox distros I provide at
><http://www.gigamonkeys.com/book/lispbox/> clean out
>CL-USER so it doesn't use any implementation-specific
>packages (which would fix this problem).
There don't seem to be any windows distros on that site, though.
Windows support is actually my main reason for using clisp, and I am
thinking about installing linux just for access to cmucl.
>Or you can define your own package that just uses
>COMMON-LISP (a.k.a. CL) and you won't have these problems.
I'll do that. Thanks for the advice.
Really enjoying your book. I'm only about a third of the way through so
far (skipped ahead to profiling section because my project is extremely
slow, runs for hours at a time). I'm looking forward to the web
programming practicals.
On 25 Apr 2005 17:00:45 -0700, "Eric Lavigne" <············@gmail.com> wrote:
> Really enjoying your book. I'm only about a third of the way through
> so far (skipped ahead to profiling section because my project is
> extremely slow, runs for hours at a time).
If you're able to post the code here (assuming it's not too long) I'm
sure a lot of people will try to help you to optimize it.
The initial standard question in this case is whether maybe you've
forgotten to compile your code but I guess you aren't /that/ newbie
anymore... :)
Cheers,
Edi.
--
Lisp is not dead, it just smells funny.
Real email: (replace (subseq ·········@agharta.de" 5) "edi")
>If you're able to post the code here (assuming it's not too long)
>I'm sure a lot of people will try to help you to optimize it.
It is on the long side... I have posted below what I think is the
bottleneck code. The rest of it is available under "Diffusion Code 3"
on my website: http://plaza.ufl.edu/lavigne/
I would appreciate any help on this. It is getting very frustrating.
>The initial standard question in this case is whether maybe
>you've forgotten to compile your code but I guess you aren't
>/that/ newbie anymore... :)
Well, forgotten isn't quite the right word... I never saw a very big
performance change from compiling in Lisp, which surprised me quite a
bit the first time I tried it. In this case, compiling improved
performance by around 20%. I need much better than that. I'm willing to
wait for a few hours to get an answer, but I'm not willing to wait a
few hours after every time I make a change to the code. I can't make
any more progress on this code until it is at least several times
faster.
(defun gauss-seidel (source flux dx)
(let ((flux^ nil)
(done t))
(do () (nil)
(incf *iteration-count-inner*)
(setf flux^ (copy-flux flux))
(with-timing "phi-calc"
(do ((i 0 (1+ i)))
((>= i (array-dimension flux 0)))
(do ((j 0 (1+ j)))
((>= j (array-dimension flux 1)))
(do ((k 0 (1+ k)))
((>= k (array-dimension flux 2)))
(setf (aref flux i j k)
(phi-calc
(aref source i j k)
dx
(node-material (aref flux i j k))
(if (eql (1+ i) (array-dimension flux 0))
(make-node :flux 0)
(aref flux (1+ i) j k))
(if (eql i 0)
(make-node :flux 0)
(aref flux (1- i) j k))
(if (eql (1+ j) (array-dimension flux 1))
(make-node :flux 0)
(aref flux i (1+ j) k))
(if (eql j 0)
(make-node :flux 0)
(aref flux i (1- j) k))
(if (eql (1+ k) (array-dimension flux 2))
(make-node :flux 0)
(aref flux i j (1+ k)))
(if (eql k 0)
(make-node :flux 0)
(aref flux i j (1- k)))))))))
(setf done t)
(with-timing "exit-cond"
(do ((i 0 (1+ i)))
((>= i (array-dimension flux 0)))
(do ((j 0 (1+ j)))
((>= j (array-dimension flux 0)))
(do ((k 0 (1+ k)))
((>= k (array-dimension flux 0)))
(when (> (node-flux (aref flux^ i j k)) 1e-3)
(when (< (* (node-flux (aref flux^ i j k))
*flux-tolerance*)
(abs (- (node-flux (aref flux i j k))
(node-flux (aref flux^ i j k)))))
(setf done nil)))))))
(when done
(return-from gauss-seidel flux)))))
;; calculates flux at one node
(defun phi-calc (source dx material
phi-x1 phi-x2
phi-y1 phi-y2
phi-z1 phi-z2)
; n is the number of no-return boundaries
(let (x-contrib y-contrib z-contrib (n 0)
(D (material-D material))
(sigma-a (material-sigma-a material))
(x1 (node-flux phi-x1))
(x2 (node-flux phi-x2))
(y1 (node-flux phi-y1))
(y2 (node-flux phi-y2))
(z1 (node-flux phi-z1))
(z2 (node-flux phi-z2)))
; if flux very small on one side, treat that side
; as no-return boundary
(if (< (min x1 x2)
(* (max x1 x2)
(- (/ 4 (+ 2 (/ dx D))) 1)))
(progn
(setq x-contrib (* 2 (max x1 x2)))
(setf n (1+ n)))
(setq x-contrib (+ x1 x2)))
(if (< (min y1 y2)
(* (max y1 y2)
(- (/ 4 (+ 2 (/ dx D))) 1)))
(progn
(setq y-contrib (* 2 (max y1 y2)))
(setf n (1+ n)))
(setq y-contrib (+ y1 y2)))
(if (< (min z1 z2)
(* (max z1 z2)
(- (/ 4 (+ 2 (/ dx D))) 1)))
(progn
(setq z-contrib (* 2 (max z1 z2)))
(setf n (1+ n)))
(setq z-contrib (+ z1 z2)))
(make-node
:material material
:flux
(/ (+ x-contrib y-contrib z-contrib
(* dx dx source (/ D)))
(+ (* dx dx (/ D) sigma-a)
6 (* (sqrt n) dx (/ D)))))))
To run the code, these are the lines I use. It takes less than a
second, but this piece of bottleneck code gets called many times in the
full program.
(setf flux (make-array '(9 9 9))) t ;; t is used to hide long output
(setf my-material (make-material :d .9 :v-sigma-f .07 :sigma-a .066))
(dotimes (i (array-dimension flux 0))
(dotimes (j (array-dimension flux 1))
(dotimes (k (array-dimension flux 2))
(setf (aref flux i j k)
(make-node :material my-material :flux 1)))))
(setf source (set-source flux 1.5)) t
;; gauss-seidel also has long output,
;; so put (show-timing-data) on same line)
(time (gauss-seidel source flux (/ 40 (1- (array-dimension flux 0)))))
(show-timing-data) (clear-timing-data)
"Eric Lavigne" <············@gmail.com> wrote in message
·····························@z14g2000cwz.googlegroups.com...
> >If you're able to post the code here (assuming it's not too long)
>>I'm sure a lot of people will try to help you to optimize it.
> It is on the long side... I have posted below what I think is the
> bottleneck code. The rest of it is available under "Diffusion Code 3"
> on my website: http://plaza.ufl.edu/lavigne/
> I would appreciate any help on this. It is getting very frustrating.
>
>
> (defun gauss-seidel (source flux dx)
> (let ((flux^ nil)
> (done t))
> (do () (nil)
> (incf *iteration-count-inner*)
> (setf flux^ (copy-flux flux))
> (with-timing "phi-calc"
> (do ((i 0 (1+ i)))
> ((>= i (array-dimension flux 0)))
> (do ((j 0 (1+ j)))
> ((>= j (array-dimension flux 1)))
> (do ((k 0 (1+ k)))
> ((>= k (array-dimension flux 2)))
> (setf (aref flux i j k)
> (phi-calc
> (aref source i j k)
> dx
> (node-material (aref flux i j k))
> (if (eql (1+ i) (array-dimension flux 0))
> (make-node :flux 0)
> (aref flux (1+ i) j k))
> (if (eql i 0)
> (make-node :flux 0)
> (aref flux (1- i) j k))
> (if (eql (1+ j) (array-dimension flux 1))
> (make-node :flux 0)
> (aref flux i (1+ j) k))
> (if (eql j 0)
> (make-node :flux 0)
> (aref flux i (1- j) k))
> (if (eql (1+ k) (array-dimension flux 2))
> (make-node :flux 0)
> (aref flux i j (1+ k)))
> (if (eql k 0)
> (make-node :flux 0)
> (aref flux i j (1- k)))))))))
Well, I can see a couple of standard mods for any language, but
whether and how much speed you get out of them...more testing I'm afraid.
- move calls and calculations out of the inner loop in this case
i.e. in the outer let (x-dim (array-dimension flux 0)) ...
and then use x-dim
- the ifs can be gotten rid of if you use a larger array with the
:flux 0 nodes on the outside
i.e 00000
xxx 0xxx0 (do ((i 1 (+ 1 i)))
xxx becomes 0xxx0 ((>= i (1- (array-dimension flux 0))))
xxx 0xxx0 (...
00000
now you don't need to check boundaries.
- declare type-information for slots, etc. Helps the compiler avoid
having to determine type at run-time.
- cache previous calculations you've done, if you can keep a partial
calculation you can use the results in this case, the previous plane
and previous line on the current plane perhaps. Part of the trick here
is to use and replace with the new as you go.
xxx x-processed
xc. c-current
... .-not processed
-reduce dimensions and calculate the offset yourself
(dotimes (x xdim)
(setf partial (* x ydim zdim))
(dotimes (y ydim)
(incf partial (* y zdim))
(dotimes (z zdim)
(aref flux (+ partial z))
-reuse structs, setf the slots rather than creating a new one, cuts down
on memory usage, less gc, no calls for allocation.
--
Geoff
Eric Lavigne wrote:
> It is on the long side... I have posted below what I think is the
> bottleneck code. The rest of it is available under "Diffusion Code 3"
> on my website: http://plaza.ufl.edu/lavigne/
...
> Well, forgotten isn't quite the right word... I never saw a very big
> performance change from compiling in Lisp, which surprised me quite a
> bit the first time I tried it. In this case, compiling improved
> performance by around 20%. I need much better than that. I'm willing to
> wait for a few hours to get an answer, but I'm not willing to wait a
> few hours after every time I make a change to the code. I can't make
> any more progress on this code until it is at least several times
> faster.
You might try using an implementation other than CLISP.
CLISP is excellent for some purposes, but heavy numerical
code generally isn't one of them. If you're on Windows
then the Allegro and LispWorks demos might suit your needs;
if on Unix, CMUCL or SBCL is probably your best bet.
You'll likely get a substantial improvement in speed
just from switching implementation; further improvements
may then be possible from (e.g.) adding declarations,
which generally don't help much in CLISP.
--
Gareth McCaughan
.sig under construc
Gareth McCaughan <················@pobox.com> writes:
> You might try using an implementation other than CLISP.
> CLISP is excellent for some purposes, but heavy numerical
> code generally isn't one of them.
Unless it's bignums or bigfloats :-)
--
__Pascal Bourguignon__ http://www.informatimago.com/
Wanna go outside.
Oh, no! Help! I got outside!
Let me back inside!
"Eric Lavigne" <············@gmail.com> writes:
>>I'm guessing you're using CLISP, in which there's already
>>a WITH-GENSYMS macro defined in the EXT package which
>>is used by CL-USER.
> Yes, I'm using clisp.
>
>>The Lispbox distros I provide at
>><http://www.gigamonkeys.com/book/lispbox/> clean out
>>CL-USER so it doesn't use any implementation-specific
>>packages (which would fix this problem).
> There don't seem to be any windows distros on that site, though.
> Windows support is actually my main reason for using clisp, and I am
> thinking about installing linux just for access to cmucl.
Yes. Windows versions are coming as soon as I can get some time on a
Windows box to make them. Sorry about that.
-Peter
--
Peter Seibel ·····@gigamonkeys.com
Lisp is the red pill. -- John Fraser, comp.lang.lisp
"Eric Lavigne" <············@gmail.com> writes:
>>It looks like you didn't enter the definition for
>>WITH-GENSYMS. It's shown earlier in the book.
>
> Yes I missed that. Thanks.
>
> I pasted the with-gensyms definition into my REPL (what do the letters
> stand for?) and got a new error:
>
> (defmacro with-gensyms ((&rest names) &body body)
> `(let ,(loop for n in names collect `(,n (gensym)))
> ,@body))
>
> DEFUN/DEFMACRO(WITH-GENSYMS):
> #<PACKAGE EXT> is locked
> [Condition of type SYSTEM::SIMPLE-PACKAGE-ERROR]
>
> My solution, since with-gensyms is a fairly simple macro that was only
> needed once, was to perform the macro expansion manually and remove the
> profiling code's dependency on with-gensyms. This works fine, but I am
> still curious why my with-gensyms definition was rejected.
The implementations are allowed to polute the COMMON-LISP-USER package
with any implementation specific package. For example, clisp imports
the EXT package amongst others.
To avoid this kind of problem, and to establish a uniform environment
in all the Common Lisp implementations I use, I put the following
expressions in ~/.common.lisp:
(IN-PACKAGE "COMMON-LISP-USER")
(DEFPACKAGE "COM.INFORMATIMAGO.PJB" (:USE "COMMON-LISP"))
;; Clean the packages imported into COMMON-LISP-USER:
(MAPC (LAMBDA (COM.INFORMATIMAGO.PJB::USED)
(UNUSE-PACKAGE COM.INFORMATIMAGO.PJB::USED "COMMON-LISP-USER"))
(REMOVE (FIND-PACKAGE "COMMON-LISP")
(COPY-SEQ (PACKAGE-USE-LIST "COMMON-LISP-USER"))))
;; and of course, a lot more of personal stuff in COM.INFORMATIMAGO.PJB...
and the following in the various rc files, ~/.clisprc.lisp,
~/.sbclrc.lisp, ~/.cmucl-init.lisp, ~/openmcl-init.lisp, etc:
(LOAD (MERGE-PATHNAMES
(MAKE-PATHNAME :NAME ".common" :TYPE "lisp") (USER-HOMEDIR-PATHNAME)))
--
__Pascal Bourguignon__ http://www.informatimago.com/
-----BEGIN GEEK CODE BLOCK-----
Version: 3.12
GCS d? s++:++ a+ C+++ UL++++ P--- L+++ E+++ W++ N+++ o-- K- w---
O- M++ V PS PE++ Y++ PGP t+ 5+ X++ R !tv b+++ DI++++ D++
G e+++ h+ r-- z?
------END GEEK CODE BLOCK------