From: R. Matthew Emerson
Subject: defun within let
Date: 
Message-ID: <6v6l3c$d8j$1@news-2.news.gte.net>
i find myself writing code like this:

(let ((n 0))
  (defun next ()
    (incf n))
  (defun reset ()
    (setf n 0)))

the outer let might create a big lookup table used only by the
internal defuns, or just might be some state variables that the
internal defuns share.

now, i've certainly not read a great deal of lisp code in my time, but
it seems that i rarely see anything like this, except in books.  is
there some reason for this?

-matt

From: Kent M Pitman
Subject: Re: defun within let
Date: 
Message-ID: <sfwsoh59ehr.fsf@world.std.com>
···@nightfly.apk.net (R. Matthew Emerson) writes:

> i find myself writing code like this:
> 
> (let ((n 0))
>   (defun next ()
>     (incf n))
>   (defun reset ()
>     (setf n 0)))
>
> now, i've certainly not read a great deal of lisp code in my time, but
> it seems that i rarely see anything like this, except in books.  is
> there some reason for this?

Three reasons come immediately to mind:

 * This used to not compile well and many old-time lisp programmers
   are not in the habit of doing it.  And in some implementations it
   might not still compile well, so some programmers nervously avoid
   it as what they perceive as a gratuitous risk if they port their
   code.

 * Some people (myself included) think this is symptomatic of a major
   split between Lisp and Scheme.  Lisp is capable of doing what you
   describe and you should feel free to do it if you like that 
   style, but many Lisp programmers prefer not to because they prefer
   to have named structures for easier debugging.  What you suggest
   is done really a lot in Scheme, and long ago when I used to program
   in Scheme it was one of those things that routinely drove me nuts.
   The issue is this:

   Since there are no closure accessors, it's hard to debug the above
   code (e.g., to extend to add another operation, to view the
   counter, or to reset the local variable to value not planned for)
   without some powerful tool capable of violating a closure's internal
   data structures in ways that no tool really ought to have the right
   to do.

   The more conventional CL style is:

   (defvar *counter* 0)
   (defun next () (incf *counter*))
   (defun reset () (setf *counter* 0))

   The effective only difference between these two approaches is that
   the counter is named, and many find that important.  Before you
   think this is silly, think about the fact that you could have written:
    (let ((n 0))
      (labels ((next () (incf n))
	       (reset () (setf n 0)))
        ... code that uses next and reset ...))
   and really the only reason you don't is that you find it useful to
   sometimes name the functions NEXT and RESET from outside that 
   context (e.g., in another file or in interactive debugging).
   So you're just drawing the line at a different place, and the
   language isn't stopping you.  Other people draw it differently,
   and for what have often been seen as good reasons.

 * There's also some messiness if you need to redefine/recompile functions
   such as the ones above.  They have to be recompiled as a pair.
   Consider this scenario:

    (a) You execute your original code above.
    (b) You do store #'next in the heap somewhere, e.g., by doing
	(defvar *saved* #'next)
    (c) You later recompile the original code you gave.

   Voila'.  You now have a bogus version of next stored in *saved*.
   And while it's true that if you had changed the definition of next
   incompatibly you'd STILL have a bogus version, at least a recompile
   without changing wouldn't hurt if you used my revised definitions.
   
   This is, in a sense, just a variant of the inability to name
   things, but it basically means that incremental development tools
   are as crippled as you when trying to keep everything up to date
   if you use this coding style.

Hope that helps.  I don't mean to suggest you have to change your
coding style.  You should do what makes you the most comfortable.
But I do think the people who use other coding styles are not crazy
for it...

Btw, some might argue the use of specials here is overkill.  The
reasonable compromise would be to use global lexicals, which Scheme
has and Common Lisp doesn't. e.g.,

 (define n 0)
 (define (next) (set! n (+ n 1)))
 (define (reset) (set! n 0))

There was an abortive attempt to add global lexicals to CL.  IMO, it's
a pity it didn't succeed.  Fortunately, we later added DEFINE-SYMBOL-MACRO,
and if you're clever with that and LOAD-TIME-VALUE, you can now add
the facility yourself in user code and it can even be quite efficient.
From: Duane Rettig
Subject: Re: defun within let
Date: 
Message-ID: <4vhlybhn2.fsf@beta.franz.com>
Kent M Pitman <······@world.std.com> writes:

[ ... Well-considered treatise on tradeoffs pro and con local lexical
  closure variables vs globals vs ... ]

>    Since there are no closure accessors, it's hard to debug the above
>    code (e.g., to extend to add another operation, to view the
>    counter, or to reset the local variable to value not planned for)
>    without some powerful tool capable of violating a closure's internal
>    data structures in ways that no tool really ought to have the right
>    to do.

I agree, but I think that that tool is/should-be the debugger itself.
It has precedents for circumventing the normal operating structures
and rules, and should be powerful enough to access these variables
with not much effort.  For example, consider the following form in
a file:

(let ((n 0))
  (declare (optimize (debug 2)))
  (defun next ()
    (break "next1 break")
    (incf n))
  (defun reset ()
    (setf n 0)))

In Allegro CL 5.0:

USER(1): (setq *load-local-names-info* t)
T
USER(2): :cl xxx
;;; Compiling file xxx.cl
;;; Writing fasl file xxx.fasl
;;; Fasl write complete
; Fast loading /force/duane/acl51/src/xxx.fasl
USER(3): (next)
Break: next1 break

Restart actions (select using :continue):
 0: return from break.
 1: Return to Top Level (an "abort" restart)
[1c] USER(4): :loc
Compiled lexical environment:
0(LOCAL): SYSTEM::.LAMBDA-LEXICAL-ENV.: #(0)
1(LOCAL): N: 0
[1c] USER(5): :loc n
0
[1c] USER(6): :set-loc n 10
[1c] USER(7): :cont 0
11
USER(8): 


-- 
Duane Rettig          Franz Inc.            http://www.franz.com/ (www)
1995 University Ave Suite 275  Berkeley, CA 94704
Phone: (510) 548-3600; FAX: (510) 548-8253   ·····@Franz.COM (internet)
From: Kent M Pitman
Subject: Re: defun within let
Date: 
Message-ID: <sfwr9wlaje1.fsf@world.std.com>
Duane Rettig <·····@franz.com> writes:

> Kent M Pitman <······@world.std.com> writes:
> 
> >    Since there are no closure accessors, it's hard to debug the above
> >    code (e.g., to extend to add another operation, to view the
> >    counter, or to reset the local variable to value not planned for)
> >    without some powerful tool capable of violating a closure's internal
> >    data structures in ways that no tool really ought to have the right
> >    to do.
> 
> I agree, but I think that that tool is/should-be the debugger itself.
> It has precedents for circumventing the normal operating structures
> and rules, and should be powerful enough to access these variables
> with not much effort.  For example, consider the following form in
> a file:

The problem is not that.  You can do a similar thing in Harlequin; see
transcript below as "proof".  (The minor difference is that we just use
the variable names themselves, and normal lisp forms, not :loc and 
:set-loc, but it's the same idea.)

Personally, I think depending on the debugger is just wrong.  You are
welcome to disagree, but here are my reasons for what they are worth:

 - A special variable offers program adaptability even in highly
   optimized code because the actual language semantics requires
   that adaptability.

 - A special variable can be program-modified in a patch file.

 - A compiler doing data flow and/or dead-code analysis might 
   determine a side-effect isn't going to happen and optimize away
   the variable.    This might be inconvenient for debugging.

 - Some environments in some implementations have no debugger, or
   may have an insufficient  debugger; a conforming implementation
   say simply type "core dumped" rather than implement a debugger.
   I figure the market will sort that out; nevertheless, there is
   value in not depending on the debugger for all your help.

Items 2 and 3 of the above are things you usually care to affect AFTER
you write your code--it's not part of the original design.  So you either
plan in that kind of flexibility or risk getting screwed for having
failed to.  The debugger can be a help in cases where you failed to
do it yourself--no doubt about that.  But, IMO, nothing about the
debugger is linguistic; and, perhaps because of my background, I see
programming as a linguistic activity--saying what I want to be true.
Requesting dynamic programming flexibility is properly said by naming
things in publicly accessible ways, not by naming things privately
and hoping for a tool that will violate that privacy.  Again, it's
not that there shouldn't be that tool--it's just that I don't rely
on it as a first line of defense.

"Show and tell" about LW debugging follows...

- - - - -

This demo is from LW 3.2.2 on Unix; we have newer Unix releases but this 
was what was handy and we've had this feature for a while, so...

Given a file containing:
  (let ((n 0))
    (defun foo () (break "Foo") n)
    (defun bar () n)
    (defun baz () (incf n)))
here's what it looks like in ours:

CL-USER 19 > (load (compile-file "/u/kmp/temp.lisp"))
;;; Compiling file /u/kmp/temp.lisp ...
;;; Safety = 3, Speed = 1, Space = 1, Float = 1, Interruptable = 0
;;; Compilation speed = 1, Debug = 2, Fixnum safety = 3, GC safety = 3
;;; Source level debugging is on 
;;; Source file recording is  on 
;;; Cross referencing is on
; (TOP-LEVEL-FORM 1)
; (TOP-LEVEL-FORM 2)
; Loading fasl file /nfs/holmes/u/ldisk/kmp/temp.wfasl
#P"/nfs/holmes/u/ldisk/kmp/temp.wfasl"

CL-USER 20 > (bar)
0

CL-USER 21 > (foo)

Foo
  1 (continue) Return from break.
  2 (abort) Return to level 0.
  3 return to top loop level 0.

Type :c followed by a number to proceed or type :? for other options

CL-USER 22 : 1 > :n
Call to (SUBFUNCTION FOO (TOP-LEVEL-FORM 1))

CL-USER 23 : 1 > :v
Call to (SUBFUNCTION FOO (TOP-LEVEL-FORM 1))
   N : 0

CL-USER 24 : 1 > (setq n 3)
3

CL-USER 25 : 1 > :c
3

CL-USER 26 > (bar)
3

CL-USER 27 > (baz)
4

CL-USER 28 > (bar)
4
From: Erik Naggum
Subject: Re: defun within let
Date: 
Message-ID: <3116498427831232@naggum.no>
* ···@nightfly.apk.net (R. Matthew Emerson)
| now, i've certainly not read a great deal of lisp code in my time, but it
| seems that i rarely see anything like this, except in books.  is there
| some reason for this?

  I regard this as an optimization technique, not as normal coding style.
  accessing a special variable is generally more expensive than accessing a
  closure variable and this may matter in some circumstances.  sometimes,
  you have to perform a number of sanity checks on global variables because
  they may get nuked by the user or set to weird values.  in such cases,
  you can save yourself a lot of trouble by inhibiting direct access to the
  variable and performing the sanity checks in the setter function, knowing
  that there are no other ways to set the variable.

  normally, I regard this kind of "information hiding" to be anathema to a
  good software development environments because recompiling the closure
  or re-loading the file they are contained in irrevocably destroys the
  encapsulated information, so it should be employed only when you know
  that the closures are loaded only once, and I prefer that to be before
  dump time for the Lisp image.

  when implementing a new fully table-driven approach for international
  character sets in Allegro CL, I found, after heavy profiling, that so
  much time was wasted in accessors into the character set definition
  objects and special variables that it payed well to make a giant closure
  with all the functions accessing pre-accessed slots from the objects as
  closure variables, set by the sole setter function for the character set.
  since this is a fundamental framework that when changed (almost) requires
  re-dumping the Lisp image, anyway, I thought this was a reasonable cost
  to obtain the needed safety and speed.

  in other words, I think this approach is fine when the code is known not
  to be changed in the life-time of the application and no access to the
  encapsulated information will ever be needed, i.e., when the code is
  completely static.  this happens less often than one might think.

#:Erik
From: Barry Margolin
Subject: Re: defun within let
Date: 
Message-ID: <so7S1.10$8A3.303311@burlma1-snr1.gtei.net>
In article <················@naggum.no>, Erik Naggum  <····@naggum.no> wrote:
>  I regard this as an optimization technique, not as normal coding style.
>  accessing a special variable is generally more expensive than accessing a
>  closure variable and this may matter in some circumstances.

While I haven't done any benchmarking to support it, I wouldn't expect this
to be true in most implementations.  Accessing a special variable is
normally just an indirection through the value cell of the symbol (most CL
systems use shallow binding); there's no package searching done at access
time -- LOAD and READ intern the symbol so the function has a pointer to
the symbol itself.  Accessing a captured lexical variable requires an
indirection through the closure's environment object.  About the only
difference at access time is that a lexical variable can't be unbound, so
no check has to be done for this.

Note that calling a lexical closure is often more expensive than calling a
plain function, because an implicit environment argument may have to be
passed to the function.

Your point about application-level checking that may be necessary for
special variables is well taken, though.  This is also the justification
for information hiding in most object oriented languages -- by limiting
access to internal state, you can often prove invariants about that state.

-- 
Barry Margolin, ······@bbnplanet.com
GTE Internetworking, Powered by BBN, Burlington, MA
*** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups.
From: Kelly Murray
Subject: Re: defun within let
Date: 
Message-ID: <36195E98.1DFB6935@IntelliMarket.Com>
(let ((n 0))
  (defun next ()
    (incf n))
  (defun reset ()
    (setf n 0)))

In my opinion, this style of using and depending on global state
is not the best.  It is difficult to maintain, and generally will fail
miserably in a parallel processing / distributed memory environment.
I suggest an object approach:

(defclass counter ()
 ((value :initform 0 :accessor counter.value)))

(defmethod next ((c counter)) (incf (counter.value c)))
(defmethod reset ((c counter)) (setf (counter.value c) 0))

;; and then you pass around a counter object:

(let cnt = (make-instance 'counter)
 do
  ... (reset cnt) )
   

-Kelly Murray  ···@intellimarket.com
From: Ralf Muschall
Subject: Re: defun within let
Date: 
Message-ID: <6vea81$mun$1@news02.btx.dtag.de>
Kelly Murray wrote:
> (let ((n 0))
>   (defun next ()
>     (incf n))
>   (defun reset ()
>     (setf n 0)))

> In my opinion, this style of using and depending on global state
...
What do you mean by _global_ state here? The variable n is only shared
between the functions next and reset.
Of course, if two process(e|or)s call the two functions created 
by the same call to let at the same time, it will fail.

But the same will happen if the two processes try to step or to
reset the same instance cnt of the counter which you defined below.

The above version would have to be changed for multiprocessing 
to something like

(defun makecounter ()
  (let ((n 0))
     (cons
        #'(lambda () (incf n))
        #'(lambda () (setq n 0)))))

(setq cnt (makecounter))
((car cnt)) ; counts
((cdr cnt)) ; resets

which is what a class is syntactical sugar for.
(I did mostly scheme for the last months, so I may have too few or too
many "#'"s or "funcall"s here.)

Ralf
From: John Gillespie
Subject: Re: defun within let
Date: 
Message-ID: <m3soh0wilb.fsf@claudio.ucdavis.edu>
I use the let for another purpose, perhaps there is a more correct
way:

(let ((ar (make-array 100)))
  (defun foo ()
       ... do some calculations that require the array ar
           to hold intermediate values ...))

Presumably, if I put the creation of ar inside defun, an array
will be created each time the function is executed, which isn't
good if the function is called a lot, as it is in my simulations.
For this particular function, it makes no sense to make ar a global
variable as its contents are of only fleeting interest within the
context of foo.

Is there a way to create a "static" array inside a function?  I assume
something like 

(defun foo ()
  (let ((ar #1A(0 0 ...) ))
        ...))

would do this, but this approach is impractical for large arrays.

Any thoughts?

John
From: Kent M Pitman
Subject: Re: defun within let
Date: 
Message-ID: <sfwu31gs761.fsf@world.std.com>
John Gillespie <····@claudio.ucdavis.edu> writes:

> Is there a way to create a "static" array inside a function?  I assume
> something like 
> 
> (defun foo ()
>   (let ((ar #1A(0 0 ...) ))
>         ...))
> 
> would do this, but this approach is impractical for large arrays.

You don't want to use #A(...) or any literal constant because you are
then forbidden from modifying the constant.

You want LOAD-TIME-VALUE.  When its second arg, read-only-p, is NIL
(the default value, so it may be omitted), then arg1 is evaluated only
once in the runtime environment and the result is still allowed to be
modified:

 (defun foo ()
   (let ((arrray (load-time-value (make-array 100))))
     ...array...))

Normally, I'd still use

 (defun *foo-array* (make-array 100))
 (defun foo () ... *foo-array* ...))

but there are times when it's better not to.  One example might
be in the implementation of a hypothetical class system where
one might turn:

 (defmessage (frob :area) () (+ (slot x) (slot y)))

into

 (setf (class-message frob :area)
       #'(lambda () (+ (slot-ref frob x)
	               (slot-ref frob y))))

where, without working this in detail, you can imagine things
like (slot-ref class x) turning into

 (cdr (load-time-value (lookup-value-cell-cons :class 'frob
					       :slot 'x)))

so that at runtime the references to slots were as fast as
(car value-cell) and (setf (car value-cell) new-value).
Here again, you could make a special variable for each slot
but it's ill-advised since it would mean a lot of clutter
to the symbol namespace for little value.

A similar strategy, using define-symbol-macro and load-time-value can
be used to implement global lexicals and "bindable constants"
needed for ISLISP.

See CLHS for more details, and particularly the LOAD-TIME-EVAL
issue referenced at the bottom of the LOAD-TIME-VALUE dictionary
entry.
From: Hrvoje Niksic
Subject: Re: defun within let
Date: 
Message-ID: <kigg1d0ntit.fsf@jagor.srce.hr>
Kent M Pitman <······@world.std.com> writes:

>  (defun *foo-array* (make-array 100))
   ^^^^^^

Did you mean to say DEFVAR here?

-- 
Hrvoje Niksic <·······@srce.hr> | Student at FER Zagreb, Croatia
--------------------------------+--------------------------------
Ask not for whom the <CONTROL-G> tolls.
From: Kent M Pitman
Subject: Re: defun within let
Date: 
Message-ID: <sfwg1czn7d4.fsf@world.std.com>
Hrvoje Niksic <·······@srce.hr> writes:

> Kent M Pitman <······@world.std.com> writes:
> 
> >  (defun *foo-array* (make-array 100))
>    ^^^^^^
> 
> Did you mean to say DEFVAR here?

Yeah.  I type pretty fast and my fingers do the typing (and even the
editing, thanks to Emacs) without any conscious thought. They're fond
of making homonymic errors (i.e., substitutions of soundalikes).  I
was surely thinking defvar but my fingers "heard" defun...  Thanks
for spotting it.
From: Kent M Pitman
Subject: Re: defun within let
Date: 
Message-ID: <sfwaf383n0a.fsf@world.std.com>
·············@t-online.de (Ralf Muschall) writes:

> What do you mean by _global_ state here? The variable n is only shared
> between the functions next and reset.

To those of us who take the point of view in question, the problem is
that the variable n is also shared by the programmer's brain and the
development environment.  You are free to say this is of no importance
to you, but should recognize that such a view is not universally
shared.

A great deal of Common Lisp's design intentionally supports incremental
development--a concept that has no place if you take the static view 
that all there is to a program is what it is at a single static point
in time when seen by a compiler.

> Of course, if two process(e|or)s call the two functions created 
> by the same call to let at the same time, it will fail.

That's not the point in issue.
 
> But the same will happen if the two processes try to step or to
> reset the same instance cnt of the counter which you defined below.
> 
> The above version would have to be changed for multiprocessing 
> to something like

I don't see how the code below is materially different than the
code which others were using, unless you mean that each thread must
somehow arrange that the counter it uses is the result of a separate
call to makecounter, which isn't born out by this example since
in the end you do a setq rather than a let to establish cnt in
what appears just as global a fashion as before.  Am I missing something?
(I'm tired and just on my way to bed, so I very well might be, but...)

> (defun makecounter ()
>   (let ((n 0))
>      (cons
>         #'(lambda () (incf n))
>         #'(lambda () (setq n 0)))))
> 
> (setq cnt (makecounter))
> ((car cnt)) ; counts
> ((cdr cnt)) ; resets
> 
> which is what a class is syntactical sugar for.
> (I did mostly scheme for the last months, so I may have too few or too
> many "#'"s or "funcall"s here.)

((car cnt)) => (funcall (car cnt))

Incidentally, in Common Lisp I would typically have written:
 (defun make-counter ()
   (let ((n 0))
     (values #'(lambda () (incf n))
             #'(lambda () (setq n 0)))))
and then later
 (multiple-value-bind (next-counter reset-counter)
     (make-counter)
   ... 
   (funcall next-counter) ; counts
   ...
   (funcall reset-counter) ;resets
   ...)

It may seem silly, but I cringe at the idea of heap consing just to
hold onto two values.
From: R. Matthew Emerson
Subject: Re: defun within let
Date: 
Message-ID: <6vh262$hdm$1@news-2.news.gte.net>
more for my own benefit than for anything else, i thought i'd try to
summarize some of the discussion in this thread.

i asked why closures didn't seem to be a common idiom in common lisp,
and gave the following trivial example:

(let ((n 0))
  (defun next ()
    (incf n))
  (defun reset ()
    (setf n 0)))

the most important point seems to be that code like this is hard to
redefine dynamically, since recompiling or reloading the code
necessarily blows away the closed-over state.  this is philosophically
at odds with the notion of easy incremental development, and of
easy dynamic re-definition.

it seems to me that common lisp exposes things a lot more than other
languages do.  the package system comes to mind---even if a symbol is
not exported, you can still get at it with the pkg::foo syntax.  so,
if you're used to a more fascist language, CL's refusal to enforce
information hiding may leave you feeling kind of naked.

it occurs to me that lisp fosters an environment which is well-suited
for a single person or a small team working uno animo.  it seems to
enable the programmer to be a spider at the center of a web of
thoughts and abstractions, all of which are interconnected to each
other and easily accessible.  this is in distinct contrast to the
gospel of strictly defined interfaces that some software engineering
people preach.  perhaps this is why the idea of the single-user lispm
was so attractive.

sorry if this is all obvious.

comp.lang.lisp has to be the best newsgroup on the net.

-matt
From: Kent M Pitman
Subject: Re: defun within let
Date: 
Message-ID: <sfwemsjn68e.fsf@world.std.com>
···@nightfly.apk.net (R. Matthew Emerson) writes:

> it seems to me that common lisp exposes things a lot more than other
> languages do. 

which is a powerful feature that supports "adaptability" and
"evolution".  what you say here is surely true, but i hope you're not
implying this is a priori bad.

> the package system comes to mind---even if a symbol is
> not exported, you can still get at it with the pkg::foo syntax.  so,
> if you're used to a more fascist language, CL's refusal to enforce
> information hiding may leave you feeling kind of naked.

i don't understand.  you have the option of using only : and you get
errors just as in other languages.  what :: provides is a way of 
discovering that a new interface is needed and working around that
problem conveniently until proper negotiation can be done to make
that formal.  surely if the negotation fails, it's obvious it's the
burden of the user of :: to remove the reference, but equally surely
you're not saying that the language would somehow be more powerful or
useful if it kept people from doing something that couldn't be 
done some other way and that was necessary to do ... are you?
 
> it occurs to me that lisp fosters an environment which is well-suited
> for a single person or a small team working 

this is true, and nothing to be ashamed of.  would you want it
not to be well-suited for this?  or are you really making a statement
about the complement set -- larger groups? if so, what statement
are you making? i don't want to put words in your mouth.  certainly
lisp has been used successfully by medium sized groups (say, a hundred
people) working to design one of the most advanced computer systems (for
its time, and in some ways still) that the world has ever seen:
lisp machines.

> uno animo.  

this term is unfamiliar to me.

> it seems to
> enable the programmer to be a spider at the center of a web of
> thoughts and abstractions, all of which are interconnected to each
> other and easily accessible.  this is in distinct contrast to the
> gospel of strictly defined interfaces that some software engineering
> people preach.  perhaps this is why the idea of the single-user lispm
> was so attractive.

lisp doesn't work against any of these things.  it simply acknowledges
that development is not a simple matter of "writing your code" and
"having it run".  development is about give and take. thinking and
rethinking.  interfaces and changed interfaces.  the language permits
you flexibility.  

some say the US is a good place to live.  it's relatively free what
you can do.  there exist societies where you can get thrown in prison
for littering.  no one in the US is arguing that they want littering.
but sometimes the cure for a bad behavior is worse than the problem.
some programming languages don't let you deviate from standard
practice--and for some people that is necessary or desirable.  but
for others it is crippling of imagination.    flexibility is not
without risk (of abuse).  but then, neither is security (of lost
freedom or creativity).  

people tend to reject what most gets in their way and what most
has recently been a burden to them.  to those tired of
crime and vandalism, a protective society seems good.  to those
tired of government running their lives, a free society seems
good.  in my opinion, governments allowed to run people's lives
usually opt to do so and i consider that risk ultimately greater
than any risk brought on by individuals.  but that's just me.

likewise, to people who haven't been good at disciplining
programs, or who have been around others who haven't, it's easy
to see why a programming language that imposes structure is good.
but having myself seen programs that were wondrous and could only
be written in lisp because other programming languages are too
oppressive, the idea of forced conformance pains me.

it may be that lisp isn't good for some because maybe some need more
structure than lisp imposes.  i don't know.  i like to think the
environment can offer enough structure that this isn't an issue.  but
classically, lisp has solved the hardest, the most complex, the most
elusive of programming problems.  and often it takes the most creative
minds to do that.  tying their hands is not first on my list of tools
to enable such people.

> sorry if this is all obvious.

honestly, i didn't understand enough of what you were trying to say
to know even what your position was.  i don't think, in any case,
that the answers are obvious.

> comp.lang.lisp has to be the best newsgroup on the net.

i agree it works pretty well. 

if i've left some point not answered to your satisfaction, i hope
you'll post clarifying your position or concern, by the way, and not
just view this as an attempt to stifle you.
From: Barry Margolin
Subject: Re: defun within let
Date: 
Message-ID: <r7XS1.5$675.92959@burlma1-snr1.gtei.net>
In article <···············@world.std.com>,
Kent M Pitman  <······@world.std.com> wrote:
>likewise, to people who haven't been good at disciplining
>programs, or who have been around others who haven't, it's easy
>to see why a programming language that imposes structure is good.

What are frequently called "bondage and discipline" languages have been
promoted mostly by organizations that have to deal with hundreds of
entry-level programmers (graduates of XXX Technical Institute).  They're
the computer equivalent of construction workers.  A software engineer
specifies the relationships between modules and tells the module
programmers what they're supposed to do, and the language enforces many of
these things.  The DoD has software factories like this, which is why they
commissioned the development of Ada.

Languages like Lisp and Perl, which provide extreme flexibility and little
enforced structure, assume that the user is more of an architect than a
worker.  Exploratory and evolutionary programming are the opposite of the
above style of rigid design, but it's mainly appropriate for experienced,
creative software designers, not software technicians.

-- 
Barry Margolin, ······@bbnplanet.com
GTE Internetworking, Powered by BBN, Burlington, MA
*** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups.
From: Pierre Mai
Subject: Re: defun within let
Date: 
Message-ID: <873e8zb5k4.fsf@dent.isdn.cs.tu-berlin.de>
···@nightfly.apk.net (R. Matthew Emerson) writes:

> it seems to me that common lisp exposes things a lot more than other
> languages do.  the package system comes to mind---even if a symbol is
> not exported, you can still get at it with the pkg::foo syntax.  so,
> if you're used to a more fascist language, CL's refusal to enforce
> information hiding may leave you feeling kind of naked.

Just a little side note:  In a number of bondage & discipline
languages, package protection can be overcome relatively simply by
just manipulating the interface definitions, since these languages
don't change object-representation or linkage depending on visibility, 
or in other words:  Information hiding is just a convention checked by 
the compiler.  This is in no way different from CL, just that CL
offers a rather more hygienic and well defined escape-hatch than
manipulating interface definitions.

Regs, Pierre.

-- 
Pierre Mai <····@cs.tu-berlin.de>	http://home.pages.de/~trillian/
  "Such is life." -- Fiona in "Four Weddings and a Funeral" (UK/1994)
From: Barry Margolin
Subject: Re: defun within let
Date: 
Message-ID: <RK6T1.16$675.335518@burlma1-snr1.gtei.net>
In article <··············@dent.isdn.cs.tu-berlin.de>,
Pierre Mai  <····@acm.org> wrote:
>Just a little side note:  In a number of bondage & discipline
>languages, package protection can be overcome relatively simply by
>just manipulating the interface definitions, since these languages
>don't change object-representation or linkage depending on visibility, 
>or in other words:  Information hiding is just a convention checked by 
>the compiler.  This is in no way different from CL, just that CL
>offers a rather more hygienic and well defined escape-hatch than
>manipulating interface definitions.

That's true.  But on the other hand, you generally have to do more work to
overcome the interface definitions, e.g. making a private copy of the
header file.  It's much more obvious that you're working around the rules,
whereas the ease of using :: can lull one into thinking that it's OK.

-- 
Barry Margolin, ······@bbnplanet.com
GTE Internetworking, Powered by BBN, Burlington, MA
*** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups.
From: Tim Bradshaw
Subject: Re: defun within let
Date: 
Message-ID: <ey3vhlum2xg.fsf@todday.aiai.ed.ac.uk>
* Barry Margolin wrote:

> That's true.  But on the other hand, you generally have to do more work to
> overcome the interface definitions, e.g. making a private copy of the
> header file.  It's much more obvious that you're working around the rules,
> whereas the ease of using :: can lull one into thinking that it's OK.

Not if you have CVS commit scripts which grep for :: and reject the
commit in that case, and mail your supervisor to say what they found
(:).

--tim
From: Barry Margolin
Subject: Re: defun within let
Date: 
Message-ID: <WzrT1.53$675.793311@burlma1-snr1.gtei.net>
In article <···············@todday.aiai.ed.ac.uk>,
Tim Bradshaw  <···@aiai.ed.ac.uk> wrote:
>* Barry Margolin wrote:
>
>> That's true.  But on the other hand, you generally have to do more work to
>> overcome the interface definitions, e.g. making a private copy of the
>> header file.  It's much more obvious that you're working around the rules,
>> whereas the ease of using :: can lull one into thinking that it's OK.
>
>Not if you have CVS commit scripts which grep for :: and reject the
>commit in that case, and mail your supervisor to say what they found
>(:).

But the point is that someone has to impose that discipline on you, whereas
it's fairly obvious to all involved that making a copy of a header file so
you can change private to public is contrary to the philosophy of the
language.  The intent isn't to stop a determined hacker, just to make it
harder to get around well-defined interfaces.  B&D languages make it hard,
while dynamic languages tend to make it easy.

But you're right that the discipline doesn't have to be part of the
language definition.  C isn't very strict (esp. K&R C, which didn't have
prototypes), which is why lint was created.  But if you can embed the rules
that a lint-like program enforces into the language definition, you don't
have to worry about people who forget to run lint and aren't working for an
organization that automates it for them.

Literate programming languages also embody a similar philosophy.  Their
designers realized that programmers lazy about documenting what how the
program works.  But if the language makes it easy to write the
documentation while you're writing the program, it might lower the bar.
This never really took off, though.

-- 
Barry Margolin, ······@bbnplanet.com
GTE Internetworking, Powered by BBN, Burlington, MA
*** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups.
From: Charles Hixson
Subject: Re: defun within let
Date: 
Message-ID: <3620E92F.19262D2D@earthlink.net>
Barry Margolin wrote:
> 
...
> Literate programming languages also embody a similar philosophy.  Their
> designers realized that programmers lazy about documenting what how the
> program works.  But if the language makes it easy to write the
> documentation while you're writing the program, it might lower the bar.
> This never really took off, though.
> 
> --
> Barry Margolin, ······@bbnplanet.com
> GTE Internetworking, Powered by BBN, Burlington, MA
> *** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups.

It may not be exactly what Literate Programming is after, but isn't that
what JavaDoc accomplishes?  And some IDE tools (e.g., IBM's VA for Java)
will write a skeleton of the JavaDoc while they create your routine
template.  I think we'll be seeing more of this in the years ahead.
From: David J Cooper Jr
Subject: Re: defun within let
Date: 
Message-ID: <36216BC1.AF509CED@genworks.com>
--------------EAC26493C4DCAE0D003E7FE2
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

Charles Hixson wrote:

> It may not be exactly what Literate Programming is after, but isn't that
> what JavaDoc accomplishes?  And some IDE tools (e.g., IBM's VA for Java)
> will write a skeleton of the JavaDoc while they create your routine
> template.  I think we'll be seeing more of this in the years ahead.

We are using a Javadoc-like approach for documenting our ICAD Design Language
(ICAD/IDL)1 functions and macros. ICAD/IDL now has a built-in documentation
writer for both functions and IDL objects (defparts). You can see an example
output from the function documentation writer at:

    http://www.genworks.com/idl-shareware/doc/catalog2oracle.html

If anyone is interested in the source for the function documentation writer,
I have it (since I wrote it)(it's only a couple lines of code). The defpart
documentation writer is probably less useful for most people, but I do have a
shareware version of that as well if anyone is interested.

By the way, I find cl-http to be very useful in developing documentation, since
you can just keep recompiling code in the Emacs buffer and reloading the doc
page from the virtual URL defined in cl-http (i.e. you don't have to keep
writing out a bunch of physical files to disk - the web pages are just computed
on a demand-driven basis).


1ICAD is a macro extension of Allegro Common Lisp, used for ``Knowledge-based
Engineering'' work (traditionally it's used mostly for generative geometrical
applications and geometric reasoning, but I use it for pretty much everything
(I'm spoiled)).


--
   "The Lisp ignorants will come up with an amazing array of bullshit
    to support their ignorance. They are like racists that way"

--Erik Naggum



--------------EAC26493C4DCAE0D003E7FE2
Content-Type: text/html; charset=us-ascii
Content-Transfer-Encoding: 7bit

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML>
Charles Hixson wrote:
<BLOCKQUOTE TYPE=CITE>It may not be exactly what Literate Programming is
after, but isn't that
<BR>what JavaDoc accomplishes?&nbsp; And some IDE tools (e.g., IBM's VA
for Java)
<BR>will write a skeleton of the JavaDoc while they create your routine
<BR>template.&nbsp; I think we'll be seeing more of this in the years ahead.</BLOCKQUOTE>

<P><BR>We are using a Javadoc-like approach for documenting our ICAD&nbsp;Design
Language (ICAD/IDL)<SUP>1</SUP> functions and macros. ICAD/IDL now has
a built-in documentation writer for both functions and IDL&nbsp;objects
(defparts). You can see an example output from the function documentation
writer at:
<P>&nbsp;&nbsp;&nbsp; <A HREF="http://www.genworks.com/idl-shareware/doc/catalog2oracle.html">http://www.genworks.com/idl-shareware/doc/catalog2oracle.html</A>
<P>If anyone is interested in the source for the function documentation
writer, I&nbsp;have it (since I&nbsp;wrote it)(it's only a couple lines
of code). The defpart documentation writer is probably less useful for
most people, but I&nbsp;do have a shareware version of that as well if
anyone is interested.
<P>By the way, I&nbsp;find cl-http to be very useful in developing documentation,
since you can just keep recompiling code in the Emacs buffer and reloading
the doc page from the virtual URL&nbsp;defined in cl-http (i.e. you don't
have to keep writing out a bunch of physical files to disk - the web pages
are just computed on a demand-driven basis).
<BR>&nbsp;
<P><SUP>1</SUP>ICAD&nbsp;is a macro extension of Allegro Common Lisp, used
for ``Knowledge-based Engineering'' work (traditionally it's used mostly
for generative geometrical applications and geometric reasoning, but I&nbsp;use
it for pretty much everything (I'm spoiled)).
<BR>&nbsp;
<PRE>--&nbsp;
&nbsp;&nbsp; "The Lisp ignorants will come up with an amazing array of bullshit
&nbsp;&nbsp;&nbsp; to support their ignorance. They are like racists that way"

--Erik Naggum</PRE>
&nbsp;</HTML>

--------------EAC26493C4DCAE0D003E7FE2--
From: David Steuber "The Interloper
Subject: Re: defun within let
Date: 
Message-ID: <36213f6c.3277643@news.newsguy.com>
On Sun, 11 Oct 1998 10:21:51 -0700, Charles Hixson
<············@earthlink.net> claimed or asked:

% It may not be exactly what Literate Programming is after, but isn't that
% what JavaDoc accomplishes?  And some IDE tools (e.g., IBM's VA for Java)
% will write a skeleton of the JavaDoc while they create your routine
% template.  I think we'll be seeing more of this in the years ahead.

What happens if the function starts to do something different than it
was originally declared to do because the design requirements changed?
I see so many cases of the comments not being updated and the function
name no longer being descriptive.

--
David Steuber (ver 1.31.2a)
http://www.david-steuber.com
To reply by e-mail, replace trashcan with david.

So I have this chicken, see?  And it hatched from this egg, see?  But
the egg wasn't laid by a chicken.  It was cross-laid by a turkey.
From: Paolo Amoroso
Subject: Re: defun within let
Date: 
Message-ID: <36212397.7330283@news.mclink.it>
On Sun, 11 Oct 1998 10:21:51 -0700, Charles Hixson
<············@earthlink.net> wrote:

> It may not be exactly what Literate Programming is after, but isn't that
> what JavaDoc accomplishes?  And some IDE tools (e.g., IBM's VA for Java)

Here are a couple of quotes from a paper ("The Design of Distributed
Hyperlinked Programming Documentation"; it's available for download at
Sun's official Web site) by Lisa Friendly, one of JavaDocs' designers:

"Java's support for automatic document generation was influenced by earlier
work on literate programming."

"Another difference between javadoc and most literate programming tools is
that its purpose is to extract documentation for the programming interface
only. For this reason we did not have to extend the program to support
method implementations, as most literate programming tools do."


Paolo
-- 
Paolo Amoroso <·······@mclink.it>
From: Tim Bradshaw
Subject: Re: defun within let
Date: 
Message-ID: <ey3pvbym4ot.fsf@todday.aiai.ed.ac.uk>
* Barry Margolin wrote:

[About using `::' to get at internal symbols is obviously bad]

> But the point is that someone has to impose that discipline on you, whereas
> it's fairly obvious to all involved that making a copy of a header file so
> you can change private to public is contrary to the philosophy of the
> language.  The intent isn't to stop a determined hacker, just to make it
> harder to get around well-defined interfaces.  B&D languages make it hard,
> while dynamic languages tend to make it easy.

I think I disagree with this, because I think that for someone
`properly trained' (for a suitable definition of that) in Lisp would
regard using `::' to get at non-exported symbols as obviously contrary
to the philosophy of the language in a serious development
environment.  The problem may be that it is harder to *see* than
copying a header file, using simple tools, but the C++ equivalent can
be hidden too I think (just put the .h file at the start of your .c
file).

So I think with C++ you'd expect to see people checking for
redefinitions of classes, with some tool, while with CL you'd expect
to see people grepping for :: and various flavours of IMPORT, with
some tool.

Unfortunately my experience of the way Lisp (CL specifically) is often
taught is that these kind of disciplines are not stressed at all, but
that tends to be because it's taught in an environment where any kind
of engineering considerations are regarded as entirely trivial and
worthless compared to the noble goal of publishing papers (:).

--tim
From: Kent M Pitman
Subject: Re: defun within let
Date: 
Message-ID: <sfwyaqm58sg.fsf@world.std.com>
Tim Bradshaw <···@aiai.ed.ac.uk> writes:

> Unfortunately my experience of the way Lisp (CL specifically) is often
> taught is that these kind of disciplines are not stressed at all, but
> that tends to be because it's taught in an environment where any kind
> of engineering considerations are regarded as entirely trivial and
> worthless compared to the noble goal of publishing papers (:).

Well said.
From: Lieven Marchand
Subject: Re: defun within let
Date: 
Message-ID: <6vqmd2$qog$2@xenon.inbe.net>
Barry Margolin <······@bbnplanet.com> writes:

> That's true.  But on the other hand, you generally have to do more work to
> overcome the interface definitions, e.g. making a private copy of the
> header file.  It's much more obvious that you're working around the rules,
> whereas the ease of using :: can lull one into thinking that it's OK.
> 

#define private public
#include "class.h"
#undef private

Note to C++ gurus: I know this violates the ODR and evokes undefined
behaviour but it has a fair chance of working on the more common
compilers.

In general the only system I have ever worked with that tried to
enforce this discipline full force ( an incredibly bad Ada-83
implementation with a library manager from hell ) just had as effect
everyone in our group had the passwords of all others and that we
disabled some general security features in order to get anything done
with the beast.

-- 
Lieven Marchand <···@bewoner.dma.be> 
------------------------------------------------------------------------------
Few people have a talent for constructive laziness. -- Lazarus Long
From: Duane Rettig
Subject: Re: defun within let
Date: 
Message-ID: <4u31ibfbi.fsf@beta.franz.com>
Barry Margolin <······@bbnplanet.com> writes:

> In article <················@naggum.no>, Erik Naggum  <····@naggum.no> wrote:
> >  I regard this as an optimization technique, not as normal coding style.
> >  accessing a special variable is generally more expensive than accessing a
> >  closure variable and this may matter in some circumstances.
> 
> While I haven't done any benchmarking to support it, I wouldn't expect this
> to be true in most implementations.  Accessing a special variable is
> normally just an indirection through the value cell of the symbol (most CL
> systems use shallow binding); there's no package searching done at access
> time -- LOAD and READ intern the symbol so the function has a pointer to
> the symbol itself.  Accessing a captured lexical variable requires an
> indirection through the closure's environment object.  About the only
> difference at access time is that a lexical variable can't be unbound, so
> no check has to be done for this.
> 
> Note that calling a lexical closure is often more expensive than calling a
> plain function, because an implicit environment argument may have to be
> passed to the function.

Mileage will vary based on usage.

Assuming aggressive optimzation settings:

For a global variable access, the name of the variable must be loaded
into a register from the function-environment, then an access of the
value slot gets the value.

For a lexical closure variable, the lexical environment must be ensured
to be in a register, then accesses to any number of the variables in
that same closure contour can take place one-load-per-instruction, assuming
the lexical environment is vector-like.

For the global accesses there is no setup of the function-environment; it
usually comes for free as part of the calling sequence.  But an instruction
scheduler at the peephole phase must take care of the back-to-back loads
in order to break any instruction pipeline interlocks.  In other words,
global accesses have relatively constant overhead, as contrasted in the
next paragraph.

For lexical closure variables, there is at least a several instruction
extra setup required per function, and the compiler must ensure that the
environment is close-at-hand for any accesses.  If the choice is made
to use a register for the lexical environment, then it might be loaded
very early (and then again after each call that the function makes), but
once the lexical environment is in a register, there would tend to be
fewer instruction pipeline interlocks, especially if many operations on
many variables at the same closure level are performed.  In other words,
lexical closure variables tend to have higher setup cost for faster
access time once the setup is done.

In conclusion, If closed-over variables are used "enough" times within
a function to offset the initial setup overhead, then it can be faster
than a global access.

Of course, if the closure structure becomes complex (e.g. multiple
variables homed in different closure levels), then the cost of closure
setup and access climbs.

> Your point about application-level checking that may be necessary for
> special variables is well taken, though.  This is also the justification
> for information hiding in most object oriented languages -- by limiting
> access to internal state, you can often prove invariants about that state.

I agree, from a compiler writer's point of view; removing external
influences on any datum makes it much easier to compile the (complete)
set of operations on that datum.

-- 
Duane Rettig          Franz Inc.            http://www.franz.com/ (www)
1995 University Ave Suite 275  Berkeley, CA 94704
Phone: (510) 548-3600; FAX: (510) 548-8253   ·····@Franz.COM (internet)