From: ccc31807
Subject: Lisp modularization
Date: 
Message-ID: <8580b7aa-5290-4492-9ac9-832024065dc2@t13g2000yqt.googlegroups.com>
Okay, here's another question that doesn't seem to be covered in the
first few chapters of the Lisp books I've been reading.

In C, you #include <stdio.h>
In Java, you import java.awt.*;
In Perl, you use DBI;
In Erlang, you -import([lists]).

Suppose you want to break your Lisp code into several modules. How to
you refer to the module data members and member functions? How do you
include code that you've placed in different modules?

Guess you can tell -- I'm just starting out, but that doesn't mean
that my questions are stupid, just ignorant, and I'm working toward
curing my ignorance.

CC

From: Robert Uhl
Subject: Re: Lisp modularization
Date: 
Message-ID: <m3tz1u8mnb.fsf@latakia.octopodial-chrome.com>
ccc31807 <········@gmail.com> writes:

> Okay, here's another question that doesn't seem to be covered in the
> first few chapters of the Lisp books I've been reading.

The reason for that is that you can go quite awhile in learning to
program without having to worry about it.  It's good that you're
thinking about it this early though; it shows that you're actually
trying to understand how to write in Lisp.  Good for you!

> In C, you #include <stdio.h>
> In Java, you import java.awt.*;
> In Perl, you use DBI;
> In Erlang, you -import([lists]).
>
> Suppose you want to break your Lisp code into several modules. How to
> you refer to the module data members and member functions? How do you
> include code that you've placed in different modules?

Lisp uses a concept called package; these are basically namespaces for
symbols.  E.g. 'foo' refers to FOO in *PACKAGE* (this is always bound to
the current package), 'bar:foo' refers to the exported FOO in the
package named BAR (and throws an error if it's not exported), and
BAR::FOO refers to FOO in BAR whether or not it's exported.

You define a package with DEFPACKAGE; among other things this indicates
what other packages that package incorporates into its own namespace.
By default you're in the COMMON-LISP-USER package which incorporates the
COMMON-LISP package.

You switch into a package with IN-PACKAGE.  You'll normally put this at
the top of a file, so that all the code in that file is in some package.

To load a file which defines a package or creates code within a package,
you can simply LOAD it as usual.  There's also REQUIRE, which does
implementation-dependent things which can often be helpful.

Practical Common Lisp, Chapter 21: Programming in the Large (Packages
and Symbols) has a much better introduction to packages than I can give.

In order to save the bother of manually LOADing stuff, folks have come
up with 'system definition' facilities.  The most popular of these at
the moment is ASDF (another system definition facility), but Mudballs
seems to be getting more popular.

I mentioned REQUIRE earlier; SBCL extends it very nicely so that
(require :cl-ppcre) will load CL-PPCRE's ASDF system definition.

I believe you use CLISP; for it, you'd have to use ASDF directly with
something like (asdf:oos 'asdf:load-op :cl-ppcre).  You could load ASDF
itself with (require :asdf #P"/path/to/asdf.lisp").

You might ask why packaging and system definition are separate.  It's
mostly historical, and in practise isn't a big deal: in fact the
flexibility can be nice.  You define your packages, and you define your
systems in ASDF and it all works out well.

-- 
Robert Uhl <http://public.xdi.org/=ruhl>
The cruelest fiction ever put forward is that every person matters.
Chances are, you don't.                                --Alex Beam
From: ccc31807
Subject: Re: Lisp modularization
Date: 
Message-ID: <e3a7787c-8a58-4f6e-aec9-935c10e81a54@x3g2000yqa.googlegroups.com>
Okay, I've read the relevant chapter in Seibel, and yes, it's toward
the end of the book. I've read all the posts. I've looked at some
code. Seibel notes that of all the languages CL has the most
commonality to Perl, and everything does seem familiar, even if
slightly weird.

Of course, when you write C or Java, you create header or class files
frequently, the aim is to have your main function or class as short as
possible, so that it does most of the work using the class properties
and methods or header functions and variables. In Perl, I use a top
down approach by first writing pseudocode in the main script and then
writing the behavior as user defined functions or modules as
appropriate to fill out the functionality. In Java, I tend to do the
opposite, writing the classes and testing them before I write the main
program -- in fact most of the time the main class is simple a rehash
of my testing classes reordering code instead of writing it.

I would imagine (since I've not written any 'real' CL code) that one
might use a blended approach, since writing the functions seems akin
to writing Java classes while developing the application logic seems
akin to a top down approach. Is there a particular idiomatic style of
CL development that people fall into as a result of the structure of
the language? I'm not asking about best practices, but what happens
when you're tired, frustrated, stressed, and trying to finish a
project with users and managers calling you every hour wanting to know
if you're finished.

Thanks, CC.
From: Robert Uhl
Subject: Re: Lisp modularization
Date: 
Message-ID: <m3d48h8yaa.fsf@latakia.octopodial-chrome.com>
ccc31807 <········@gmail.com> writes:

> Is there a particular idiomatic style of CL development that people
> fall into as a result of the structure of the language?

Well, here's what _I_ use.  I'm not saying it's best, or even
particularly good, but it works for me.

 + project
   - package.lisp # contains all DEFPACKAGEs
   - project.asd  # contains PROJECT's DEFSYSTEM
   - ...

My package.lisp will look something like:

  (in-package :cl-user)
  
  (defpackage #:project
    (:use :cl)
    (:export))

And my project.asd will look something like:

  (in-package :cl-user)
  
  (defpackage :project-system
    (:use :cl :asdf))
  
  (in-package :project-system)
  
  (defsystem :project
   :description "This is a project"
    :author "Robert A. Uhl"
     :license "LLGPL"
       :depends-on (:foo :bar :baz)
        :components ((:file "package")))

In fact, this is so common that this code here will create a new
Lisp project:

  (defparameter *project-dir* #P"/home/ruhl/lisp/")

    (defun start-lisp-project (project-name)
    (let* ((project-dir 
  	  (make-pathname :directory (append (pathname-directory *project-dir*) 
  					    (list project-name))))
  	 (package-file-name (merge-pathnames "package.lisp" project-dir))
  	 (system-file-name (merge-pathnames
  			    (concatenate 'string project-name ".asd")
  			    project-dir))
  	 (*print-readably* t)
  	 (*print-pretty* t)
  	 (*print-case* :downcase))
      (ensure-directories-exist project-dir)
      (with-open-file (*standard-output* package-file-name
  				       :direction :output
  				       :if-exists :error)
        (prin1 '(in-package :cl-user))
        (terpri)
        (terpri)
        (prin1 `(defpackage ,(make-symbol (string-upcase project-name))
  		(:use :cl)
  		(:export)))
        (terpri))
      (with-open-file (*standard-output* system-file-name
  				       :direction :output
  				       :if-exists :error)
        (let ((system-package (intern (concatenate 'string
  						 (string-upcase project-name)
  						 "-SYSTEM")
  				    :keyword))
  	    (system-name (intern (string-upcase project-name) :keyword)))
  	(prin1 '(in-package :cl-user))
  	(terpri)
  	(terpri)
  	(prin1 `(defpackage ,system-package (:use :cl :asdf)))
  	(terpri)
  	(terpri)
  	(prin1 `(in-package ,system-package))
  	(terpri)
  	(terpri)
  	(prin1 `(defsystem ,system-name
  		  :description ""
  		  :author "Robert A. Uhl"
  		  :license "LLGPL"
  		  :depends-on ()
  		  :components ((:file "package"))))
  	(terpri)))))

This can be used by calling (start-lisp-project "foo") and will create
/home/ruhl/lisp/foo/{package.lisp,foo.asd}.

Note that there's some cargo-culting going on here.  There's probably no
good reason to be using an uninterned symbol #:PROJECT for the project
name in DEFPACKAGE; using :PROJECT is there probably just fine and would
make my code look better, but I'm too lazy to retype it.

Also, note the use of *print-case*.  You might want to add this to
whatever CLISP's equivalent of .sbclrc is: (setf *print-case :downcase).
This will make Lisp not look like it's from the era of teletypes.

-- 
Robert Uhl <http://public.xdi.org/=ruhl>
An environment that puts women unwillingly at the disposal of men is
never an argument for equality.  It is quite vividly the opposite.
                                                --Kathleen Parker
From: Rob Warnock
Subject: Re: Lisp modularization
Date: 
Message-ID: <kJmdnTlEItuLLtPXnZ2dnUVZ_jBi4p2d@speakeasy.net>
Robert Uhl  <·········@NOSPAMgmail.com> wrote:
+---------------
| Also, note the use of *print-case*.  You might want to add this to
| whatever CLISP's equivalent of .sbclrc is: (setf *print-case :downcase).
| This will make Lisp not look like it's from the era of teletypes.
+---------------

Doing (setf (readtable-case *readtable*) :invert) will do much the
name thing, but without losing information in case you have some
mixed-case symbols in your environment.


-Rob

-----
Rob Warnock			<····@rpw3.org>
627 26th Avenue			<URL:http://rpw3.org/>
San Mateo, CA 94403		(650)572-2607
From: D Herring
Subject: Re: Lisp modularization
Date: 
Message-ID: <4a4eded7$0$29136$6e1ede2f@read.cnntp.org>
Rob Warnock wrote:
> Robert Uhl  <·········@NOSPAMgmail.com> wrote:
> +---------------
> | Also, note the use of *print-case*.  You might want to add this to
> | whatever CLISP's equivalent of .sbclrc is: (setf *print-case :downcase).
> | This will make Lisp not look like it's from the era of teletypes.
> +---------------
> 
> Doing (setf (readtable-case *readtable*) :invert) will do much the
> name thing, but without losing information in case you have some
> mixed-case symbols in your environment.

True, but :invert is a braindead, clever hack by people who refuse to 
move forward with life.  I don't much like it, but don't see anything 
better without changing CL's core.  And it doesn't even work all the 
time; try typing a cl: symbol with CamelCase...

Proposal:  move case out of the readtable and into the package.  Then 
the standard packages can maintain their :upcase behavior, but users 
can easily create packages where case is preserved.  All in the same 
lisp image at the same time without interference.  I have some notes 
on how this would work, but haven't had time to flesh them out.  Need 
to start hacking SBCL or the like.

- Daniel
From: Rob Warnock
Subject: Re: Lisp modularization
Date: 
Message-ID: <UKKdnR185pxAq9LXnZ2dnUVZ_qidnZ2d@speakeasy.net>
D Herring  <········@at.tentpost.dot.com> wrote:
+---------------
| Rob Warnock wrote:
| > Doing (setf (readtable-case *readtable*) :invert) will do much the
| > name thing, but without losing information in case you have some
| > mixed-case symbols in your environment.
| 
| True, but :invert is a braindead, clever hack by people who refuse to 
| move forward with life.  I don't much like it, but don't see anything 
| better without changing CL's core.  And it doesn't even work all the 
| time; try typing a cl: symbol with CamelCase...
+---------------

I'm not sure what you mean by "doesn't even work all the time".
It works as documented in the CLHS:

    23.1.2 Effect of Readtable Case on the Lisp Reader
    ...
    :invert
	When the readtable case is :invert, then if all of the
	unescaped letters in the extended token are of the same case,
	those (unescaped) letters are converted to the opposite case.

and:

    22.1.3.3.2 Effect of Readtable Case on the Lisp Printer
    ...
    :invert
	When the readtable case is :invert, the case of all
	alphabetic characters in single case symbol names is
	inverted. Mixed-case symbol names are printed as is.

To wit:

    > (readtable-case *readtable*)

    :UPCASE
    > (setf (readtable-case *readtable*) :invert)

    :invert
    > (defun show (sym)
	(list sym (symbol-name sym)))

    show
    > (show 'cl::foo)

    (common-lisp::foo "FOO")
    > (show 'cl::FooBar)

    (common-lisp::FooBar "FooBar")
    > (show 'cl::FOO)

    (common-lisp::FOO "foo")
    > 

Now you might not *like* the documented behavior, but it's useful
when one needs to read sexps from files written in CamelCase [e.g.,
EDIF files] when what one mostly wants is read/print invariance
together with the ability to still input symbols in the CL package
in lowercase.


-Rob

-----
Rob Warnock			<····@rpw3.org>
627 26th Avenue			<URL:http://rpw3.org/>
San Mateo, CA 94403		(650)572-2607
From: Richard Fateman
Subject: readtable-case   was: Re: Lisp modularization
Date: 
Message-ID: <4A4F5E17.8020705@cs.berkeley.edu>
Rob Warnock wrote:
> D Herring  <········@at.tentpost.dot.com> wrote:
> +---------------
> | Rob Warnock wrote:
> 

......
> | 
> | True, but :invert is a braindead,

...

> +---------------
> 
> I'm not sure what you mean by "doesn't even work all the time".
>
...

This discussion dates back to the ANSI CL standards committee, and has 
been revived periodically.

My view (at that time and now) is that the ANSI standard should have had 
both cases.  The consensus-builders that prevailed were used to PDP-10s 
with a much smaller character set and opted for mapping to upper-case.
(6 characters in a 36-bit word meant you had only 64 different glyphs).
The Berkeley view (from Unix on VAX with ASCII) was that both cases were 
valuable. My view did not prevail, except for our Lisp, and then as an
option for Allegro Common Lisp.

Unfortunately, the invert-casing of a readtable is insufficient 
alteration to make a winner from a loser.  Indeed, preserving case of a 
readtable is not so good either.


  Here's the problem:
the names of the standard functions must also be set up in the most 
useful case. Do you want CONS, CAR, DEFUN?  eg. is it really OK to have
(DEFUN CamelCase ( ....)...)

or do you want
(defun CamelCase ...)?

and do you want to be able to change your mind at runtime?

a discussion can be found in
http://www.franz.com/support/tutorials/casemode-tutorial.htm

In the porting of Maxima to different lisp systems, I became aware that 
Scieneer supported the "modern" mode as in Allegro.. I just checked this:
·············································@···········@2A.html
u
RJF
From: Rob Warnock
Subject: Re: readtable-case   was: Re: Lisp modularization
Date: 
Message-ID: <PKWdnYiCn5qLkc3XnZ2dnUVZ_j-dnZ2d@speakeasy.net>
Richard Fateman  <·······@cs.berkeley.edu> wrote:
+---------------
| Rob Warnock wrote:
| > D Herring  <········@at.tentpost.dot.com> wrote:
| > +---------------
| > | True, but :invert is a braindead,
...
| > I'm not sure what you mean by "doesn't even work all the time".
| ...
| This discussion dates back to the ANSI CL standards committee,
| and has been revived periodically.
+---------------

Indeed! And sometimes with far more heat than in this thread so far.  ;-}

+---------------
| My view (at that time and now) is that the ANSI standard should have had 
| both cases.  The consensus-builders that prevailed were used to PDP-10s 
| with a much smaller character set and opted for mapping to upper-case.
| (6 characters in a 36-bit word meant you had only 64 different glyphs).
+---------------

Actually, even as early as 1970 [when I started using PDP-10s], the
encoding for PDP-10 text files was *7*-bit ASCII [5 characters plus
one wasted bit per 36-bit word], which handled lowercase just fine --
provided you had a lowercase-capable terminal, which were just starting
to phase in [KSR37s plus a few 80x24 video terminals ("glass teletypes")],
as were mixed-case line printers [such as the 3rd-party Mohawk Data
Sciences printer we added to our -10]. Yes, TOPS-10 still used SIXBIT
for file & device names in system calls, but text files on disk were
all 7-bit case-sensitive ASCII. The main issue was that the terminals
people had readily available were still mostly ASR33s or KSR35s, which
were uppercase-only [albeit they (fortunately!) printed lowercase as
uppercase, having :UPCASE built into the hardware, as it were].

+---------------
| The Berkeley view (from Unix on VAX with ASCII) was that both cases
| were valuable. My view did not prevail, except for our Lisp, and then
| as an option for Allegro Common Lisp.
+---------------

Unfortunately, once the ANSI standard *was* passed with :UPCASE as
the reader default, the issue then became [and remains!] one of arguing
against The Standard, rather than any of the original technical merits.

+---------------
| Unfortunately, the invert-casing of a readtable is insufficient 
| alteration to make a winner from a loser.  Indeed, preserving case
| of a readtable is not so good either.
+---------------

No argument there! See my parallel reply to Daniel Herring. But IMHO,
*given* that :UPCASE *is* the ANSI default, :INVERT is a much better
alternative to use than :PRESERVE, but it is certainly no panacea.
You still have to revert to :UPCASE when reading/compiling legacy code.

+---------------
| Here's the problem:
| the names of the standard functions must also be set up in the most 
| useful case. Do you want CONS, CAR, DEFUN?  eg. is it really OK to have
| (DEFUN CamelCase ( ....)...) or do you want (defun CamelCase ...)?
+---------------

That's why, when forced to deal with CamelCase, I prefer :INVERT;
it lets you write the latter version.

+---------------
| and do you want to be able to change your mind at runtime?
+---------------

Absolutely!! I need :UPCASE to LOAD or COMPILE-FILE legacy code
and :INVERT when READ-ing CamelCase files (e.g., EDIF data).

+---------------
| a discussion can be found in
| http://www.franz.com/support/tutorials/casemode-tutorial.htm
+---------------

Thanks for the ref. Also see the many past "interesting" discussions
in this group, e.g., one memorable exchange[1] between Erik Naggum,
Kent Pitman, Bruno Haible, and others way back in 1998!  ;-}  ;-}


-Rob

[1] Advanced Search in Google Groups is down at the moment [all queries
    yield null results!!], but when it comes back you might start at or
    before Message-ID <················@naggum.no>.

-----
Rob Warnock			<····@rpw3.org>
627 26th Avenue			<URL:http://rpw3.org/>
San Mateo, CA 94403		(650)572-2607
From: Richard Fateman
Subject: Re: readtable-case   was: Re: Lisp modularization
Date: 
Message-ID: <h2qln9$r6r$1@aioe.org>
Rob Warnock wrote:
> Richard Fateman  <·······@cs.berkeley.edu> wrote:
> +---------------
> | Rob Warnock wrote:
> | > D Herring  <········@at.tentpost.dot.com> wrote:
> | > +---------------
> | > | True, but :invert is a braindead,
> ...
> | > I'm not sure what you mean by "doesn't even work all the time".
> | ...
> | This discussion dates back to the ANSI CL standards committee,
> | and has been revived periodically.
> +---------------
> 
> Indeed! And sometimes with far more heat than in this thread so far.  ;-}
> 
> +---------------
> | My view (at that time and now) is that the ANSI standard should have had 
> | both cases.  The consensus-builders that prevailed were used to PDP-10s 
> | with a much smaller character set and opted for mapping to upper-case.
> | (6 characters in a 36-bit word meant you had only 64 different glyphs).
> +---------------
> 
> Actually, even as early as 1970 [when I started using PDP-10s], the
> encoding for PDP-10 text files was *7*-bit ASCII [5 characters plus
> one wasted bit per 36-bit word], which handled lowercase just fine --
> provided you had a lowercase-capable terminal, which were just starting
> to phase in [KSR37s plus a few 80x24 video terminals ("glass teletypes")],
....

The persons who preferred the upper-casification on the CL committee are 
still alive, and presumably could try to present their arguments again 
if they cared to. I doubt they care to. Glancing through the old 
comp.lang.lisp, I think they have not weighed in, and notes from
others not involved in the decision raise new thoughts. Color? 
Distinguishing the case of initial letters by position vs. implicitly 
capital?  etc.


Regarding sixbits, yest there were upper/lowercase display terminals way 
back then.  But I found this:

"  (pnget symbol  n) returns  the pname  of the  symbol as  a list  of 
fixnums containing  packed  n-bit bytes.   The  legal  values of  n 
depend  on the implementation; in the pdp-10 implementation, 6 (SIXBIT) 
and 7  (ASCII) are allowed.  If this seems obscure, that's because it 
is. ..."
from 
http://pdp-10.trailing-edge.com/mit_emacs_170_teco_1220/01/info/lisp.info.html

The importance of six characters was impressed on novice programmers (I 
was using Maclisp at MIT for Macsyma starting in 1968). That if a symbol 
had a 6-letter print-name, then the storage of the character string took 
only one 36-bit word.  If the print-name had fewer letters, it still 
took a 36-bit word.  If the print-name had only one letter more, another 
36-bit word was needed.  Thus FOOBAR is OK. FOOBAR2 was wasteful.
Actually, I don't know for a fact that any of this was true, or whether 
the Lisp used for Macsyma used 6 or 7 bits per character, but memory was 
always a concern. The PDP-10 computers used for Macsyma had address 
spaces of 1.2 megabytes, and that included the operating system.
Multics and Lisp machines had much larger address spaces, but they 
didn't catch on.

  By the time the common lisp commitee was meeting, the VAX and Berkeley 
UNIX were widespread and Berkeley's Franz Lisp were probably the most 
widely used Lisp systems, using lower case.  MIT, Stanford, CMU people 
were largely hostile to suggestions from Berkeley, so much so that I 
suspect the preserving-case proposal might have been rejected primarily 
because it came from Berkeley.
...


> 
> Unfortunately, once the ANSI standard *was* passed with :UPCASE as
> the reader default, the issue then became [and remains!] one of arguing
> against The Standard, rather than any of the original technical merits.
> 
...
In some sense yes, but in others no.  Every CL implementation these days 
exceeds the standard in some way, and so any sufficiently-ambitious 
program that is portable to several systems tends to require 
conditionalizations in the code to make it work on different 
implementations. I've found that writing in lower case makes everything 
work nicely, most of the time, as presumably does everyone else.
With the exception of a Lisp implementation of a Mathematica-style 
system with symbols like PolynomialGCD in it.  It was hugely more 
convenient to write it using Allegro CL "modern" mode. Though not 
impossible to write in "standard mode" if you don't mind writing stuff 
like |PolynomialGCD| all over the place.  (And mercilessly hacking the 
GCL readtable which does not implement readtable-case).

RJF
From: Rainer Joswig
Subject: Re: readtable-case was: Re: Lisp modularization
Date: 
Message-ID: <b8bd0aae-8ee2-475b-bc9c-a9e4a73661cb@a36g2000yqc.googlegroups.com>
On 5 Jul., 18:51, Richard Fateman <·······@cs.berkeley.edu> wrote:

> In some sense yes, but in others no.  Every CL implementation these days
> exceeds the standard in some way, and so any sufficiently-ambitious
> program that is portable to several systems tends to require
> conditionalizations in the code to make it work on different
> implementations. I've found that writing in lower case makes everything
> work nicely, most of the time, as presumably does everyone else.
> With the exception of a Lisp implementation of a Mathematica-style
> system with symbols like PolynomialGCD in it.  It was hugely more
> convenient to write it using Allegro CL "modern" mode. Though not
> impossible to write in "standard mode" if you don't mind writing stuff
> like |PolynomialGCD| all over the place.  (And mercilessly hacking the
> GCL readtable which does not implement readtable-case).

Or use readmacros like CCL does for foreign identifiers.

Examples:

Foreign function:

  #_NSClassFromString

Foreign variable:

  #$kProcessTransformToForegroundApplication

Foreign message:

  #/loadNibNamed:owner:

Which has the added benefit that the definitions of these identifiers
can be looked
up incrementally during read time and the reader can generate code
around those.
It also works fine in standard ANSI Common Lisp.

> RJF
From: Alan Bawden
Subject: Re: readtable-case   was: Re: Lisp modularization
Date: 
Message-ID: <w2d8wj2srkg.fsf@shaggy.csail.mit.edu>
Just to clarify the record about MacLisp...

Richard Fateman <·······@cs.berkeley.edu> writes:
> ...
> Regarding sixbits, yest there were upper/lowercase display terminals way
> back then.  But I found this:
> 
> "  (pnget symbol  n) returns  the pname  of the  symbol as  a list  of
> fixnums containing  packed  n-bit bytes.   The  legal  values of  n depend
> on the implementation; in the pdp-10 implementation, 6 (SIXBIT) and 7
> (ASCII) are allowed.  If this seems obscure, that's because it is. ..."
> from
> http://pdp-10.trailing-edge.com/mit_emacs_170_teco_1220/01/info/lisp.info.html

MacLisp symbols were stored in 7-bit ASCII.  (PNGET x 7) did no consing and
returned the symbol's actual printname.  (PNGET x 6) computed its answer.
The latter was sometimes useful when you were using a operating system like
ITS that stored it's filenames in SIXBIT and you were issuing system calls
manually, which was indeed an obscure thing to do.

I could have sworn that (PNGET x 5) also gave you SQUOZE (radix 40), but a
glance at the MacLisp sources show that in fact the only allowed 2nd
arguments are 6 and 7.

> The importance of six characters was impressed on novice programmers (I was
> using Maclisp at MIT for Macsyma starting in 1968). That if a symbol had a
> 6-letter print-name, then the storage of the character string took only one
> 36-bit word.  If the print-name had fewer letters, it still took a 36-bit
> word.  If the print-name had only one letter more, another 36-bit word was
> needed.  Thus FOOBAR is OK. FOOBAR2 was wasteful.
> Actually, I don't know for a fact that any of this was true, or whether the
> Lisp used for Macsyma used 6 or 7 bits per character, but memory was always
> a concern....

That may have been true in 1968, but by the time I was using MacLisp in
1976 symbols contained 7-bit characters.  So "FOOBAR" was already too long!

Note that since symbol names were -lists- of fixnums, it actually cost
-two- 36-bit words for every 5 characters: one for the CONS and one for the
FIXNUM.  I always thought that that was incredibly wastefull given, as you
note, we were always worrying about exhausting the PDP10 address space.
But perhaps the effort required to re-architect MacLisp to avoid that would
have exceeded the benefit.
From: Vassil Nikolov
Subject: Re: readtable-case   was: Re: Lisp modularization
Date: 
Message-ID: <snz1votdqs0.fsf@luna.vassil.nikolov.name>
On 05 Jul 2009 18:17:19 -0400, Alan Bawden <····@shaggy.csail.mit.edu> said:
> ...
> That may have been true in 1968, but by the time I was using MacLisp in
> 1976 symbols contained 7-bit characters.  So "FOOBAR" was already too long!

  The following quote might be of (minor) interest in this regard:

    *Compatibility note:* In earlier versions of Lisp, such as MacLisp
    and InterLisp, the print name of a gensym was of fixed length,
    consisting of a single letter and a fixed-length decimal
    representation with leading zeros if necessary, for example,
    G0007.  This convention was motivated by an implementation
    consideration, namely that the name should fit into a single
    machine word, allowing a quick and clever implementation.  Such
    considerations are less relevant in Common Lisp.  ...

  (Guy Steele, _Common LISP: The Language_, Digital Press, 1984;
  10.3. Creating Symbols, GENSYM, p. 170).

  ---Vassil.


-- 
"Even when the muse is posting on Usenet, Alexander Sergeevich?"
From: D Herring
Subject: Re: Lisp modularization
Date: 
Message-ID: <4a4fbe7c$0$29138$6e1ede2f@read.cnntp.org>
Rob Warnock wrote:
> D Herring  <········@at.tentpost.dot.com> wrote:
> +---------------
> | Rob Warnock wrote:
> | > Doing (setf (readtable-case *readtable*) :invert) will do much the
> | > name thing, but without losing information in case you have some
> | > mixed-case symbols in your environment.
> | 
> | True, but :invert is a braindead, clever hack by people who refuse to 
> | move forward with life.  I don't much like it, but don't see anything 
> | better without changing CL's core.  And it doesn't even work all the 
> | time; try typing a cl: symbol with CamelCase...
> +---------------
> 
> I'm not sure what you mean by "doesn't even work all the time".
> It works as documented in the CLHS:

Please ignore my error wrt CamelCase for a moment.  The thing that 
really bothers me with :invert is that it hides a dirty secret; 
everything that touches the internal representation must participate 
or things will break.

e.g.
(setf (readtable-case *readtable*) :invert)
(symbol-name 'cons)
=> "CONS"
(symbol-name 'CONS)
=> "cons"
(intern (concatenate 'string
                      (symbol-name 'cons)
                      (symbol-name 'CONS)))
=> CONScons ; expect consCONS
(symbol-name *)
=> "CONScons"

Note how :invert breaks the algebra of treating symbol names as 
strings?  The CL API leaks this implementation detail.


This gets really fun when defining bindings to other languages; it 
pretty much forces the use of strings (or read macros which generate 
strings) instead of transparent lisp symbols.  The same goes for 
programs which try using symbols for user interaction (e.g. computer 
algebra).


Here's my big beef with the spec for :invert.

>     23.1.2 Effect of Readtable Case on the Lisp Reader
>     ...
>     :invert
> 	When the readtable case is :invert, then if all of the
> 	unescaped letters in the extended token are of the same case,
> 	those (unescaped) letters are converted to the opposite case.

Why not always invert?  Why the special behavior for CamelCase? 
Without this exception, the above string algebra would behave 
properly...  Isn't it inefficient to scan the whole symbol before 
knowing its case?  Wouldn't it be more efficient to preserve the case 
MOST PEOPLE WANT TO USE?

In this day and age, it is customary to write IN LOWERCASE; it seems 
like CL AND OLD FORTRAN are the only languages which don't comply.  It 
seems to me that :invert was the ugly hack which made ALL CAPS 
acceptable in the CL spec; I often wish it hadn't been invented.

Later,
Daniel

P.S.  As mentioned in my previous post, I think case conversion would 
be more useful if it was moved out of the reader and into the package 
framework.  Packages care about case; the whole lisp image shouldn't 
have to agree to a single convention.

P.P.S  In the grand scheme of things, symbol case isn't all that 
important; but it can be real turn-off for smart people coming from 
other languages.  A common WTF that makes them question CL's sanity.
From: Rob Warnock
Subject: Re: Lisp modularization
Date: 
Message-ID: <j-2dnfUblsQxadLXnZ2dnUVZ_jWdnZ2d@speakeasy.net>
D Herring  <········@at.tentpost.dot.com> wrote:
+---------------
| Rob Warnock wrote:
| > I'm not sure what you mean by "doesn't even work all the time".
| > It works as documented in the CLHS:
| 
| Please ignore my error wrt CamelCase for a moment.  The thing that 
| really bothers me with :invert is that it hides a dirty secret; 
| everything that touches the internal representation must participate 
| or things will break.
+---------------

Agreed, but then again, that's usually the case when one
"touches the internal representation" of almost anything...  ;-}  ;-}

Look, I was just quibbling over calling :INVERT "broken" when
in fact it does "work" as ANSI CL requires.

Now if you want to call ANSI CL *itself* "broken" for not having gone
with a full case-sensitive symbol reader as the default -- instead
of picking the historical :UPCASE as the default and then slipping
in :INVERT as a clumsy hack for preserving read/write case-invariance
while not breaking source code written in lowercase -- well, that's
a whole 'nother argument, one that continues from time to time [as
Professor Fateman points out in his parallel reply], for example,
the [sometimes rancorous!] discussions of Allegro Common Lisp's
"modern" mode a while back.

And by the way, I never meant to imply that :INVERT completely
"solves" the problem; it only ameliorates it under most
circumstances... and makes things worse in others. For example,
I've seen plenty of legacy CL code that, while written *mostly*
in lowercase, for stylistic & readability reasons freely uses
both uppercase and lowercase to designate the *same* symbol.
E.g., the CMUCL compiler sources tend to make tags in TAGBODYs
and VOP bodies [think LAP] be all uppercase, while *not* always
making all references to them be the same. [Though to be fair,
that "mismatch" mostly occurs in VOPs; in TAGBODYs the authors
*did* mostly say (go FOO) to go to FOO and (go bar) to go to bar.]

So you still have to be careful when using :INVERT. That said,
given that ANSI CL defaults to :UPCASE, I'm still glad :INVERT
is available [and think that :PRESERVE is almost *never* the
right thing to use!!]..

+---------------
| ...[example elided]...
| 
| Note how :invert breaks the algebra of treating symbol names as 
| strings?  The CL API leaks this implementation detail.
+---------------

But no such "algebra of treating symbol names as strings" was ever
promised by the standard. That is, I would claim that this isn't
"leaking an implementation detail", it's quite clearly adhering to
the published spec [which in turn was a compromise of the times].

+---------------
| 
| >     23.1.2 Effect of Readtable Case on the Lisp Reader
| >     ...
| >     :invert
| > 	When the readtable case is :invert, then if all of the
| > 	unescaped letters in the extended token are of the same case,
| > 	those (unescaped) letters are converted to the opposite case.
| 
| Why not always invert?  Why the special behavior for CamelCase? 
+---------------

Simple: historical compatibility with legacy code containing *lots*
of uppercase strings in symbol contexts: (intern "FOO" "BAR"),
(in-package "GORP"), etc.

+---------------
| In this day and age, it is customary to write IN LOWERCASE; it seems 
| like CL AND OLD FORTRAN are the only languages which don't comply.
+---------------

A sizable number of people agree with you. [See many previous threads
in this group.] Neverthless, as we all know, ANSI CL is a *compromise*
standard reflecting the practices and -- most importantly -- the existing
corpus of code at that time, and as a result :UPCASE was chosen as
the default.

+---------------
| It seems to me that :invert was the ugly hack which made ALL CAPS 
| acceptable in the CL spec; I often wish it hadn't been invented.
+---------------

You've got it backwards, I think. Given that :UPCASE *was* chosen as
the default [which made (eq 'ALL-CAPS 'all-caps) not only "acceptable"
but mandatory, by default], :INVERT was the ugly hack glued on afterwards
to provide *some* measure of read/write mixed-case stmbol invariance
while not totally breaking code written in all lowercase. So *given*
:UPCASE as the default, I personally am glad that :INVERT exists.


-Rob

-----
Rob Warnock			<····@rpw3.org>
627 26th Avenue			<URL:http://rpw3.org/>
San Mateo, CA 94403		(650)572-2607
From: Thomas F. Burdick
Subject: Re: Lisp modularization
Date: 
Message-ID: <dd5f935a-3ee5-404c-8ba2-002862aa577c@13g2000prl.googlegroups.com>
On Jul 5, 2:32 am, ····@rpw3.org (Rob Warnock) wrote:
> D Herring  <········@at.tentpost.dot.com> wrote:
>
> +---------------
> | It seems to me that :invert was the ugly hack which made ALL CAPS
> | acceptable in the CL spec; I often wish it hadn't been invented.
> +---------------
>
> You've got it backwards, I think. Given that :UPCASE *was* chosen as
> the default [which made (eq 'ALL-CAPS 'all-caps) not only "acceptable"
> but mandatory, by default], :INVERT was the ugly hack glued on afterwards
> to provide *some* measure of read/write mixed-case stmbol invariance
> while not totally breaking code written in all lowercase. So *given*
> :UPCASE as the default, I personally am glad that :INVERT exists.

Are you sure about that? Not being present when it was being
discussed, I have no idea how things actually played out. But what
Daniel seems to be arguing here, at it certainly seems plausible, is
that the choice to make CL a LISP THAT SHOUTS AT YOU might not have
made it through the committee if the dissenters hadn't been offered
the ugly (but useful!!!) hack of :invert case. Although from what
Richard Fateman said above, it sounds like that might not have been
the case.
From: Richard Fateman
Subject: Re: Lisp modularization
Date: 
Message-ID: <4A51FE68.7050208@cs.berkeley.edu>
Thomas F. Burdick wrote:
..

f the dissenters hadn't been offered
> the ugly (but useful!!!) hack of :invert case. Although from what
> Richard Fateman said above, it sounds like that might not have been
> the case.

My recollection is that readtable-case was added afterward, and was not 
part of the first complete draft. I don't have documents to support 
this, but observe that GCL does not have readtable-case.

I note (once again) that changing the case from :upper to :preserve at 
runtime probably requires (if you are reading lisp code with the changed 
readtable), that all built-in functions be renamed, e.g. from CAR to 
car.  Thus the transformation between versions of lisp is not quite as 
simple as you might first image.  Of course if you are only preserving 
case in (say) a different language Q implemented in lisp, then the 
readtable for Q may be different from the readtable for lisp.

RJF
From: D Herring
Subject: Re: Lisp modularization
Date: 
Message-ID: <4a52ad86$0$29139$6e1ede2f@read.cnntp.org>
Thomas F. Burdick wrote:
> On Jul 5, 2:32 am, ····@rpw3.org (Rob Warnock) wrote:
>> D Herring  <········@at.tentpost.dot.com> wrote:
>>
>> +---------------
>> | It seems to me that :invert was the ugly hack which made ALL CAPS
>> | acceptable in the CL spec; I often wish it hadn't been invented.
>> +---------------
>>
>> You've got it backwards, I think. Given that :UPCASE *was* chosen as
>> the default [which made (eq 'ALL-CAPS 'all-caps) not only "acceptable"
>> but mandatory, by default], :INVERT was the ugly hack glued on afterwards
>> to provide *some* measure of read/write mixed-case stmbol invariance
>> while not totally breaking code written in all lowercase. So *given*
>> :UPCASE as the default, I personally am glad that :INVERT exists.
> 
> Are you sure about that? Not being present when it was being
> discussed, I have no idea how things actually played out. But what
> Daniel seems to be arguing here, at it certainly seems plausible, is
> that the choice to make CL a LISP THAT SHOUTS AT YOU might not have
> made it through the committee if the dissenters hadn't been offered
> the ugly (but useful!!!) hack of :invert case. Although from what
> Richard Fateman said above, it sounds like that might not have been
> the case.

That's what my guess was.  I have no knowledge of what the original 
thoughts were, but today :invert is the poor bone used to pacify 
barking dogs such as myself.

- Daniel
From: Robert Uhl
Subject: Re: Lisp modularization
Date: 
Message-ID: <m34otrn12k.fsf@latakia.octopodial-chrome.com>
D Herring <········@at.tentpost.dot.com> writes:
>
> P.P.S  In the grand scheme of things, symbol case isn't all that
> important; but it can be real turn-off for smart people coming from
> other languages.  A common WTF that makes them question CL's sanity.

I just (setf *print-case* :downcase), as that Does the Right Thing
generally.

As for sanity...yeah, IMHO the whole upcasing-read thing is pretty
goofy.  If anything, the reverse would have made much better sense,
since it would case-fold to a reasonable case.  In my nastier, more
opinionated moments I like to think that preferring all-caps is a kind
of mental illness, but that's uncharitable and probably not true.

-- 
Robert Uhl <http://public.xdi.org/=ruhl>
It's all a conspiracy.  A conspiracy by us elites to shut people out
by giving away all our code under open source licenses.
 --Peter Samuelson in comp.os.linux.development.system
From: Raffael Cavallaro
Subject: Re: Lisp modularization
Date: 
Message-ID: <h2qfil$df7$1@news.eternal-september.org>
On 2009-07-05 01:36:19 -0400, Robert Uhl <·········@NOSPAMgmail.com> said:

> In my nastier, more
> opinionated moments I like to think that preferring all-caps is a kind
> of mental illness, but that's uncharitable and probably not true.

Don't be so hard on yourself.

;^)
-- 
Raffael Cavallaro
From: Alberto Riva
Subject: Re: Lisp modularization
Date: 
Message-ID: <h2lc7e$7qc$1@usenet.osg.ufl.edu>
ccc31807 wrote:
> Okay, I've read the relevant chapter in Seibel, and yes, it's toward
> the end of the book. I've read all the posts. I've looked at some
> code. Seibel notes that of all the languages CL has the most
> commonality to Perl, and everything does seem familiar, even if
> slightly weird.

Just imagine how weird Perl looks from our point of view... :)

> Of course, when you write C or Java, you create header or class files
> frequently, the aim is to have your main function or class as short as
> possible, so that it does most of the work using the class properties
> and methods or header functions and variables. In Perl, I use a top
> down approach by first writing pseudocode in the main script and then
> writing the behavior as user defined functions or modules as
> appropriate to fill out the functionality. In Java, I tend to do the
> opposite, writing the classes and testing them before I write the main
> program -- in fact most of the time the main class is simple a rehash
> of my testing classes reordering code instead of writing it.
> 
> I would imagine (since I've not written any 'real' CL code) that one
> might use a blended approach, since writing the functions seems akin
> to writing Java classes while developing the application logic seems
> akin to a top down approach. Is there a particular idiomatic style of
> CL development that people fall into as a result of the structure of
> the language?

Lisp is a multi-paradigm language, so you can essentially use whatever 
style you're more comfortable with. Nevertheless, the design of the 
language favors a bottom-up approach, similar to what you were 
describing above about Java, but more extreme: writing a Lisp program 
means interactively and dynamically evolving the language so that it 
fits your problem domain, and once you get there, expressing the 
solution to the problem becomes very natural. You obviously write 
functions that act as building blocks for other functions, but you can 
also create new syntax, new control structures, new data structures, etc.

> I'm not asking about best practices, but what happens
> when you're tired, frustrated, stressed, and trying to finish a
> project with users and managers calling you every hour wanting to know
> if you're finished.

I think every programmer has a different answer to this question, but in 
general Lisp is a language that allows you to develop working prototypes 
very quickly, thanks to incremental development, powerful debugger, 
macros, malleable data structures, the condition system, untyped 
variables, etc. So in a situation like that, I'd rather be using Lisp 
than any other language.

Alberto
From: Paul Donnelly
Subject: Re: Lisp modularization
Date: 
Message-ID: <87tz1u31t7.fsf@plap.localdomain>
ccc31807 <········@gmail.com> writes:

> Okay, here's another question that doesn't seem to be covered in the
> first few chapters of the Lisp books I've been reading.
>
> In C, you #include <stdio.h>
> In Java, you import java.awt.*;
> In Perl, you use DBI;
> In Erlang, you -import([lists]).
>
> Suppose you want to break your Lisp code into several modules. How to
> you refer to the module data members and member functions? How do you
> include code that you've placed in different modules?

I'm surprised none of your books have covered that yet, but maybe they
will in a few chapters. Short answer: use packages and some system
definition facility (which will be a library). Packages are purely
namespaces, and are not a way to load code. For that, ASDF is the de
facto standard. It's a library that provides a way to define and load
“systems”. Mudballs is a new alternative that will hopefully catch
on. Likely it will. And there's always the option to write a short
program that loads some code into your Lisp (for example, ASDF has to do
this since it can't load itself without being installed).

Packages are simple. First you define a package somewhere with
DEFPACKAGE, then at the top of any source files for that package, you
use IN-PACKAGE. You export any symbols (functions or variables) from
that package that you want to be public, and can access them later with
PACKAGE:SYMBOL. These are globally accessible. If you want to cut
typing, you can state that a package uses another package, and the
symbols from the latter will be usable in the former. A double colon
accesses a package's unexported symbols, if that should be
necessary. There's more to know about them, but that's basically how
they work.

One important thing to keep in mind is that if you're doing cut'n'paste
REPL interaction and you paste from a file having USE-PACKAGE at the top
into a REPL that's in the CL-USER package, for example, your pasted code
won't end up in the right place, because the IN-PACKAGE form doesn't
magically go along with it. You'll need to watch your packages if you
paste code into the REPL. SLIME *will* magically take care of it for you
if you evaluate a definition directly from a file.

For ASDF or Mudballs, find yourself an example and read the docs. Using
them isn't particularly complicated.

You'll notice that while in some languages you import systems or
packages per-file, in CL systems just need to be loaded into your Lisp
at some point before use, and any packages used by the package you're in
are available without qualifiers to any file in that package. So
required systems are noted in your program's system definition (if
you've got one — otherwise you load them by hand), and if a package uses
another, it's simplest to put it in the definition of that former
package.
From: Paul Donnelly
Subject: Re: Lisp modularization
Date: 
Message-ID: <87prci30ow.fsf@plap.localdomain>
Paul Donnelly <·············@sbcglobal.net> writes:

> You export any symbols (functions or variables) from that package that
> you want to be public, and can access them later with
> PACKAGE:SYMBOL. These are globally accessible. If you want to cut
> typing, you can state that a package uses another package, and the
> symbols from the latter will be usable in the former. A double colon
> accesses a package's unexported symbols, if that should be necessary.

Oh, and unexported symbols are also globally accessible. So
PACKAGE::SYMBOL will always get you symbol “symbol” in the package
“package”, regardless of what package uses what (likewise PACKAGE:SYMBOL
for exported symbols). It's up to you not to stick your fingers where
they don't belong.
From: blandest
Subject: Re: Lisp modularization
Date: 
Message-ID: <63d13312-39f9-453b-a093-3f4cd776fe31@g31g2000yqc.googlegroups.com>
> Mudballs is a new alternative that will hopefully catch
> on. Likely it will. And there's always the option to write a short
> program that loads some code into your Lisp (for example, ASDF has to do
> this since it can't load itself without being installed).

Unfortunately the future of Mudballs is uncertain. The author recently
announced that he will no longer maintain this project (http://
groups.google.com/group/mudballs/browse_thread/thread/15755f04ff62629c?
hl=en).
From: Alberto Riva
Subject: Re: Lisp modularization
Date: 
Message-ID: <h2jqg6$lm$1@usenet.osg.ufl.edu>
ccc31807 wrote:
> Okay, here's another question that doesn't seem to be covered in the
> first few chapters of the Lisp books I've been reading.
> 
> In C, you #include <stdio.h>
> In Java, you import java.awt.*;
> In Perl, you use DBI;
> In Erlang, you -import([lists]).

Note that these look superficially similar, but they actually do very 
different things.

> Suppose you want to break your Lisp code into several modules.

The closest approximation in Lisp is called 'packages'. See Chapter 21 
of Practical Common Lisp.

> How to you refer to the module data members and member functions?

There is no such thing as 'data members' and 'member functions' in Lisp. 
Packages are containers for symbols, and symbols can be used to name 
variables, or functions, or both (and several other things too).

> How do you include code that you've placed in different modules?

Look up USE-PACKAGE. Feel free to come back here if anything is not clear :)

Alberto
From: D Herring
Subject: Re: Lisp modularization
Date: 
Message-ID: <4a4d6e84$0$29138$6e1ede2f@read.cnntp.org>
ccc31807 wrote:
> Okay, here's another question that doesn't seem to be covered in the
> first few chapters of the Lisp books I've been reading.
> 
> In C, you #include <stdio.h>
> In Java, you import java.awt.*;
> In Perl, you use DBI;
> In Erlang, you -import([lists]).
> 
> Suppose you want to break your Lisp code into several modules. How to
> you refer to the module data members and member functions? How do you
> include code that you've placed in different modules?

Common Lisp uses packages.
http://www.lispworks.com/documentation/HyperSpec/Body/11_aa.htm

Use DEFPACKAGE to define a new package, specify which symbols it will 
export, and specify which other packages it will use.

Use IN-PACKAGE to tell CL where the following code should be stored.

ASDF and REQUIRE serve to autoload other packages as needed.

lispdoc.com is the best search engine for standard CL functions.

Later,
Daniel
From: Pascal J. Bourguignon
Subject: Re: Lisp modularization
Date: 
Message-ID: <871voyo5p3.fsf@galatea.local>
ccc31807 <········@gmail.com> writes:

> Suppose you want to break your Lisp code into several modules. How to
> you refer to the module data members and member functions? How do you
> include code that you've placed in different modules?
>
> Guess you can tell -- I'm just starting out, but that doesn't mean
> that my questions are stupid, just ignorant, and I'm working toward
> curing my ignorance.

When you have code in various files, you can load them with various
LOAD expressions.
See: http://www.lispworks.com/documentation/lw50/CLHS/Body/24_.htm


When you have symbols defined in various packages, you can use them
with various package operations.  You may _use_ one package in
another, you may _import_ symbols from one package to another, etc.
See: http://www.lispworks.com/documentation/lw50/CLHS/Body/11_.htm


If you don't want to deal with file compilation and loading manually,
you may use a system construction package such as asdf (defsystem is
an earlier used only in legacy systems; xcvb will be the asdf
successor).
See: http://constantly.at/lisp/asdf/

-- 
__Pascal Bourguignon__
From: budden
Subject: Re: Lisp modularization
Date: 
Message-ID: <0c420c45-b683-4812-8460-d9952bba679d@y7g2000yqa.googlegroups.com>
Hi!
  CL package systems looks nice at the glance, but its design is
rather inconvinient
in practice and there are always a pitfall waiting for you when you
change your package/module
structure. You really _can_ make CL modular, but this is a pain.

Try to read this: www.flownet.com/gat/packages.pdf